### BullMQ Quick Start Example Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Standard BullMQ setup for creating a queue, adding a job, and setting up a worker with event listeners for job completion and failure. ```typescript // BullMQ import { Queue, Worker } from 'bullmq'; const connection = { host: 'localhost', port: 6379 }; const queue = new Queue('tasks', { connection }); await queue.add('send-email', { to: 'user@example.com' }); const worker = new Worker( 'tasks', async (job) => { await sendEmail(job.data.to); }, { connection, concurrency: 10 }, ); worker.on('completed', (job) => console.log(job.id, 'done')); worker.on('failed', (job, err) => console.error(job.id, err.message)); ``` -------------------------------- ### Install and Run Dashboard Demo Server Source: https://github.com/avifenesh/glide-mq/blob/main/docs/OBSERVABILITY.md Quick start for the @glidemq/dashboard package. Navigate to the demo directory, install dependencies, and run the dashboard server. ```bash cd demo npm install npm run dashboard # http://localhost:3000 ``` -------------------------------- ### Quick Start: Broadcast Setup and Publishing Source: https://github.com/avifenesh/glide-mq/blob/main/docs/BROADCAST.md Demonstrates how to initialize a Broadcast instance, create BroadcastWorkers for different subscriptions, and publish a message. Ensure the connection details are correctly configured. ```typescript import { Broadcast, BroadcastWorker } from 'glide-mq'; const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const broadcast = new Broadcast('events', { connection, maxMessages: 1000, // retain at most 1000 messages in the stream }); // Each subscriber is identified by a unique subscription name (becomes a consumer group) const inventoryWorker = new BroadcastWorker( 'events', async (job) => { console.log('Inventory update:', job.data); }, { connection, subscription: 'inventory-service' }, ); const emailWorker = new BroadcastWorker( 'events', async (job) => { console.log('Send notification:', job.data); }, { connection, subscription: 'email-service' }, ); // Publish - every subscriber receives this message await broadcast.publish('order.placed', { event: 'order.placed', orderId: 42 }); ``` -------------------------------- ### glide-mq Quick Start Example Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md glide-mq setup mirroring BullMQ's structure, using the `addresses` array for connection. The processor function signature and event handling remain identical. ```typescript // glide-mq import { Queue, Worker } from 'glide-mq'; const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); await queue.add('send-email', { to: 'user@example.com' }); const worker = new Worker( 'tasks', async (job) => { await sendEmail(job.data.to); }, { connection, concurrency: 10 }, ); worker.on('completed', (job) => console.log(job.id, 'done')); worker.on('failed', (job, err) => console.error(job.id, err.message)); ``` -------------------------------- ### Install glide-mq and Update Imports Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Instructions for removing BullMQ and installing glide-mq, followed by an example of updating import paths. ```bash npm remove bullmq npm install glide-mq ``` ```typescript // Before import { Queue, Worker, Job, QueueEvents, FlowProducer } from 'bullmq'; // After import { Queue, Worker, Job, QueueEvents, FlowProducer } from 'glide-mq'; ``` -------------------------------- ### Quick Start: Setup Queue and Worker Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/SKILL.md Initialize a Queue and a Worker for a specific task. Configure connection options and set concurrency for the worker. Handles job completion and failure events. ```typescript import { Queue, Worker } from 'glide-mq'; const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); await queue.add('send-email', { to: 'user@example.com', subject: 'Hello' }); const worker = new Worker('tasks', async (job) => { console.log(`Processing ${job.name}:`, job.data); return { sent: true }; }, { connection, concurrency: 10 }); worker.on('completed', (job) => console.log(`Done: ${job.id}`)); worker.on('failed', (job, err) => console.error(`Failed: ${job.id}`, err.message)); ``` -------------------------------- ### Python Example: Adding and Reading Jobs with redis-py Source: https://github.com/avifenesh/glide-mq/blob/main/docs/WIRE_PROTOCOL.md This example demonstrates adding a job using glidemq_addJob via redis-py's fcall, then reading the job's data and checking job counts. Ensure redis-py is installed and Redis is running. ```python import redis import json import time r = redis.Redis(host='localhost', port=6379, decode_responses=True) queue_name = 'tasks' prefix = 'glide' p = f'{prefix}:{{{queue_name}}}' # Add a simple job job_data = json.dumps({'to': 'user@example.com', 'subject': 'Hello'}) job_opts = json.dumps({}) timestamp = str(int(time.time() * 1000)) result = r.fcall( 'glidemq_addJob', 4, # numkeys f'{p}:id', # key 1: id counter f'{p}:stream', # key 2: stream f'{p}:scheduled', # key 3: scheduled ZSet f'{p}:events', # key 4: events stream 'send-email', # arg 1: jobName job_data, # arg 2: jobData job_opts, # arg 3: jobOpts timestamp, # arg 4: timestamp '0', # arg 5: delay '0', # arg 6: priority '', # arg 7: parentId '0', # arg 8: maxAttempts '', # arg 9: orderingKey '0', # arg 10: groupConcurrency '0', # arg 11: groupRateMax '0', # arg 12: groupRateDuration '0', # arg 13: tbCapacity '0', # arg 14: tbRefillRate '0', # arg 15: jobCost '0', # arg 16: ttl '', # arg 17: customJobId '0', # arg 18: lifo '', # arg 19: parentQueue '', # arg 20: schedulerName '0', # arg 21: skipEvents ) print(f'Job created with ID: {result}') # Read the job back job_hash = r.hgetall(f'{p}:job:{result}') print(f'Job data: {job_hash}') # Get job counts stream_len = r.xlen(f'{p}:stream') completed = r.zcard(f'{p}:completed') failed = r.zcard(f'{p}:failed') delayed = r.zcard(f'{p}:scheduled') print(f'Waiting: {stream_len}, Completed: {completed}, Failed: {failed}, Delayed: {delayed}') ``` -------------------------------- ### Install glide-mq Source: https://github.com/avifenesh/glide-mq/blob/main/README.md Install the glide-mq package using npm. ```bash npm install glide-mq ``` -------------------------------- ### Basic Connection Setup Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/connection.md Establishes a basic connection to a local Glide instance and initializes a new Queue. ```typescript const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); ``` -------------------------------- ### AWS Lambda Example with ServerlessPool Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/serverless.md Example demonstrating how to use the ServerlessPool within an AWS Lambda handler for efficient connection management. Ensure to close the pool on SIGTERM. ```typescript import { serverlessPool } from 'glide-mq'; const CONNECTION = { addresses: [{ host: process.env.VALKEY_HOST!, port: 6379 }], }; export async function handler(event: any) { const producer = serverlessPool.getProducer('notifications', { connection: CONNECTION, }); const id = await producer.add('push-notification', { userId: event.userId, message: event.message, }); return { statusCode: 200, body: JSON.stringify({ jobId: id }) }; } process.on('SIGTERM', async () => { await serverlessPool.closeAll(); }); ``` -------------------------------- ### BullMQ Pro Setup for Per-Client Isolation Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md This snippet shows the original BullMQ Pro setup for enqueuing tasks with per-client isolation and rate limiting. ```typescript // BullMQ Pro - original setup import { Queue, Worker } from 'bullmq'; const connection = { host: 'localhost', port: 6379 }; const queue = new Queue('jobs', { connection }); // Enqueue with per-client isolation, max 2 concurrent await queue.add( 'task', { clientId: 'acme', payload: {} }, { group: { id: 'acme', limit: { max: 2, duration: 0 } }, attempts: 5, backoff: { type: 'exponential', delay: 1000 }, }, ); const worker = new Worker( 'jobs', async (job) => { const results = job.data.partialResults ?? []; results.push(await doWork(job.data)); await job.updateData({ ...job.data, partialResults: results }); return results; }, { connection, concurrency: 20, limiter: { max: 2, duration: 100 }, }, ); ``` -------------------------------- ### HTTP Proxy Creation Source: https://github.com/avifenesh/glide-mq/blob/main/docs/SERVERLESS.md Example of creating an HTTP proxy server using `createProxyServer` for cross-language communication. ```APIDOC ## HTTP Proxy Use `createProxyServer()` when you need cross-language producers, request-reply, or SSE consumers from environments that should not host long-lived queue objects. ```typescript import { createProxyServer } from 'glide-mq/proxy'; const proxy = createProxyServer({ connection: CONNECTION, queues: ['emails', 'events'], // optional allowlist compression: 'gzip', }); proxy.app.listen(3000); ``` ### Proxy Options | Option | Type | Notes | | ------ | ---- | ----- | | `connection` | `ConnectionOptions` | Required unless `client` is provided. Required for queue-wide/broadcast SSE routes. | | `client` | `Client` | Shared GLIDE client for non-blocking routes. | | `prefix` | `string` | Key prefix (default: `glide`). | | `queues` | `string[]` | Optional allowlist. Unlisted queue names return `403`. | | `compression` | `'none' | 'gzip'` | Transparent job payload compression. | | `onError` | `(err, queueName) => void` | Queue-level error hook. | ### Route Surface The proxy now covers the main queue-management surface: - Queue writes: add, bulk add, add-and-wait, pause/resume, drain, retry, clean - Job operations: fetch job, list jobs by state, change priority, change delay, promote delayed jobs, stream job output over SSE, send suspend/resume signals - Queue telemetry: counts, metrics, live workers, queue-wide events over SSE - Scheduler APIs: list, fetch, upsert, remove - Flow APIs: create tree flows or DAGs, inspect flow snapshots, read nested tree views, revoke outstanding flow jobs - AI APIs: flow usage, flow budget, rolling usage summary - Broadcast APIs: publish plus SSE fan-out with `subscription` and `subjects` filters - Health: `/health` See [Usage - Proxy Endpoints](./USAGE.md#proxy-endpoints) for the exact method/path table. - Add your own auth and rate limiting middleware before exposing the proxy publicly. `glide-mq/proxy` does not ship built-in auth. - Queue-wide SSE (`/queues/:name/events`) and broadcast SSE (`/broadcast/:name/events`) need `connection`, not only a shared `client`, because they allocate blocking readers. ``` -------------------------------- ### Uninstall Bee-Queue and Install Glide-MQ Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bee/SKILL.md Use these commands to remove the Bee-Queue package and install Glide-MQ in your project. ```bash npm uninstall bee-queue @types/bee-queue npm install glide-mq ``` -------------------------------- ### Producer with Compression Source: https://github.com/avifenesh/glide-mq/blob/main/docs/SERVERLESS.md Example of creating a producer with gzip compression enabled at the producer level. ```typescript const compressed = new Producer('q', { connection: CONNECTION, compression: 'gzip', }); ``` -------------------------------- ### General Queue Usage Source: https://github.com/avifenesh/glide-mq/blob/main/README.md Demonstrates basic queue and worker setup for general task processing. Ensure a connection to Valkey/Redis is established. ```typescript import { Queue, Worker } from 'glide-mq'; const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); await queue.add('send-email', { to: 'user@example.com', subject: 'Welcome' }); const worker = new Worker( 'tasks', async (job) => { await sendEmail(job.data.to, job.data.subject); return { sent: true }; }, { connection, concurrency: 10 }, ); ``` -------------------------------- ### Install glide-mq and remove BullMQ Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bullmq/SKILL.md Use npm to uninstall BullMQ and install glide-mq. This is the first step in migrating your project. ```bash npm remove bullmq npm install glide-mq ``` -------------------------------- ### Create HTTP Proxy Server Source: https://github.com/avifenesh/glide-mq/blob/main/docs/SERVERLESS.md Instantiates and starts an HTTP proxy server for Glide-MQ, optionally with a queue allowlist and compression. ```typescript import { createProxyServer } from 'glide-mq/proxy'; const proxy = createProxyServer({ connection: CONNECTION, queues: ['emails', 'events'], // optional allowlist compression: 'gzip', }); proxy.app.listen(3000); ``` -------------------------------- ### Job Retrieval and Management Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bee/references/api-mapping.md Demonstrates how to get individual jobs and multiple jobs, and how to remove jobs. ```APIDOC ## getJob ### Description Retrieves a single job by its ID. ### Method `queue.getJob(jobId)` ### Parameters #### Path Parameters - **jobId** (string) - Required - The ID of the job to retrieve. ### Request Example ```typescript const job = await queue.getJob('42'); ``` ## getJobs ### Description Retrieves a list of jobs based on their type and a range. ### Method `queue.getJobs(type, start, end)` ### Parameters #### Path Parameters - **type** (string) - Required - The type of jobs to retrieve (e.g., 'waiting', 'failed'). - **start** (number) - Required - The starting index for the job list. - **end** (number) - Required - The ending index for the job list. ### Request Example ```typescript const waiting = await queue.getJobs('waiting', 0, 25); const failed = await queue.getJobs('failed', 0, 100); ``` ## removeJob ### Description Removes a job from the queue. ### Method `queue.removeJob(jobId)` or `job.remove()` ### Parameters #### Path Parameters - **jobId** (string) - Required - The ID of the job to remove. ### Request Example ```typescript // Remove by ID on queue await queue.removeJob('42'); // Remove via Job instance const job = await queue.getJob('42'); await job.remove(); ``` ``` -------------------------------- ### HTTP Proxy Setup Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/serverless.md Set up an Express-based HTTP proxy for enqueueing, request-reply, queue telemetry, and SSE consumption. Configure connection options and an optional allowlist for queues. ```typescript import { createProxyServer } from 'glide-mq/proxy'; const proxy = createProxyServer({ connection: ConnectionOptions, // required unless client provided client?: Client, // pre-existing GLIDE client prefix?: string, // default: 'glide' queues?: string[], // allowlist (403 for unlisted queues) compression?: 'none' | 'gzip', onError?: (err, queueName) => void, }); proxy.app.listen(3000); await proxy.close(); // shuts down all cached Queue instances ``` -------------------------------- ### Import Testing Utilities Source: https://github.com/avifenesh/glide-mq/blob/main/CLAUDE.md Import `TestQueue` and `TestWorker` from `glide-mq/testing` for in-memory testing. This setup is useful when a Valkey instance is not available. ```typescript import { TestQueue, TestWorker } from "glide-mq/testing" ``` -------------------------------- ### Bee-Queue vs glide-mq Connection Setup Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bee/SKILL.md Illustrates the difference in establishing a connection to Redis between Bee-Queue and glide-mq. glide-mq uses a structured 'connection' object. ```typescript // BEFORE (Bee-Queue) const Queue = require('bee-queue'); const queue = new Queue('tasks', { redis: { host: 'localhost', port: 6379 } }); // AFTER (glide-mq) import { Queue, Worker } from 'glide-mq'; const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); ``` -------------------------------- ### QueueEvents: Replaying All Events Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Example of configuring `QueueEvents` to replay all events from the beginning by setting `lastEventId: '0'`. This is compatible with BullMQ's behavior. ```typescript const qe = new QueueEvents('tasks', { connection, lastEventId: '0', }); ``` -------------------------------- ### HTTP Proxy - AI Usage Summary Source: https://context7.com/avifenesh/glide-mq/llms.txt Get a rolling summary of AI usage over a specified time window for one or more queues. ```bash # Rolling AI usage summary (last hour) curl 'http://localhost:3000/usage/summary?windowMs=3600000&queues=llm-tasks,embeddings' ``` -------------------------------- ### OpenTelemetry Integration Setup Source: https://github.com/avifenesh/glide-mq/blob/main/docs/OBSERVABILITY.md glide-mq emits OpenTelemetry spans automatically when `@opentelemetry/api` is installed. No code changes are required beyond installing the package and initializing your tracer provider. ```bash npm install @opentelemetry/api ``` -------------------------------- ### Quick Start Producer Source: https://github.com/avifenesh/glide-mq/blob/main/docs/SERVERLESS.md Initialize a Producer for serverless environments. Set `events: false` to skip XADD event emission, saving a Redis call per add operation. ```typescript import { Producer } from 'glide-mq'; const producer = new Producer('emails', { connection: { addresses: [{ host: 'localhost', port: 6379 }] }, events: false, // skip XADD event emission - saves 1 redis.call() per add }); const jobId = await producer.add('send-welcome', { to: 'user@example.com', subject: 'Welcome!', }); console.log(`Enqueued job ${jobId}`); await producer.close(); ``` -------------------------------- ### ServerlessPool Usage Source: https://github.com/avifenesh/glide-mq/blob/main/docs/SERVERLESS.md Demonstrates how to use the serverlessPool singleton or create a custom ServerlessPool instance. ```APIDOC ## ServerlessPool ```typescript import { serverlessPool, ServerlessPool } from 'glide-mq'; // Use the module-level singleton const producer = serverlessPool.getProducer('queue', { connection }); // Or create your own pool const pool = new ServerlessPool(); pool.getProducer('queue', { connection }); await pool.closeAll(); ``` ``` -------------------------------- ### Creating an Index Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/search.md Demonstrates how to create a Valkey Search index for job hashes, with options for including vector fields for KNN search and additional schema fields. ```APIDOC ## Creating an Index ```typescript import { Queue } from 'glide-mq'; import type { Field } from 'glide-mq'; const queue = new Queue('embeddings', { connection }); // Minimal index (base fields only, no vector search) await queue.createJobIndex(); // Index with vector field for KNN search await queue.createJobIndex({ name: 'embeddings-idx', // default: '{queueName}-idx' vectorField: { name: 'embedding', // field name in the job hash dimensions: 1536, // vector dimensions (e.g. OpenAI ada-002) algorithm: 'HNSW', // 'HNSW' (default) or 'FLAT' distanceMetric: 'COSINE', // 'COSINE' (default), 'L2', or 'IP' }, fields: [ // additional schema fields { type: 'TAG', name: 'category' } as Field, { type: 'TEXT', name: 'summary' } as Field, { type: 'NUMERIC', name: 'score' } as Field, ], }); ``` ### Auto-Included Base Fields Every index automatically includes: | Field | Type | |-------|------| | `name` | TAG | | `state` | TAG | | `timestamp` | NUMERIC | | `priority` | NUMERIC | ### JobIndexOptions ```typescript interface JobIndexOptions { name?: string; fields?: Field[]; vectorField?: { name: string; dimensions: number; algorithm?: 'HNSW' | 'FLAT'; distanceMetric?: 'COSINE' | 'L2' | 'IP'; }; createOptions?: IndexCreateOptions; } ``` ### IndexCreateOptions ```typescript interface IndexCreateOptions { score?: number; language?: string; skipInitialScan?: boolean; minStemSize?: number; withOffsets?: boolean; noOffsets?: boolean; noStopWords?: boolean; stopWords?: string[]; punctuation?: string; } ``` ``` -------------------------------- ### Per-Group Rate Limiting Source: https://github.com/avifenesh/glide-mq/blob/main/docs/ADVANCED.md Control the rate at which jobs for a specific key can start within a time window using `ordering.rateLimit`. Both `concurrency` and `rateLimit` must be satisfied for a job to start. ```typescript // Max 10 jobs per 60 seconds for each tenant await queue.add('sync', data, { ordering: { key: `tenant-${tenantId}`, concurrency: 3, rateLimit: { max: 10, duration: 60_000 }, }, }); ``` -------------------------------- ### Upsert Bounded Scheduler with Interval, Delayed Start, and Hard Stop Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/schedulers.md Schedule a job with a fixed interval, a delayed start time, and a hard end date, also including a job creation limit. ```typescript await queue.upsertJobScheduler( 'warmup-cache', { every: 30_000, startDate: Date.now() + 60_000, endDate: new Date('2026-12-31'), limit: 100, }, { name: 'warmup', data: { region: 'us-east' } }, ); ``` -------------------------------- ### Reproduce Performance Benchmarks Source: https://github.com/avifenesh/glide-mq/blob/main/README.md Command to reproduce the performance benchmarks shown in the documentation. This can be run using npm or npx. ```bash npm run bench ``` ```bash npx tsx benchmarks/elasticache-head-to-head.ts ``` -------------------------------- ### Get Repeatable Jobs Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Retrieve a list of configured schedulers using `getRepeatableJobs()`. ```typescript const schedulers = await queue.getRepeatableJobs(); // [{ name: 'daily-report', entry: { pattern, startDate?, endDate?, limit?, iterationCount?, nextRun, lastRun? } }] ``` -------------------------------- ### Exponential Backoff (BullMQ) Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Example of configuring exponential backoff for job retries in BullMQ. ```typescript await queue.add( 'process', { clientId, payload }, { attempts: 5, backoff: { type: 'exponential', delay: 1000 }, }, ); ``` -------------------------------- ### Create a Queue Source: https://github.com/avifenesh/glide-mq/blob/main/docs/USAGE.md Instantiate a new queue by providing a name and connection configuration. Ensure the connection details are correctly set for your message broker. ```typescript import { Queue } from 'glide-mq'; const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); ``` -------------------------------- ### Queue Constructor Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/queue.md Instantiate a new Queue with various configuration options. ```APIDOC ## Constructor ```typescript import { Queue } from 'glide-mq'; const queue = new Queue('tasks', { connection: ConnectionOptions, // required unless `client` provided client?: Client, // pre-existing GLIDE client (not owned) prefix?: string, // key prefix (default: 'glide') compression?: 'none' | 'gzip', // default: 'none' serializer?: Serializer, // default: JSON_SERIALIZER events?: boolean, // emit 'added' events (default: true) deadLetterQueue?: { name: string; maxRetries?: number }, }); ``` ``` -------------------------------- ### Add and Manage Jobs with Queue Source: https://context7.com/avifenesh/glide-mq/llms.txt Demonstrates adding single and bulk jobs, request-reply patterns, inspecting job states, fetching metrics, pausing/resuming, draining, cleaning old jobs, and obliterating the queue. ```typescript import { Queue } from 'glide-mq'; const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); // Add a single job with options const job = await queue.add( 'send-email', { to: 'user@example.com', subject: 'Welcome' }, { delay: 5_000, // run after 5 seconds priority: 1, // lower = higher priority attempts: 3, // max 3 total attempts backoff: { type: 'exponential', delay: 1_000 }, timeout: 30_000, // fail if processor exceeds 30s removeOnComplete: true, removeOnFail: false, }, ); // Bulk add — uses GLIDE Batch API for high throughput await queue.addBulk([ { name: 'job1', data: { a: 1 } }, { name: 'job2', data: { a: 2 } }, ]); // Request-reply — add a job and wait for its result const result = await queue.addAndWait('inference', { prompt: 'Hello' }, { waitTimeout: 30_000 }); // Inspect jobs by state const waiting = await queue.getJobs('waiting', 0, 49); const active = await queue.getJobs('active', 0, 49); const failed = await queue.getJobs('failed', 0, 49); // Lightweight metadata fetch (omits data/returnvalue) const lite = await queue.getJobs('waiting', 0, 99, { excludeData: true }); // Count jobs per state const counts = await queue.getJobCounts(); // { waiting: 12, active: 3, delayed: 5, completed: 842, failed: 7 } // Per-minute throughput metrics (retained 24 h) const metrics = await queue.getMetrics('completed', { start: -10 }); // last 10 buckets // Pause / resume await queue.pause(); await queue.resume(); // Remove waiting jobs await queue.drain(); // waiting only await queue.drain(true); // + delayed // Clean old jobs const removedIds = await queue.clean(60_000 * 60, 1000, 'completed'); // Wipe all queue data await queue.obliterate({ force: true }); await queue.close(); ``` -------------------------------- ### Get Dead Letter Jobs Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Glide-MQ only: Retrieves jobs that have been moved to the dead-letter queue. ```APIDOC ## Get Dead Letter Jobs ### Description This method is exclusive to glide-mq. It retrieves jobs that have failed and been moved to the dead-letter queue. This allows for inspection and potential reprocessing of failed jobs. ### Method `queue.getDeadLetterJobs(start, end)` ### Parameters - **start** (number) - Required - The starting index for the range of jobs. - **end** (number) - Required - The ending index for the range of jobs. ``` -------------------------------- ### Get All Registered Schedulers Source: https://github.com/avifenesh/glide-mq/blob/main/docs/ADVANCED.md Retrieve a list of all currently defined repeatable job schedulers. ```typescript // List all registered schedulers const schedulers = await queue.getRepeatableJobs(); ``` -------------------------------- ### Queue Rate Limit Management Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Methods for getting and removing global rate limits for queues. ```APIDOC ## Queue Rate Limit Management ### `queue.getGlobalRateLimit()` #### Description Retrieves the global rate limit settings for the queue. ### `queue.removeGlobalRateLimit()` #### Description Removes the global rate limit settings for the queue. ``` -------------------------------- ### In-Memory Queue and Worker Setup Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/serverless.md Set up an in-memory queue and worker for testing purposes. No connection configuration is needed. The worker processes jobs and emits 'completed' or 'failed' events. ```typescript import { TestQueue, TestWorker } from 'glide-mq/testing'; const queue = new TestQueue('tasks'); // no connection config needed const worker = new TestWorker(queue, async (job) => { return { processed: job.data }; }); worker.on('completed', (job, result) => { ... }); worker.on('failed', (job, err) => { ... }); await queue.add('send-email', { to: 'user@example.com' }); const counts = await queue.getJobCounts(); // { waiting: 0, active: 0, delayed: 0, completed: 1, failed: 0 } await worker.close(); await queue.close(); ``` -------------------------------- ### Create Glide-MQ Proxy Server Source: https://github.com/avifenesh/glide-mq/blob/main/docs/USAGE.md Instantiate a proxy server with optional queue allowlist. Ensure you add your own authentication and rate-limiting middleware before exposing the proxy to a network. ```typescript import { createProxyServer } from 'glide-mq/proxy'; const proxy = createProxyServer({ connection, queues: ['tasks', 'broadcast-events'], // optional allowlist }); proxy.app.listen(3000); ``` -------------------------------- ### Get Job Logs Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Retrieves logs for a specific job. This method is fully compatible with BullMQ. ```APIDOC ## Get Job Logs ### Description Retrieves the logs associated with a specific job ID. This method is fully compatible with BullMQ. ### Method `queue.getJobLogs(id, start, end)` ### Parameters - **id** (string | number) - Required - The ID of the job. - **start** (number) - Required - The starting line number for logs. - **end** (number) - Required - The ending line number for logs. ``` -------------------------------- ### Get Metrics Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Retrieves queue metrics. Glide-MQ's signature is simplified compared to BullMQ. ```APIDOC ## Get Metrics ### Description Retrieves performance metrics for the queue. Glide-MQ's signature is simplified, accepting only a `type` argument, unlike BullMQ which also accepts `start` and `end` times. ### Method `queue.getMetrics(type)` ### Parameters - **type** (string) - Required - The type of metrics to retrieve (e.g., 'completed', 'failed', 'delayed'). ``` -------------------------------- ### Count Jobs Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Gets the total number of jobs in the queue. This method is fully compatible with BullMQ. ```APIDOC ## Count Jobs ### Description Returns the total number of jobs currently in the queue, regardless of their status. This method is fully compatible with BullMQ. ### Method `queue.count()` ### Returns - (number) - The total number of jobs in the queue. ``` -------------------------------- ### Initialize Queue Instance Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/schedulers.md Instantiate a new Queue with a given name and connection details. ```typescript const queue = new Queue('tasks', { connection }); ``` -------------------------------- ### Basic Connection Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/connection.md Establishes a basic connection to a local Glide MQ instance. ```APIDOC ## Basic Connection ```typescript const connection = { addresses: [{ host: 'localhost', port: 6379 }] }; const queue = new Queue('tasks', { connection }); ``` ``` -------------------------------- ### Get Job by ID (Queue API) Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bee/references/api-mapping.md The API for retrieving a single job by its ID remains the same. ```typescript // BEFORE const job = await queue.getJob('42'); // AFTER - same API const job = await queue.getJob('42'); ``` -------------------------------- ### Run Benchmarks Source: https://github.com/avifenesh/glide-mq/blob/main/CLAUDE.md Execute benchmarks for glide-mq, comparing its performance against BullMQ. This helps in evaluating performance characteristics. ```bash npm run bench ``` -------------------------------- ### Instantiate a Queue Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/queue.md Create a new Queue instance. Connection options are required unless a pre-existing client is provided. Other options control prefix, compression, serialization, event emission, and dead-letter queue configuration. ```typescript import { Queue } from 'glide-mq'; const queue = new Queue('tasks', { connection: ConnectionOptions, // required unless `client` provided client?: Client, // pre-existing GLIDE client (not owned) prefix?: string, // key prefix (default: 'glide') compression?: 'none' | 'gzip', // default: 'none' serializer?: Serializer, // default: JSON_SERIALIZER events?: boolean, // emit 'added' events (default: true) deadLetterQueue?: { name: string; maxRetries?: number }, }); ``` -------------------------------- ### Get Job Details Source: https://github.com/avifenesh/glide-mq/blob/main/docs/OBSERVABILITY.md Retrieves the details, state, and logs for a specific job within a queue. ```APIDOC ## GET /api/queues/:name/jobs/:id ### Description Single job details, state, and logs. ### Method GET ### Endpoint /api/queues/:name/jobs/:id ``` -------------------------------- ### Load and Call Speedkey Functions in TypeScript Source: https://github.com/avifenesh/glide-mq/blob/main/docs/ARCHITECTURE.md Demonstrates how to load a library of functions and then call a specific function using the client. In cluster mode, the library can be loaded to all nodes. ```typescript // Load library (once, on init) await client.functionLoad(librarySource, { replace: true }); // Call function await client.fcall('glidemq_addJob', [key1, key2, key3, key4], [arg1, arg2, ...]); // In cluster mode, load to all nodes await clusterClient.functionLoad(librarySource, true, { route: 'allPrimaries' }); ``` -------------------------------- ### Get Queue Details Source: https://github.com/avifenesh/glide-mq/blob/main/docs/OBSERVABILITY.md Fetches detailed information about a specific queue, including recent jobs. ```APIDOC ## GET /api/queues/:name ### Description Queue details + recent jobs. ### Method GET ### Endpoint /api/queues/:name ``` -------------------------------- ### Add Job using go-valkey Source: https://github.com/avifenesh/glide-mq/blob/main/docs/WIRE_PROTOCOL.md Demonstrates how to add a job to the GlideMQ queue using the `go-valkey` library. Ensure the Valkey server is running and accessible at localhost:6379. ```go package main import ( "context" "fmt" "strconv" "time" "github.com/valkey-io/valkey-go" ) func main() { client, err := valkey.NewClient(valkey.ClientOption{ InitAddress: []string{"localhost:6379"}, }) if err != nil { panic(err) } defer client.Close() ctx := context.Background() queueName := "tasks" prefix := "glide" p := fmt.Sprintf("%s:{%s}", prefix, queueName) timestamp := strconv.FormatInt(time.Now().UnixMilli(), 10) jobData := `{"to":"user@example.com","subject":"Hello"}` jobOpts := `{}` result := client.Do(ctx, client.B().Fcall(). Function("glidemq_addJob"). Numkeys(4). Key(p+":id"). Key(p+":stream"). Key(p+":scheduled"). Key(p+":events"). Arg("send-email"). // jobName Arg(jobData). // jobData Arg(jobOpts). // jobOpts Arg(timestamp). // timestamp Arg("0"). // delay Arg("0"). // priority Arg(""). // parentId Arg("0"). // maxAttempts Arg(""). // orderingKey Arg("0"). // groupConcurrency Arg("0"). // groupRateMax Arg("0"). // groupRateDuration Arg("0"). // tbCapacity Arg("0"). // tbRefillRate Arg("0"). // jobCost Arg("0"). // ttl Arg(""). // customJobId Arg("0"). // lifo Arg(""). // parentQueue Arg(""). // schedulerName Arg("0"). // skipEvents Build(), ) jobId, err := result.ToString() if err != nil { panic(err) } fmt.Printf("Job created with ID: %s\n", jobId) } ``` -------------------------------- ### Get Workers Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Retrieves information about active workers processing jobs. This method is fully compatible with BullMQ. ```APIDOC ## Get Workers ### Description Retrieves a list of active workers currently processing jobs in the queue. This method is fully compatible with BullMQ. ### Method `queue.getWorkers()` ### Returns - (array) - An array of worker objects. ``` -------------------------------- ### Get Job Scheduler Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Retrieves a specific job scheduler configuration. Glide-MQ uses `name` instead of `id`. ```APIDOC ## Get Job Scheduler ### Description Retrieves the configuration for a specific job scheduler. Glide-MQ uses the job `name` as the identifier, differing from BullMQ's use of `id`. ### Method `queue.getJobScheduler(name)` ### Parameters - **name** (string) - Required - The name of the job scheduler. ``` -------------------------------- ### Recommended Valkey Config for Production Queues Source: https://github.com/avifenesh/glide-mq/blob/main/docs/DURABILITY.md This configuration enables AOF persistence with a balance between performance and durability, and optionally uses RDB preamble for faster restarts. Tune RDB snapshots based on backup needs. ```conf appendonly yes appendfsync everysec # If supported by your Valkey version: aof-use-rdb-preamble yes ``` ```conf save 60 10000 ``` -------------------------------- ### Queue Constructor Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Instantiate a new queue with glide-mq, which offers additional options compared to BullMQ. ```APIDOC ## Queue Constructor ### Description Initializes a new queue instance. Glide-MQ extends the BullMQ constructor with `deadLetterQueue` and `compression` options. ### Method `new Queue(name, { connection, prefix, deadLetterQueue, compression })` ### Parameters - **name** (string) - Required - The name of the queue. - **connection** (object) - Required - The Redis connection options. - **prefix** (string) - Optional - The prefix for Redis keys. - **deadLetterQueue** (object) - Optional - Configuration for the dead-letter queue. - **compression** (boolean) - Optional - Enables or disables job data compression. ``` -------------------------------- ### Get Job Count By Types Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Retrieves the count of jobs for specified types. This method is fully compatible with BullMQ. ```APIDOC ## Get Job Count By Types ### Description Retrieves the number of jobs that match the specified types. This method is fully compatible with BullMQ. ### Method `queue.getJobCountByTypes(...types)` ### Parameters - **...types** (string) - Optional - A list of job types to count. ``` -------------------------------- ### Web UI Integration: Bee-Queue Arena vs. Glide-MQ Dashboard Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bee/SKILL.md Shows how to integrate the web UI for monitoring, comparing Bee-Queue's Arena with Glide-MQ's Dashboard. ```typescript // BEFORE (Bee-Queue) - Arena const Arena = require('bull-arena'); app.use('/', Arena({ Bee: require('bee-queue'), queues: [{ name: 'tasks' }] })); // AFTER (glide-mq) - Dashboard import { createDashboard } from '@glidemq/dashboard'; app.use('/dashboard', createDashboard([queue])); ``` -------------------------------- ### Get Job by ID Source: https://github.com/avifenesh/glide-mq/blob/main/docs/MIGRATION.md Retrieves a specific job from the queue using its ID. This method is fully compatible with BullMQ. ```APIDOC ## Get Job by ID ### Description Retrieves a job object based on its unique ID. This method is fully compatible with BullMQ. ### Method `queue.getJob(id)` ### Parameters - **id** (string | number) - Required - The ID of the job to retrieve. ``` -------------------------------- ### Bee-Queue Worker Processing vs glide-mq Worker Setup Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bee/SKILL.md Shows how to set up a worker to process jobs. Bee-Queue integrates processing directly with the queue, whereas glide-mq uses a separate 'Worker' class with its own event listeners. ```typescript // BEFORE (Bee-Queue) queue.process(10, async (job) => { return { processed: true }; }); queue.on('succeeded', (job, result) => console.log('Done:', result)); // AFTER (glide-mq) - separate Worker class const worker = new Worker('tasks', async (job) => { return { processed: true }; }, { connection, concurrency: 10 }); worker.on('completed', (job) => console.log('Done:', job.returnValue)); ``` -------------------------------- ### Get Usage Summary Source: https://github.com/avifenesh/glide-mq/blob/main/docs/USAGE.md Retrieves a rolling summary of AI usage. Supports time window and queue filtering. ```APIDOC ## GET /usage/summary ### Description Rolling AI usage summary. Supports `windowMs` (or legacy `window`) in ms, `start`/`end` as epoch ms, and `queues=a,b` (for example `?windowMs=3600000`). ### Method GET ### Endpoint /usage/summary ### Query Parameters - **windowMs** (integer) - Optional - Time window in milliseconds. - **window** (integer) - Optional - Legacy time window in milliseconds. - **start** (integer) - Optional - Start epoch time in milliseconds. - **end** (integer) - Optional - End epoch time in milliseconds. - **queues** (string) - Optional - Comma-separated list of queues to filter by. ``` -------------------------------- ### Get a Specific Job Source: https://github.com/avifenesh/glide-mq/blob/main/docs/USAGE.md Retrieve a job by its unique ID. This is useful for checking the status or details of a particular job. ```typescript const job = await queue.getJob('42'); ``` -------------------------------- ### Integrate Glide-MQ Dashboard Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq-migrate-bee/references/new-features.md Embed the Glide-MQ dashboard into an Express.js application for monitoring and managing queues. Requires installation of `@glidemq/dashboard`. ```typescript import { createDashboard } from '@glidemq/dashboard'; import express from 'express'; const app = express(); app.use('/dashboard', createDashboard([queue])); ``` -------------------------------- ### Valkey Modules (Search / JSON / Bloom) Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/connection.md Information on using Valkey modules like Search, JSON, and Bloom with Glide MQ, often requiring `valkey-bundle`. ```APIDOC ## Valkey Modules (Search / JSON / Bloom) Vector search (`queue.createJobIndex()`, `queue.vectorSearch()`) requires the `valkey-search` module loaded on the server. The easiest way to get all modules is to use `valkey-bundle`, which bundles search, JSON, bloom, and other modules: ```bash # Docker (standalone with all modules) docker run -p 6379:6379 valkey/valkey-bundle:latest ``` ``` -------------------------------- ### glidemq_addFlow Source: https://github.com/avifenesh/glide-mq/blob/main/docs/WIRE_PROTOCOL.md Atomically creates a parent job and all child jobs. The parent starts in `waiting-children` state and is re-queued when all children complete. ```APIDOC ## FCALL glidemq_addFlow ### Description Atomically creates a parent job and all child jobs. The parent starts in `waiting-children` state and is re-queued when all children complete. ### Keys (4 + 4 per child) - **1**: `glide:{parentQueue}:id` - **2**: `glide:{parentQueue}:stream` - **3**: `glide:{parentQueue}:scheduled` - **4**: `glide:{parentQueue}:events` - **4+(i-1)*4+1**: `glide:{childQueue_i}:id` - **4+(i-1)*4+2**: `glide:{childQueue_i}:stream` - **4+(i-1)*4+3**: `glide:{childQueue_i}:scheduled` - **4+(i-1)*4+4**: `glide:{childQueue_i}:events` ### Args (9 parent + 9 per child + extra deps) **Parent args (positions 1-9):** - **1**: `parentName` (string) - **2**: `parentData` (string (JSON)) - **3**: `parentOpts` (string (JSON)) - **4**: `timestamp` (string (int)) - **5**: `parentDelay` (string (int)) - **6**: `parentPriority` (string (int)) - **7**: `parentMaxAttempts` (string (int)) - **8**: `numChildren` (string (int)) - **9**: `parentCustomId` (string) **Child args (9 per child, starting at position 10):** For child `i` (1-based), base = `9 + (i-1) * 9`: - **base+1**: `childName` (string) - **base+2**: `childData` (string (JSON)) - **base+3**: `childOpts` (string (JSON)) - **base+4**: `childDelay` (string (int)) - **base+5**: `childPriority` (string (int)) - **base+6**: `childMaxAttempts` (string (int)) - **base+7**: `childQueuePrefix` (string) - **base+8**: `childParentQueue` (string) - **base+9**: `childCustomId` (string) **Extra deps (after all children):** - **9 + numChildren*9 + 1**: `numExtraDeps` (string int) - **9 + numChildren*9 + 2..N**: `extraDepsMember` (string) ### Return Value JSON-encoded array: `["parentId", "child1Id", "child2Id", ...]` Returns `["duplicate"]` if any custom job ID already exists, or `"ERR:COST_EXCEEDS_CAPACITY"` if a cost exceeds bucket capacity. ``` -------------------------------- ### Load Valkey Search Module Source: https://github.com/avifenesh/glide-mq/blob/main/skills/glide-mq/references/connection.md Load the Valkey Search module explicitly using the valkey-server command. This is required for vector search functionality. ```bash valkey-server --loadmodule /path/to/valkeysearch.so ```