### Minimal Example Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/REFERENCE.md A basic example demonstrating how to create and start a consumer with a simple message handler. ```javascript import { Consumer } from "sqs-consumer"; const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/123456789012/queue", handleMessage: async (message) => { console.log("Processing:", message.Body); return message; // acknowledge }, }); consumer.on("error", (err) => console.error(err)); consumer.start(); ``` -------------------------------- ### Basic sqs-consumer Usage Source: https://github.com/bbc/sqs-consumer/blob/main/README.md This snippet demonstrates the basic setup for an SQS consumer. It shows how to create a consumer instance, define a message handler, and start the consumer. Ensure you replace the placeholder queue URL with your actual SQS queue URL. ```javascript import { Consumer } from "sqs-consumer"; const app = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { // do some work with `message` }, }); app.on("error", (err) => { console.error(err.message); }); app.on("processing_error", (err) => { console.error(err.message); }); app.start(); ``` -------------------------------- ### Example: Listening for 'started' and 'stopped' events once Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md This example shows how to use the `once` method to execute a callback function only the first time the 'started' or 'stopped' events are emitted. This is useful for performing actions like logging or exiting the process exactly once. ```javascript const consumer = Consumer.create({ /* ... */ }); consumer.once("started", (metadata) => { console.log("Consumer started, beginning processing"); }); consumer.once("stopped", (metadata) => { console.log("Consumer stopped, exiting gracefully"); process.exit(0); }); consumer.start(); ``` -------------------------------- ### Example: Update Batch Size and Listen for Updates Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/consumer.md Demonstrates how to create a consumer, start it, update its batch size, and listen for the 'option_updated' event. This is useful for dynamically scaling message processing. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", batchSize: 1, handleMessage: async (message) => { return message; }, }); consumer.start(); // Later, increase batch size consumer.updateOption("batchSize", 5); // Listen for updates consumer.on("option_updated", (option, value) => { console.log(`Updated ${option} to ${value}`); }); ``` -------------------------------- ### SQS Consumer Example Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/consumer.md Demonstrates creating, configuring, and starting an SQS consumer. Includes event handlers for message reception, processing, errors, and queue status. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { return message; }, }); consumer.on("message_received", (message, metadata) => { console.log("Message received:", message.MessageId); }); consumer.on("message_processed", (message, metadata) => { console.log("Message processed:", message.MessageId); }); consumer.on("error", (err, messages, metadata) => { console.error("Error:", err.message, "Queue:", metadata.queueUrl); }); consumer.on("processing_error", (err, message, metadata) => { console.error("Processing error:", err.message); }); consumer.on("empty", (metadata) => { console.log("Queue is empty"); }); consumer.start(); ``` -------------------------------- ### Quick Start SQS Consumer Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/README.md A basic JavaScript example demonstrating how to create, start, and gracefully shut down an SQS consumer. It includes error handling for general and processing errors. ```javascript import { Consumer } from "sqs-consumer"; const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/123456789012/my-queue", handleMessage: async (message) => { console.log("Message:", message.Body); return message; // acknowledge and delete }, }); consumer.on("error", (err) => { console.error("Error:", err.message); }); consumer.on("processing_error", (err) => { console.error("Handler error:", err.message); // Message will be retried }); consumer.start(); // Graceful shutdown process.on("SIGTERM", () => { consumer.stop(); consumer.once("stopped", () => { console.log("Consumer stopped"); process.exit(0); }); }); ``` -------------------------------- ### Install sqs-consumer Package Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/getting-started.md Install the sqs-consumer package and its AWS SDK dependency using npm. ```bash npm install sqs-consumer @aws-sdk/client-sqs ``` -------------------------------- ### Example: Listening for message_received and error events Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md This example demonstrates how to use the `on` method to log received message IDs and any errors encountered during processing. It requires a Consumer instance to be created. ```javascript const consumer = Consumer.create({ /* ... */ }); consumer.on("message_received", (message, metadata) => { console.log(`Received from ${metadata.queueUrl}:`, message.MessageId); }); consumer.on("error", (err, messages, metadata) => { console.error("Error:", err.message); }); ``` -------------------------------- ### Configuring SQS Client with Custom Options Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Example of initializing an SQSClient with custom configuration, including region, credentials, and retry strategy. ```typescript import { SQSClient, ReceiveMessageCommand, DeleteMessageCommand, ChangeMessageVisibilityCommand, } from "@aws-sdk/client-sqs"; ``` ```javascript import { SQSClient } from "@aws-sdk/client-sqs"; const sqsClient = new SQSClient({ region: "eu-west-1", credentials: { accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, }, maxAttempts: 5, }); const consumer = Consumer.create({ queueUrl: "...", sqs: sqsClient, handleMessage: async (message) => message, }); ``` -------------------------------- ### Install sqs-consumer using npm Source: https://github.com/bbc/sqs-consumer/blob/main/README.md Install the sqs-consumer package using npm. This command is used to add the package to your project's dependencies. ```bash npm install sqs-consumer ``` -------------------------------- ### Minimal SQS Consumer Setup Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/getting-started.md This is the most basic setup for consuming SQS messages. It defines the queue URL and a handler function for processing each message. Errors are logged to the console. ```javascript import { Consumer } from "sqs-consumer"; const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/123456789012/my-queue", handleMessage: async (message) => { console.log("Processing message:", message.Body); // Return the message to acknowledge and delete it return message; }, }); consumer.on("error", (err) => { console.error("Consumer error:", err.message); }); consumer.start(); ``` -------------------------------- ### Type-Safe SQS Consumer Setup and Event Handling Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Demonstrates a complete TypeScript setup for the SQS Consumer, including type-safe message processing, consumer creation, and event handlers for 'message_received', 'processing_error', and 'error'. ```typescript import type { Message, MessageAttributeValue } from "@aws-sdk/client-sqs"; import { Consumer, SQSError, StandardError, TimeoutError } from "sqs-consumer"; interface ProcessingResult { success: boolean; message: string; } // Type-safe message processing async function processMessage(message: Message): Promise { if (!message.Body) { return { success: false, message: "Empty message body" }; } try { const data = JSON.parse(message.Body); // Process data return { success: true, message: "Processed" }; } catch (err) { return { success: false, message: String(err) }; } } // Create consumer with type safety const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account/queue", handleMessage: async (message: Message): Promise => { const result = await processMessage(message); return result.success ? message : undefined; }, }); // Type-safe event handlers consumer.on("message_received", (message: Message) => { console.log("Received:", message.MessageId); }); consumer.on("processing_error", (err: StandardError, message: Message) => { console.error("Processing failed:", err.cause?.message); }); consumer.on("error", (err: Error, messages: Message | Message[] | undefined) => { if (err instanceof SQSError) { console.error("SQS Error:", err.code); } }); consumer.start(); ``` -------------------------------- ### Example: Tracking completed batches Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md This example utilizes the `response_processed` event to count the number of completed batches of messages. It logs the completion of each batch. ```javascript let batchCount = 0; consumer.on("response_processed", (metadata) => { batchCount++; console.log(`Completed batch #${batchCount}`); }); ``` -------------------------------- ### Minimal SQS Consumer Setup Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/README.md Sets up a basic SQS consumer to process messages from a specified queue URL. It logs message bodies and handles errors during operation. ```javascript import { Consumer } from "sqs-consumer"; const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account/queue", handleMessage: async (message) => { console.log("Processing:", message.Body); return message; // acknowledge }, }); consumer.on("error", (err) => console.error(err)); consumer.start(); ``` -------------------------------- ### SQS Consumer Configuration Options Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/README.md Provides an example of creating an SQS consumer with various configuration options, including batch size, polling settings, timeouts, and visibility configurations. ```javascript const consumer = Consumer.create({ queueUrl: "...", batchSize: 5, waitTimeSeconds: 15, visibilityTimeout: 300, heartbeatInterval: 60, handleMessageTimeout: 30000, pollingCompleteWaitTimeMs: 10000, handleMessage: async (message) => message, }); ``` -------------------------------- ### Example: Tracking received message count Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md This example demonstrates how to use the `message_received` event to count the number of messages received from SQS. It logs the count for each received message. ```javascript let receivedCount = 0; consumer.on("message_received", (message, metadata) => { receivedCount++; console.log(`Received message #${receivedCount}: ${message.MessageId}`); }); ``` -------------------------------- ### QueueMetadata Example Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/types.md Demonstrates how to access the `queueUrl` from the `metadata` object within an event listener, such as `message_received`, to identify the source queue. ```javascript consumer.on("message_received", (message, metadata) => { console.log(`Message from queue: ${metadata.queueUrl}`); }); ``` -------------------------------- ### Basic SQS Consumer with Message Handling Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/getting-started.md A standard setup for an SQS consumer, demonstrating message processing and error logging. It includes the necessary imports and consumer creation. ```javascript import { Consumer } from "sqs-consumer"; const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/123456789012/my-queue", handleMessage: async (message) => { // Process the message console.log("Message body:", message.Body); // Return to acknowledge, undefined to skip return message; }, }); consumer.on("error", (err) => { console.error("Error:", err); }); consumer.start(); ``` -------------------------------- ### Example: Tracking successfully processed messages per queue Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md This example uses the `message_processed` event to maintain a count of successfully processed messages for each queue. It logs these counts periodically. ```javascript const successCount = new Map(); consumer.on("message_processed", (message, metadata) => { const queue = metadata.queueUrl; successCount.set(queue, (successCount.get(queue) || 0) + 1); }); setInterval(() => { console.log("Processed counts:", Object.fromEntries(successCount)); }, 10000); ``` -------------------------------- ### Handle Consumer Started Event Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md Emitted when start() is called and polling begins. This event is emitted before the first poll is initiated. ```typescript consumer.on("started", (metadata: QueueMetadata) => { // ... }); ``` ```javascript consumer.on("started", (metadata) => { console.log(`Consumer started polling ${metadata.queueUrl}`); startMetricsReporting(); }); ``` -------------------------------- ### Consuming Queue Attributes Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Example of creating an SQS consumer and specifying queue attribute names to retrieve. ```javascript const consumer = Consumer.create({ queueUrl: "...", attributeNames: ["ApproximateNumberOfMessages", "VisibilityTimeout"], // Note: attributeNames are queue attributes, not message attributes handleMessage: async (message) => { return message; }, }); ``` -------------------------------- ### Listen for Consumer Started Event Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/consumer.md Be notified when the consumer begins its polling process by subscribing to the 'started' event. This is typically emitted after calling the `start()` method. ```typescript consumer.on("started", (metadata: QueueMetadata) => { console.log("Consumer started polling"); }); ``` -------------------------------- ### Example: Handling System Attributes Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Demonstrates how to configure and access system attributes like SenderId, SentTimestamp, and ApproximateReceiveCount within the message handler. It shows parsing timestamps and retry counts. ```javascript const consumer = Consumer.create({ queueUrl: "...", messageSystemAttributeNames: ["SenderId", "SentTimestamp", "ApproximateReceiveCount"], handleMessage: async (message) => { const attrs = message.Attributes || {}; console.log({ senderId: attrs.SenderId, sentAt: new Date(parseInt(attrs.SentTimestamp || "0")), retryCount: parseInt(attrs.ApproximateReceiveCount || "0"), }); return message; }, }); ``` -------------------------------- ### Implement Pre/Post Message Hooks Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/advanced-usage.md Use preReceiveMessageCallback and postReceiveMessageCallback to execute setup and cleanup logic around each polling cycle. These callbacks are useful for initializing request contexts, setting up tracing, or flushing metrics. ```javascript const consumer = Consumer.create({ queueUrl: "...", preReceiveMessageCallback: async () => { // Setup before poll console.log("Starting poll cycle"); // Could initialize request context, set up tracing, etc. }, postReceiveMessageCallback: async () => { // Cleanup after poll console.log("Completed poll cycle"); // Could cleanup context, flush metrics, etc. }, handleMessage: async (message) => message, }); ``` -------------------------------- ### once(event, listener) Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md Register a listener to be called only once when the event is emitted, then automatically removed. This is useful for one-time setup or cleanup tasks. ```APIDOC ## once(event, listener) ### Description Register a listener to be called only once when the event is emitted, then automatically removed. ### Method `once` ### Parameters #### Path Parameters - **event** (keyof Events) - Required - Event name to listen for - **listener** (function) - Required - Callback function invoked once when event is emitted ### Returns `this` (allows chaining) ### Example ```javascript const consumer = Consumer.create({ /* ... */ }); consumer.once("started", (metadata) => { console.log("Consumer started, beginning processing"); }); consumer.once("stopped", (metadata) => { console.log("Consumer stopped, exiting gracefully"); process.exit(0); }); consumer.start(); ``` ``` -------------------------------- ### Consumer Lifecycle Methods Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/README.md Control the consumer's lifecycle using `start()`, `stop()`, and `updateOption()`. Check status with the `status` property. ```typescript consumer.start(): void consumer.stop(options?: StopOptions): void consumer.status: { isRunning: boolean; isPolling: boolean } consumer.updateOption(option: UpdatableOptions, value: any): void ``` -------------------------------- ### Configure SQS Consumer with Polling Options Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/configuration.md Set up a consumer with custom batch size, long polling wait time, and polling delay between requests. This example demonstrates fetching 5 messages per poll, waiting up to 15 seconds for messages, and introducing a 100ms delay between polls. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", batchSize: 5, // fetch 5 messages per poll waitTimeSeconds: 15, // wait up to 15 seconds for messages pollingWaitTimeMs: 100, // wait 100ms between polls handleMessageBatch: async (messages) => { console.log(`Processing batch of ${messages.length}`); return messages; }, }); ``` -------------------------------- ### Example: Handling an SQS Message Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Demonstrates how to process a message within the `handleMessage` function. It shows accessing message ID, body, custom attributes, system attributes, and verifying the MD5 hash of the body. The handler must return the message to acknowledge it. ```javascript const consumer = Consumer.create({ queueUrl: "...", messageAttributeNames: ["userId", "eventType"], handleMessage: async (message) => { console.log("Message ID:", message.MessageId); console.log("Body:", message.Body); // Access custom attributes const userId = message.MessageAttributes?.userId?.StringValue; const eventType = message.MessageAttributes?.eventType?.StringValue; // Access system attributes const senderId = message.Attributes?.SenderId; const timestamp = message.Attributes?.SentTimestamp; // Verify integrity console.log("MD5 of body:", message.Md5OfBody); // Process the message await processMessage(JSON.parse(message.Body)); // Return to acknowledge return message; }, }); ``` -------------------------------- ### Example: Accessing Message Attributes Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Shows how to retrieve and check the data type of custom message attributes. It demonstrates accessing both string and binary attributes, converting binary data to a Buffer. ```javascript const message = /* from consumer */; // Access string attribute const userId = message.MessageAttributes?.userId; if (userId?.DataType === "String") { console.log("User ID:", userId.StringValue); } // Access binary attribute const binary = message.MessageAttributes?.data; if (binary?.DataType === "Binary") { const buffer = Buffer.from(binary.BinaryValue); console.log("Binary data:", buffer); } ``` -------------------------------- ### Handling Timeout Errors Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/errors.md Example of configuring a consumer with a message handler timeout and listening for the 'timeout_error' event. This snippet demonstrates how to identify and log timeout errors. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessageTimeout: 5000, // 5 second timeout handleMessage: async (message) => { // If this takes > 5000ms, TimeoutError is emitted await longRunningOperation(); return message; }, }); consumer.on("timeout_error", (err, message) => { if (err instanceof TimeoutError) { console.error("Handler timed out:", err.message); console.error("Message ID:", err.messageIds[0]); // Message will be retried (stays on queue) } }); consumer.start(); ``` -------------------------------- ### SQS Consumer Configuration with Type Safety Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Example of creating an SQS consumer instance with type-safe attribute names. Ensure the necessary types are imported from '@aws-sdk/client-sqs'. ```typescript import type { QueueAttributeName, MessageSystemAttributeName } from "@aws-sdk/client-sqs"; const consumer = Consumer.create({ queueUrl: "...", attributeNames: ["ApproximateNumberOfMessages"] as QueueAttributeName[], messageSystemAttributeNames: ["SentTimestamp", "ApproximateReceiveCount"] as MessageSystemAttributeName[], handleMessage: async (message) => message, }); ``` -------------------------------- ### TypeScript Usage with SQS Consumer Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/getting-started.md Example demonstrating the use of the SQS Consumer library with TypeScript, including type definitions for messages and errors. Handles message processing and error logging with type safety. ```typescript import { Consumer, SQSError } from "sqs-consumer"; import { Message } from "@aws-sdk/client-sqs"; const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/123456789012/my-queue", handleMessage: async (message: Message): Promise => { try { const data = JSON.parse(message.Body!); await processData(data); return message; } catch (err) { console.error("Error:", err); return undefined; } }, }); consumer.on("error", (err: Error, messages?: Message | Message[]) => { if (err instanceof SQSError) { console.error("SQS Error:", err.code, err.statusCode); } }); consumer.start(); ``` -------------------------------- ### Creating Type-Safe Batch Message Handler Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Example of a type-safe handler for processing batches of SQS messages, using Promise.all for concurrent processing. ```typescript import type { Message } from "@aws-sdk/client-sqs"; import { Consumer } from "sqs-consumer"; // Type-safe batch handler const handleMessageBatch = async (messages: Message[]): Promise => { const results = await Promise.all( messages.map(async (msg) => { try { await processData(JSON.parse(msg.Body || "{}")); return msg; } catch { return null; } }) ); return results.filter((msg): msg is Message => msg !== null); }; const consumer = Consumer.create({ queueUrl: "...", batchSize: 10, handleMessageBatch }); ``` -------------------------------- ### SQS Consumer with Batch Handler and Timeout Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/configuration.md Configure a consumer to process messages in batches with a specified timeout. This example processes messages concurrently and acknowledges only those that were successfully processed, returning their messages. ```javascript // Batch handler with timeout const consumer2 = Consumer.create({ queueUrl: "...", handleMessageTimeout: 30000, // 30 second timeout per batch handleMessageBatch: async (messages) => { // Process all messages concurrently const results = await Promise.all( messages.map(msg => processMessage(msg)) ); // Acknowledge successfully processed messages return results.filter(r => r.success).map(r => r.message); }, }); ``` -------------------------------- ### Set Environment Variables for SQS Consumer Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/configuration.md Configure AWS region and credentials using environment variables before starting the consumer. This is useful for local development or specific deployment scenarios. ```bash export AWS_REGION=eu-west-1 export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY npm start ``` -------------------------------- ### Graceful Shutdown Implementation Source: https://github.com/bbc/sqs-consumer/blob/main/README.md Implements a graceful shutdown for the SQS consumer, allowing time for in-flight messages to be processed before stopping. This example uses pollingCompleteWaitTimeMs and event listeners for shutdown. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { await doWork(message); return message; }, pollingCompleteWaitTimeMs: 10_000, // This will allow up to 10s for the last poll + handler }); const shutdown = (signal) => { console.log(`Received ${signal}, waiting for in-flight work...`); consumer.stop(); consumer.once("waiting_for_polling_to_complete", () => { console.log("Still processing in-flight messages..."); }); consumer.once("waiting_for_polling_to_complete_timeout_exceeded", () => { console.warn("Graceful shutdown timed out, continuing shutdown anyway."); }); consumer.once("stopped", () => { console.log("Consumer stopped cleanly"); process.exit(0); }); }; process.once("SIGINT", shutdown); process.once("SIGTERM", shutdown); consumer.start(); ``` -------------------------------- ### Start SQS Consumer polling Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/consumer.md Call `consumer.start()` to begin long polling for messages. This method is idempotent and creates a new AbortController internally on each call. A warning may be logged for FIFO queues if `suppressFifoWarning` is not enabled. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { // process message return message; }, }); consumer.start(); ``` -------------------------------- ### SQS Consumer with Single Message Handler Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/configuration.md Implement a consumer that processes individual messages. The handler should return the message to acknowledge it or undefined to skip it. This example logs the message body and performs asynchronous work. ```javascript // Single message handler const consumer1 = Consumer.create({ queueUrl: "...", handleMessage: async (message) => { console.log("Processing:", message.Body); await doWork(message.Body); return message; // acknowledge }, }); ``` -------------------------------- ### Process Messages from Multiple SQS Queues Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/advanced-usage.md Configure and manage multiple SQS consumers to process messages from different queues concurrently. This example demonstrates mapping over a list of queue URLs, creating individual consumers, and handling graceful shutdown. ```javascript import { Consumer } from "sqs-consumer"; const queues = [ "https://sqs.eu-west-1.amazonaws.com/account/queue-1", "https://sqs.eu-west-1.amazonaws.com/account/queue-2", "https://sqs.eu-west-1.amazonaws.com/account/queue-3", ]; const consumers = queues.map(queueUrl => Consumer.create({ queueUrl, handleMessage: async (message) => { console.log(`Processing from ${queueUrl}`); await processMessage(message); return message; }, }) ); // Start all consumers consumers.forEach(consumer => { consumer.on("error", console.error); consumer.start(); }); // Graceful shutdown for all async function gracefulShutdown(signal) { console.log(`Received ${signal}`); consumers.forEach(consumer => consumer.stop()); await Promise.all( consumers.map(consumer => new Promise(resolve => { consumer.once("stopped", resolve); }) ) ); console.log("All consumers stopped"); process.exit(0); } process.on("SIGTERM", () => gracefulShutdown("SIGTERM")); process.on("SIGINT", () => gracefulShutdown("SIGINT")); ``` -------------------------------- ### Consumer Lifecycle Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/REFERENCE.md Illustrates the lifecycle of a sqs-consumer instance, from creation to stopping. ```text new Consumer(options) ↓ consumer.start() ↓ (polling begins) receive messages from SQS ↓ call handler function (handleMessage or handleMessageBatch) ↓ (handler succeeds) delete message from queue ↓ emit "message_processed" event ↓ continue polling ↓ consumer.stop() ↓ (polling stops) emit "stopped" event ``` -------------------------------- ### Consumer Class Methods Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/COMPLETION_SUMMARY.md Documentation for the main Consumer class, including static and instance methods for managing the SQS consumer. ```APIDOC ## Consumer Class ### Description Provides methods to create, manage, and interact with an SQS message consumer. ### Methods #### static create(options: ConsumerOptions): Consumer - **Description**: Creates a new instance of the SQS consumer. - **Parameters**: - **options** (ConsumerOptions) - Required - Configuration options for the consumer. - **Returns**: A new Consumer instance. #### start(): Promise - **Description**: Starts the SQS message consumer to begin processing messages. - **Returns**: A Promise that resolves when the consumer has started. #### stop(): Promise - **Description**: Stops the SQS message consumer gracefully. - **Returns**: A Promise that resolves when the consumer has stopped. #### updateOption(optionName: string, value: any): void - **Description**: Updates a specific option of the running consumer dynamically. - **Parameters**: - **optionName** (string) - Required - The name of the option to update. - **value** (any) - Required - The new value for the option. #### status (getter): string - **Description**: Returns the current status of the consumer. - **Returns**: A string representing the consumer's status. ``` -------------------------------- ### Create a Consumer Instance Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/README.md Use `Consumer.create()` to instantiate a consumer. Pass an object with configuration options. ```typescript Consumer.create(options: ConsumerOptions): Consumer ``` -------------------------------- ### consumer.start() Source: https://github.com/bbc/sqs-consumer/blob/main/README.md Initiates the message polling process. Once called, the consumer will begin to fetch and process messages from the configured SQS queue according to the specified options. ```APIDOC ## consumer.start() ### Description Start polling the queue for messages. ### Method `consumer.start()` ### Request Example ```js consumer.start(); ``` ``` -------------------------------- ### Creating Type-Safe Single Message Handler Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/message-types.md Example of a type-safe handler for processing individual SQS messages, including error handling and JSON parsing. ```typescript import type { Message } from "@aws-sdk/client-sqs"; import { Consumer } from "sqs-consumer"; // Type-safe single message handler const handleMessage = async (message: Message): Promise => { if (!message.Body) { console.error("Message missing body"); return message; // acknowledge malformed } try { const data = JSON.parse(message.Body); await processData(data); return message; // acknowledge } catch (err) { console.error("Processing failed:", err); return undefined; // retry } }; const consumer = Consumer.create({ queueUrl: "...", handleMessage }); ``` -------------------------------- ### Chaining Event Listeners Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md Chain multiple event listeners for started, message_received, message_processed, error, and stopped events. This allows for a fluent API usage pattern. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue", handleMessage: async (message) => message, }); consumer .on("started", () => console.log("Started")) .on("message_received", (msg) => console.log("Received:", msg.MessageId)) .on("message_processed", (msg) => console.log("Processed:", msg.MessageId)) .on("error", (err) => console.error("Error:", err.message)) .on("stopped", () => process.exit(0)); consumer.start(); ``` -------------------------------- ### SQS Consumer Project File Structure Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/INDEX.md This snippet displays the directory structure of the SQS Consumer project, highlighting key documentation files and their locations. ```tree output/ ├── INDEX.md ← Navigation guide (this file) ├── README.md ← Start here ├── REFERENCE.md ← Master index ├── COMPLETION_SUMMARY.md ← Generation summary │ ├── getting-started.md ← Quick start guide ├── configuration.md ← Options reference ├── types.md ← Type definitions ├── errors.md ← Error handling ├── advanced-usage.md ← Production patterns │ └── api-reference/ ← Detailed API docs ├── consumer.md ← Consumer class ├── events.md ← Event reference ├── message-types.md ← Message types └── typed-event-emitter.md ← Event system ``` -------------------------------- ### ConsumerOptions Interface Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/types.md Configuration object passed to `Consumer.create()` to configure the consumer instance. ```APIDOC ## ConsumerOptions Configuration object passed to `Consumer.create()` to configure the consumer instance. ### Parameters #### queueUrl - **Type**: `string` - **Required**: ✓ - **Description**: The SQS queue URL (e.g., `https://sqs.region.amazonaws.com/account/queue-name`) #### attributeNames - **Type**: `QueueAttributeName[]` - **Required**: - **Default**: `[]` - **Description**: List of queue attributes to retrieve. See [AWS docs](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-sqs/Variable/QueueAttributeName/) for valid values #### messageAttributeNames - **Type**: `string[]` - **Required**: - **Default**: `[]` - **Description**: List of message attribute names to retrieve (e.g., `['name', 'address']`) #### messageSystemAttributeNames - **Type**: `MessageSystemAttributeName[]` - **Required**: - **Default**: `[]` - **Description**: List of message system attributes to retrieve. See [AWS docs](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-sqs/Variable/MessageSystemAttributeName/) #### batchSize - **Type**: `number` - **Required**: - **Default**: `1` - **Description**: Number of messages to request from SQS per poll. Must be 1-10 #### visibilityTimeout - **Type**: `number` - **Required**: - **Default**: — - **Description**: Duration in seconds that messages are hidden after being retrieved. Passed to SQS ReceiveMessage. Used for heartbeat extension #### waitTimeSeconds - **Type**: `number` - **Required**: - **Default**: `20` - **Description**: Duration in seconds for long polling. Must be 1-20 #### authenticationErrorTimeout - **Type**: `number` - **Required**: - **Default**: `10000` - **Description**: Milliseconds to wait before retrying after authentication errors #### pollingWaitTimeMs - **Type**: `number` - **Required**: - **Default**: `0` - **Description**: Milliseconds to wait between polls (delay between successive SQS requests) #### pollingCompleteWaitTimeMs - **Type**: `number` - **Required**: - **Default**: `0` - **Description**: Milliseconds to wait for final poll and in-flight messages to complete before emitting `stopped`. Set for graceful shutdown #### terminateVisibilityTimeout - **Type**: `boolean | number | ((messages: Message[]) => number)` - **Required**: - **Default**: `false` - **Description**: If `true`, sets message visibility to 0 after processing error. If number, sets to that value. If function, calls function with messages to determine timeout #### heartbeatInterval - **Type**: `number` - **Required**: - **Default**: — - **Description**: Seconds between heartbeat requests to extend message visibility. Must be less than `visibilityTimeout` #### sqs - **Type**: `SQSClient` - **Required**: - **Default**: — - **Description**: Pre-configured SQS Client instance. If not provided, a new client is created #### region - **Type**: `string` - **Required**: - **Default**: `process.env.AWS_REGION` or `'eu-west-1'` - **Description**: AWS region for the SQS client #### useQueueUrlAsEndpoint - **Type**: `boolean` - **Required**: - **Default**: `true` - **Description**: If `false`, uses the SQS client's resolved endpoint instead of `queueUrl` #### handleMessageTimeout - **Type**: `number` - **Required**: - **Default**: — - **Description**: Milliseconds to wait for `handleMessage` to complete before timing out. Emits `timeout_error` on timeout #### shouldDeleteMessages - **Type**: `boolean` - **Required**: - **Default**: `true` - **Description**: If `false`, messages are not deleted from queue after processing (useful for testing) #### alwaysAcknowledge - **Type**: `boolean` - **Required**: - **Default**: `false` - **Description**: If `true`, all messages are acknowledged regardless of handler return value #### handleMessage - **Type**: `(message: Message) => Promise` - **Required**: - **Default**: — - **Description**: Handler function for single message processing. Return the message to acknowledge, `undefined` to skip. Required if `handleMessageBatch` not set #### handleMessageBatch - **Type**: `(messages: Message[]) => Promise` - **Required**: - **Default**: — - **Description**: Handler function for batch message processing. Return array of messages to acknowledge, `undefined` to skip. Overrides `handleMessage` if both set #### preReceiveMessageCallback - **Type**: `() => Promise` - **Required**: - **Default**: — - **Description**: Async hook called before each ReceiveMessage request to SQS #### postReceiveMessageCallback - **Type**: `() => Promise` - **Required**: - **Default**: — - **Description**: Async hook called after each ReceiveMessage request to SQS #### extendedAWSErrors - **Type**: `boolean` - **Required**: - **Default**: `false` - **Description**: If `true`, error events include full AWS error response and metadata #### suppressFifoWarning - **Type**: `boolean` - **Required**: - **Default**: `false` - **Description**: If `true`, suppresses warning when FIFO queue is detected #### strictReturn - **Type**: `boolean` - **Required**: - **Default**: `false` - **Description**: If `true`, handler returning `null` throws error instead of treating as "do not acknowledge". Future default **Used by:** `Consumer.create()`, `Consumer.constructor()` **Remarks:** ``` -------------------------------- ### Handling Message Processing Errors Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/errors.md Example of setting up listeners for 'processing_error' and 'timeout_error' events to manage errors occurring during message handling. This allows for custom retry logic or routing. ```javascript consumer.on("processing_error", (err, message) => { console.error("Message processing failed:", err.message); // Implement custom retry logic, dead-letter routing, etc. }); consumer.on("timeout_error", (err, message) => { console.error("Message handler timed out"); // Could route to DLQ manually or implement exponential backoff }); ``` -------------------------------- ### Create SQS Consumer with Default Credentials and Region Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/configuration.md This snippet shows how to create an SQS consumer using default AWS credentials and region detection (from environment variables or a fallback). ```javascript const consumer1 = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue", // Uses AWS_REGION env var or defaults to eu-west-1 handleMessage: async (message) => message, }); ``` -------------------------------- ### Consumer.create Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/consumer.md A static factory method used to instantiate a new Consumer. It requires a configuration object with details such as the SQS queue URL and a message handler function. ```APIDOC ## Consumer.create ### Description Static factory method to create a new Consumer instance. ### Method `static create(options: ConsumerOptions): Consumer` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **options** (ConsumerOptions) - Required - Configuration object for the consumer ### Request Example ```javascript import { Consumer } from "sqs-consumer"; const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { console.log("Processing message:", message.Body); return message; // acknowledge and delete }, }); ``` ### Response #### Success Response (200) - **Consumer** - A new consumer instance #### Response Example (No specific response example provided, but the return type is `Consumer`) ``` -------------------------------- ### on(event, listener) Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/events.md Register a listener to be called every time the event is emitted. This method allows for continuous monitoring of specific events throughout the consumer's lifecycle. ```APIDOC ## on(event, listener) ### Description Register a listener to be called every time the event is emitted. ### Method `on` ### Parameters #### Path Parameters - **event** (keyof Events) - Required - Event name to listen for - **listener** (function) - Required - Callback function invoked when event is emitted ### Returns `this` (allows chaining) ### Example ```javascript const consumer = Consumer.create({ /* ... */ }); consumer.on("message_received", (message, metadata) => { console.log(`Received from ${metadata.queueUrl}:`, message.MessageId); }); consumer.on("error", (err, messages, metadata) => { console.error("Error:", err.message); }); ``` ``` -------------------------------- ### Create SQS Consumer with Manual Credentials Source: https://github.com/bbc/sqs-consumer/blob/main/README.md Instantiate an SQS consumer with manually provided AWS credentials via an SQSClient instance. This is useful when environment variables are not sufficient. ```javascript import { Consumer } from "sqs-consumer"; import { SQSClient } from "@aws-sdk/client-sqs"; const app = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { // ... }, sqs: new SQSClient({ region: "my-region", credentials: { accessKeyId: "yourAccessKey", secretAccessKey: "yourSecret", }, }), }); app.on("error", (err) => { console.error(err.message); }); app.on("processing_error", (err) => { console.error(err.message); }); app.on("timeout_error", (err) => { console.error(err.message); }); app.start(); ``` -------------------------------- ### Check SQS Consumer Status Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/api-reference/consumer.md Access the `consumer.status` property to determine if the consumer is running and actively polling. `isRunning` indicates if `start()` has been called and `stop()` has not, while `isPolling` reflects active message retrieval. ```javascript const consumer = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { return message; }, }); consumer.start(); console.log(consumer.status); // { isRunning: true, isPolling: true } consumer.stop(); setTimeout(() => { console.log(consumer.status); // { isRunning: false, isPolling: false } }, 100); ``` -------------------------------- ### Configure SQS Consumer with Custom SQS Client Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/advanced-usage.md Use a pre-configured AWS SDK SQS client for advanced scenarios, such as adding custom middleware via customUserAgent. Ensure the custom client is passed to the Consumer.create method. ```javascript import { SQSClient } from "@aws-sdk/client-sqs"; import { Consumer } from "sqs-consumer"; // Custom client with middleware const customClient = new SQSClient({ region: "eu-west-1", credentials: { accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, }, // Add custom middleware customUserAgent: [["my-app", "1.0.0"]], }); const consumer = Consumer.create({ queueUrl: "...", sqs: customClient, // use custom client handleMessage: async (message) => message, }); ``` -------------------------------- ### Graceful Shutdown of SQS Consumer Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/README.md Demonstrates how to gracefully shut down the SQS consumer by stopping it and handling termination signals (SIGTERM, SIGINT). ```javascript consumer.stop(); consumer.once("stopped", () => process.exit(0)); process.on("SIGTERM", () => consumer.stop()); process.on("SIGINT", () => consumer.stop()); ``` -------------------------------- ### Consumer.create(options) Source: https://github.com/bbc/sqs-consumer/blob/main/README.md Creates a new SQS consumer instance. This method initializes the consumer with the provided configuration options, which define aspects like the queue URL, message handling logic, and AWS SQS client configuration. ```APIDOC ## Consumer.create(options) ### Description Creates a new SQS consumer using the defined options. ### Method `Consumer.create(options)` ### Parameters #### Options Object - **queueUrl** (string) - Required - The URL of the SQS queue to consume messages from. - **handleMessage** (function) - Required - An asynchronous function that processes each received message. - **sqs** (SQSClient) - Optional - A pre-configured instance of the AWS SQS Client. If not provided, the SDK will attempt to find credentials automatically. - **pollingWaitTimeMs** (number) - Optional - The time in milliseconds to wait for messages in each polling cycle. - **batchSize** (number) - Optional - The number of messages to retrieve in a single poll. - **maxNumberOfMessages** (number) - Optional - The maximum number of messages to retrieve in a single poll. - **visibilityTimeout** (number) - Optional - The duration in seconds for which a message will be invisible to other consumers after being received. - **authenticationErrorRetries** (number) - Optional - The number of times to retry when an authentication error occurs. - **messageAttributeFilters** (object) - Optional - Filters for message attributes. - **stop Emitter** (EventEmitter) - Optional - An EventEmitter instance to manage stopping the consumer. - **region** (string) - Optional - The AWS region for the SQS client. - **credentials** (object) - Optional - AWS credentials for the SQS client. - **pollingCompleteWaitTimeMs** (number) - Optional - The maximum time in milliseconds to wait for the last poll and message handler to complete during a graceful shutdown. ### Request Example ```js import { Consumer } from "sqs-consumer"; import { SQSClient } from "@aws-sdk/client-sqs"; const app = Consumer.create({ queueUrl: "https://sqs.eu-west-1.amazonaws.com/account-id/queue-name", handleMessage: async (message) => { // Process message console.log(message.Body); }, sqs: new SQSClient({ region: "my-region", credentials: { accessKeyId: "yourAccessKey", secretAccessKey: "yourSecret", }, }), pollingCompleteWaitTimeMs: 10000 }); ``` ``` -------------------------------- ### Integrating SQS Consumer with an Express Server Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/getting-started.md Embed an SQS consumer within an Express.js application, providing a health check endpoint to monitor the consumer's status. Includes setup for graceful shutdown. ```javascript import express from "express"; import { Consumer } from "sqs-consumer"; const app = express(); const consumer = Consumer.create({ queueUrl: "...", handleMessage: async (message) => message, }); // Health check endpoint app.get("/health", (req, res) => { const status = consumer.status; res.json({ healthy: status.isRunning && status.isPolling, isRunning: status.isRunning, isPolling: status.isPolling, }); }); // Start consumer consumer.on("error", console.error); consumer.start(); // Graceful shutdown process.on("SIGTERM", () => { consumer.stop(); consumer.once("stopped", () => { process.exit(0); }); }); app.listen(3000); ``` -------------------------------- ### Configure AWS Credentials via Environment Variables Source: https://github.com/bbc/sqs-consumer/blob/main/_autodocs/getting-started.md Set AWS region and access keys as environment variables for the consumer to authenticate with AWS. This is a common and simple method for local development. ```bash export AWS_REGION=eu-west-1 export AWS_ACCESS_KEY_ID=your-access-key export AWS_SECRET_ACCESS_KEY=your-secret-key ```