### Install Better Queue via npm Source: https://github.com/diamondio/better-queue/blob/master/README.md Instructions to install the Better Queue library using npm, saving it as a dependency in your project. This command adds 'better-queue' to your project's dependencies. ```bash npm install --save better-queue ``` -------------------------------- ### Install better-queue TypeScript Type Definitions Source: https://github.com/diamondio/better-queue/blob/master/README.md Instructions for installing the TypeScript type definitions for 'better-queue'. These types are available from the Definitely Typed repository and enable type-safe development when using the library in TypeScript projects. ```bash npm install --save @types/better-queue ``` -------------------------------- ### Initialize Better Queue and Push Tasks Source: https://github.com/diamondio/better-queue/blob/master/README.md Demonstrates the basic setup of a Better Queue instance. It initializes a new queue with a processing function that takes an input and a callback, then shows how to push various types of data (numbers, objects) into the queue for processing. ```javascript var Queue = require('better-queue'); var q = new Queue(function (input, cb) { // Some processing here ... cb(null, result); }) q.push(1) q.push({ x: 1 }) ``` -------------------------------- ### Configure better-queue with SQLite Store Source: https://github.com/diamondio/better-queue/blob/master/README.md Shows how to configure 'better-queue' to use an SQLite database for persistent storage. This requires installing `better-queue-sql` or `better-queue-sqlite` and specifying the store type, dialect, and file path in the queue options. ```javascript var q = new Queue(fn, { store: { type: 'sql', dialect: 'sqlite', path: '/path/to/sqlite/file' } }); ``` -------------------------------- ### Import better-queue in TypeScript Project Source: https://github.com/diamondio/better-queue/blob/master/README.md Shows the syntax for importing and instantiating 'better-queue' within a TypeScript project. After installing the type definitions, the library can be imported using a `require` style import, allowing for type inference and checking. ```typescript import Queue = require('better-queue') const q: Queue = new Queue(() => {}); ``` -------------------------------- ### Use better-queue with Webpack and MemoryStore Source: https://github.com/diamondio/better-queue/blob/master/README.md Provides an example of how to bundle and use 'better-queue' in a browser environment with Webpack. It demonstrates explicitly importing `Queue` and `MemoryStore` and passing an instance of `MemoryStore` to the queue constructor, as the default in-memory store is not automatically available in browser builds. ```javascript import Queue = require('better-queue') import MemoryStore = require('better-queue-memory') var q = new Queue(function (input, cb) { // Some processing here ... cb(null, result); }, { store: new MemoryStore(), } ) ``` -------------------------------- ### Ticket Instance Events Source: https://github.com/diamondio/better-queue/blob/master/README.md Describes the events emitted by a Ticket object, which is returned when a task is pushed to the queue. These events provide granular updates on the individual task's lifecycle, including acceptance, queuing, start, progress, completion, and failure. ```APIDOC Ticket Events: accepted: Emitted when the corresponding task is accepted (has passed filter). queued: Emitted when the corresponding task is queued (and saved into the store). started: Emitted when the corresponding task is started. progress: Emitted when the corresponding task progress changes. finish: Emitted when the corresponding task completes. failed: Emitted when the corresponding task fails. ``` -------------------------------- ### Configure better-queue with PostgreSQL Store Source: https://github.com/diamondio/better-queue/blob/master/README.md Demonstrates how to set up 'better-queue' with a PostgreSQL database for task persistence. This configuration requires the `pg` npm package and involves providing detailed connection parameters such as host, port, credentials, database name, and table name. ```javascript var q = new Queue(fn, { store: { type: 'sql', dialect: 'postgres', host: 'localhost', port: 5432, username: 'username', password: 'password', dbname: 'template1', tableName: 'tasks' } }); ``` -------------------------------- ### Use a Custom Store with better-queue Source: https://github.com/diamondio/better-queue/blob/master/README.md Illustrates two methods for integrating a custom storage solution with 'better-queue'. A custom store can be passed directly during queue instantiation or dynamically applied using the `q.use()` method after the queue has been created. ```javascript var q = new Queue(fn, { store: myStore }); ``` ```javascript q.use(myStore); ``` -------------------------------- ### Queue Constructor: new Queue(process, options) Source: https://github.com/diamondio/better-queue/blob/master/README.md Documents the constructor for creating a new Queue instance. It requires a 'process' function to handle tasks and accepts an 'options' object for extensive configuration, including task filtering, merging, priority, concurrency, batching, retries, and store integration. ```APIDOC Queue: __init__(process: function | object, options: object) process: function description: Function to process tasks. Will be run with either one single task (if `batchSize` is 1) or as an array of at most `batchSize` items. The second argument will be a callback `cb(error, result)` that must be called regardless of success or failure. options: object (optional) filter: function description: Function to filter input. Will be run with `input` whatever was passed to `q.push()`. If you define this function, then you will be expected to call the callback `cb(error, task)`. If an error is sent in the callback then the input is rejected. merge: function description: Function to merge tasks with the same task ID. Will be run with `oldTask`, `newTask` and a callback `cb(error, mergedTask)`. If you define this function then the callback is expected to be called. priority: function description: Function to determine the priority of a task. Takes in a task and returns callback `cb(error, priority)`. precondition: function description: Function that runs a check before processing to ensure it can process the next batch. Takes a callback `cb(error, passOrFail)`. id: string | function description: The property to use as the task ID. This can be a string or a function (for more complicated IDs). The function `(task, cb)` and must call the callback with `cb(error, taskId)`. cancelIfRunning: boolean description: If true, when a task with the same ID is running, its worker will be cancelled. default: false autoResume: boolean description: If true, tasks in the store will automatically start processing once it connects to the store. default: true failTaskOnProcessException: boolean description: If true, when the process function throws an error the batch fails. default: true filo: boolean description: If true, tasks will be completed in a first in, last out order. default: false batchSize: number description: The number of tasks (at most) that can be processed at once. default: 1 batchDelay: number description: Number of milliseconds to delay before starting to popping items off the queue. default: 0 batchDelayTimeout: number description: Number of milliseconds to wait for a new task to arrive before firing off the batch. default: Infinity concurrent: number description: Number of workers that can be running at any given time. default: 1 maxTimeout: number description: Number of milliseconds before a task is considered timed out. default: Infinity afterProcessDelay: number description: Number of milliseconds to delay before processing the next batch of items. default: 1 maxRetries: number description: Maximum number of attempts to retry on a failed task. default: 0 retryDelay: number description: Number of milliseconds before retrying. default: 0 storeMaxRetries: number description: Maximum number of attempts before giving up on the store. default: Infinity storeRetryTimeout: number description: Number of milliseconds to delay before trying to connect to the store again. default: 1000 preconditionRetryTimeout: number description: Number of milliseconds to delay before checking the precondition function again. default: 1000 store: object | instance description: Represents the options for the initial store. Can be an object containing `{ type: storeType, ... options ... }`, or the store instance itself. ``` -------------------------------- ### Queue Instance Methods Source: https://github.com/diamondio/better-queue/blob/master/README.md Details the methods available on a Queue instance for interacting with the queue's lifecycle and tasks, including pushing new tasks, controlling processing flow, managing the underlying store, and retrieving performance statistics. ```APIDOC Queue Methods: push(task: any, cb: function = null): Ticket description: Push a task onto the queue, with an optional callback when it completes. parameters: - task: any - cb: function (optional callback when task completes) returns: Ticket object pause(): void description: Pauses the queue: tries to pause running tasks and prevents tasks from getting processed until resumed. resume(): void description: Resumes the queue and its running tasks. destroy(cb: function = null): void description: Destroys the queue: closes the store, tries to clean up. parameters: - cb: function (optional callback) use(store: object): void description: Sets the queue to read from and write to the given store. parameters: - store: object (store instance) getStats(): object description: Gets the aggregate stats for the queue. returns: object properties: - successRate: number (success rate on tasks) - peak: number (peak number of items queued) - total: number (total number of items processed) - average: number (average processing time in milliseconds) resetStats(): void description: Resets all of the aggregate stats. ``` -------------------------------- ### Subscribe to Queue Events Using Task ID Source: https://github.com/diamondio/better-queue/blob/master/README.md Demonstrates how to subscribe to queue events, such as `task_finish`, where the task ID is provided as an argument. This allows for specific handling or logging based on the completed task's identifier. ```js var counter = new Queue(fn) counter.on('task_finish', function (taskId, result) { // taskId will be 'jim' or 'bob' }) counter.push({ id: 'jim', count: 2 }); counter.push({ id: 'bob', count: 1 }); ``` -------------------------------- ### Define Custom better-queue Store Interface Source: https://github.com/diamondio/better-queue/blob/master/README.md Outlines the essential methods that a custom 'better-queue' store implementation must provide. These methods define the contract for how the queue interacts with the underlying storage mechanism to manage tasks, including connection, retrieval, saving, and removal operations. ```APIDOC q.use({ connect: function (cb) { // Connect to your db }, getRunningTasks: function (cb) { // Returns a map of running tasks (lockId => taskIds) }, getTask: function (taskId, cb) { // Retrieves a task }, putTask: function (taskId, task, priority, cb) { // Save task with given priority }, takeFirstN: function (n, cb) { // Removes the first N items (sorted by priority and age) }, takeLastN: function (n, cb) { // Removes the last N items (sorted by priority and recency) } }) ``` -------------------------------- ### Handle Task-Specific Events on Push Source: https://github.com/diamondio/better-queue/blob/master/README.md Illustrates how to chain event listeners directly onto the result of a `q.push()` call. This allows for handling success (`on('finish')`) or failure (`on('failed')`) events for individual tasks, providing fine-grained control over task outcomes. ```javascript var q = new Queue(fn); q.push(1) .on('finish', function (result) { // Task succeeded with {result}! }) .on('failed', function (err) { // Task failed! }) ``` -------------------------------- ### Configure FILO (Stack) Behavior Source: https://github.com/diamondio/better-queue/blob/master/README.md Explains how to change the queue's behavior from FIFO (First-In, First-Out) to FILO (First-In, Last-Out). By setting the `filo` option to `true` during initialization, the queue will process the most recently pushed items first, effectively turning it into a stack. ```javascript var q = new Queue(fn, { filo: true }) ``` -------------------------------- ### Queue Instance Events Source: https://github.com/diamondio/better-queue/blob/master/README.md Lists the events emitted by a Queue instance, providing notifications for various stages of task processing, from queuing and acceptance to completion, failure, and batch-level progress. ```APIDOC Queue Events: task_queued: Emitted when a task is queued. task_accepted: Emitted when a task is accepted. task_started: Emitted when a task begins processing. task_finish: Emitted when a task is completed. task_failed: Emitted when a task fails. task_progress: Emitted when a task progress changes. batch_finish: Emitted when a batch of tasks (or worker) completes. batch_failed: Emitted when a batch of tasks (or worker) fails. batch_progress: Emitted when a batch of tasks (or worker) updates its progress. ``` -------------------------------- ### Subscribe to Queue-Level Events Source: https://github.com/diamondio/better-queue/blob/master/README.md Demonstrates how to subscribe to various events emitted by the queue itself. These events include `task_finish` (when any task completes), `task_failed` (when any task fails), `empty` (when all tasks have been pulled from the queue), and `drain` (when no tasks are left and none are running). ```javascript var q = new Queue(fn); q.on('task_finish', function (taskId, result, stats) { // taskId = 1, result: 3, stats = { elapsed: