### Mix Promises and Functions with `queue.all` Source: https://github.com/henrygd/queue/blob/main/readme.md Demonstrates using `queue.all` with a mix of existing promises and function wrappers. Note that only the wrapper functions are queued, as existing promises start execution immediately. ```typescript const existingPromise = fetch("https://pokeapi.co/api/v2/pokemon/ditto").then( (r) => r.json() ); const results = await queue.all([ existingPromise, () => fetch("https://pokeapi.co/api/v2/pokemon/pidgeot").then((r) => r.json()), ]); ``` -------------------------------- ### Add Task to Queue and Get Result Source: https://context7.com/henrygd/queue/llms.txt The `add` method enqueues an async function or promise-returning function. It returns a promise that resolves with the function's return value or rejects if the task throws an error. Tasks are executed respecting the queue's concurrency. ```typescript import { newQueue } from '@henrygd/queue' const queue = newQueue(2) // Basic usage - add returns a promise with the result const response = await queue.add(() => fetch('https://pokeapi.co/api/v2/pokemon')) console.log(response.ok, response.status) // true 200 // Add multiple tasks and get individual results const fetchPokemon = (name: string) => async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`) return res.json() } const dittoPromise = queue.add(fetchPokemon('ditto')) const pikachuPromise = queue.add(fetchPokemon('pikachu')) const charmanderPromise = queue.add(fetchPokemon('charmander')) // Each promise resolves with the return value const ditto = await dittoPromise console.log(ditto.name, ditto.id) // ditto 132 // Handle errors gracefully try { await queue.add(async () => { throw new Error('Something went wrong') }) } catch (error) { console.error(error.message) // Something went wrong } ``` -------------------------------- ### Create and Use a Basic Queue Source: https://github.com/henrygd/queue/blob/main/readme.md Demonstrates creating a new queue with a specified concurrency limit and adding async tasks to it. Use `queue.done()` to wait for all tasks to complete. ```typescript import { newQueue } from '@henrygd/queue' // create a new queue with a concurrency of 2 const queue = newQueue(2) const pokemon = ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl', 'golem', 'charizard'] for (const name of pokemon) { queue.add(async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`) const json = await res.json() console.log(`${json.name}: ${json.height * 10}cm | ${json.weight / 10}kg`) }) } console.log('running') await queue.done() console.log('done') ``` -------------------------------- ### Performance Benchmarks Source: https://github.com/henrygd/queue/blob/main/readme.md Comparison of @henrygd/queue with other popular queue libraries across different environments. ```APIDOC ## Comparisons and Benchmarks ### Library Comparison Table | Library | Version | Bundle size (B) | Weekly downloads | | :-------------------------------------------------------------- | :------ | :-------------- | :--------------- | | @henrygd/queue | 1.2.0 | 472 | hundreds :) | @henrygd/queue/rl | 1.2.0 | 685 | - | [p-limit](https://github.com/sindresorhus/p-limit) | 5.0.0 | 1,763 | 118,953,973 | [async.queue](https://github.com/caolan/async) | 3.2.5 | 6,873 | 53,645,627 | [fastq](https://github.com/mcollina/fastq) | 1.17.1 | 3,050 | 39,257,355 | [queue](https://github.com/jessetane/queue) | 7.0.0 | 2,840 | 4,259,101 | [promise-queue](https://github.com/promise-queue/promise-queue) | 2.2.5 | 2,200 | 1,092,431 ### Benchmark Notes - All libraries run the same test: resolving 1,000 async functions. - Completion is checked inside the function to avoid penalizing libraries that don't natively support a 'done' promise like `Promise.all`. ### Browser Benchmark - Test environment: Chromium. - Results indicate `@henrygd/queue` performs well, with Firefox and Safari showing slightly slower but comparable results to `promise-queue`. - Image: ![@henrygd/queue - 13,665 Ops/s. fastq - 7,661 Ops/s. promise-queue - 7,650 Ops/s. async.queue - 4,060 Ops/s. p-limit - 1,067 Ops/s. queue - 721 Ops/s](https://henrygd-assets.b-cdn.net/queue/106/browser-benchmark.png) ### Node.js Benchmarks - Note: `p-limit` 6.1.0 performance may vary. - Ryzen 5 4500U | 8GB RAM | Node 22.3.0: ![@henrygd/queue - 1.9x faster than fastq. 2.03x promise-queue. 3.86x async.queue. 20x queue. 86x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/node-4500.png) - Ryzen 7 6800H | 32GB RAM | Node 22.3.0: ![@henrygd/queue - 1.9x faster than fastq. 2.01x promise-queue. 3.98x async.queue. 6.86x queue. 88x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/node-6800h.png) ### Deno Benchmarks - Note: `p-limit` 6.1.0 performance may vary. - Ryzen 5 4500U | 8GB RAM | Deno 1.44.4: ![@henrygd/queue - 1.9x faster than fastq. 2.01x promise-queue. 4.7x async.queue. 7x queue. 28x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/deno-4500.png) - Ryzen 7 6800H | 32GB RAM | Deno 1.44.4: ![@henrygd/queue - 1.82x faster than fastq. 1.91x promise-queue. 3.47x async.queue. 7x queue. 26x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/deno-6800h.png) ### Bun Benchmarks - Ryzen 5 4500U | 8GB RAM | Bun 1.1.17: ![@henrygd/queue - 1.25x faster than promise-queue. 1.66x fastq. 2.73x async.queue. 5.44x p-limit. 12x queue.](https://henrygd-assets.b-cdn.net/queue/106/bun-4500.png) - Ryzen 7 6800H | 32GB RAM | Bun 1.1.17: ![@henrygd/queue - 1.17x faster than promise-queue. 1.51x fastq. 2.53x async.queue. 5.25x p-limit. 5.39x queue.](https://henrygd-assets.b-cdn.net/queue/106/bun-6800h.png) ``` -------------------------------- ### queue.all - Process Multiple Tasks Like Promise.all Source: https://context7.com/henrygd/queue/llms.txt The `all` method provides a way to process multiple tasks concurrently, similar to `Promise.all`. It accepts an array of promise-returning functions or existing promises and returns a single promise that resolves with an array of results, maintaining the original order. ```APIDOC ## queue.all - Process Multiple Tasks Like Promise.all ### Description Works like `Promise.all` but with concurrency control. It accepts an array of promise-returning functions (or existing promises) and returns a promise that resolves with an array of results in the same order as the input. This is useful when you have a batch of tasks to process with controlled concurrency. ### Method `queue.all(tasks: Array<() => Promise | Promise>): Promise>` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```typescript import { newQueue } from '@henrygd/queue' const queue = newQueue(2) // Process array of tasks with concurrency control const tasks = ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl'].map( (name) => async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`) return await res.json() } ) // Results are returned in input order regardless of completion order const results = await queue.all(tasks) console.log(results.map(p => p.name)) // ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl'] // Mix existing promises and wrapper functions const existingPromise = fetch('https://pokeapi.co/api/v2/pokemon/ditto').then(r => r.json()) const mixedResults = await queue.all([ existingPromise, () => fetch('https://pokeapi.co/api/v2/pokemon/pidgeot').then(r => r.json()) ]) console.log(mixedResults.map(p => p.name)) // ['ditto', 'pidgeot'] // Note: existing promises start immediately when created, // only wrapper functions are queued ``` ### Response #### Success Response (200) Resolves with an array of results from the processed tasks, in the same order as the input array. #### Response Example ```json { "example": "[result1, result2, ...]" } ``` ``` -------------------------------- ### Time-Based Rate Limiting Queue Source: https://github.com/henrygd/queue/blob/main/readme.md Illustrates creating a queue with both concurrency and time-based rate limiting using `@henrygd/queue/rl`. This is useful for APIs with strict rate limits. ```typescript import { newQueue } from '@henrygd/queue/rl' // max 10 concurrent requests, but only 3 can start per second const queue = newQueue(10, 3, 1000) const start = Date.now() for (let i = 1; i <= 10; i++) { queue.add(async () => console.log(`Task ${i} started at ${Date.now() - start}ms`)) } await queue.done() ``` -------------------------------- ### newQueue - Create a Queue with Concurrency Control Source: https://context7.com/henrygd/queue/llms.txt The `newQueue` function initializes a new queue instance, allowing you to specify the maximum number of concurrent asynchronous operations that can run at any given time. This is crucial for managing resources and adhering to API rate limits. ```APIDOC ## newQueue - Create a Queue with Concurrency Control ### Description Creates a new queue instance with a specified maximum concurrency level. The concurrency parameter determines how many async operations can run simultaneously. Once created, you can add async functions to the queue and they will be executed respecting the concurrency limit. ### Method `newQueue(concurrency: number)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```typescript import { newQueue } from '@henrygd/queue' // Create a queue that allows 2 concurrent operations const queue = newQueue(2) const pokemon = ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl', 'golem', 'charizard'] for (const name of pokemon) { queue.add(async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`) const json = await res.json() console.log(`${json.name}: ${json.height * 10}cm | ${json.weight / 10}kg`) }) } console.log('running') await queue.done() console.log('done') ``` ### Response #### Success Response (200) N/A (This function creates a queue instance, it does not perform an operation that returns a value directly in this context.) #### Response Example N/A ``` -------------------------------- ### Queue Interface Methods Source: https://github.com/henrygd/queue/blob/main/readme.md This section describes the core methods of the Queue interface for adding, managing, and querying the status of asynchronous tasks. ```APIDOC ## Queue Interface ### Methods - **add(promiseFunction: () => PromiseLike): Promise** - Description: Adds an async function or promise wrapper to the queue. - Returns: A Promise that resolves with the result of the promiseFunction. - **all(promiseFunctions: Array | (() => PromiseLike)>): Promise** - Description: Adds multiple promises (or wrappers) to the queue and resolves like Promise.all. - Returns: A Promise that resolves with an array of results from all promiseFunctions. - **done(): Promise** - Description: Returns a promise that resolves when the queue is empty. - Returns: A Promise that resolves when all pending and active tasks are completed. - **clear(): void** - Description: Empties the queue. Pending promises are rejected, and active promises continue to run. - **active(): number** - Description: Returns the number of promises currently being executed. - Returns: The count of active promises. - **size(): number** - Description: Returns the total number of promises currently in the queue (pending and active). - Returns: The total size of the queue. ``` -------------------------------- ### queue.add - Add a Task to the Queue Source: https://context7.com/henrygd/queue/llms.txt The `add` method allows you to enqueue an asynchronous function or a function that returns a promise. It returns a promise that resolves with the result of the executed task. Tasks are processed respecting the queue's concurrency limit and in the order they were added. ```APIDOC ## queue.add - Add a Task to the Queue ### Description Adds an async function or promise-returning function to the queue and returns a promise that resolves with the function's return value. Tasks are executed in the order they are added, limited by the queue's concurrency setting. If a task throws an error, the returned promise rejects with that error. ### Method `queue.add(task: () => Promise | any): Promise` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```typescript import { newQueue } from '@henrygd/queue' const queue = newQueue(2) // Basic usage - add returns a promise with the result const response = await queue.add(() => fetch('https://pokeapi.co/api/v2/pokemon')) console.log(response.ok, response.status) // true 200 // Add multiple tasks and get individual results const fetchPokemon = (name: string) => async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`) return res.json() } const dittoPromise = queue.add(fetchPokemon('ditto')) const pikachuPromise = queue.add(fetchPokemon('pikachu')) const charmanderPromise = queue.add(fetchPokemon('charmander')) // Each promise resolves with the return value const ditto = await dittoPromise console.log(ditto.name, ditto.id) // ditto 132 // Handle errors gracefully try { await queue.add(async () => { throw new Error('Something went wrong') }) } catch (error) { console.error(error.message) // Something went wrong } ``` ### Response #### Success Response (200) Resolves with the return value of the executed task. #### Response Example ```json { "example": "The return value of the task function" } ``` ``` -------------------------------- ### Create Queue with Concurrency Control Source: https://context7.com/henrygd/queue/llms.txt Use `newQueue` to create a queue instance with a specified maximum concurrency level. Add async functions to the queue; they will execute respecting the concurrency limit. ```typescript import { newQueue } from '@henrygd/queue' // Create a queue that allows 2 concurrent operations const queue = newQueue(2) const pokemon = ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl', 'golem', 'charizard'] for (const name of pokemon) { queue.add(async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`) const json = await res.json() console.log(`${json.name}: ${json.height * 10}cm | ${json.weight / 10}kg`) }) } console.log('running') await queue.done() console.log('done') ``` -------------------------------- ### Queue Interface Methods Source: https://github.com/henrygd/queue/blob/main/readme.md Defines the core methods for interacting with the queue, including adding tasks, managing queue state, and retrieving queue information. Ensure proper async function or promise wrapper is used when adding to the queue. ```typescript /** Add an async function / promise wrapper to the queue */ queue.add(promiseFunction: () => PromiseLike): Promise ``` ```typescript /** Adds promises (or wrappers) to the queue and resolves like Promise.all */ queue.all(promiseFunctions: Array | (() => PromiseLike)>): Promise ``` ```typescript /** Returns a promise that resolves when the queue is empty */ queue.done(): Promise ``` ```typescript /** Empties the queue - pending promises are rejected, active promises continue */ queue.clear(): void ``` ```typescript /** Returns the number of promises currently running */ queue.active(): number ``` ```typescript /** Returns the total number of promises in the queue */ queue.size(): number ``` -------------------------------- ### Monitor Queue Status with queue.active() and queue.size() Source: https://context7.com/henrygd/queue/llms.txt The `active` method returns the number of currently running tasks, while `size` returns the total number of tasks in the queue. Useful for monitoring progress and custom flow control. ```typescript import { newQueue } from '@henrygd/queue' const queue = newQueue(4) console.log(`Initial - Active: ${queue.active()}, Size: ${queue.size()}`) // Initial - Active: 0, Size: 0 // Add 10 tasks that take 10ms each for (let i = 0; i < 10; i++) { queue.add(() => new Promise(r => setTimeout(r, 10))) } console.log(`After adding - Active: ${queue.active()}, Size: ${queue.size()}`) // After adding - Active: 4, Size: 10 // Check status mid-execution await new Promise(r => setTimeout(r, 25)) console.log(`Mid-execution - Active: ${queue.active()}, Size: ${queue.size()}`) // Mid-execution - Active: 2, Size: 2 await queue.done() console.log(`After done - Active: ${queue.active()}, Size: ${queue.size()}`) // After done - Active: 0, Size: 0 ``` -------------------------------- ### Queue Task Return Value Source: https://github.com/henrygd/queue/blob/main/readme.md Shows that the return value of `queue.add` is the same as the return value of the supplied async function. This allows capturing results directly from queued tasks. ```typescript const response = await queue.add(() => fetch("https://pokeapi.co/api/v2/pokemon") ); console.log(response.ok, response.status, response.headers); ``` -------------------------------- ### Process Multiple Tasks with Concurrency Control Source: https://context7.com/henrygd/queue/llms.txt Use `queue.all` to process an array of promise-returning functions or existing promises with controlled concurrency, similar to `Promise.all`. It returns a promise that resolves with an array of results in the input order. ```typescript import { newQueue } from '@henrygd/queue' const queue = newQueue(2) // Process array of tasks with concurrency control const tasks = ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl'].map( (name) => async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`) return await res.json() } ) // Results are returned in input order regardless of completion order const results = await queue.all(tasks) console.log(results.map(p => p.name)) // ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl'] // Mix existing promises and wrapper functions const existingPromise = fetch('https://pokeapi.co/api/v2/pokemon/ditto').then(r => r.json()) const mixedResults = await queue.all([ existingPromise, () => fetch('https://pokeapi.co/api/v2/pokemon/pidgeot').then(r => r.json()) ]) console.log(mixedResults.map(p => p.name)) // ['ditto', 'pidgeot'] // Note: existing promises start immediately when created, // only wrapper functions are queued ``` -------------------------------- ### Wait for Queue to Empty with queue.done() Source: https://context7.com/henrygd/queue/llms.txt The `done` method returns a promise that resolves when all tasks in the queue have completed. It can be called multiple times and reused after the queue empties. ```typescript import { newQueue } from '@henrygd/queue' const queue = newQueue(3) // Add multiple tasks for (let i = 1; i <= 10; i++) { queue.add(async () => { await new Promise(r => setTimeout(r, 100)) console.log(`Task ${i} completed`) }) } console.log(`Queue size: ${queue.size()}`) // Queue size: 10 // Wait for all tasks to complete await queue.done() console.log(`Queue size after done: ${queue.size()}`) // Queue size after done: 0 // done() resolves immediately on empty queue await queue.done() console.log('Resolved immediately') // Resolved immediately // Can be reused after queue empties queue.add(async () => 'new task') await queue.done() console.log('Second batch complete') // Second batch complete ``` -------------------------------- ### Clear Pending Tasks with queue.clear() Source: https://context7.com/henrygd/queue/llms.txt The `clear` method empties the queue by rejecting all pending tasks with an error. Tasks already running will continue. Use this to cancel unneeded work. ```typescript import { newQueue } from '@henrygd/queue' const queue = newQueue(2) const results: number[] = [] // Add 10 tasks that take 50ms each for (let i = 0; i < 10; i++) { queue.add(async () => { await new Promise(r => setTimeout(r, 50)) results.push(i) }).catch(() => { // Handle rejection from cleared tasks console.log(`Task ${i} was cleared`) }) } console.log(`Initial size: ${queue.size()}`) // Initial size: 10 // Clear the queue after 110ms (after ~4 tasks complete) setTimeout(() => { queue.clear() console.log(`Size after clear: ${queue.size()}`) // Size after clear: 2 (active tasks continue) }, 110) await queue.done() console.log(`Completed tasks: ${results.length}`) // Completed tasks: 4 ``` -------------------------------- ### Time-Based Rate Limiting with @henrygd/queue/rl Source: https://context7.com/henrygd/queue/llms.txt The rate-limited queue (`@henrygd/queue/rl`) adds time-based rate limiting on top of concurrency control. Use `newQueue(concurrency, rate, interval)` for APIs with request limits. ```typescript import { newQueue } from '@henrygd/queue/rl' // Max 10 concurrent requests, but only 3 can start per second const queue = newQueue(10, 3, 1000) const start = Date.now() for (let i = 1; i <= 10; i++) { queue.add(async () => { console.log(`Task ${i} started at ${Date.now() - start}ms`) }) } await queue.done() // Output (approximate): // Task 1 started at 0ms // Task 2 started at 1ms // Task 3 started at 2ms // Task 4 started at 1000ms (rate limited) // Task 5 started at 1001ms // Task 6 started at 1002ms // Task 7 started at 2000ms (rate limited) // Task 8 started at 2001ms // Task 9 started at 2002ms // Task 10 started at 3000ms (rate limited) // Without rate and interval, works like the main package const simpleQueue = newQueue(5) // concurrency only, no rate limit ``` -------------------------------- ### Process All Tasks Concurrently with `queue.all` Source: https://github.com/henrygd/queue/blob/main/readme.md Utilizes `queue.all` to process an array of async tasks concurrently, respecting the queue's concurrency limit. It waits for all tasks to complete and returns their results. ```typescript import { newQueue } from "@henrygd/queue"; const queue = newQueue(2); const tasks = ["ditto", "hitmonlee", "pidgeot", "poliwhirl"].map( (name) => async () => { const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`); return await res.json(); } ); // Process all tasks concurrently (limited by queue concurrency) and wait for all to complete const results = await queue.all(tasks); console.log(results); // [{ name: 'ditto', ... }, { name: 'hitmonlee', ... }, ...] ``` -------------------------------- ### Preserve Execution Context with AsyncLocalStorage Source: https://context7.com/henrygd/queue/llms.txt Use this snippet to maintain execution context, such as request IDs, across asynchronous operations. It requires importing `newQueue` from '@henrygd/queue/async-storage' and `AsyncLocalStorage` from 'async_hooks'. The context is preserved even after awaiting operations within the queued task. ```typescript import { newQueue } from '@henrygd/queue/async-storage' import { AsyncLocalStorage } from 'async_hooks' const queue = newQueue(2) const store = new AsyncLocalStorage<{ requestId: string }>() async function processRequest(requestId: string) { // Run within AsyncLocalStorage context return store.run({ requestId }, async () => { // Queue the actual work return queue.add(async () => { // Context is preserved even after awaiting await new Promise(r => setTimeout(r, 10)) // Access the store value const context = store.getStore() console.log(`Processing request: ${context?.requestId}`) return `Completed ${requestId}` }) }) } // Process multiple requests concurrently const results = await Promise.all([ processRequest('req-001'), processRequest('req-002'), processRequest('req-003'), processRequest('req-004'), ]) // Output: // Processing request: req-001 // Processing request: req-002 // Processing request: req-003 // Processing request: req-004 console.log(results) // ['Completed req-001', 'Completed req-002', 'Completed req-003', 'Completed req-004'] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.