### Install p-map Source: https://github.com/sindresorhus/p-map/blob/main/readme.md Install the p-map package using npm. ```sh npm install p-map ``` -------------------------------- ### p-map Usage Example with Options Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Demonstrates how to import and use the p-map function with a custom options object, setting concurrency, stopOnError, and signal. ```typescript import pMap, {type Options} from 'p-map'; const options: Options = { concurrency: 5, stopOnError: false, signal: abortController.signal }; const results = await pMap(items, mapper, options); ``` -------------------------------- ### pMapIterable Combined Configuration Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md An example of `pMapIterable` with combined concurrency and backpressure settings for efficient fetching and processing. ```javascript import {pMapIterable} from 'p-map'; // Fetch URLs concurrently, buffer results, process sequentially const mapper = async url => { const response = await fetch(url); return response.json(); }; for await (const data of pMapIterable(urls, mapper, { concurrency: 20, backpressure: 5 })) { await database.insert(data); } ``` -------------------------------- ### Type-Safe Mapper Usage Example Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Demonstrates how to create and use a type-safe mapper function with p-map. This example specifies the input element type as string and the return type as number. ```typescript import pMap, {type Mapper} from 'p-map'; // Type-safe mapper with specific element and return types const stringLengthMapper: Mapper = (str, index) => { return str.length; }; const results = await pMap(['hello', 'world'], stringLengthMapper); // results type: number[] ``` -------------------------------- ### MaybePromise Usage Example Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Illustrates the usage of the `MaybePromise` type, showing how it can represent both synchronous and asynchronous string values. ```typescript // MaybePromise means it can be either: // - a string // - a Promise type Result = MaybePromise; const sync: Result = 'hello'; const async: Result = Promise.resolve('hello'); ``` -------------------------------- ### pMapIterable Usage Example Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Demonstrates how to use the `pMapIterable` function with custom `IterableOptions` for concurrent asynchronous iteration. ```typescript import {pMapIterable, type IterableOptions} from 'p-map'; const options: IterableOptions = { concurrency: 10, backpressure: 3 }; for await (const result of pMapIterable(items, mapper, options)) { console.log(result); } ``` -------------------------------- ### Mapper Rejection with stopOnError: true Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Demonstrates how p-map rejects immediately with the first error when `stopOnError` is true (default). Already started mappers continue but their results are ignored. ```javascript import pMap from 'p-map'; const items = [1, 2, 3, 4, 5]; const mapper = async item => { if (item === 3) throw new Error(`Failed on item ${item}`); return item * 2; }; // stopOnError: true (default) try { const results = await pMap(items, mapper, {concurrency: 2}); } catch (error) { console.error(error.message); // Failed on item 3 // Thrown immediately, results not collected } ``` -------------------------------- ### Implementing Retry Logic with p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md This example shows how to implement retry logic within the mapper function for p-map. The mapper will attempt to process an item up to 3 times before throwing an error. ```javascript const mapper = async (item, index) => { for (let attempt = 1; attempt <= 3; attempt++) { try { return await process(item); } catch (error) { if (attempt === 3) throw error; } } }; const results = await pMap(items, mapper, {concurrency: 5}); ``` -------------------------------- ### Handling AbortError with AbortSignal Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Example of how p-map throws an AbortError (DOMException) if the provided AbortSignal is aborted during the mapping process. Includes setting up an AbortController and aborting it after a delay. ```javascript import pMap from 'p-map'; const abortController = new AbortController(); setTimeout(() => { abortController.abort(new Error('Timeout after 500ms')); }, 500); const slowMapper = async value => { await new Promise(resolve => setTimeout(resolve, 1000)); return value; }; try { await pMap([1, 2, 3], slowMapper, {signal: abortController.signal}); } catch (error) { console.error(error.name); // AbortError console.error(error.message); // Timeout after 500ms } ``` -------------------------------- ### pMapIterable Type Inference Example Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Shows how pMapIterable infers the AsyncIterable type when mapping an array of numbers. The `concurrency` option is demonstrated. ```typescript const items = [1, 2, 3]; // number[] const mapper = async (item: number, index: number) => item * 2; // returns number const iterable = pMapIterable(items, mapper, {concurrency: 5}); // iterable type is inferred as: AsyncIterable for await (const result of iterable) { // result type is: number } ``` -------------------------------- ### pMap Type Inference Example Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Demonstrates how pMap infers the result type when mapping an array of strings to an array of numbers. Ensure `pMap` is imported. ```typescript const items = ['a', 'b', 'c']; // string[] const mapper = async (item: string, index: number) => item.length; // returns number const results = await pMap(items, mapper); // results type is inferred as: Promise ``` -------------------------------- ### Fix Invalid Backpressure in pMapIterable Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Provides correct examples for the backpressure option in pMapIterable, showing it must be greater than or equal to concurrency, contrasting with invalid values. ```javascript // ✗ Wrong {concurrency: 5, backpressure: 3} {concurrency: 5, backpressure: 4.5} {concurrency: 5, backpressure: 'unlimited'} // ✓ Correct - backpressure must be >= concurrency {concurrency: 5, backpressure: 5} {concurrency: 5, backpressure: 10} {concurrency: 5, backpressure: Infinity} ``` -------------------------------- ### p-map Abort Signal Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Use an AbortSignal to cancel the promise chain. The example demonstrates setting a timeout to abort the operation. ```javascript import pMap from 'p-map'; const abortController = new AbortController(); // Start timeout to abort after 5 seconds setTimeout(() => abortController.abort(), 5000); try { await pMap(items, mapper, {signal: abortController.signal}); } catch (error) { if (error.name === 'AbortError') { console.log('Aborted'); } } ``` -------------------------------- ### Fix Invalid Concurrency in p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Shows correct values for the concurrency option in p-map, including positive integers and Infinity, contrasting them with invalid examples. ```javascript // ✗ Wrong {concurrency: 2.5} {concurrency: 0} {concurrency: -1} {concurrency: '5'} // ✓ Correct {concurrency: 2} {concurrency: 5} {concurrency: Infinity} ``` -------------------------------- ### p-map-iterable Without Backpressure Control Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-iterable.md Demonstrates a scenario where the mapper can get far ahead of the consumer, leading to potential memory issues. Set `backpressure` to match `concurrency`. ```javascript for await (const item of pMapIterable(largeList, fastMapper, { concurrency: 100, backpressure: 100 // matches concurrency })) { await slowDatabaseSave(item); } ``` -------------------------------- ### Minimal pMapIterable Configuration Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Demonstrates the basic usage of `pMapIterable` with default settings. ```javascript import {pMapIterable} from 'p-map'; // Use all defaults for await (const result of pMapIterable(items, mapper)) { console.log(result); } ``` -------------------------------- ### p-map File Structure Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md Illustrates the directory structure of the p-map project, showing the location of key files like the main implementation, type definitions, package metadata, and documentation. ```text p-map/ ├── index.js # Implementation (pMap, pMapIterable, pMapSkip) ├── index.d.ts # TypeScript definitions ├── package.json # Package metadata ├── readme.md # User-facing documentation ├── test.js # Test suite └── test-multiple-pmapskips-performance.js # Performance tests ``` -------------------------------- ### Basic p-map Usage with Concurrency Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap.md Demonstrates how to use p-map to fetch data from multiple URLs concurrently, limiting the number of simultaneous requests. ```javascript import pMap from 'p-map'; const urls = ['https://example.com', 'https://example.org', 'https://example.net']; const fetchSite = async url => { const response = await fetch(url); return response.status; }; const results = await pMap(urls, fetchSite, {concurrency: 2}); console.log(results); // [200, 200, 200] ``` -------------------------------- ### p-map Minimal Configuration Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Use all default options or explicitly set them. ```javascript import pMap from 'p-map'; // Use all defaults await pMap(items, mapper); // Or explicitly set defaults await pMap(items, mapper, {}); ``` -------------------------------- ### pMapIterable Backpressure Control (Infinite) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Demonstrates using `pMapIterable` with infinite concurrency and backpressure, allowing all items to be fetched before consumption. ```javascript import {pMapIterable} from 'p-map'; // High concurrency with matching backpressure // Both allow all items to be fetched before consuming for await (const item of pMapIterable(items, mapper, { concurrency: Infinity, backpressure: Infinity })) { console.log(item); } ``` -------------------------------- ### Basic Streaming with Concurrency Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-iterable.md Demonstrates basic streaming usage of `p-mapIterable` with a specified concurrency limit. This is useful for fetching data from multiple URLs concurrently. ```javascript import {pMapIterable} from 'p-map'; const urls = ['https://example.com', 'https://example.org', 'https://example.net']; const fetchSite = async url => { const response = await fetch(url); return {url, status: response.status}; }; for await (const result of pMapIterable(urls, fetchSite, {concurrency: 2})) { console.log(result); // {url: '...', status: 200} } ``` -------------------------------- ### Import pMapSkip Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-skip.md Import pMapSkip along with pMap or pMapIterable from the 'p-map' package. ```javascript import pMap, {pMapSkip} from 'p-map'; ``` ```javascript import {pMapIterable, pMapSkip} from 'p-map'; ``` -------------------------------- ### pMapIterable Backpressure Control (Low) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Illustrates using `pMapIterable` with high concurrency and low backpressure to manage memory when the consumer is slower than the mapper. ```javascript import {pMapIterable} from 'p-map'; // High concurrency with low backpressure = fast mapper, slow consumer // Mapper fetches 100 items, but only keeps 5 resolved results buffered for await (const item of pMapIterable(items, fastFetch, { concurrency: 100, backpressure: 5 })) { await slowDatabaseSave(item); } ``` -------------------------------- ### pMapIterable Usage Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-iterable.md Demonstrates basic streaming usage with concurrency. It takes an iterable, a mapper function, and an options object with concurrency. It yields results as they become available. ```APIDOC ## pMapIterable ### Description Processes an iterable concurrently and returns an async iterable of results. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **input** (Iterable | AsyncIterable) - The iterable or async iterable to process. - **mapper** (Function) - An async function to call for each item in the input. - **options** (Object) - Optional configuration object. - **concurrency** (number) - The maximum number of promises to run concurrently. Defaults to `Infinity`. - **backpressure** (number) - The maximum number of resolved promises to keep buffered. Defaults to `concurrency`. ### Return Type `AsyncIterable>` - An async iterable that streams each return value from `mapper` in order. Values returned as `pMapSkip` are excluded from the stream. The iterable rejects if any mapper invocation rejects. ### Throws - `TypeError` - If `input` is not an Iterable or AsyncIterable. - `TypeError` - If `mapper` is not a function. - `TypeError` - If `concurrency` is not an integer from 1 and up or `Infinity`. - `TypeError` - If `backpressure` is not an integer from `concurrency` and up or `Infinity`. - Any error from mapper - If mapper rejects during iteration. ### Examples #### Basic streaming usage with concurrency ```javascript import {pMapIterable} from 'p-map'; const urls = ['https://example.com', 'https://example.org', 'https://example.net']; const fetchSite = async url => { const response = await fetch(url); return {url, status: response.status}; }; for await (const result of pMapIterable(urls, fetchSite, {concurrency: 2})) { console.log(result); // {url: '...', status: 200} } ``` #### Using backpressure to limit resolved promises ```javascript import {pMapIterable} from 'p-map'; const postIds = [1, 2, 3, 4, 5, 6, 7, 8]; const getPostMetadata = async id => { // Simulate slow API call const response = await fetch(`/api/posts/${id}`); return response.json(); }; // Fetch 8 posts concurrently, but keep at most 3 resolved promises waiting for await (const post of pMapIterable(postIds, getPostMetadata, { concurrency: 8, backpressure: 3 })) { // Save to database - slow operation await saveToDatabase(post); console.log(`Saved post ${post.id}`); } ``` #### Filtering results with pMapSkip ```javascript import {pMapIterable, pMapSkip} from 'p-map'; const items = ['apple', 'banana', 'cherry', 'date']; const filterEvenLength = async item => { // Skip items with odd length return item.length % 2 === 0 ? item : pMapSkip; }; for await (const item of pMapIterable(items, filterEvenLength, {concurrency: 2})) { console.log(item); // 'date' (length 4) } ``` #### Processing async iterable source ```javascript import {pMapIterable} from 'p-map'; async function* getAsyncItems() { for (let i = 1; i <= 5; i++) { yield i; } } const double = async item => item * 2; for await (const result of pMapIterable(getAsyncItems(), double, {concurrency: 2})) { console.log(result); // 2, 4, 6, 8, 10 (in order) } ``` #### Error handling in async iterable ```javascript import {pMapIterable} from 'p-map'; const items = [1, 2, 3, 4, 5]; const mapper = async item => { if (item === 3) throw new Error('Item 3 failed'); return item * 2; }; try { for await (const result of pMapIterable(items, mapper, {concurrency: 2})) { console.log(result); } } catch (error) { console.error(error.message); // 'Item 3 failed' } ``` #### Streaming with limited backpressure to prevent memory buildup ```javascript import {pMapIterable} from 'p-map'; async function* readLargeDataset() { // Simulate reading from a large data source for (let i = 0; i < 1000; i++) { yield i; } } const processItem = async item => { // Simulate heavy computation return item * item; }; // Process 100 items concurrently, but keep at most 10 results buffered for await (const result of pMapIterable(readLargeDataset(), processItem, { concurrency: 100, backpressure: 10 })) { // Process results one at a time - backpressure prevents memory buildup console.log(result); } ``` ``` -------------------------------- ### Filter Products by Stock and Price with pMapSkip Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-skip.md Select products that are in stock and below a certain price threshold, skipping others. ```javascript import pMap, {pMapSkip} from 'p-map'; const products = [ {id: 1, price: 10, inStock: true}, {id: 2, price: 5, inStock: false}, {id: 3, price: 20, inStock: true}, {id: 4, price: 8, inStock: false} ]; const getAffordableInStockProduct = async product => { // Skip products that are out of stock or too expensive if (!product.inStock || product.price > 15) { return pMapSkip; } const enrichedData = await fetchProductDetails(product.id); return {...product, ...enrichedData}; }; const availableProducts = await pMap(products, getAffordableInStockProduct, {concurrency: 2}); // Only in-stock products under $15 ``` -------------------------------- ### Full Parallelism with p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md To run all tasks concurrently without any limit, set concurrency to Infinity. Use with caution as it can consume significant resources. ```javascript const results = await pMap(items, mapper, {concurrency: Infinity}); ``` -------------------------------- ### pMapIterable Backpressure Control (Balanced) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Shows a balanced configuration for `pMapIterable` where concurrency and backpressure are equal, allowing for efficient processing. ```javascript import {pMapIterable} from 'p-map'; // Balanced concurrency and backpressure // Mapper fetches 10 items, keeps up to 10 results buffered for await (const item of pMapIterable(items, mapper, { concurrency: 10, backpressure: 10 // Defaults to concurrency })) { console.log(item); } ``` -------------------------------- ### BaseOptions Type Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Base configuration object providing concurrency control for p-map. ```APIDOC ## BaseOptions Base configuration object with concurrency control. ```typescript type BaseOptions = { readonly concurrency?: number; } ``` ### Properties | Property | Type | Required | Default | Description | |----------|------|----------|---------|-------------| | concurrency | `number` | No | `Infinity` | Number of concurrently pending promises returned by `mapper`. Must be an integer from 1 and up or `Infinity`. | ``` -------------------------------- ### Basic p-map Usage Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md Use this snippet for basic concurrent mapping of items. Specify the items, an async mapper function, and concurrency options. ```javascript import pMap from 'p-map'; const results = await pMap(items, asyncMapper, {concurrency: 5}); ``` -------------------------------- ### Configure High Backpressure for High Throughput Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md For high throughput scenarios, increase the backpressure value (e.g., 500) with pMapIterable to buffer more results. ```javascript // High throughput: high backpressure for await (const item of pMapIterable(items, mapper, { concurrency: 50, backpressure: 500 // Buffer more results })) { await process(item); } ``` -------------------------------- ### p-map without pMapSkip (two passes) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-skip.md Demonstrates the traditional approach using p-map for mapping followed by a separate, non-concurrent filter operation. This involves two distinct passes over the data. ```javascript const allResults = await pMap(items, mapper, {concurrency: 5}); const filtered = allResults.filter(result => result !== null); ``` -------------------------------- ### p-map Combined Configuration Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Combine multiple configuration options, including concurrency, error handling, and abort signal, in a single p-map call. ```javascript import pMap from 'p-map'; const abortController = new AbortController(); const results = await pMap(items, mapper, { concurrency: 5, stopOnError: false, signal: abortController.signal }); ``` -------------------------------- ### ESM Import Syntax for p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md The library is ESM-only. Use the `import` statement for default and named exports. CommonJS `require` is not supported. ```javascript // ✓ ESM (default export) import pMap from 'p-map'; // ✓ ESM (named export) import {pMapIterable, pMapSkip} from 'p-map'; // ✗ CommonJS (not supported) const pMap = require('p-map'); // Error ``` -------------------------------- ### Check Node.js Version Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Verify that your Node.js version is 18.0.0 or higher, as required by the library. ```bash node --version # v18.0.0 or higher required ``` -------------------------------- ### pMapIterable(input, mapper, options?) Source: https://github.com/sindresorhus/p-map/blob/main/readme.md Creates an async iterable that streams results from applying an asynchronous mapper function to each element of an input iterable. It supports concurrency and backpressure. ```APIDOC ## pMapIterable(input, mapper, options?) ### Description Creates an async iterable that streams results from applying an asynchronous mapper function to each element of an input iterable. It supports concurrency and backpressure. ### Parameters #### input Type: `AsyncIterable | unknown> | Iterable | unknown>` Synchronous or asynchronous iterable that is iterated over concurrently, calling the `mapper` function for each element. Each iterated item is `await`'d before the `mapper` is invoked so the iterable may return a `Promise` that resolves to an item. #### mapper(element, index) Type: `Function` Expected to return a `Promise` or value. #### options Type: `object` ##### concurrency Type: `number` *(Integer)* Default: `Infinity` Minimum: `1` Number of concurrently pending promises returned by `mapper`. ##### backpressure **Only for `pMapIterable`** Type: `number` *(Integer)* Default: `options.concurrency` Minimum: `options.concurrency` Maximum number of promises returned by `mapper` that have resolved but not yet collected by the consumer of the async iterable. Calls to `mapper` will be limited so that there is never too much backpressure. ``` -------------------------------- ### p-map Function Options Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Configuration options for the p-map function, allowing control over concurrency, error handling, and abort signals. ```APIDOC ## Options Configuration object for `p-map` function. ```typescript export type Options = BaseOptions & { readonly stopOnError?: boolean; readonly signal?: AbortSignal | undefined; } ``` ### Properties | Property | Type | Required | Default | Description | |----------|------|----------|---------|-------------| | concurrency | `number` | No | `Infinity` | Number of concurrently pending promises returned by `mapper`. Must be an integer from 1 and up or `Infinity`. Inherited from `BaseOptions`. | | stopOnError | `boolean` | No | `true` | When `true`, the first mapper rejection will be rejected back to the consumer. When `false`, instead of stopping when a promise rejects, it will wait for all the promises to settle and then reject with an `AggregateError` containing all the errors from the rejected promises. | | signal | `AbortSignal | undefined` | No | `undefined` | An AbortSignal from an AbortController to abort the promises. | ### Usage ```typescript import pMap, {type Options} from 'p-map'; const options: Options = { concurrency: 5, stopOnError: false, signal: abortController.signal }; const results = await pMap(items, mapper, options); ``` ``` -------------------------------- ### Configure Low Backpressure for Memory Constraints Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md When memory is constrained, use a low backpressure value (e.g., 5) with pMapIterable to keep memory usage minimal. ```javascript import {pMapIterable} from 'p-map'; // Memory-constrained: low backpressure for await (const item of pMapIterable(items, mapper, { concurrency: 50, backpressure: 5 // Keep memory usage low })) { await save(item); } ``` -------------------------------- ### Unlimited Parallelism with Concurrency Infinity Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Using `concurrency: Infinity` allows all mapper function calls to execute simultaneously. This provides the maximum possible parallelism but can lead to high resource consumption. ```javascript const results = await pMap(items, slowMapper, {concurrency: Infinity}); // Timeline: // [mapper('a')] // [mapper('b')] // [mapper('c')] // [mapper('d')] ← All run in parallel ``` -------------------------------- ### Streaming Usage with p-mapIterable Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md Utilize pMapIterable for streaming results as they become available. This is useful for processing large datasets without loading all results into memory. ```javascript import {pMapIterable} from 'p-map'; for await (const result of pMapIterable(items, asyncMapper, {concurrency: 5})) { console.log(result); } ``` -------------------------------- ### Streaming with Limited Backpressure for Memory Prevention Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-iterable.md Shows how to use `backpressure` with a large dataset to prevent memory buildup. By limiting the number of buffered results, it ensures that memory usage remains stable even when processing a vast amount of data. ```javascript import {pMapIterable} from 'p-map'; async function* readLargeDataset() { // Simulate reading from a large data source for (let i = 0; i < 1000; i++) { yield i; } } const processItem = async item => { // Simulate heavy computation return item * item; }; // Process 100 items concurrently, but keep at most 10 results buffered for await (const result of pMapIterable(readLargeDataset(), processItem, { concurrency: 100, backpressure: 10 })) { // Process results one at a time - backpressure prevents memory buildup console.log(result); } ``` -------------------------------- ### p-map with pMapSkip (single pass) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-skip.md Shows how to use pMapSkip within the mapper function to filter out unwanted results during the concurrent mapping process. This achieves filtering in a single, efficient pass. ```javascript const mapper = async item => { const result = await process(item); return result.isValid ? result : pMapSkip; }; const filtered = await pMap(items, mapper, {concurrency: 5}); ``` -------------------------------- ### Choose Concurrency for I/O Tasks Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md For I/O-bound tasks like network requests, a higher concurrency value (e.g., 20) can improve throughput by overlapping waiting times. ```javascript // I/O tasks (network): use higher concurrency const results = await pMap(urls, fetchUrl, {concurrency: 20}); ``` -------------------------------- ### pMap(input, mapper, options?) Source: https://github.com/sindresorhus/p-map/blob/main/readme.md Processes an iterable of values concurrently, applying an asynchronous mapper function to each. It returns a Promise that resolves to an array of the mapper's results in the original order, or rejects if any mapper promise rejects. ```APIDOC ## pMap(input, mapper, options?) ### Description Processes an iterable of values concurrently, applying an asynchronous mapper function to each. It returns a Promise that resolves to an array of the mapper's results in the original order, or rejects if any mapper promise rejects. ### Parameters #### input Type: `AsyncIterable | unknown> | Iterable | unknown>` Synchronous or asynchronous iterable that is iterated over concurrently, calling the `mapper` function for each element. Each iterated item is `await`'d before the `mapper` is invoked so the iterable may return a `Promise` that resolves to an item. #### mapper(element, index) Type: `Function` Expected to return a `Promise` or value. #### options Type: `object` ##### concurrency Type: `number` *(Integer)* Default: `Infinity` Minimum: `1` Number of concurrently pending promises returned by `mapper`. ##### stopOnError Type: `boolean` Default: `true` When `true`, the first mapper rejection will be rejected back to the consumer. When `false`, it waits for all promises to settle and then rejects with an `AggregateError`. ##### signal Type: `AbortSignal` You can abort the promises using `AbortController`. ``` -------------------------------- ### Error Handling with p-map (stopOnError) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap.md Illustrates how p-map handles errors. By default (`stopOnError: true`), it rejects immediately upon the first error. When `stopOnError` is `false`, it collects all errors into an `AggregateError`. ```javascript import pMap from 'p-map'; const items = [1, 2, 3, 4, 5]; const mapper = async item => { if (item === 3) throw new Error('Item 3 failed'); return item * 2; }; // stopOnError: true (default) - rejects immediately try { await pMap(items, mapper, {concurrency: 2}); } catch (error) { console.error(error.message); // 'Item 3 failed' } // stopOnError: false - collects all errors try { await pMap(items, mapper, {concurrency: 2, stopOnError: false}); } catch (error) { console.error(error); // AggregateError with all errors } ``` -------------------------------- ### Limited Parallelism with p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md This is the typical usage for p-map, allowing a specified number of tasks to run concurrently. Adjust the concurrency value based on your needs. ```javascript const results = await pMap(items, mapper, {concurrency: 5}); ``` -------------------------------- ### Choose Concurrency for Database Queries Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Tune concurrency for database queries based on the capacity of your database connection pool. ```javascript // Database queries: tune based on connection pool const results = await pMap(queries, runQuery, {concurrency: 10}); ``` -------------------------------- ### Streaming Large Datasets Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md Process results progressively from a large dataset without buffering everything in memory. `pMapIterable` supports backpressure to manage memory usage. ```javascript for await (const item of pMapIterable(largeList, process, { concurrency: 50, backpressure: 5 })) { await save(item); } ``` -------------------------------- ### Stop on First Error with p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md This pattern demonstrates the default behavior of p-map, where processing stops immediately upon encountering the first error. It uses a try-catch block to handle the error. ```javascript import pMap from 'p-map'; try { const results = await pMap(items, mapper, {concurrency: 5}); console.log('All items processed:', results); } catch (error) { console.error('Processing stopped at first error:', error); } ``` -------------------------------- ### Rate Limiting with p-throttle Source: https://github.com/sindresorhus/p-map/blob/main/readme.md Compose p-map with p-throttle to limit the execution rate of mapper functions. Ensure p-throttle is imported and configured with desired limits. ```javascript import pThrottle from 'p-throttle'; import pMap from 'p-map'; const throttle = pThrottle({ limit: 1, interval: 1000, strict: true, }); const result = await pMap(input, throttle(mapper), {concurrency: 2}); ``` -------------------------------- ### Handling Mapper Errors with try-catch Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md A basic approach to handle potential errors during p-map execution using a standard try-catch block. ```javascript // Option 1: Use try-catch try { const results = await pMap(items, mapper, {concurrency: 5}); } catch (error) { console.error('Mapping failed:', error); } ``` -------------------------------- ### Catching AbortError in p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Shows the recommended way to handle AbortError by checking the error's name property within a catch block. ```javascript const abortController = new AbortController(); try { const results = await pMap(items, mapper, {signal: abortController.signal}); } catch (error) { if (error.name === 'AbortError') { console.log('Mapping was aborted'); } else { console.error('Mapping failed:', error); } } ``` -------------------------------- ### Serial Processing with p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md Use this pattern when tasks must be executed sequentially. Set concurrency to 1 to ensure only one task runs at a time. ```javascript const results = await pMap(items, mapper, {concurrency: 1}); ``` -------------------------------- ### Cancellation with AbortSignal in p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap.md Shows how to cancel p-map operations using an `AbortSignal`. The `catch` block will capture an `AbortError` if the signal is triggered. ```javascript import pMap from 'p-map'; const abortController = new AbortController(); setTimeout(() => { abortController.abort(); }, 500); const slowTask = async value => { await new Promise(resolve => setTimeout(resolve, 1000)); return value; }; try { await pMap([1, 2, 3], slowTask, {signal: abortController.signal}); } catch (error) { console.error(error.name); // 'AbortError' } ``` -------------------------------- ### Handling Mapper Errors within the Mapper Function Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Shows how to handle errors directly within the mapper function using try-catch and returning `pMapSkip` to ignore the failed item. ```javascript // Option 2: Handle errors in mapper const safeMapper = async item => { try { return await riskyOperation(item); } catch (error) { console.error(`Failed on item ${item}:`, error); return pMapSkip; // Skip this item instead of rejecting } }; const results = await pMap(items, safeMapper); ``` -------------------------------- ### Fix Invalid Input Error in p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Illustrates the correct way to provide input to p-map, emphasizing the use of arrays, Sets, Maps, or async iterables instead of invalid types like strings. ```javascript // ✗ Wrong await pMap('string', item => item); // ✓ Correct - use array, Set, Map, or async iterable await pMap(['a', 'b', 'c'], item => item); ``` -------------------------------- ### Throughput Optimized Backpressure with pMapIterable Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Set `backpressure` to a higher value than `concurrency` to optimize throughput. This allows more results to be buffered, potentially increasing processing speed at the cost of higher memory usage. ```javascript // Mapper fetches 10 URLs in parallel // Keeps up to 100 results buffered before pausing // Better throughput, higher memory usage for await (const result of pMapIterable(urls, fetch, { concurrency: 10, backpressure: 100 // Much higher })) { await quickProcess(result); } // Memory usage could be high: up to 100 results ``` -------------------------------- ### BaseOptions Type Definition Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/types.md Defines the base configuration object for p-map, primarily specifying the concurrency setting. ```typescript type BaseOptions = { readonly concurrency?: number; } ``` -------------------------------- ### Abort pMap Operation with AbortSignal Source: https://github.com/sindresorhus/p-map/blob/main/readme.md Demonstrates how to abort a pMap operation using an AbortController and AbortSignal. The operation will throw an AbortError if the signal is aborted before all promises settle. ```javascript import pMap from 'p-map'; import delay from 'delay'; const abortController = new AbortController(); setTimeout(() => { abortController.abort(); }, 500); const mapper = async value => value; await pMap([delay(1000), delay(1000)], mapper, {signal: abortController.signal}); // Throws AbortError (DOMException) after 500 ms. ``` -------------------------------- ### Filtering Results with pMapSkip Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap.md Demonstrates using `pMapSkip` to exclude specific results from the final output array. This is useful for filtering out failed or irrelevant operations. ```javascript import pMap, {pMapSkip} from 'p-map'; const urls = ['https://example.com', 'https://invalid', 'https://example.org']; const fetchWithFallback = async url => { try { const response = await fetch(url); if (response.ok) return response.url; return pMapSkip; } catch { return pMapSkip; } }; const results = await pMap(urls, fetchWithFallback, {concurrency: 2}); // Results will only contain successfully fetched URLs ``` -------------------------------- ### Processing Async Iterable Source with p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap.md Shows how p-map can directly process an async iterable as its input source, mapping over its yielded values. ```javascript import pMap from 'p-map'; async function* getAsyncItems() { yield 1; yield 2; yield 3; } const double = async item => item * 2; const results = await pMap(getAsyncItems(), double, {concurrency: 2}); console.log(results); // [2, 4, 6] ``` -------------------------------- ### Fix Invalid Mapper Error in p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Demonstrates the correct way to provide a mapper function to p-map, contrasting incorrect non-function values with a valid function. ```javascript // ✗ Wrong await pMap([1, 2, 3], 42); // ✓ Correct - pass a function await pMap([1, 2, 3], item => item * 2); ``` -------------------------------- ### Default Backpressure with pMapIterable Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md The default `backpressure` behavior for `pMapIterable` matches the `concurrency` setting, buffering up to that many results. This is suitable when the consumer can keep up with the mapper's output rate. ```javascript import {pMapIterable} from 'p-map'; // Mapper fetches 20 URLs in parallel // Consumer processes results one at a time // Up to 20 results can be buffered waiting for consumer for await (const result of pMapIterable(urls, fetch, { concurrency: 20, backpressure: 20 // Default })) { await slowProcess(result); } ``` -------------------------------- ### AggregateError Details with stopOnError: false Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Demonstrates the structure of an AggregateError when multiple mappers reject and `stopOnError` is false. It shows how to access the array of individual errors. ```javascript import pMap from 'p-map'; const items = [1, 2, 3, 4, 5]; const mapper = async item => { if (item === 2 || item === 4) { throw new Error(`Item ${item} failed`); } return item * 2; }; try { const results = await pMap(items, mapper, { concurrency: 2, stopOnError: false }); } catch (error) { console.error(error instanceof AggregateError); // true console.error(error.errors.length); // 2 console.error(error.errors[0].message); // Item 2 failed console.error(error.errors[1].message); // Item 4 failed } ``` -------------------------------- ### Error Handling in Async Iterable Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-iterable.md Illustrates how errors thrown by the mapper function are propagated and can be caught using a try-catch block around the `for await...of` loop. This ensures robust handling of potential failures during processing. ```javascript import {pMapIterable} from 'p-map'; const items = [1, 2, 3, 4, 5]; const mapper = async item => { if (item === 3) throw new Error('Item 3 failed'); return item * 2; }; try { for await (const result of pMapIterable(items, mapper, {concurrency: 2})) { console.log(result); } } catch (error) { console.error(error.message); // 'Item 3 failed' } ``` -------------------------------- ### Implementing Timeout with p-map Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md This snippet demonstrates how to add a timeout to p-map operations using AbortController. Ensure the timeout is cleared in a finally block to prevent memory leaks. ```javascript const controller = new AbortController(); const timeout = setTimeout(() => controller.abort(), 30000); try { return await pMap(items, mapper, {signal: controller.signal}); } finally { clearTimeout(timeout); } ``` -------------------------------- ### Filter Users Based on Status with pMapSkip Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-skip.md Conditionally return a user object or pMapSkip based on an asynchronous status check. ```javascript import pMap, {pMapSkip} from 'p-map'; const users = [ {id: 1, name: 'Alice'}, {id: 2, name: 'Bob'}, {id: 3, name: 'Charlie'} ]; const getActiveUser = async user => { const isActive = await checkUserStatus(user.id); return isActive ? user : pMapSkip; }; const activeUsers = await pMap(users, getActiveUser, {concurrency: 2}); // Only active users are in the result ``` -------------------------------- ### pMapIterable Concurrency Control (Low) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Sets a low concurrency limit for `pMapIterable`, suitable for operations that are I/O bound or have limited resources. ```javascript import {pMapIterable} from 'p-map'; // Fetch 5 items concurrently for await (const result of pMapIterable(items, fetch, {concurrency: 5})) { console.log(result); } ``` -------------------------------- ### Mapper Rejection with stopOnError: false Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/errors.md Illustrates how p-map collects all rejections into an AggregateError when `stopOnError` is false. All mapper calls complete, and the AggregateError is thrown afterward. ```javascript import pMap from 'p-map'; const items = [1, 2, 3, 4, 5]; const mapper = async item => { if (item === 3) throw new Error(`Failed on item ${item}`); return item * 2; }; // stopOnError: false try { const results = await pMap(items, mapper, {concurrency: 2, stopOnError: false}); } catch (error) { console.error(error); // AggregateError console.error(error.errors); // Array of all errors from mapper // [Error: Failed on item 3] } ``` -------------------------------- ### Optimize Error Handling with stopOnError: true Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Setting `stopOnError: true` offers a slight performance advantage when errors are frequent, as it halts processing immediately. ```javascript // stopOnError: true is slightly faster when errors are common // because it stops immediately (less cleanup) await pMap(items, mapper, {stopOnError: true}); ``` -------------------------------- ### Error Handling: Fail Fast (Default) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md This snippet demonstrates the default error handling behavior where p-map stops processing on the first encountered error. ```javascript // Fail fast on first error (default) try { await pMap(items, mapper, {stopOnError: true}); } catch (error) { // Single error } ``` -------------------------------- ### Skip Results with pMapSkip Source: https://github.com/sindresorhus/p-map/blob/main/readme.md Illustrates how to use pMapSkip within a mapper function to exclude specific results from the final output array. This is useful for filtering out errors or unwanted data during concurrent processing. ```javascript import pMap, {pMapSkip} from 'p-map'; import got from 'got'; const sites = [ getWebsiteFromUsername('sindresorhus'), //=> Promise 'https://avajs.dev', 'https://example.invalid', 'https://github.com' ]; const mapper = async site => { try { const {requestUrl} = await got.head(site); return requestUrl; } catch { return pMapSkip; } }; const result = await pMap(sites, mapper, {concurrency: 2}); console.log(result); //=> ['https://sindresorhus.com/', 'https://avajs.dev/', 'https://github.com/'] ``` -------------------------------- ### pMapIterable Concurrency Control (High) Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/configuration.md Sets a high concurrency limit for `pMapIterable`, useful for very fast operations where overhead is minimal. ```javascript import {pMapIterable} from 'p-map'; // Fetch 100 items concurrently (for fast operations) for await (const result of pMapIterable(items, fetch, {concurrency: 100})) { console.log(result); } ``` -------------------------------- ### pMapIterable Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/api-reference/pmap-iterable.md Maps over an iterable of promises or values concurrently, returning an async iterable that streams results in order. It supports configuration for concurrency and backpressure. ```APIDOC ## pMapIterable ### Description Maps over an iterable of promises or values concurrently, returning an async iterable that streams results in order. It supports configuration for concurrency and backpressure. ### Signature ```typescript export function pMapIterable(input: AsyncIterable> | Iterable>, mapper: Mapper, options?: IterableOptions): AsyncIterable> ``` ### Parameters #### input - **Type**: `AsyncIterable> | Iterable>` - **Required**: Yes - **Description**: Synchronous or asynchronous iterable that is iterated over concurrently. Each iterated item is awaited before the mapper is invoked. The iterable may return a Promise that resolves to an item. Asynchronous iterables can be used when the next item may not be ready without waiting for an asynchronous process to complete (e.g., reading from a remote queue, reading lines from a stream). #### mapper - **Type**: `Mapper` - **Required**: Yes - **Description**: Function called for every item in `input`. Expected to return a Promise or value. The function receives the element and its index as parameters. Return `pMapSkip` to skip including the value in the result stream. #### options - **Type**: `IterableOptions` - **Required**: No - **Default**: `{}` - **Description**: Configuration object for concurrency and backpressure control. ### Options #### concurrency - **Type**: `number` - **Required**: No - **Default**: `Infinity` - **Description**: Number of concurrently pending promises returned by `mapper`. Must be an integer from 1 and up or `Infinity`. #### backpressure - **Type**: `number` - **Required**: No - **Default**: `options.concurrency` - **Description**: Maximum number of promises returned by `mapper` that have resolved but not yet collected by the consumer of the async iterable. Must be an integer from `concurrency` and up or `Infinity`. Calls to `mapper` will be limited so that there is never too much backpressure. Useful when consuming the iterable slower than what the mapper function can produce concurrently. ``` -------------------------------- ### Error Handling: Collect All Errors Source: https://github.com/sindresorhus/p-map/blob/main/_autodocs/OVERVIEW.md Configure p-map to continue processing even after errors occur and collect all errors using the AggregateError. ```javascript // Collect all errors try { await pMap(items, mapper, {stopOnError: false}); } catch (error) { // error instanceof AggregateError with error.errors array } ```