### Quick Start: Benchmark JavaScript Functions with Overtake Source: https://github.com/3axap4ehko/overtake/blob/master/README.md A 5-second quick start guide demonstrating how to benchmark two different sum implementations (for loop vs. reduce) on an array of 1 million numbers using Overtake. It shows the basic setup of a benchmark suite, targets, and measures, along with how to run the benchmarks via the CLI. ```typescript // benchmark.ts const suite = benchmark('1M numbers', () => Array.from({ length: 1e6 }, (_, i) => i)); suite.target('for loop').measure('sum', (_, arr) => { let sum = 0; for (let i = 0; i < arr.length; i++) sum += arr[i]; }); suite.target('reduce').measure('sum', (_, arr) => { arr.reduce((a, b) => a + b); }); ``` -------------------------------- ### Benchmark Target Setup with Async Operations (TypeScript) Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Illustrates how to define a setup function for a benchmark target that performs asynchronous operations, such as importing modules. The return value of the setup function is passed as the `ctx` argument to the measure function, allowing access to setup resources like crypto hashes and caches. ```typescript const suite = benchmark('data', () => Buffer.from('test data')); suite .target('with setup', async () => { // Setup runs once before measurements const { createHash } = await import('node:crypto'); const cache = new Map(); return { createHash, cache }; // Available as ctx in measure }) .measure('hash', ({ createHash, cache }, input) => { // ctx contains setup return value const hash = createHash('sha256').update(input).digest(); cache.set(input, hash); }); ``` -------------------------------- ### Example Benchmark Command with Multiple Metrics Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Shows how to specify multiple metrics to report for a benchmark, including operations per second, mean duration, standard deviation, relative margin of error, and specific percentiles. ```bash npx overtake bench.ts -r ops mean sd rme p50 p95 p99 ``` -------------------------------- ### Install Overtake JavaScript Benchmarking Library Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Installs the Overtake library as a development dependency using npm, pnpm, or yarn. This is the first step to using Overtake for JavaScript benchmarking. ```bash npm install -D overtake pnpm add -D overtake yarn add -D overtake ``` -------------------------------- ### Example Histogram Output for Benchmark Comparison Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Illustrates the ASCII histogram output format, visually comparing the performance of different benchmarks. This provides a quick, graphical representation of speed differences. ```text for loop - sum 1M numbers | ████████████████████████████████████████ 2,189 ops/s reduce - sum 1M numbers | ████ 233 ops/s ``` -------------------------------- ### Define Benchmark Targets with Async Setup and GC Prevention - Overtake Source: https://context7.com/3axap4ehko/overtake/llms.txt Illustrates defining benchmark targets using `target()`, including support for asynchronous setup functions that provide context to measurements and teardown functions for cleanup. It also shows a pattern for preventing garbage collection interference during measurements. ```typescript import 'overtake'; const suite = benchmark('test data', () => Buffer.from('hello world')); // Target with async setup - imports available in worker context suite .target('crypto hashing', async () => { const { createHash } = await import('node:crypto'); const cache = new Map(); return { createHash, cache }; }) .teardown(async (ctx) => { ctx.cache.clear(); // Cleanup after all measurements }) .measure('sha256', ({ createHash, cache }, buffer) => { const hash = createHash('sha256').update(buffer).digest('hex'); cache.set(buffer.toString(), hash); }); // Target with GC prevention pattern suite .target('memory safe', () => { const gcBlock = new Set(); // Keeps references alive return { gcBlock }; }) .measure('buffer concat', ({ gcBlock }, buffer) => { const result = Buffer.concat([buffer, buffer]); gcBlock.add(result); // Prevent GC during measurement return result.length; }); // Run: npx overtake benchmark.ts -f table ``` -------------------------------- ### Example Markdown Output for Benchmark Results Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Demonstrates the Markdown output format for a benchmark, showing operations per second with confidence intervals. This format is suitable for documentation. ```markdown ## for loop - sum | Feed | ops | | ---------- | --------------------- | | 1M numbers | 2,189 ops/s +/- 0.17% | ``` -------------------------------- ### Overtake CLI Usage and Options Source: https://context7.com/3axap4ehko/overtake/llms.txt Provides examples of how to use the Overtake command-line interface (CLI) for running benchmarks. It covers basic usage, different output formats (table, JSON, markdown, histogram), enabling progress bars, customizing worker counts and cycles, setting convergence thresholds, and disabling the GC observer. ```bash # Basic usage - run all benchmarks matching pattern npx overtake "**/*.bench.ts" # Table output format npx overtake benchmark.ts -f table # Detailed statistics npx overtake bench.ts -r ops mean sd rme p50 p95 p99 # JSON output for CI pipelines npx overtake bench.ts -f json > results.json # Markdown output for documentation/PRs npx overtake bench.ts -f markdown # ASCII histogram visualization npx overtake bench.ts -f histogram # Progress bar for long benchmarks npx overtake bench.ts --progress # Custom worker count and cycles npx overtake bench.ts -w 8 --min-cycles 100 --max-cycles 500 # Convergence thresholds npx overtake bench.ts --abs-threshold 500 --rel-threshold 0.01 # Disable GC detection npx overtake bench.ts --no-gc-observer # Output examples: # simple format (default): # for loop sum # 1M numbers: 1,607 ops/s +/- 0.15% (heap: 1024KB) # # histogram format: # for loop - sum # 1M numbers | ████████████████████████████████████████ 1,607 ops/s # reduce - sum # 1M numbers | ██████ 238 ops/s ``` -------------------------------- ### Example Benchmark Output with Memory Tracking Source: https://github.com/3axap4ehko/overtake/blob/master/README.md An example of benchmark output that includes heap memory allocation information. This helps in identifying potential memory leaks or excessive memory usage during benchmark runs. ```text 1M numbers ops: 233 ops/s +/- 0.13% (heap: 1794KB) ``` -------------------------------- ### Compare Algorithm Performance (TypeScript) Source: https://github.com/3axap4ehko/overtake/blob/master/README.md A minimal example demonstrating how to benchmark two different implementations of a summing function. It sets up a benchmark for an array of 1 million numbers and measures the performance of a traditional for loop against the `reduce` method. ```typescript // examples/quick-start.ts const sumBenchmark = benchmark('1M numbers', () => Array.from({ length: 1_000_000 }, (_, i) => i)); sumBenchmark.target('for loop').measure('sum', (_, numbers) => { let sum = 0; for (let i = 0; i < numbers.length; i++) sum += numbers[i]; }); sumBenchmark.target('reduce').measure('sum', (_, numbers) => { numbers.reduce((a, b) => a + b, 0); }); ``` -------------------------------- ### Define Benchmark Operations with Pre and Post Hooks (TypeScript) Source: https://context7.com/3axap4ehko/overtake/llms.txt Demonstrates how to define benchmark operations using the `measure()` function within the Overtake library. It illustrates the use of `pre()` and `post()` hooks for setup and cleanup tasks that are not included in the timing measurements. This is useful for preparing data or cleaning up resources before and after each benchmark cycle. ```typescript import { Benchmark, printJSONReports } from 'overtake'; const suite = new Benchmark('data', () => ({ items: [1, 2, 3, 4, 5] })); const target = suite.target('serialization', () => { const { parse, stringify } = JSON; let serialized: string = ''; return { parse, stringify, serialized }; }); // Measure stringify operation target .measure('stringify', ({ stringify }, input) => { stringify(input); }) .pre(async (ctx, input) => { // Runs before each measurement - not timed }) .post(async (ctx, input) => { // Runs after each measurement - not timed }); // Measure parse with pre-serialization target .measure('parse', ({ parse, serialized }) => { parse(serialized); }) .pre(async (ctx, input) => { // Serialize before measuring parse ctx.serialized = ctx.stringify(input); }) .post(async (ctx) => { // Clean up to avoid GC during next measurement ctx.serialized = ''; }); const reports = await suite.execute({ workers: 4, reportTypes: ['ops', 'mean', 'median', 'p95'], }); printJSONReports(reports, 2); ``` -------------------------------- ### Run Benchmark Suite via CLI (TypeScript) Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Executes a benchmark suite defined in a TypeScript file using the `npx overtake` command. This mode provides a global `benchmark` function, eliminating the need for explicit imports within the benchmark file. The output format can be specified, for example, as a table. ```typescript // benchmark.ts - No imports needed! const suite = benchmark('small', () => generateSmallData()).feed('large', () => generateLargeData()); suite.target('algorithm A').measure('process', (_, input) => { processA(input); }); suite.target('algorithm B').measure('process', (_, input) => { processB(input); }); ``` ```bash npx overtake benchmark.ts --format table ``` -------------------------------- ### Programmatic Benchmark Execution (TypeScript) Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Integrates Overtake into custom applications by importing the `Benchmark` class. This allows for more control over benchmark setup and execution. Benchmarks are defined, executed with specified worker counts and report types, and results are printed to the console. ```typescript import { Benchmark, printTableReports } from 'overtake'; const suite = new Benchmark('dataset', () => getData()); suite.target('impl').measure('op', (_, input) => { process(input); }); // Must explicitly execute const reports = await suite.execute({ workers: 4, reportTypes: ['ops', 'mean', 'p95'], }); printTableReports(reports); ``` -------------------------------- ### Prevent Garbage Collection During Measurement (TypeScript) Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Shows a technique to prevent the JavaScript garbage collector from interfering with benchmark measurements. By storing references to generated data in a `Set` within the target's setup, these objects are kept alive during the measurement phase. ```typescript const suite = benchmark('data', () => [1, 2, 3, 4, 5]); suite .target('no GC', () => { const gcBlock = new Set(); // Keeps references alive return { gcBlock }; }) .measure('process', ({ gcBlock }, input) => { const result = input.map((x) => x * x); gcBlock.add(result); // Prevent GC during measurement }); ``` -------------------------------- ### Handle Dynamic Imports in Worker Context with TypeScript Source: https://context7.com/3axap4ehko/overtake/llms.txt This snippet illustrates how to correctly perform dynamic imports within the functions executed by Overtake's workers. Since functions are stringified and run in isolated contexts, external dependencies must be imported dynamically inside the function body. It shows examples for Node.js built-in modules like 'node:v8', 'node:path', and 'node:crypto'. ```typescript import 'overtake'; // WRONG: closes over imported binding - undefined in worker // import { serialize } from 'node:v8'; // benchmark('data', getData).target('v8', () => ({ serialize })); // CORRECT: import inside worker-run function const suite = benchmark('data', () => ({ key: 'value', num: 42 })); suite .target('v8 serialization', async () => { // Dynamic import works in worker context const { serialize, deserialize } = await import('node:v8'); return { serialize, deserialize }; }) .measure('serialize', ({ serialize }, input) => { serialize(input); }); // Local file imports (CLI mode - relative to benchmark file) suite .target('local module', async () => { const { join } = await import('node:path'); const modulePath = join(process.cwd(), './build/myModule.js'); const { myHelper } = await import(modulePath); return { myHelper }; }) .measure('use helper', ({ myHelper }, input) => { myHelper(input); }); // Node built-in modules suite .target('crypto', async () => { const { createHash } = await import('node:crypto'); const { gzip } = await import('node:zlib'); const { promisify } = await import('node:util'); return { createHash, gzip: promisify(gzip) }; }) .measure('hash', ({ createHash }, input) => { createHash('md5').update(JSON.stringify(input)).digest('hex'); }); // Run: npx overtake benchmark.ts ``` -------------------------------- ### Create Benchmark Suite (CLI Mode) - Overtake Source: https://context7.com/3axap4ehko/overtake/llms.txt Demonstrates how to create a benchmark suite using the global `benchmark` function in CLI mode. This function initializes a suite with initial data and allows adding more data feeds and defining implementation targets with measurements. No explicit imports are needed for `benchmark` in this mode. ```typescript // benchmark.ts - CLI mode (no imports needed) import 'overtake'; // Create benchmark with initial data feed const suite = benchmark('1M numbers', () => Array.from({ length: 1_000_000 }, (_, i) => i) ); // Add additional data feeds for comparison suite.feed('100K numbers', () => Array.from({ length: 100_000 }, (_, i) => i) ); // Define implementation targets and measurements suite.target('for loop').measure('sum', (_, numbers) => { let sum = 0; for (let i = 0; i < numbers.length; i++) { sum += numbers[i]; } }); suite.target('reduce').measure('sum', (_, numbers) => { numbers.reduce((acc, curr) => acc + curr, 0); }); // Run: npx overtake benchmark.ts // Output: // for loop sum // 1M numbers: 1,607 ops/s // 100K numbers: 15,234 ops/s // // reduce sum // 1M numbers: 238 ops/s (6.7x slower) // 100K numbers: 2,456 ops/s (6.2x slower) ``` -------------------------------- ### Create Benchmark Suite with Multiple Feeds (TypeScript) Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Demonstrates the creation of a benchmark suite using the `benchmark` function and chaining the `.feed()` method to include multiple datasets. It also shows how to define targets and measures for comparison. ```typescript // Create with initial feed const suite = benchmark('initial data', () => data).feed('more data', () => moreData); // Add more datasets // Define what to compare suite.target('implementation A').measure('operation', (ctx, input) => { /* ... */ }); suite.target('implementation B').measure('operation', (ctx, input) => { /* ... */ }); ``` -------------------------------- ### Output Formats Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Details the different output formats available for Overtake benchmarks, allowing customization for console display, CI integration, and documentation. ```APIDOC ## Output Formats | Format | Description | | ----------- | -------------------------------- | | `simple` | Grouped console output (default) | | `table` | Console table format | | `json` | Compact JSON | | `pjson` | Pretty-printed JSON | | `markdown` | Markdown table for docs/PRs | | `histogram` | ASCII bar chart comparing ops/s | ### Markdown Example ```bash npx overtake bench.ts -f markdown ``` ```markdown ## for loop - sum | Feed | ops | | ---------- | --------------------- | | 1M numbers | 2,189 ops/s +/- 0.17% | ``` ### Histogram Example ```bash npx overtake bench.ts -f histogram ``` ``` for loop - sum 1M numbers | ████████████████████████████████████████ 2,189 ops/s reduce - sum 1M numbers | ████ 233 ops/s ``` ``` -------------------------------- ### Baseline Comparison for Performance Regression Detection (Bash) Source: https://context7.com/3axap4ehko/overtake/llms.txt Illustrates how to use the Overtake CLI to save benchmark results as baselines and compare subsequent runs against them. This is crucial for detecting performance regressions in CI pipelines. It shows commands for saving a baseline and comparing against an existing one, along with an explanation of the output indicators. ```bash # Save current results as baseline npx overtake bench.ts --save-baseline baseline.json # Later, compare against baseline npx overtake bench.ts --compare-baseline baseline.json # Output shows: # + Green: Performance improved (>5% better) # ! Red: Performance regressed (>5% worse) # No indicator: Within threshold # CI usage example npx overtake bench.ts --compare-baseline main-baseline.json -f json ``` -------------------------------- ### Show Detailed Benchmark Statistics Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Runs a specific benchmark file and reports detailed statistics including operations per second, mean duration, and various dispersion and confidence metrics. This provides in-depth performance insights. ```bash npx overtake bench.ts -r ops mean p95 p99 ``` -------------------------------- ### Run Overtake Benchmarks with Table Output Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Executes all benchmark files matching the specified pattern and displays the results in a console table format. This is useful for a human-readable overview of benchmark performance. ```bash npx overtake "**/*.bench.ts" -f table ``` -------------------------------- ### Overtake: Importing Local Files in CLI vs. Programmatic Mode Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Illustrates how to import local files within Overtake benchmarks, differentiating between CLI and programmatic usage. For CLI mode, relative paths work directly. For programmatic mode, `baseUrl` must be explicitly provided using `import.meta.url` for correct resolution. ```typescript // CLI usage – relative path is fine benchmark('local', () => 1) .target('helper', async () => { const { helper } = await import('./helpers.js'); return { helper }; }) .measure('use helper', ({ helper }) => helper()); // Programmatic usage – provide baseUrl const suite = new Benchmark('local'); suite .target('helper', async () => { const { helper } = await import('./helpers.js'); return { helper }; }) .measure('use helper', ({ helper }) => helper()); await suite.execute({ baseUrl: import.meta.url }); ``` -------------------------------- ### Display Progress Bar During Benchmark Execution Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Runs a benchmark file while showing a progress bar, which is helpful for long-running benchmarks to provide visual feedback on the execution status. ```bash npx overtake bench.ts --progress ``` -------------------------------- ### Baseline Comparison Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Explains how to use Overtake for performance regression tracking by saving and comparing benchmark results against a baseline file. ```APIDOC ## Baseline Comparison Track performance regressions by saving and comparing baselines: ```bash # Save current results as baseline npx overtake bench.ts --save-baseline baseline.json # Later, compare against baseline npx overtake bench.ts --compare-baseline baseline.json ``` **Output shows:** - `+` Green: Performance improved (>5% better) - `!` Red: Performance regressed (>5% worse) - No indicator: Within threshold **CI usage:** ```bash # In CI, fail if regression detected npx overtake bench.ts --compare-baseline main-baseline.json ``` ``` -------------------------------- ### Generate ASCII Histogram for Benchmark Comparison Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Runs a benchmark file and displays the results as an ASCII histogram, visually comparing the operations per second across different benchmarks. This offers a quick, graphical overview of performance differences. ```bash npx overtake bench.ts -f histogram ``` -------------------------------- ### Output Benchmark Results as JSON Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Executes a benchmark file and outputs the results in JSON format, suitable for programmatic consumption or integration into CI/CD pipelines. The output can be redirected to a file. ```bash npx overtake bench.ts -f json > results.json ``` -------------------------------- ### Generate Markdown Table for Benchmark Results Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Executes a benchmark file and formats the output as a Markdown table. This is ideal for embedding performance results directly into documentation or pull requests. ```bash npx overtake bench.ts -f markdown ``` -------------------------------- ### Compare Current Benchmark Results Against Baseline Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Executes a benchmark file and compares its results against a previously saved baseline JSON file. The output indicates performance improvements or regressions relative to the baseline. ```bash npx overtake bench.ts --compare-baseline baseline.json ``` -------------------------------- ### Programmatic Benchmark Suite Execution - Overtake Source: https://context7.com/3axap4ehko/overtake/llms.txt Shows how to use the `Benchmark` class for programmatic benchmark creation and execution. It allows detailed configuration of workers, cycles, thresholds, and report types, providing full control over the benchmarking process. Results can be printed in table or JSON format. ```typescript import { Benchmark, printTableReports, printJSONReports } from 'overtake'; import { randomUUID, randomInt } from 'node:crypto'; // Create benchmark suite programmatically const suite = new Benchmark('objects', () => Array.from({ length: 10_000 }, (_, idx) => ({ id: randomUUID(), value: randomInt(10_000), active: idx % 3 === 0, })) ) .feed('strings', () => Array.from({ length: 10_000 }, () => randomUUID())) .feed('numbers', () => Array.from({ length: 10_000 }, () => randomInt(10_000))); // Define targets with measurements suite.target('JSON').measure('stringify', (_, input) => { JSON.stringify(input); }); suite.target('v8', async () => { const { serialize } = await import('node:v8'); return { serialize }; }).measure('serialize', ({ serialize }, input) => { serialize(input); }); // Execute with custom options const reports = await suite.execute({ workers: 4, warmupCycles: 20, minCycles: 50, maxCycles: 1000, absThreshold: 1_000, // Stop if stddev < 1μs relThreshold: 0.02, // Stop if CoV < 2% gcObserver: true, progress: true, reportTypes: ['ops', 'mean', 'p95', 'p99'], }); // Output results printTableReports(reports); // Or: printJSONReports(reports, 2); ``` -------------------------------- ### Configure Report Types and Output Formats in TypeScript Source: https://context7.com/3axap4ehko/overtake/llms.txt This snippet demonstrates how to configure Overtake to generate reports with various statistical measures and output them in different formats. It includes setting up a benchmark suite, executing it with specified report types, and then printing the results using different formatters like simple console output, tables, JSON, Markdown, and histograms. Dependencies include the 'overtake' library. ```typescript import { Benchmark, printSimpleReports, printTableReports, printJSONReports, printMarkdownReports, printHistogramReports } from 'overtake'; const suite = new Benchmark('data', () => Array.from({ length: 10_000 }, Math.random)); suite.target('Math.floor').measure('round', (_, nums) => { for (const n of nums) Math.floor(n); }); // Available report types: // Core: 'ops', 'min', 'max', 'mean', 'median', 'mode' // Dispersion: 'sd', 'variance', 'sem', 'mad', 'iqr' // Confidence: 'moe', 'rme', 'ci_lower', 'ci_upper' // Percentiles: 'p1' through 'p99' const reports = await suite.execute({ workers: 4, reportTypes: ['ops', 'mean', 'median', 'sd', 'rme', 'p50', 'p95', 'p99', 'min', 'max'] as const, }); // Different output formats printSimpleReports(reports); // Grouped console output printTableReports(reports); // Console table format printJSONReports(reports, 2); // Pretty JSON printMarkdownReports(reports); // Markdown table printHistogramReports(reports); // ASCII bar chart // JSON output structure: // { // "Math.floor round": { // "data": { // "ops": "15,234 ops/s +/- 0.12%", // "mean": "65.64 μs", // "p95": "68.21 μs" // } // } // } ``` -------------------------------- ### Run Overtake Benchmarks from CLI Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Executes benchmark tests defined in a TypeScript file using the Overtake CLI. This command runs the benchmarks and displays the results, including operations per second and comparative performance. ```bash npx overtake benchmark.ts # Output: # for loop sum # 1M numbers: 1,607 ops/s # # reduce sum # 1M numbers: 238 ops/s (6.7x slower) ``` -------------------------------- ### Configure Overtake Programmatically with TypeScript Source: https://github.com/3axap4ehko/overtake/blob/master/README.md This TypeScript code snippet demonstrates how to programmatically configure the Overtake benchmark suite. It allows setting parameters like the number of workers, warmup and measurement cycles, thresholds for stopping, GC observer, report types, and progress bar options. ```typescript const reports = await suite.execute({ workers: 4, // Concurrent workers warmupCycles: 20, // Warmup iterations minCycles: 50, // Minimum measurement iterations maxCycles: 1000, // Maximum measurement iterations absThreshold: 1_000, // Stop if stddev < 1us relThreshold: 0.02, // Stop if CoV < 2% gcObserver: true, // Discard GC-affected samples reportTypes: ['ops', 'mean', 'p95'], progress: true, // Show progress bar progressInterval: 100, // Progress update interval (ms) }); ``` -------------------------------- ### Overtake CLI Usage Source: https://github.com/3axap4ehko/overtake/blob/master/README.md The primary command for running Overtake benchmarks. It accepts a pattern to find benchmark files and various options to control the benchmarking process and output. ```APIDOC ## CLI Options ```bash npx overtake [options] ``` ### Parameters #### Path Parameters - **pattern** (string) - Required - A glob pattern to find benchmark files (e.g., `**/*.bench.ts`). #### Query Parameters - **--format** or **-f** (string) - Optional - Output format. Supported formats include `simple`, `table`, `json`, `pjson`, `markdown`, `histogram`. Defaults to `simple`. - **--report-types** or **-r** (array of strings) - Optional - Specifies which metrics to report. See Available Metrics section for a list of supported metrics. Defaults to `['ops']`. - **--workers** or **-w** (number) - Optional - Number of concurrent workers to use for benchmarking. Defaults to the number of CPU cores. - **--min-cycles** (number) - Optional - Minimum number of measurement iterations for each benchmark. Defaults to 50. - **--max-cycles** (number) - Optional - Maximum number of measurement iterations for each benchmark. Defaults to 1000. - **--warmup-cycles** (number) - Optional - Number of warmup iterations before actual measurement. Defaults to 20. - **--abs-threshold** (number) - Optional - Absolute error threshold in nanoseconds for baseline comparison. Defaults to 1000. - **--rel-threshold** (number) - Optional - Relative error threshold (0-1) for baseline comparison. Defaults to 0.02. - **--no-gc-observer** - Optional - Disables garbage collection overlap detection. - **--progress** - Optional - Shows a progress bar during benchmark execution. - **--save-baseline** (string) - Optional - Saves the current benchmark results to the specified file path for baseline comparison. - **--compare-baseline** (string) - Optional - Compares current benchmark results against the specified baseline file. ### Example Commands ```bash # Run all benchmarks with table output npx overtake "**/*.bench.ts" -f table # Show detailed statistics npx overtake bench.ts -r ops mean p95 p99 # Output JSON for CI npx overtake bench.ts -f json > results.json # Show progress bar for long benchmarks npx overtake bench.ts --progress # Markdown output for docs/PRs npx overtake bench.ts -f markdown # ASCII histogram chart npx overtake bench.ts -f histogram ``` ``` -------------------------------- ### Available Metrics Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Lists and describes the various performance metrics that can be reported by Overtake, including core, dispersion, confidence, and percentile metrics. ```APIDOC ## Available Metrics Specify with `--report-types` or `reportTypes` option. ### Core Metrics | Metric | Description | | ------------- | ------------------------------- | | `ops` | Operations per second (default) | | `mean` | Average duration | | `median` | Middle value (p50) | | `min` / `max` | Range bounds | | `mode` | Most frequent duration | ### Dispersion Metrics | Metric | Description | | ---------- | ------------------------- | | `sd` | Standard deviation | | `variance` | Statistical variance | | `sem` | Standard error of mean | | `mad` | Median absolute deviation | | `iqr` | Interquartile range | ### Confidence Metrics | Metric | Description | | ---------- | ---------------------------- | | `moe` | Margin of error (95% CI) | | `rme` | Relative margin of error (%) | | `ci_lower` | Lower bound of 95% CI | | `ci_upper` | Upper bound of 95% CI | ### Percentiles `p1` through `p99` - any percentile **Example:** ```bash npx overtake bench.ts -r ops mean sd rme p50 p95 p99 ``` ``` -------------------------------- ### Programmatic Baseline Comparison (TypeScript) Source: https://context7.com/3axap4ehko/overtake/llms.txt Demonstrates how to programmatically save benchmark results as a baseline and compare them against a previously saved baseline using the Overtake library in TypeScript. This approach is useful for integrating performance regression checks directly into application code or build scripts. ```typescript import { Benchmark, reportsToBaseline, printComparisonReports, BaselineData } from 'overtake'; import { readFile, writeFile } from 'node:fs/promises'; const suite = new Benchmark('perf test', () => [1, 2, 3, 4, 5]); suite.target('impl').measure('op', (_, data) => data.map(x => x * 2)); const reports = await suite.execute({ reportTypes: ['ops', 'mean'] }); // Save baseline programmatically const baseline = reportsToBaseline(reports); await writeFile('baseline.json', JSON.stringify(baseline, null, 2)); // Load and compare against baseline const savedBaseline: BaselineData = JSON.parse( await readFile('baseline.json', 'utf8') ); printComparisonReports(reports, savedBaseline, 5); // 5% threshold ``` -------------------------------- ### Save Benchmark Results to Baseline File Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Executes a benchmark file and saves the resulting performance metrics to a specified JSON file. This baseline can be used later for comparison to detect performance regressions. ```bash npx overtake bench.ts --save-baseline baseline.json ``` -------------------------------- ### Overtake: Handling Capture-Free Functions in Workers Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Demonstrates the correct way to handle functions that rely on external modules within Overtake benchmarks. It highlights the issue of 'closing over' variables and shows how to import dependencies inside the worker-run function using dynamic import. ```typescript // ❌ WRONG: closes over serialize; it is undefined in the worker import { serialize } from 'node:v8'; benchmark('data', getData).target('v8', () => ({ serialize })); // ✅ CORRECT: import inside the worker-run function benchmark('data', getData) .target('v8', async () => { const { serialize } = await import('node:v8'); return { serialize }; }) .measure('serialize', ({ serialize }, input) => serialize(input)); ``` -------------------------------- ### Import Local Modules for Benchmarking (TypeScript) Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Demonstrates the correct way to import local modules within an Overtake benchmark, especially when dealing with compiled JavaScript files. It uses dynamic `import()` with `node:path` to resolve the module path relative to the current working directory. ```typescript // examples/imports.ts - Correct way to import local files const suite = benchmark('local modules', () => testData); suite .target('local files', async () => { const { join } = await import('node:path'); const modulePath = join(process.cwd(), './build/myModule.js'); const { myFunction } = await import(modulePath); return { myFunction }; }) .measure('call function', ({ myFunction }, input) => { myFunction(input); }); ``` -------------------------------- ### Additional Output Information Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Details additional information provided in Overtake benchmark outputs, such as memory tracking. ```APIDOC ## Additional Output Information ### Memory Tracking Each benchmark reports heap memory delta: ``` 1M numbers ops: 233 ops/s +/- 0.13% (heap: 1794KB) ``` This indicates memory allocated during the benchmark run. ``` -------------------------------- ### CI Usage: Fail on Performance Regression Source: https://github.com/3axap4ehko/overtake/blob/master/README.md Configures the Overtake CLI to fail the build or process if a performance regression is detected when comparing against a baseline file. This is crucial for maintaining performance stability in automated environments. ```bash npx overtake bench.ts --compare-baseline main-baseline.json ``` -------------------------------- ### Handle DCE Warnings in Benchmarks Source: https://github.com/3axap4ehko/overtake/blob/master/README.md When V8 eliminates benchmark code, leading to a '[DCE warning]', ensure your function returns a value, uses provided input data, or has observable side effects. The benchmark internally uses atomic operations, but simple operations might still trigger this. ```text 1M numbers ops: 5,000,000,000 ops/s [DCE warning] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.