### Original Bencher Benchmark Example

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/bencher_compatibility.md

This is a standard benchmark example using the `bencher` crate.

```rust
use bencher::{benchmark_group, benchmark_main, Bencher};

fn a(bench: &mut Bencher) {
    bench.iter(|| {
        (0..1000).fold(0, |x, y| x + y)
    })
}

fn b(bench: &mut Bencher) {
    const N: usize = 1024;
    bench.iter(|| {
        vec![0u8; N]
    });

    bench.bytes = N as u64;
}

benchmark_group!(benches, a, b);
benchmark_main!(benches);
```

--------------------------------

### Install cargo-wasi

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Install the cargo-wasi command-line tool to simplify building WASI programs.

```properties
cargo install cargo-wasi
```

--------------------------------

### Example Benchmark Output

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/bencher_compatibility.md

This is an example of the output you might see after running `cargo bench` with the compatibility layer.

```text
     Running target/release/deps/bencher_example-d865087781455bd5
a                       time:   [234.58 ps 237.68 ps 241.94 ps]
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

b                       time:   [23.972 ns 24.218 ns 24.474 ns]
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
```

--------------------------------

### Install cargo-criterion

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/cargo_criterion/cargo_criterion.md

Install the cargo-criterion tool using the cargo install command.

```bash
cargo install cargo-criterion
```

--------------------------------

### Install and Run Clippy

Source: https://github.com/bheisler/criterion.rs/blob/master/CONTRIBUTING.md

Install the clippy linter and run it on the entire workspace to check for common Rust errors and style issues. Fix any warnings before submitting changes.

```bash
rustup component add clippy
cargo clippy --workspace --all-targets
```

--------------------------------

### Per-Iteration Setup Without Timing Setup Cost with Bencher::iter_batched

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Generates a batch of inputs using a setup closure, times only the benchmark closure across the batch, and drops outputs after measurement. Ideal for algorithms that consume or mutate their input.

```rust
use criterion::{criterion_group, criterion_main, BatchSize, Criterion};
use std::hint::black_box;

fn bench_sort_batched(c: &mut Criterion) {
    c.bench_function("sort_unstable_batched", |b| {
        b.iter_batched(
            // Setup: called once per batch to produce fresh unsorted data
            || { let mut v: Vec<u64> = (0..1000).rev().collect(); v },
            // Routine: timed; receives ownership of setup output
            |mut data| { data.sort_unstable(); black_box(data) },
            // BatchSize controls memory/overhead trade-off
            BatchSize::SmallInput,
        )
    });
}

criterion_group!(benches, bench_sort_batched);
criterion_main!(benches);
```

--------------------------------

### Full Benchmark Output Example

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_output.md

This is a comprehensive example of the output produced by Criterion.rs during a benchmark run, showing warmup, sample collection, analysis, and detailed statistics.

```text
Benchmarking alloc
Benchmarking alloc: Warming up for 1.0000 s
Benchmarking alloc: Collecting 100 samples in estimated 13.354 s (5050 iterations)
Benchmarking alloc: Analyzing
alloc                   time:   [2.5094 ms 2.5306 ms 2.5553 ms]
                        thrpt:  [391.34 MiB/s 395.17 MiB/s 398.51 MiB/s]
                        change: [-38.292% -37.342% -36.524%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
slope  [2.5094 ms 2.5553 ms] R^2            [0.8660614 0.8640630]
mean   [2.5142 ms 2.5557 ms] std. dev.      [62.868 us 149.50 us]
median [2.5023 ms 2.5262 ms] med. abs. dev. [40.034 us 73.259 us]
```

--------------------------------

### Example Benchmark with `#[criterion]`

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/custom_test_framework.md

This example demonstrates how to define benchmarks using the `#[criterion]` attribute. It includes enabling unstable features, setting the test runner, and defining benchmarks with both default and custom Criterion configurations.

```rust
#![feature(custom_test_frameworks)]
#![test_runner(criterion::runner)]

use criterion::Criterion;
use criterion_macro::criterion;
use std::hint::black_box;

fn fibonacci(n: u64) -> u64 {
    match n {
        0 | 1 => 1,
        n => fibonacci(n - 1) + fibonacci(n - 2),
    }
}

fn custom_criterion() -> Criterion {
    Criterion::default()
        .sample_size(50)
}

#[criterion]
fn bench_simple(c: &mut Criterion) {
    c.bench_function("Fibonacci-Simple", |b| b.iter(|| fibonacci(black_box(10))));
}

#[criterion(custom_criterion())]
fn bench_custom(c: &mut Criterion) {
    c.bench_function("Fibonacci-Custom", |b| b.iter(|| fibonacci(black_box(20))));
}
```

--------------------------------

### Format Code with Rustfmt

Source: https://github.com/bheisler/criterion.rs/blob/master/CONTRIBUTING.md

Install the rustfmt-preview component and format the entire project's code to maintain a consistent style. This ensures adherence to the project's code style guidelines.

```bash
rustup component add rustfmt-preview
cargo fmt --all
```

--------------------------------

### Install wasmer-js for NodeJS

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Install the wasmer-cli package globally for NodeJS to run WASM modules.

```properties
npm install -g @wasmer/cli
```

--------------------------------

### Run Benchmarks with cargo-criterion

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/cargo_criterion/cargo_criterion.md

Execute benchmarks using the installed cargo-criterion tool.

```bash
cargo criterion
```

--------------------------------

### Benchmark Code Example for CSV Output

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/csv_output.md

An example of Rust code used to define a benchmark group and individual benchmarks, which corresponds to the data found in the `raw.csv` file.

```rust
fn compare_fibonaccis(c: &mut Criterion) {
    let mut group = c.benchmark_group("Fibonacci");
    group.bench_with_input("Recursive", 20, |b, i| b.iter(|| fibonacci_slow(*i)));
    group.bench_with_input("Iterative", 20, |b, i| b.iter(|| fibonacci_fast(*i)));
    group.finish();
}

```

--------------------------------

### Criterion.toml Configuration Example

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/cargo_criterion/configuring_cargo_criterion.md

This TOML snippet demonstrates various configuration options for cargo-criterion, including setting the output directory, output format, plotting backend, and custom colors for charts.

```toml
# This is used to override the directory where cargo-criterion saves 
# its data and generates reports.
criterion_home = "./target/criterion"

# This is used to configure the format of cargo-criterion's command-line output.
# Options are:
# criterion: Prints confidence intervals for measurement and throughput, and 
#   indicates whether a change was detected from the previous run. The default.
# quiet: Like criterion, but does not indicate changes. Useful for simply 
#   presenting output numbers, eg. on a library's README.
# verbose: Like criterion, but prints additional statistics.
# bencher: Emulates the output format of the bencher crate and nightly-only 
#   libtest benchmarks.
output_format = "criterion"

# This is used to configure the plotting backend used by cargo-criterion. 
# Options are "gnuplot" and "plotters", or "auto", which will use gnuplot if it's
# available or plotters if it isn't.
plotting_backend = "auto"

# The colors table allows users to configure the colors used by the charts 
# cargo-criterion generates.
[colors]
# These are used in many charts to compare the current measurement against 
# the previous one.
current_sample = {r = 31, g = 120, b = 180}
previous_sample = {r = 7, g = 26, b = 28}

# These are used by the full PDF chart to highlight which samples were outliers.
not_an_outlier = {r = 31, g = 120, b = 180}
mild_outlier = {r = 5, g = 127, b = 0}
severe_outlier = {r = 7, g = 26, b = 28}

# These are used for the line chart to compare multiple different functions.
comparison_colors = [
    {r = 8, g = 34, b = 34},
    {r = 6, g = 139, b = 87},
    {r = 0, g = 139, b = 139},
    {r = 5, g = 215, b = 0},
    {r = 0, g = 0, b = 139},
    {r = 0, g = 20, b = 60},
    {r = 9, g = 0, b = 139},
    {r = 0, g = 255, b = 127},
]

```

--------------------------------

### Original libtest benchmark

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/migrating_from_libtest.md

This is the starting benchmark code using Rust's libtest harness.

```rust
#![feature(test)]
extern crate test;
use test::Bencher;
use test::black_box;

fn fibonacci(n: u64) -> u64 {
    match n {
        0 => 1,
        1 => 1,
        n => fibonacci(n-1) + fibonacci(n-2),
    }
}

#[bench]
fn bench_fib(b: &mut Bencher) {
    b.iter(|| fibonacci(black_box(20)));
}
```

--------------------------------

### Benchmark Async Function with FuturesExecutor

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/benchmarking_async.md

This example demonstrates benchmarking an async function using the `FuturesExecutor`. Ensure the `async_futures` feature is enabled for Criterion.rs.

```rust
use criterion::BenchmarkId;
use criterion::Criterion;
use criterion::{criterion_group, criterion_main};

// This is a struct that tells Criterion.rs to use the "futures" crate's current-thread executor
use criterion::async_executor::FuturesExecutor;

// Here we have an async function to benchmark
async fn do_something(size: usize) {
    // Do something async with the size
}

fn from_elem(c: &mut Criterion) {
    let size: usize = 1024;

    c.bench_with_input(BenchmarkId::new("input_example", size), &size, |b, &s| {
        // Insert a call to `to_async` to convert the bencher to async mode.
        // The timing loops are the same as with the normal bencher.
        b.to_async(FuturesExecutor).iter(|| do_something(s));
    });
}

criterion_group!(benches, from_elem);
criterion_main!(benches);
```

--------------------------------

### Benchmark setup for Fibonacci function

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/getting_started.md

This Rust code sets up a benchmark for the fibonacci function using Criterion.rs. It imports necessary items and defines the benchmark function.

```rust
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;
use mycrate::fibonacci;

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("fib 20", |b| b.iter(|| fibonacci(black_box(20))));
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
```

--------------------------------

### Fibonacci function implementation

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/getting_started.md

An example of a function to be benchmarked. Ensure it's public if benchmarking from a separate crate.

```rust
#[inline]
pub fn fibonacci(n: u64) -> u64 {
    match n {
        0 => 1,
        1 => 1,
        n => fibonacci(n-1) + fibonacci(n-2),
    }
}
```

--------------------------------

### Custom Measurement with MicroSeconds

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Implement the `Measurement` and `ValueFormatter` traits to benchmark using custom metrics. This example defines a `MicroSeconds` measurement that counts nanoseconds but reports in microseconds.

```rust
use criterion::measurement::{Measurement, ValueFormatter};
use criterion::{criterion_group, criterion_main, Throughput, Criterion};
use std::time::{Duration, Instant};

/// A measurement that counts nanoseconds but reports in microseconds.
struct MicroSeconds;

struct MicroSecondsFormatter;
impl ValueFormatter for MicroSecondsFormatter {
    fn scale_values(&self, _typical: f64, values: &mut [f64]) -> &'static str {
        for v in values { *v /= 1_000.0; }  // ns → µs
        "µs"
    }
    fn scale_throughputs(&self, typical: f64, throughput: &Throughput, values: &mut [f64]) -> &'static str {
        self.scale_values(typical, values);
        "µs"
    }
    fn scale_for_machines(&self, values: &mut [f64]) -> &'static str {
        for v in values { *v /= 1_000.0; }
        "µs"
    }
}

impl Measurement for MicroSeconds {
    type Intermediate = Instant;
    type Value = Duration;
    fn start(&self) -> Instant { Instant::now() }
    fn end(&self, i: Instant) -> Duration { i.elapsed() }
    fn add(&self, v1: &Duration, v2: &Duration) -> Duration { *v1 + *v2 }
    fn zero(&self) -> Duration { Duration::from_secs(0) }
    fn to_f64(&self, v: &Duration) -> f64 { v.as_nanos() as f64 }
    fn formatter(&self) -> &dyn ValueFormatter { &MicroSecondsFormatter }
}

fn bench_with_custom_measurement(c: &mut Criterion<MicroSeconds>) {
    c.bench_function("custom_metric", |b| {
        b.iter(|| (0u64..1000).sum::<u64>())
    });
}

fn custom_criterion() -> Criterion<MicroSeconds> {
    Criterion::default().with_measurement(MicroSeconds)
}

criterion_group! {
    name = benches;
    config = custom_criterion();
    targets = bench_with_custom_measurement
}
criterion_main!(benches);

```

--------------------------------

### Formatting Nanoseconds with SI Prefixes

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/custom_measurements.md

Example of formatting time values in nanoseconds using SI prefixes for better human readability. It converts ns to ps, ns, us, ms, and s based on magnitude.

```rust
if ns < 1.0 {
    format!("{:>6} ps", ns * 1e3)
} else if ns < 10f64.powi(3) {
    format!("{:>6} ns", ns)
} else if ns < 10f64.powi(6) {
    format!("{:>6} us", ns / 1e3)
} else if ns < 10f64.powi(9) {
    format!("{:>6} ms", ns / 1e6)
} else {
    format!("{:>6} s", ns / 1e9)
}
```

--------------------------------

### Clone Criterion.rs Repository

Source: https://github.com/bheisler/criterion.rs/blob/master/CONTRIBUTING.md

Clone the Criterion.rs repository to your local machine to start making changes. Ensure you replace 'your-username' with your GitHub username.

```bash
git clone git@github.com:your-username/criterion.rs.git
```

--------------------------------

### Criterion.rs Benchmark Output Analysis

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/getting_started.md

Example output from a Criterion.rs benchmark run for a Fibonacci function. It shows statistical analysis, including time measurements, change percentages, and outlier detection, indicating performance improvements.

```text
     Running target/release/deps/example-423eedc43b2b3a93
Benchmarking fib 20
Benchmarking fib 20: Warming up for 3.0000 s
Benchmarking fib 20: Collecting 100 samples in estimated 5.0000 s (13548862800 iterations)
Benchmarking fib 20: Analyzing
fib 20                  time:   [353.59 ps 356.19 ps 359.07 ps]
                        change: [-99.999% -99.999% -99.999%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 99 measurements (6.06%)
  4 (4.04%) high mild
  2 (2.02%) high severe
slope  [353.59 ps 359.07 ps] R^2            [0.8734356 0.8722124]
mean   [356.57 ps 362.74 ps] std. dev.      [10.672 ps 20.419 ps]
median [351.57 ps 355.85 ps] med. abs. dev. [4.6479 ps 10.059 ps]
```

--------------------------------

### Per-Iteration Setup With Mutable Reference using Bencher::iter_batched_ref

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Similar to `iter_batched`, but the benchmark closure receives a `&mut T` instead of owning the input. Use when the routine should mutate but not consume the input.

```rust
use criterion::{criterion_group, criterion_main, BatchSize, Criterion};
use std::hint::black_box;

fn bench_sort_ref(c: &mut Criterion) {
    c.bench_function("sort_unstable_ref", |b| {
        b.iter_batched_ref(
            // Setup: produce unsorted Vec
            || (0u64..1000).rev().collect::<Vec<_>>(),
            // Routine: receives &mut Vec<u64>; does not consume it
            |data| { data.sort_unstable(); black_box(data.len()) },
            BatchSize::SmallInput,
        )
    });
}

criterion_group!(benches, bench_sort_ref);
criterion_main!(benches);
```

--------------------------------

### Generate Firefox Baseline and Export

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Commands to run benchmarks with Firefox, save a baseline, and export results. The export requires piping to a download mechanism.

```properties
hex --bench --save-baseline firefox
```

```properties
hex --bench --export firefox | download
```

--------------------------------

### Generate Chromium Baseline and Export

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Commands to run benchmarks with Chromium, save a baseline, and export results. The export requires piping to a download mechanism.

```properties
hex --bench --save-baseline chromium
```

```properties
hex --bench --export chromium | download
```

--------------------------------

### Run WASI benchmark in browser via webassembly.sh

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Execute the WASM benchmark in a web browser using webassembly.sh, which shims WASI. Drag-and-drop the WASM file and run the benchmark command.

```properties
hex --bench
```

--------------------------------

### Override Plotting Backend

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Specify a different backend for generating plots, such as 'gnuplot' or 'plotters'. 'gnuplot' is the default if installed.

```bash
cargo bench -- --plotting-backend gnuplot
```

```bash
cargo bench --plotting-backend plotters
```

--------------------------------

### Run WASI benchmark with wasmer

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Execute the compiled WASM benchmark using the wasmer runtime, mounting the current directory and passing benchmark arguments.

```properties
wasmer run --dir=. hex.wasm -- --bench
```

--------------------------------

### Run Benchmarks for Testing

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Execute benchmarks without measurement or analysis, suitable for CI environments to verify that benchmarks run successfully.

```bash
cargo test --benches
```

--------------------------------

### Import necessary items for benchmarking

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/getting_started.md

Imports Criterion.rs types, black_box for preventing optimizations, and the function to be benchmarked from the external crate.

```rust
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;
use mycrate::fibonacci;
```

--------------------------------

### Run WASI benchmark with wasmtime

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Execute the compiled WASM benchmark using the wasmtime runtime, mounting the current directory and passing benchmark arguments.

```properties
wasmtime run --dir=. hex.wasm -- --bench
```

--------------------------------

### Custom Async Executor Integration

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Implement the `AsyncExecutor` trait to integrate custom async runtimes with Criterion.rs. This example uses a simple blocking executor.

```rust
// Cargo.toml: criterion = { version = "0.7", features = ["async"] }
use criterion::async_executor::AsyncExecutor;
use criterion::{criterion_group, criterion_main, Criterion};
use std::future::Future;

/// Custom executor using a simple blocking approach
struct MyExecutor;

impl AsyncExecutor for MyExecutor {
    fn block_on<T>(&self, future: impl Future<Output = T>) -> T {
        // Use your custom runtime here
        futures::executor::block_on(future)
    }
}

async fn my_async_work() -> u64 { 42 }

fn bench_custom_executor(c: &mut Criterion) {
    c.bench_function("custom_async", |b| {
        b.to_async(MyExecutor).iter(|| my_async_work())
    });
}

criterion_group!(benches, bench_custom_executor);
criterion_main!(benches);

```

--------------------------------

### Generate Wasmtime Baseline and JSON

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Use these commands to run benchmarks with Wasmtime, save a baseline, and export results to a JSON file.

```properties
wasmtime run --dir=. hex.wasm -- --bench --save-baseline wasmtime
```

```properties
wasmtime run --dir=. hex.wasm -- --bench --export wasmtime > wasmtime.json
```

--------------------------------

### Run Benchmarks

Source: https://github.com/bheisler/criterion.rs/blob/master/CONTRIBUTING.md

Execute the project's benchmarks to assess performance. This is typically run alongside tests.

```bash
cargo bench
```

--------------------------------

### Run Project Tests and Benchmarks

Source: https://github.com/bheisler/criterion.rs/blob/master/plot/CONTRIBUTING.md

After making code changes, run all project tests and benchmarks to ensure everything is functioning correctly. This requires Cargo and the Rust toolchain.

```bash
cargo test --all

```

```bash
cargo bench
```

--------------------------------

### Global Benchmark Configuration with `Criterion::default()`

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Configures global settings for all benchmarks in a group using a builder chain on `Criterion::default()`. Allows customization of sample size, warm-up time, measurement time, and other parameters.

```rust
use criterion::{criterion_group, criterion_main, Criterion, PlottingBackend};
use std::time::Duration;

fn my_bench(c: &mut Criterion) {
    c.bench_function("example", |b| b.iter(|| 42u64));
}

fn custom_criterion() -> Criterion {
    Criterion::default()
        .sample_size(200)                               // default 100; min 10
        .warm_up_time(Duration::from_secs(5))           // default 3s
        .measurement_time(Duration::from_secs(10))      // default 5s
        .nresamples(50_000)                             // default 100_000
        .noise_threshold(0.02)                          // default 0.01 (1%)
        .confidence_level(0.95)                         // default 0.95
        .significance_level(0.05)                       // default 0.05
        .plotting_backend(PlottingBackend::Plotters)    // Gnuplot or Plotters
        .with_output_color(true)
        .save_baseline("my-baseline".to_string())
}

criterion_group! {
    name = benches;
    config = custom_criterion();
    targets = my_bench
}
criterion_main!(benches);
```

--------------------------------

### Generate main function with criterion_main! Macro

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Expands to a `main` function that runs all specified benchmark groups and prints the final summary. Must be called exactly once in a benchmark file.

```rust
use criterion::{criterion_group, criterion_main, Criterion};

fn bench_a(c: &mut Criterion) {
    c.bench_function("bench_a", |b| b.iter(|| 1 + 1));
}

fn bench_b(c: &mut Criterion) {
    c.bench_function("bench_b", |b| b.iter(|| 2 + 2));
}

criterion_group!(group_a, bench_a);
criterion_group!(group_b, bench_b);
criterion_main!(group_a, group_b);
// Expands to:
// fn main() {
//     group_a();
//     group_b();
//     Criterion::default().configure_from_args().final_summary();
// }
```

--------------------------------

### Criterion.rs Measurement Trait Definition

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/custom_measurements.md

Defines the core `Measurement` trait for custom timing measurements in Criterion.rs. Implementors must provide methods for starting, ending, accumulating, and formatting measurements.

```rust
pub trait Measurement {
    type Intermediate;
    type Value: MeasuredValue;

    fn start(&self) -> Self::Intermediate;
    fn end(&self, i: Self::Intermediate) -> Self::Value;

    fn add(&self, v1: &Self::Value, v2: &Self::Value) -> Self::Value;
    fn zero(&self) -> Self::Value;
    fn to_f64(&self, val: &Self::Value) -> f64;

    fn formatter(&self) -> &dyn ValueFormatter;
}
```

--------------------------------

### Generate Wasmer Baseline and JSON

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Use these commands to run benchmarks with Wasmer, save a baseline, and export results to a JSON file.

```properties
wasmer run --dir=. hex.wasm -- --bench --save-baseline wasmer
```

```properties
wasmer run --dir=. hex.wasm -- --bench --export wasmer > wasmer.json
```

--------------------------------

### Generate benchmark group and main function

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/getting_started.md

Uses macros to create a benchmark group and a main function to run the benchmarks.

```rust
criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
```

--------------------------------

### Build WASI benchmark with cargo-wasi

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Compile your benchmark for WASI using cargo-wasi, which automatically selects the correct target and optimizes the output.

```properties
cargo wasi build --bench=hex --release
```

--------------------------------

### Add Iai to Cargo.toml

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/iai/getting_started.md

Include Iai as a development dependency and configure the benchmark harness in your Cargo.toml file.

```toml
[dev-dependencies]
iai = "0.1"

[[bench]]
name = "my_benchmark"
harness = false

```

--------------------------------

### Benchmark a Single Parameterless Function with Criterion::bench_function

Source: https://context7.com/bheisler/criterion.rs/llms.txt

The simplest benchmarking entry point. Runs the provided closure under the timing loop and reports statistics. Use `black_box` to prevent the compiler from optimizing away the work being benchmarked.

```rust
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;

fn my_sort(data: &mut Vec<u64>) {
    data.sort_unstable();
}

fn bench_sort(c: &mut Criterion) {
    let data: Vec<u64> = (0..1000u64).rev().collect();

    c.bench_function("sort 1000 reversed u64", |b| {
        b.iter(|| {
            let mut v = black_box(data.clone());
            my_sort(&mut v);
            v
        })
    });
}

criterion_group!(benches, bench_sort);
criterion_main!(benches);
// Output:
// sort 1000 reversed u64  time: [xx.x µs xx.x µs xx.x µs]
```

--------------------------------

### Generate Node.js Baseline and JSON

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Use these commands to run benchmarks with wasmer-js (for Node.js), save a baseline, and export results to a JSON file.

```properties
wasmer-js run --dir=. hex.wasm -- --bench --save-baseline nodejs
```

```properties
wasmer-js run --dir=. hex.wasm -- --bench --export nodejs > nodejs.json
```

--------------------------------

### Criterion.rs Benchmark File Structure

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/csv_output.md

Illustrates the directory structure where Criterion.rs saves benchmark results, including base, change, and new measurement directories.

```text
$BENCHMARK/
├── base/
│  ├── raw.csv
│  ├── estimates.json
│  ├── sample.json
│  └── tukey.json
├── change/
│  └── estimates.json
├── new/
│  ├── raw.csv
│  ├── estimates.json
│  ├── sample.json
│  └── tukey.json

```

--------------------------------

### Define Benchmark Functions in Rust

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/iai/getting_started.md

Create benchmark functions in your Rust code, utilizing `iai::black_box` to prevent optimizations and `iai::main` to register them.

```rust
use iai::{black_box, main};

fn fibonacci(n: u64) -> u64 {
    match n {
        0 => 1,
        1 => 1,
        n => fibonacci(n-1) + fibonacci(n-2),
    }
}

fn iai_benchmark_short() -> u64 {
    fibonacci(black_box(10))
}

fn iai_benchmark_long() -> u64 {
    fibonacci(black_box(30))
}


iai::main!(iai_benchmark_short, iai_benchmark_long);

```

--------------------------------

### Run Benchmarks Quickly

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Execute benchmarks faster with reduced statistical accuracy. This option is useful for quick checks when high precision is not required.

```bash
cargo bench -- --quick
```

--------------------------------

### Run Criterion.rs Tests and Benchmarks

Source: https://github.com/bheisler/criterion.rs/blob/master/bencher_compat/CONTRIBUTING.md

Execute all tests and benchmarks for the Criterion.rs project using Cargo. This ensures your changes do not break existing functionality.

```bash
cargo test --all
```

```bash
cargo bench
```

--------------------------------

### Run All Tests

Source: https://github.com/bheisler/criterion.rs/blob/master/CONTRIBUTING.md

Execute all tests for the project to ensure code integrity. This command should be run after making changes to the code.

```bash
cargo test --all
```

--------------------------------

### Tune Statistical Parameters with BenchmarkGroup Configuration Methods

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Allows overriding statistical settings for a benchmark group, such as sample size, warm-up time, measurement time, and sampling mode. This enables fine-tuning without altering global defaults.

```rust
use criterion::{criterion_group, criterion_main, AxisScale, Criterion,
                PlotConfiguration, SamplingMode};
use std::time::Duration;

fn slow_fn() { std::thread::sleep(Duration::from_millis(10)); }
fn fast_fn() -> u64 { (0u64..1000).sum() }

fn bench_configured(c: &mut Criterion) {
    // Group for a slow function: use flat sampling, fewer samples
    let mut slow_group = c.benchmark_group("slow-ops");
    slow_group
        .sample_size(10)
        .warm_up_time(Duration::from_secs(1))
        .measurement_time(Duration::from_secs(20))
        .sampling_mode(SamplingMode::Flat);
    slow_group.bench_function("slow_fn", |b| b.iter(|| slow_fn()));
    slow_group.finish();

    // Group for a fast function: high precision settings + log-scale axis
    let plot_config = PlotConfiguration::default()
        .summary_scale(AxisScale::Logarithmic);
    let mut fast_group = c.benchmark_group("fast-ops");
    fast_group
        .sample_size(500)
        .significance_level(0.01)
        .noise_threshold(0.005)
        .confidence_level(0.99)
        .nresamples(200_000)
        .plot_config(plot_config);
    fast_group.bench_function("fast_fn", |b| b.iter(|| fast_fn()));
    fast_group.finish();
}

criterion_group!(benches, bench_configured);
criterion_main!(benches);
```

--------------------------------

### Baseline Comparison Commands

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Use these bash commands to manage and compare performance baselines across different runs of your benchmarks. This helps in structured A/B testing of code changes.

```bash
# Save current results as the "main" baseline
cargo bench -- --save-baseline main
```

```bash
# After making changes, compare against the saved baseline (strict: fails if any benchmark is missing)
cargo bench -- --baseline main
```

```bash
# Lenient comparison: skips missing benchmarks instead of failing (useful in CI)
cargo bench -- --baseline-lenient main
```

```bash
# Profile mode: iterate for N seconds without saving/analyzing (for use with profilers)
cargo bench -- --profile-time 10
```

```bash
# Run only benchmarks matching a regex filter
cargo bench -- "fibonacci.*Iterative"
```

```bash
# List all benchmarks without running them
cargo bench -- --list
```

```bash
# Run in test mode (verifies benchmarks compile and execute without measuring)
cargo test --benches
```

--------------------------------

### Async Benchmark Support with `Bencher::to_async`

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Converts a synchronous `Bencher` into an `AsyncBencher` using a provided async executor. Requires the `async_tokio` feature. All standard timing loops are available on `AsyncBencher`.

```rust
// Cargo.toml: criterion = { version = "0.7", features = ["async_tokio"] }
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};
use tokio::runtime::Runtime;

async fn async_fetch(n: u64) -> u64 {
    // Simulate async I/O work
    tokio::task::yield_now().await;
    n * 2
}

fn bench_async(c: &mut Criterion) {
    let rt = Runtime::new().unwrap();
    let mut group = c.benchmark_group("async-ops");

    for &n in &[10u64, 100, 1000] {
        group.bench_with_input(BenchmarkId::new("async_fetch", n), &n, |b, &val| {
            b.to_async(&rt).iter(|| async move { async_fetch(val).await });
        });
    }

    group.finish();
}

criterion_group!(benches, bench_async);
criterion_main!(benches);
```

--------------------------------

### Basic Timing Loop with Bencher::iter

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Uses the default timing loop to call a closure N times and record total elapsed time. Suitable for benchmarks where the returned value has no expensive `Drop`.

```rust
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;

fn bench_iter(c: &mut Criterion) {
    let data: Vec<u32> = (0..10_000).collect();

    c.bench_function("sum_iter", |b| {
        b.iter(|| {
            // The entire closure body is timed.
            // black_box prevents the compiler from optimizing away data access.
            black_box(&data).iter().map(|&x| x as u64).sum::<u64>()
        })
    });
}

criterion_group!(benches, bench_iter);
criterion_main!(benches);
```

--------------------------------

### Save and Compare Baselines

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Manage benchmark baselines by saving current results to a named baseline or comparing against an existing one. This is crucial for tracking performance regressions.

```bash
cargo bench -- --save-baseline <name>
```

```bash
cargo bench -- --baseline <name>
```

--------------------------------

### Baseline Comparison Options

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Provides options for comparing against named baselines, including strict comparison, lenient comparison, and loading a baseline as the new data set.

```bash
cargo bench -- --baseline-lenient <name>
```

```bash
cargo bench -- --load-baseline <name>
```

--------------------------------

### Run Criterion.rs Benchmarks with Specific Arguments

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/faq.md

When `cargo bench` gives 'Unrecognized Option' errors, run only your Criterion benchmark with specific arguments by prefixing them with `--`.

```bash
cargo bench --bench my_benchmark -- --verbose
```

--------------------------------

### Manage Custom Baselines

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Utilize custom baselines for comparing benchmark results across different branches or states. Options include saving, loading, and lenient comparison.

```bash
git checkout master
cargo bench -- --save-baseline master
git checkout feature
cargo bench -- --save-baseline feature
git checkout optimizations

# Some optimization work here

# Measure again
cargo bench
# Now compare against the stored baselines without overwriting it or re-running the measurements
cargo bench -- --load-baseline new --baseline master
cargo bench -- --load-baseline new --baseline feature
```

--------------------------------

### Export WASI benchmark results

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Export benchmark results to JSON format using the --export flag and pipe to a download command.

```properties
hex --bench --export=base | download
```

--------------------------------

### Add Criterion.rs dependency to Cargo.toml

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/getting_started.md

Add this to your Cargo.toml to include Criterion.rs as a development dependency and configure a benchmark.

```toml
[dev-dependencies]
criterion = "0.5.1"

[[bench]]
name = "my_benchmark"
harness = false
```

--------------------------------

### Enabling In-Process Profiling with Custom Profiler

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/profiling.md

Demonstrates how to configure Criterion.rs to use a custom in-process profiler. The profiler hook only takes effect when running in `--profile-time` mode.

```rust
extern crate my_custom_profiler;
use my_custom_profiler::MyCustomProfiler;

fn fibonacci_profiled(criterion: &mut Criterion) {
    // Use the criterion struct as normal here.
}

fn profiled() -> Criterion {
    Criterion::default().with_profiler(MyCustomProfiler)
}

criterion_group! {
    name = benches;
    config = profiled();
    targets = fibonacci_profiled
}
```

--------------------------------

### Change CLI Output Format

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Select an alternative output format for the command-line interface, such as 'criterion' or 'bencher'. The 'bencher' format is compatible with external parsing tools.

```bash
cargo bench -- --output-format <name>
```

--------------------------------

### Add Criterion.rs to Cargo.toml

Source: https://context7.com/bheisler/criterion.rs/llms.txt

Add Criterion.rs as a dev-dependency and declare a benchmark target with `harness = false` to replace the default benchmark harness.

```toml
[dev-dependencies]
criterion = { version = "0.7", features = ["html_reports"] }

[[bench]]
name = "my_benchmark"
harness = false
```

--------------------------------

### Profile Benchmark Execution Time

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/command_line_options.md

Run benchmarks for a fixed duration without saving or analyzing results, useful for profiling. This minimizes overhead and focuses on execution time.

```bash
cargo bench -- --profile-time <num_seconds>
```

--------------------------------

### Define and run Criterion.rs benchmarks

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/migrating_from_libtest.md

Use `criterion_group!` and `criterion_main!` macros to define the benchmark group and generate the main function for running benchmarks with Criterion.rs.

```rust
criterion_group!(benches, bench_fib);
criterion_main!(benches);
```

--------------------------------

### Build WASI benchmark without cargo-wasi

Source: https://github.com/bheisler/criterion.rs/blob/master/book/src/user_guide/wasi.md

Compile your benchmark for WASI manually using cargo, specifying the target and release mode.

```properties
cargo build --bench=hex --release --target wasm32-wasi
```