### Setup and Teardown Phases in Julia Benchmarks

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Demonstrates the use of `setup` and `teardown` expressions in `@benchmarkable`. The `setup` expression is evaluated before each sample, allowing for preparation of the environment without affecting benchmark results. This is useful for tasks like copying data.

```julia
julia> x = rand(100000)

# For each sample, bind a variable `y` to a fresh copy of `x`. As you
# can see, `y` is accessible within the scope of the core expression.
julia> b = @benchmarkable sort!(y) setup=(y = copy($x)) # setup expression
Benchmark(evals=1, seconds=5.0, samples=10000)

julia> run(b)
BenchmarkTools.Trial: 819 samples with 1 evaluations.
 Range (min … max):  5.983 ms …  6.954 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     6.019 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   6.029 ms ± 46.222 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

        ▃▂▂▄█▄▂▃                                                  
  ▂▃▃▄▆▅████████▇▆▆▅▄▄▄▅▆▄▃▄▅▄▃▂▂▃▁▂▂▂▁▂▂▂▂▂▂▁▁▁▁▂▂▁▁▁▂▂▁▁▂▁▁▂
  5.98 ms           Histogram: frequency by time           6.18 ms (top 1%)

 Memory estimate: 0 bytes, allocs estimate: 0.
```

--------------------------------

### Install BenchmarkTools.jl

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/README.md

To install BenchmarkTools, open Julia's REPL, enter package mode by pressing ']', and then type 'add BenchmarkTools'.

```julia
pkg> add BenchmarkTools
```

--------------------------------

### Multiple Assignments in Julia Benchmark Setup

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Shows how to correctly provide multiple assignments within the `setup` expression for Julia benchmarks. Assignments must be separated by semicolons; commas will result in an error.

```julia
julia> @btime x + y setup = (x=1; y=2)  # works
  1.238 ns (0 allocations: 0 bytes)
3
```

```julia
julia> @btime x + y setup = (x=1, y=2)  # errors
ERROR: UndefVarError: `x` not defined
```

```julia
julia> @btime exp(x) setup = (x=1,)  # errors
ERROR: UndefVarError: `x` not defined
```

--------------------------------

### Benchmark Code Execution with @benchmark

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/README.md

The @benchmark macro is used to benchmark code. The `setup` expression is run once per sample and is not included in timing results. Each sample can require multiple evaluations.

```julia
julia> using BenchmarkTools

# The `setup` expression is run once per sample, and is not included in the
# timing results. Note that each sample can require multiple evaluations
# benchmark kernel evaluations. See the BenchmarkTools manual for details.
julia> @benchmark sort(data) setup=(data=rand(10))
BenchmarkTools.Trial: 10000 samples with 972 evaluations.
 Range (min … max):  69.399 ns …  1.066 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     83.850 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   89.471 ns ± 53.666 ns  ┊ GC (mean ± σ):  3.25% ± 5.16%

          ▁▄▇█▇▆▃▁                                                 
  ▂▁▁▂▂▃▄▆████████▆▅▄▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂
  69.4 ns           Histogram: frequency by time             145 ns (top 1%)

 Memory estimate: 160 bytes, allocs estimate: 1.
```

--------------------------------

### Install cset for Processor Shielding

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/linuxtips.md

Installs the 'cset' utility on Ubuntu for managing processor shields. This tool helps protect dedicated processors and memory nodes from the Linux scheduler.

```bash
➜ sudo apt-get install cpuset
```

--------------------------------

### Benchmark Julia Code with @benchmark

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/index.md

Use the @benchmark macro to measure the performance of a Julia expression. The setup expression is run once per sample and is not included in the timing results. Output includes median time, mean and standard deviation, and a histogram of results.

```julia
using BenchmarkTools

# The `setup` expression is run once per sample, and is not included in the
# timing results. Note that each sample can require multiple evaluations
benchmark kernel evaluations. See the BenchmarkTools manual for details.
@benchmark sort(data) setup=(data=rand(10))
```

--------------------------------

### Start ASLR-Disabled Shell

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/linuxtips.md

Launches a new shell session with ASLR disabled for the current user, without affecting the global system setting. Useful for specific benchmark runs.

```bash
setarch $(uname -m) -R /bin/sh
```

--------------------------------

### Filter BenchmarkGroup by Tagged Keys and Parent Tags

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Demonstrates filtering a BenchmarkGroup using the @tagged macro with tags that are keys of child groups or tags of parent groups. This example selects groups tagged '8' or '9'.

```julia
g[@tagged "8" || "9"]
```

--------------------------------

### @benchmarkable — Define a reusable benchmark object

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Creates a `Benchmark` instance without running it, enabling explicit `tune!` / `run` workflows. This is the low-level primitive underlying all benchmark macros and is what gets stored in `BenchmarkGroup` suites.

```APIDOC
## @benchmarkable — Define a reusable benchmark object

### Description
Creates a `Benchmark` instance without running it, enabling explicit `tune!` / `run` workflows. This is the low-level primitive underlying all benchmark macros and is what gets stored in `BenchmarkGroup` suites. Accepts the same keyword arguments as `@benchmark` plus `setup` and `teardown` expressions.

### Usage
```julia
using BenchmarkTools

# Define without running
b = @benchmarkable sort!(y) setup=(y = copy($(rand(1000)))) evals=1

# Tune to find the optimal evals/sample
tune!(b)

# Run and capture the full Trial
t = run(b)

# Override parameters at run-time
t2 = run(b, seconds=2, samples=200)

# Inspect the benchmark's parameters
params(b)

# Manually set parameters (bypasses tune!)
b2 = @benchmarkable sum($( rand(100) )) seconds=3 samples=1000 evals=50
run(b2)
```

### Parameters
Accepts the same keyword arguments as `@benchmark` (`seconds`, `samples`, `evals`, `setup`, `teardown`), plus:
- `setup` (Any): Expression to run once per sample before each evaluation (excluded from timing).
- `teardown` (Any): Expression to run once per sample after each evaluation (excluded from timing).
```

--------------------------------

### Filter BenchmarkGroup by Tagged Predicate

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Applies a complex boolean predicate using the @tagged macro to filter a BenchmarkGroup. This example selects groups tagged with '3' or '7' but not '1'.

```julia
g[@tagged ("3" || "7") && !("1")]
```

--------------------------------

### Define a reusable benchmark object with @benchmarkable

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Use @benchmarkable to create a Benchmark instance without running it, facilitating explicit tune!/run workflows. It accepts keyword arguments similar to @benchmark, plus setup and teardown expressions.

```julia
using BenchmarkTools

# Define without running
b = @benchmarkable sort!(y) setup=(y = copy($(rand(1000)))) evals=1
```

```julia
# Tune to find the optimal evals/sample
tune!(b)
```

```julia
# Run and capture the full Trial
t = run(b)
```

```julia
# Override parameters at run-time
t2 = run(b, seconds=2, samples=200)
```

```julia
# Inspect the benchmark's parameters
params(b)
```

```julia
# Manually set parameters (bypasses tune!)
b2 = @benchmarkable sum($( rand(100) )) seconds=3 samples=1000 evals=50
run(b2)
```

--------------------------------

### Mutating External State in Julia Benchmarks

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Shows how to benchmark code that mutates external state. The `@benchmarkable` macro is used to create a benchmarkable object, which can then be run. The example demonstrates that repeated runs of the benchmarkable object modify the external array `A`.

```julia
julia> A = zeros(3)

 # each evaluation will modify A
julia> b = @benchmarkable fill!($A, rand())

julia> run(b, samples = 1)

julia> A
3-element Vector{Float64}:
 0.4615582142515109
 0.4615582142515109
 0.4615582142515109

julia> run(b, samples = 1)

julia> A
3-element Vector{Float64}:
 0.06373849439691504
 0.06373849439691504
 0.06373849439691504
```

--------------------------------

### Benchmark with CPU profiling using @bprofile

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Use @bprofile to run @benchmark with CPU profiling enabled during the timed execution phase. GC is disabled by default during profiling. Inspect results with Profile.print() or ProfileView.view().

```julia
using BenchmarkTools, Profile

# Profile a matrix factorization
@bprofile lu(rand(100, 100))

# View the profile flamegraph
Profile.print(format=:flat, sortedby=:count)

# Or use a graphical viewer
using ProfileView
ProfileView.view()

# Profile with explicit evals to control sample density
@bprofile sum($(rand(10000))) evals=100
```

--------------------------------

### Run Benchmarkable with Parameters

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Execute a pre-configured benchmarkable. This is equivalent to passing the parameters directly to the `run` function.

```julia
run(b) # equivalent to run(b, seconds = 1, time_tolerance = 0.01)
```

--------------------------------

### Run a full benchmark trial with @benchmark

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Use @benchmark for comprehensive benchmarking. It tunes evaluations, collects samples within a budget, and provides detailed statistical summaries. Interpolate external variables with $ to avoid benchmarking overhead.

```julia
using BenchmarkTools

# Basic usage
@benchmark sin(1)
```

```julia
# Use setup= to run per-sample initialization (excluded from timing)
@benchmark sort!(y) setup=(y = copy(x)) evals=1 seconds=2
```

```julia
# Interpolate globals to avoid benchmarking type dispatch overhead
A = rand(3, 3)
@benchmark inv($A)
```

```julia
# Interpolate a computed value (rand(3,3) is called once before benchmarking)
@benchmark inv($(rand(3, 3)))
```

```julia
# Prevent compiler hoisting of compile-time constants with Ref
a = 1; b = 2
@benchmark $(Ref(a))[] + $(Ref(b))[]
```

```julia
# Custom parameters: limit time budget and tighten noise tolerance
@benchmark eigen(rand(10, 10)) seconds=1 samples=500 time_tolerance=0.01
```

--------------------------------

### Return named tuple of timing statistics with @btimed

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Use @btimed to get a NamedTuple with performance metrics like time, bytes, allocations, and GC time. Useful for programmatic access to performance data.

```julia
using BenchmarkTools

result = @btimed sort($(rand(100)))
# (value = [...], time = 8.4e-8, bytes = 896, alloc = 1, gctime = 0.0)

result.time    # minimum elapsed time in seconds
result.bytes   # bytes allocated
result.alloc   # number of allocations
result.gctime  # GC time in seconds
result.value   # the expression's return value

# Use in scripts to record performance
data = @btimed lu($(rand(50, 50)))
@assert data.alloc < 20 "Too many allocations: $(data.alloc)"
```

--------------------------------

### Run Benchmark Suite

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Execute a benchmark suite using an old version of a package and save the results.

```julia
old_results = run(suite, verbose = true)
```

```julia
BenchmarkTools.save("old_results.json", old_results)
```

```julia
results = run(suite, verbose = true)
```

```julia
old_results = BenchmarkTools.load("old_results.json")
```

```julia
BenchmarkTools.judge(minimum(results), minimum(old_results))
```

--------------------------------

### Configure Benchmarkable with Parameters

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Define a benchmarkable with specific execution parameters. Use `seconds` to set a time budget and `time_tolerance` for result analysis.

```julia
b = @benchmarkable sin(1) seconds=1 time_tolerance=0.01
```

--------------------------------

### Return minimum elapsed time as Float64 with @belapsed

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Use @belapsed to get the minimum elapsed time in seconds as a Float64 scalar, suitable for programmatic comparisons or test assertions. It accepts all @benchmark keyword arguments.

```julia
using BenchmarkTools

t = @belapsed sin(1)
```

```julia
# Use in a conditional test
x = rand(100)
t_sort = @belapsed sort($x)
t_sort_bang = @belapsed sort!(y) setup=(y = copy($x)) evals=1
println("sort!  is ", t_sort / t_sort_bang, "× faster than sort")
```

```julia
# With custom time budget
t = @belapsed fft($(rand(ComplexF64, 256))) seconds=2
```

--------------------------------

### Run Benchmark and Inspect Trial

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Run a benchmark using the @benchmark macro and inspect the resulting Trial object to see detailed timing and memory information.

```julia
t = @benchmark eigen(rand(10, 10))
```

```julia
dump(t) # here's what's actually stored in a Trial
BenchmarkTools.Trial
  params: BenchmarkTools.Parameters
    seconds: Float64 5.0
    samples: Int64 10000
    evals: Int64 1
    overhead: Float64 0.0
    gctrial: Bool true
    gcsample: Bool false
    time_tolerance: Float64 0.05
    memory_tolerance: Float64 0.01
  times: Array{Float64}((10000,)) [26549.0, 26960.0, 27030.0, 27171.0, 27211.0, 27261.0, 27270.0, 27311.0, 27311.0, 27321.0  …  55383.0, 55934.0, 58649.0, 62847.0, 68547.0, 75761.0, 247081.0, 1.421718e6, 1.488322e6, 1.50329e6]
  gctimes: Array{Float64}((10000,)) [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.366184e6, 1.389518e6, 1.40116e6]
  memory: Int64 16752
  allocs: Int64 19
```

--------------------------------

### Comparing TrialEstimates with ratio

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Demonstrates comparing two TrialEstimate instances using the ratio function. This function compares fields like time, gctime, memory, and allocs.

```julia
julia> using BenchmarkTools

julia> b = @benchmarkable eigen(rand(10, 10));

julia> tune!(b);

julia> m1 = median(run(b))
BenchmarkTools.TrialEstimate:
  time:             38.638 μs
  gctime:           0.000 ns (0.00%)
  memory:           9.30 KiB
  allocs:           28

julia> m2 = median(run(b))
BenchmarkTools.TrialEstimate:
  time:             38.723 μs
  gctime:           0.000 ns (0.00%)
  memory:           9.30 KiB
  allocs:           28

julia> ratio(m1, m2)
BenchmarkTools.TrialRatio:
  time:             0.997792009916587
  gctime:           1.0
  memory:           1.0
  allocs:           1.0
```

--------------------------------

### Defining a BenchmarkGroup Suite

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Demonstrates how to define a parent BenchmarkGroup and add child groups with specific tags for organizing benchmark suites.

```julia
# Define a parent BenchmarkGroup to contain our suite
suite = BenchmarkGroup()

# Add some child groups to our benchmark suite. The most relevant BenchmarkGroup constructor
# for this case is BenchmarkGroup(tags::Vector). These tags are useful for
# filtering benchmarks by topic, which we'll cover in a later section.
suite["utf8"] = BenchmarkGroup(["string", "unicode"])
suite["trig"] = BenchmarkGroup(["math", "triangles"])

# Add some benchmarks to the "utf8" group
teststr = join(rand('a':'d', 10^4));
suite["utf8"]["replace"] = @benchmarkable replace($teststr, "a" => "b")
suite["utf8"]["join"] = @benchmarkable join($teststr, $teststr)
```

--------------------------------

### Analyze Trial Data in BenchmarkGroup

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Demonstrates how to access and analyze trial data stored within a `BenchmarkGroup` after running benchmarks. Functions like `median` and `judge` can be applied to these groups.

```julia
m1 = median(results["utf8"])

m2 = median(run(suite["utf8"]))

judge(m1, m2; time_tolerance = 0.001)
```

--------------------------------

### BenchmarkTools.load

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/reference.md

Loads benchmark results from a file.

```APIDOC
## BenchmarkTools.load

### Description
Loads benchmark results from a file.

### Method
(Not specified, likely a function call in Julia)

### Endpoint
(Not applicable for Julia functions)

### Parameters
(Specific parameters not detailed in the source)

### Request Example
(Not applicable for Julia functions)

### Response
(Not applicable for Julia functions)
```

--------------------------------

### Load Benchmark Parameters

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Load previously saved benchmark parameters into a suite for consistent configuration. Ensure the JSON file contains the parameters.

```julia
# syntax is loadparams!(group, paramsgroup, fields...)
julia> loadparams!(suite, BenchmarkTools.load("params.json")[1], :evals, :samples);
```

--------------------------------

### Tune and Run BenchmarkGroup

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Executes `tune!` on all benchmarks within a `BenchmarkGroup` and then runs them with a specified time limit. Returns a `BenchmarkGroup` containing trial data.

```julia
tune!(suite)

results = run(suite, verbose = true, seconds = 1)
```

--------------------------------

### Tune and Save Benchmark Parameters

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Tune a benchmark suite to configure parameters and save them using a JSON wrapper.

```julia
# untuned example suite
julia> suite
BenchmarkTools.BenchmarkGroup:
  tags: []
  "utf8" => BenchmarkGroup(["string", "unicode"])
  "trig" => BenchmarkGroup(["math", "triangles"])

# tune the suite to configure benchmark parameters
julia> tune!(suite);

# save the suite's parameters using a thin wrapper
# over JSON (this wrapper maintains compatibility
# across BenchmarkTools versions)
julia> BenchmarkTools.save("params.json", params(suite));
```

--------------------------------

### Explicit Benchmark Definition, Tuning, and Running

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Define a benchmarkable expression, automatically tune its parameters (evaluations per sample, number of samples), and then run the benchmark. This provides more control than @benchmark.

```julia
julia> b = @benchmarkable sin(1); # define the benchmark with default parameters

# find the right evals/sample and number of samples to take for this benchmark
julia> tune!(b);

julia> run(b)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.442 ns … 4.308 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.453 ns             ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.456 ns ± 0.056 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                  █                              
  ▂▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▃
  1.44 ns          Histogram: frequency by time           1.46 ns (top 1%)

 Memory estimate: 0 bytes, allocs estimate: 0.
```

--------------------------------

### @benchmark — Run a full benchmark trial

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

The primary macro for benchmarking an expression. It automatically tunes the number of evaluations per sample, collects samples within a budget, and displays a detailed statistical summary. Use $ interpolation to pass external variables as constants.

```APIDOC
## @benchmark — Run a full benchmark trial

### Description
The primary macro for benchmarking an expression. It automatically tunes the number of evaluations per sample, collects up to 10,000 samples within a 5-second budget, and displays a detailed statistical summary including min/max, median, mean ± σ, a histogram, and memory/allocation estimates. Use `$` interpolation to pass external variables as constants rather than globals.

### Usage
```julia
using BenchmarkTools

# Basic usage
@benchmark sin(1)

# Use setup= to run per-sample initialization (excluded from timing)
@benchmark sort!(y) setup=(y = copy(x)) evals=1 seconds=2

# Interpolate globals to avoid benchmarking type dispatch overhead
A = rand(3, 3)
@benchmark inv($A)

# Interpolate a computed value (rand(3,3) is called once before benchmarking)
@benchmark inv($(rand(3, 3)))

# Prevent compiler hoisting of compile-time constants with Ref
a = 1; b = 2
@benchmark $(Ref(a))[] + $(Ref(b))[]

# Custom parameters: limit time budget and tighten noise tolerance
@benchmark eigen(rand(10, 10)) seconds=1 samples=500 time_tolerance=0.01
```

### Parameters
- `seconds` (Real): Time budget for collecting samples.
- `samples` (Integer): Maximum number of samples to collect.
- `evals` (Integer): Number of evaluations per sample (tuned automatically by default).
- `time_tolerance` (Real): Tightens the noise tolerance for tuning.
- `setup` (Any): Expression to run once per sample before each evaluation (excluded from timing).
- `teardown` (Any): Expression to run once per sample after each evaluation (excluded from timing).
```

--------------------------------

### Benchmark Expression with Time Budget

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/README.md

Use the `@btime` macro to benchmark an expression. The `seconds` keyword argument sets a time budget for the benchmark.

```julia
julia> @btime sin(x) setup=(x=rand()) seconds=3
  4.361 ns (0 allocations: 0 bytes)
0.49587200950472454
```

--------------------------------

### Visualize Benchmark Trial with BenchmarkPlots

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Visualize benchmark trial results as a violin plot using BenchmarkPlots and StatsPlots. Supports standard Plots.jl keyword arguments for customization.

```julia
using BenchmarkPlots, StatsPlots
b = @benchmarkable lu(rand(10,10))
t = run(b)

plot(t)
```

--------------------------------

### Create a Processor Shield with cset

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/linuxtips.md

Activates a processor shield using 'cset' to isolate processors 1 and 3, and includes kernel threads. It shows the tasks moved and remaining.

```bash
➜ sudo cset shield -c 1,3 -k on
cset: --> activating shielding:
cset: moving 67 tasks from root into system cpuset...
[==================================================]%
cset: kthread shield activated, moving 91 tasks into system cpuset...
[==================================================]%
cset: **> 34 tasks are not movable, impossible to move
cset: "system" cpuset of CPUSPEC(0,2) with 124 tasks running
cset: "user" cpuset of CPUSPEC(1,3) with 0 tasks running
```

--------------------------------

### Compare benchmark results with ratio and judge

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Use ratio to compute field-wise ratios between two TrialEstimates, returning a TrialRatio. Use judge to classify these ratios as :regression, :improvement, or :invariant based on tolerance thresholds.

```julia
using BenchmarkTools

b = @benchmarkable eigen(rand(10, 10))
tune!(b)

m1 = median(run(b))
m2 = median(run(b))
```

--------------------------------

### Benchmark with Timing and Allocation Information

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Use @btime to benchmark an expression and display its execution time and memory allocations. This is similar to Julia's built-in @time macro.

```julia
julia> @btime sin(1)
  13.612 ns (0 allocations: 0 bytes)
0.8414709848078965
```

--------------------------------

### BenchmarkTools.save

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/reference.md

Saves benchmark results to a file.

```APIDOC
## BenchmarkTools.save

### Description
Saves benchmark results to a file.

### Method
(Not specified, likely a function call in Julia)

### Endpoint
(Not applicable for Julia functions)

### Parameters
(Specific parameters not detailed in the source)

### Request Example
(Not applicable for Julia functions)

### Response
(Not applicable for Julia functions)
```

--------------------------------

### Benchmark Simple Addition (Optimized)

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

This benchmark measures the time to add two integers. Due to compile-time evaluation, Julia optimizes this to a constant, resulting in a very low benchmark time.

```julia
julia> a = 1; b = 2
2

julia> @btime $a + $b
  0.024 ns (0 allocations: 0 bytes)
3
```

--------------------------------

### Create a Tagged BenchmarkGroup

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Constructs a nested BenchmarkGroup with various tags applied to child groups. This demonstrates the structure for tag-based filtering.

```julia
g = BenchmarkGroup([], # no tags in the parent
                          "c" => BenchmarkGroup(["5", "6", "7"]),
                          "b" => BenchmarkGroup(["3", "4", "5"]),
                          "a" => BenchmarkGroup(["1", "2", "3"],  # contains tags and child groups
                                                "d" => BenchmarkGroup(["8"], 1 => 1),
                                                "e" => BenchmarkGroup(["9"], 2 => 2)));
```

--------------------------------

### Calculate Trial Estimates

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Calculate various statistical estimates (minimum, maximum, median, mean, std) from a Trial object to understand benchmark performance.

```julia
minimum(t)
BenchmarkTools.TrialEstimate: 
  time:             26.549 μs
  gctime:           0.000 ns (0.00%)
  memory:           16.36 KiB
  allocs:           19
```

```julia
maximum(t)
BenchmarkTools.TrialEstimate: 
  time:             1.503 ms
  gctime:           1.401 ms (93.21%)
  memory:           16.36 KiB
  allocs:           19
```

```julia
median(t)
BenchmarkTools.TrialEstimate: 
  time:             30.818 μs
  gctime:           0.000 ns (0.00%)
  memory:           16.36 KiB
  allocs:           19
```

```julia
mean(t)
BenchmarkTools.TrialEstimate: 
  time:             31.777 μs
  gctime:           415.686 ns (1.31%)
  memory:           16.36 KiB
  allocs:           19
```

```julia
std(t)
BenchmarkTools.TrialEstimate: 
  time:             25.161 μs
  gctime:           23.999 μs (95.38%)
  memory:           16.36 KiB
  allocs:           19
```

--------------------------------

### Create Nested BenchmarkGroup

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Creates a nested BenchmarkGroup by indexing keys, automatically creating intermediate groups as needed. This simplifies hierarchical benchmark organization.

```julia
suite2 = BenchmarkGroup()

suite2["my"]["nested"]["benchmark"] = @benchmarkable sum(randn(32))
```

--------------------------------

### Quick Benchmarking with @btime

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/README.md

The @btime macro is a convenience wrapper around @benchmark, providing output similar to Julia's built-in @time macro for quick sanity checks.

```julia
julia> using BenchmarkTools

# The `setup` expression is run once per sample, and is not included in the
# timing results. Note that each sample can require multiple evaluations
# benchmark kernel evaluations. See the BenchmarkTools manual for details.
julia> @btime sort(data) setup=(data=rand(10))
  83.850 ns (1 allocation: 160 bytes)

```

--------------------------------

### View CPU Thread Siblings

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/linuxtips.md

Examine the `/sys/devices/system/cpu/cpu*/topology/thread_siblings_list` file to identify which logical processors share a physical core. This is crucial for understanding hyperthreading configuration.

```bash
➜ cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list
0,4
1,5
2,6
3,7
0,4
1,5
2,6
3,7
```

--------------------------------

### Judge benchmark results with default tolerances

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Compares two benchmark results using default tolerances (5% time, 1% memory). Indicates if the new result is an improvement, regression, or invariant.

```julia
j = judge(m1, m2)
```

--------------------------------

### Save and load benchmark results

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Persists benchmark results to a JSON file and loads them back. This allows for cross-session comparison and analysis.

```julia
using BenchmarkTools

suite = BenchmarkGroup()
suite["sort"] = @benchmarkable sort($(rand(1000)))
suite["sum"]  = @benchmarkable sum($(rand(1000)))
tune!(suite)

# --- Session 1: baseline ---
old_results = run(suite)
BenchmarkTools.save("baseline.json", old_results)

# Cache tuned parameters to reuse next session
BenchmarkTools.save("params.json", params(suite))
```

```julia
# --- Session 2: after a code change ---
# Reload parameters (avoids re-tuning, ensures same evals)
loadparams!(suite, BenchmarkTools.load("params.json")[1], :evals, :samples)
new_results = run(suite)

# Load the baseline and compare
old = BenchmarkTools.load("baseline.json")[1]
judgements = judge(minimum(new_results), minimum(old))
```

```julia
# Real-world caching pattern (from benchmark/benchmarks.jl)
paramspath = joinpath(@__DIR__, "params.json")
if isfile(paramspath)
    loadparams!(suite, BenchmarkTools.load(paramspath)[1], :evals)
else
    tune!(suite)
    BenchmarkTools.save(paramspath, params(suite))
end
```

--------------------------------

### Benchmark Julia Expression with @benchmark

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Quickly benchmark a Julia expression. This macro automatically configures, tunes, and runs the benchmark, providing a detailed trial result.

```julia
julia> @benchmark sin(1)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.442 ns … 53.028 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.453 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.462 ns ±  0.566 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                   █                              
  ▂▁▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▃
  1.44 ns           Histogram: frequency by time           1.46 ns (top 1%)

 Memory estimate: 0 bytes, allocs estimate: 0.
```

--------------------------------

### Auto-configure benchmark evaluation counts with tune!

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Call tune! on a Benchmark or BenchmarkGroup to automatically set the optimal number of evaluations per sample. This helps overcome timer resolution noise for fast operations.

```julia
using BenchmarkTools

b = @benchmarkable sin(1)
tune!(b)
params(b).evals  # e.g. 1000

# Tune an entire suite
suite = BenchmarkGroup()
suite["trig"] = BenchmarkGroup()
suite["trig"]["sin"] = @benchmarkable sin($(rand()))
suite["trig"]["cos"] = @benchmarkable cos($(rand()))
tune!(suite, verbose=true)

# Run after tuning
results = run(suite)

# Skip tuning for mutations where evals must be 1
b_mut = @benchmarkable fill!($( zeros(100) ), rand()) evals=1
run(b_mut)  # tune! is a no-op when evals is manually set
```

--------------------------------

### Visualize BenchmarkGroup with BenchmarkPlots

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Visualize results of a BenchmarkGroup containing only Trials by plotting each Trial as a violin plot.

```julia
using BenchmarkPlots, StatsPlots
t = run(g)
plot(t)
```

--------------------------------

### Benchmark with Detailed Timing Information

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Use @btimed to benchmark an expression and return a tuple containing the value, time, bytes allocated, allocations, and GC time. This is similar to Julia's built-in @timed macro.

```julia
julia> @btimed sin(1)
(value = 0.8414709848078965, time = 9.16e-10, bytes = 0, alloc = 0, gctime = 0.0)
```

--------------------------------

### Index BenchmarkGroup with another BenchmarkGroup

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Index into a BenchmarkGroup using the keys from another BenchmarkGroup. This is useful for selecting subsets of benchmarks based on external criteria.

```julia
julia> g # leaf values are integers
BenchmarkTools.BenchmarkGroup:
  tags: []
  "c" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => 1
	  "2" => 2
	  "3" => 3
  "b" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => 1
	  "2" => 2
	  "3" => 3
  "a" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => 1
	  "2" => 2
	  "3" => 3
  "d" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => 1
	  "2" => 2
	  "3" => 3

julia> x # note that leaf values are characters
BenchmarkTools.BenchmarkGroup:
  tags: []
  "c" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "2" => '2'
  "a" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => '1'
	  "3" => '3'
  "d" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => '1'
	  "2" => '2'
	  "3" => '3'

julia> g[x] # index into `g` with the keys of `x`
BenchmarkTools.BenchmarkGroup:
  tags: []
  "c" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "2" => 2
  "a" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => 1
	  "3" => 3
  "d" => BenchmarkTools.BenchmarkGroup:
	  tags: []
	  "1" => 1
	  "2" => 2
	  "3" => 3
```

--------------------------------

### Benchmark Global vs. Interpolated Variable in Julia

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/manual.md

Demonstrates the difference in benchmarking when a global variable is used directly versus when it's interpolated with '$'. Interpolation prevents the variable lookup from being part of the benchmarked code.

```julia
julia> @benchmark [i*i for i in A]
BenchmarkTools.Trial: 10000 samples with 54 evaluations.
 Range (min … max):  889.241 ns … 29.584 μs  ┊ GC (min … max):  0.00% … 93.33%
 Time  (median):       1.073 μs              ┊ GC (median):     0.00%
 Time  (mean ± σ):     1.296 μs ±  2.004 μs  ┊ GC (mean ± σ):  14.31% ±  8.76%

      ▃█▆                                                           
  ▂▂▄▆███▇▄▄▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▁▂▂▂▁▂▂▁▁▁▁▁▂▁▁▁▁▂▂▁▁▁▁▂▁▁▁▁▁▁▂▂▂▂▂▂▂▂▂▂
  889 ns             Histogram: frequency by time            2.92 μs (top 1%)

 Memory estimate: 7.95 KiB, allocs estimate: 2.
```

```julia
julia> @benchmark [i*i for i in $A]
BenchmarkTools.Trial: 10000 samples with 121 evaluations.
 Range (min … max):  742.455 ns … 11.846 μs  ┊ GC (min … max):  0.00% … 88.05%
 Time  (median):     909.959 ns              ┊ GC (median):     0.00%
 Time  (mean ± σ):     1.135 μs ±  1.366 μs  ┊ GC (mean ± σ):  16.94% ± 12.58%

  ▇█▅▂                                                             ▁
  ████▇▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▅▆██
  742 ns          Histogram: log(frequency) by time          10.3 μs (top 1%)

 Memory estimate: 7.94 KiB, allocs estimate: 1.
```

--------------------------------

### Load cached benchmark parameters

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Restores benchmark parameters, such as evaluation counts and samples, from a JSON file. This avoids re-tuning and ensures reproducible trial conditions.

```julia
using BenchmarkTools

b = @benchmarkable fft($(rand(ComplexF64, 512)))
tune!(b)
println(params(b).evals)  # e.g. 8

# Save parameters
BenchmarkTools.save("fft_params.json", params(b))

# In a new session: restore without re-tuning
b2 = @benchmarkable fft($(rand(ComplexF64, 512)))
loadparams!(b2, BenchmarkTools.load("fft_params.json")[1], :evals, :samples)
println(params(b2).evals)  # same value as before

# For an entire suite
suite = BenchmarkGroup()
suite["a"] = @benchmarkable sin($(rand()))
loadparams!(suite, BenchmarkTools.load("suite_params.json")[1], :evals)
```

```julia
# Change global defaults for all future benchmarks
BenchmarkTools.DEFAULT_PARAMETERS.seconds = 2.5
BenchmarkTools.DEFAULT_PARAMETERS.time_tolerance = 0.10
```

--------------------------------

### Quick Performance Check with @btime

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/index.md

The @btime macro is a convenience wrapper around @benchmark for quick sanity checks. Its output is similar to Julia's built-in @time macro, showing execution time and memory allocations.

```julia
@btime sin(x) setup=(x=rand())
```

--------------------------------

### Organize hierarchical benchmark suites with BenchmarkGroup

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Use BenchmarkGroup as a dictionary-like container to structure benchmark definitions, results, and nested groups. Supports tagged filtering, indexing, and bulk operations.

```julia
using BenchmarkTools

# Build a nested suite
suite = BenchmarkGroup()
suite["utf8"] = BenchmarkGroup(["string", "unicode"])
suite["trig"] = BenchmarkGroup(["math"])

teststr = join(rand('a':'d', 10^4))
suite["utf8"]["replace"] = @benchmarkable replace($teststr, "a" => "b")
suite["utf8"]["join"]    = @benchmarkable join($teststr, $teststr)
for f in (sin, cos, tan)
    suite["trig"][string(f)] = @benchmarkable $(f)($(rand()))
end

# Auto-create nested groups by deep indexing
suite["linalg"]["dot"]["float64"] = @benchmarkable dot($(rand(100)), $(rand(100)))

# Tune and run
tune!(suite)
results = run(suite, verbose=true, seconds=1)
# (1/2) benchmarking "utf8"...
# (2/2) benchmarking "trig"...

# Statistical summaries over the whole group
m = median(results)
m["utf8"]["replace"]  # TrialEstimate(203 μs)

# Filter by tag
results[@tagged "math"]
results[@tagged "string" && !"unicode"]  # string but not unicode

# Vector indexing into nested groups
suite[["utf8", "replace"]]

# Iterate leaves
for (keys, bench) in leaves(suite)
    println(keys, " => ", bench)
end
```

--------------------------------

### Set CPU Governor to Performance

Source: https://github.com/juliaci/benchmarktools.jl/blob/main/docs/src/linuxtips.md

Forces all CPU cores to run at their maximum frequency by setting the 'performance' governor. This prevents dynamic frequency scaling from interfering with benchmarks.

```bash
echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
```

--------------------------------

### Visualize benchmark results with BenchmarkPlots

Source: https://context7.com/juliaci/benchmarktools.jl/llms.txt

Generates violin plots for benchmark results using BenchmarkPlots and StatsPlots. Allows for customization of plot appearance and REPL histogram alignment.

```julia
# Install companion packages
# pkg> add BenchmarkPlots StatsPlots

using BenchmarkTools, BenchmarkPlots, StatsPlots

# Plot a single Trial as a violin plot
b = @benchmarkable lu(rand(10, 10))
t = run(b)
plot(t)  # displays violin plot of timing samples

# Plot a BenchmarkGroup (one violin per benchmark)
suite = BenchmarkGroup()
suite["lu"]  = @benchmarkable lu(rand(10, 10))
suite["qr"]  = @benchmarkable qr(rand(10, 10))
suite["svd"] = @benchmarkable svd(rand(10, 10))
tune!(suite)
results = run(suite)
plot(results)  # side-by-side violin plots
```

```julia
# Align REPL histograms for manual comparison
io = IOContext(stdout, :histmin=>1, :histmax=>30, :logbins=>true)
show(io, MIME("text/plain"), @benchmark x^3   setup=(x=rand()))
show(io, MIME("text/plain"), @benchmark x^3.0 setup=(x=rand()))
```

```julia
# Customize the violin plot
plot(t, st=:box, yaxis=:log10, title="LU Factorization Timings")
```