### Quick Start Benchmarking

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

A basic example to get started with Criterium. It requires importing the 'criterium.bench' namespace and then calling the 'bench' macro with the expression to be benchmarked.

```clojure
(require '[criterium.bench :as bench])

(bench/bench (+ 1 1))
```

--------------------------------

### Example: Spawning a Clojure Subprocess with Agent

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Demonstrates how to spawn a Clojure subprocess and pass the necessary agent JVM options to it. This example executes a simple addition operation in the subprocess.

```clojure
(require '[clojure.java.shell :as shell])
(let [opts (agent/jvm-opts)]
  (apply shell/sh "clojure" (concat opts ["-e" "(+ 1 2)"])))
```

--------------------------------

### Basic Domain Analysis Setup

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Initializes domain analysis with required namespaces and benchmarks sorting across a range of input sizes.

```clojure
(require '[criterium.domain :as domain]
         '[criterium.domain.builder :as builder]
         '[criterium.domain-plans :as domain-plans])

;; Benchmark sorting across input sizes
(domain/bench
 (domain/domain-expr
  [n (builder/log-range 10 1000 5)]
  (sort (vec (range n)))))
```

--------------------------------

### Using a Specific Bench Plan

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Shows how to specify a particular bench plan for analysis. This example uses the 'distribution-analysis' plan.

```clojure
(bench/bench (sort data) :bench-plan criterium.bench-plans/distribution-analysis)
```

--------------------------------

### Install R on Ubuntu/Debian

Source: https://github.com/hugoduncan/criterium/blob/develop/bases/criterium/validation/README.md

Install R on Ubuntu/Debian systems using apt-get. This is a prerequisite for running validation tests.

```bash
sudo apt-get install r-base
```

--------------------------------

### Basic CMake Setup

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Sets the minimum CMake version, enables compile command export, and defines the C++ standard to C++17.

```cmake
cmake_minimum_required(VERSION 3.14)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
```

--------------------------------

### Start Rserve from Command Line

Source: https://github.com/hugoduncan/criterium/blob/develop/bases/criterium/validation/README.md

Start the Rserve background process from the command line. This is an alternative to starting it within an R session.

```bash
R -e "library(Rserve); Rserve()"
```

--------------------------------

### Benchmarking with Progress Reporting

Source: https://github.com/hugoduncan/criterium/blob/develop/README.md

Wrap benchmarking calls with `with-progress-reporting` to see progress information. This example uses `bench` and `quick-bench` with verbose output.

```clojure
(with-progress-reporting (bench (Thread/sleep 1000) :verbose))
(with-progress-reporting (quick-bench (Thread/sleep 1000) :verbose))
```

--------------------------------

### Changing Output Format

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Demonstrates how to change the output format of the benchmark results using the :viewer option. This example sets the viewer to :pprint.

```clojure
(bench/bench (sort data) :viewer :pprint)
```

--------------------------------

### Install R on macOS

Source: https://github.com/hugoduncan/criterium/blob/develop/bases/criterium/validation/README.md

Install R on macOS using Homebrew. This is a prerequisite for running validation tests.

```bash
brew install r
```

--------------------------------

### Benchmark with Portal Viewer and Histogram Plan

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Benchmark an expression using Criterium with the Portal viewer and a histogram for the benchmark plan. Requires Portal to be installed.

```clojure
(require 'criterium.viewer.portal)
(require '[criterium.bench :as bench])
(bench/bench (criterium.jvm/wait 10000)
  :viewer :portal
  :bench-plan criterium.bench-plans/histogram)
```

--------------------------------

### Basic Benchmarking with `bench`

Source: https://github.com/hugoduncan/criterium/blob/develop/README.md

Use the `bench` function for a simple way to measure the execution time of an expression. This example benchmarks a 1-second sleep.

```clojure
(use 'criterium.core)

(bench (Thread/sleep 1000))
```

--------------------------------

### Install Criterium with Deps.edn

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Add Criterium as a dependency alias in your deps.edn file for benchmarking. Includes JVM options for dead-code elimination.

```clojure
{:aliases
 {:bench {:extra-deps {org.hugoduncan/criterium {:mvn/version "0.5.153-ALPHA"}}
          ;; JDK 17+ options for optimal dead-code elimination
          :jvm-opts ["-XX:+UnlockExperimentalVMOptions"
                     "-XX:CompileCommand=blackhole,criterium.blackhole.Blackhole::consume"]}}}
```

--------------------------------

### Load Criterium Agent at JVM Startup (macOS)

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Start a Clojure REPL with the Criterium agent loaded using the -agentpath JVM option. This is the recommended method for enabling allocation tracking.

```bash
clojure -J-agentpath:/path/to/libcriterium.dylib -M:dev
```

--------------------------------

### Accessing Benchmark Results Programmatically

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Provides an example of how to access the full benchmark results programmatically after a benchmark has been run. It also shows how to extract specific statistical values.

```clojure
(bench/bench (reduce + (range 100)))

;; Get full results
(bench/last-bench)

;; Extract specific values
(require '[criterium.util.helpers :as util])
(util/stats-value (:data (bench/last-bench)) :stats :elapsed-time :mean)
```

--------------------------------

### Start REPL with Custom-Built Agent

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Load a custom-built Criterium agent using the -agentpath JVM argument. This is an advanced option for development or when using non-bundled agent binaries.

```bash
# Start REPL with custom-built agent
clojure -J-agentpath:/path/to/custom/libcriterium.so -M:dev
```

--------------------------------

### Benchmarking with Local Bindings

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Shows how the 'bench' macro captures local bindings from its enclosing scope. This example benchmarks the reduction of a vector.

```clojure
(let [data (vec (range 1000))]
  (bench/bench (reduce + data)))
```

--------------------------------

### Start Rserve in R

Source: https://github.com/hugoduncan/criterium/blob/develop/bases/criterium/validation/README.md

Start the Rserve background process from within an R session. Rserve defaults to port 6311.

```r
library(Rserve)
Rserve()
```

--------------------------------

### Install Criterium and Arg-Gen with Deps.edn

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Add Criterium and its argument generation library as dependencies in deps.edn. Includes JVM options for dead-code elimination.

```clojure
{:aliases
 {:bench {:extra-deps {org.hugoduncan/criterium {:mvn/version "0.5.153-ALPHA"}
                       org.hugoduncan/criterium.arg-gen {:mvn/version "0.5.153-ALPHA"}}
          :jvm-opts ["-XX:+UnlockExperimentalVMOptions"
                     "-XX:CompileCommand=blackhole,criterium.blackhole.Blackhole::consume"]}}}
```

--------------------------------

### Benchmark with Generated Arguments (Basic)

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Benchmark a function using `bench/bench-measured` and `arg-gen/measured` with automatically generated arguments. This example uses two large integers for addition.

```clojure
(require '[criterium.arg-gen :as arg-gen])
(require '[clojure.test.check.generators :as gen])
(require '[criterium.bench :as bench])

;; Benchmark with generated arguments
(bench/bench-measured
  (arg-gen/measured
    [a gen/large-integer
     b gen/large-integer]
    (+ a b)))
```

--------------------------------

### Time an Expression with Criterium

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Use the `bench/bench` function to time a Clojure expression. This example times a 10 microsecond busy wait.

```clojure
(require '[criterium.bench :as bench])
(bench/bench (criterium.jvm/wait 10000)) ; 10us busy wait
```

--------------------------------

### Install Required R Packages for Validation

Source: https://github.com/hugoduncan/criterium/blob/develop/bases/criterium/validation/README.md

Install additional R packages like 'multimode' which are necessary for specific validation tests, such as silverman-test and ACR test validation.

```r
# For silverman-test and ACR test validation
install.packages("multimode")
```

--------------------------------

### Use Local Agent Build on Linux

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Start a Clojure development environment using a locally built Criterium agent on Linux. This alias configures the -agentpath to point to the compiled local binary.

```bash
# Linux  
clojure -M:dev:with-agent-linux
```

--------------------------------

### Install Rserve Package in R

Source: https://github.com/hugoduncan/criterium/blob/develop/bases/criterium/validation/README.md

Install the Rserve package within an R session. This package is required for remote connections to R.

```r
install.packages("Rserve", repos = "http://rforge.net")
```

--------------------------------

### Get Native Agent Path

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Command to find the path to the Criterium native agent, which is required for detailed allocation tracking.

```bash
# Get the agent path
clojure -Sdeps '{:deps {org.hugoduncan/criterium {:mvn/version "0.5.153-ALPHA"}}}'
  -e '(require '"'"'[criterium.agent :as agent]) (println (first (agent/jvm-opts)))'
```

--------------------------------

### Limiting Benchmark Duration

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Illustrates how to limit the total duration of a benchmark run. This example sets the limit to 5 seconds using the :limit-time-s option.

```clojure
(bench/bench (sort data) :limit-time-s 5)
```

--------------------------------

### Common Argument Generation Patterns

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Provides examples of generating arguments for common tasks like string processing, collection operations, and map operations.

```clojure
;; String processing
(arg-gen/measured
 [s gen/string-alphanumeric]
 (clojure.string/upper-case s))

;; Collection operations
(arg-gen/measured {:size 100}
 [v (gen/vector gen/small-integer)]
 (sort v))

;; Map operations
(arg-gen/measured {:size 20}
 [m (gen/map gen/keyword gen/small-integer)]
 (vals m))
```

--------------------------------

### Check Agent Loading Status

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Start a Clojure REPL and check if the C++ agent has been successfully loaded by Criterium. This is a targeted test for agent functionality.

```clojure
(require 'criterium.agent)
(criterium.agent/loaded?)
```

--------------------------------

### Benchmark with Generated Arguments (Configurable)

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Benchmark a function with generated arguments, specifying options for size and seed for reproducibility. This example generates a vector of small integers for summation.

```clojure
;; With options for size and reproducibility
(bench/bench-measured
  (arg-gen/measured {:size 50 :seed 12345}
    [coll (gen/vector gen/small-integer)]
    (reduce + coll)))
```

--------------------------------

### Use Local Agent Build on macOS

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Start a Clojure development environment using a locally built Criterium agent on macOS. This alias configures the -agentpath to point to the compiled local binary.

```bash
# macOS
clojure -M:dev:with-agent-mac
```

--------------------------------

### Get Agent Path for JVM Startup

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Retrieve the agent path from Criterium's dependencies to use with the -agentpath JVM option. This is useful for automating the agent loading process.

```bash
clojure -Sdeps '{:deps {org.hugoduncan/criterium {:mvn/version "0.5.x"}}}' \
  -e '(require '"'"'[criterium.agent :as agent]) (println (first (agent/jvm-opts)))'
```

--------------------------------

### Basic Usage of :bench-measured with Argument Generation

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Measures a function using generated arguments, starting with a small integer for 'n'. Requires the 'criterium.arg-gen' artifact.

```clojure
(require '[criterium.arg-gen :as arg-gen]
         '[clojure.test.check.generators :as gen])

;; Basic usage - each iteration gets fresh generated values
(bench/bench-measured
 (bench/options->bench-plan)
 (arg-gen/measured
  [n gen/small-integer]
  (* n n)))
```

--------------------------------

### Check if Criterium Agent is Loaded

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Verify whether the Criterium agent is currently loaded and active in the JVM. Returns true if the agent was started with the -agentpath option, otherwise false.

```clojure
;; Check if agent is loaded
(agent/loaded?)
;; => true (if started with -agentpath) or false
```

--------------------------------

### Basic Benchmarking with :pprint Viewer

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Demonstrates a simple benchmark using the :pprint viewer for structured output.

```clojure
(bench/bench (+ 1 1) :viewer :pprint)
```

--------------------------------

### Interactive Benchmarking with :portal Viewer

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Sets up and uses the :portal viewer for interactive analysis of benchmark results. Requires Portal to be connected to tap>.

```clojure
;; Setup: connect Portal to tap>
(require '[portal.api :as p])
(def portal (p/open))
(add-tap #'p/submit)

;; Use portal viewer
(bench/bench (+ 1 1) :viewer :portal)
```

--------------------------------

### Setting and Checking Default Viewer

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Shows how to set a default viewer for all subsequent benchmark calls and how to check the currently set default viewer.

```clojure
;; Set for all subsequent bench calls
(bench/set-default-viewer! :kindly)

;; Check current default
(bench/default-viewer)
```

--------------------------------

### Using the Histogram Bench Plan

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Demonstrates how to use the 'histogram' bench plan for non-parametric distribution analysis. This plan includes histogram visualization, KDE density estimation, and mode detection.

```clojure
(require '[criterium.bench-plans :as plans])

(bench/bench (my-function)
             :bench-plan plans/histogram)
```

--------------------------------

### Using the Distribution Analysis Bench Plan

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Shows how to use the 'distribution-analysis' bench plan for parametric distribution fitting. This plan adds distribution fitting, shape statistics, goodness-of-fit tests, and Q-Q plots.

```clojure
(bench/bench (my-function)
             :bench-plan plans/distribution-analysis)
```

--------------------------------

### Domain Analysis with Custom Options

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Demonstrates how to customize domain analysis benchmarks with options like disabling the progress reporter and setting a time limit per benchmark.

```clojure
(domain/bench
 (domain/domain-expr ...)
 :domain-plan domain-plans/complexity-analysis
 :reporter nil                        ; Silent (no progress dots)
 :bench-options {:limit-time-s 2})    ; Per-benchmark time limit
```

--------------------------------

### Default Viewer (:print)

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Illustrates the default output viewer, which is :print. This viewer provides human-readable text output to standard output.

```clojure
(bench/bench (+ 1 1))  ; uses :print
```

--------------------------------

### Lower-Level Benchmarking Functions

Source: https://github.com/hugoduncan/criterium/blob/develop/README.md

Utilize `benchmark` and `quick-benchmark` for more control, separating statistic generation from reporting. Results are returned to prevent JIT optimization issues.

```clojure
(report-result (benchmark (Thread/sleep 1000) {:verbose true}))
(report-result (quick-benchmark (Thread/sleep 1000)))
```

--------------------------------

### Argument Generation with Size and Seed Options

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Shows how to control the size parameter for generators (e.g., gen/vector) and how to ensure reproducible generation using a seed.

```clojure
;; Control generator size (affects sized generators like gen/vector)
(arg-gen/measured {:size 50}
 [coll (gen/vector gen/small-integer)]
 (sort coll))

;; Reproducible generation with seed
(arg-gen/measured {:seed 12345}
 [n gen/small-integer]
 (* n n))
```

--------------------------------

### Benchmark with Allocation Tracking

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Benchmark an expression using `bench/bench` after loading the native agent. The results will now include allocation tracking data.

```clojure
;; Benchmarks now include allocation tracking
(require '[criterium.bench :as bench])
(bench/bench (reduce + (range 1000)))
```

--------------------------------

### Create Agent Resource Directories

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Create the necessary directory structure for storing agent binaries and their hash files within the Criterium project's resource paths. This follows the convention bases/agent/resources/native/{platform}/.

```bash
mkdir -p bases/agent/resources/native/linux-x64
mkdir -p bases/agent/resources/native/macos-x64
mkdir -p bases/agent/resources/native/macos-arm64
```

--------------------------------

### Load Local Agent Build for Linux

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Configure the JVM to load the locally built agent for Linux. This uses a Clojure alias to point to the local build directory.

```bash
clojure -M:dev:with-agent-linux
```

--------------------------------

### Build Agent Locally

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Build the agent locally from the agent-cpp directory. Use this for development with local agent modifications.

```bash
cd agent-cpp
cmake -B build
cmake --build build
```

--------------------------------

### Fetching and Integrating GoogleTest

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Fetches the GoogleTest framework using FetchContent and makes it available for use in the project. It also prevents GoogleTest from overriding compiler/linker options.

```cmake
# Testing with GoogleTest
if(BUILD_TESTS)
    include(FetchContent)
    FetchContent_Declare(
        googletest
        GIT_REPOSITORY https://github.com/google/googletest.git
        GIT_TAG v1.14.0
    )
    # Prevent GoogleTest from overriding compiler/linker options
    set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
    FetchContent_MakeAvailable(googletest)

    enable_testing()
    add_subdirectory(test)
endif()
```

--------------------------------

### Agent binary resource paths

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Lists the paths within the JAR where agent binaries are stored for different platforms.

```text
resources/native/linux-x64/libcriterium.so
resources/native/macos-x64/libcriterium.dylib
resources/native/macos-arm64/libcriterium.dylib
```

--------------------------------

### Collecting Allocation Data

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Demonstrates how to collect allocation data during a benchmark. This requires a native agent and is enabled using the :with-allocation-trace option.

```clojure
(bench/bench (sort data) :with-allocation-trace true)
```

--------------------------------

### Load Local Agent Build for macOS

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Configure the JVM to load the locally built agent for macOS. This uses a Clojure alias to point to the local build directory.

```bash
clojure -M:dev:with-agent-mac
```

--------------------------------

### Comparing Implementations in Domain Analysis

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Compares the performance of different implementations (e.g., sort vs. sort-by) within a domain analysis benchmark. Uses a specific domain plan for implementation comparison.

```clojure
(domain/bench
 (domain/domain-expr
  [n (builder/log-range 100 10000 5)]
  {:sort    (sort (vec (range n)))
   :sort-by (sort-by identity (vec (range n)))})
 :domain-plan domain-plans/implementation-comparison)
```

--------------------------------

### Benchmarking with :kindly Viewer for Notebooks

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Configures Criterium to use the :kindly viewer, suitable for rendering tables and charts in Clay/Clerk notebooks.

```clojure
(bench/set-default-viewer! :kindly)
(bench/bench (+ 1 1))
```

--------------------------------

### Verify Native Agent Loading

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

In your REPL, call `agent/loaded?` to confirm that the Criterium native agent has been successfully loaded.

```clojure
(require '[criterium.agent :as agent])

;; Verify agent is loaded
(agent/loaded?)
;; => true
```

--------------------------------

### Download macOS Agent Artifacts

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Download the agent-cpp-macos-x64 and agent-cpp-macos-arm64 artifacts from a specific GitHub Actions workflow run. These artifacts contain the macOS agent binaries and their SHA256 hashes for both Intel and Apple Silicon architectures.

```bash
gh run download <run-id> -n agent-cpp-macos-x64
gh run download <run-id> -n agent-cpp-macos-arm64
```

--------------------------------

### Copy Downloaded Agent Binaries to Resources

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Copy the downloaded agent binaries and their corresponding SHA256 hash files from the CI artifact directories to their designated resource paths within the Criterium project. This ensures the agent is correctly placed for runtime loading.

```bash
cp agent-cpp-linux-x64/libcriterium.* bases/agent/resources/native/linux-x64/
cp agent-cpp-macos-x64/libcriterium.* bases/agent/resources/native/macos-x64/
cp agent-cpp-macos-arm64/libcriterium.* bases/agent/resources/native/macos-arm64/
```

--------------------------------

### Run integration tests

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Execute integration tests that specifically require the agent binary. This command focuses on tests with agent dependencies.

```shell
clojure -M:kaocha:dev:test --focus :requires-agent
```

--------------------------------

### Agent binary SHA256 hash file paths

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Lists the paths to SHA256 hash files for agent binaries, used for version tracking.

```text
resources/native/linux-x64/libcriterium.so.sha256
resources/native/macos-x64/libcriterium.dylib.sha256
resources/native/macos-arm64/libcriterium.dylib.sha256
```

--------------------------------

### Restart JVM with Native Agent

Source: https://github.com/hugoduncan/criterium/blob/develop/README.ALPHA.md

Restart your Clojure process with the Criterium native agent loaded using the path obtained previously. This enables allocation tracking.

```bash
# Restart with the agent (use the path from above)
clojure -J-agentpath:/tmp/criterium-agent-macos-x64-abc123.dylib -M:dev
```

--------------------------------

### Configure JVM Options for Agent Loading

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Obtain the necessary JVM arguments to restart your application with the Criterium agent loaded. This allows Criterium to manage the agent path configuration.

```clojure
(require '[criterium.agent :as agent])

;; Get JVM arguments for restarting with agent
(agent/jvm-opts)
;; => ["-agentpath:/tmp/criterium-agent-macos-x64-abc123.dylib"]
```

--------------------------------

### Compare Implementations using Domain Analysis

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Compare the performance of different implementations of a function using Criterium's domain analysis. This plan is suitable for analyzing how different approaches scale.

```clojure
(require '[criterium.domain :as domain]
         '[criterium.domain.builder :as builder]
         '[criterium.domain-plans :as domain-plans])

(domain/bench
 (domain/domain-expr
  [n (builder/log-range 100 10000 5)]
  {:impl-a (sort (vec (range n)))
   :impl-b (sort-by identity (vec (range n)))})
 :domain-plan domain-plans/implementation-comparison)
```

--------------------------------

### Agent Behavior on Unsupported Platforms

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Illustrates the expected behavior of Criterium functions when the agent is unavailable due to an unsupported platform or missing binaries. Allocation tracking will be disabled, but basic timing will still function.

```clojure
;; On unsupported platform
(agent/loaded?)
;; => false

(agent/agent-path)
;; => nil

;; Benchmarks still work, but without allocation tracking
(bench/bench (reduce + (range 1000)))
;; => Results show timing but no allocation data
```

--------------------------------

### Run All Criterium Tests with Bundled Agent

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Execute all Criterium tests, including those that utilize the bundled C++ agent. This command assumes the agent binaries have been correctly placed in the resources directory.

```bash
clojure -M:kaocha:dev:test --reporter dots
```

--------------------------------

### Build Agent Locally with Debug Symbols

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Build the agent locally with debug symbols enabled for development. This provides more diagnostic information during testing.

```bash
cmake -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build
```

--------------------------------

### Download Linux Agent Artifact

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Download the agent-cpp-linux-x64 artifact from a specific GitHub Actions workflow run using the GitHub CLI. This artifact contains the Linux agent binary and its SHA256 hash.

```bash
gh run download <run-id> -n agent-cpp-linux-x64
```

--------------------------------

### Warning and Optimization Flags

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Sets warning flags and conditionally adds -Werror for Release builds. Also configures optimization flags for Release builds.

```cmake
# Warning flags - treat warnings as errors in Release builds
set(WARNING_FLAGS -Wall -Wextra -Wpedantic)
if(CMAKE_BUILD_TYPE STREQUAL "Release")
    list(APPEND WARNING_FLAGS -Werror)
endif()

# Optimization flags
if(CMAKE_BUILD_TYPE STREQUAL "Release")
    set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O2")
endif()
```

--------------------------------

### Basic Single Expression Benchmark

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Benchmark a single Clojure expression. The optional :viewer option can be used to control output formatting. Use `bench/last-bench` to access the results of the most recent benchmark.

```clojure
(require '[criterium.bench :as bench])

(bench/bench (my-function arg1 arg2))
(bench/bench (my-function arg1 arg2) :viewer :pprint)
(bench/last-bench)  ; Access results
```

--------------------------------

### Criterium agent API: jvm-opts

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Returns JVM arguments for spawning subprocesses with the agent. Returns an empty vector if unavailable.

```clojure
(jvm-opts)
```

--------------------------------

### Complexity Analysis using Domain Analysis

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Analyze the time complexity of an algorithm using Criterium's domain analysis. This plan helps in understanding how the algorithm's performance scales with input size.

```clojure
(domain/bench
 (domain/domain-expr
  [n (builder/log-range 10 10000 7)]
  (my-algorithm n))
 :domain-plan domain-plans/complexity-analysis)
```

--------------------------------

### Building the Criterium Shared Library

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Adds the Criterium shared library target, compiles it with specified warning flags, and sets clang-tidy properties if available.

```cmake
add_library(criterium SHARED ${sources})
target_compile_options(criterium PRIVATE ${WARNING_FLAGS})
if(CLANG_TIDY_EXE)
    set_target_properties(criterium PROPERTIES CXX_CLANG_TIDY "${CLANG_TIDY_EXE}")
endif()
```

--------------------------------

### Build Criterium Agent Locally

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Compile the native Criterium agent from source using the 'make' command. This is part of the local development workflow for making changes to the agent's native code.

```bash
# From the agent-cpp/ directory
make

# Build with debug symbols
make DEBUG=1
```

--------------------------------

### Run Tests with Local Agent on macOS

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Execute the project's tests using a locally built Criterium agent on macOS. This ensures that tests are run with the latest native agent code.

```bash
# Run tests with local agent
clojure -M:kaocha:dev:test:with-agent-mac --reporter dots
```

--------------------------------

### Complexity Analysis in Domain Analysis

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Performs complexity analysis by fitting O(log n), O(n), O(n log n), O(n²) models to benchmark results. Uses a specialized range generator for O(n log n) algorithms.

```clojure
(domain/bench
 (domain/domain-expr
  [n (builder/n-log-n-range 10 10000 7)]
  (sort (vec (range n))))
 :domain-plan domain-plans/complexity-analysis)
```

--------------------------------

### Project Definition

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Defines the project name for the Criterium build.

```cmake
project(criterium)
```

--------------------------------

### View Specific Workflow Run

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Use the GitHub CLI to view details of a specific workflow run, identified by its run ID. This is a prerequisite for downloading artifacts.

```bash
gh run view <run-id>
```

--------------------------------

### Cross-Compile Agent for Apple Silicon

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Cross-compile agent binaries specifically for Apple Silicon (arm64) architecture on macOS using CMake. This targets newer Mac hardware.

```bash
cmake -B build -DCMAKE_OSX_ARCHITECTURES=arm64
cmake --build build
```

--------------------------------

### Spawn Subprocess with Criterium Agent

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Generate JVM arguments required to spawn a subprocess with the Criterium agent enabled. This is useful for running benchmarks or other tasks in isolated environments with agent capabilities.

```clojure
(require '[criterium.agent :as agent])

;; Get JVM arguments for subprocess
(agent/jvm-opts)
;; => ["-agentpath:/tmp/criterium-agent-linux-x64-abc123.so"] or []
```

--------------------------------

### Cross-Compile Agent for Intel Macs

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Cross-compile agent binaries specifically for Intel (x86_64) architecture on macOS using CMake. This targets traditional Mac hardware.

```bash
cmake -B build -DCMAKE_OSX_ARCHITECTURES=x86_64
cmake --build build
```

--------------------------------

### Build Tests Option

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Configures an option to control whether tests should be built.

```cmake
# Testing option
option(BUILD_TESTS "Build tests" ON)
```

--------------------------------

### Run Criterium Validation Tests

Source: https://github.com/hugoduncan/criterium/blob/develop/bases/criterium/validation/README.md

Execute all validation tests located in the `bases/criterium/validation/` directory from the project root. Tests will automatically skip if R/Rserve is unavailable.

```bash
clojure -M:validation
```

--------------------------------

### Discover Criterium Tests with Google Test

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/test/CMakeLists.txt

Includes the Google Test module and discovers all tests within the Criterium test executable.

```cmake
include(GoogleTest)
gtest_discover_tests(criterium_tests)
```

--------------------------------

### Set Include Directories for Criterium Tests

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/test/CMakeLists.txt

Specifies private include directories for the Criterium test executable, including the project's source directory.

```cmake
target_include_directories(criterium_tests PRIVATE
    ${CMAKE_SOURCE_DIR}
)
```

--------------------------------

### List Recent Workflow Runs

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/building-agent.md

Use the GitHub CLI to list recent workflow runs for the agent-cpp.yml workflow. This helps identify the correct run ID for downloading artifacts.

```bash
gh run list --workflow="agent-cpp.yml" --limit 5
```

--------------------------------

### Source Files Definition

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Defines the source files for the Criterium library.

```cmake
set(sources agent.cpp)
```

--------------------------------

### Finding and Configuring clang-tidy

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Locates the clang-tidy executable and issues a warning if not found. This is used for code analysis on the project's targets.

```cmake
# Find clang-tidy for use on our targets only (not dependencies)
find_program(CLANG_TIDY_EXE NAMES "clang-tidy")
if(NOT CLANG_TIDY_EXE)
    # On mac, this can be installed with `brew install llvm`
    message(WARNING "clang-tidy not found")
endif()
```

--------------------------------

### Merge develop into master

Source: https://github.com/hugoduncan/criterium/blob/develop/docs/contributor/releasing.md

Before performing a release, merge the latest changes from the 'develop' branch into the 'master' branch and push the changes.

```bash
git checkout master
git pull origin master
git merge develop
git push origin master
```

--------------------------------

### Debug Build Option

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Configures a DEBUG option to enable debug mode and add the DEBUG definition if enabled.

```cmake
# Debug configuration
option(DEBUG "Enable debug mode" OFF)
if(DEBUG)
    add_definitions(-DDEBUG)
endif()
```

--------------------------------

### Add Criterium Dependency with Leiningen

Source: https://github.com/hugoduncan/criterium/blob/develop/README.md

Add this to your project's :dependencies in your project.clj file to include Criterium.

```clojure
[criterium "0.4.6"]
```

--------------------------------

### Criterium agent API: agent-path

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Returns the absolute path to the extracted agent binary. Returns nil if the agent is unavailable.

```clojure
(agent-path)
```

--------------------------------

### criterium.agent.core low-level API

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Provides low-level functions for checking the agent's attachment status and retrieving the current JVM process ID.

```APIDOC
## Low-level API (`criterium.agent.core`)

### Description
Core low-level functions for agent interaction.

### Functions

- **`(attached?)`**
  - Checks if the agent was loaded via the `-agentpath` JVM argument.

- **`(pid)`**
  - Returns the current JVM process ID.
```

--------------------------------

### Finding and Including JNI

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/CMakeLists.txt

Finds the Java Native Interface (JNI) package and includes its directories for compilation.

```cmake
find_package(JNI REQUIRED)
include_directories(${JNI_INCLUDE_DIRS})
```

--------------------------------

### Link Libraries for Criterium Tests

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/test/CMakeLists.txt

Links the Criterium test executable against Google Test libraries (gtest_main and gmock).

```cmake
target_link_libraries(criterium_tests
    GTest::gtest_main
    GTest::gmock
)
```

--------------------------------

### criterium.agent namespace functions

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Provides functions to interact with the Criterium agent, such as checking its loaded status, retrieving its file path, and obtaining JVM arguments for subprocesses.

```APIDOC
## `criterium.agent` namespace

### Description
Functions for interacting with the Criterium agent.

### Functions

- **`(loaded?)`**
  - Returns `true` if the agent is currently loaded via the `-agentpath` JVM argument, `false` otherwise.

- **`(agent-path)`**
  - Returns the absolute path to the extracted agent binary. Returns `nil` if the agent is unavailable.

- **`(jvm-opts)`**
  - Returns a vector of JVM arguments required for spawning subprocesses that use the agent (e.g., `["-agentpath:/tmp/..."]`). Returns an empty vector if the agent is unavailable.

### Deprecated

- **`(load-agent!)`**
  - **Note:** Programmatic loading is deprecated and does not work for allocation tracking due to JVMTI limitations. The `can_generate_sampled_object_alloc_events` capability must be requested at VM startup. Use `-agentpath` at JVM startup instead.
```

--------------------------------

### Benchmark with Generated Arguments

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Benchmark a function using arguments generated by Criterium's argument generation utilities. This is useful for testing functions with complex or varied input types.

```clojure
(require '[criterium.arg-gen :as arg-gen]
         '[clojure.test.check.generators :as gen])

(bench/bench-measured
 (bench/options->bench-plan)
 (arg-gen/measured {:size 100}
  [coll (gen/vector gen/small-integer)]
  (sort coll)))
```

--------------------------------

### Add Criterium Dependency with Maven

Source: https://github.com/hugoduncan/criterium/blob/develop/README.md

Include this dependency in your Maven project's pom.xml to use Criterium.

```xml
<dependency>
  <groupId>criterium</groupId>
  <artifactId>criterium</artifactId>
  <version>0.4.6</version>
</dependency>
```

--------------------------------

### Set Compile Options for Criterium Tests

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/test/CMakeLists.txt

Applies private compile options, specifically warning flags, to the Criterium test executable.

```cmake
target_compile_options(criterium_tests PRIVATE ${WARNING_FLAGS})
```

--------------------------------

### Define Criterium Test Executable

Source: https://github.com/hugoduncan/criterium/blob/develop/agent-cpp/test/CMakeLists.txt

Defines the main executable for Criterium tests and lists all associated source files.

```cmake
add_executable(criterium_tests
    test_main.cpp
    agent_state_test.cpp
    all_tags_test.cpp
    call_tree_test.cpp
    integration_test.cpp
    is_system_class_test.cpp
    message_queue_test.cpp
    method_tracing_state_transitions_test.cpp
    state_transitions_test.cpp
)
```

--------------------------------

### Criterium agent API: loaded?

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Checks if the agent is currently loaded via the -agentpath JVM argument. Returns a boolean.

```clojure
(loaded?)
```

--------------------------------

### Force Overhead Recalculation

Source: https://github.com/hugoduncan/criterium/blob/develop/README.md

Manually recalculate the measurement overhead if needed by calling `estimated-overhead!`. This ensures accurate timing, especially if the JVM is busy.

```clojure
(criterium.core/estimated-overhead!)
```

--------------------------------

### Criterium agent core API: pid

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Returns the current JVM process ID. This is a low-level function.

```clojure
(pid)
```

--------------------------------

### Set Constant Measurement Overhead

Source: https://github.com/hugoduncan/criterium/blob/develop/README.md

For consistency across JVM processes, explicitly set the measurement overhead to a constant value using `estimated-overhead!`.

```clojure
(criterium.core/estimated-overhead! 10000)
```

--------------------------------

### Criterium agent core API: attached?

Source: https://github.com/hugoduncan/criterium/blob/develop/projects/agent/README.md

Checks if the agent was loaded via the -agentpath JVM argument. Returns a boolean.

```clojure
(attached?)
```

--------------------------------

### Avoiding Dead Code Elimination in Benchmarks

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Ensure benchmarked code's results are used to prevent the JVM from optimizing it away. Use the result of the computation in the benchmark.

```clojure
;; BAD - side-effect only, result discarded
(bench/bench (do (sort data) nil))

;; GOOD - return the result
(bench/bench (sort data))
```

--------------------------------

### Avoiding Side Effects in Benchmarks

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Isolate computations from side effects like I/O or mutation. Ensure the benchmark measures the intended computation, not external operations.

```clojure
;; BAD - file I/O dominates timing
(bench/bench (spit "test.txt" (str data)))

;; GOOD - separate I/O from computation
(bench/bench (str data))
```

--------------------------------

### Dependent Argument Generation

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Generates arguments where one binding (n) influences another (the size of the vector 'coll'), enabling dependent input generation.

```clojure
(arg-gen/measured
 [n (gen/choose 10 100)                    ; n bound first
  coll (gen/vector gen/small-integer n)]   ; n used to size the vector
 (reduce + coll))
```

--------------------------------

### Avoiding Constant Folding in Benchmarks

Source: https://github.com/hugoduncan/criterium/blob/develop/skills/criterium/SKILL.md

Prevent the compiler from optimizing constant expressions by using local bindings. This ensures the expression is evaluated during the benchmark.

```clojure
;; BAD - may be optimized to constant
(bench/bench (+ 1 2))

;; BETTER - use local bindings
(let [a 1 b 2]
  (bench/bench (+ a b)))
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.