### Complete Benchmark Class Example

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

A full example of a benchmark class including state, setup, teardown, and the benchmark method.

```kotlin
import kotlinx.benchmark.*

@State(Scope.Benchmark)
class MyBenchmark {
    private val size = 10
    private val list = ArrayList<Int>()

    @Setup
    fun prepare() {
        for (i in 0..<size) {
            list.add(i)
        }
    }

    @TearDown
    fun cleanup() {
        list.clear()
    }

    @Benchmark
    fun benchmarkMethod(): Int {
        return list.sum()
    }
}
```

--------------------------------

### Setup and Teardown Methods

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Use `@Setup` to initialize resources before a benchmark run and `@TearDown` to clean them up afterward.

```kotlin
@Setup
fun prepare() {
    for (i in 0..<size) {
        list.add(i)
    }
}

@TearDown
fun cleanup() {
    list.clear()
}
```

--------------------------------

### Multiplatform Benchmark Example

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/writing-benchmarks.md

This example demonstrates a basic multiplatform benchmark class. It includes annotations for benchmark mode, output time unit, warmup and measurement iterations, state scope, parameterization, setup, teardown, and the benchmark method itself. Use this to test the performance of operations like summing elements in a list with varying input sizes.

```kotlin
import kotlinx.benchmark.*

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(BenchmarkTimeUnit.MILLISECONDS)
@Warmup(iterations = 10, time = 500, timeUnit = BenchmarkTimeUnit.MILLISECONDS)
@Measurement(iterations = 20, time = 1, timeUnit = BenchmarkTimeUnit.SECONDS)
@State(Scope.Benchmark)
class ExampleBenchmark {

    // Parameterizes the benchmark to run with different list sizes
    @Param("4", "10")
    var size: Int = 0

    private val list = ArrayList<Int>()

    // Prepares the test environment before each benchmark run
    @Setup
    fun prepare() {
        for (i in 0..<size) {
            list.add(i)
        }
    }

    // Cleans up resources after each benchmark run
    @TearDown
    fun cleanup() {
        list.clear()
    }

    // The actual benchmark method
    @Benchmark
    fun benchmarkMethod(): Int {
        return list.sum()
    }
}
```

--------------------------------

### Writing a Basic Kotlin Benchmark

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Define benchmark classes using JMH annotations like @State, @Setup, and @Benchmark. Ensure necessary imports from org.openjdk.jmh.annotations are included.

```kotlin
import org.openjdk.jmh.annotations.*
import java.util.concurrent.*
import kotlin.math.cos
import kotlin.math.sqrt
import kotlin.random.Random

@State(Scope.Benchmark)
@Fork(1)
@Warmup(iterations = 10)
@Measurement(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS)
class MyBenchmark {
    private var data = 0.0

    @Setup
    fun setUp() {
        data = Random.nextDouble()
    }

    @Benchmark
    fun sqrtBenchmark(): Double {
        return sqrt(data)
    }

    @Benchmark
    fun cosBenchmark(): Double {
        return cos(data)
    }
}
```

--------------------------------

### Example Benchmark Class with @State Annotation

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

An example of a Kotlin benchmark class annotated with @State(Scope.Benchmark). The allopen plugin ensures this class and its methods are treated as open, which is required by JMH.

```kotlin
// MyBenchmark.kt
@State(Scope.Benchmark)
class MyBenchmark {
    // Benchmarking-related methods and variables
    @Benchmark
    fun benchmarkMethod() {
        // benchmarking logic
    }
}
```

--------------------------------

### Configure Notebook Environment

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Sets up the necessary extensions for the notebook environment, including serialization, dataframes, and plotting capabilities.

```python
%use serialization, dataframe, kandy
```

--------------------------------

### Configure Warmup Phase

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/writing-benchmarks.md

Use @Warmup to specify the preliminary phase before actual benchmarking. This allows the system to reach optimal performance.

```kotlin
class MyBenchmark {

    @Warmup(iterations = 10, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Benchmark
    fun benchmarkMethod() {
        // benchmark code here
    }
}
```

--------------------------------

### Apply kotlinx-benchmark Plugin (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Apply the kotlinx-benchmark plugin to your build.gradle.kts file.

```kotlin
plugins {
    id("org.jetbrains.kotlinx.benchmark") version "0.4.13"
}
```

--------------------------------

### Apply kotlinx-benchmark Plugin (Groovy DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Apply the kotlinx-benchmark plugin to your build.gradle file.

```groovy
plugins {
    id 'org.jetbrains.kotlinx.benchmark' version '0.4.13'
}
```

--------------------------------

### Register Native Target for Benchmarking

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Define a Kotlin/Native target, such as 'linuxX64', and register it for benchmarking in your build.gradle.kts file. Benchmarks can only be executed for the host target.

```kotlin
kotlin {
    linuxX64()
}

benchmark {
    targets {
        register("linuxX64")
    }
}
```

--------------------------------

### Run JVM Benchmark

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Execute the JVM benchmark for the Kotlin Multiplatform project using Gradle. This command should be run from the project's root directory.

```shell
./gradlew :examples:kotlin-multiplatform:jvmBenchmark
```

--------------------------------

### Configure Plugin Repository (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Ensure the Gradle Plugin Portal is available for plugin resolution in settings.gradle.kts.

```kotlin
pluginManagement {
    repositories {
        gradlePluginPortal()
    }
}
```

--------------------------------

### Configure Benchmark Profiles and Targets

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/tasks-overview.md

This configuration block defines custom benchmark profiles and registers targets for benchmarking. Ensure your build script includes this to set up custom benchmark runs.

```kotlin
benchmark {
    configurations {
        named("main") {
            iterations = 20
            warmups = 20
            iterationTime = 1
            iterationTimeUnit = "s"
        }
        register("smoke") {
            include("Essential")
            iterations = 10
            warmups = 10
            iterationTime = 200
            iterationTimeUnit = "ms"
        }
    }

    targets {
        register("jvm")
        register("js")
    }
}
```

--------------------------------

### Designate Main Source Set as Benchmark Target (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Configure the benchmark extension to register the 'main' source set as a benchmark target in build.gradle.kts.

```kotlin
benchmark {
    targets {
        register("main")
    }
}
```

--------------------------------

### Configuring Benchmark Profiles in Gradle

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Customize benchmark execution by defining multiple configuration profiles within the `benchmark` block in your `build.gradle.kts` file. The 'main' configuration can be modified, and new ones like 'smoke' can be registered with specific include patterns and settings.

```kotlin
// build.gradle.kts
benchmark {
    configurations {
        named("main") {
            warmups = 20
            iterations = 10
            iterationTime = 3
            iterationTimeUnit = "s"
        }
        register("smoke") {
            include("<pattern of fully qualified name>")
            warmups = 5
            iterations = 3
            iterationTime = 500
            iterationTimeUnit = "ms"
        }
    }
}
```

--------------------------------

### Registering a New Benchmark Configuration Profile

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/configuration-options.md

Define a new benchmark configuration profile named 'smoke' within the `configurations` block. This allows for custom benchmark execution settings.

```kotlin
benchmark {
    configurations {
        register("smoke") {
            // Configure this configuration profile here
        }
        // here you can create additional profiles
    }
}
```

--------------------------------

### Run Gradle Benchmark Task

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-hypothesis.ipynb

Execute the Gradle task to run benchmarks. This command should be run from the root of the project.

```shell
> ./gradlew :examples:kotlin-jvm-compare-hypothesis:benchmark
```

--------------------------------

### Define Benchmark Source Set (Kotlin/JVM)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/separate-benchmark-source-set.md

Create a new source set named 'benchmark' in your project. This source set will contain your benchmark code.

```kotlin
sourceSets {
    create("benchmark")
}
```

--------------------------------

### Register JS Target for Benchmarking

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Configure a Kotlin/JS target with Node.js execution and register it for benchmarking in your build.gradle.kts file.

```kotlin
kotlin {
    js {
        nodejs()
    }
}

benchmark {
    targets {
        register("js")
    }
}
```

--------------------------------

### Register JVM Target for Benchmarking

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Register the 'jvm' target within the benchmark extension in your build.gradle.kts file.

```kotlin
kotlin {
    jvm()
}

benchmark {
    targets {
        register("jvm")
    }
}
```

--------------------------------

### Add kotlinx-benchmark-runtime Dependency (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Include the kotlinx-benchmark-runtime dependency in your build.gradle.kts file.

```kotlin
dependencies {
    implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.13")
}
```

--------------------------------

### Configure Gradle Plugin Portal (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Ensure the Gradle Plugin Portal is included in repositories for plugin lookup in settings.gradle.kts.

```kotlin
pluginManagement {
    repositories {
        gradlePluginPortal()
    }
}
```

--------------------------------

### Configure AllOpen Plugin for Benchmarks (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Apply the allopen plugin and configure it to annotate @State classes as open in build.gradle.kts. This is necessary for JMH to operate correctly with Kotlin's final-by-default classes.

```kotlin
plugins {
    kotlin("plugin.allopen") version "2.2.0"
}

allOpen {
    annotation("org.openjdk.jmh.annotations.State")
}
```

--------------------------------

### Add kotlinx-benchmark-runtime Dependency (Groovy DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Include the kotlinx-benchmark-runtime dependency in your build.gradle file.

```groovy
dependencies {
    implementation 'org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.13'
}
```

--------------------------------

### Configure AllOpen Plugin for Benchmarks (Groovy DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Apply the allopen plugin and configure it to annotate @State classes as open in build.gradle. This is necessary for JMH to operate correctly with Kotlin's final-by-default classes.

```groovy
plugins {
    id 'org.jetbrains.kotlin.plugin.allopen' version "2.2.0"
}

allOpen {
    annotation("org.openjdk.jmh.annotations.State")
}
```

--------------------------------

### Configure Benchmark Profiles in build.gradle.kts

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Define custom benchmark configurations, including warmups, iterations, and time units, within the `benchmark` block.

```kotlin
// build.gradle.kts
benchmark {
    configurations {
        named("main") {
            warmups = 20
            iterations = 10
            iterationTime = 3
            iterationTimeUnit = "s"
        }
        register("smoke") {
            include("<pattern of fully qualified name>")
            warmups = 5
            iterations = 3
            iterationTime = 500
            iterationTimeUnit = "ms"
        }
    }
}
```

--------------------------------

### Register Wasm Target for Benchmarking

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Configure a Kotlin/Wasm target with Node.js execution and register it for benchmarking in your build.gradle.kts file. Note that Kotlin/Wasm is an experimental target.

```kotlin
kotlin {
    wasmJs {
        nodejs()
    }
}

benchmark {
    targets {
        register("wasmJs")
    }
}
```

--------------------------------

### Load Benchmark Results from JSON

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Finds the latest benchmark results file and parses it into a list of Benchmark objects. It filters directories by creation time to identify the most recent run.

```kotlin
import java.nio.file.Files
import java.nio.file.attribute.BasicFileAttributes
import kotlin.io.path.exists
import kotlin.io.path.forEachDirectoryEntry
import kotlin.io.path.isDirectory
import kotlin.io.path.listDirectoryEntries
import kotlin.io.path.readText

// Find latest result file, based on the their timestamp.
val runsDir = notebook.workingDir.resolve("kotlin-multiplatform/build/reports/benchmarks/main")
val lastRunDir = runsDir.listDirectoryEntries()
    .filter { it.isDirectory() }
    .sortedByDescending { dir -> Files.readAttributes(dir, BasicFileAttributes::class.java).creationTime() }
    .first()
val outputFile = lastRunDir.resolve("jvm.json")
val json = Json { ignoreUnknownKeys = true }
val benchmarkData = json.decodeFromString<List<Benchmark>>(outputFile.readText())
```

--------------------------------

### Include Maven Central Repository (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/kotlin-jvm-project-setup.md

Add Maven Central to the repositories in your build.gradle.kts file for dependency lookup.

```kotlin
repositories {
    mavenCentral()
}
```

--------------------------------

### Define Benchmark Method

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Annotate the method you want to benchmark with `@Benchmark`.

```kotlin
@Benchmark
fun benchmarkMethod(): Int {
    return list.sum()
}
```

--------------------------------

### Prepare Benchmark DataFrames

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-hypothesis.ipynb

Adds calculated columns for error margins and formats benchmark labels. It also determines bar colors based on performance differences (green for improvement, red for degradation, grey for overlap).

```kotlin
val plotData = benchmarkGroups.mapValues {
    it.value
        .add("errorMin") { it.getValue<Double>("score") - it.getValue<Double>("error") }
        .add("errorMax") { it.getValue<Double>("score") + it.getValue<Double>("error") }
        .add("errorMin1") { it.getValue<Double>("score1") - it.getValue<Double>("error1") }
        .add("errorMax1") { it.getValue<Double>("score1") + it.getValue<Double>("error1") }
        .add("diff") { (it.getValue<Double>("score1") - it.getValue<Double>("score")) / it.getValue<Double>("score")  * 100.0 }
        .insert("label") {
            // Re-format the benchmark labels to make them look "nicer"
            if (!it.getValue<String>("params").isBlank()) {
                it.getValue<String>("params").replace(",", "\n")
            } else {
                it.getValue<String>("name").substringAfterLast(".").removeSuffix(baselineSuffix)
            }
        }.at(0)
        .add("barColor") {
            val diff = get("diff") as Double
            val interval1 = (get("errorMin") as Double)..(get("errorMax") as Double)
            val interval2 = (get("errorMin1") as Double)..(get("errorMax1") as Double)
            val overlap = interval1.start <= interval2.endInclusive && interval2.start <= interval1.endInclusive
            when {
                overlap -> "grey"
                diff > 0 -> "green"
                else -> "red"
            }
        }
        .remove("name", "params")
}
```

--------------------------------

### Specify Runtime Repository (Kotlin DSL)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Ensure mavenCentral() is configured for dependency resolution in build.gradle.kts.

```kotlin
repositories {
    mavenCentral()
}
```

--------------------------------

### Register Benchmark Source Set (Kotlin/JVM)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/separate-benchmark-source-set.md

Register the 'benchmark' source set with the kotlinx-benchmark tool. This ensures that benchmarks within this source set are recognized and executed.

```kotlin
benchmark {
    targets { 
        register("benchmark")
    }
}
```

--------------------------------

### Group and Compare Benchmarks

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-hypothesis.ipynb

Groups benchmark results by their base name and then joins the baseline and optimized implementations for comparison. It extracts relevant information like parameters, score, error, and unit for each benchmark.

```kotlin
import kotlinx.serialization.json.*

// Helper class for tracking the information we need to use.
data class Benchmark(val name: String, val params: String, val score: Double, val error: Double, val unit: String)

// Split benchmark results into groups. Generally, each group consist of all tests from one test file,
// except when it is an parameterized test. In this case, each test (with all its variants) are put
// in its own group.
val benchmarkGroups = benchmarkData
    .groupBy {
        if (it.benchmark.endsWith(optimizedSuffix))
            it.benchmark.removeSuffix(optimizedSuffix)
        else
            it.benchmark.removeSuffix(baselineSuffix)
    }
    .mapValues { group ->
        val benchmarks = group.value.map { benchmark ->
            // Parameters are specific to each test. `deserializeJson()` will generate the appropriate data classes,
            // but for generic handling of parameters we would need to fallback to reading the JSON. In this case
            // we just handle them through the typed API.
            val paramInfo = benchmark.params?.entries.orEmpty()
                .sortedBy { it.key }
                .joinToString(",") { "${it.key}=${it.value.jsonPrimitive.content}" }
            val name = benchmark.benchmark
            Benchmark(
                name,
                paramInfo,
                benchmark.primaryMetric.score,
                benchmark.primaryMetric.scoreError,
                benchmark.primaryMetric.scoreUnit
            )
        }
        val baseline = benchmarks.filter { it.name.endsWith("Baseline") }.toDataFrame()
        val optimized = benchmarks.filter { it.name.endsWith("Optimized") }.toDataFrame()
        baseline.join(optimized, "params")
    }

```

--------------------------------

### Create Benchmark Class with State

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Annotate your benchmark class with `@State(Scope.Benchmark)` to manage its state during benchmark execution.

```kotlin
import kotlinx.benchmark.*

@State(Scope.Benchmark)
class MyBenchmark {

}
```

--------------------------------

### Consume Benchmark Results with Blackhole

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/writing-benchmarks.md

Inject Blackhole into your benchmark method to consume results of computations, preventing unwanted optimizations like dead-code elimination.

```kotlin
@Benchmark
fun iterateBenchmark(bh: Blackhole) {
    for (e in myList) {
        bh.consume(e)
    }
}
```

--------------------------------

### Find Latest Benchmark Result Files

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Locates the two most recent benchmark result JSON files from the specified directory. It sorts directories by creation time to ensure the latest runs are selected.

```kotlin
import java.nio.file.Files
import java.nio.file.attribute.BasicFileAttributes
import kotlin.io.path.exists
import kotlin.io.path.forEachDirectoryEntry
import kotlin.io.path.isDirectory
import kotlin.io.path.listDirectoryEntries
import kotlin.io.path.readText

// Find latest result file, based on the their timestamp.
val runsDir = notebook.workingDir.resolve("kotlin-multiplatform/build/reports/benchmarks/main")
val outputFiles = runsDir.listDirectoryEntries()
    .filter { it.isDirectory() }
    .sortedByDescending { dir -> Files.readAttributes(dir, BasicFileAttributes::class.java).creationTime() }
    .subList(0, 2)
    .map { it.resolve("jvm.json") }
```

--------------------------------

### Find and Deserialize Latest Benchmark Results

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-hypothesis.ipynb

Locates the latest benchmark results JSON file and deserializes its content into a list of Benchmark objects. It assumes results are stored in a specific directory structure relative to the working directory.

```kotlin
import kotlinx.serialization.json.Json
import java.nio.file.Files
import java.nio.file.attribute.BasicFileAttributes
import kotlin.io.path.exists
import kotlin.io.path.forEachDirectoryEntry
import kotlin.io.path.isDirectory
import kotlin.io.path.listDirectoryEntries
import kotlin.io.path.readText

// Find latest result file, based on the their timestamp.
val runsDir = notebook.workingDir.resolve("kotlin-jvm-compare-hypothesis/build/reports/benchmarks/main")
val lastRunDir = runsDir.listDirectoryEntries()
    .filter { it.isDirectory() }
    .sortedByDescending { dir -> Files.readAttributes(dir, BasicFileAttributes::class.java).creationTime() }
    .first()
val outputFile = lastRunDir.resolve("main.json")
val json = Json { ignoreUnknownKeys = true }
val benchmarkData = json.decodeFromString<List<Benchmark>>(outputFile.readText())
```

--------------------------------

### Define Benchmark Suffixes

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-hypothesis.ipynb

Define string constants for the suffixes used to identify baseline and optimized benchmark implementations.

```kotlin
// Benchmarks for a "baseline" implementation have a "Baseline" suffix in their names,
// while benchmarks for an "opimized" implementation have a "Optimized" suffix.
val baselineSuffix = "Baseline"
val optimizedSuffix = "Optimized"
```

--------------------------------

### Select Built-in JS Benchmarks Executor

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/configuration-options.md

Configure the Kotlin/JS target to use the library's built-in benchmarking implementation instead of the default 'benchmark.js' library.

```kotlin
benchmark {
    targets {
        register("jsBenchmarks") {
            this as JsBenchmarkTarget
            jsBenchmarksExecutor = JsBenchmarksExecutor.BuiltIn
        }
    }
}
```

--------------------------------

### Format Combined Data for Analysis

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Reduces the combined benchmark data into a more streamlined format, extracting relevant fields like name, parameters, mode, unit, and scores from both old and new runs. Parameters are formatted into a comma-separated string.

```kotlin
import kotlinx.serialization.json.encodeToJsonElement

// Reduce the combined data into the exact format we need
val resultData = combinedData.mapToFrame {
    "name" from { it.benchmark }
    "params" from {
        it.params?.entries.orEmpty()
            .sortedBy { it.key }
            .joinToString(",") { entry -> "${entry.key}=${entry.value.jsonPrimitive.content}" }
    }
    "mode" from { it.mode } // "avgt" or "thrpt"
    "unit" from { it.primaryMetric.scoreUnit }
    "runOld" {
        "score" from { it.primaryMetric.score }
        "range" from { it.primaryMetric.scoreConfidence[0]..it.primaryMetric.scoreConfidence[1] }
    }
    "runNew" {
        "score" from { it.primaryMetric1.score }
        "range" from { it.primaryMetric1.scoreConfidence[0]..it.primaryMetric1.scoreConfidence[1] }
    }
}
// Un-commont this to see the intermediate dataframe:
// resultData
```

--------------------------------

### Run JVM Benchmark JAR

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/tasks-overview.md

This task generates a self-contained executable JAR for JVM benchmarks. It can be run using `java -jar` and supports JMH profilers. The JAR is located in `build/benchmarks/<targetName>/jars/`.

```bash
java -jar path-to-the.jar -h
```

--------------------------------

### Associate Benchmark Compilation with Main (Kotlin/JVM)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/separate-benchmark-source-set.md

Associate the 'benchmark' compilation with the 'main' compilation. This enables the benchmark compilation to access internal APIs from 'main' and propagate dependencies.

```kotlin
kotlin {
    target {
        compilations.getByName("benchmark")
            .associateWith(compilations.getByName("main"))
    }
}
```

--------------------------------

### Configure Measurement Phase

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/writing-benchmarks.md

Use @Measurement to control the properties of the actual benchmarking phase, including the number and duration of iterations.

```kotlin
class MyBenchmark {

    @Measurement(iterations = 20, time = 1, timeUnit = TimeUnit.SECONDS)
    @Benchmark
    fun benchmarkMethod() {
        // benchmark code here
    }
}
```

--------------------------------

### Group and Format Benchmark Data

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Groups benchmark results by their source and formats them into DataFrames. Parameterized tests are grouped individually, while others are grouped by their test file.

```kotlin
import kotlinx.serialization.json.encodeToJsonElement

// Helper class for tracking the information we need to use.
data class Benchmark(val name: String, val params: String, val score: Double, val error: Double, val unit: String)

// Split benchmark results into groups. Generally, each group consist of all tests from one test file,
// except when it is an parameterized test. In this case, each test (with all its variants) are put
// in its own group.
val benchmarkGroups = benchmarkData
    .groupBy {
        if (it.params != null) {
            it.benchmark
        } else {
            it.benchmark.substringBeforeLast(".")
        }
    }
    .mapValues { group ->
        val benchmarks = group.value.map { benchmark ->
            val paramInfo = benchmark.params?.entries.orEmpty()
                .sortedBy { it.key }
                .joinToString(",") { "${it.key}=${it.value.jsonPrimitive.content}" }
            val name = benchmark.benchmark
            Benchmark(
                name,
                paramInfo,
                benchmark.primaryMetric.score,
                benchmark.primaryMetric.scoreError,
                benchmark.primaryMetric.scoreUnit
            )
        }
        benchmarks.toDataFrame()
    }

// Un-commont this to see the benchmark data as DataFrames
// benchmarkGroups.forEach {
//     DISPLAY(it.value)
// }
```

--------------------------------

### Define New Compilation for Benchmarks (Kotlin Multiplatform)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/separate-benchmark-source-set.md

Create a new compilation named 'benchmark' and associate it with the 'main' compilation of the 'jvm' target. This allows the benchmark compilation to access internal APIs of the main compilation.

```kotlin
kotlin {
    jvm {
        compilations.create("benchmark") {
            associateWith(this@jvm.compilations.getByName("main"))
        }
    }
}
```

--------------------------------

### Prepare Data for Plotting

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Transforms the benchmark DataFrames by adding error bounds and creating a 'label' column for plotting. It formats parameter information for display and removes intermediate columns.

```kotlin
// Prepare the data frames for plotting by:
// - Add calculated columns for errorMin / errorMax
// - Tests with parameters use the parameter values as the label
// - Tests without paramaters use the test name as the label
val plotData = benchmarkGroups.mapValues {
    it.value
        .add("errorMin") { it.getValue<Double>("score") - it.getValue<Double>("error") }
        .add("errorMax") { it.getValue<Double>("score") + it.getValue<Double>("error") }
        .insert("label") {
            // Re-format the benchmark labels to make them look "nicer"
            if (!it.getValue<String>("params").isBlank()) {
                it.getValue<String>("params").replace(",", "\n")
            } else {
                it.getValue<String>("name").substringAfterLast(".").removeSuffix("Benchmark")
            }
        }.at(0)
        .remove("name", "params")

```

--------------------------------

### Parameterize Benchmark with @Param

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/writing-benchmarks.md

Use @Param to pass different parameter values to your benchmark method, allowing you to test performance with various inputs.

```kotlin
class MyBenchmark {

    @Param("4", "10")
    var size: String? = null

    @Benchmark
    fun benchmarkMethod(): Int {
        var sum = 0
        for (i in 0 until size!!.toInt()) {
            sum += i
        }
        return sum
    }
}
```

--------------------------------

### Normalize Benchmark Data for Comparison

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Calculates score differences and percentages, and adds labels and colors for plotting. This data processing is essential before visualization.

```kotlin
val plotData = mergedData
    .add("diffScore") {
        when (mode) {
            "avgt" -> score - score1
            "thrpt" -> score1 - score
            else -> error("Unknown mode: $mode")
        }
    }
    .add("diffScorePercentage") {
        (get("diffScore") as Double) * 100.0 / score
    }
    .add("testLabel") {
        if (params.isNullOrBlank()) {
            name
        } else {
            "$name\n[$params]"
        }
    }
    .add("barColor") {
        val value = get("diffScorePercentage") as Double
        if (value < 0.0) "neg" else "pos"
    }
plotData
```

--------------------------------

### Convert Benchmark Data to DataFrames

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Transforms the deserialized benchmark results into DataFrames for easier manipulation. A 'rowIndex' is added as a primary key, assuming the order of benchmarks in both files is consistent.

```kotlin
// Convert to DataFrames for easier processing. As there is not "id" keys for the benchmark, we invent one by just
// assigning the test row index as their "primary key". We could attempt to use the benchmark name and param values,
// but that is complicated by how paramers are represented in the JSON file. So, since we assume that the two files
// are equal using row index should be safe.
val oldDf = oldRun.toDataFrame().addId("rowIndex")
val newDf = newRun.toDataFrame().addId("rowIndex")
```

--------------------------------

### Plot Benchmark Data with Lets-Plot

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-hypothesis.ipynb

Generates bar plots with error bars for each benchmark group using Lets-Plot. Adjusts plot size dynamically based on the number of tests. Consider using a log scale or modifying limits if plots appear squished.

```kotlin
plotData.forEach { (fileName, dataframe) ->
    val plot = dataframe.plot {
        bars {
            x("label") {
                axis.name = ""
            }
            y("diff")
            fillColor("barColor") {
                scale = categorical("red" to Color.RED, "green" to Color.GREEN, "grey" to Color.GREY)
                legend.type = LegendType.None
            }
        }
        coordinatesTransformation = CoordinatesTransformation.cartesianFlipped()
        layout {
            this.yAxisLabel = "Diff, %"
            style {
                global {
                    title {
                        margin(10.0, -10.0)
                    }
                    text {
                        fontFamily = FontFamily.MONO
                    }
                }
            }
            // Adjust the height of the Kandy plot based on the number of tests.
            size = 800 to ((50 * dataframe.size().nrow) + 100)
        }
    }
    DISPLAY(HTML("<h4 >$fileName</h4>"))
    DISPLAY(plot)
}
```

--------------------------------

### Define Benchmark Variables

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/README.md

Declare variables within the benchmark class that will be used during the benchmark execution.

```kotlin
private val size = 10
private val list = ArrayList<Int>()
```

--------------------------------

### Set Kotlin/Native Build Type to DEBUG

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/configuration-options.md

Change the native binary build type to DEBUG for benchmarks. This includes debug information and is not optimized.

```kotlin
benchmark {
    targets {
        register("native") {
            this as NativeBenchmarkTarget
            buildType = NativeBuildType.DEBUG
        }
    }
}
```

--------------------------------

### Register Benchmark Compilation (Kotlin Multiplatform)

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/separate-benchmark-source-set.md

Register the new benchmark compilation using its default source set name, 'jvmBenchmark'. This informs the benchmark tool about the location of benchmark code.

```kotlin
benchmark {
    targets {
        register("jvmBenchmark")
    }
}
```

--------------------------------

### Define JMH-alike Serialization Classes

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-hypothesis.ipynb

Define data classes for deserializing JMH-alike JSON benchmark results. These classes match the structure of the benchmark output, including handling parameters as a generic JsonObject.

```kotlin
@Serializable
public data class Benchmark(
    public val benchmark: String,
    public val mode: String,
    public val warmupIterations: Int,
    public val warmupTime: String,
    public val measurementIterations: Int,
    public val measurementTime: String,
    public val primaryMetric: PrimaryMetric,
    public val secondaryMetrics: Map<String, PrimaryMetric>,
    public val params: JsonObject? = null
)

@Serializable
public data class PrimaryMetric(
    public val score: Double,
    public val scoreError: Double,
    public val scoreConfidence: List<Double>,
    public val scorePercentiles: Map<String, Double>,
    public val scoreUnit: String,
    public val rawData: List<List<Double>>,
)
```

--------------------------------

### Deserialize Benchmark JSON to Data Objects

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Decodes the JSON content of the latest two benchmark result files into Kotlin data objects using the defined `Benchmark` and `PrimaryMetric` classes. Assumes JSON format with `ignoreUnknownKeys` enabled.

```kotlin
// Convert to typed JSON
val json = Json { ignoreUnknownKeys = true }
val newRun = json.decodeFromString<List<Benchmark>>(outputFiles.first().readText())
val oldRun = json.decodeFromString<List<Benchmark>>(outputFiles.last().readText())
```

--------------------------------

### Define Benchmark Data Classes

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Defines Kotlin data classes to match the JMH-alike JSON format for benchmark results. This allows for generic handling of benchmark parameters.

```kotlin
@Serializable
public data class Benchmark(
    public val benchmark: String,
    public val mode: String,
    public val forks: Int = 1,
    public val warmupIterations: Int,
    public val warmupTime: String,
    public val measurementIterations: Int,
    public val measurementTime: String,
    public val primaryMetric: PrimaryMetric,
    public val secondaryMetrics: Map<String, PrimaryMetric>,
    public val params: JsonObject? = null
)

@Serializable
public data class PrimaryMetric(
    public val score: Double,
    public val scoreError: Double,
    public val scoreConfidence: List<Double>,
    public val scorePercentiles: Map<String, Double>,
    public val scoreUnit: String,
    public val rawData: List<List<Double>>,
)
```

--------------------------------

### Plot Benchmark Results

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Generates a bar plot for each benchmark group, displaying the score with error bars. This uses the Lets-Plot library for visualization.

```kotlin
import org.jetbrains.letsPlot.Geom
import org.jetbrains.letsPlot.core.spec.plotson.coord
import org.jetbrains.letsPlot.themes.margin

// Plot each group as a bar plot with the error displayed as error bars.

```

--------------------------------

### Plot Benchmark Score Differences

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Visualizes the calculated score differences as horizontal bars. It uses color to indicate positive or negative differences and sorts the bars by percentage difference.

```kotlin
plotData.sortBy { diffScorePercentage }.plot {
    barsH {
        x(diffScorePercentage) {
            axis.name = "Diff %"
        }
        y(testLabel) {
            axis.name = ""
        }
        fillColor(barColor) {
            scale = categorical("neg" to Color.RED, "pos" to Color.GREEN)
            legend.type = LegendType.None
        }
        tooltips {
            line(diffScorePercentage, format = ".2f")
        }
    }
    layout {
        size = 800 to ((40 * plotData.size().nrow) + 100)
        style {
            global {
                title {
                    margin(10.0, 0.0)
                }
            }
        }
    }
}
```

--------------------------------

### Specify JMH Version for JVM Benchmarks

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/docs/configuration-options.md

Customize the Java Microbenchmark Harness (JMH) version used for JVM benchmarks. Ensure compatibility between kotlinx-benchmark and JMH versions.

```kotlin
benchmark {
    targets {
        register("jvmBenchmarks") {
            this as JvmBenchmarkTarget
            jmhVersion = "1.38"
        }
    }
}
```

--------------------------------

### Flatten Data for Plotting

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Flattens the structured benchmark data, separating the 'runOld' and 'runNew' score and range information into distinct columns for easier plotting and analysis.

```kotlin
// Flatten the data so it is easier to plot
val mergedData = resultData.unfold { runOld and runNew }.flatten()
// Un-commont this to see the intermediate dataframe:
// mergedData
```

--------------------------------

### Combine Benchmark DataFrames

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Merges the two benchmark DataFrames based on the 'rowIndex' to align corresponding benchmark results from different runs.

```kotlin
val combinedData = oldDf.innerJoin(newDf) { rowIndex }
// Un-commont this to see the intermediate dataframe:
// combinedData
```

--------------------------------

### Calculate Score Difference Between Runs

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Calculates the difference in benchmark scores between the new and old runs. The calculation method ('newScore - oldScore' or 'oldScore - newScore') depends on the test mode ('avgt' or 'thrpt') to ensure positive values indicate improvement.

```kotlin
// Before plotting the data, we calculate the change between the two runs. This is saved
// in "scoreDiff". This is done slightly different depending on the test mode:
//
// - "avgt": For the average time we use "oldScore - newScore", so improvements in the
//   benchmark result in positive numbers.
// - "thrpt": For throughput, we use "newScore - oldScore", so improvements here also
//   result in positive numbers.
//

```

--------------------------------

### Plot Benchmark Data with Error Bars

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/simple-benchmark-analysis.ipynb

Iterates through benchmark dataframes to generate plots. Use this for visualizing performance scores with associated error margins. Customizes plot appearance, axis labels, and size dynamically.

```kotlin
plotData.forEach { (fileName, dataframe) ->
    val plot = dataframe.plot {
        bars {
            x("label") {
                axis.name = ""
            }
            y("score")
        }
        errorBars {
            x("label")
            y("score")
            yMin("errorMin")
            yMax("errorMax")
        }
        coordinatesTransformation = CoordinatesTransformation.cartesianFlipped()
        // y.axis.limits = dataframe.min("errorMin")..dataframe.max("errorMax")
        layout {
            this.yAxisLabel = dataframe.first().getValue<String>("unit")
            style {
                global {
                    title {
                        margin(10.0, 0.0)
                    }
                    text {
                        fontFamily = FontFamily.MONO
                    }
                }
            }
            // Adjust the height of the Kandy plot based on the number of tests.
            size = 800 to ((50 * dataframe.size().nrow) + 100)
        }
    }
    DISPLAY(HTML("<h4 >$fileName</h4>"))
    DISPLAY(plot)
}
```

--------------------------------

### Define Serialization Classes for Benchmark Results

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Defines Kotlin data classes to match the JMH-alike JSON benchmark result format. These classes are used for deserializing benchmark output, allowing flexible handling of parameters.

```kotlin
// Serialization classes matching the JMH-alike JSON format.
// We define these classes manually so we can keep `params` as a JsonObject, as it means we can handle them
// in a generic manner. If you benchmark have fixed params, using `"<jsonText>".deserializeThis()` is
// faster and easier.

@Serializable
public data class Benchmark(
    public val benchmark: String,
    public val mode: String,
    public val warmupIterations: Int,
    public val warmupTime: String,
    public val measurementIterations: Int,
    public val measurementTime: String,
    public val primaryMetric: PrimaryMetric,
    public val secondaryMetrics: Map<String, PrimaryMetric>,
    public val params: JsonObject? = null
)

@Serializable
public data class PrimaryMetric(
    public val score: Double,
    public val scoreError: Double,
    public val scoreConfidence: List<Double>,
    public val scorePercentiles: Map<String, Double>,
    public val scoreUnit: String,
    public val rawData: List<List<Double>>,
)
```

--------------------------------

### Filter for Interesting Benchmarks

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Identifies benchmarks where the score ranges do not overlap, indicating significant differences. This filtering helps focus on the most impactful changes.

```kotlin
fun kotlin.ranges.ClosedFloatingPointRange<kotlin.Double>.overlaps(other: ClosedFloatingPointRange<kotlin.Double>): Boolean = 
    this.start <= other.endInclusive && other.start <= this.endInclusive

val interestingBenchmarks = plotData.filter {
    !it.range.overlaps(it.range1)
}
interestingBenchmarks
```

--------------------------------

### Plot Interesting Benchmark Score Differences

Source: https://github.com/kotlin/kotlinx-benchmark/blob/master/examples/compare-benchmark-runs.ipynb

Visualizes the score differences for the filtered 'interesting' benchmarks. This plot is similar to the general score difference plot but focuses on significant changes.

```kotlin
interestingBenchmarks.sortBy { diffScorePercentage }.plot {
    barsH {
        x(diffScorePercentage) {
            axis.name = "Diff %"
        }
        y(testLabel) {
            axis.name = ""
        }
        fillColor(barColor) {
            scale = categorical("neg" to Color.RED, "pos" to Color.GREEN)
            legend.type = LegendType.None
        }
        tooltips {
            line(diffScorePercentage, format = ".2f")
        }
    }
    layout {
        size = 800 to ((40 * interestingBenchmarks.size().nrow) + 100)
        style {
            global {
                title {
                    margin(10.0, 0.0)
                }
            }
        }
    }
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.