### Install pyperf and its dependencies

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

Provides commands to install the pyperf benchmarking tool using pip. It also includes commands to upgrade setuptools to resolve potential installation errors and install the optional psutil dependency for memory tracking.

```bash
python3 -m pip install pyperf
```

```bash
python3 -m pip install -U setuptools
```

```bash
python3 -m pip install -U psutil
```

--------------------------------

### Example Output of pyperf metadata Command

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

Provides an example of the metadata displayed by the `pyperf metadata` command when run on `telco.json`. It shows various system and benchmark-specific details like ASLR status, boot time, CPU configuration, and affinity.

```Shell
$ python3 -m pyperf metadata telco.json
Metadata:
- aslr: Full randomization
- boot_time: 2016-10-19 01:10:08
- cpu_affinity: 2-3
- cpu_config: 2-3=driver:intel_pstate, intel_pstate:turbo, governor:performance, isolated; idle:intel_idle
```

--------------------------------

### Install pyperf Python Module

Source: https://github.com/psf/pyperf/blob/main/README.rst

Provides the standard command to install the `pyperf` module using pip. This ensures the benchmarking toolkit is available in your Python 3 environment for use.

```Shell
python3 -m pip install pyperf
```

--------------------------------

### Install CPU Performance Tools on Fedora

Source: https://github.com/psf/pyperf/blob/main/doc/system.rst

This command installs `turbostat` and `cpupower` utilities on Fedora-based systems. These tools are essential for monitoring and analyzing dynamic CPU features like Turbo Boost, P-state, and C-state, which significantly impact CPU performance and benchmark stability.

```Shell
dnf install -y kernel-tools
```

--------------------------------

### Write Benchmark Script with pyperf.Runner

Source: https://github.com/psf/pyperf/blob/main/README.rst

Illustrates how to create a dedicated Python script to define and execute a benchmark using the `pyperf.Runner` class. This approach offers more control over the benchmark setup, including the statement to be timed and any necessary setup code.

```Python
#!/usr/bin/env python3
import pyperf

runner = pyperf.Runner()
runner.timeit(name="sort a sorted list",
                  stmt="sorted(s, key=f)",
                  setup="f = lambda x: x; s = list(range(1000))")
```

--------------------------------

### Example Output of pyperf dump Command

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

Demonstrates the standard output of the `pyperf dump` command when run on a `telco.json` benchmark file. It shows calibration steps, warmup values, and benchmark results.

```Shell
$ python3 -m pyperf dump telco.json
Run 1: calibrate the number of loops: 8
- calibrate 1: 23.1 ms (loops: 1, raw: 23.1 ms)
- calibrate 2: 22.5 ms (loops: 2, raw: 45.0 ms)
- calibrate 3: 22.5 ms (loops: 4, raw: 89.9 ms)
- calibrate 4: 22.4 ms (loops: 8, raw: 179 ms)
Run 2: 1 warmup, 3 values, 8 loops
- warmup 1: 22.5 ms
- value 1: 22.8 ms
- value 2: 22.5 ms
- value 3: 22.6 ms
(...)
Run 41: 1 warmup, 3 values, 8 loops
- warmup 1: 22.5 ms
- value 1: 22.6 ms
- value 2: 22.4 ms
- value 3: 22.4 ms
```

--------------------------------

### Run a basic pyperf timeit benchmark

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

Demonstrates how to use the pyperf timeit command to measure the execution time of a Python instruction. It shows the command-line syntax and an example of the output, including mean and standard deviation.

```bash
$ python3 -m pyperf timeit '[1,2]*1000'
```

--------------------------------

### pyperf Benchmark Result JSON Format Example

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Illustrates the default JSON structure used by `pyperf` to store benchmark results. This example shows the top-level `benchmarks` array, individual run details, associated `metadata`, and `warmups` data, demonstrating the hierarchical organization of benchmark data.

```JSON
{
    "benchmarks": [
        {
            "runs": [
                {
                    "metadata": {
                        "date": "2016-10-21 03:14:19.670631",
                        "duration": 0.33765527700597886
                    },
                    "warmups": [
                        [
                            1,
                            0.023075559991411865
                        ],
                        [
                            2,
                            0.022522017497976776
                        ],
                        [
                            4,
                            0.02247579424874857
                        ],
                        [
                            8,
                            0.02237467262420978
                        ]
                    ]
                }
            ]
        }
    ]
}
```

--------------------------------

### Example Output of pyperf dump in Verbose Mode

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

Illustrates the detailed output of the `pyperf dump` command when executed with the `--verbose` option. It includes extensive metadata about the system, CPU, and run specifics for each benchmark run.

```Shell
$ python3 -m pyperf dump telco.json -v
Metadata:
  cpu_affinity: 2-3
  cpu_config: 2-3=driver:intel_pstate, intel_pstate:turbo, governor:performance, isolated; idle:intel_idle
  cpu_count: 4
  cpu_model_name: Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
  hostname: selma
  loops: 8
  name: telco
  perf_version: 0.8.2
  ...

Run 1: calibrate the number of loops
- calibrate 1: 23.1 ms (loops: 1, raw: 23.1 ms)
- calibrate 2: 22.5 ms (loops: 2, raw: 45.0 ms)
- calibrate 3: 22.5 ms (loops: 4, raw: 89.9 ms)
- calibrate 4: 22.4 ms (loops: 8, raw: 179 ms)
- Metadata:
  cpu_freq: 2=3596 MHz, 3=1352 MHz
  cpu_temp: coretemp:Physical id 0=67 C, coretemp:Core 0=51 C, coretemp:Core 1=67 C
  date: 2016-10-21 03:14:19.670631
  duration: 338 ms
  load_avg_1min: 0.29
  ...
Run 2:
- warmup 1: 22.5 ms
- value 1: 22.8 ms
- value 2: 22.5 ms
- value 3: 22.6 ms
- Metadata:
  cpu_freq: 2=3596 MHz, 3=2998 MHz
  cpu_temp: coretemp:Physical id 0=67 C, coretemp:Core 0=51 C, coretemp:Core 1=67 C
  date: 2016-10-21 03:14:20.496710
  duration: 723 ms
  load_avg_1min: 0.29
  ...
...
```

--------------------------------

### Python: Runner.timeit Method

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Runs a benchmark using `timeit.Timer` functionality. It allows specifying a statement (`stmt`), setup code (`setup`), teardown code (`teardown`), and a custom namespace (`globals`). The `teardown` parameter was added in version 1.6.0, and `stmt` became optional in the same version.

```APIDOC
timeit(name, stmt=None, setup="pass", teardown="pass", inner_loops=None, duplicate=None, metadata=None, globals=None)
	name: The benchmark name, which must be unique within the same script.
	stmt: A Python statement or sequence of statements to benchmark. Optional since 1.6.0.
	setup: A Python statement or sequence of statements executed before computing each benchmark value.
	teardown: A Python statement or sequence of statements executed after computing each benchmark value. Added in 1.6.0.
	inner_loops: Number of inner-loops, used when 'stmt' manually duplicates the same expression multiple times.
	duplicate: Duplicates the 'stmt' statement 'duplicate' times to reduce the cost of the outer loop.
	metadata: Metadata for this benchmark, added to the runner's metadata.
	globals: Namespace used to run 'setup', 'teardown', and 'stmt'. By default, an empty namespace is created.
```

--------------------------------

### Benchmark String Strip Operation with pyperf timeit

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

This example demonstrates how to use `pyperf timeit` to benchmark the `strip()` method on a string. It utilizes the `--duplicate` option to multiply inner loops, reducing overhead and providing more accurate performance measurements for simple operations.

```bash
python3 -m pyperf timeit '" abc ".strip()' --duplicate=1024
```

--------------------------------

### IRQ Affinity Management

Source: https://github.com/psf/pyperf/blob/main/doc/system.rst

Handles the state of the `irqbalance` service and manages CPU affinity for interruptions. `tune` stops the `irqbalance` service, and `reset` starts it. It also reads/writes to `/proc/irq/default_smp_affinity` and `/proc/irq/N/smp_affinity`.

```APIDOC
Operation: IRQ affinity
Service: irqbalance
Files: /proc/irq/default_smp_affinity, /proc/irq/N/smp_affinity
Description: Handle the state of the irqbalance service and CPU affinity of interruptions.
Tune Action: Stops irqbalance service; manages smp_affinity.
Reset Action: Starts irqbalance service; manages smp_affinity.
```

--------------------------------

### Get Benchmark Dates (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the start date of the first run and the end date of the last run. Returns a (start, end) tuple of datetime.datetime objects if date metadata exists for at least one run, otherwise returns None.

```APIDOC
get_dates() -> (datetime.datetime, datetime.datetime) or None
  Returns: (start, end) tuple of datetime.datetime or None
```

--------------------------------

### Warning for `nohz_full` with `intel_pstate` in `check` and `system`

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

The `check` and `system` commands now issue a warning if the `nohz_full` kernel parameter is detected in conjunction with the `intel_pstate` driver. This helps users identify potential configuration conflicts or suboptimal setups.

```APIDOC
Commands: check, system
Warning Condition: nohz_full used with intel_pstate driver.
```

--------------------------------

### Run pyperf benchmark with Python 3.8

Source: https://github.com/psf/pyperf/blob/main/doc/analyze.rst

This command executes a `pyperf timeit` benchmark using Python 3.8, measuring the execution time of the same list multiplication as the Python 3.6 example. The results are saved to `py38.json`.

```Shell
python3.8 -m pyperf timeit '[1,2]*1000' -o py38.json
```

--------------------------------

### Run pyperf Benchmark to Observe Outliers

Source: https://github.com/psf/pyperf/blob/main/doc/analyze.rst

This example shows how to run a pyperf timeit benchmark that is likely to produce outliers, saving the results to a JSON file. The output includes a warning about unstable results, indicating the presence of values significantly higher than the mean.

```Shell
$ python3 -m pyperf timeit '[1,2]*1000' -o outliers.json
```

--------------------------------

### Example Output of pyperf hist Command

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

Shows a sample text-based histogram generated by `pyperf hist` from a `telco.json` benchmark file. The output displays frequency counts for different performance values, visualizing the distribution of benchmark results.

```Shell
$ python3 -m pyperf hist telco.json
26.4 ms:  1 ##
26.4 ms:  1 ##
26.4 ms:  2 #####
26.5 ms:  1 ##
26.5 ms:  1 ##
26.5 ms:  4 #########
26.6 ms:  8 ###################
26.6 ms:  6 ##############
26.7 ms: 11 ##########################
26.7 ms: 13 ##############################
26.7 ms: 18 ##########################################
26.8 ms: 21 #################################################
26.8 ms: 34 ###############################################################################
26.8 ms: 26 ############################################################
26.9 ms: 11 ##########################
26.9 ms: 14 #################################
27.0 ms: 17 ########################################
27.0 ms: 14 #################################
27.0 ms: 10 #######################
27.1 ms: 10 #######################
27.1 ms:  7 ################
27.1 ms: 12 ############################
27.2 ms:  5 ############
27.2 ms:  2 #####
27.3 ms:  0 |
27.3 ms:  1 ##
```

--------------------------------

### Compare Python Versions with pyperf timeit

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

This example illustrates how to compare the performance of a Python operation between two different Python versions using `pyperf timeit`. The `--compare-to` option facilitates direct comparison, showing performance differences and percentage changes between the specified Python executables.

```bash
python3.8 -m pyperf timeit '" abc ".strip()' --duplicate=1024 --compare-to=python3.6
```

--------------------------------

### Get All Benchmark Values (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves all recorded values from all runs of the benchmark.

```APIDOC
get_values()
  Returns: (unspecified, likely list of values)
```

--------------------------------

### Get List of Run Objects (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves a list of all Run objects associated with the benchmark.

```APIDOC
get_runs() -> List[Run]
  Returns: List[Run]
```

--------------------------------

### Demonstrate unstable pyperf benchmark with low loops

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

Shows an example of an unstable benchmark result when the --loops parameter is set too low. It highlights the warnings generated by pyperf regarding high standard deviation and large maximum values, indicating unreliable measurements.

```bash
$ python3 -m pyperf timeit --loops=10 pass
```

--------------------------------

### pyperf Benchmark Result JSON Structure

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

This snippet provides a detailed example of the JSON format used by pyperf to store benchmark results. It includes an array of individual run results, each with metadata (date, duration), measured values, and warmup data. Additionally, it shows top-level metadata about the system (CPU, hostname) and the benchmark itself (description, version).

```JSON
]
                    },
                    {
                        "metadata": {
                            "date": "2016-10-21 03:14:20.496710",
                            "duration": 0.7234010050015058,
                        },
                        "values": [
                            0.022752201875846367,
                            0.022529058374857414,
                            0.022569017250134493
                        ],
                        "warmups": [
                            [
                                8,
                                0.02249833550013136
                            ]
                        ]
                    },
                    ...
                    {
                        "metadata": {
                            "date": "2016-10-21 03:14:52.549713",
                            "duration": 0.719920061994344,
                            ...
                        },
                        "values": [
                            0.022562820375242154,
                            0.022442164625317673,
                            0.02241712374961935
                        ],
                        "warmups": [
                            [
                                8,
                                0.02249412499986647
                            ]
                        ]
                    }
                ]
            }
        ],
        "metadata": {
            "cpu_count": 4,
            "cpu_model_name": "Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz",
            "description": "Telco decimal benchmark",
            "hostname": "selma",
            "loops": 8,
            "name": "telco",
            "perf_version": "0.8.2",
            "tags": ["numeric"],
            ...
        },
        "version": "1.0"
    }
```

--------------------------------

### Get Common Benchmark Metadata (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves metadata common to all runs within the benchmark. The format_metadata function can be used for value formatting. Refer to the 'Metadata' section for more details.

```APIDOC
get_metadata() -> dict
  Returns: dict
```

--------------------------------

### Avoid Aggressive PyPy JIT Tuning for Benchmarks

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

This snippet illustrates an example of an overly aggressive JIT configuration for PyPy (`pypy --jit threshold=1,function_threshold=1`) that should be avoided during benchmarking. Such settings can lead to unrepresentative results, excessive tracing, and increased garbage collector pressure, making benchmarks unreliable.

```Shell
pypy --jit threshold=1,function_threshold=1
```

--------------------------------

### Configure Linux Kernel for CPU Isolation (isolcpus)

Source: https://github.com/psf/pyperf/blob/main/doc/system.rst

This snippet illustrates how to modify the Linux kernel command line via GRUB to add the `isolcpus` parameter. This isolates specific CPU cores from the general scheduler, dedicating them for critical tasks like benchmarks to reduce interference and improve stability. The example isolates logical CPUs 2, 3, 6, and 7.

```Shell
isolcpus=2,3,6,7
```

--------------------------------

### CLI: pyperf Command Line Options

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

Documentation for various command-line options available in the pyperf benchmarking tool, including options for running benchmarks, managing environment, and converting data.

```CLI
--worker-task <identifier>: Execute a specific benchmark function by its identifier.
--python: Specify the Python executable to use for benchmarks.
--name <name>: (timeit) Assign a name to the benchmark run.
--inner-loops <count>: (timeit) Specify the number of inner loops for timeit benchmarks.
--compare-to <file>: (timeit) Compare current benchmark results to a specified file.
--affinity: (metadata command) Control CPU affinity for metadata collection.
--inherit-environ: Inherit environment variables in worker processes.
--tracemalloc: Use the tracemalloc module to track Python memory allocation and get peak usage.
--track-memory: Run a thread to read memory usage every millisecond and store the peak.
--remove-all-metadata: (convert command) Remove all metadata from a benchmark file.
--update-metadata: (convert command) Update metadata in a benchmark file.
--extract-metadata=NAME: (convert command) Extract specific metadata by name.
--group-by-speed (-G): (compare_to command) Group comparison results by speed.
--min-speed: (compare_to command) Set a minimum speed threshold for comparison.
-b: (show/stats commands) Specify benchmark(s) to operate on.
--append: Append results to an existing benchmark file.
slowest: Display the slowest benchmarks.
stats: Display statistics about benchmarks.
```

--------------------------------

### Run pyperf benchmark with JSON output and warnings

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

Illustrates running a pyperf timeit benchmark and directing its output to a JSON file using the -o option. It also highlights common warnings that indicate unstable benchmark results and suggests ways to address them.

```bash
$ python3 -m pyperf timeit '[1,2]*1000' -o json2
```

--------------------------------

### Get Benchmark Name (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the name of the benchmark as a string.

```APIDOC
get_name() -> str
  Returns: str
```

--------------------------------

### Execute pyperf Benchmark Script

Source: https://github.com/psf/pyperf/blob/main/README.rst

Shows how to run a Python benchmark script (e.g., `bench.py`) created with `pyperf.Runner` from the command line. The `-o` flag directs the benchmark results to a specified JSON output file.

```Shell
$ python3 bench.py -o bench.json
```

--------------------------------

### Get Number of Runs (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the total number of runs recorded for the benchmark.

```APIDOC
get_nrun() -> int
  Returns: int
```

--------------------------------

### Verify Isolated and NOHZ Full CPU Settings (Linux Shell)

Source: https://github.com/psf/pyperf/blob/main/doc/system.rst

These commands check the current configuration of isolated CPUs and CPUs configured for nohz_full by reading respective sysfs entries. This helps confirm that kernel boot parameters have been applied correctly and are active on the system.

```Shell
cat /sys/devices/system/cpu/isolated
```

```Shell
cat /sys/devices/system/cpu/nohz_full
```

--------------------------------

### Run Benchmark with pyperf timeit Command

Source: https://github.com/psf/pyperf/blob/main/README.rst

Demonstrates how to run a simple benchmark directly from the command line using the `pyperf timeit` utility. This method is ideal for quick performance checks of Python expressions or statements, saving the results to a specified JSON file.

```Shell
$ python3 -m pyperf timeit '[1,2]*1000' -o bench.json
```

--------------------------------

### Get Total Number of Values (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the total count of all values across all runs.

```APIDOC
get_nvalue() -> int
  Returns: int
```

--------------------------------

### Get Value Unit (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the unit of the benchmark's values. Possible units include 'byte' (file size), 'integer' (integer number), or 'second' (duration).

```APIDOC
get_unit() -> str
  Returns: str
  Possible values: 'byte', 'integer', 'second'
```

--------------------------------

### New `system tune` Command for Benchmark System Tuning

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

A new `system tune` command has been introduced to optimize the system for benchmarking. It provides functionalities such as disabling Turbo Boost, verifying isolated CPUs, setting CPU frequency, and configuring the CPU scaling governor to 'performance'.

```APIDOC
Command: system tune
Purpose: Tune system for benchmarks.
Capabilities:
  - Disable Turbo Boost
  - Check isolated CPUs
  - Set CPU frequency
  - Set CPU scaling governor to "performance"
```

--------------------------------

### Get Total Benchmark Duration (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Calculates the total duration of the benchmark in seconds. This is derived from the 'duration' metadata of runs or by summing their raw values, including warmup values.

```APIDOC
get_total_duration() -> float
  Returns: float
```

--------------------------------

### Compare Benchmark Results with pyperf compare_to

Source: https://github.com/psf/pyperf/blob/main/README.rst

Illustrates how to use the `pyperf compare_to` command to compare multiple benchmark suites. This tool helps determine if performance differences between various runs or environments are statistically significant, presenting the comparison in a tabular format.

```Shell
$ python3 -m pyperf compare_to --table mult_list_py36.json mult_list_py37.json mult_list_py38.json
```

--------------------------------

### Get Number of Warmup Values (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the number of warmup values per run. Returns an integer if all runs have the same number of warmups, or a float representing the average otherwise.

```APIDOC
get_nwarmup() -> int or float
  Returns: int or float
```

--------------------------------

### Add pybench Program

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

Introduces a new `pybench` program, designed to be similar in functionality to `python3 -m perf`, providing an alternative entry point for performance benchmarking.

```CLI
pybench
python3 -m perf
```

--------------------------------

### Configure pyperf for Stable Benchmarking

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

This section describes command-line options and general advice for reducing system noise and ensuring consistent benchmark results. It highlights the `--no-locale` option for locale-independent runs and refers to system tuning commands.

```Shell
--no-locale
```

```Shell
pyperf system tune
```

--------------------------------

### pyperf Runner CLI Command-Line Options

Source: https://github.com/psf/pyperf/blob/main/doc/runner.rst

Comprehensive documentation of command-line arguments available for the `pyperf.Runner` class. This includes options for benchmark execution control, output formatting, JSON result handling, and various utility settings.

```APIDOC
Runner CLI Options:
  Loop iterations:
    --rigorous: Spend longer running tests to get more accurate results. Multiply the number of PROCESSES by 2. Default: 40 processes and 3 values per process (120 values).
    --fast: Get rough answers quickly. Divide the number of PROCESSES by 2 and multiply the number of VALUES by 2/3 (0.6). Default: 10 processes and 2 values per process (total: 20 values).
    -p PROCESSES, --processes=PROCESSES: number of processes used to run the benchmark (default: 20, or 6 with a JIT)
    -n VALUES, --values=VALUES: number of values per process (default: 3, or 10 with a JIT)
    -l LOOPS, --loops=LOOPS: number of loops per value. x^y syntax is accepted, example: --loops=2^8 uses 256 iterations. By default, the timer is calibrated to get raw values taking at least MIN_TIME seconds.
    -w WARMUPS, --warmups=WARMUPS: the number of ignored values used to warmup to benchmark (default: 1, or 10 with a JIT)
    --min-time=MIN_TIME: Minimum duration of a single raw value in seconds (default: 100 ms)

  Output options:
    -d, --dump: displays the benchmark run results, see pyperf dump command
    -m, --metadata: displays metadata: see pyperf show metadata command
    -g, --hist: renders an histogram of values, see pyperf hist command
    -t, --stats: displays statistics (min, max, ...), see pyperf stats command
    -v, --verbose: enables the verbose mode
    -q, --quiet: enables the quiet mode

  JSON output:
    -o FILENAME, --output=FILENAME: writes the benchmark result as JSON into FILENAME
    --append=FILENAME: appends the benchmark runs to benchmarks of the JSON file FILENAME. The file is created if it doesn't exist.
    --pipe=FD: writes benchmarks encoded as JSON into the pipe FD.

  Misc:
    -h, --help: Displays help message and exits.
    --python=PYTHON: Python executable. By default, use the running Python (sys.executable). The Python executable must have the pyperf module installed.
    --compare-to=REF_PYTHON: Run benchmark on the Python executable REF_PYTHON, run benchmark on Python executable PYTHON, and then compare REF_PYTHON result to PYTHON result.
    --python-names=REF_NAME:CHANGED_NAME: Option used with --compare-to to name PYTHON as CHANGED_NAME and name REF_PYTHON as REF_NAME in results. For example, ./python ... --compare-to=../ref/python --python-names=ref:patch uses "ref" name for ../ref/python and use "patch" name for ./python.
    --affinity=CPU_LIST: Specify CPU affinity for worker processes. This way, benchmarks can be forced to run on a given set of CPUs to minimize run to run variation. By default, worker processes are pinned to isolate CPUs if isolated CPUs are found. See CPU pinning and CPU isolation.
    --inherit-environ=VARS: VARS is a comma-separated list of environment variable names which are inherited by worker child processes. By default, only the following variables are inherited: PATH, PYTHONPATH, HOME, TEMP, COMSPEC, SystemRoot, SystemDrive, and locale environment variables. See the --no-locale below for locale environment variables.
    --copy-env: Inherit all environment variables.
    --no-locale: Don't inherit locale environment variables: LANG, LC_ADDRESS, LC_ALL, LC_COLLATE, LC_CTYPE, LC_IDENTIFICATION, LC_MEASUREMENT, LC_MESSAGES, LC_MONETARY, LC_NAME, LC_NUMERIC, LC_PAPER, LC_TELEPHONE, LC_TIME
    --timeout TIMEOUT: set a timeout in seconds for an execution of the benchmark. If the benchmark execution times out, pyperf exits with error code 124. There is no time out by default.
    --tracemalloc: Use the tracemalloc module to track Python memory
```

--------------------------------

### pyperf timeit Command Line Interface

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

This entry outlines the command-line syntax and available options for `pyperf timeit`, a tool designed for benchmarking Python code. It details parameters such as benchmark naming, Python executable selection, inner loop configuration, and setup/teardown statements for precise performance measurement.

```APIDOC
pyperf timeit command:
  Usage: python3 -m pyperf timeit [options] [--name BENCHMARK_NAME] [--python PYTHON] [--compare-to REF_PYTHON] [--inner-loops INNER_LOOPS] [--duplicate DUPLICATE] [-s SETUP] [--teardown TEARDOWN] [--profile PROFILE] stmt [stmt ...]
  Options:
    [options]: Refer to Runner CLI for additional options.
    stmt: Python code to be executed in the benchmark. Multiple statements are supported.
    -s SETUP, --setup SETUP: Statement(s) run before the tested statement. Can be specified multiple times.
    --teardown TEARDOWN: Statement(s) run after the tested statement. Can be specified multiple times.
    --name=BENCHMARK_NAME: Custom name for the benchmark (default: 'timeit').
    --inner-loops=INNER_LOOPS: Number of inner loops per value to reduce outer loop overhead.
    --compare-to=REF_PYTHON: Path to a Python executable for comparison against the primary Python executable.
    --duplicate=DUPLICATE: Multiplies inner loops by duplicating statements to reduce outer loop overhead.
    --profile=PROFILE: Path to output file for cProfile profiler results. Note: profiling affects timing accuracy.
```

--------------------------------

### Runner Class API Documentation

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Documentation for the `Runner` class, a tool for executing benchmarks in text mode, managing worker processes, and configuring benchmark execution parameters.

```APIDOC
Runner class:
  Runner(values=3, warmups=1, processes=20, loops=0, min_time=0.1, metadata=None, show_name=True, program_args=None, add_cmdline_args=None)
    Description: Tool to run a benchmark in text mode. Spawn processes worker processes to run the benchmark. Only one instance of Runner must be created. Use the same instance to run all benchmarks.
    Parameters:
      values: The default number of values.
      warmups: The default number of warmup values.
      processes: The default number of processes.
      loops: (Not explicitly described, but part of signature)
      min_time: (Not explicitly described, but part of signature)
      metadata: Passed to the Run constructor.
      show_name: If true, displays the benchmark name.
      program_args: A list of strings passed to Python on the command line to run the program. By default, (sys.argv[0],) is used.
      add_cmdline_args: An optional callback used to add command line arguments to the command line of worker processes. The callback is called with add_cmdline_args(cmd, args).

  Methods to run benchmarks:
    bench_func
    bench_async_func
    timeit
    bench_command
    bench_time_func
```

--------------------------------

### `system tune` Manages `irqbalance` and IRQ Affinity

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

The `system tune` command now includes functionality to stop the `irqbalance` service and configure the CPU affinity for interruptions (IRQs). This provides more precise control over system resources during benchmarking.

```APIDOC
Command: system tune
Actions:
  - Stops irqbalance service.
  - Sets CPU affinity for interruptions (IRQ).
```

--------------------------------

### BenchmarkSuite Class API Documentation

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Documentation for the `BenchmarkSuite` class, which manages a collection of `Benchmark` objects, including methods for adding, retrieving, and serializing benchmarks.

```APIDOC
BenchmarkSuite class:
  BenchmarkSuite(benchmarks, filename=None)
    Description: A benchmark suite is made of Benchmark objects.
    Parameters:
      benchmarks: A non-empty sequence of Benchmark objects.
      filename: The name of the file from which the suite was loaded.

  Methods:
    add_benchmark(benchmark: Benchmark)
      Description: Add a Benchmark object. A suite cannot contain two benchmarks with the same name, because the name is used as a unique key.
      Parameters:
        benchmark: The Benchmark object to add.

    add_runs(bench: Benchmark or BenchmarkSuite)
      Description: Add runs of benchmarks.
      Parameters:
        bench: Can be a Benchmark or a BenchmarkSuite.

    dump(file, compact=True, replace=False)
      Description: Dump the benchmark suite as JSON into file. If file is a filename ending with .gz, the file is compressed by gzip. If file is a filename and replace is false, the function fails if the file already exists.
      Parameters:
        file: Can be a filename, or a file object open for write.
        compact: If true, generate compact file. Otherwise, indent JSON.
        replace: If file is a filename and replace is false, the function fails if the file already exists.

    get_benchmark(name: str) -> Benchmark
      Description: Get the benchmark called name. Raises KeyError if there is no benchmark called name.
      Parameters:
        name: Must be non-empty.

    get_benchmark_names() -> List[str]
      Description: Get the list of benchmark names.

    get_benchmarks() -> List[Benchmark]
      Description: Get the list of benchmarks.

    get_dates() -> (datetime.datetime, datetime.datetime) or None
      Description: Get the start date of the first benchmark and end date of the last benchmark. Returns a (start, end) tuple where start and end are datetime.datetime objects if at least one benchmark has dates. Returns None if no benchmark has dates.

    get_metadata() -> dict
      Description: Get metadata common to all benchmarks (common to all runs of all benchmarks).

    get_total_duration() -> float
      Description: Get the total duration of all benchmarks in seconds.

    __iter__()
      Description: Iterate on benchmarks.

    __len__() -> int
      Description: Get the number of benchmarks.

    load(file) (classmethod)
      Description: Load a benchmark suite from a JSON file which was created by dump.
      Parameters:
        file: Can be a filename, '-' string to load from sys.stdin, or a file object open to read.

    loads(string) -> Benchmark (classmethod)
      Description: Load a benchmark suite from a JSON string.
      Parameters:
        string: The JSON string to load.

  Attributes:
    filename
      Description: Name of the file from which the benchmark suite was loaded. It can be None.
```

--------------------------------

### Get Total Loops Per Value (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Calculates the total number of loops per value (outer-loops x inner-loops). Returns an integer if all runs have the same total loops, or a float representing the average otherwise.

```APIDOC
get_total_loops() -> int or float
  Returns: int or float
```

--------------------------------

### Get Inner Loop Iterations (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the number of inner loop iterations for runs. Returns an integer if all runs have the same number of inner loops, or a float representing the average otherwise. Added in version 1.3.

```APIDOC
get_inner_loops() -> int or float
  Returns: int or float
```

--------------------------------

### Get Outer Loop Iterations (pyperf)

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Retrieves the number of outer loop iterations for runs. Returns an integer if all runs have the same number of outer loops, or a float representing the average otherwise. Added in version 1.3.

```APIDOC
get_loops() -> int or float
  Returns: int or float
```

--------------------------------

### Profile pyperf Benchmarks using Linux perf record

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

This snippet describes how `pyperf` integrates with the Linux `perf record` tool for profiling benchmark execution. It specifies the naming convention for generated profile data files (`perf.data.<uuid>`) and provides environment variables (`PYPERF_PERF_RECORD_DATA_DIR`, `PYPERF_PERF_RECORD_EXTRA_OPTS`) to customize output directory and `perf record` command-line options.

```Shell
perf record
```

```File System
perf.data.<uuid>
```

```Environment Variable
PYPERF_PERF_RECORD_DATA_DIR
PYPERF_PERF_RECORD_EXTRA_OPTS
```

--------------------------------

### New pyperf Benchmark and Runner Methods

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

This section documents new methods added to the `Benchmark` and `Runner` classes, enhancing pyperf's capabilities for statistical analysis and command execution. These methods provide more granular control and access to benchmark data.

```APIDOC
Benchmark:
  percentile()
  mean()
  median_abs_dev()
  stdev()

Runner:
  bench_command()
  timeit()
```

--------------------------------

### pyperf.Run Class Attributes

Source: https://github.com/psf/pyperf/blob/main/doc/api.rst

Documents the key attributes of the `pyperf.Run` class, which manages benchmark execution and data collection. It details how arguments are stored, the `argparse` object used, and the `metadata` dictionary for benchmark information.

```APIDOC
Class: Run
  Attributes:
    args:
      Description: Namespace of arguments; result of the parse_args method, None before parse_args is called.
      Type: argparse.Namespace or None
    argparser:
      Description: An argparse.ArgumentParser object used to parse command line options.
      Type: argparse.ArgumentParser
    metadata:
      Description: Benchmark metadata.
      Type: dict
```

--------------------------------

### New Run Class for Normalized Samples

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

Introduces a new `Run` class designed to store normalized samples, moving away from raw sample storage for improved data consistency and analysis.

--------------------------------

### CLI Commands Support Multiple Files and Suites

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

Enhances the `perf` command-line interface to allow most commands to operate on multiple input files and to support benchmark suites, improving flexibility for complex benchmarking scenarios.

```CLI
perf <command> file1.json file2.json
```

--------------------------------

### API: pyperf.Benchmark Class Methods

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

Documentation for methods of the `Benchmark` class in pyperf, used for managing benchmark data and properties.

```APIDOC
Benchmark.dump(replace: bool = False):
  Dumps the benchmark data to a file.
  Parameters:
    replace (bool): If True, allows overwriting an existing file. Defaults to False.
Benchmark.get_unit():
  Returns the unit of the benchmark (e.g., 'second', 'byte', 'integer').
Benchmark.update_metadata():
  Updates the metadata associated with the benchmark.
```

--------------------------------

### CPU Scaling Governor Management (intel_pstate driver)

Source: https://github.com/psf/pyperf/blob/main/doc/system.rst

Manages the CPU scaling governor via the intel_pstate driver. The `tune` operation sets the governor to 'performance', while `reset` sets it to 'powersave'.

```APIDOC
Operation: CPU scaling governor (intel_pstate driver)
Description: Get/Set the CPU scaling governor.
Tune Action: Sets governor to 'performance'.
Reset Action: Sets governor to 'powersave'.
```

--------------------------------

### Analyze pyperf benchmark results with dump command

Source: https://github.com/psf/pyperf/blob/main/doc/run_benchmark.rst

Explains how to use the pyperf dump command with the --verbose option to inspect the detailed results of a previously saved benchmark run. This command helps in understanding the timing and process information.

```bash
python3 -m pyperf dump --verbose bench.json
```

--------------------------------

### pyperf system: Manage System State for Benchmarks

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

The `pyperf system` command allows users to inspect, tune, or reset the system's state to optimize for benchmark execution. It can also specify CPU affinity for the operations.

```APIDOC
python3 -m pyperf system
  [--affinity=CPU_LIST]
  [{show,tune,reset}]

Commands:
  pyperf system show: Shows the current state of the system.
  pyperf system tune: Tunes the system to run benchmarks.
  pyperf system reset: Resets the system to the default state.

Options:
  --affinity=CPU_LIST: Specify CPU affinity. By default, use isolate CPUs.
```

--------------------------------

### Handle Missing MSR Device in `system tune` Command

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

The `system tune` command now gracefully handles scenarios where the `/dev/cpu/N/msr` device is missing. It logs an error and suggests loading the `msr` kernel module, preventing crashes related to device unavailability.

```APIDOC
Command: system tune
Behavior: Logs error and suggests loading 'msr' kernel module if /dev/cpu/N/msr is missing.
```

--------------------------------

### Analyze Benchmark Results with pyperf stats

Source: https://github.com/psf/pyperf/blob/main/README.rst

Demonstrates using the `pyperf stats` command to display a detailed statistical analysis of benchmark results stored in a JSON file. It provides metrics such as mean, median, standard deviation, and various percentiles.

```Shell
$ python3 -m pyperf stats telco.json
```

--------------------------------

### New Benchmark Metadata Fields

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

Introduces new metadata fields for benchmarks, including `cpu_config`, `cpu_freq`, `cpu_temp`, and `load_avg_1min`, providing more detailed system information with benchmark results.

```APIDOC
Metadata:
  cpu_config: string
  cpu_freq: float
  cpu_temp: float
  load_avg_1min: float
```

--------------------------------

### API: pyperf.BenchmarkSuite Class Methods

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

Documentation for methods of the `BenchmarkSuite` class in pyperf, used for managing collections of benchmarks.

```APIDOC
BenchmarkSuite.dump(replace: bool = False):
  Dumps the benchmark suite data to a file.
  Parameters:
    replace (bool): If True, allows overwriting an existing file. Defaults to False.
BenchmarkSuite.get_metadata():
  Returns the metadata associated with the benchmark suite.
```

--------------------------------

### nohz_full Kernel Option Compatibility Check

Source: https://github.com/psf/pyperf/blob/main/doc/system.rst

Ensures that the `nohz_full` kernel option is not used with the `intel_pstate` CPU driver, as they are incompatible (referencing Bug 1378529).

```APIDOC
Check: nohz_full
Condition: CPU driver is intel_pstate
Description: Make sure that nohz_full kernel option is not used with intel_pstate driver due to incompatibility (Bug 1378529).
```

--------------------------------

### pyperf metadata Command-Line Options

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

Outlines the command-line arguments and options for the `pyperf metadata` command, which is used to display metadata from benchmark files. It allows filtering metadata by specific benchmark names.

```APIDOC
pyperf metadata:
  Usage: [-b NAME/--benchmark NAME] filename [filename2 ...]
  Options:
    --benchmark NAME: only displays the benchmark called NAME. Can be specified multiple times.
```

--------------------------------

### New and Renamed pyperf Command-Line Options

Source: https://github.com/psf/pyperf/blob/main/doc/changelog.rst

This section documents new command-line options added to the pyperf runner and existing options that have been renamed. These updates provide users with more control and flexibility when configuring and running benchmarks.

```APIDOC
Renamed CLI Options:
  --name=NAME -> --benchmark=NAME

New CLI Options:
  --python-names
  --compare-to
  --compare-to --table
  --no-locale
```

--------------------------------

### pyperf command: Run and Measure Benchmarks

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

This command is used to execute a program and measure its performance. It supports options for tracking maximum RSS memory, assigning a custom name to the benchmark, and specifying the command to be tested.

```APIDOC
pyperf command
  --track-memory: Use the maximum RSS memory of the command instead of the time.
  --name=BENCHMARK_NAME: Benchmark name (default: command).
  program [arg1 arg2 ...]: The tested command.
```

```shell
python3 -m pyperf command -- python3.6 -c pass
```

--------------------------------

### Compare pyperf benchmark suites

Source: https://github.com/psf/pyperf/blob/main/doc/cli.rst

The `pyperf compare_to` command compares multiple benchmark suites, using the first specified file as the reference. It uses a Student's two-sample, two-tailed t-test to determine significant differences and can compute the geometric mean for overall suite comparison. Results can be grouped by speed or displayed in a table format.

```Shell
python3 -m pyperf compare_to
    [-v/--verbose] [-q/--quiet]
    [-G/--group-by-speed]
    [--min-speed=MIN_SPEED]
    [--table]
    [--table-format=rest|md]
    [-b NAME/--benchmark NAME]
    reference.json changed.json [changed2.json ...]
```

```Shell
$ python3 -m pyperf compare_to py36.json py38.json
Mean +- std dev: [py36] 4.70 us +- 0.18 us -> [py38] 4.22 us +- 0.08 us: 1.11x faster
```

```Shell
$ python3 -m pyperf compare_to --table mult_list_py36.json mult_list_py37.json mult_list_py38.json
+----------------+----------------+-----------------------+-----------------------+
| Benchmark      | mult_list_py36 | mult_list_py37        | mult_list_py38        |
+================+================+=======================+=======================+
| [1]*1000       | 2.13 us        | 2.09 us: 1.02x faster | not significant       |
```