Hyperfoil

https://hyperfoil.io/docs
Admin
Exhaustive documentation for the Hyperfoil tool, providing overviews, quickstarts, user guides,...
Tokens:189,928
Snippets:2,952
Trust Score:6.6
Update:4 months ago
Show doc for...
Tokens:
Raw
### Run Hyperfoil Example Server

Source: https://hyperfoil.io/docs/getting-started/quickstart2

Start a local container to serve example requests. Replace 'podman' with 'docker' if you are using Docker.

```bash
podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples
```

--------------------------------

### Running Example Benchmark with Online Statistics

Source: https://hyperfoil.io/docs/_print

This command starts an example benchmark and allows for observing online statistics as it progresses. The `podman run` command is used to launch the example environment.

```bash
podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

```

```bash
[hyperfoil@localhost]$ upload .../eshop-scale.hf.yaml
Loaded benchmark eshop-scale, uploading...
... done.
[hyperfoil@localhost]$ run eshop-scale
Started run 0002
Run 0002, benchmark eshop-scale
...

```

--------------------------------

### Running the Hyperfoil Example Server

Source: https://hyperfoil.io/docs/howtos/credentials

This command starts the Hyperfoil example server using Podman. Ensure the server is running before executing benchmarks.

```bash
podman run --rm -p 8084:8084 quay.io/hyperfoil/hyperfoil-examples
```

--------------------------------

### Start Example Workload Server

Source: https://hyperfoil.io/docs/getting-started/quickstart7

Starts a Hyperfoil example workload server using Podman. This command maps port 8080 on the host to port 8083 in the container, ensuring the server is accessible.

```bash
podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

```

--------------------------------

### Example: OutOfBand and Final Phase Configuration

Source: https://hyperfoil.io/docs/user-guide/_print

Demonstrates how to configure an 'outOfBand' phase to start users after a delay and a 'final' phase to execute after other phases have terminated.

```yaml
- outOfBand:
    always:
      users: 5
      startTime: 2m
      duration: 2m
      scenario: ...
- final:
    atOnce:
      users: 1
      # Do something at the end: make sure that both rampUp and steadyState are terminated
      startAfterStrict:
      - rampUp
      - steadyState
      scenario: ...
```

--------------------------------

### Example Phase Configuration

Source: https://hyperfoil.io/docs/_print

This example demonstrates configuring a ramp-up phase followed by a steady-state phase. It shows how to define user rates, session limits, durations, and scenarios for each phase.

```yaml
phases:
# Over one minute ramp the number of users started each second from 1 to 100
- rampUp:
    increasingRate:
      initialUsersPerSec: 1
      targetUsersPerSec: 100
      # We expect at most 200 users being active at one moment - see below
      maxSessions: 200
      duration: 1m
      scenario: ...
# After rampUp is finished, run for 5 minutes and start 100 new users each second
- steadyState:
    constantRate:
      usersPerSec: 100
      maxSessions: 200
      startAfter: rampUp
      duration: 5m
      # If some users get stuck, forcefully terminate them after 6 minutes from the phase start
      maxDuration: 6m
      scenario: ...
```

--------------------------------

### Example: Running a Specific Benchmark

Source: https://hyperfoil.io/docs/howtos/_print

This is a concrete example of running a benchmark file named 'first-benchmark.yml' and saving its report to '/tmp/reports'.

```bash
./distribution/bin/run.sh -o /tmp/reports /tmp/first-benchmark.yml

```

--------------------------------

### Start Server and Fire Scenario

Source: https://hyperfoil.io/docs/getting-started/_print

Command to start the server and execute the defined Hyperfoil scenario.

```bash
hyperfoil --config /path/to/your/config.conf run --scenario /path/to/your/scenario.yaml
```

--------------------------------

### Starting Hyperfoil CLI and Running wrk

Source: https://hyperfoil.io/docs/migration/_print

This demonstrates starting the Hyperfoil CLI in interactive mode and then running a wrk benchmark locally.

```bash
podman run -it --rm quay.io/hyperfoil/hyperfoil cli
[hyperfoil]$ start-local --quiet
[hyperfoil@in-vm]$ wrk -t 2 -c 10 -H 'accept: application/json' -d 10s http://example.com
```

--------------------------------

### Start Server and Run Scenario

Source: https://hyperfoil.io/docs/getting-started/quickstart3

Command to start the Hyperfoil server and execute the defined scenario.

```bash
hyperfoil start
hyperfoil run --scenario quickstart3.yaml
```

--------------------------------

### Running a Benchmark via CLI

Source: https://hyperfoil.io/docs/user-guide/installation/k8s

Example of running a previously uploaded benchmark using the Hyperfoil CLI. This will start agent pods in the cluster.

```bash
[hyperfoil@hyperfoil-hyperfoil]$ run k8s-hello-world
Started run 0000
Run 0000, benchmark k8s-hello-world
Agents: agent-one[STARTING]
Started: 2019/11/18 19:07:36.752    Terminated: 2019/11/18 19:07:41.778
NAME  STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION               DESCRIPTION
main  TERMINATED  19:07:36.753             19:07:41.778  5025 ms (exceeded by 25 ms)  5.00 users per second

```

--------------------------------

### Start Hyperfoil Controller

Source: https://hyperfoil.io/docs/user-guide/installation/start_manual

Use this script to start the Hyperfoil controller. Arguments are passed directly to the Java process.

```bash
bin/controller.sh
```

--------------------------------

### Running a More Complex Benchmark (eshop-scale)

Source: https://hyperfoil.io/docs/getting-started/_print

Example of running a more complex benchmark 'eshop-scale' to observe online statistics during execution.

```bash
podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

[hyperfoil@localhost]$ upload .../eshop-scale.hf.yaml
Loaded benchmark eshop-scale, uploading...
... done.
[hyperfoil@localhost]$ run eshop-scale
Started run 0002
Run 0002, benchmark eshop-scale
...
```

--------------------------------

### Basic YAML Structure Example

Source: https://hyperfoil.io/docs/reference

Illustrates the fundamental key-value pair structure for Hyperfoil benchmark configurations.

```yaml
EXAMPLE:
  Key: Value

```

--------------------------------

### Example Benchmark Configuration

Source: https://hyperfoil.io/docs/getting-started/quickstart6

This is a basic configuration for a single-request benchmark, defining the benchmark name, HTTP target, and phases.

```yaml
name: single-request
http:
  host: http://hyperfoil.io
phases:

```

--------------------------------

### Install Ansible Galaxy Roles

Source: https://hyperfoil.io/docs/_print

Installs necessary Hyperfoil Ansible Galaxy roles for setup, testing, and shutdown.

```bash
ansible-galaxy install hyperfoil.hyperfoil_setup,{{ site.last_release.galaxy_version }}
ansible-galaxy install hyperfoil.hyperfoil_shutdown,{{ site.last_release.galaxy_version }}
ansible-galaxy install hyperfoil.hyperfoil_test,{{ site.last_release.galaxy_version }}

```

--------------------------------

### Hyperfoil Benchmark Configuration

Source: https://hyperfoil.io/docs/getting-started/quickstart6

Defines a simple benchmark with a single phase 'example' that performs an HTTP GET request synchronously.

```yaml
example:
    atOnce:
      users: 1
      scenario:
      - test:
        - httpRequest:
            GET: /
            sync: true

```

--------------------------------

### Basic Benchmark Definition

Source: https://hyperfoil.io/docs/getting-started/_print

Defines a simple benchmark with a single phase 'example' that performs one HTTP GET request synchronously.

```yaml
example:
    atOnce:
      users: 1
      scenario:
      - test:
        - httpRequest:
            GET: /
            sync: true
```

--------------------------------

### Example Phase Configuration

Source: https://hyperfoil.io/docs/user-guide/benchmark/phases

Defines two phases: an 'outOfBand' phase to spawn users after a delay and a 'final' phase to execute at the end of the benchmark.

```yaml
- outOfBand:
    always:
      users: 5
      startTime: 2m
      duration: 2m
      scenario: ...
- final:
    atOnce:
      users: 1
      startAfterStrict:
      - rampUp
      - steadyState
      scenario: ...
```

--------------------------------

### Install Hyperfoil Ansible Roles

Source: https://hyperfoil.io/docs/user-guide/installation/ansible

Fetch the necessary Ansible Galaxy roles for Hyperfoil setup, testing, and shutdown. Ensure you use the correct version for your release.

```bash
ansible-galaxy install hyperfoil.hyperfoil_setup,{{ site.last_release.galaxy_version }}
ansible-galaxy install hyperfoil.hyperfoil_shutdown,{{ site.last_release.galaxy_version }}
ansible-galaxy install hyperfoil.hyperfoil_test,{{ site.last_release.galaxy_version }}
```

--------------------------------

### Run Local Server

Source: https://hyperfoil.io/docs/_print/index.html

Starts a local server to serve requests for the benchmark. Replace 'podman' with 'docker' if using Docker.

```bash
podman run --rm -p 8080:8080 quay.io/hyperfoil/hyperfoil-examples
```

--------------------------------

### Example Phase Configuration with Increasing Rate

Source: https://hyperfoil.io/docs/user-guide/benchmark/_print

This snippet shows how to configure a phase with an increasing user rate over a specified duration. It includes settings for initial and target users per second, maximum active sessions, and the scenario to be executed.

```yaml
phases:
# Over one minute ramp the number of users started each second from 1 to 100
- rampUp:
    increasingRate:
      initialUsersPerSec: 1
      targetUsersPerSec: 100
      # We expect at most 200 users being active at one moment - see below
      maxSessions: 200
      duration: 1m
      scenario: ...
```

--------------------------------

### Ansible Hosts File Example

Source: https://hyperfoil.io/docs/_print

An example configuration for the Ansible `hosts` file, defining controller and agent nodes.

```ini
[hyperfoil-controller]
controller ansible_host=localhost

[hyperfoil-agent]
agent-1 ansible_host=localhost

```

--------------------------------

### Starting Hyperfoil CLI and Uploading Benchmark

Source: https://hyperfoil.io/docs/howtos/credentials

This command launches the Hyperfoil CLI in host-network mode and uploads the benchmark configuration. Connect to the CLI to manage and run benchmarks.

```bash
podman run --rm -it --net host quay.io/hyperfoil/hyperfoil cli
[hyperfoil]$ upload https://hyperfoil.io/benchmarks/credentials.hf.yaml
```

--------------------------------

### Start Hyperfoil Standalone Server

Source: https://hyperfoil.io/docs/getting-started/quickstart6

Use this script to start the Hyperfoil controller and an agent in a single JVM for standalone operation.

```shell
bin/standalone.sh
```

--------------------------------

### Hyperfoil Stats Output Example

Source: https://hyperfoil.io/docs/migration/_print

Example output from the 'stats' command, showing throughput, latency, and error metrics for different phases.

```text
Total stats from run 0000
Extensions (use -e to show): transfer
PHASE        METRIC   THROUGHPUT    REQUESTS  MEAN      p50       p90        p99        p99.9      p99.99     TIMEOUTS  ERRORS  BLOCKED  2xx   3xx  4xx  5xx  CACHE
calibration  request   96.58 req/s       608  99.39 ms  98.57 ms  100.14 ms  117.96 ms  406.85 ms  406.85 ms         0       0     0 ns   608    0    0    0      0
test         request  101.44 req/s      1025  97.99 ms  98.04 ms   99.09 ms  100.14 ms  101.19 ms  101.19 ms         0       0     0 ns  1015    0    0    0      0
```

--------------------------------

### Forked Phases Example

Source: https://hyperfoil.io/docs/user-guide/benchmark/phases

Demonstrates how to define 'forks' within a 'steadyState' phase to distribute users between 'sellShares' and 'buyShares' sub-phases with different weights.

```yaml
phases:
- steadyState:
    constantRate:
      usersPerSec: 30
      duration: 5m
      forks:
        sellShares:
          weight: 1
          scenario: ...
        buyShares:
          weight: 2
          scenario: ...
```

--------------------------------

### wrk Output Example

Source: https://hyperfoil.io/docs/migration/_print

Example output from a wrk load test, showing statistics like latency and requests per second.

```text
Running 10s test @ http://example.com
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    97.64ms  489.01us 101.11ms   73.66%
    Req/Sec    50.73      4.81    80.00     91.92%
  1010 requests in 10.01s, 1.53MB read
Requests/sec:    100.85
Transfer/sec:    156.91KB
```

--------------------------------

### Upload Benchmark and Run

Source: https://hyperfoil.io/docs/_print/index.html

Upload the benchmark configuration to the Hyperfoil controller and start the execution. Monitor the run status and statistics.

```bash
[hyperfoil]$ upload https://hyperfoil.io/benchmarks/credentials.hf.yaml
Loaded benchmark credentials, uploading...
... done.

[hyperfoil]$ run
Started run 0000
Run 0000, benchmark credentials
Agents: in-vm[STOPPED]
Started: 2022/10/27 15:48:39.271    Terminated: 2022/10/27 15:48:39.301
NAME              STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION             DESCRIPTION</span>
use-bearer-token  TERMINATED  15:48:39.271             15:48:39.301  30 ms (exceeded by 31 ms)  1 users at once
use-cookie        TERMINATED  15:48:39.271             15:48:39.300  29 ms (exceeded by 30 ms)  1 users at once

[hyperfoil]$ stats
Total stats from run 0000

PHASE             METRIC                 THROUGHPUT   REQUESTS  MEAN     p50      p90      p99      p99.9    p99.99   TIMEOUTS  ERRORS  BLOCKED  2xx  3xx  4xx  5xx  CACHE</span>
use-bearer-token  after-login            33.33 req/s         1  3.79 ms  3.80 ms  3.80 ms  3.80 ms  3.80 ms  3.80 ms         0       0     0 ns    1    0    0    0      0
use-bearer-token  login-with-basic-auth  33.33 req/s         1  4.57 ms  4.59 ms  4.59 ms  4.59 ms  4.59 ms  4.59 ms         0       0     0 ns    0    1    0    0      0
use-bearer-token  request-login-page     33.33 req/s         1  7.49 ms  7.50 ms  7.50 ms  7.50 ms  7.50 ms  7.50 ms         0       0     0 ns    0    0    1    0      0
```

--------------------------------

### Benchmark Results (CSV Table)

Source: https://hyperfoil.io/docs/getting-started/_print

Example output of the total statistics for the 'example' benchmark run, showing request counts and response times.

```text
Phase    Name  Requests  Responses  Mean       Min        p50.0      p90.0      p99.0      p99.9      p99.99     Max        MeanSendTime  ConnFailure  Reset  Timeouts  2xx  3xx  4xx  5xx  Other  Invalid  BlockedCount  BlockedTime  MinSessions  MaxSessions
example  test  1         1          267911168  267386880  268435455  268435455  268435455  268435455  268435455  268435455  2655879       0            0      0         0    1    0    0    0      0        0             0
```

--------------------------------

### Running the Benchmark

Source: https://hyperfoil.io/docs/howtos/credentials

This command executes the uploaded benchmark. It starts the run and provides a summary of the status and duration for each scenario.

```bash
[hyperfoil@in-vm]$ run
```

--------------------------------

### Hyperfoil Container wrk Output Example

Source: https://hyperfoil.io/docs/migration/_print

Example output when running wrk via the Hyperfoil container, similar to the native wrk output.

```text
Running 10s test @ http://example.com/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    98.61ms    9.58ms 402.65ms   98.92%
    Req/Sec   101.70      3.35   110.00     80.00
  1017 requests in 10.023s,   1.59MB read
Requests/sec: 101.47
Transfer/sec: 162.84kB
```

--------------------------------

### Uploading and Running a Scaled Benchmark

Source: https://hyperfoil.io/docs/_print/index.html

This sequence of commands uploads a more complex benchmark definition ('eshop-scale.hf.yaml') and starts its execution.

```bash
[hyperfoil@localhost]$ upload .../eshop-scale.hf.yaml
Loaded benchmark eshop-scale, uploading...
... done.
[hyperfoil@localhost]$ run eshop-scale
Started run 0002
Run 0002, benchmark eshop-scale
...
```

--------------------------------

### Run Hyperfoil CLI

Source: https://hyperfoil.io/docs/_print

Start the Hyperfoil CLI in host-network mode to allow communication with the server. This command also starts an in-VM controller.

```bash
podman run --rm -it --net host quay.io/hyperfoil/hyperfoil cli
```

--------------------------------

### Hyperfoil CLI Commands

Source: https://hyperfoil.io/docs/getting-started/_print

Demonstrates the command-line interface commands for starting a local Hyperfoil instance, uploading a benchmark configuration, running the benchmark, and viewing statistics.

```bash
[hyperfoil]$ start-local
...
[hyperfoil@in-vm]$ upload .../random-urls.hf.yaml
...
[hyperfoil@in-vm]$ run
Started run 0002
Run 0002, benchmark random-urls
Agents: in-vm[STARTING]
Started: 2019/11/15 17:49:45.859    Terminated: 2019/11/15 17:49:50.904
NAME  STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION               DESCRIPTION
main  TERMINATED  17:49:45.859             17:49:50.903  5044 ms (exceeded by 44 ms)  10.00 users per second

[hyperfoil@in-vm]$ stats
```

--------------------------------

### Start Local Hyperfoil Controller

Source: https://hyperfoil.io/docs/getting-started/quickstart1

Initiate a local Hyperfoil controller within the CLI for testing and development. This command starts the controller and connects to it.

```bash
[hyperfoil]$ start-local
Starting controller in default directory (/tmp/hyperfoil)
Controller started, listening on 127.0.0.1:41621
Connecting to the controller...
Connected!
```
Tokens:
Raw