### Initial Project Setup

Source: https://github.com/ai-dynamo/aiperf/blob/main/CONTRIBUTING.md

Clone the repository and run the one-command setup to create a virtual environment, install the project, and set up pre-commit hooks.

```bash
git clone <repository-url>
cd aiperf

make first-time-setup
```

```bash
make setup-venv       # Create virtual environment
make install          # Install project + mock server in editable mode
pre-commit install    # Install pre-commit hooks
```

--------------------------------

### Install Dependencies and Start Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/plugins/creating-your-first-plugin.md

Commands to install necessary Python packages and start the Echo API server in the background for testing.

```bash
uv pip install fastapi uvicorn orjson cyclopts
python echo_server.py &
```

--------------------------------

### Native Installation: Start SGLang Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/sglang-video-generation.md

Start the SGLang server natively with specified model path, resource offloading, GPU count, port, and host.

```bash
sglang serve \
    --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
    --text-encoder-cpu-offload \
    --pin-cpu-memory \
    --num-gpus 1 \
    --port 30010 \
    --host 0.0.0.0
```

--------------------------------

### Setup Dynamo Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/gpu-telemetry.md

This snippet sets environment variables, pulls the Dynamo container, downloads the docker-compose file, and starts the necessary services and the Dynamo server in the background.

```bash
# Set environment variables
export AIPERF_REPO_TAG="main"
export DYNAMO_PREBUILT_IMAGE_TAG="nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.6.1"
export MODEL="Qwen/Qwen3-0.6B"

# Download the Dynamo container
docker pull ${DYNAMO_PREBUILT_IMAGE_TAG}
export DYNAMO_REPO_TAG=$(docker run --rm --entrypoint "" ${DYNAMO_PREBUILT_IMAGE_TAG} cat /workspace/version.txt | cut -d'+' -f2)

# Start up required services
curl -O https://raw.githubusercontent.com/ai-dynamo/dynamo/${DYNAMO_REPO_TAG}/deploy/docker-compose.yml
docker compose -f docker-compose.yml down || true
docker compose -f docker-compose.yml up -d

# Launch Dynamo in the background
docker run \
  --rm \
  --gpus all \
  --network host \
  ${DYNAMO_PREBUILT_IMAGE_TAG} \
    /bin/bash -c "python3 -m dynamo.frontend & python3 -m dynamo.vllm --model ${MODEL} --enforce-eager --no-enable-prefix-caching" > server.log 2>&1 &
```

--------------------------------

### Start vLLM Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/custom-dataset.md

Starts a vLLM server using Docker. Ensure you have Docker installed and the necessary NVIDIA drivers for GPU acceleration.

```bash
docker pull vllm/vllm-openai:latest
docker run --gpus all -p 8000:8000 vllm/vllm-openai:latest \
  --model Qwen/Qwen3-0.6B \
  --host 0.0.0.0 --port 8000 &
```

--------------------------------

### Setup AIPerf and Mock Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/tests/integration/README.md

This bash command installs AIPerf and its associated mock server, which is necessary for running integration tests.

```bash
make first-time-setup

```

--------------------------------

### Install Project and Mock Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/AGENTS.md

Installs the project dependencies along with a mock server.

```bash
make install
```

--------------------------------

### Initial Environment Setup

Source: https://github.com/ai-dynamo/aiperf/blob/main/AGENTS.md

Run this command for the initial setup of your development environment.

```bash
make first-time-setup
```

--------------------------------

### Launch vLLM Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/mmstar.md

Start a vLLM server with a vision language model. Ensure Docker is installed and accessible.

```bash
docker pull vllm/vllm-openai:latest
docker run --gpus all -p 8000:8000 vllm/vllm-openai:latest \
  --model Qwen/Qwen2-VL-2B-Instruct
```

--------------------------------

### Install AI-Perf Mock Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/tests/aiperf_mock_server/README.md

Installs the AI-Perf mock server. Can be installed from the project root or standalone.

```bash
# From project root
make install-mock-server
```

```bash
# Or standalone
cd tests/aiperf_mock_server
uv pip install -e ".[dev]"
```

--------------------------------

### Start vLLM Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/custom-prompt-benchmarking.md

Starts a vLLM server with the specified model. Ensure the server is accessible before running benchmarks.

```bash
docker pull vllm/vllm-openai:latest
docker run --gpus all -p 8000:8000 vllm/vllm-openai:latest \
  --model Qwen/Qwen3-0.6B \
  --host 0.0.0.0 --port 8000 &
```

--------------------------------

### Install JupyterLab in Triton Container

Source: https://github.com/ai-dynamo/aiperf/blob/main/notebooks/TCO_calculator.ipynb

Installs JupyterLab within the Triton container and starts the server, making it accessible on port 8888.

```bash
pip install jupyterlab
jupyter lab --ip 0.0.0.0 --port=8888 --allow-root --notebook-dir=/myworkspace
```

--------------------------------

### Initialize Inline Dataset Tutorial Template

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/inline-datasets.md

Use this command to create a new benchmark YAML file pre-configured with inline dataset examples.

```bash
aiperf config init --template inline_dataset --output bench.yaml
```

--------------------------------

### Scaffold Configuration from Template

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/yaml-config.md

Initialize a new benchmark configuration by scaffolding from a template. Use --list to browse templates, --search to find templates by keyword, and specify model, URL, and output file.

```bash
aiperf config init --list                          # browse
```

```bash
aiperf config init --search sweep                  # search by keyword
```

```bash
aiperf config init --template goodput_slo \
  --model meta-llama/Llama-3.1-70B-Instruct \
  --url http://localhost:8000/v1/chat/completions \
  --output benchmark.yaml
```

--------------------------------

### Module Import Error Example

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/plugins/plugin-system.md

This error indicates a problem importing the module for a specific endpoint. Verify the class path format is correct ('module.path:ClassName') and all dependencies are installed.

```text
ImportError: Failed to import module for endpoint:my_plugin
```

--------------------------------

### Basic User-Centric Rate Example

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/benchmark-modes/timing-modes-reference.md

Sets up a benchmark with 15 users, each sending requests at a rate that results in 1 QPS total. This mode is suitable for multi-turn KV cache benchmarking.

```bash
aiperf profile --url localhost:8000 --model llama \
    --user-centric-rate 1.0 \
    --num-users 15 \
    --session-turns-mean 20 \
    --streaming \
    --benchmark-duration 300
```

--------------------------------

### Minimal Warmup Example

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/warmup.md

Use this for basic connection and cache warmup. It sets a fixed number of warmup requests.

```bash
aiperf profile \
    --model Qwen/Qwen2.5-7B-Instruct \
    --url localhost:8000 \
    --endpoint-type chat \
    --streaming \
    --concurrency 50 \
    --warmup-request-count 20 \
    --request-count 500
```

--------------------------------

### Start vLLM Server for Cohere Re-ranking

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/rankings.md

Launches a vLLM server configured for re-ranking using the pooling runner. This setup is for profiling models compatible with the Cohere Re-Rank API.

```bash
docker run --gpus all -p 8080:8000 \
  -e HF_TOKEN=<HF_TOKEN> \
  vllm/vllm-openai:latest \
  --model BAAI/bge-reranker-v2-m3 \
  --runner pooling
```

--------------------------------

### Start vLLM Server with OpenAI API

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/benchmark-modes/trace-replay.md

Launches a vLLM server using a specified chat model. Ensure Docker is installed and accessible. The server will be available on port 8000.

```bash
docker pull vllm/vllm-openai:latest
docker run --gpus all -p 8000:8000 vllm/vllm-openai:latest \
  --model Qwen/Qwen3-0.6B
```

--------------------------------

### Install AIPerf and Dependencies

Source: https://github.com/ai-dynamo/aiperf/blob/main/README.md

Create a virtual environment, activate it, and install the AIPerf package. Note potential build toolchain requirements for specific architectures.

```bash
python3 -m venv venv
source venv/bin/activate
pip install aiperf
```

--------------------------------

### Production-Like Warmup Example

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/warmup.md

Simulates gradual traffic increase during warmup, mimicking production load patterns. Configure request rate, concurrency, and ramp durations.

```bash
aiperf profile \
    --model Qwen/Qwen2.5-7B-Instruct \
    --url localhost:8000 \
    --endpoint-type chat \
    --streaming \
    --request-rate 100 \
    --concurrency 200 \
    --concurrency-ramp-duration 60 \
    --warmup-request-rate 20 \
    --warmup-concurrency 50 \
    --warmup-concurrency-ramp-duration 15 \
    --warmup-duration 30 \
    --benchmark-duration 300
```

--------------------------------

### Start vLLM OpenAI Responses Server

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/openai-responses.md

Launches a vLLM server compatible with the OpenAI Responses API. Ensure Docker is installed and GPUs are available. The server will be accessible at http://localhost:8000.

```bash
docker pull vllm/vllm-openai:latest
docker run --gpus all -p 8000:8000 vllm/vllm-openai:latest \
  --model Qwen/Qwen3-0.6B --reasoning-parser qwen3
```

--------------------------------

### Multi-Turn Warmup Configuration

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/warmup.md

Configure warmup for multi-turn benchmarks by specifying the number of warmup sessions and the average session turns. This example runs 10 warmup sessions, each with an average of 5 turns, before starting the main profiling.

```bash
aiperf profile \
    --model your-model \
    --url localhost:8000 \
    --endpoint-type chat \
    --streaming \
    --session-turns-mean 5 \
    --num-warmup-sessions 10 \
    --request-count 500
```

--------------------------------

### Best Configurations Example (Single Parameter)

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/api/sweep-aggregates.md

This section identifies the best performing configurations based on specific metrics. It lists the configuration name, the parameter value, the metric value, and the unit.

```csv
Best Configurations
Configuration,concurrency,Metric,Unit
Best Throughput,40,285.50,requests/sec
Best Latency P99,10,120.50,ms
```

--------------------------------

### Aiperf Mock Server End-to-End Run Example

Source: https://github.com/ai-dynamo/aiperf/blob/main/tests/aiperf_mock_server/README.md

This bash script demonstrates how to start the aiperf mock server in record mode, run a profiling command, shut down the server gracefully, and then analyze the generated request logs and summary.

```bash
aiperf-mock-server --record-requests /tmp/req.jsonl --fast &
aiperf profile \
    --endpoint-type chat \
    --url http://localhost:8000 \
    --model Qwen/Qwen3-0.6B \
    --random-range-ratio 0.2 --isl-mean 1024 --osl-mean 256 \
    --request-count 200
kill %1     # graceful shutdown: worker flushes JSONL + summary
cat /tmp/req.jsonl.summary.json
python -c "import pandas as pd; print(pd.read_json('/tmp/req.jsonl', lines=True).describe())"
```

--------------------------------

### Comparing Configurations with Confidence Intervals

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/multi-run-confidence.md

Example demonstrating how to compare two configurations based on their confidence intervals. No overlap suggests a significant difference.

```text
Config A: mean=150ms, CI=[145, 155]
Config B: mean=180ms, CI=[175, 185]
```

--------------------------------

### Native Installation: Install SGLang with Diffusion

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/sglang-video-generation.md

Install or upgrade pip, install uv, and then install SGLang with diffusion support for native installation.

```bash
pip install --upgrade pip
pip install uv
uv pip install "sglang[diffusion]" --prerelease=allow
```

--------------------------------

### Install Optional BoTorch Dependency

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/troubleshooting/sweeps.md

Install the optional BoTorch dependency for OptunaSearchPlanner. Use the editable install for development or the PyPI install for general use.

```bash
uv pip install -e ".[botorch]"
```

```bash
pip install "aiperf[botorch]"
```

--------------------------------

### Install FFmpeg

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/synthetic-video.md

If FFmpeg is not found, use these commands to install it. After installation, restart your terminal.

```bash
sudo apt update && sudo apt install ffmpeg
```

```bash
conda install -c conda-forge ffmpeg
```

--------------------------------

### Install Plugin Locally with uv

Source: https://github.com/ai-dynamo/aiperf/blob/main/docs/plugins/creating-your-first-plugin.md

Installs the plugin in editable mode into the current Python environment. Ensure AIPerf is installed in the same environment for discovery.

```bash
uv pip install -e .

```