### Install EvalScope via PyPI

Source: https://github.com/modelscope/evalscope/blob/main/README.md

This command installs the EvalScope package from the Python Package Index (PyPI). This is the recommended method for most users as it provides a stable and easy way to get started with EvalScope.

```shell
pip install evalscope
```

--------------------------------

### Quick Experience: Clone and Run EvalScope Visualization

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/visualization.md

Clones the EvalScope repository and starts the visualization service with example data. This allows for a quick demonstration of the visualization features using pre-configured evaluation results for multiple models and datasets.

```bash
git clone https://github.com/modelscope/evalscope
evalscope app --outputs evalscope/examples/viz
```

--------------------------------

### Install EvalScope from Source with All Dependencies

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope from source with all available optional dependencies.

```shell
pip install '.[all]'
```

--------------------------------

### Using ClearML with EvalScope

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/stress_test/quick_start.md

Integrate EvalScope with ClearML for experiment tracking. Install the ClearML library, initialize the ClearML server, and then configure EvalScope by specifying 'clearml' as the visualizer and providing a task name.

```bash
pip install clearml
```

```bash
clearml-init
```

```bash
--visualizer clearml
--name 'name_of_clearml_task'
```

--------------------------------

### Install EvalScope from Source with Optional RAGEval Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope from source with the RAGEval evaluation backend.

```shell
pip install '.[rag]'
```

--------------------------------

### Install EvalScope with Performance Dependencies (pip)

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/stress_test/quick_start.md

Installs EvalScope and its performance testing dependencies using pip. This is the recommended method for most users to set up the environment for performance evaluations.

```shell
pip install evalscope[perf] -U
```

--------------------------------

### Start vLLM OpenAI API Server (Shell)

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/basic_usage.md

Start a model service using vLLM to make it compatible with the OpenAI API format. This involves installing vLLM, setting an environment variable, and running the OpenAI API server with specified model and port.

```shell
# Please install vLLM first: pip install vllm
export VLLM_USE_MODELSCOPE=True
python -m vllm.entrypoints.openai.api_server \
  --model Qwen/Qwen2.5-0.5B-Instruct \
  --served-model-name qwen2.5 \
  --trust-remote-code \
  --port 8801
```

--------------------------------

### Install and Launch Visualization WebUI

Source: https://context7.com/modelscope/evalscope/llms.txt

Installs the necessary dependencies for the visualization web UI and starts the server. The UI is accessible at http://127.0.0.1:7861.

```bash
# Install visualization dependencies
pip install 'evalscope[app]'

# Start WebUI server
evalscope app

# Access at http://127.0.0.1:7861
```

--------------------------------

### Installing and Using SwanLab with EvalScope

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/stress_test/quick_start.md

Install the SwanLab library to enable logging and visualization. Configure EvalScope by setting the visualizer to 'swanlab' and providing a log name. SwanLab can be used for cloud-based logging or a local dashboard.

```bash
pip install swanlab
```

```bash
--visualizer swanlab
--name 'name_of_swanlab_log'
```

```bash
pip install 'swanlab[dashboard]'
```

```bash
--swanlab-api-key local
```

--------------------------------

### Install EvalScope from Source with Optional Visualization Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope from source, including the dependencies needed for its visualization features.

```shell
pip install '.[app]'
```

--------------------------------

### Clone EvalScope Repository for Source Installation

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Downloads the EvalScope source code from its GitHub repository. This is the first step for installing from source, enabling direct code modification and debugging.

```shell
git clone https://github.com/modelscope/evalscope.git
```

--------------------------------

### Install and Use ClearML for Visualization

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/stress_test/examples.md

This snippet details how to set up ClearML for experiment management with EvalScope. ClearML provides an end-to-end platform for ML operations. Install the ClearML SDK using pip, initialize your ClearML server, and then use the --visualizer and --name flags to log your experiments. The CLEARML_PROJECT_NAME environment variable can also specify the project.

```bash
pip install clearml

# Initialize the ClearML server:
clearml-init

# Add the following parameters before starting the test:
# You can use the CLEARML_PROJECT_NAME environment variable to specify the project name
--visualizer clearml
--name 'name_of_clearml_task'
```

--------------------------------

### Install EvalScope from Source with Optional VLMEvalKit Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope from source with the VLMEvalKit evaluation backend.

```shell
pip install '.[vlmeval]'
```

--------------------------------

### Start Evaluation via Command Line

Source: https://github.com/modelscope/evalscope/blob/main/README.md

Execute the 'evalscope eval' command to initiate model evaluation on specified datasets with a limit on the number of samples. This is a straightforward way to get started with basic evaluations.

```bash
evalscope eval \
 --model Qwen/Qwen2.5-0.5B-Instruct \
 --datasets gsm8k arc \
 --limit 5
```

--------------------------------

### Install EvalScope with All Dependencies via Pip

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope along with all available optional dependencies, including performance testing, visualization, and all supported evaluation backends.

```shell
pip install 'evalscope[all]'
```

--------------------------------

### Install EvalScope with Optional RAGEval Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope with the RAGEval evaluation backend. This integration allows for using RAGEval's capabilities in evaluating generative models.

```shell
pip install 'evalscope[rag]'
```

--------------------------------

### Install and Use WandB for Visualization

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/stress_test/examples.md

This snippet shows how to install the Weights & Biases (WandB) library and configure EvalScope to use it for experiment logging. WandB helps track and visualize metrics, hyperparameters, and model performance during training. Ensure WandB is installed via pip and specify the visualizer and a log name when running tests.

```bash
pip install wandb

# Add these parameters before starting the test:
--visualizer wandb
--name 'name_of_wandb_log'
```

--------------------------------

### Start EvalScope Visualization Service

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/visualization.md

Starts the EvalScope web service for visualization. Access the interface at http://127.0.0.1:7860. Customizable parameters include output directory, language, public sharing, server name, and port.

```bash
evalscope app

```

--------------------------------

### Install and Use SwanLab for Visualization

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/stress_test/examples.md

This snippet outlines the steps to install SwanLab and integrate it with EvalScope for experiment tracking. SwanLab is a lightweight experiment tracking tool. Install it using pip and then use the --visualizer and --name arguments to direct logs to SwanLab. The SWANLAB_PROJ_NAME environment variable can also set the project name.

```bash
pip install swanlab

# You can use the SWANLAB_PROJ_NAME environment variable to specify the project name
--visualizer swanlab
--name 'name_of_swanlab_log'
```

--------------------------------

### Generation Configuration Examples (Bash)

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/parameters.md

Demonstrates how to set generation parameters for image models using either a simple key-value format or a more complex JSON string format. This allows for fine-tuning parameters like sampling, temperature, and chat template options.

```bash
# key=value format
--generation-config do_sample=true,temperature=0.5

# JSON string format (supports more complex parameters)
--generation-config '{"do_sample":true,"temperature":0.5,"chat_template_kwargs":{"enable_thinking":false}}'
```

--------------------------------

### Using Weights & Biases with EvalScope

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/stress_test/quick_start.md

Configure EvalScope to log evaluation results to Weights & Biases. This requires specifying the 'wandb' visualizer and a name for the log. Ensure Weights & Biases is installed and configured in your environment.

```bash
--visualizer wandb
--name 'name_of_wandb_log'
```

--------------------------------

### Install τ²-bench and evalscope

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/third_party/tau2_bench.md

Installs the evalscope library and the τ²-bench package from GitHub. This is a prerequisite for using the τ²-bench evaluation framework.

```bash
pip install evalscope
# Install tau2-bench
pip install "git+https://github.com/sierra-research/tau2-bench@v0.2.0"
```

--------------------------------

### Launch Local Inference Server with vLLM

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/best_practice/eval_qwq.md

Launches a local OpenAI API-compatible inference server using vLLM for the Qwen/QwQ-32B model. This setup is crucial for evaluating models that produce long chains of thought, optimizing reasoning speed.

```bash
VLLM_USE_MODELSCOPE=True CUDA_VISIBLE_DEVICES=0 python -m vllm.entrypoints.openai.api_server --model Qwen/QwQ-32B  --served-model-name QwQ-32B --trust_remote_code --port 8801
```

--------------------------------

### Install EvalScope from Source with Optional OpenCompass Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope from source with the OpenCompass evaluation backend.

```shell
pip install '.[opencompass]'
```

--------------------------------

### Install EvalScope

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/best_practice/qwen3_vl.md

Installs the EvalScope model evaluation framework with application and performance testing modules. Ensure you have pip installed and accessible.

```bash
pip install 'evalscope[app,perf]' -U
```

--------------------------------

### Install vLLM from Source for Qwen3-Omni

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/best_practice/qwen3_omni.md

Installs the vLLM framework from a specific branch designed for Qwen3-Omni. This process requires cloning the repository, installing build and CUDA dependencies, and then installing vLLM in editable mode. It's recommended to use a new Python environment to prevent conflicts.

```shell
git clone -b qwen3_omni https://github.com/wangxiongts/vllm.git
cd vllm
pip install -r requirements/build.txt
pip install -r requirements/cuda.txt
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://wheels.vllm.ai/a5dd03c1ebc5e4f56f3c9d3dc0436e9c582c978f/vllm-0.9.2-cp38-abi3-manylinux1_x86_64.whl
VLLM_USE_PRECOMPILED=1 pip install -e . -v --no-build-isolation
# If you encounter an "Undefined symbol" error while using VLLM_USE_PRECOMPILED=1, please use "pip install -e . -v" to build from source.
# Install the Transformers
pip install git+https://github.com/huggingface/transformers
pip install accelerate
pip install qwen-omni-utils -U
pip install -U flash-attn --no-build-isolation
```

--------------------------------

### Install EvalScope with Optional VLMEvalKit Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope with the VLMEvalKit evaluation backend. This enables using VLMEvalKit for evaluating vision-language models.

```shell
pip install 'evalscope[vlmeval]'
```

--------------------------------

### Install EvalScope from Source

Source: https://github.com/modelscope/evalscope/blob/main/README.md

This procedure installs EvalScope directly from its source code repository. It involves cloning the Git repository and then performing an editable installation, which is typically used by developers who need to modify the EvalScope codebase.

```shell
git clone https://github.com/modelscope/evalscope.git
cd evalscope
pip install -e .
```

--------------------------------

### Install EvalScope with Optional OpenCompass Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope with the OpenCompass evaluation backend. This allows using OpenCompass for comprehensive model evaluation tasks.

```shell
pip install 'evalscope[opencompass]'
```

--------------------------------

### Install EvalScope with Optional Visualization Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope with the dependencies needed for its visualization features. This enables features like dashboards and result plotting.

```shell
pip install 'evalscope[app]'
```

--------------------------------

### Install vLLM for Model Inference

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/best_practice/qwen3_next.md

Installs the vLLM framework, which is required for launching the model service locally. This command specifically installs the nightly version of vLLM along with its dependencies.

```shell
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
```

--------------------------------

### Install Older EvalScope Versions (llmuses)

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs versions of the package prior to v0.4.3, which was known as llmuses. This is for compatibility with older projects or specific version requirements.

```shell
pip install llmuses<=0.4.3
```

--------------------------------

### Install EvalScope Service (Bash)

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/user_guides/service.md

Installs the EvalScope package with service capabilities using pip. The full installation includes all necessary dependencies for running the service.

```bash
pip install evalscope[service]
```

--------------------------------

### Download Local Models for Offline Evaluation (Bash)

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/basic_usage.md

Prepare local models for offline evaluation by downloading them using the `modelscope download` command. Specify the model ID and a local directory for storage. The downloaded directory can then be used for offline model loading.

```bash
modelscope download --model modelscope/Qwen2.5-0.5B-Instruct --local_dir ./model/qwen2.5
```

--------------------------------

### Install EvalScope from Source with Optional Performance Testing Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope from source, including the specific dependencies required for model service inference performance testing.

```shell
pip install '.[perf]'
```

--------------------------------

### Install EvalScope and Transformers

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/best_practice/qwen3_next.md

Installs the EvalScope model evaluation framework along with necessary components for application and performance testing. It also installs the latest version of the Hugging Face Transformers library.

```bash
pip install 'evalscope[app,perf]' -U
pip install git+https://github.com/huggingface/transformers.git@main
```

--------------------------------

### Install τ²-bench

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/supported_dataset/agent.md

Installs the τ²-bench library from GitHub. This library is an extension of τ-bench, designed for evaluating conversational AI agents that interact with domain-specific API tools.

```bash
pip install git+https://github.com/sierra-research/tau2-bench@v0.2.0
```

--------------------------------

### Launch EvalScope Visualization Interface

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/best_practice/eval_qwq.md

This command launches the EvalScope web-based visualization interface. After running this command, users can select an evaluation report to view detailed model outputs and overall accuracy.

```bash
evalscope app --lang en
```

--------------------------------

### Install EvalScope with Optional Performance Testing Dependency

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs EvalScope along with the specific dependencies required for model service inference performance testing. This is useful for benchmarking model inference speeds.

```shell
pip install 'evalscope[perf]'
```

--------------------------------

### Install EvalScope from Source with Core Dependencies

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/get_started/installation.md

Installs the EvalScope package in editable mode from the local source code. This method is recommended for developers who need to modify the library or use the latest unreleased code.

```shell
cd evalscope/
pip install -e .
```

--------------------------------

### Launch Qwen3-Next Model Service with vLLM

Source: https://github.com/modelscope/evalscope/blob/main/docs/en/best_practice/qwen3_next.md

Launches the Qwen3-Next-80B-A3B-Instruct model as an OpenAI API-compatible service using vLLM. This command configures the service with specific parameters like port, tensor parallelism, max model length, and GPU utilization.

```bash
VLLM_USE_MODELSCOPE=true VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve Qwen/Qwen3-Next-80B-A3B-Instruct --served-model-name Qwen3-Next-80B-A3B-Instruct --port 8801 --tensor-parallel-size 4 --max-model-len 8000 --gpu-memory-utilization 0.9 --max-num-seqs 32
```