### Install uv and Setup Project

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/AGENTS.md

Installs the `uv` package manager and sets up a new virtual environment for project development. This is the initial setup step for new contributors.

```bash
# Install uv (first time)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Setup blank project if needed
uv init && uv venv --python 3.12
source .venv/bin/activate
```

--------------------------------

### Install Browser Extras and Example Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_dom_example/README.md

Installs necessary browser dependencies and the specific example environment using uv pip.

```bash
# Install browser extras
uv pip install -e ".[browser]"

# Install this example environment
uv pip install -e ./environments/browser_dom_example
```

--------------------------------

### Install Browser Extras and Example Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Installs necessary browser extras and the specific example environment using uv pip.

```bash
uv pip install -e ".[browser]"

# Install this example environment
uv pip install -e ./environments/browser_cua_example
```

--------------------------------

### Example Workflow: Start and Resume Evaluation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/evaluation.md

Demonstrates the complete workflow of starting a large evaluation with checkpointing and then resuming it from a specific checkpoint if it gets interrupted.

```bash
# Start a large evaluation with checkpointing
prime eval run math-python -n 500 -r 3 -s

# If interrupted, find the run directory
ls ./environments/math_python/outputs/evals/math-python--openai--gpt-4.1-mini/

# Resume from the checkpoint
prime eval run math-python -n 500 -r 3 -s \
  --resume ./environments/math_python/outputs/evals/math-python--openai--gpt-4.1-mini/abc12345
```

--------------------------------

### Install Environment from Repository

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/create-environments/SKILL.md

When examples are available in a repository, use `prime env install --from-repo` to install directly.

```bash
prime env install math-python --from-repo
```

--------------------------------

### Download Example Configuration Files

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/training.md

Use the prime lab setup script to download example configuration files for Hosted Training, evaluation, and GEPA into your workspace.

```bash
prime lab setup
```

--------------------------------

### Development Setup Commands

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/development.md

Essential commands for setting up the development environment, installing pre-commit hooks, and running various test suites.

```bash
# Development setup
uv sync                               # CPU-only
uv sync --all-extras                  # With RL/training extras
uv run pre-commit install             # One-time per clone (installs pre-commit + pre-push)
```

```bash
# Run tests
uv run pytest tests/                  # All tests
uv run pytest tests/ -xvs             # Debug mode
uv run pytest tests/ --cov=verifiers  # With coverage
```

```bash
# Run environment tests
uv run pytest tests/test_envs.py -vv              # All environments
uv run pytest tests/test_envs.py -k math_python   # Specific environment
```

```bash
# Linting
uv run ruff check --fix .             # Fix lint errors
uv run ruff format --check verifiers tests  # Verify Python formatting
uv run ty check verifiers             # Type check (matches CI Ty target)
```

```bash
# Environment tools
prime env init new-env                       # Create v0 environment stub
prime eval run new-env -m openai/gpt-4.1-mini -n 5  # Test environment
prime eval view                              # Browse evals in the tree browser
```

--------------------------------

### Setup prime-rl Workspace

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/training.md

Run this command to clone and install the prime-rl trainer and its dependencies. Refer to the prime-rl documentation for configuration files and launch commands.

```bash
prime lab setup --prime-rl
```

--------------------------------

### Install Hub Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/create-environments/SKILL.md

Use these bash commands to install or pull existing environments from the hub. This is useful for starting with a baseline environment.

```bash
prime env install owner/name
```

```bash
prime env pull owner/name -t ./tmp-env
```

--------------------------------

### Install Environment from Repository

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/browse-environments/SKILL.md

Install an environment directly from the repository using a specific path.

```bash
prime env install reverse-text --from-repo
```

--------------------------------

### List and Install Existing Environments

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/create-environments/SKILL.md

Search for existing environments using `prime env list --search` and retrieve information with `prime env info`. Install a found environment using `prime env install`.

```bash
prime env list --search "keyword"
prime env info owner/name
prime env install owner/name
```

--------------------------------

### Start CUA Server Locally

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/envs/integrations/README.md

Instructions to start the CUA server locally for manual server mode. This involves navigating to the assets directory, installing dependencies, and running the start script.

```bash
cd assets/templates/browserbase/cua
pnpm install
./start.sh
```

--------------------------------

### Install Local Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/assets/lab/environments/AGENTS.md

Install a local environment using the `prime env install` command. Use the `-p` flag for custom paths.

```bash
prime env install my-env                    # from ./environments/my_env
prime env install my-env -p /path/to/environments   # custom path

```

--------------------------------

### Install Dependencies

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/development.md

Ensures all necessary packages are installed for development. Use `--extra` flags to install optional dependencies for specific integrations.

```bash
# Ensure package is installed in development mode
uv sync
```

```bash
# Install optional dependencies for specific integrations
uv sync --extra ta   # for TextArenaEnv
uv sync --extra rg   # for ReasoningGymEnv
uv sync --extra modal     # for the v1 Modal runtime
uv sync --extra notebook  # for generate_sync() in Jupyter
uv sync --extra quest     # for QUEST PDF evaluation
```

--------------------------------

### Install Verifiers and Prime CLI

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/overview.md

Install the `uv` package manager, the `prime` CLI tool, and log in to the Prime Intellect platform.

```bash
# install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# install the prime CLI
uv tool install prime
# log in to the Prime Intellect platform
prime login
```

--------------------------------

### Initialize and Install a New Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/create-environments/SKILL.md

Use `prime env init` to create a new environment and `prime env install` to set it up. The `--v1` flag specifies the version, and `--with-harness` is used when the environment requires an explicit reusable harness.

```bash
prime env init my-env --v1
prime env install my-env
```

--------------------------------

### Install Verifiers V1

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/v1/README.md

Installs the core verifiers package along with shipped packages and examples.

```bash
uv sync   # core + the shipped packages + examples
```

--------------------------------

### Clone and Install Verifiers

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/development.md

Clone the repository and install dependencies for development. Use 'uv sync' for CPU-only or 'uv sync --all-extras' for GPU-based development.

```bash
git clone https://github.com/PrimeIntellect-ai/verifiers.git
cd verifiers

# CPU-only development:
uv sync

# GPU-based trainer development:
uv sync --all-extras

# Install pre-commit hooks (including pre-push Ty gate):
uv run pre-commit install
```

--------------------------------

### Install and Evaluate Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/browse-environments/SKILL.md

Install an environment and run a smoke evaluation with a specified model.

```bash
prime env install owner/name
prime eval run name -m openai/gpt-4.1-mini -n 5
```

--------------------------------

### Run CUA Example in Manual Server Mode

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Executes the browser CUA example with sandbox disabled, connecting to a manually started CUA server for local development.

```bash
prime eval run browser-cua-example -m gpt-4.1-mini -b https://api.openai.com/v1 -k OPENAI_API_KEY -a '{"use_sandbox": false}'
```

--------------------------------

### Hosted Training Configuration Example

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/training.md

Example TOML configuration for Hosted Training using the primeintellect/reverse-text environment with a Qwen/Qwen3.5-4B model. Ensure only one model is uncommented.

```toml
# Qwen3.5 dense models. Uncomment exactly one model.
# model = "Qwen/Qwen3.5-0.8B"
# model = "Qwen/Qwen3.5-2B"
model = "Qwen/Qwen3.5-4B"
# model = "Qwen/Qwen3.5-9B"

max_steps = 100
batch_size = 128
rollouts_per_example = 8

[sampling]
max_tokens = 1024

[[env]]
id = "primeintellect/reverse-text"
```

--------------------------------

### Load Example Dataset

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/reference.md

Loads a built-in example dataset by its name.

```python
vf.load_example_dataset(name: str) -> Dataset
```

--------------------------------

### Install and Run Environment Evaluation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/review-environments/SKILL.md

Use these commands to install a local environment and run an evaluation against a specified model. Ensure the environment is correctly installed before running evaluations.

```bash
prime env install <env>
prime eval run <env> -m openai/gpt-4.1-mini -n 5
```

--------------------------------

### CLI Examples for Environment Secrets and Variables

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/envs/integrations/browser_env/README.md

Examples of using the CLI to create and link secrets, and create variables for an environment.

```bash
prime env secret create owner/my-env --name BROWSERBASE_API_KEY --value "..."
```

```bash
prime env secret link <global-secret-id> owner/my-env
```

```bash
prime env var create owner/my-env --name MAX_TURNS --value 10
```

--------------------------------

### Install and Run Environment Evaluation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/train-with-environments/SKILL.md

Use these commands to install a new environment and run a smoke evaluation before training. Ensure you keep the default save behavior unless explicitly overriding with `--skip-upload`.

```bash
prime env install my-env
```

```bash
prime eval run my-env -m openai/gpt-4.1-mini -n 20 -r 3 -s
```

--------------------------------

### Local Environment Installation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/AGENTS.md

Installs a local environment using `prime env install`. This command runs `uv pip install -e` for editable installs.

```bash
prime env install my-env                    # from ./environments/my_env
prime env install my-env -p /path/to/environments   # custom path
```

--------------------------------

### Implement Taskset Lifecycle Hooks in Python

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/v1/GUIDE.md

Example of a Taskset class implementing `setup`, `finalize`, and `reward` hooks. The `setup` hook clones a repository, `finalize` scrapes a diff, and `reward` runs tests. `NEEDS_CONTAINER` is set to `True` to ensure a containerized runtime.

```python
import vf

class SWETaskset(vf.Taskset[SWETask, SWEConfig]):
    NEEDS_CONTAINER = True   # the only Taskset class var: refuse the subprocess runtime

    async def setup(self, task, runtime) -> None:
        await runtime.run(["git", "clone", task.repo_url, "/repo"], {})
        await runtime.run(["git", "-C", "/repo", "checkout", task.base_commit], {})

    async def finalize(self, task, trace, runtime) -> None:
        diff = await runtime.run(["git", "-C", "/repo", "diff"], {})
        trace.info["diff"] = diff.stdout   # scrape the agent's diff off the live runtime

    @vf.reward()
    async def tests_pass(self, task, trace, runtime) -> float:
        result = await runtime.run(["bash", "-lc", task.test_cmd], {})
        return 1.0 if result.exit_code == 0 else 0.0
```

--------------------------------

### Run CUA Example with Pre-built Docker Image

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Executes the browser CUA example using a pre-built Docker image for fast startup, specifying the model, backend URL, and API key.

```bash
prime eval run browser-cua-example -m gpt-4.1-mini -b https://api.openai.com/v1 -k OPENAI_API_KEY
```

--------------------------------

### Install Extras for Integrations

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/envs/AGENTS.md

Install specific extras for TextArenaEnv, ReasoningGymEnv, or BrowserEnv using uv sync. This ensures all necessary dependencies for these integrations are met.

```bash
uv sync --extra ta       # for TextArenaEnv
uv sync --extra rg       # for ReasoningGymEnv
uv sync --extra browser  # for BrowserEnv
```

--------------------------------

### Install verifiers-rl from GitHub main with RL extras

Source: https://github.com/primeintellect-ai/verifiers/blob/main/packages/verifiers-rl/verifiers_rl/rl/README.md

Install the verifiers-rl package directly from the GitHub main branch with RL extras.

```bash
uv add 'verifiers-rl @ git+https://github.com/PrimeIntellect-ai/verifiers.git@main#subdirectory=packages/verifiers-rl'
```

--------------------------------

### Install a Verifiers Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/README.md

Installs a Verifiers environment, making it available for use in training or evaluation. This command typically follows the creation or configuration of an environment.

```bash
prime env install my-env
```

--------------------------------

### Install and Run Verifiers V1 Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/v1/GUIDE.md

Use `uv sync` to install core packages and `uv run eval` to execute evaluations. The `uv run eval -h` command provides typed help for available tasksets and harnesses.

```bash
uv sync                          # core + shipped packages + examples
```

```bash
uv run eval gsm8k-v1 -n 5 -r 3   # 5 tasks, 3 rollouts each; default harness, subprocess runtime
```

```bash
uv run eval -h                   # typed help (lists local tasksets + harnesses)
```

--------------------------------

### Install CUA Server Prerequisites

Source: https://github.com/primeintellect-ai/verifiers/blob/main/assets/templates/browserbase/cua/README.md

This command installs the necessary Node.js packages for the CUA server, including Stagehand and Fastify.

```bash
npm install @browserbasehq/stagehand fastify
```

--------------------------------

### Install BrowserEnv with Extras

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/envs/integrations/browser_env/README.md

Install BrowserEnv with the necessary browser extras from the verifiers repo or a project using verifiers.

```bash
uv sync --extra browser
```

```bash
uv pip install -e ".[browser]"
```

--------------------------------

### Run Evaluation with Default Settings

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/reasoning_gym_env/README.md

Execute an evaluation using the reasoning-gym-env with its default configuration. This is the simplest way to start an evaluation.

```bash
prime eval run reasoning-gym-env
```

--------------------------------

### Install Verifiers with Optional Features

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/overview.md

Install Verifiers with optional features like `modal` for the v1 Modal sandbox, `notebook` for Jupyter integration, or `quest` for PDF parsing.

```bash
uv add "verifiers[modal]"
uv add "verifiers[notebook]"
uv add "verifiers[quest]"
```

--------------------------------

### Installing Pre-commit Hooks

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/development.md

Command to install the pre-commit hooks. These hooks run automatically on commits and pushes to enforce code style and quality.

```bash
uv run pre-commit install
```

--------------------------------

### Run CUA Example with Different Modes

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Executes the browser CUA example using the prime eval command with different configurations for model, pre-built image usage, and sandbox.

```bash
# Default - pre-built image (fastest)
prime eval run browser-cua-example -m openai/gpt-4o-mini

# Binary upload (custom server)
prime eval run browser-cua-example -m openai/gpt-4o-mini -a '{"use_prebuilt_image": false}'

# Local development
prime eval run browser-cua-example -m openai/gpt-4o-mini -a '{"use_sandbox": false}'
```

--------------------------------

### Run CUA Example with Custom Server URL

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Executes the browser CUA example, specifying a custom URL for the CUA server when running in manual mode with sandbox disabled.

```bash
prime eval run browser-cua-example -m gpt-4.1-mini -b https://api.openai.com/v1 -k OPENAI_API_KEY -a '{"use_sandbox": false, "server_url": "http://localhost:8080"}'
```

--------------------------------

### Run CUA Example with Binary Upload Mode

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Executes the browser CUA example using binary upload mode, disabling the prebuilt image to build and upload a custom CUA server binary.

```bash
prime eval run browser-cua-example -m gpt-4.1-mini -b https://api.openai.com/v1 -k OPENAI_API_KEY -a '{"use_prebuilt_image": false}'
```

--------------------------------

### Run Toxicity Explanation with Defaults

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/toxicity_explanation/README.md

Execute an evaluation for the toxicity-explanation environment using default settings. This is a quick way to start an evaluation.

```bash
prime eval run toxicity-explanation
```

--------------------------------

### Async Test Example with MockClient

Source: https://github.com/primeintellect-ai/verifiers/blob/main/tests/README.md

Demonstrates testing an async environment using the mock_client fixture. It asserts that the client was called once.

```python
@pytest.mark.asyncio
async def test_my_async_function(mock_client):
    env = SingleTurnEnv(client=mock_client, model="test", ...)
    result = await env.rollout(...)
    assert mock_client.call_count == 1
```

--------------------------------

### Add Verifiers to Existing Project

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/overview.md

Add the `verifiers` library to an existing project and set up the lab environment, skipping the initial installation.

```bash
uv add verifiers && prime lab setup --skip-install
```

--------------------------------

### EnvGroup Initialization Example

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/reference.md

Demonstrates how to combine multiple environments into a single EnvGroup for mixed-task training. Optional environment names can be provided for internal routing.

```python
env_group = vf.EnvGroup(
    envs=[env1, env2, env3],
    env_names=["math", "code", "qa"]  # optional
)
```

--------------------------------

### Get Typed Help for Evaluation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/v1/README.md

Displays typed help information for the 'eval' command, including local example tasksets and harnesses.

```bash
uv run eval -h                   # typed help (+ the local example tasksets/harnesses)
```

--------------------------------

### Run Basic Evaluation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/evaluation.md

Execute evaluations against a local or Hub environment. This command resolves and installs the environment, loads its module, runs examples, scores them, and prints aggregate metrics.

```bash
prime eval run my-env -m openai/gpt-4.1-mini -n 10
```

--------------------------------

### Print Prompt Completions Sample

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/reference.md

Pretty-prints a sample of rollouts from the provided outputs. Allows specifying the number of samples to display.

```python
vf.print_prompt_completions_sample(outputs: GenerateOutputs, n: int = 3)
```

--------------------------------

### Initialize BrowserEnv in DOM and CUA Modes

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/envs/integrations/README.md

Quick start example for initializing BrowserEnv. Choose 'dom' mode for natural language operations or 'cua' mode for vision-based primitives. Ensure your dataset and rubric are defined.

```python
from verifiers.envs.integrations.browser_env import BrowserEnv
from datasets import Dataset
import verifiers as vf

# Create your dataset
dataset = Dataset.from_list([
    {"prompt": [{"role": "user", "content": "Navigate to example.com and find the main heading"}]}
])

# Create a rubric
rubric = vf.Rubric(funcs=[my_reward_func])

# DOM mode (natural language)
env = BrowserEnv(
    mode="dom",
    dataset=dataset,
    rubric=rubric,
)

# CUA mode (vision-based) - auto-deploys server to sandbox (no setup required)
env = BrowserEnv(
    mode="cua",
    dataset=dataset,
    rubric=rubric,
)
```

--------------------------------

### Initialize New Environment Project

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/AGENTS.md

Initializes a new environment project. Use the `--v1` flag to create a project based on the v1 Taskset/Harness template.

```bash
prime env init my-env       # v0 stub
prime env init my-env --v1  # v1 Taskset/Harness template
```

--------------------------------

### Install Test Dependencies with uv

Source: https://github.com/primeintellect-ai/verifiers/blob/main/tests/README.md

Installs test dependencies using the uv package manager. Ensure uv and Python 3.11+ are installed.

```bash
uv sync
```

--------------------------------

### Run hello-mcp-harbor Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/hello_mcp_harbor/README.md

Execute a single rollout of the hello-mcp-harbor environment against your configured agent.

```bash
prime eval run hello-mcp-harbor
```

--------------------------------

### Start CUA Server Locally

Source: https://github.com/primeintellect-ai/verifiers/blob/main/assets/templates/browserbase/cua/README.md

These bash commands show how to start the CUA server locally for development. Use 'pnpm dev' for hot reloading or 'pnpm start' for a standard start. The port can be customized using the CUA_SERVER_PORT environment variable.

```bash
# Start the server (with hot reload)
pnpm dev
```

```bash
# Or start without hot reload
pnpm start
```

```bash
# Custom port via environment variable
CUA_SERVER_PORT=8080 pnpm dev
```

--------------------------------

### Implement a Custom StatefulToolEnv

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/AGENTS.md

Extend StatefulToolEnv to manage per-rollout state. Use args_to_skip to hide parameters from the model and update_tool_args to inject state into tool calls at runtime. The setup_state method initializes resources for each rollout.

```python
class MySandboxEnv(vf.StatefulToolEnv):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.add_tool(self.run_code, args_to_skip=["session_id"])

    async def setup_state(self, state, **kwargs):
        state["session_id"] = await create_session()
        await super().setup_state(state, **kwargs)

    def update_tool_args(self, tool_name, tool_args, messages, state, **kwargs):
        if tool_name == "run_code":
            tool_args["session_id"] = state["session_id"]
        return tool_args

    async def run_code(self, code: str, session_id: str) -> str:
        """Execute code in the sandbox."""
        return await execute_in_session(session_id, code)
```

--------------------------------

### Initialize Environment Project

Source: https://github.com/primeintellect-ai/verifiers/blob/main/docs/environments.md

Use `prime env init` to create a new environment project structure. This command sets up the basic files for your custom environment.

```bash
prime env init my-env       # v0 stub
```

--------------------------------

### Clone Verifiers and Install Dependencies

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/AGENTS.md

Clones the verifiers repository and installs core dependencies. Includes options for CPU-only or GPU-based development environments. Also installs pre-commit hooks for code quality checks.

```bash
git clone https://github.com/PrimeIntellect-ai/verifiers.git
cd verifiers

# CPU-only development:
uv sync

# GPU-based trainer development:
uv sync --all-extras && uv pip install flash-attn --no-build-isolation

# Install pre-commit hooks:
uv run pre-commit install
```

--------------------------------

### Add Verifiers and Scaffold New Environment

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/AGENTS.md

Use `uv add` to install verifiers and their dependencies. Then, use `prime env init` to scaffold a new environment.

```bash
# Add verifiers
uv add verifiers            # core
uv add 'verifiers[all]'     # + training

# Scaffold new environment
prime env init new-environment
```

--------------------------------

### Initialize and Explore Taskset

Source: https://github.com/primeintellect-ai/verifiers/blob/main/verifiers/envs/experimental/composable/README.md

Demonstrates creating a R2EGymTaskSet, accessing individual tasks, and inspecting task properties like prompt and sandbox specifications.

```python
from verifiers.envs.experimental.composable.tasksets.swe.r2e_gym import R2EGymTaskSet
from verifiers.envs.experimental.composable.harnesses.opencode import opencode_harness
from verifiers.envs.experimental.composable import ComposableEnv

# Create a taskset
taskset = R2EGymTaskSet()                 # 4578 SWE instances

# Explore instances
task = taskset[0]                          # one Task
task.prompt                                # the problem statement
task.sandbox_spec                          # SandboxSpec(image=..., cpu=4, ...)
task.sandbox_spec.image                    # per-instance docker image
```

--------------------------------

### Shuffle Examples Before Evaluation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/evaluate-environments/SKILL.md

Shuffle examples before selecting the evaluation subset. This ensures a random sample for evaluation.

```bash
prime eval run my-env -n 200 -r 3 --shuffle --shuffle-seed 123 -s
```

--------------------------------

### Initialize a New Verifiers Environment Template

Source: https://github.com/primeintellect-ai/verifiers/blob/main/README.md

Creates a new, basic environment template within the `./environments/` directory. Use `--with-harness` if the environment manages its own harness behavior.

```bash
prime env init my-env # creates a new template in ./environments/my_env
```

```bash
prime env init my-env --with-harness
```

--------------------------------

### Example: Create and Interact with a Session

Source: https://github.com/primeintellect-ai/verifiers/blob/main/assets/templates/browserbase/cua/README.md

This bash script demonstrates a complete workflow: creating a session, navigating to a URL, clicking an element, typing text, pressing Enter, and scrolling down. It uses `curl` for API requests and `jq` for parsing JSON responses.

```bash
# Create a session
SESSION=$(curl -s -X POST http://localhost:3000/sessions | jq -r '.sessionId')
```

```bash
# Navigate to a website
curl -X POST http://localhost:3000/sessions/$SESSION/action \
  -H "Content-Type: application/json" \
  -d '{"type": "goto", "url": "https://example.com"}'
```

```bash
# Click a button
curl -X POST http://localhost:3000/sessions/$SESSION/action \
  -H "Content-Type: application/json" \
  -d '{"type": "click", "x": 150, "y": 300}'
```

```bash
# Type into an input
curl -X POST http://localhost:3000/sessions/$SESSION/action \
  -H "Content-Type: application/json" \
  -d '{"type": "type", "text": "Hello, World!"}'
```

```bash
# Press Enter
curl -X POST http://localhost:3000/sessions/$SESSION/action \
  -H "Content-Type: application/json" \
  -d '{"type": "keypress", "keys": "Enter"}'
```

```bash
# Scroll down
curl -X POST http://localhost:3000/sessions/$SESSION/action \
  -H "Content-Type: application/json" \
  -d '{"type": "scroll", "x": 640, "y": 360, "scroll_y": 500}'
```

--------------------------------

### Initialize MCPEnv with MCP Servers

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/AGENTS.md

Configure and initialize MCPEnv by providing a list of MCP server configurations, including name, command, and arguments.

```python
mcp_servers = [
    {
        "name": "fetch",
        "command": "uvx",
        "args": ["mcp-server-fetch"],
    },
]

vf_env = vf.MCPEnv(
    mcp_servers=mcp_servers,
    dataset=dataset,
    rubric=rubric,
)

```

--------------------------------

### Run Package Install Tests

Source: https://github.com/primeintellect-ai/verifiers/blob/main/skills/create-environments/SKILL.md

Executes pytest for environment tests, particularly useful when packaging or dependencies have changed. Assumes 'uv' is installed.

```bash
uv run pytest tests/test_envs.py -k my_env -vv
```

--------------------------------

### Run Browser CUA Example with Custom Image

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Use this command to run the browser-cua-example, specifying your custom Docker image for the CUA server. Ensure the image name and tag match what you pushed.

```bash
prime eval run browser-cua-example -m openai/gpt-4.1-mini -a '{"prebuilt_image": "myuser/cua-server:v1.0.0"}'
```

--------------------------------

### Start CUA Server for Manual Mode

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_cua_example/README.md

Starts the CUA server manually in a separate terminal for local development, setting the OpenAI API key and running the development server.

```bash
cd assets/templates/browserbase/cua
export OPENAI_API_KEY="your-openai-key"
pnpm dev
```

--------------------------------

### Configure MCP Tool Environments

Source: https://github.com/primeintellect-ai/verifiers/blob/main/assets/lab/environments/AGENTS.md

Set up MCP servers by providing a list of dictionaries, each specifying the server name, command, and arguments. This configuration is used to initialize MCPEnv.

```python
mcp_servers = [
    {
        "name": "fetch",
        "command": "uvx",
        "args": ["mcp-server-fetch"],
    },
]

vf_env = vf.MCPEnv(
    mcp_servers=mcp_servers,
    dataset=dataset,
    rubric=rubric,
)
```

--------------------------------

### Define a Group-Based Reward Function

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/AGENTS.md

Example of a reward function that operates at the group level, processing multiple completions from the same input example. It returns a list of scores, one for each completion.

```python
async def diversity_bonus(completions) -> list[float]:
    """Reward unique responses within a group."""
    responses = [c[-1]["content"] for c in completions]
    unique = set(responses)
    # Higher reward if this response is unique
    return [0.2 if responses.count(r) == 1 else 0.0 for r in responses]

rubric = vf.Rubric(funcs=[correct_answer, diversity_bonus])
```

--------------------------------

### Run Browser DOM Example Evaluation

Source: https://github.com/primeintellect-ai/verifiers/blob/main/environments/browser_dom_example/README.md

Executes the browser DOM example evaluation task using the 'prime eval run' command, specifying the model and API endpoint.

```bash
prime eval run browser-dom-example -m gpt-4.1-mini -b https://api.openai.com/v1 -k OPENAI_API_KEY
```

--------------------------------

### Initialize a new Verifiers environment (v1)

Source: https://github.com/primeintellect-ai/verifiers/blob/main/assets/agents/common_best_practices.md

Use this command to initialize a new Verifiers environment. The `--v1` flag specifies the version, and `--with-harness` can be added to include a harness. Edit the generated TasksetConfig, Taskset.load_tasks(), Taskset.load_toolsets(), User subclasses, and @vf.* methods for specific configurations and logic.

```bash
prime env init <name> --v1
prime env init <name> --v1 --with-harness
```