HUD

https://docs.hud.ai
Admin
HUD is a platform for building RL environments for AI agents to take actions, complete tasks, and...
Tokens:169,849
Snippets:2,162
Trust Score:7
Update:4 months ago
Show doc for...
Tokens:
Raw
### Full Scenario Run (Setup + Grade)

Source: https://docs.hud.ai/reference/cli/misc

Executes a full scenario run, including setup (getting the prompt) and grading the answer. Requires `hud dev` to be running.

```bash
hud scenario run my-env:echo --args '{"message": "hello"}' --answer "hello"
# Prompt: Repeat this back exactly: hello
# {"reward": 1.0, "done": true}
```

--------------------------------

### Get Help Command

Source: https://docs.hud.ai/platform/slack

Displays available commands and usage examples.

```text
@hud help
```

--------------------------------

### Setup Coding Environment

Source: https://docs.hud.ai/tools/filesystem

Initialize an Environment and add standard tools for a coding setup.

```python
from hud import Environment
from hud.tools import BashTool, EditTool
from hud.tools.filesystem import ReadTool, GrepTool, GlobTool, ListTool

env = Environment("coding-env")
env.add_tool(BashTool())
env.add_tool(EditTool())
env.add_tool(ReadTool())
env.add_tool(GrepTool())
env.add_tool(GlobTool())
env.add_tool(ListTool())
```

--------------------------------

### HUD Init Examples

Source: https://docs.hud.ai/reference/cli/init

Examples demonstrating various ways to use the hud init command, including interactive preset selection, specifying presets and directories, and forcing overwrites.

```bash
# Choose preset interactively (default blank)
hud init

# Create a blank template in a new directory
hud init my-env -p blank

# Browser presets
hud init my-browser -p browser

# Deep research preset (remote browser)
hud init my-deep -p deep-research

# Force overwrite
hud init my-env -p blank --force
```

--------------------------------

### Run Scenario Examples

Source: https://docs.hud.ai/platform/slack

Specific examples of running different types of agent scenarios.

```text
@hud run browser-env search-task query="best restaurants in SF"
```

```text
@hud run code-sandbox debug-task file=main.py line=42
```

```text
@hud run ops-diagnostics investigate issue_url="https://sentry.io/issues/12345"
```

--------------------------------

### Live Analysis with Environment Variables Example

Source: https://docs.hud.ai/llms-full.txt

Example of performing a live analysis with environment variables passed to the container.

```bash
# Live analysis with env vars
hud analyze my-env:latest --live -e API_KEY=test
```

--------------------------------

### Run Scenario Setup

Source: https://docs.hud.ai/llms-full.txt

Sets up a scenario with specific arguments. The arguments are provided as a JSON string.

```bash
hud scenario setup my-env:echo --args '{"message": "hello"}'
```

--------------------------------

### Typical Setup for OpenAI

Source: https://docs.hud.ai/llms-full.txt

Sets up the HUD environment with ShellTool and ApplyPatchTool for use with OpenAI.

```python
from hud.tools.coding import ShellTool, ApplyPatchTool

env = Environment("coding-env")
env.add_tool(ShellTool())
env.add_tool(ApplyPatchTool())
```

--------------------------------

### Example HUD Workflow Commands

Source: https://docs.hud.ai/

Provides a sequence of commands for local development and evaluation of HUD environments. Includes login, initialization, local server start, evaluation, and deployment.

```bash
hud login                   # Authenticate (one-time)
hud init my-env             # Scaffold environment
cd my-env
hud dev env:env -w env.py   # Run MCP server locally with hot-reload on watched paths
hud eval tasks.json claude  # Run an eval locally
hud deploy                  # Deploy to platform → run at scale

```

--------------------------------

### Analysis using Configuration File

Source: https://docs.hud.ai/llms-full.txt

Example of analyzing an environment using a configuration file.

```bash
hud analyze --config mcp-config.json
```

--------------------------------

### Create and Run ClaudeAgent Example

Source: https://docs.hud.ai/reference/agents

Example demonstrating how to create a `ClaudeAgent` instance with specific model and token limits, and then use it to run a task derived from an environment.

```python
from hud import Environment
from hud.agents.claude import ClaudeAgent

env = Environment("browser").connect_hub("hud-evals/browser")

agent = ClaudeAgent.create(
    model="claude-sonnet-4-5",
    max_tokens=8192,
)

# Create task from scenario
task = env("navigate", url="https://example.com")
result = await agent.run(task)
```

--------------------------------

### Install HUD CLI with uv

Source: https://docs.hud.ai/llms-full.txt

Installs the HUD CLI using uv, the recommended package installer. This is followed by logging into the HUD service.

```bash
uv tool install hud-python@latest --python 3.12
hud login
```

--------------------------------

### Typical Environment Setup with Web Tools

Source: https://docs.hud.ai/tools/web

Set up the environment and add both client-side (PlaywrightTool) and hosted (WebSearchTool) web tools.

```python
from hud import Environment
from hud.tools import PlaywrightTool
from hud.tools.hosted import WebSearchTool

env = Environment("web-env")
env.add_tool(PlaywrightTool())
env.add_tool(WebSearchTool())
```

--------------------------------

### serve()

Source: https://docs.hud.ai/reference/environments

Starts a standalone MCP server using the specified transport protocol.

```APIDOC
## serve()

### Description
Starts a standalone MCP server. This method is blocking.

### Parameters
#### Query Parameters
- **transport** (Literal["stdio", "sse", "streamable-http"]) - Optional - Transport protocol. Default: "streamable-http"
- **host** (str) - Optional - Host address to bind. Default: "0.0.0.0"
- **port** (int) - Optional - Port to bind. Default: 8000
```

--------------------------------

### Starting Development Server with HUD

Source: https://docs.hud.ai/reference/cli/init

Commands to start the development server for a HUD environment. Use --watch for hot-reloading and --inspector for the HTTP inspector.

```bash
# Inspector (HTTP, visual)
hud dev --inspector

# Interactive TUI (arrow keys)
hud dev --interactive

# Hot-reload specific paths
hud dev -w controller -w environment --inspector
```

--------------------------------

### CI/CD Integration Example

Source: https://docs.hud.ai/reference/cli/overview

A bash script demonstrating how to debug and verify tools within a CI pipeline.

```bash
#!/bin/bash
set -e

# Test environment
hud debug "$IMAGE_NAME"

# Verify tools
TOOLS=$(hud analyze "$IMAGE_NAME" --format json | jq '.tools | length')
if [ "$TOOLS" -lt 3 ]; then
  echo "Not enough tools!"
  exit 1
fi
```

--------------------------------

### Install HUD Python in Development Mode

Source: https://docs.hud.ai/contributing

Install the SDK in development mode using uv, including development dependencies.

```bash
uv pip install -e ".[dev]"
uv tool install --force --from "." hud-python --refresh
```

--------------------------------

### Setup Environment and GroundedComputerTool

Source: https://docs.hud.ai/llms-full.txt

Register a computer tool with the environment and create a Grounder instance. This setup is necessary for using grounded calls within an environment context.

```python
from hud import Environment
from hud.tools import AnthropicComputerTool
from hud.tools.grounding import GroundedComputerTool, Grounder, GrounderConfig

# Setup environment with computer tool
env = Environment("grounded-env")
env.add_tool(AnthropicComputerTool())

# Create grounder
config = GrounderConfig(
    api_base="https://api.openai.com/v1",
    model="gpt-4o",
    api_key="your-api-key",
)
grounder = Grounder(config=config)
```

--------------------------------

### Setup Scenario Environment

Source: https://docs.hud.ai/reference/cli/misc

Retrieves the prompt for a specific scenario and environment. Requires `hud dev` to be running.

```bash
hud scenario setup my-env:echo --args '{"message": "hello"}'
```

--------------------------------

### Typical Coding Environment Setup

Source: https://docs.hud.ai/llms-full.txt

Sets up a standard coding environment by adding essential file system and shell tools to the HUD environment.

```python
from hud import Environment
from hud.tools import BashTool, EditTool
from hud.tools.filesystem import ReadTool, GrepTool, GlobTool, ListTool

env = Environment("coding-env")
env.add_tool(BashTool())
env.add_tool(EditTool())
env.add_tool(ReadTool())
env.add_tool(GrepTool())
env.add_tool(GlobTool())
env.add_tool(ListTool())
```

--------------------------------

### Install and Login to HUD CLI

Source: https://docs.hud.ai/

Installs the HUD CLI and logs in to authenticate with hud.ai, storing the API key locally. Can also set the API key manually.

```bash
# Install CLI
uv tool install hud-python --python 3.12

# Login
hud login

```

```bash
hud set HUD_API_KEY=your-key-here
```

--------------------------------

### Fast Metadata Analysis Example

Source: https://docs.hud.ai/llms-full.txt

Example of performing a fast metadata analysis for an environment.

```bash
# Fast metadata analysis
hud analyze hudpython/text-2048:latest
```

--------------------------------

### Start HUD Development Server

Source: https://docs.hud.ai/llms-full.txt

Commands to start the HUD development server. Use `--watch` for hot-reloading specific paths or directories.

```bash
# Inspector (HTTP, visual)
hud dev --inspector
```

```bash
# Interactive TUI (arrow keys)
hud dev --interactive
```

```bash
# Hot-reload specific paths
hud dev -w controller -w environment --inspector
```

--------------------------------

### Development Server

Source: https://docs.hud.ai/llms-full.txt

Starts the development server with hot-reloading enabled for specified paths.

```bash
hud dev -w controller -w environment
```

--------------------------------

### Typical Setup for Claude

Source: https://docs.hud.ai/llms-full.txt

Sets up the HUD environment with BashTool and EditTool for use with Claude.

```python
from hud import Environment
from hud.tools import BashTool, EditTool

env = Environment("coding-env")
env.add_tool(BashTool())
env.add_tool(EditTool())
```

--------------------------------

### Example .dockerignore File

Source: https://docs.hud.ai/llms-full.txt

A sample .dockerignore file to exclude specific files and directories from the Docker build context.

```ignore
# .dockerignore
.git
.venv
__pycache__
*.pyc
node_modules
.env
.env.*
```

--------------------------------

### Setup Gemini Agent Environment

Source: https://docs.hud.ai/tools/filesystem

Initialize an Environment and add tools specifically for Gemini agents.

```python
from hud import Environment
from hud.tools.coding import GeminiShellTool, GeminiEditTool
from hud.tools.filesystem import (
    GeminiReadTool,
    GeminiSearchTool,
    GeminiGlobTool,
    GeminiListTool,
)

env = Environment("gemini-env")
env.add_tool(GeminiShellTool())
env.add_tool(GeminiEditTool())
env.add_tool(GeminiReadTool())
env.add_tool(GeminiSearchTool())
env.add_tool(GeminiGlobTool())
env.add_tool(GeminiListTool())
```

--------------------------------

### Get Environment Tool

Source: https://docs.hud.ai/llms-full.txt

Retrieves environment details and build status.

```text
`get_environment`
```

--------------------------------

### Start Development Server with Build and HTTP Transport

Source: https://docs.hud.ai/reference/cli/dev

This command starts the development server, builds the project, and uses HTTP transport on port 8765. It's often used for integrations like Cursor.

```bash
hud dev . --build --transport http --port 8765
```

--------------------------------

### Start HTTP Transport Development Server

Source: https://docs.hud.ai/reference/cli/dev

Use this command to start the development server with HTTP transport, enabling web browser access and the MCP Inspector. Specify a port for the server.

```bash
hud dev . --transport http --port 8765
```

--------------------------------

### Build and Start Hud Dev

Source: https://docs.hud.ai/reference/cli/dev

Use the `--build` or `-b` flag to build the Docker image before starting the development server. This ensures the latest image is used.

```bash
hud dev --build
```

--------------------------------

### Analysis using Cursor and Live Mode

Source: https://docs.hud.ai/llms-full.txt

Example of analyzing an environment using a Cursor server in live mode.

```bash
# Cursor and config
hud analyze --cursor my-dev-server --live
```

--------------------------------

### Instantiate OpenAIAgent

Source: https://docs.hud.ai/llms-full.txt

Example of creating an OpenAIAgent with specified model, token limits, and temperature.

```python
agent = OpenAIAgent.create(
    model="gpt-4o",
    max_output_tokens=2048,
    temperature=0.7,
)
```

--------------------------------

### Quick Example: Using Native Graders

Source: https://docs.hud.ai/reference/native-graders

Demonstrates how to use `hud.native` graders within a scenario to perform parallel evaluations and combine results.

```APIDOC
## Quick Example: Using Native Graders

This example shows how to integrate `hud.native` graders into a scenario for parallel execution and result aggregation.

```python theme={null}
from hud import Environment
from hud.native import BashGrader, Grade, exact_match

env = Environment("coding-env")

@env.scenario("fix-tests")
async def fix_tests():
    yield "Make the checkout tests pass"

    yield await Grade.gather(
        BashGrader.grade(weight=0.7, command="pytest tests/test_checkout.py -q"),
        BashGrader.grade(weight=0.3, command="ruff check ."),
    )
```
```

--------------------------------

### Local Development Server

Source: https://docs.hud.ai/llms-full.txt

Starts a local development server. Use the --watch flag for hot-reloading, specifying the controller directory.

```bash
hud dev . -w controller
```

--------------------------------

### Instantiate GeminiAgent

Source: https://docs.hud.ai/llms-full.txt

Example of creating a GeminiAgent with a specific model, temperature, and token limit.

```python
agent = GeminiAgent.create(
    model="gemini-2.5-pro",
    temperature=0.7,
    max_output_tokens=4096,
)
```

--------------------------------

### Install HUD CLI

Source: https://docs.hud.ai/llms-full.txt

Installs the HUD Python CLI for version 3.12. This is a one-time setup step.

```bash
uv tool install hud-python --python 3.12

hud login
```

--------------------------------

### Analyze Output Formats

Source: https://docs.hud.ai/reference/cli/overview

Examples of retrieving analysis results in different formats.

```bash
hud analyze my-env
```

```bash
hud analyze my-env --format json

{
  "tools": [{
    "name": "click",
    "description": "Click at coordinates",
    "parameters": {...}
  }]
}
```

```bash
hud analyze my-env --format markdown > docs/tools.md
```

--------------------------------

### Serve and Connect to Environment

Source: https://docs.hud.ai/guides/mcp-to-a2a

Commands to launch the environment server and initiate a client connection.

```bash
# Serve it
HUD_ENV=github-assistant HUD_SCENARIO=chat \
    uv run python examples/03_a2a_chat_server.py

# Talk to it
uv run python examples/05_a2a_simple_client.py
```

--------------------------------

### Start Development Server for Cursor Integration

Source: https://docs.hud.ai/llms-full.txt

Starts the HUD development server with build enabled, HTTP transport, and on port 8765. This setup is required for integrating with Cursor.

```bash
hud dev . --build --transport http --port 8765
```

--------------------------------

### Cursor One-Click Install for HUD MCP Server

Source: https://docs.hud.ai/llms-full.txt

Use this link to automatically install the HUD MCP server configuration in Cursor. It simplifies the setup process for integrating HUD context.

```html
<a href="https://cursor.com/en/install-mcp?name=docs-hud-so&config=eyJ1cmwiOiJodHRwczovL2RvY3MuaHVkLmFpL21jcCJ9">Install MCP Server</a>
```

--------------------------------

### Get Help via Slack Command

Source: https://docs.hud.ai/llms-full.txt

Displays available commands and usage examples for the HUD AI bot in Slack.

```bash
@hud help
```

--------------------------------

### Python Testing Example

Source: https://docs.hud.ai/reference/mcpserver

Connect to an environment and interact with tools using the HUD Python client. Demonstrates listing tools and calling a tool.

```python
# Python testing
async def test():
    from hud import Environment
    
    env = Environment("test").connect_mcp_config({
        "env": {"command": "docker", "args": ["run", "-i", "my-env"]}
    })
    
    async with env:
        tools = await env.list_tools()
        result = await env.call_tool("setup", value=0)
```

--------------------------------

### HUD Deploy JSON Configuration

Source: https://docs.hud.ai/llms-full.txt

Example of the .hud/deploy.json file created after a successful deployment, linking the directory to the platform environment.

```json
{
  "registryId": "abc123-def456-...",
  "version": "0.1.0"
}
```

--------------------------------

### Typical Setup for Generic Agent with Session Memory

Source: https://docs.hud.ai/tools/memory

Sets up a generic environment with BashTool and SessionMemoryTool. This is suitable for any agent that requires simple, session-based key-value memory.

```python
from hud import Environment
from hud.tools import BashTool
from hud.tools.memory import SessionMemoryTool

env = Environment("generic-env")
env.add_tool(BashTool())
env.add_tool(SessionMemoryTool())
```

--------------------------------

### MCP Environment Dockerfile

Source: https://docs.hud.ai/reference/mcpserver

Dockerfile for building an MCP environment image. It installs dependencies and configures the server to start the optional backend and the MCP controller.

```dockerfile
FROM python:3.11-slim

WORKDIR /app

# Copy and install
COPY pyproject.toml ./
COPY controller/ ./controller/
COPY environment/ ./environment/
RUN pip install --no-cache-dir -e .

ENV ENV_SERVER_PORT=8005

# Start optional backend, then MCP controller on stdio
CMD ["sh", "-c", "uvicorn environment.server:app --host 0.0.0.0 --port $ENV_SERVER_PORT --log-level warning & python -m controller"]

```

--------------------------------

### Complete HUD Codex Example Script

Source: https://docs.hud.ai/cookbooks/codex-coding

A full runnable Python script demonstrating the setup of a HUD environment with ShellTool and ApplyPatchTool, defining a coding task scenario, and running an agent.

```python
import asyncio
import os
import hud
from hud.agents import create_agent
from hud.tools.coding import ShellTool, ApplyPatchTool

async def main():
    # Set up working directory
    work_dir = "./codex_output"
    os.makedirs(work_dir, exist_ok=True)

    # Create environment with Codex tools
    env = hud.Environment("my-codex")
    env.add_tool(ShellTool())
    env.add_tool(ApplyPatchTool(base_path=work_dir))

    # Define scenario for evaluation
    @env.scenario("coding_task")
    async def coding_task(task: str):
        yield f"""You are a skilled software developer. Complete:

{task}

Use `shell` to run commands and `apply_patch` to create/modify files."""
        yield 1.0

    # Create agent and run
    agent = create_agent("gpt-4o", verbose=True)
    task = "Create a Python script called main.py that prints Hello World"

    async with hud.eval(env("coding_task", task=task), name="codex-local") as ctx:
        await agent.run(ctx, max_steps=20)

    print(f"Reward: {ctx.reward}")
    print(f"Files: {os.listdir(work_dir)}")

asyncio.run(main())
```

--------------------------------

### HUD CLI Command Examples

Source: https://docs.hud.ai/reference/cli/overview

Commonly used commands for environment management, task synchronization, and local development.

```bash
hud init my-env
```

```bash
hud deploy .
```

```bash
hud sync env my-env
```

```bash
hud sync tasks my-taskset
```

```bash
hud sync
```

```bash
hud dev . -w controller
```

```bash
hud build . --tag v1.0
```

```bash
hud scenario list
```

```bash
hud eval tasks.py claude
```

```bash
hud analyze org/env
```

```bash
hud debug my-env:latest
```

```bash
hud rl run my-taskset -m <model-id>
```

```bash
hud push .
```

```bash
hud cancel <job_id>
```

```bash
hud login
```

```bash
hud set HUD_API_KEY=...
```

```bash
hud models --json
```

```bash
hud convert ./tasks --from harbor
```

```bash
hud version
```

--------------------------------

### @initialize Decorator

Source: https://docs.hud.ai/reference/mcpserver

The @initialize decorator allows you to define an asynchronous setup function that runs during the MCP initialize request. This is useful for setting up environment resources before the server starts handling agent requests.

```APIDOC
## @initialize Decorator

### Description
Run async setup during MCP initialize request.

### Usage
```python
mcp = MCPServer(name="my-env")

@mcp.initialize
async def setup_environment(ctx):
    # Setup logic here
    pass
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Arguments for decorated function
- **ctx** (RequestContext): Contains `meta` (client metadata dict) and `session` (MCP ServerSession).
```

--------------------------------

### Environment Variable Setup

Source: https://docs.hud.ai/llms-full.txt

Configure API keys for local or hub mode execution by creating a .env file in your project root. HUD_API_KEY is recommended for traces.

```bash
# For local mode (calls OpenAI directly)
OPENAI_API_KEY=sk-...

# For hub mode OR traces (recommended)
HUD_API_KEY=sk-hud-...
```

--------------------------------

### Orchestrator Setup with Separate Environments

Source: https://docs.hud.ai/llms-full.txt

Configures an orchestrator with multiple specialist agents, each having its own isolated environment and tools. This example shows setting up researcher and coder environments and then creating an orchestrator environment with tools for each specialist.

```python
from hud import Environment
from hud.tools import AgentTool, BashTool, EditTool
from hud.agents import create_agent
import hud

researcher_env = Environment("researcher")
# ... setup researcher tools and scenario

coder_env = Environment("coder")
coder_env.add_tool(BashTool())
coder_env.add_tool(EditTool())

@coder_env.scenario()
async def fix_bug(description: str):
    yield f"Fix the bug: {description}"
    yield 1.0

orchestrator = Environment("orchestrator")
orchestrator.add_tool(
    AgentTool(
        researcher_env("investigate"), model="gpt-4o", name="research",
    )
)
orchestrator.add_tool(
    AgentTool(
        coder_env("fix_bug"), model="claude-sonnet-4-5", name="fix_code",
    )
)

@orchestrator.scenario()
async def handle_ticket(ticket_id: str):
    yield f"Handle support ticket {ticket_id}"
    yield 1.0

async with hud.eval(orchestrator("handle_ticket", ticket_id="TICKET-456")) as ctx:
    await create_agent("gpt-4o").run(ctx)
```

--------------------------------

### Install HUD CLI with pip

Source: https://docs.hud.ai/llms-full.txt

Installs the HUD CLI using pip. After installation, you need to log in to the HUD service.

```bash
pip install hud-python
hud login
```

--------------------------------

### Run Environment with Variants

Source: https://docs.hud.ai/llms-full.txt

Demonstrates how to use an environment with specific variants and run a client completion.

```python
from hud import Environment

env = Environment("my-env")

@env.tool()
def count_letter(text: str, letter: str) -> int:
    return text.lower().count(letter.lower())

@env.scenario("count")
async def count_scenario(sentence: str, letter: str):
    answer = yield f"How many '{letter}' in '{sentence}'?"
    correct = str(sentence.lower().count(letter.lower()))
    yield correct in answer

# Run with variants
async with env("count", sentence="Strawberry", letter="r", variants={"model": ["gpt-4o", "claude"]}) as ctx:
    response = await client.chat.completions.create(
        model=ctx.variants["model"],
        messages=[{"role": "user", "content": ctx.prompt}],
        tools=ctx.as_openai_chat_tools(),
    )
    await ctx.submit(response.choices[0].message.content or "")
```

--------------------------------

### Install HUD CLI

Source: https://docs.hud.ai/reference/cli/overview

Install the HUD Python package using various package managers. Authentication is required after installation.

```bash
uv tool install hud-python@latest --python 3.12
hud login
```

```bash
pip install hud-python
hud login
```

```bash
pipx install hud-python
hud login
```

--------------------------------

### Run RL Training from Local File

Source: https://docs.hud.ai/llms-full.txt

Starts an RL training job, loading task definitions from a local JSON file.

```bash
hud rl run tasks.json -m mdl_abc123
```

--------------------------------

### Deterministic Scenario Setup: Bad vs. Good

Source: https://docs.hud.ai/llms-full.txt

Demonstrates how to ensure reproducible scenario execution. The 'Good' version explicitly seeds the database with necessary data before the agent runs, avoiding reliance on pre-existing state.

```python
# Bad: Depends on whatever state exists — non-reproducible
@env.scenario("find-user")
async def find_user(name: str):
    answer = yield f"Find the user named {name}"
    yield 1.0 if name in answer else 0.0
```

```python
# Good: Seeds known state — every run starts the same
@env.scenario("find-user")
async def find_user(name: str):
    await db.clear()
    await db.insert("users", name=name, email=f"{name}@example.com")
    
    answer = yield f"Find the user named {name}"
    yield 1.0 if name in answer else 0.0
```

--------------------------------

### Install HUD CLI with pipx

Source: https://docs.hud.ai/llms-full.txt

Installs the HUD CLI using pipx, which provides isolated environments. Log in to the HUD service after installation.

```bash
pipx install hud-python
hud login
```

--------------------------------

### Quick Start Chat with env.chat()

Source: https://docs.hud.ai/guides/chat

Use `env.chat()` for a simple way to create a chat instance. It defaults to `trace=False` and `quiet=True`, suitable for server and app usage.

```python
chat = env.chat("help", model="claude-haiku-4-5")

r1 = await chat.send("Look into account ABC-123")
print(r1.content)

r2 = await chat.send("What's their current plan?")
print(r2.content)
```

--------------------------------

### Create OpenAIChatAgent with Custom Client

Source: https://docs.hud.ai/reference/agents

Demonstrates initializing an OpenAIChatAgent with a pre-configured asynchronous OpenAI client.

```python
from hud.agents import OpenAIChatAgent
from openai import AsyncOpenAI

agent = OpenAIChatAgent.create(
    openai_client=AsyncOpenAI(base_url="http://localhost:8000/v1"),
    model="served-model",
)
```

--------------------------------

### Initialize and Deploy HUD Environment

Source: https://docs.hud.ai/reference/cli/overview

Commands to bootstrap a new project and push it to the HUD platform.

```bash
hud init my-env && cd my-env
```

```bash
hud deploy
```

--------------------------------

### V4A Diff Format Example

Source: https://docs.hud.ai/cookbooks/codex-coding

An example illustrating the V4A diff format used by the ApplyPatchTool for specifying changes to files.

```diff
@@ def hello():
    print("Hello")
+    print("Hello, World!")
```

--------------------------------

### Install Agent Dependencies

Source: https://docs.hud.ai/llms-full.txt

Installs the necessary Python packages for running Claude or Gemini agents locally. This is not required for remote execution.

```bash
uv add "hud-python[agents]"
```

--------------------------------

### Deploy an environment

Source: https://docs.hud.ai/reference/cli/deploy

Basic syntax for deploying an environment from a directory.

```bash
hud deploy [DIRECTORY] [OPTIONS]
```

--------------------------------

### Install Local Execution Dependencies

Source: https://docs.hud.ai/reference/cli/eval

Install the necessary packages for running Claude or Gemini agents locally. This is not required for remote execution.

```bash
uv add "hud-python[agents]"
```

--------------------------------

### Callback Payload Example

Source: https://docs.hud.ai/llms-full.txt

Example of a POST request body received by the webhook URL when a trace completes, containing status, results, and custom metadata.

```json
{
  "trace_id": "030083f2-...",
  "status": "completed",
  "completed_at": "2026-02-25T06:00:27Z",
  "reward": 0.6,
  "response": "I've added Essential hypertension...",
  "metadata": {"my_custom_id": "abc-123"}
}
```

--------------------------------

### Create New Environment

Source: https://docs.hud.ai/llms-full.txt

Initializes a new environment in the specified directory.

```bash
hud init my-env
```

--------------------------------

### Typical HUD Sync Workflow

Source: https://docs.hud.ai/llms-full.txt

Demonstrates the basic commands for deploying, syncing tasks, and running evaluations, highlighting the additive and diff-aware nature of `hud sync`.

```bash
hud deploy                           # deploy environment to platform
hud sync tasks my-taskset            # push local tasks to 'my-taskset'
hud eval my-taskset claude --full    # run evals against the synced tasks
```

--------------------------------

### Build a Custom QA Agent

Source: https://docs.hud.ai/platform/agents/qa

Use the prepare_qa_context helper to initialize trace data for analysis. The scenario must accept trace_id and hud_api_key as arguments.

```python
from pydantic import BaseModel, Field
from env import env
from qa_common import prepare_qa_context

class MyResult(BaseModel):
    verdict: str = Field(description="Your analysis verdict")
    confidence: float = Field(ge=0.0, le=1.0)

@env.scenario("my_analysis", returns=MyResult)
async def my_analysis(
    trace_id: str,
    hud_api_key: str,
    query: str = "",
    ground_truth: str | None = None,
) -> Any:
    _, _, context = await prepare_qa_context(
        trace_id, hud_api_key, "My analysis"
    )

    prompt = f"""Your analysis instructions here.

{context}

## Focus
{query or "Default analysis question."}"""

    response: MyResult = yield prompt

    if ground_truth is not None:
        yield 1.0 if response.verdict == ground_truth else 0.0
    else:
        yield 1.0
```

--------------------------------

### CLI Command: hud init

Source: https://docs.hud.ai/reference/cli/init

Scaffolds a working MCP environment using templates from the public SDK.

```APIDOC
## hud init

### Description
The `hud init` command scaffolds a working MCP environment using templates from the public SDK. It creates a directory structure containing a controller, backend, and configuration files.

### Usage
`hud init [NAME] [OPTIONS]`

### Arguments
- **name** (string) - Optional - Environment name. If omitted, the current directory name is used.

### Options
- **--preset** (string) - Optional (default: blank) - Template preset: `blank`, `deep-research`, or `browser`. Short: `-p`
- **--dir** (string) - Optional (default: .) - Target directory where the environment will be created. Short: `-d`
- **--force** (boolean) - Optional (default: false) - Overwrite existing files if they exist. Short: `-f`

### Examples
```bash
# Choose preset interactively
hud init

# Create a blank template in a new directory
hud init my-env -p blank

# Force overwrite
hud init my-env -p blank --force
```
```

--------------------------------

### Linking on a New Machine

Source: https://docs.hud.ai/reference/cli/link

When setting up your project on a new machine, clone the repository and run `hud link`. You can then select your environment from the interactive list to re-establish the link.

```bash
# On new machine, link to existing environment
git clone your-repo && cd your-repo
hud link
# Select your environment from the list
```

--------------------------------

### Running HUD Codex CLI Example - Custom Task

Source: https://docs.hud.ai/cookbooks/codex-coding

Command to run the HUD Codex example script in local mode with a custom task defined for the agent.

```bash
# Custom task
uv run python examples/06_codex_coding_agent.py --local \
  --task "Create a Python script that prints the Fibonacci sequence up to 10 numbers"
```

--------------------------------

### Minimal MCPServer Environment Setup

Source: https://docs.hud.ai/reference/mcpserver

Sets up a minimal MCPServer environment named 'counter-env' with tools for initializing, incrementing, and evaluating a counter. This is suitable for basic testing and development.

```python
# src/hud_controller/server.py
from hud.server import MCPServer
from mcp.types import TextContent

mcp = MCPServer(name="counter-env")
counter = {"value": 0}

@mcp.tool()
async def setup(start_value: int = 0):
    """Initialize counter."""
    counter["value"] = start_value
    return {"status": "ready", "counter": counter["value"]}

@mcp.tool()
async def increment():
    """Increment counter."""
    counter["value"] += 1
    return [TextContent(text=f"Counter: {counter['value']}", type="text")]

@mcp.tool()
async def evaluate(target: int):
    """Check if target reached."""
    from hud.tools.types import EvaluationResult
    return EvaluationResult(
        reward=1.0 if counter["value"] >= target else 0.0,
        done=counter["value"] >= target
    )

if __name__ == "__main__":
    mcp.run()
```

--------------------------------

### Running HUD Codex CLI Example - Hub Mode

Source: https://docs.hud.ai/cookbooks/codex-coding

Command to run the HUD Codex example script in hub mode, which uses full cloud execution.

```bash
# Hub mode - full cloud execution (default)
uv run python examples/06_codex_coding_agent.py
```

--------------------------------

### Initialize a New HUD Environment

Source: https://docs.hud.ai/llms-full.txt

Use `hud init` to create a new HUD environment with essential configuration files and directories. Navigate into the created directory to proceed.

```bash
hud init my-env && cd my-env
```

--------------------------------

### Running HUD Codex CLI Example - Local Mode

Source: https://docs.hud.ai/cookbooks/codex-coding

Command to run the HUD Codex example script in local mode, where tools execute on the local machine.

```bash
# Local mode - tools run on your machine
uv run python examples/06_codex_coding_agent.py --local
```

--------------------------------

### Basic Build

Source: https://docs.hud.ai/llms-full.txt

Builds the Docker image with an automatically generated tag.

```bash
hud build
```

--------------------------------

### Start Standalone MCP Server

Source: https://docs.hud.ai/reference/environments

Use `env.serve()` to start a standalone MCP server. This method blocks execution. It supports various transport protocols like stdio, sse, and streamable-http.

```python
from hud import Environment

env = Environment("my-env")

@env.tool()
def greet(name: str) -> str:
    return f"Hello, {name}!"

# Run as MCP server (blocking)
env.serve()
```

```python
# Serve over stdio (for CLI tools)
env.serve(transport="stdio")
```

```python
# Serve over HTTP on custom port
env.serve(transport="streamable-http", host="0.0.0.0", port=8765)
```

--------------------------------

### Create Task with Environment

Source: https://docs.hud.ai/llms-full.txt

Demonstrates how to create a task by instantiating an Environment and calling it with a scenario name and arguments. Returns a Task object.

```python
from hud import Environment

env = Environment("my-env")
task = env("scenario_name", arg1="value")  # Returns Task
```

--------------------------------

### Running HUD Codex CLI Example - Local Mode with Persistent Output

Source: https://docs.hud.ai/cookbooks/codex-coding

Command to run the HUD Codex example script in local mode, specifying a persistent directory for output files.

```bash
# Local mode with persistent output directory
uv run python examples/06_codex_coding_agent.py --local --work-dir ./codex_output
```

--------------------------------

### Run Scenario MCP Protocol Mapping

Source: https://docs.hud.ai/llms-full.txt

This snippet demonstrates how to interact with MCP endpoints for prompt retrieval and resource reading within a scenario. It shows the asynchronous flow of getting a prompt, submitting an answer, and then reading a resource to get a reward.

```python
async with env:
    prompt_result = await env.get_prompt(
        "myenv:checkout", 
        {"product": "laptop", "user_id": "alice"}
    )
    print(f"Prompt: {prompt_result.messages[0].content}")
    
    await env.submit("checkout", answer="Order completed successfully")
    resource_result = await env.read_resource("myenv:checkout")
    print(f"Reward: {resource_result}")  # {"reward": 1.0}
```

--------------------------------

### Running the Example Script

Source: https://docs.hud.ai/llms-full.txt

Execute the HUD AI script using 'uv run'. Options include local mode, specifying a work directory, or running in hub mode. Custom tasks and verbose output are also supported.

```bash
# Local mode - tools run on your machine
uv run python examples/06_codex_coding_agent.py --local

# Local mode with persistent output directory
uv run python examples/06_codex_coding_agent.py --local --work-dir ./codex_output

# Hub mode - full cloud execution (default)
uv run python examples/06_codex_coding_agent.py

# Custom task
uv run python examples/06_codex_coding_agent.py --local \
  --task "Create a Python script that prints the Fibonacci sequence up to 10 numbers"

# Verbose output
uv run python examples/06_codex_coding_agent.py --local --verbose
```

--------------------------------

### Full Scenario Run

Source: https://docs.hud.ai/llms-full.txt

Executes a full scenario, including setup and grading, with provided arguments and answer. The output includes the prompt, reward, and done status.

```bash
hud scenario run my-env:echo --args '{"message": "hello"}' --answer "hello"
# Prompt: Repeat this back exactly: hello
# {"reward": 1.0, "done": true}
```

--------------------------------

### Get Job Tool

Source: https://docs.hud.ai/llms-full.txt

Retrieves job details and summary.

```text
`get_job`
```

--------------------------------

### Setup Environment with GroundedComputerTool

Source: https://docs.hud.ai/tools/grounding

Integrate GroundedComputerTool within an HUD Environment, registering a computer tool and setting up the grounder. This allows for grounded actions within the environment.

```python
from hud import Environment
from hud.tools import AnthropicComputerTool
from hud.tools.grounding import GroundedComputerTool, Grounder, GrounderConfig

# Setup environment with computer tool
env = Environment("grounded-env")
env.add_tool(AnthropicComputerTool())

# Create grounder
config = GrounderConfig(
    api_base="https://api.openai.com/v1",
    model="gpt-4o",
    api_key="your-api-key",
)
grounder = Grounder(config=config)

async with env:
    # Wrap environment for grounded calls
    grounded = GroundedComputerTool(grounder=grounder, ctx=env)
    
    # Take screenshot via environment
    result = await env.call_tool("computer", action="screenshot")
    
    # Use grounded tool for element-based actions
    await grounded(
        action="click",
        element_description="the login button",
        screenshot_b64=result.content[0].data,  # base64 from screenshot
    )
```

--------------------------------

### Define a Sample Environment

Source: https://docs.hud.ai/building/tasks-and-evaluation

Create a tool and a scenario within an environment to enable agent interaction and scoring.

```python
from hud import Environment

env = Environment("letter-counter")

@env.tool()
def count_letter(text: str, letter: str) -> int:
    """Count occurrences of a letter in text."""
    return text.lower().count(letter.lower())

@env.scenario("count")
async def count(word: str, letter: str):
    answer = yield f"How many '{letter}' in '{word}'?"

    correct = str(word.lower().count(letter.lower()))
    yield 1.0 if answer and correct in answer else 0.0
```

--------------------------------

### Get Evalset Tasks Tool

Source: https://docs.hud.ai/llms-full.txt

Retrieves tasks within a specific evalset.

```text
`get_evalset_tasks`
```

--------------------------------

### Initialize a Task

Source: https://docs.hud.ai/reference/types

Create a task by initializing an Environment and calling it with a scenario name.

```python
from hud import Environment

env = Environment("my-env")
task = env("scenario_name", arg1="value")  # Returns Task
```

--------------------------------

### Get Trace Tool

Source: https://docs.hud.ai/llms-full.txt

Retrieves the full trace, including trajectory and logs.

```text
`get_trace`
```

--------------------------------

### Initialize Environment

Source: https://docs.hud.ai/reference/environments

Create an instance of the Environment class to manage tools and integrations.

```python
from hud import Environment

env = Environment("my-env")
```

--------------------------------

### Get Job Traces Tool

Source: https://docs.hud.ai/llms-full.txt

Retrieves traces within a specific job.

```text
`get_job_traces`
```

--------------------------------

### Taskset Creation Response

Source: https://docs.hud.ai/platform/rest-api

Example JSON response returned after creating or updating a taskset.

```json
{
  "evalset_id": "a1b2c3d4-...",
  "evalset_name": "my-checkout-taskset",
  "tasks_created": 2,
  "tasks_updated": 0
}
```

--------------------------------

### Execute Commands with BashTool

Source: https://docs.hud.ai/tools/coding

Demonstrates command execution, chaining, and session management.

```python
# Execute command
result = await bash(command="ls -la")

# Chain commands (session persists)
await bash(command="cd /app")
await bash(command="npm install")

# Restart if session dies
await bash(restart=True)
```

--------------------------------

### Configure Environment with Native Tools

Source: https://docs.hud.ai/building/scaffolding

Initialize an environment and register tools that automatically adapt to the connected agent's provider.

```python
from hud import Environment
from hud.tools import AnthropicComputerTool, BashTool, EditTool

env = Environment("desktop-agent")
env.add_tool(AnthropicComputerTool())
env.add_tool(BashTool())
env.add_tool(EditTool())
```

```python
from hud import Environment
from hud.tools import AnthropicComputerTool, BashTool, EditTool
from hud.tools.filesystem import ReadTool, GrepTool

env = Environment("desktop-agent")
env.add_tool(AnthropicComputerTool())
env.add_tool(BashTool())
env.add_tool(EditTool())
env.add_tool(ReadTool())
env.add_tool(GrepTool())
```

--------------------------------

### Skip Confirmation Prompt

Source: https://docs.hud.ai/llms-full.txt

Runs the evaluation without prompting for user confirmation before starting.

```bash
# Skip confirmation
hud eval tasks.json claude --full -y
```
Tokens:
Raw