===============
LIBRARY RULES
===============
From library maintainers:
- Coval evaluates AI agents — it is NOT an AI agent framework or LLM wrapper
- Voice agent testing is a primary use case — always consider voice-specific features like personas, accents, and background noise
- "Simulations" means automated test conversations between a persona and the user's agent, not generic simulations
- "Metrics" are scoring criteria applied to simulation results — types include LLM judge, audio analysis, regex, tool call, and composite 
- "Personas" are simulated callers/users with configurable voice, accent, behavior, and background noise
- "Mutations" are agent configuration variants for A/B testing, not code mutations
- The CLI is installed via Homebrew (brew install coval-ai/tap/coval), Cargo, or binary download
- The API base URL is https://api.coval.dev/v1/ and requires a Bearer token (API key)
- When showing CLI examples, use the "coval" command prefix
- "Runs" are evaluation sessions that execute multiple simulations against an agent with a test set, persona, and metrics
- Coval supports monitoring production conversations, not just pre-deployment testing
- Test cases can include expected behaviors for composite metric evaluation

### Core Configuration Example

Source: https://docs.coval.dev/guides/simulations/chat

A minimal configuration to get started with chat simulations.

```APIDOC
## Quick Start Configuration

### Minimum Required Configuration:

1.  **Chat Endpoint**: The URL where your agent receives messages.
2.  **Authorization Header**: Authentication credentials for your API.

That’s it! All other fields are optional and depend on your specific API requirements.
```

--------------------------------

### Example Agent Context

Source: https://docs.coval.dev/concepts/metrics/prompting

A sample system prompt configuration for a healthcare scheduling assistant to guide evaluation.

```text
This is a healthcare scheduling assistant that helps patients with:
- Booking new appointments with available providers
- Rescheduling existing appointments (requires 24-hour notice)
- Canceling appointments
- Answering questions about office locations and hours
```

--------------------------------

### Example Initialization Endpoint URL

Source: https://docs.coval.dev/guides/simulations/chat

URL called once before a conversation starts to set up session state. Must use HTTPS.

```text
https://api.yourdomain.com/init
```

--------------------------------

### Example Annotation Creation

Source: https://docs.coval.dev/cli/human-review

A practical example of how to invoke the create command with specific flags.

```shellscript
# Create a basic annotation
coval review-annotations create
```

--------------------------------

### Project Configuration JSON Example

Source: https://docs.coval.dev/guides/human-review-api

Example configuration object for a voice agent review project.

```json
"display_name": "Q1 Voice Agent Review",
 "description": "Review accuracy and latency for Q1 voice simulations",
 "assignees": ["alice@company.com", "bob@company.com"],
 "linked_simulation_ids": ["sim-output-001", "sim-output-002"],
 "linked_metric_ids": ["metric-accuracy", "metric-latency"],
 "project_type": "PROJECT_COLLABORATIVE",
 "notifications": true
```

--------------------------------

### Example API requests

Source: https://docs.coval.dev/api-reference/v1/introduction

Example commands for interacting with the OpenAPI endpoints.

```shellscript
# List all available specs
curl -s https://api.coval.dev/v1/openapi
```

--------------------------------

### Coval CLI Usage Examples

Source: https://docs.coval.dev/cli/runs

Examples demonstrating how to use the Coval CLI with different options.

```APIDOC
## Coval CLI Usage Examples

### Description
Provides examples of how to execute Coval commands.

### Request Example
```shell
# Basic run
coval runs launch ""

# Run with options
coval runs launch " --agent-id ag_abc123" " --persona-id per_xyz789" " --test-set-id ts_123456"
```
```

--------------------------------

### Print Audio URL Example

Source: https://docs.coval.dev/cli/simulations

Example usage for retrieving the audio URL for a specific simulation ID.

```shellscript
# Print audio URL
coval simulations audio sim_abc123
```

--------------------------------

### Install OpenTelemetry Packages

Source: https://docs.coval.dev/concepts/simulations/traces/opentelemetry

Install the necessary Python packages for OpenTelemetry SDK and OTLP HTTP exporter.

```bash
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```

--------------------------------

### Create Chart Widget Example

Source: https://docs.coval.dev/cli/dashboards

Example of how to create a chart widget using the Coval CLI.

```APIDOC
## POST /api/widgets/create (Example)

### Description
This endpoint allows for the creation of new widgets within a dashboard.

### Method
POST

### Endpoint
/api/widgets/create

### Request Body
- **dashboard_id** (string) - Required - The ID of the dashboard to add the widget to.
- **widget_type** (string) - Required - The type of widget to create (e.g., 'chart', 'table', 'text').
- **widget_name** (string) - Required - The name of the widget.
- **configuration** (object) - Optional - Configuration options specific to the widget type.
  - **--grid-w** (number) - Optional - Grid width.
  - **--grid-h** (number) - Optional - Grid height.
  - **--grid-x** (number) - Optional - Grid X position.
  - **--grid-y** (number) - Optional - Grid Y position.

### Request Example
```json
{
  "dashboard_id": "db_abc123",
  "widget_type": "chart",
  "widget_name": "Score Trends",
  "configuration": {
    "--grid-w": 4,
    "--grid-h": 3,
    "--grid-x": 0,
    "--grid-y": 0
  }
}
```

### Response
#### Success Response (200)
- **widget_id** (string) - The ID of the newly created widget.
- **message** (string) - Confirmation message.

#### Response Example
```json
{
  "widget_id": "wid_xyz789",
  "message": "Chart widget 'Score Trends' created successfully."
}
```
```

--------------------------------

### Verify Coval CLI Installation

Source: https://docs.coval.dev/cli/installation

Run this command after installation to ensure the Coval CLI is set up correctly and accessible.

```bash
coval --help
```

--------------------------------

### Coval API Usage Examples

Source: https://docs.coval.dev/api-reference/v1/introduction

Examples for listing specs and fetching them in different formats using curl.

```shellscript
# List all available specs
curl -s https://api.coval.dev/v1/openapi

# Fetch a spec as YAML (default)
curl -s https://api.coval.dev/v1/openapi/agents

# Fetch a spec as JSON
curl -s -H "Accept: application/json" https://api.coval.dev/v1/openapi/agents
```

--------------------------------

### Create a widget with configuration

Source: https://docs.coval.dev/cli/dashboards

Example of creating a widget with specific metrics, grid dimensions, and configuration settings.

```shellscript
coval dashboards widgets create db_abc123 \
 --name "Detailed Report" \
 --type table \
 --config @widget-config.json
```

--------------------------------

### Setup Instructions

Source: https://docs.coval.dev/concepts/agents/connections/pipecat

Step-by-step guide to deploy your agent and connect it to Coval.

```APIDOC
## Setup Instructions

1.  **Deploy your agent to Pipecat Cloud**
    *   Follow the [Pipecat Cloud Quickstart](https://docs.pipecat.daily.co/quickstart) to deploy your agent.
    *   Confirm your agent is listed in the Pipecat Cloud dashboard.

2.  **Connect it to Coval**
    *   Go to [app.coval.dev/coval/agents/create](https://app.coval.dev/coval/agents/create).
    *   Enter the exact agent name as it appears in Pipecat Cloud.
    *   Input your Pipecat API key.
    *   Add any required custom data in JSON format.

3.  **Run a simulation**
    *   Create a test set with scenarios for your agent.
    *   Launch a simulation to verify the connection works end-to-end.
```

--------------------------------

### Initialization JSON Payload

Source: https://docs.coval.dev/concepts/agents/connections/websocket

Example of the JSON object sent immediately upon establishing a connection for session setup.

```json
{
  "action": "start_session",
  "session_type": "simulation",
  "metadata": {
    "source": "coval",
    "test_mode": true
  }
}
```

--------------------------------

### Create a Review Project

Source: https://docs.coval.dev/cli/human-review

Example command to initialize a new collaborative review project with specific name, assignees, and configuration flags.

```shellscript
# Create a collaborative review project
coval review-projects create \
 --name "Q1 Voice Agent Review" \
 --assignees "alice@company.com,bob@company.com" \

```

--------------------------------

### Get Run Status via JSON Output

Source: https://docs.coval.dev/cli/overview

Example of using the --format json flag to retrieve machine-readable status information.

```shellscript
# Get run status
coval
```

--------------------------------

### Create Tool Usage Metrics

Source: https://docs.coval.dev/cli/metrics

Example command for initializing a tool usage metric via the Coval CLI.

```bash
_jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#953800\",\n \"--shiki-dark\": \"#DCDCAA\"\n },\n children: \"coval\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" metrics\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" create\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --name\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\"Tool Usage\\"\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --description\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \"
```

--------------------------------

### Full monitoring example in Python

Source: https://docs.coval.dev/concepts/simulations/traces/opentelemetry

A complete implementation for importing necessary libraries and setting up OpenTelemetry tracing.

```python
import requests
from opentelemetry.sdk import trace as trace_sdk
```

--------------------------------

### Coval Configuration in Code

Source: https://docs.coval.dev/cli/installation

Example of setting Coval API key and URL using JSX syntax. This is typically part of a larger component or setup.

```jsx
_jsx("div", {
  children: [
    _jsx(_components.span, {
      style: {
        color: "#953800",
        "--shiki-dark": "#DCDCAA"
      },
      children: "coval"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " config"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " set"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " api_key"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " sk_your_api_key"
    })
  ]
}),
"\n",
_jsxs(_components.span, {
  className: "line",
  children: [
    _jsx(_components.span, {
      style: {
        color: "#953800",
        "--shiki-dark": "#DCDCAA"
      },
      children: "coval"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " config"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " set"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " api_url"
    }),
    _jsx(_components.span, {
      style: {
        color: "#0A3069",
        "--shiki-dark": "#CE9178"
      },
      children: " https://api.coval.dev"
    })
  ]
}),
"\n"

```

--------------------------------

### Run Coval MCP Server Locally with NPX

Source: https://docs.coval.dev/mcp/installation

Execute this command in your terminal to start the Coval MCP server locally. Ensure you have Node.js and npm installed.

```bash
npx @covalai/mcp-server
```

--------------------------------

### Example Initialization Response

Source: https://docs.coval.dev/guides/simulations/chat

JSON response from the initialization endpoint, providing session and user details.

```json
{
  "sessionId": "abc-123-def",
  "userId": "user-456",
  "conversationId": "conv-789"
}
```

--------------------------------

### Create a Basic Run Template

Source: https://docs.coval.dev/cli/run-templates

Create a new run template with essential configuration details like name, agent, persona, and test set.

```bash
coval run-templates create \
  --name "Nightly Regression" \
  --agent-id ag_abc123 \
  --persona-id per_xyz789 \
  --test-set-id ts_123456
```

--------------------------------

### Get Agent Status with jq

Source: https://docs.coval.dev/cli/overview

This command retrieves the status of agents and formats the output as JSON, then uses jq to extract the status field. Ensure you have jq installed.

```bash
coval runs --format json | jq '.status'
```

--------------------------------

### Full Monitoring Example with OpenTelemetry

Source: https://docs.coval.dev/concepts/simulations/traces/opentelemetry

This snippet shows how to set up OpenTelemetry tracing, instrument LLM and TTS calls, and export spans to Coval. Ensure you have the necessary libraries installed and replace placeholders like '<COVAL_API_KEY>'.

```python
import requests
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.sdk.resources import SERVICE_NAME, Resource

COVAL_API_KEY = "<COVAL_API_KEY>"

# --- Call setup: buffer spans in memory ---
resource = Resource.create({SERVICE_NAME: "my-agent"})
in_memory_exporter = InMemorySpanExporter()
provider = trace_sdk.TracerProvider(resource=resource)
provider.add_span_processor(SimpleSpanProcessor(in_memory_exporter))
tracer = provider.get_tracer("my-agent")

# --- During the call: instrument as normal ---
with tracer.start_as_current_span("llm") as span:
    span.set_attribute("metrics.ttfb", 0.42)
    response = call_llm()

with tracer.start_as_current_span("tts") as span:
    span.set_attribute("metrics.ttfb", 0.18)
    audio = synthesize_speech(response)

# --- After the call ends: submit transcript, then export spans ---
submit_response = requests.post(
    "https://api.coval.dev/v1/conversations:submit",
    headers={"x-api-key": COVAL_API_KEY, "Content-Type": "application/json"},
    json={"transcript": transcript},
)
conversation_id = submit_response.json()["conversation"]["conversation_id"]

otlp_exporter = OTLPSpanExporter(
    endpoint="https://api.coval.dev/v1/traces",
    headers={"X-API-Key": COVAL_API_KEY, "X-Conversation-Id": conversation_id},
    timeout=30,
)
finished_spans = in_memory_exporter.get_finished_spans()
if finished_spans:
    otlp_exporter.export(list(finished_spans))
```

--------------------------------

### Configure HTTP-First Connection

Source: https://docs.coval.dev/concepts/agents/connections/chat-websocket

Use HTTP-First connection mode with a setup endpoint.

```text
Connection Mode: HTTP-First
Endpoint: https://chat.example.com/api/v1/provision
```

--------------------------------

### Persona Prompt for Filler Words

Source: https://docs.coval.dev/concepts/personas/overview

Include explicit TTS-friendly instructions in your persona prompt to guide the AI on using natural filler words and pauses. This example specifies allowed filler words and punctuation for hesitations.

```text
<speaking_style>
You use natural filler words in conversation. When hesitating, use only these words: \"um\", \"uh\", \"hmm\", \"oh\", \"well\". Write them as single short words. Use commas for pauses instead of ellipses or repeated letters.
Example: \"Um, I think the order number is, uh, let me check, it's 12345.\"
</speaking_style>
```

--------------------------------

### Run Coval Onboarding

Source: https://docs.coval.dev/agents/onboarding

Use the `/onboard` skill to guide through setting up your first Coval evaluation. The AI coding agent will ask questions about your use case and create necessary resources.

```shellscript
coval onboard
```

--------------------------------

### Clone and Run Coval MCP Server from Source

Source: https://docs.coval.dev/mcp/installation

Steps to clone the Coval MCP server repository, install dependencies, build the project, set your API key, and start the server for local development. This method is suitable for contributing to the project or advanced customization.

```bash
git clone https://github.com/coval-ai/mcp-server
cd coval-mcp-server
npm install
npm run build

# Set your API key
export COVAL_API_KEY=your_api_key_here

# Run the server
npm start
```

--------------------------------

### Install Coval MCP Server

Source: https://docs.coval.dev/mcp/installation

Use this command to install the Coval MCP server. Ensure you have Node.js and npm installed.

```shell
npm install coval-mcp-server
```

--------------------------------

### Install Coval CLI with Cargo

Source: https://docs.coval.dev/cli/installation

Install the Coval CLI using Cargo, Rust's package manager. Requires Rust to be installed.

```bash
cargo install coval
```

--------------------------------

### List Review Projects Command Example

Source: https://docs.coval.dev/cli/human-review

An example of the 'list review projects' command, demonstrating its usage. This snippet shows the basic command structure.

```shellscript
coval review-projects list
```

--------------------------------

### GET /v1/simulations/{simulation_id}

Source: https://docs.coval.dev/cli/overview

Get a simulation by ID.

```APIDOC
## GET /v1/simulations/{simulation_id}

### Description
Get a simulation by ID.

### Method
GET

### Endpoint
/v1/simulations/{simulation_id}

### Parameters
#### Path Parameters
- **simulation_id** (string) - Required - The unique identifier of the simulation.
```

--------------------------------

### GET get_test_case

Source: https://docs.coval.dev/mcp/tools

Get detailed information about a test case.

```APIDOC
## GET get_test_case

### Description
Get detailed information about a test case.

### Method
GET

### Parameters
#### Path Parameters
- **test_case_id** (string) - Yes - Test case ID from list_test_cases
```

--------------------------------

### Coval CLI Command Examples

Source: https://docs.coval.dev/cli/human-review

Examples of using the Coval CLI to create review projects with specific configurations and parameters.

```bash
\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --simulation-ids\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\\"sim-output-001,sim-output-002\\\"\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --metric-ids\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\\"metric-accuracy,metric-latency\\\"\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --type\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" collaborative\"\n })]\n }), \"\\n\", _jsx(_components.span, {\n className: \"line\"\n }), \"\\n\", _jsx(_components.span, {\n className: \"line\",\n children: _jsx(_components.span, {\n style: {\n color: \"#6E7781\",\n \"--shiki-dark\": \"#6A9955\"\n },\n children: \"# Create with description and notifications disabled\"\n })\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#953800\",\n \"--shiki-dark\": \"#DCDCAA\"\n },\n children: \"coval\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" review-projects\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" create\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --name\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\\"Internal Audit\\\"\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --assignees\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\\"reviewer@company.com\\\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --simulation-ids\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\\"sim-output-003\\\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --metric-ids\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\\"metric-accuracy\\\"\n }), _jsx(_components.span, {\n style: {\n color: \"#CF222E\",\n \"--shiki-dark\": \"#D7BA7D\"\n },\n children: \" \\\\\"\n })]\n }), \"\\n\", _jsxs(_components.span, {\n className: \"line\",\n children: [_jsx(_components.span, {\n style: {\n color: \"#0550AE\",\n \"--shiki-dark\": \"#569CD6\"\n },\n children: \" --description\"\n }), _jsx(_components.span, {\n style: {\n color: \"#0A3069\",\n \"--shiki-dark\": \"#CE9178\"\n },\n children: \" \\\"Spot-check accuracy labels\\\"\n }), _jsx(_components.span, {\n style: {\n color: \"#C
```

--------------------------------

### GET /get_metric

Source: https://docs.coval.dev/mcp/tools

Get detailed configuration for a specific metric.

```APIDOC
## GET /get_metric

### Description
Get detailed configuration for a specific metric.

### Method
GET

### Endpoint
/get_metric

### Parameters
#### Query Parameters
- **metric_id** (string) - Yes - Metric ID from list_metrics
```

--------------------------------

### Create Review Project

Source: https://docs.coval.dev/cli/human-review

Commands to initialize new review projects with specific assignees, simulations, and metrics.

```bash
# Create a collaborative review project
coval review-projects create \
  --name "Q1 Voice Agent Review" \
  --assignees "alice@company.com,bob@company.com" \
  --simulation-ids "sim-output-001,sim-output-002" \
  --metric-ids "metric-accuracy,metric-latency" \
  --type collaborative

# Create with description and notifications disabled
coval review-projects create \
  --name "Internal Audit" \
  --assignees "reviewer@company.com" \
  --simulation-ids "sim-output-003" \
  --metric-ids "metric-accuracy" \
  --description "Spot-check accuracy labels" \
  --notifications false
```

--------------------------------

### Create a Persona via CLI

Source: https://docs.coval.dev/cli/personas

Example command for initializing a new persona with specific name and voice parameters.

```shellscript
# Create a basic persona
coval personas create \
 --name "Frustrated Customer" \
 --voice "Aria" \

```

--------------------------------

### GET /v1/review-annotations/{annotation_id}

Source: https://docs.coval.dev/cli/human-review

Get a single review annotation by ID.

```APIDOC
## GET /v1/review-annotations/{annotation_id}

### Description
Get a single review annotation by ID.

### Method
GET

### Endpoint
/v1/review-annotations/{annotation_id}

### Parameters
#### Path Parameters
- **annotation_id** (string) - Required - The unique identifier of the annotation.
```

--------------------------------

### Initialize Coval Onboarding

Source: https://docs.coval.dev/agents/onboarding

Commands to install the necessary skills and initiate the interactive onboarding process within an AI coding agent.

```bash
# 1. Install Coval skills
npx skills add coval-ai/coval-external-skills

# 2. Open your AI coding agent (Claude Code, Cursor, etc.)

# 3. Run the onboarding skill
/onboard
```

--------------------------------

### GET run-templates get

Source: https://docs.coval.dev/cli/run-templates

Retrieves details for a specific run template.

```APIDOC
## GET run-templates get

### Description
Retrieves the details of a specific run template by its ID.

### Parameters
#### Path Parameters
- **run_template_id** (string) - Required - The run template ID

### Request Example
```shell
coval run-templates get rt_abc123
```
```

--------------------------------

### GET coval personas get

Source: https://docs.coval.dev/cli/personas

Retrieves a specific persona by its ID.

```APIDOC
## GET coval personas get

### Description
Retrieves details for a specific persona using its unique identifier.

### Endpoint
coval personas get <persona_id>

### Parameters
#### Path Parameters
- **persona_id** (string) - Required - The persona ID

### Request Example
coval personas get per_abc123
```

--------------------------------

### Create a review project

Source: https://docs.coval.dev/guides/human-review-api

Use this command to initialize a review project by linking simulations, metrics, and assignees. The API automatically generates annotations for every combination of these resources.

```shellscript
curl -X POST https://api.coval.dev/v1/review-projects \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Q3 Evaluation",
    "simulation_ids": ["sim_123", "sim_456"],
    "metric_ids": ["met_789"],
    "assignee_emails": ["reviewer@example.com"]
  }'
```

--------------------------------

### GET /simulations/download-audio

Source: https://docs.coval.dev/cli/simulations

Download or get the audio URL for a simulation recording.

```APIDOC
## GET /simulations/download-audio

### Description
Download or get the audio URL for a simulation recording.
```

--------------------------------

### GET /v1/review-projects/{project_id}

Source: https://docs.coval.dev/cli/scheduled-runs

Get a single review project by ID.

```APIDOC
## GET /v1/review-projects/{project_id}

### Description
Get a single review project by ID.

### Method
GET

### Endpoint
/v1/review-projects/{project_id}

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The unique identifier of the review project.
```

--------------------------------

### Webhook Configuration Examples

Source: https://docs.coval.dev/concepts/agents/connections/outbound-voice

Examples of JSON-formatted configuration values for HTTP headers, phone number keys, and trigger payloads.

```json
{"Content-Type": "application/json", "Authorization": "Bearer token123"}
```

```json
"phone_number"
```

```json
"recipient_phone"
```

```json
"target_number"
```

```json
{"sequence_code": "ABC123", "campaign_id": "summer2024"}
```

--------------------------------

### Install Claude Desktop via Terminal

Source: https://docs.coval.dev/mcp/beginners-guide

Use these commands to install Claude Desktop and open its configuration file via the terminal. Ensure Claude Desktop is installed first.

```shellscript
# 1. Install Claude Desktop (if you haven't already)
# Download from anthropic.com/download and run the installer

# 2. Open the Claude Desktop config file
open ~/Library/Application\ Support/Claude/claude_desktop_config.json
```

--------------------------------

### Example Prompt: Verify Tool Usage with Trace Context

Source: https://docs.coval.dev/concepts/metrics/prompting

This prompt demonstrates how to instruct the LLM Judge to verify if a specific tool was used correctly based on the transcript and trace context. It outlines conditions for returning YES or NO.

```text
Given the transcript and trace context, did the assistant call the `lookup_account` function before providing account balance information?

Return YES if:
• The TRACE CONTEXT shows a tool call to `lookup_account` (or equivalent) occurring before the agent stated the balance
• The transcript confirms the agent provided balance details

Return NO if:
• The agent mentioned account balance information but no `lookup_account` tool call appears in the TRACE CONTEXT
• The tool call appears AFTER the agent has already stated the balance (out of order)
```

--------------------------------

### Run Coval Launch Commands

Source: https://docs.coval.dev/cli/runs

Examples of using the coval CLI to launch runs with various parameters.

```shellscript
# Basic run
coval runs launch \
 --agent-id ag_abc123 \
 --persona-id per_xyz789 \
 --test-set-id ts_123456

# Run with options
coval runs launch \
 --agent-id ag_abc123 \
 --persona-id per_xyz789 \
 --test-set-id ts_123456
```

--------------------------------

### Install OpenTelemetry Dependencies

Source: https://docs.coval.dev/concepts/simulations/traces/opentelemetry

Command to install the necessary Python packages for OpenTelemetry.

```shellscript
pip install
```

--------------------------------

### GET /v1/simulations/{simulation_id}/metrics/{metric_output_id}

Source: https://docs.coval.dev/cli/overview

Get a specific metric result.

```APIDOC
## GET /v1/simulations/{simulation_id}/metrics/{metric_output_id}

### Description
Get a specific metric result.

### Method
GET

### Endpoint
/v1/simulations/{simulation_id}/metrics/{metric_output_id}

### Parameters
#### Path Parameters
- **simulation_id** (string) - Required - The unique identifier of the simulation.
- **metric_output_id** (string) - Required - The unique identifier of the metric output.
```

--------------------------------

### Create a Run Template with Advanced Options

Source: https://docs.coval.dev/cli/run-templates

Create a run template including advanced configurations such as metrics, mutations, iteration count, and concurrency.

```bash
coval run-templates create \
  --name "Full Evaluation" \
  --agent-id ag_abc123 \
  --persona-id per_xyz789 \
  --test-set-id ts_123456 \
  --metric-ids "met_001,met_002,met_003"
```

--------------------------------

### Install OpenTelemetry Packages

Source: https://docs.coval.dev/concepts/simulations/traces/opentelemetry

Command to install the necessary OpenTelemetry SDK packages for Python.

```shellscript
pip install opentelemetry-sdk opentelemetry-exporter-otlp
```

--------------------------------

### List all annotations

Source: https://docs.coval.dev/cli/human-review

Use this command to list all review annotations. No setup or imports are required.

```shellscript
# List all annotations
coval review-annotations list
```

--------------------------------

### Comparison of Installation Commands

Source: https://docs.coval.dev/agents/skills

Installation commands for Skills, MCP Server, and CLI tools.

```shellscript
npx skills add coval-ai/coval-external-skills
```

```shellscript
npx coval-mcp
```

```shellscript
brew install coval-ai/tap/coval
```

--------------------------------

### Create a collaborative review project

Source: https://docs.coval.dev/cli/human-review

Use this command to initialize a new review project with specified assignees, simulations, and metrics.

```shellscript
# Create a collaborative review project
coval review-projects create \
 --name "Q1 Voice Agent Review" \
 --assignees "alice@company.com,bob@company.com" \
 --simulation-ids "sim-output-001,sim-output-002" \
 --metric-ids "metric-accuracy,metric-latency" \
 --type collaborative

# C
```