### Manual Development Setup and Testing

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Commands for cloning the repository, installing dependencies with uv, running unit and integration tests, and linting the code manually.

```bash
# Clone and install
git clone https://github.com/voicetestdev/voicetest
cd voicetest
uv sync
```

```bash
# Run unit tests
uv run pytest tests/unit
```

```bash
# Run integration tests (requires Ollama with qwen2.5:0.5b)
uv run pytest tests/integration
```

```bash
# Lint
uv run ruff check voicetest/ tests/
```

--------------------------------

### Start Frontend Development Server with Bun

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Navigate to the web directory, install dependencies using Bun, and start the Vite development server with hot reloading. This command is intended for manual frontend development.

```bash
# Terminal 1 - Frontend dev server with hot reload
cd web
mise exec -- bun install
mise exec -- bun run dev   # http://localhost:5173
```

--------------------------------

### Install Prerequisites for Demo Recordings

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/demos/README.md

Installs necessary tools for generating CLI and Web UI demos. Ensure you are in the correct directory before running these commands.

```bash
# CLI demo (VHS by Charmbracelet)
brew install vhs

# Web UI demo (Playwright + ffmpeg)
cd ../../web
mise exec -- bun add -D @playwright/test
mise exec -- bunx playwright install chromium
brew install ffmpeg
```

--------------------------------

### Start Voicetest Development Environment with Docker Compose

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Clone the repository and start all development services using Docker Compose. This is the recommended method for a full development environment.

```bash
git clone https://github.com/voicetestdev/voicetest
cd voicetest
docker compose -f docker-compose.dev.yml up
```

--------------------------------

### Install Voicetest CLI

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Install the Voicetest CLI tool using uv or pip. Add it to your project or install it globally.

```bash
# Install as a CLI tool
uv tool install voicetest

# Or add to a project
uv add voicetest

# Or with pip
pip install voicetest
```

--------------------------------

### GitHub Actions Workflow for Voice Agent Tests

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Example GitHub Actions workflow to run voice agent tests on push events. It checks out code, sets up Python with uv, installs voicetest, and runs tests with environment variables for API keys.

```yaml
name: Voice Agent Tests
on:
  push:
    paths:
      - "agents/**"  # Trigger on agent config or test changes

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv tool install voicetest
      - run: voicetest run --agent agents/receptionist.json --tests agents/tests.json --all
        env:
          GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
```

--------------------------------

### Install voicetest with uv

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Install voicetest using the uv tool. This command adds the tool to your environment.

```bash
uv tool install voicetest
```

--------------------------------

### Install LiveKit CLI on Linux

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Install the LiveKit CLI tool on Linux by downloading and executing the official installation script. This tool is required for LiveKit integration tests.

```bash
curl -sSL https://get.livekit.io/cli | bash
```

--------------------------------

### CLI: Start REST API Server

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Commands to launch the VoiceTest REST API server with various configurations.

```APIDOC
## CLI: Start REST API Server

Launch the web server with REST API and web UI.

### Start server with default settings
```bash
voicetest serve
```

### Start with custom host and port
```bash
voicetest serve --host 0.0.0.0 --port 9000
```

### Start with hot reload for development
```bash
voicetest serve --reload
```

### Start with linked agents and tests
```bash
voicetest serve --agent agent.json --tests tests.json
```
```

--------------------------------

### Start voicetest REST API server with Web UI

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Launch the REST API server along with its accompanying Web UI for voicetest.

```bash
voicetest serve
```

--------------------------------

### Install voicetest with pip

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Install voicetest using pip. This is a standard Python package installation.

```bash
pip install voicetest
```

--------------------------------

### Start voicetest infrastructure services

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Bring up essential infrastructure services like LiveKit, Whisper, and Kokoro, along with the backend for live calls.

```bash
voicetest up
```

--------------------------------

### Run voicetest demo with Groq API key

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Set your Groq API key and run the voicetest demo. This is useful for quick testing without full setup.

```bash
# Set up an API key (free, no credit card at https://console.groq.com)
export GROQ_API_KEY=gsk_...
```

```bash
# Load demo and start interactive shell
voicetest demo
```

```bash
# Or load demo and start web UI
voicetest demo --serve
```

--------------------------------

### Launch Voicetest Demo

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Launch the Voicetest demo with a sample healthcare agent. Requires a Groq API key. Can start an interactive shell or the web UI.

```bash
# Set up an API key (free at https://console.groq.com)
export GROQ_API_KEY=gsk_...

# Load demo and start interactive shell
voicetest demo

# Or load demo and start web UI
voicetest demo --serve
```

--------------------------------

### Run LiveKit Agent Session

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/RESEARCH.md

Demonstrates how to initiate and run an agent session using the LiveKit Agents SDK. It includes starting the agent, sending user input, and asserting expected assistant responses and intents.

```python
async with AgentSession(llm=llm) as sess:
    await sess.start(MyAgent())
    result = await sess.run(user_input="Hello")
    result.expect.next_event().is_message(role="assistant")
    await result.expect.next_event().judge(llm, intent="should ask what user wants")
```

--------------------------------

### Tool Mocks Example

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/references/test-format.md

Shows how to provide mock responses for external tools called by an agent. This is useful for testing tool-calling functionality in isolation.

```json
{
  "tool_mocks": [
    {
      "name": "lookup_account",
      "response": { "name": "Jane Smith", "balance": 150.00 }
    }
  ]
}
```

--------------------------------

### Install LiveKit CLI on macOS

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Install the LiveKit CLI tool using Homebrew on macOS. This tool is required for LiveKit integration tests.

```bash
brew install livekit-cli
```

--------------------------------

### Manage Voicetest Infrastructure

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Use CLI commands to manage the Docker infrastructure for live voice calls, including LiveKit, Whisper STT, and Kokoro TTS. Start the infrastructure with or without the backend server, or stop it.

```bash
voicetest up
```

```bash
voicetest up --detach
```

```bash
voicetest down
```

--------------------------------

### Convert Retell to LiveKit Python

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-convert.md

Example of converting a Retell agent to the LiveKit Python format.

```bash
voicetest export -a retell.json -f livekit
```

--------------------------------

### Configure Punq Dependency Injection Container

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/RESEARCH.md

Setup the Punq container to manage application dependencies with singleton scope for the registry.

```python
import punq

def create_container() -> punq.Container:
    container = punq.Container()
    container.register(ImporterRegistry, factory=_create_registry, scope=punq.Scope.singleton)
    container.register(AgentRepository, factory=lambda: AgentRepository(get_connection()))
    # ... other registrations
    return container
```

--------------------------------

### Convert Any Format to Agent Graph JSON

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-convert.md

Example of converting an agent from any format to the Agent Graph JSON format.

```bash
voicetest export -a agent.json -f agentgraph
```

--------------------------------

### Install Voicetest Claude Code Plugin

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Install the voicetest plugin for Claude Code from the marketplace or via pip. This enables agent-assisted voice testing features.

```bash
/plugin marketplace add voicetestdev/voicetest
/plugin install voicetest@voicetest-plugins
```

```bash
cd your-project
voicetest init-claude
```

--------------------------------

### Convert Retell to VAPI

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-convert.md

Example of converting a Retell agent to the VAPI format.

```bash
voicetest export -a retell.json -f vapi
```

--------------------------------

### Dynamic Variables Example

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/references/test-format.md

Demonstrates how to define dynamic variables within a test case. These variables can be injected into agent prompts using `{{variable_name}}` syntax.

```json
{
  "dynamic_variables": {
    "customer_name": "Jane Smith",
    "account_number": "12345",
    "appointment_date": "2024-03-15"
  }
}
```

--------------------------------

### OpenClaw Configuration Example

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/MOLTBOT.md

This JSON snippet shows a sample OpenClaw configuration, specifically defining an agent named 'voicetest-runner' with 'coding' profile and allowing 'shell' and 'fs' tools.

```json
{
  "agents": {
    "list": [
      {
        "id": "voicetest-runner",
        "tools": {
          "profile": "coding",
          "allow": ["shell", "fs"]
        }
      }
    ]
  }
}

```

--------------------------------

### Convert Retell to Mermaid Diagram

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-convert.md

Example of converting a Retell agent to a Mermaid diagram format.

```bash
voicetest export -a retell.json -f mermaid
```

--------------------------------

### Start LiveKit Server Manually

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Run the LiveKit server in development mode using a Docker container. This is necessary for manual frontend development when live voice calls are needed.

```bash
# Terminal 3 - LiveKit server (for live voice calls)
docker run --rm -p 7880:7880 -p 7881:7881 -p 7882:7882/udp livekit/livekit-server --dev
```

--------------------------------

### Example Test JSON Format

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/RESEARCH.md

JSON structure for defining tests that can be stored in external files.

```json
[
  {
    "name": "Greeting test",
    "user_prompt": "Hello, I need help",
    "metrics": ["Agent greeted user warmly"],
    "type": "llm"
  }
]
```

--------------------------------

### Start Voicetest REST API Server

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Launch the Voicetest web server with REST API and web UI. Default settings can be used, or custom host and port can be specified. Hot reload is available for development.

```bash
voicetest serve
```

```bash
voicetest serve --host 0.0.0.0 --port 9000
```

```bash
voicetest serve --reload
```

```bash
voicetest serve --agent agent.json --tests tests.json
```

--------------------------------

### Start Backend API Server Manually

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Run the FastAPI backend server with hot reloading enabled. This command is used for manual backend development.

```bash
# Terminal 2 - Backend API
uv run voicetest serve --reload   # http://localhost:8000
```

--------------------------------

### Integrate voicetest with GitHub Actions

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Automate voice agent testing by installing the tool and running test suites in a CI/CD pipeline.

```yaml
name: Voice Agent Tests
on:
  push:
    paths: ["agents/**"]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5

      - name: Install voicetest
        run: uv tool install voicetest

      - name: Run smoke test
        run: voicetest smoke-test
        env:
          GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}

      - name: Run full test suite
        run: voicetest run --agent agents/receptionist.json --tests agents/tests.json --all --output results.json
        env:
          GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}

      - name: Upload results
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: results.json
```

--------------------------------

### Simulation Test Case Example

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/references/test-format.md

Defines a simulation test where an LLM acts as the user. Use for evaluating agent responses with natural language metrics.

```json
{
  "name": "Customer billing inquiry",
  "user_prompt": "When asked for name, say Jane Smith. Say you're confused about a charge on your bill.",
  "metrics": [
    "Agent greeted the customer and addressed the billing concern.",
    "Agent was helpful and professional."
  ],
  "dynamic_variables": {
    "account_number": "12345"
  },
  "tool_mocks": [],
  "type": "simulation",
  "llm_model": "gpt-4o-mini"
}
```

--------------------------------

### Pattern Matching Test Case Example

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/references/test-format.md

A rule-based test case that uses regex patterns to validate specific formats in the agent's response, such as confirmation numbers.

```json
{
  "name": "Confirmation number format",
  "user_prompt": "Book an appointment for tomorrow at 3pm.",
  "includes": ["confirmed", "appointment"],
  "patterns": ["[A-Z]{2,4}-[0-9]{4,8}"],
  "type": "unit"
}
```

--------------------------------

### Build Frontend for Production

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Build the frontend application for production deployment using Bun. This command should be run from the 'web' directory.

```bash
# Build for production
cd web && mise exec -- bun run build
```

--------------------------------

### List Available Platforms

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Display a list of supported platforms and their current configuration status.

```bash
voicetest platforms
```

--------------------------------

### Stop voicetest infrastructure services

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Shut down the infrastructure services that were started with `voicetest up`.

```bash
voicetest down
```

--------------------------------

### Configure Platform Credentials

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-export.md

Set up authentication credentials for a specific platform. You will need to provide your API key.

```bash
voicetest platform configure <platform> --api-key <key>
```

--------------------------------

### List voicetest capabilities and configuration

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-info.md

Use these commands to inspect supported formats, platform integrations, and current settings.

```bash
voicetest --json importers
```

```bash
voicetest --json exporters
```

```bash
voicetest --json platforms
```

```bash
voicetest --json settings
```

--------------------------------

### Generate Demo Recordings

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/demos/README.md

Commands to generate various demo recordings. Navigate to the 'docs/demos' directory before executing these commands. Options include generating all demos, specific types (CLI, Web UI, DRY), or cleaning up generated files.

```bash
cd docs/demos

# Generate all (cli + web + dry, both themes)
make all

# CLI demo only
make cli

# Web UI demos (light + dark)
make web

# DRY analysis demos (light + dark)
make dry

# Clean generated files
make clean
```

--------------------------------

### Get Agent Details

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Retrieve detailed information about a specific agent using its unique ID.

```bash
voicetest agent get <agent-id>
```

--------------------------------

### Set Configuration Value

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Modify a specific configuration setting. For example, to change the default agent model.

```bash
voicetest settings --set models.agent=openai/gpt-4o
```

--------------------------------

### Show Available Importers and Exporters

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-convert.md

Run these commands to list the supported agent formats for importing and exporting.

```bash
voicetest --json importers
```

```bash
voicetest --json exporters
```

--------------------------------

### Launch voicetest TUI

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Start the Text User Interface (TUI) for voicetest. This provides an interactive terminal-based interface.

```bash
voicetest tui --agent agent.json --tests tests.json
```

--------------------------------

### Platform Integration Commands

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Configure platform credentials, list agents on remote platforms, import agents, and push local agents to remote platforms. Lists available platforms.

```bash
# Configure platform credentials
voicetest platform configure retell --api-key sk-xxx

# List agents on a remote platform
voicetest platform list-agents retell

# Import an agent from Retell
voicetest platform import retell agent_123abc -o imported.json

# Push a local agent to VAPI
voicetest platform push vapi -a agent.json --agent-name "My Agent"

# List available platforms with configuration status
voicetest platforms
```

--------------------------------

### Configure Models and Run Options in Settings File

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Define LLM models, run settings like max_turns, and audio configurations in the .voicetest/settings.toml file.

```toml
[models]
agent = "groq/llama-3.1-8b-instant"
simulator = "groq/llama-3.1-8b-instant"
judge = "groq/llama-3.1-8b-instant"

[run]
max_turns = 20
audio_eval = false
streaming = false

[audio]
tts_url = "http://localhost:8002/v1"
stt_url = "http://localhost:8001/v1"

[cache]
cache_backend = "disk"
```

--------------------------------

### Define DSPy Signatures

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Use type annotations and descriptions to guide LLM output formatting and semantic extraction.

```python
class MySignature(dspy.Signature):
    """Docstring becomes the prompt context."""

    input_text: str = dspy.InputField(desc="What this input contains")
    count: int = dspy.InputField(desc="Numeric input")

    result: str = dspy.OutputField(desc="What the LLM should produce")
    score: float = dspy.OutputField(desc="Numeric score from 0.0 to 1.0")
    items: list[str] = dspy.OutputField(desc="List of extracted items")
    valid: bool = dspy.OutputField(desc="True/False judgment")
```

--------------------------------

### Initialize and Run ConversationEngine in Python

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Use the ConversationEngine to process turns against an agent graph. Requires an initialized agent service and RunOptions configuration.

```python
import asyncio
from voicetest.engine.conversation import ConversationEngine
from voicetest.services import get_agent_service
from voicetest.models.test_case import RunOptions

async def main():
    agent_svc = get_agent_service()
    graph = await agent_svc.import_agent("agent.json")

    # Initialize engine with dynamic variables
    engine = ConversationEngine(
        graph=graph,
        model="openai/gpt-4o-mini",
        options=RunOptions(max_turns=50),
        dynamic_variables={"customer_name": "Jane", "account_id": "12345"}
    )

    # Process conversation turns
    while not engine.end_call_invoked:
        user_input = input("You: ")
        if user_input.lower() in ("quit", "exit"):
            break

        await engine.add_user_message(user_input)
        result = await engine.advance()

        print(f"Agent: {result.response}")
        if result.transitioned_to:
            print(f"  [Transitioned to: {result.transitioned_to}]")

    print(f"\nConversation ended. Nodes visited: {engine.nodes_visited}")

asyncio.run(main())
```

--------------------------------

### Integrate with Platforms using voicetest CLI

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/SKILL.md

Commands for listing supported platforms, configuring platform credentials, listing remote agents, and importing/pushing agents to specific platforms.

```bash
voicetest platforms                                       # List platforms
```

```bash
voicetest platform configure retell --api-key <key>       # Set credentials
```

```bash
voicetest platform list-agents retell                     # List remote agents
```

```bash
voicetest platform import retell <agent-id> -o agent.json # Import agent
```

```bash
voicetest platform push retell -a agent.json              # Push to platform
```

--------------------------------

### Get Claude Plugin Directory Path

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Retrieve the directory path for the Claude Code plugin to enable manual plugin loading.

```bash
claude --plugin-dir $(voicetest claude-plugin-path)
```

--------------------------------

### Configure and Manage Platform Agents via CLI

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/references/platforms.md

Use these commands to manage credentials, list, import, and push agents to supported platforms.

```bash
# Configure credentials
voicetest platform configure retell --api-key sk-xxx

# List agents on the platform
voicetest platform list-agents retell

# Import an agent from the platform
voicetest platform import retell agent_abc123 -o imported.json

# Push a local agent to the platform
voicetest platform push retell -a agent.json --agent-name "My Agent"
```

--------------------------------

### Run Frontend Tests with Vitest

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Execute the frontend unit tests using Vitest. Ensure you are in the 'web' directory before running this command.

```bash
# Run frontend tests
cd web && npx vitest run
```

--------------------------------

### Run History API

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Commands for managing and viewing test run history, including listing runs, getting details, and deleting runs.

```APIDOC
## Run History

### List Runs

**Description**: Lists past test runs for a specific agent.

**Command**: `voicetest runs list <agent-id>`

### Get Run Details

**Description**: Retrieves detailed results for a specific test run.

**Command**: `voicetest runs get <run-id>`

### Delete Run

**Description**: Deletes a specific test run from the history.

**Command**: `voicetest runs delete <run-id>`
```

--------------------------------

### Show Current Settings

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Display the current configuration settings for Voicetest.

```bash
voicetest settings
```

--------------------------------

### Import Agent from Platform

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Import an agent from a remote platform into your local Voicetest environment. Specify the platform and the agent ID, and an output file name.

```bash
voicetest platform import retell <agent-id> -o imported.json
```

--------------------------------

### Add voicetest to a project with uv

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Add voicetest as a dependency to your project using uv. You can then run it using `uv run voicetest`.

```bash
uv add voicetest
```

--------------------------------

### JSON Output for Snippet Analyze

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Analyze agent snippets for repeated text and get the results in JSON format. Standard error will contain progress updates.

```bash
voicetest --json snippet analyze --agent agent.json
```

--------------------------------

### Rule-Based Test Case Example

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/references/test-format.md

Defines a rule-based test for validating agent responses using inclusion and exclusion criteria. Suitable for checking specific keywords or phrases.

```json
{
  "name": "Greeting includes company name",
  "user_prompt": "Say hello and ask about business hours.",
  "includes": ["Acme", "welcome", "help"],
  "excludes": ["goodbye", "transfer"],
  "patterns": [],
  "type": "unit"
}
```

--------------------------------

### List available importers

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Use this command to see all the available data importers supported by voicetest.

```bash
voicetest importers
```

--------------------------------

### CLI: Infrastructure Management

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Commands for managing Docker infrastructure for live voice calls.

```APIDOC
## CLI: Infrastructure Management

Start Docker infrastructure for live voice calls with LiveKit, Whisper STT, and Kokoro TTS.

### Start infrastructure + backend server
```bash
voicetest up
```

### Start infrastructure only (detached)
```bash
voicetest up --detach
```

### Stop infrastructure
```bash
voicetest down
```
```

--------------------------------

### List Available Export Formats

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-export.md

Run this command to see all supported export formats for voice agents. Common formats include mermaid, livekit, retell-llm, vapi, bland, and agentgraph.

```bash
voicetest --json exporters
```

--------------------------------

### Parse CLI Output with --json Flag and jq

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/SKILL.md

Utilize the --json flag with voicetest commands to get machine-parseable output, which can then be processed by tools like jq for specific data extraction.

```bash
voicetest --json agent list | jq '.[].name'
```

```bash
voicetest --json run -a agent.json -t tests.json --all | jq '.results[] | {name: .test_name, status}'
```

--------------------------------

### Retell Simulation Test Case Format

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/RESEARCH.md

Defines the structure for a Retell simulation test case using markdown. It includes sections for identity, goal, and personality to guide the agent's behavior during simulation.

```markdown
## Identity
Your name is Mike.
Your date of birth is June 10, 1999.
Your order number is 7891273.

## Goal
Your primary objective is to return the package you received and get a refund.

## Personality
You are a patient customer. However, if the conversation becomes too long
or complicated, you will show signs of impatience.
```

--------------------------------

### List Supported Platforms

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-export.md

View the list of supported platforms for voice agent operations. This command helps identify which platforms can be interacted with.

```bash
voicetest --json platforms
```

--------------------------------

### Configure DSPy with BAMLAdapter

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/RESEARCH.md

Sets up DSPy to use BAMLAdapter for efficient structured outputs and prompt optimization. This configuration is typically done globally.

```python
from dspy.adapters.baml_adapter import BAMLAdapter

llm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=llm, adapter=BAMLAdapter())
```

```python
import dspy
from dspy.adapters.baml_adapter import BAMLAdapter
dspy.configure(adapter=BAMLAdapter())
```

--------------------------------

### Manage Agents with voicetest CLI

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/SKILL.md

Commands for listing, retrieving, creating, updating, deleting, and visualizing agent structures stored in the database. Agent creation requires a JSON file.

```bash
voicetest agent list                           # List agents in database
```

```bash
voicetest agent get <id>                       # Get agent details
```

```bash
voicetest agent create -a agent.json           # Create from file
```

```bash
voicetest agent update <id> --name "New Name"  # Update properties
```

```bash
voicetest agent delete <id>                    # Delete agent
```

```bash
voicetest agent graph <id>                     # Show graph structure
```

--------------------------------

### List agents in database

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-info.md

Displays a list of agents currently loaded in the database.

```bash
voicetest --json agent list
```

--------------------------------

### Create Agent

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Create a new agent from a JSON definition file. Provide a name for the agent.

```bash
voicetest agent create -a agent.json --name "My Agent"
```

--------------------------------

### Manage Infrastructure Services

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Commands to manage Docker-based infrastructure services required for live voice calls.

```bash
# Start infrastructure + backend server
voicetest up

# Or start infrastructure only (e.g., to run the backend separately)
voicetest up --detach

# Stop infrastructure when done
voicetest down
```

--------------------------------

### Configure LLM Models with LiteLLM Format

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Configure different LLM models for agent, simulator, and judge roles using LiteLLM format. Set maximum conversation turns.

```python
from voicetest.models.test_case import RunOptions

options = RunOptions(
    agent_model="openai/gpt-4o-mini",
    simulator_model="gemini/gemini-1.5-flash",
    judge_model="anthropic/claude-3-haiku-20240307",
    max_turns=20,
)
```

--------------------------------

### Run Pre-commit Checks

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Execute all configured pre-commit hooks to ensure code quality standards are met before committing.

```bash
uv run pre-commit run --all-files
```

--------------------------------

### Run All Tests with voicetest CLI

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/SKILL.md

Execute all defined test cases for an agent. Use --all to run all tests, or --test "Name" to run a specific test by name. The --json flag provides machine-parseable output.

```bash
voicetest run -a agent.json -t tests.json --all          # Run all tests
```

```bash
voicetest run -a agent.json -t tests.json --test "Name"  # Run specific test
```

```bash
voicetest --json run -a agent.json -t tests.json --all   # JSON output for parsing
```

--------------------------------

### Push Local Agent to Platform

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Upload a local agent definition to a specified remote platform.

```bash
voicetest platform push retell -a agent.json
```

--------------------------------

### Create Test Case from JSON

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Create a new test case for an agent using a JSON file as input.

```bash
voicetest test create <agent-id> -f test.json
```

--------------------------------

### Manage Platform Operations via REST API

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Interact with external platforms to import or export agent configurations.

```bash
# List available platforms
curl http://localhost:8000/api/platforms

# List agents on a platform
curl http://localhost:8000/api/platforms/retell/agents

# Import agent from platform
curl -X POST http://localhost:8000/api/platforms/retell/import \
  -H "Content-Type: application/json" \
  -d '{"agent_id": "agent_abc123"}'

# Push agent to platform
curl -X POST http://localhost:8000/api/platforms/vapi/export \
  -H "Content-Type: application/json" \
  -d '{
    "graph": {...},
    "name": "My VAPI Agent"
  }'
```

--------------------------------

### Configure voicetest settings

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Define model, run, audio, cache, and environment settings in the .voicetest/settings.toml file.

```toml
[models]
agent = "groq/llama-3.1-8b-instant"
simulator = "groq/llama-3.1-8b-instant"
judge = "anthropic/claude-3-haiku-20240307"

[run]
max_turns = 50
verbose = false
flow_judge = false
streaming = false
audio_eval = false
pattern_engine = "fnmatch"

[audio]
tts_url = "http://localhost:8002/v1"
stt_url = "http://localhost:8001/v1"

[cache]
cache_backend = "disk"
# For shared caching:
# cache_backend = "s3"
# s3_bucket = "my-bucket"
# s3_prefix = "dspy-cache/"

[env]
GROQ_API_KEY = "gsk_..."
OPENAI_API_KEY = "sk-..."
```

--------------------------------

### Manage Settings via REST API

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Retrieve and modify global configuration settings for models and execution parameters.

```bash
# Get current settings
curl http://localhost:8000/api/settings

# Update settings
curl -X PUT http://localhost:8000/api/settings \
  -H "Content-Type: application/json" \
  -d '{
    "models": {
      "agent": "openai/gpt-4o-mini",
      "simulator": "groq/llama-3.1-8b-instant",
      "judge": "anthropic/claude-3-haiku-20240307"
    },
    "run": {
      "max_turns": 30,
      "streaming": true,
      "audio_eval": false
    }
  }'
```

--------------------------------

### Convert Agent Formats

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

CLI commands to convert agent configurations between different platform formats using the unified AgentGraph representation.

```bash
# Convert Retell Conversation Flow to Retell LLM format
voicetest export --agent retell-cf-agent.json --format retell-llm > retell-llm-agent.json

# Convert VAPI assistant to Retell LLM format
voicetest export --agent vapi-assistant.json --format retell-llm > retell-agent.json

# Convert Retell LLM to VAPI format
voicetest export --agent retell-llm-agent.json --format vapi-assistant > vapi-agent.json
```

--------------------------------

### Settings and Platforms API

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Commands for managing Voicetest settings, viewing platform configurations, and interacting with remote platforms.

```APIDOC
## Settings and Platforms

### Show Settings

**Description**: Displays the current Voicetest configuration settings.

**Command**: `voicetest settings`

### Set Setting

**Description**: Sets a specific configuration value.

**Command**: `voicetest settings --set <setting_path>=<value>`

**Example**: `voicetest settings --set models.agent=openai/gpt-4o`

### List Platforms

**Description**: Lists available platforms and their configuration status.

**Command**: `voicetest platforms`

### Configure Platform

**Description**: Configures credentials for a specific platform.

**Command**: `voicetest platform configure <platform_name> --api-key <api_key>`

**Example**: `voicetest platform configure retell --api-key sk-xxx`

### List Remote Agents

**Description**: Lists agents available on a remote platform.

**Command**: `voicetest platform list-agents <platform_name>`

### Import Agent from Platform

**Description**: Imports an agent from a remote platform.

**Command**: `voicetest platform import <platform_name> <agent-id> -o <output_file.json>`

### Push Agent to Platform

**Description**: Pushes a local agent definition to a remote platform.

**Command**: `voicetest platform push <platform_name> -a <agent_file>`
```

--------------------------------

### Launch interactive voicetest shell

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Launch the interactive shell for voicetest. Use this for direct command execution within the tool.

```bash
# Launch interactive shell (default)
uv run voicetest
```

```bash
# In the shell:
> agent tests/fixtures/retell/sample_config.json
> tests tests/fixtures/retell/sample_tests.json
> set agent_model ollama_chat/qwen2.5:0.5b
> run
```

--------------------------------

### Push Agent to Platform

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-export.md

Upload or update a voice agent definition to a specified platform. Requires the agent path and the target platform.

```bash
voicetest --json platform push <platform> -a <agent-path>
```

--------------------------------

### Resolve Dependencies from Container

Source: https://github.com/voicetestdev/voicetest/blob/main/docs/RESEARCH.md

Retrieve registered services from the global container instance.

```python
from voicetest.container import get_container
repo = get_container().resolve(AgentRepository)
```

--------------------------------

### Resolve DI Container Instances

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Import helper functions to access registered singletons and repositories from the Punq container.

```python
from voicetest.container import get_session, get_importer_registry
```

--------------------------------

### Configure voicetest Settings

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/SKILL.md

Commands to view current settings, modify specific settings (e.g., LLM models), and reset settings to their default values.

```bash
voicetest settings                                     # Show settings
```

```bash
voicetest settings --set models.agent=openai/gpt-4o    # Set a value
```

```bash
voicetest settings --defaults                          # Show defaults
```

--------------------------------

### Configure LLM Models via CLI

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/SKILL.md

Set the LLM models used for different roles (agent, simulator, judge) directly from the command line. These settings can also be configured in .voicetest.toml.

```bash
voicetest settings --set models.agent=openai/gpt-4o
```

```bash
voicetest settings --set models.judge=anthropic/claude-sonnet-4-20250514
```

--------------------------------

### Import and Test Agent via Python SDK

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Use the Python library to programmatically import agents and execute test cases.

```python
import asyncio
from voicetest.services import get_agent_service, get_test_execution_service
from voicetest.models.test_case import TestCase, RunOptions

async def main():
    agent_svc = get_agent_service()
    exec_svc = get_test_execution_service()

    # Import agent from file
    graph = await agent_svc.import_agent("agent.json")
    print(f"Imported: {graph.source_type} with {len(graph.nodes)} nodes")

    # Define test case
    test = TestCase(
        name="Greeting test",
        user_prompt="Say hello and ask about store hours",
        metrics=["Agent greeted warmly", "Agent provided hours info"],
        dynamic_variables={"store_name": "Acme Store"}
    )

    # Run test
    options = RunOptions(
        agent_model="groq/llama-3.1-8b-instant",
        simulator_model="groq/llama-3.1-8b-instant",
        judge_model="groq/llama-3.1-8b-instant",
        max_turns=20
    )

    result = await exec_svc.run_test(graph, test, options)

    print(f"Status: {result.status}")
    print(f"Turns: {result.turn_count}")
    for metric in result.metric_results:
        print(f"  {metric.metric}: {'PASS' if metric.passed else 'FAIL'} ({metric.score:.2f})")

asyncio.run(main())
```

--------------------------------

### List Defined Snippets

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

List all snippets that have been defined for a specific agent.

```bash
voicetest snippet list --agent agent.json
```

--------------------------------

### Run tests against an agent definition

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Execute tests against a specified agent configuration. The `--all` flag ensures all tests are run.

```bash
voicetest run --agent agent.json --tests tests.json --all
```

--------------------------------

### Run All Voice Agent Tests

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-run.md

Execute all defined tests for a voice agent. Ensure the agent and test file paths are correctly specified.

```bash
voicetest --json run -a <agent-path> -t <tests-path> --all
```

--------------------------------

### List Test Cases for Agent

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Retrieve a list of all test cases associated with a specific agent.

```bash
voicetest test list <agent-id>
```

--------------------------------

### View Run Details

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Retrieve detailed results and information for a specific test run using its ID.

```bash
voicetest runs get <run-id>
```

--------------------------------

### Manage Test Cases via REST API

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Create, update, and link test cases to agents for evaluation.

```bash
# List tests for an agent
curl http://localhost:8000/api/agents/{agent_id}/tests

# Create a test case
curl -X POST http://localhost:8000/api/agents/{agent_id}/tests \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Billing inquiry",
    "user_prompt": "Ask about a charge on your bill",
    "metrics": ["Agent addressed the billing concern"],
    "dynamic_variables": {"account_id": "12345"},
    "type": "llm"
  }'

# Update a test case
curl -X PUT http://localhost:8000/api/tests/{test_id} \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Updated test",
    "user_prompt": "New prompt",
    "metrics": ["New metric"]
  }'

# Link external test file to agent
curl -X POST http://localhost:8000/api/agents/{agent_id}/tests-paths \
  -H "Content-Type: application/json" \
  -d '{"path": "/path/to/tests.json"}'
```

--------------------------------

### Configure Local LLM Models with Ollama

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Use Ollama for local LLM execution by specifying the Ollama model names in RunOptions.

```python
options = RunOptions(
    agent_model="ollama_chat/qwen2.5:0.5b",
    simulator_model="ollama_chat/qwen2.5:0.5b",
    judge_model="ollama_chat/qwen2.5:0.5b",
)
```

--------------------------------

### Set LLM Models via Shell

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Configure LLM models for agent and simulator roles directly from the shell.

```bash
> set agent_model gemini/gemini-1.5-flash
> set simulator_model ollama_chat/qwen2.5:0.5b
```

--------------------------------

### JSON Output for Run Command

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Execute the 'run' command with specified agent and tests, and receive the output in JSON format. Progress details are directed to stderr.

```bash
voicetest --json run -a agent.json -t tests.json --all
```

--------------------------------

### Import Agent from Platform

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-export.md

Import a voice agent from a specified platform using its unique agent ID. This is useful for migrating agents or backing them up.

```bash
voicetest --json platform import <platform> <agent-id>
```

--------------------------------

### Create or Update Snippet

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Create a new snippet or update an existing one for an agent. Provide the snippet name and its text content.

```bash
voicetest snippet set --agent agent.json greeting "Hello, how can I help?"
```

--------------------------------

### Export and Convert Agent Definitions with voicetest CLI

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/skills/voicetest/SKILL.md

Commands to export agent definitions into various formats like Mermaid diagrams, LiveKit Python agents, or VAPI format. Also lists available importers and exporters.

```bash
voicetest export -a agent.json -f mermaid      # Mermaid diagram
```

```bash
voicetest export -a agent.json -f livekit      # LiveKit Python agent
```

```bash
voicetest export -a agent.json -f vapi         # VAPI format
```

```bash
voicetest importers                            # List importers
```

```bash
voicetest exporters                            # List export formats
```

--------------------------------

### Export agent to Mermaid diagram

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Export the agent configuration to a Mermaid diagram format for visualization.

```bash
voicetest export --agent agent.json --format mermaid         # Diagram
```

--------------------------------

### Analyze Agent for Repeated Text

Source: https://github.com/voicetestdev/voicetest/blob/main/claude-plugin/commands/voicetest-convert.md

After conversion, use this command to analyze the agent for repeated text snippets. Replace <agent-path> with the path to your converted agent file.

```bash
voicetest --json snippet analyze --agent <agent-path>
```

--------------------------------

### Testing Commands

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

Commands for running tests, chatting with agents, evaluating transcripts, diagnosing failures, and decomposing agents.

```APIDOC
## Testing Commands

### Run Tests

**Description**: Executes tests against an agent definition.

**Command**: `voicetest run --agent <agent_file> --tests <tests_file> [--all]

### Chat with Agent

**Description**: Allows interactive chat with an agent.

**Command**: `voicetest chat -a <agent_file> --model <model_name> [--var <key>=<value>]

### Evaluate Transcript

**Description**: Evaluates a transcript against specified metrics without simulation.

**Command**: `voicetest evaluate -t <transcript_file> -m "<metric_description>"

### Diagnose Failures

**Description**: Diagnoses test failures and suggests fixes. Can optionally auto-fix and save changes.

**Command**: `voicetest diagnose -a <agent_file> -t <tests_file> [--auto-fix] [--save <output_file>]

### Decompose Agent

**Description**: Decomposes an agent into sub-agents.

**Command**: `voicetest decompose -a <agent_file> -o <output_directory> [--num-agents <N>] [--model <model_id>]
```

--------------------------------

### List Agents

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

List all agents currently stored in the Voicetest database.

```bash
voicetest agent list
```

--------------------------------

### Configure Rule-Based Test Cases

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Define deterministic tests using inclusion, exclusion, and regex pattern matching. Useful for validating specific output requirements or PII constraints.

```json
[
  {
    "name": "Greeting includes company name",
    "user_prompt": "Say hello",
    "includes": ["Acme", "welcome", "help"],
    "excludes": ["goodbye", "transfer"],
    "type": "rule"
  },
  {
    "name": "Confirmation number format",
    "user_prompt": "Book an appointment for tomorrow at 3pm",
    "includes": ["confirmed", "appointment"],
    "excludes": ["error", "unable"],
    "patterns": ["[A-Z]{2,4}-[0-9]{4,8}"],
    "type": "rule"
  },
  {
    "name": "No PII in response",
    "user_prompt": "My SSN is 123-45-6789, verify my identity",
    "includes": ["verified"],
    "excludes": ["123-45-6789", "123456789"],
    "type": "rule"
  }
]
```

--------------------------------

### List Agents on Remote Platform

Source: https://github.com/voicetestdev/voicetest/blob/main/README.md

View a list of agents that are currently deployed on a specified remote platform.

```bash
voicetest platform list-agents retell
```

--------------------------------

### Import Agent Configuration via REST API

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Import an agent configuration from any supported format using a POST request to the `/api/agents/import` endpoint. The request body should contain the agent configuration and source type.

```bash
curl -X POST http://localhost:8000/api/agents/import \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "nodes": {
        "greeting": {
          "id": "greeting",
          "state_prompt": "Greet the user warmly and ask how you can help.",
          "transitions": [
            {
              "target_node_id": "handle_query",
              "condition": {"type": "llm_prompt", "value": "User states their question"}
            }
          ]
        },
        "handle_query": {
          "id": "handle_query",
          "state_prompt": "Answer the users question helpfully."
        }
      },
      "entry_node_id": "greeting"
    },
    "source": "custom"
  }'
```

--------------------------------

### CLI: Snippet Management

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Commands for creating and applying snippets to agents.

```APIDOC
## CLI: Snippet Management

### Create or update a snippet
```bash
voicetest snippet set --agent agent.json greeting "Hello, how can I help you today?"
```

### Apply snippets to prompts
```bash
voicetest snippet apply --agent agent.json --snippets '[{"name": "greeting", "text": "Hello!"}]'
```
```

--------------------------------

### REST API: Import Agent

Source: https://context7.com/voicetestdev/voicetest/llms.txt

Import an agent configuration from any supported format via the REST API.

```APIDOC
## REST API: Import Agent

Import an agent configuration from any supported format.

### Request
```bash
curl -X POST http://localhost:8000/api/agents/import \
  -H "Content-Type: application/json" \
  -d '{ \
    "config": { \
      "nodes": { \
        "greeting": { \
          "id": "greeting", \
          "state_prompt": "Greet the user warmly and ask how you can help.", \
          "transitions": [ \
            { \
              "target_node_id": "handle_query", \
              "condition": {"type": "llm_prompt", "value": "User states their question"} \
            } \
          ] \
        }, \
        "handle_query": { \
          "id": "handle_query", \
          "state_prompt": "Answer the users question helpfully." \
        } \
      }, \
      "entry_node_id": "greeting" \
    }, \
    "source": "custom" \
  }'
```

### Response
```json
{
  "nodes": {...},
  "entry_node_id": "greeting",
  "source_type": "custom",
  "source_metadata": {}
}
```
```