ValueHead

https://valuehead.ai/docs
Admin
ValueHead is an LLM-as-judge observability platform that evaluates AI agent conversations by...
Tokens:5,703
Snippets:54
Trust Score:2.1
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install ValueHead SDK from Source

Source: https://valuehead.ai/docs

Install the ValueHead Python SDK directly from its GitHub repository.

```bash
pip install git+https://github.com/Aradhye2002/valuehead-sdk.git
```

--------------------------------

### Install ValueHead SDK via Pip

Source: https://valuehead.ai/docs

Use this command to install the ValueHead Python SDK using pip.

```bash
pip install valuehead
```

--------------------------------

### Clone and Run ValueHead Locally

Source: https://valuehead.ai/docs

Instructions for self-hosting ValueHead by cloning the repository, setting up environment variables, installing dependencies, and running the application.

```bash
git clone https://github.com/Aradhye2002/valuehead-sdk.git
cd valuehead
cp .env.example .env      # add your OpenAI API key
pip install -r requirements.txt
cd frontend && npm install && npm run build && cd ..
python run.py
```

--------------------------------

### Quick Start: Submit Conversation Trace

Source: https://valuehead.ai/docs

Submit a conversation trace to ValueHead for evaluation and retrieve session results. Ensure you have your API key.

```python
from valuehead import ValueHead

client = ValueHead(api_key="vh_your_api_key")

result = client.submit(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the weather in London?"},
        {"role": "assistant", "content": "Let me check.", "tool_calls": [
            {"id": "call_1", "type": "function", "function": {
                "name": "get_weather",
                "arguments": "{\"city\": \"London\"}"
            }}
        ]},
        {"role": "tool", "content": "{\"temp\": 15, \"condition\": \"cloudy\"}",
         "tool_call_id": "call_1"},
        {"role": "assistant", "content": "London is 15°C and cloudy.",}
    ],
    metadata={"agent": "weather-bot"},
)

# Wait for evaluation
session = client.wait(result.session_id)

# Print results
print(f"Turns: {session.total_turns}, Score: {session.summary.net_score}")
for turn_num, j in session.judgements.items():
    print(f"  Turn {j.turn}: {j.score} — {j.turn_reasoning[:100]}")
```

--------------------------------

### Get profile

Source: https://valuehead.ai/docs

Retrieves the profile information for the authenticated user. Requires authentication.

```APIDOC
## GET /api/v1/auth/me

### Description
Get the profile information for the authenticated user.

### Method
GET

### Endpoint
/api/v1/auth/me

### Response
#### Success Response (200)
- **email** (string) - User's email address.
- **api_key** (string) - User's API key.

### Headers
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Get JWT token

Source: https://valuehead.ai/docs

Authenticates a user and returns a JWT token. Does not require authentication.

```APIDOC
## POST /api/v1/auth/login

### Description
Authenticate a user and obtain a JWT token.

### Method
POST

### Endpoint
/api/v1/auth/login

### Parameters
#### Request Body
- **email** (string) - Required - User's email address.
- **password** (string) - Required - User's password.

### Response
#### Success Response (200)
- **access_token** (string) - The JWT access token.
```

--------------------------------

### Get Trace Results with Curl

Source: https://valuehead.ai/docs

Retrieve the results of a submitted trace using its session ID. Replace '{session_id}' and 'vh_your_key' with the appropriate values.

```bash
curl https://valuehead-production.up.railway.app/api/v1/traces/{session_id} \
  -H "X-Api-Key: vh_your_key"
```

--------------------------------

### Get session + judgements

Source: https://valuehead.ai/docs

Retrieves a specific session along with its associated judgements. Requires authentication.

```APIDOC
## GET /api/v1/traces/{id}

### Description
Get session details and judgements for a given session ID.

### Method
GET

### Endpoint
/api/v1/traces/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The ID of the session to retrieve.

### Response
#### Success Response (200)
- **SessionDetail** (object) - Contains session ID, status, turns, judgements, trajectory, summary, error, and metadata.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Initialize ValueHead Client

Source: https://valuehead.ai/docs

Create an instance of the ValueHead client with your API key.

```APIDOC
## Initialize ValueHead Client

### Description
Create a client instance to interact with the ValueHead API. The client can be used as a context manager.

### Parameters
#### Path Parameters
- **api_key** (string) - Required - Your API key (starts with vh_)
- **base_url** (string) - Optional - API base URL (defaults to valuehead.ai)
- **timeout** (float) - Optional - HTTP timeout in seconds (defaults to 30.0)

### Request Example
```python
from valuehead import ValueHead

client = ValueHead(api_key="vh_your_api_key")
```
```

--------------------------------

### Initialize ValueHead Client

Source: https://valuehead.ai/docs

Create an instance of the ValueHead client with your API key. The base URL and timeout can also be configured.

```python
from valuehead import ValueHead

client = ValueHead(api_key="vh_your_api_key")
```

--------------------------------

### Configure ValueHead SDK for Self-Hosted Instance

Source: https://valuehead.ai/docs

Instantiate the ValueHead client, pointing it to your self-hosted instance by providing the base URL and API key.

```python
client = ValueHead(api_key="vh_...", base_url="http://localhost:8000")
```

--------------------------------

### Create account

Source: https://valuehead.ai/docs

Creates a new user account. Does not require authentication.

```APIDOC
## POST /api/v1/auth/register

### Description
Create a new user account.

### Method
POST

### Endpoint
/api/v1/auth/register

### Parameters
#### Request Body
- **email** (string) - Required - User's email address.
- **password** (string) - Required - User's password.

### Response
#### Success Response (200)
- **message** (string) - Confirmation message.
```

--------------------------------

### Submit Trace with Curl

Source: https://valuehead.ai/docs

Use this command to submit a structured trace to the ValueHead API. Ensure you replace 'vh_your_key' with your actual API key.

```bash
curl -X POST https://valuehead-production.up.railway.app/api/v1/traces \
  -H "Content-Type: application/json" \
  -H "X-Api-Key: vh_your_key" \
  -d '{
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."}, 
      {"role": "user", "content": "Hello"},
      {"role": "assistant", "content": "Hi! How can I help you?"}
    ]
  }'
```

--------------------------------

### Batch Submission via REST API

Source: https://valuehead.ai/docs

Submit multiple conversation traces simultaneously using the ValueHead REST API. Requires your API key.

```bash
curl -X POST https://valuehead-production.up.railway.app/api/v1/traces/batch \
  -H "Content-Type: application/json" \
  -H "X-Api-Key: vh_your_key" \
  -d '{"traces": [{"messages": [...], "metadata": {...}}, ...]}'
```

--------------------------------

### Submit multiple traces

Source: https://valuehead.ai/docs

Submits multiple traces in a batch for processing. Requires authentication.

```APIDOC
## POST /api/v1/traces/batch

### Description
Submit multiple traces in a batch for processing.

### Method
POST

### Endpoint
/api/v1/traces/batch

### Parameters
#### Request Body
- **traces** (array) - Required - An array of trace objects to submit.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Submit a trace (async)

Source: https://valuehead.ai/docs

Submits a structured trace for asynchronous processing. Requires authentication.

```APIDOC
## POST /api/v1/traces

### Description
Submit a structured trace for asynchronous processing.

### Method
POST

### Endpoint
/api/v1/traces

### Request Body
- **messages** (array) - Required - Array of message objects, each with 'role' and 'content'.
- **metadata** (object) - Optional - Key-value pairs for additional context.

### Request Example
```json
{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."}, 
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi! How can I help you?"}
  ]
}
```

### Response
#### Success Response (200)
- **id** (str) - The ID of the submitted session.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Stream Judgements with SSE

Source: https://valuehead.ai/docs

Use this cURL command to stream judgements as they are produced by the ValueHead API. This is useful for live evaluation displays.

```bash
curl -N https://valuehead-production.up.railway.app/api/v1/traces/{session_id}/stream \
  -H "X-Api-Key: vh_your_key"
```

--------------------------------

### Submit Structured Conversation Trace

Source: https://valuehead.ai/docs

Submit a conversation trace in OpenAI format for evaluation. Each user message initiates a new turn.

```python
result = client.submit(
    messages=[
        {"role": "system", "content": "You are a booking assistant."},
        {"role": "user", "content": "Book me a flight to Paris."},
        {"role": "assistant", "content": "Searching.", "tool_calls": [
            {"id": "c1", "type": "function", "function": {
                "name": "search_flights",
                "arguments": "{\"destination\": \"Paris\"}"
            }}
        ]},
        {"role": "tool", "content": "{\"flights\": [{\"id\": \"FL123\"}]}",
         "tool_call_id": "c1"},
        {"role": "assistant", "content": "Found a flight for $450. Book it?"},
    ],
    metadata={"agent": "travel-bot"},
)
```

--------------------------------

### Submit Raw Text Transcript for Evaluation

Source: https://valuehead.ai/docs

Submit unstructured text for evaluation. ValueHead's LLM parser will identify speakers and split into turns.

```python
result = client.submit_text(
    text="""
    Agent: How can I help you today?
    Customer: My internet has been down since yesterday.
    Agent: Let me check your account status.
    """,
    context="ISP customer support call",
    metadata={"department": "technical"},
)
```

--------------------------------

### Submit Voice Recording for Evaluation

Source: https://valuehead.ai/docs

Upload an audio file for automatic transcription, speaker diarization, and evaluation. Supports multilingual audio.

```python
result = client.submit_voice(
    "recording.ogg",
    speakers_expected=2,
    first_speaker_role="user",      # or "assistant" for outbound calls
    metadata={"call_type": "support"},
)
session = client.wait(result.session_id)
```

--------------------------------

### Submit voice recording

Source: https://valuehead.ai/docs

Submits an audio recording for processing. Requires authentication.

```APIDOC
## POST /api/v1/traces/voice

### Description
Submit an audio recording for processing.

### Method
POST

### Endpoint
/api/v1/traces/voice

### Parameters
#### Request Body
- **file_path** (string) - Required - Path to the audio file.
- **context** (string) - Optional - Context for the transcription.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Submit Voice Recordings

Source: https://valuehead.ai/docs

Upload audio files for transcription and evaluation.

```APIDOC
## Submit Voice Recordings

### Description
Upload audio files (wav, mp3, ogg, m4a) for automatic transcription with speaker diarization, followed by evaluation. Supports multilingual audio with automatic language detection.

### Method
`client.submit_voice`

### Parameters
#### Path Parameters
- **file_path** (str) - Required - Path to audio file.
- **speakers_expected** (int | None) - Optional - Diarization hint (e.g. 2).
- **human_speaker** (str | None) - Optional - Override speaker-to-role mapping.
- **first_speaker_role** (str) - Optional - Defaults to "user", can be "assistant" for outbound calls.
- **metadata** (dict | None) - Optional - Extra metadata tags.

### Request Example
```python
result = client.submit_voice(
    "recording.ogg",
    speakers_expected=2,
    first_speaker_role="user",      # or "assistant" for outbound calls
    metadata={"call_type": "support"},
)
session = client.wait(result.session_id)
```
```

--------------------------------

### SSE stream of live judgements

Source: https://valuehead.ai/docs

Provides a Server-Sent Events (SSE) stream for live judgements of a session. Requires authentication.

```APIDOC
## GET /api/v1/traces/{id}/stream

### Description
Get a Server-Sent Events (SSE) stream of live judgements for a session.

### Method
GET

### Endpoint
/api/v1/traces/{id}/stream

### Parameters
#### Path Parameters
- **id** (string) - Required - The ID of the session to stream judgements for.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Batch Submission (REST API)

Source: https://valuehead.ai/docs

Submit multiple conversation traces at once via the REST API.

```APIDOC
## Batch Submission (REST API)

### Description
Submit multiple conversation traces simultaneously using the ValueHead REST API.

### Method
`POST`

### Endpoint
`/api/v1/traces/batch`

### Headers
- **Content-Type**: `application/json`
- **X-Api-Key**: `vh_your_key`

### Request Body
- **traces** (list) - Required - A list of trace objects, where each object contains `messages` and optional `metadata`.

### Request Example
```bash
curl -X POST https://valuehead-production.up.railway.app/api/v1/traces/batch \
  -H "Content-Type: application/json" \
  -H "X-Api-Key: vh_your_key" \
  -d '{"traces": [{"messages": [...], "metadata": {...}}, ...] }'
```
```

--------------------------------

### List sessions (paginated)

Source: https://valuehead.ai/docs

Retrieves a paginated list of sessions, ordered by newest first. Requires authentication.

```APIDOC
## GET /api/v1/traces

### Description
List sessions, ordered by newest first. Supports pagination.

### Method
GET

### Endpoint
/api/v1/traces

### Parameters
#### Query Parameters
- **limit** (integer) - Optional - Maximum number of sessions to return.
- **offset** (integer) - Optional - Number of sessions to skip for pagination.

### Response
#### Success Response (200)
- **SessionsListResponse** (object) - Contains a list of session objects.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Submit Structured Conversation Traces

Source: https://valuehead.ai/docs

Submit OpenAI-format messages for evaluation.

```APIDOC
## Submit Structured Conversation Traces

### Description
Submit conversation traces in OpenAI message format for evaluation. Each user message initiates a new evaluation turn. Supports both OpenAI-style `tool_calls` and XML-style tags.

### Method
`client.submit`

### Parameters
#### Request Body
- **messages** (list) - Required - List of message objects in OpenAI format.
- **metadata** (dict) - Optional - Extra metadata tags for the submission.

### Request Example
```python
result = client.submit(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the weather in London?"},
        {"role": "assistant", "content": "Let me check.", "tool_calls": [
            {"id": "call_1", "type": "function", "function": {
                "name": "get_weather",
                "arguments": "{\"city\": \"London\"}"
            }}
        ]},
        {"role": "tool", "content": "{\"temp\": 15, \"condition\": \"cloudy\"}",
         "tool_call_id": "call_1"},
        {"role": "assistant", "content": "London is 15°C and cloudy."},
    ],
    metadata={"agent": "weather-bot"},
)
```
```

--------------------------------

### Rotate API key

Source: https://valuehead.ai/docs

Rotates the user's API key. Requires authentication.

```APIDOC
## POST /api/v1/auth/rotate-key

### Description
Rotate the user's API key.

### Method
POST

### Endpoint
/api/v1/auth/rotate-key

### Response
#### Success Response (200)
- **new_api_key** (string) - The newly generated API key.

### Headers
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Submit raw text transcript

Source: https://valuehead.ai/docs

Submits a raw text transcript for processing. Requires authentication.

```APIDOC
## POST /api/v1/traces/text

### Description
Submit a raw text transcript for processing.

### Method
POST

### Endpoint
/api/v1/traces/text

### Parameters
#### Request Body
- **text** (string) - Required - The raw text transcript.
- **context** (string) - Optional - Context for the transcript.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Synchronous evaluation

Source: https://valuehead.ai/docs

Performs a synchronous evaluation of a session. Requires authentication.

```APIDOC
## POST /api/v1/traces/{id}/evaluate

### Description
Perform a synchronous evaluation of a session.

### Method
POST

### Endpoint
/api/v1/traces/{id}/evaluate

### Parameters
#### Path Parameters
- **id** (string) - Required - The ID of the session to evaluate.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Handle Evaluation Timeouts in Python

Source: https://valuehead.ai/docs

This Python snippet demonstrates how to handle `EvaluationTimeoutError` when waiting for evaluation results. If a timeout occurs, it retrieves the session data.

```python
from valuehead import ValueHead, EvaluationTimeoutError

try:
    session = client.wait(result.session_id, timeout=60)
except EvaluationTimeoutError:
    print("Still evaluating, check back later")
    session = client.get(result.session_id)
```

--------------------------------

### Submit Raw Text Transcripts

Source: https://valuehead.ai/docs

Submit unstructured text for parsing and evaluation.

```APIDOC
## Submit Raw Text Transcripts

### Description
Submit unstructured text. ValueHead's LLM parser identifies speakers and splits the text into turns for evaluation.

### Method
`client.submit_text`

### Parameters
#### Request Body
- **text** (str) - Required - The unstructured text transcript.
- **context** (str) - Optional - Context for the text, e.g., "ISP customer support call".
- **metadata** (dict) - Optional - Extra metadata tags.

### Request Example
```python
result = client.submit_text(
    text="""
    Agent: How can I help you today?
    Customer: My internet has been down since yesterday.
    Agent: Let me check your account status.
    """,
    context="ISP customer support call",
    metadata={"department": "technical"},
)
```
```

--------------------------------

### Delete all sessions

Source: https://valuehead.ai/docs

Deletes all existing sessions. Requires authentication.

```APIDOC
## DELETE /api/v1/traces

### Description
Delete all existing sessions.

### Method
DELETE

### Endpoint
/api/v1/traces

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```

--------------------------------

### Wait for Evaluation Results

Source: https://valuehead.ai/docs

Poll for the completion of an evaluation session.

```APIDOC
## Wait for Evaluation Results

### Description
Poll for the completion of an evaluation session using its session ID and retrieve the results.

### Method
`client.wait`

### Parameters
#### Path Parameters
- **session_id** (string) - Required - The ID of the evaluation session.

### Request Example
```python
session = client.wait(result.session_id)
```
```

--------------------------------

### Delete a session

Source: https://valuehead.ai/docs

Deletes a specific session by its ID. Requires authentication.

```APIDOC
## DELETE /api/v1/traces/{id}

### Description
Delete a specific session by its ID.

### Method
DELETE

### Endpoint
/api/v1/traces/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The ID of the session to delete.

### Headers
- **X-Api-Key** (string) - Required - Your API key (e.g., vh_your_key).
- **Authorization** (string) - Required - Bearer token (e.g., Bearer <jwt>).
```
Tokens:
Raw