Transluce Docent

https://docs.transluce.org
Admin
Docent is a tool that summarizes, clusters, and searches agent transcripts to surface observations,...
Tokens:70,556
Snippets:661
Trust Score:9
Update:4 months ago
Show doc for...
Tokens:
Raw
### Quick Example: Create Collection and Query Data

Source: https://docs.transluce.org/sdk/overview

This snippet demonstrates how to initialize the Docent client, create a new collection for agent runs, and execute a DQL query to retrieve data. Ensure you have your API key and have installed the Docent SDK.

```python
from docent import Docent

client = Docent(api_key="your-api-key")

# Create a collection
collection_id = client.create_collection(name="My Agent Runs")

# Query with DQL
result = client.execute_dql(collection_id, "SELECT id, name, metadata_json->>'model' AS model FROM agent_runs LIMIT 5")
rows = client.dql_result_to_dicts(result)
```

--------------------------------

### Install Docent SDK with uv

Source: https://docs.transluce.org/installation

Install the Docent Python SDK in your project using the `uv` package manager. Ensure you are in your project directory before running the commands.

```shell
cd your-project
uv init && uv add docent-python
```

--------------------------------

### Install Docent Plugin for Codex

Source: https://docs.transluce.org/installation

Add the Transluce AI marketplace and install the Docent plugin for Codex. Start a new Codex session to load the Docent skills and MCP tools.

```shell
codex plugin marketplace add TransluceAI/codex-plugins
codex plugin add docent@transluce-plugins
```

--------------------------------

### Install Docent Python SDK

Source: https://docs.transluce.org/ingestion/tracing

Install the Docent Python SDK using pip. This package includes the tracing system.

```bash
pip install docent-python
```

--------------------------------

### Install Inspect AI Package (pip)

Source: https://docs.transluce.org/ingestion/sdk

Installs the inspect-ai package using pip.

```bash
pip install inspect-ai
```

--------------------------------

### Install Docent Plugin for Claude Code

Source: https://docs.transluce.org/installation

Add the Transluce AI marketplace and install the Docent plugin for Claude Code. Restart Claude Code for the plugin to be recognized. Verify by typing '/'.

```shell
claude plugin marketplace add TransluceAI/claude-code-plugins
claude plugin install docent@transluce-plugins
```

--------------------------------

### Install Inspect AI Package

Source: https://docs.transluce.org/ingestion/sdk

Installs the inspect-ai package using uv, a fast Python package installer.

```bash
uv add inspect-ai
```

--------------------------------

### Config File Format Example

Source: https://docs.transluce.org/sdk/authentication

Configuration files use dotenv format. Supported keys include API key, URLs, domain, and collection ID.

```bash
DOCENT_API_KEY=your-api-key
DOCENT_API_URL=https://api.docent.transluce.org
DOCENT_FRONTEND_URL=https://docent.transluce.org
DOCENT_DOMAIN=docent.transluce.org
DOCENT_COLLECTION_ID=my-collection
```

--------------------------------

### Example: Poll Until Complete

Source: https://docs.transluce.org/sdk/jobs

Provides a Python function example demonstrating how to poll for the completion status of multiple jobs, useful when automatic waiting is not enabled.

```APIDOC
## Example: Poll Until Complete

### Description
Demonstrates how to poll for the completion status of multiple jobs using `get_agent_run_job_statuses`.

### Usage
This function is useful when `wait=False` is used with methods like `add_agent_runs`, requiring manual polling for job completion.

### Code Example
```python
import time

def wait_for_jobs(client, collection_id, job_ids, poll_interval=2.0):
    pending = set(job_ids)
    while pending:
        statuses = client.get_agent_run_job_statuses(collection_id, list(pending))
        for s in statuses:
            if s["status"] in ("completed", "canceled", "cancelling"):
                pending.discard(s["job_id"])
                print(f"Job {s['job_id']}: {s['status']}")
        if pending:
            time.sleep(poll_interval)
    print("All jobs finished")
```

### Note
When using `add_agent_runs` with `wait=True` (the default), job polling is handled automatically. Manual polling is only necessary when `wait=False`.
```

--------------------------------

### Docent Client Initialization with Full Configuration

Source: https://docs.transluce.org/sdk/client

Initialize the Docent client with all available configuration options, including custom API and frontend URLs. This example also prints the resolved frontend and backend URLs.

```python
from docent import Docent

client = Docent(
    api_key="your-api-key",
    collection_id="my-collection",
    api_url="https://api.my-docent.example.com",
    frontend_url="https://my-docent.example.com",
)

print(client.frontend_url)  # https://my-docent.example.com
print(client.backend_url)   # https://api.my-docent.example.com/rest
```

--------------------------------

### Start a Chat

Source: https://docs.transluce.org/sdk/ui

Creates an interactive chat session with specified agent runs or transcripts as context, and opens it in the browser. Returns the session ID.

```APIDOC
## Start a Chat

### Description
Creates an interactive chat session with specified agent runs or transcripts as context, and opens it in the browser. Returns the session ID.

### Method
client.start_chat

### Parameters
#### Path Parameters
- **context** (LLMContext | list[AgentRun | Transcript]) - Required - Objects to include as chat context. Can be a list of `AgentRun` or `Transcript` instances, or a pre-built `LLMContext`.
- **model_string** (str | None) - Optional - Model to use, in `"provider/model_name"` format.
- **reasoning_effort** (Literal['minimal', 'low', 'medium', 'high'] | None) - Optional - Reasoning effort hint passed to the LLM provider.

### Returns
#### Success Response
- **session_id** (str) - The session ID of the created chat session.
```

--------------------------------

### Export all transcripts and metadata

Source: https://docs.transluce.org/analysis/exporting

Download all transcripts joined with agent run metadata. Ensure the 'docent' library is installed and a 'Docent' client is initialized. The 'collection_id' must be specified.

```python
from docent import Docent
import json

client = Docent()
collection_id = "your-collection-id"

# Download all transcripts joined with agent run metadata
result = client.execute_dql(
    collection_id,
    """SELECT
  t.id AS transcript_id,
  t.name AS transcript_name,
  t.messages,
  t.metadata_json AS transcript_metadata,
  ar.id AS agent_run_id,
  ar.name AS agent_run_name,
  ar.metadata_json AS agent_run_metadata
FROM transcripts t
JOIN agent_runs ar ON ar.id = t.agent_run_id"""
)

rows = client.dql_result_to_dicts(result)

# Parse the messages JSON from each transcript
for row in rows:
    row["messages"] = json.loads(row["messages"]) if isinstance(row["messages"], str) else row["messages"]

print(f"Downloaded {len(rows)} transcripts")
print(f"First transcript has {len(rows[0]['messages'])} messages")

```

--------------------------------

### Start a Chat Session with Context in Browser

Source: https://docs.transluce.org/sdk/ui

Initiates an interactive chat session in the browser, using specified agent runs as context. Returns the session ID.

```python
run1 = client.get_agent_run("my-collection-id", "run-1")
run2 = client.get_agent_run("my-collection-id", "run-2")

session_id = client.start_chat([run1, run2])
# Opens browser to chat UI with both runs as context
```

--------------------------------

### Example Agent Run Metadata Dictionary

Source: https://docs.transluce.org/concepts/metadata

This dictionary demonstrates a typical structure for agent run metadata, including conventional fields for scores and custom fields for tracking specific run details.

```python
metadata = {
    # Optional conventional fields
    "scores": {"reward_1": 0.1, "reward_2": 0.5, "reward_3": 0.8},
    # Custom fields
    "episode": 42,
    "policy_version": "v1.2.3",
    "training_step": 12500,
}
```

--------------------------------

### Execute a DQL Query

Source: https://docs.transluce.org/sdk/dql/execute

Use the `execute_dql` method to run a DQL query against a specified collection. This example demonstrates fetching the first 10 agent runs, selecting their IDs and model metadata.

```python
from docent import Docent

client = Docent()

result = client.execute_dql(
    "my-collection-id",
    "SELECT id, metadata_json->>'model' AS model FROM agent_runs LIMIT 10",
)
```

--------------------------------

### Get and Use a Judge for Agent Runs

Source: https://docs.transluce.org/sdk/rubrics/manage

Downloads a rubric configuration to create a callable judge instance. It demonstrates inspecting the judge's configuration and asynchronously evaluating an agent run.

```python
judge = client.get_judge("my-collection-id", rubric_id)

# Inspect the configuration
print(judge.cfg.rubric_text)
print(judge.cfg.judge_model)

# Run locally (async)
import asyncio

async def evaluate():
    run = client.get_agent_run("my-collection-id", "run-id-123")
    result = await judge(run)
    print(result.output)  # {"label": "pass", "explanation": "..."}
    print(result.result_type)  # ResultType.DIRECT_RESULT

asyncio.run(evaluate())
```

--------------------------------

### Get a Judge

Source: https://docs.transluce.org/sdk/rubrics/manage

Downloads a rubric configuration and creates a callable judge instance for evaluating agent runs. LLM provider API keys are read from environment variables.

```APIDOC
## Get a Judge

### Description
Downloads a rubric configuration and creates a callable judge instance for evaluating agent runs. LLM provider API keys are read from environment variables.

### Method
`client.get_judge(collection_id: str, rubric_id: str, version: int | None = None)`

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **rubric_id** (str) - Required - ID of the rubric/judge to retrieve.

#### Query Parameters
- **version** (int | None) - Optional - Specific version number. If `None`, returns the latest version.

### Returns
#### Success Response
- **judge** (BaseJudge) - A callable judge instance. Use `await judge(agent_run)` to evaluate a run.
  - **cfg** (Rubric) - The underlying rubric configuration.
    - **rubric_text** (str)
    - **judge_model** (str)
  - **__call__(agent_run, *, temperature=1.0, max_new_tokens=16384, timeout=180.0)** (async -> JudgeResult) - Evaluate an agent run. Returns a `JudgeResult` with `output`, `result_type`, and `result_metadata` fields.
    - **output** (dict)
    - **result_type** (ResultType)
    - **result_metadata** (dict)

### Note
Running a judge locally requires the appropriate LLM provider API key set in your environment (e.g., `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`). The required provider depends on the rubric's `judge_model` configuration.
```

--------------------------------

### Get Collection Overview

Source: https://docs.transluce.org/analysis/quickstart

A simple prompt to retrieve a general overview of a specified collection. This is useful for initial data exploration and understanding the contents of a collection. Ensure auto-approval is enabled for the analysis plan.

```text
/docent Give me an overview of this collection.

- Collection ID: 479b7093-5a33-47f1-8d7b-fc9f6f16bb75
- Auto accept analysis plan
```

--------------------------------

### Run and Monitor Rubric Evaluation

Source: https://docs.transluce.org/sdk/rubrics/evaluate

Starts a rubric evaluation job, polls for its completion, and then analyzes the results. Ensure you have initialized the Docent client and have valid collection and rubric IDs.

```python
import time
from docent import Docent

client = Docent()
collection_id = "my-collection-id"
rubric_id = "rubric-123"

# Start evaluation
job_id = client.start_rubric_eval_job(collection_id, rubric_id)
print(f"Started job: {job_id}")

# Poll for completion
while True:
    state = client.get_rubric_run_state(collection_id, rubric_id)
    current = state.get("current_results_count", 0)
    total = state.get("total_results_needed", 0)
    print(f"Progress: {current}/{total}")

    if state.get("job_status") in ("completed", "canceled"):
        break
    time.sleep(5)

# Analyze results — each entry groups judge results by agent run
for entry in state.get("results", []):
    for judge_result in entry["results"]:
        print(f"Run {entry['agent_run_id']}: {judge_result['output']}")
```

--------------------------------

### Get Rubric Run State

Source: https://docs.transluce.org/sdk/rubrics/evaluate

Fetches the current state of a rubric evaluation. Use this to check progress and retrieve results after starting an evaluation job.

```python
state = client.get_rubric_run_state("my-collection-id", "rubric-123")
print(f"Total results needed: {state['total_results_needed']}")
print(f"Results so far: {len(state.get('results', []))}")
```

--------------------------------

### Docent Client Initialization

Source: https://docs.transluce.org/sdk/client

Demonstrates how to initialize the Docent client with minimal configuration (reading from environment/config files) and with explicit parameters.

```APIDOC
## Initialization

```python
from docent import Docent

# Minimal — reads API key from a discovered config file or environment
client = Docent()

# With explicit configuration
client = Docent(
    api_key="your-api-key",
    collection_id="my-default-collection",
)
```

### Constructor Parameters

<ParamField body="api_key" type="str | None">
  API key for authentication. Falls back to the discovered config file
  (`~/.docent/docent.env` when no project-level file is found), then the
  `DOCENT_API_KEY` environment variable. See the
  [configuration priority table](/sdk/authentication#configuration-priority) for
  the full resolution order. **Required** — if no key is found, raises `ValueError`.
</ParamField>

<ParamField body="collection_id" type="str | None">
  Stored as `default_collection_id` for your convenience, but SDK methods
  do **not** fall back to it automatically — you must pass `collection_id`
  explicitly to each method call. Falls back to `DOCENT_COLLECTION_ID` from
  the config file.
</ParamField>

<ParamField body="api_url" type="str | None">
  Direct URL of the Docent API server. Overrides URL derived from `domain`.
  Falls back to `DOCENT_API_URL` environment variable.
</ParamField>

<ParamField body="frontend_url" type="str | None">
  Direct URL of the Docent frontend UI. Overrides URL derived from `domain`.
  Falls back to `DOCENT_FRONTEND_URL` environment variable.
</ParamField>

<ParamField body="domain" type="str | None" default="docent.transluce.org">
  Domain of the Docent instance. API and frontend URLs are derived as
  `https://api.{domain}` and `https://{domain}` unless overridden.
</ParamField>

<ParamField body="use_https" type="bool" default="True">
  Whether to use HTTPS when constructing URLs from the domain.
</ParamField>

<ParamField body="config_file" type="str | Path | None">
  Explicit path to a dotenv config file. If not provided, the SDK searches for
  project-level `docent.env` overrides from the current directory upward, then
  checks the default global config file at `~/.docent/docent.env`.
</ParamField>

<ParamField body="log_stream" type="IO[str] | None">
  Output stream for SDK log messages. Defaults to `sys.stdout`.
</ParamField>

## Properties

<ResponseField name="frontend_url" type="str">
  The resolved Docent frontend base URL.
</ResponseField>

<ResponseField name="backend_url" type="str">
  The resolved Docent API base URL.
</ResponseField>

<ResponseField name="default_collection_id" type="str | None">
  The default collection ID, if configured.
</ResponseField>

## Example: Full Configuration

```python
from docent import Docent

client = Docent(
    api_key="your-api-key",
    collection_id="my-collection",
    api_url="https://api.my-docent.example.com",
    frontend_url="https://my-docent.example.com",
)

print(client.frontend_url)  # https://my-docent.example.com
print(client.backend_url)   # https://api.my-docent.example.com/rest
```
```

--------------------------------

### Get Rubric Run State

Source: https://docs.transluce.org/sdk/rubrics/evaluate

Retrieves the current state of a rubric evaluation, including results and progress. This method does not start an evaluation; it only fetches the current status.

```APIDOC
## Get Rubric Run State

### Description
Retrieves the current state of a rubric evaluation, including results and progress. This method does not start an evaluation; it only fetches the current status.

### Method
GET (or equivalent SDK method call)

### Endpoint
(Conceptual - SDK method call)

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **rubric_id** (str) - Required - ID of the rubric.

#### Query Parameters
- **version** (int | None) - Optional - Rubric version. If `None`, uses the latest version.
- **filter_dict** (dict | None) - Optional - Optional filter to apply to results.
- **include_failures** (bool) - Optional - Whether to include failed judge results in the response. Defaults to `False`.

### Returns
#### Success Response
- **state** (dict) - Evaluation state.
  - **results** (list[dict]) - List of per-agent-run result groups. Each entry contains:
    - **agent_run_id** (str) - The agent run that was evaluated.
    - **rubric_id** (str) - The rubric used.
    - **rubric_version** (int) - The rubric version used.
    - **results** (list[dict]) - List of individual judge results, each with `output`, `result_type`, and `result_metadata`.
    - **reflection** (dict | None) - Reflection data, if the judge variant uses multi-reflection.
  - **job_id** (str | None) - ID of the evaluation job, if one exists.
  - **job_status** (str | None) - Status of the job: `"pending"`, `"running"`, `"completed"`, or `"canceled"`.
  - **total_results_needed** (int | None) - Total number of results expected when evaluation is complete.
  - **current_results_count** (int | None) - Number of results completed so far.

### Request Example
```python
state = client.get_rubric_run_state("my-collection-id", "rubric-123")
print(f"Total results needed: {state['total_results_needed']}")
print(f"Results so far: {len(state.get('results', []))}")
```

### Note
`get_rubric_run_state` does **not** start an evaluation. Use `start_rubric_eval_job()`
first, then poll `get_rubric_run_state()` to check progress.
```

--------------------------------

### Create and Use a Global Config File

Source: https://docs.transluce.org/sdk/authentication

Create a global config file at ~/.docent/docent.env to store API keys and collection IDs. Project-level files can override these settings.

```bash
mkdir -p ~/.docent
cat <<'EOF' > ~/.docent/docent.env
DOCENT_API_KEY=your-api-key
DOCENT_COLLECTION_ID=my-default-collection
EOF
```

--------------------------------

### Configure Docent API Key and Domain

Source: https://docs.transluce.org/installation

Create the default global Docent configuration file `~/.docent/docent.env` and set your API key and domain. This file can be overridden by project-level `docent.env` files.

```shell
mkdir -p ~/.docent
cat <<'EOF' > ~/.docent/docent.env
DOCENT_API_KEY=<YOUR_API_KEY>
DOCENT_DOMAIN=docent.transluce.org
EOF
```

--------------------------------

### Docent Client Initialization with Explicit Configuration

Source: https://docs.transluce.org/sdk/client

Initialize the Docent client by explicitly providing the API key and a default collection ID.

```python
from docent import Docent

# With explicit configuration
client = Docent(
    api_key="your-api-key",
    collection_id="my-default-collection",
)
```

--------------------------------

### Initialize Docent Tracing (Basic)

Source: https://docs.transluce.org/ingestion/tracing

Perform basic initialization of the Docent tracing system by providing an application collection name.

```python
from docent.trace import initialize_tracing

# Basic initialization
initialize_tracing("my-application")
```

--------------------------------

### Initialize Docent with Explicit Config File Path

Source: https://docs.transluce.org/sdk/authentication

Specify an explicit path to a configuration file when initializing the Docent client.

```python
client = Docent(config_file="/path/to/my-config.env")
```

--------------------------------

### Get Comments

Source: https://docs.transluce.org/sdk/feedback/labels

Retrieves comments associated with a collection or a specific agent run.

```APIDOC
## Get Comments

### Description
Retrieves comments associated with a collection or a specific agent run.

### Method
```python
client.get_comments(collection_id)
client.get_comments_for_agent_run(collection_id, run_id)
```

### Parameters
- **collection_id** (string) - Required - The ID of the collection.
- **run_id** (string) - Required - The ID of the agent run (for `get_comments_for_agent_run`).
```

--------------------------------

### Minimal Docent Client Initialization

Source: https://docs.transluce.org/sdk/client

Initialize the Docent client with minimal configuration. It automatically reads the API key from a discovered config file or environment variable.

```python
from docent import Docent

# Minimal — reads API key from a discovered config file or environment
client = Docent()
```

--------------------------------

### Get Collection Metadata

Source: https://docs.transluce.org/sdk/collections/metadata

Fetches the complete metadata dictionary for a specified collection.

```APIDOC
## Get Collection Metadata

### Description
Fetches the complete metadata dictionary for a specified collection.

### Method
GET (Implied by `get_collection_metadata`)

### Endpoint
`/collections/{collection_id}/metadata` (Implied)

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.

### Returns
#### Success Response
- **metadata** (dict) - The collection's metadata dictionary.

### Response Example
```json
{
  "metadata": {
    "team": "support",
    "model": "gpt-4"
  }
}
```
```

--------------------------------

### Initialize Docent with API Key Parameter

Source: https://docs.transluce.org/sdk/authentication

Provide your API key directly as a parameter when initializing the Docent client.

```python
from docent import Docent

client = Docent(api_key="your-api-key")
```

--------------------------------

### Get General Label

Source: https://docs.transluce.org/sdk/feedback/labels

Fetches a specific general label by its ID within a given collection.

```APIDOC
## Get General Label

### Description
Retrieves a specific general label using its ID.

### Method
client.get_general_label

### Parameters
- **collection_id** (str) - Required - ID of the collection.
- **label_id** (str) - Required - ID of the general label to retrieve.
```

--------------------------------

### Get Cluster Centroids

Source: https://docs.transluce.org/sdk/rubrics/clustering

Retrieves a list of centroid information dictionaries for a specified collection and rubric.

```APIDOC
## Get Cluster Centroids

### Description
Retrieves a list of centroid information dictionaries for a specified collection and rubric.

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **rubric_id** (str) - Required - ID of the rubric.

### Returns
#### Success Response (200)
- **centroids** (list[dict]) - List of centroid information dictionaries.
```

--------------------------------

### Get Transcript Group Metadata

Source: https://docs.transluce.org/sdk/agent-runs/metadata

Retrieves the metadata for a specific transcript group within a collection.

```APIDOC
## Get Transcript Group Metadata

### Description
Retrieves the metadata for a specific transcript group.

### Method
GET (Implied by client.get_transcript_group_metadata)

### Endpoint
/collections/{collection_id}/transcript-groups/{transcript_group_id}/metadata (Inferred)

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **transcript_group_id** (str) - Required - ID of the transcript group.

### Response
#### Success Response (200)
- **metadata** (dict) - The metadata associated with the transcript group.
```

--------------------------------

### Complete Search-and-Cluster Pipeline

Source: https://docs.transluce.org/analysis/analysis-plans

This snippet demonstrates a full pipeline involving querying, reading, and clustering data. It shows how to define steps, chain them using handles, and trigger execution.

```python
from docent import Docent

client = Docent()
collection_id = "<your-collection-id>"

# Step $1: a DQL step selecting inputs (no LLM involved)
sampled = client.query(
    collection_id,
    "SELECT transcripts.id AS transcript FROM transcripts ORDER BY transcripts.id LIMIT 100",
    name="Sample 100 transcripts",
)

# Step $2: a Reading step that runs once per row of $1
summarize = client.read(
    prompt_template=[
        sampled.transcript.as_type("transcript"),
        "Write a 1-2 sentence summary of any mistakes the agent made.",
    ],
    model="openai/gpt-5.4-mini",
    name="Summarize mistakes per transcript",
)

# Step $3: a DQL step that gathers all of $2's outputs into one row.
# The f-string interpolates the Reading handle as its alias ('$2');
# the server substitutes the real reading ID at execution time.
summaries = client.query(
    collection_id,
    f"""
    SELECT array_agg(rr.id ORDER BY rr.id) AS summaries
    FROM reading_results rr
    JOIN reading_result_links rrl ON rrl.result_id = rr.id
    WHERE rrl.reading_id = '{summarize}'
    """,
    name="Collect all summaries",
)

# Step $4: a Reading step that sees every summary at once
clusters = client.read(
    prompt_template=[
        "Cluster these mistake summaries into 5-10 categories: ",
        summaries.summaries.as_type("reading_result", is_list=True),
    ],
    model="openai/gpt-5.5",
    name="Cluster mistake summaries",
)

# Nothing has run yet. Accessing .results forces evaluation of $4 and
# everything upstream, blocks until complete, and returns the output.
print(clusters.results[0].output)
```

--------------------------------

### Get Tags

Source: https://docs.transluce.org/sdk/feedback/labels

Retrieves tags associated with a collection, optionally filtered by value or for a specific agent run.

```APIDOC
## Get Tags

### Description
Retrieves tags associated with a collection, optionally filtered by value or for a specific agent run.

### Method
```python
client.get_tags(collection_id, value=None)
client.get_tags_for_agent_run(collection_id, run_id)
```

### Parameters
- **collection_id** (string) - Required - The ID of the collection.
- **run_id** (string) - Required - The ID of the agent run (for `get_tags_for_agent_run`).
- **value** (string) - Optional - Filters tags by the specified value.
```

--------------------------------

### Get a Rubric

Source: https://docs.transluce.org/sdk/rubrics/manage

Retrieves a specific rubric by its collection and rubric ID. Optionally, a specific version can be requested.

```APIDOC
## Get a Rubric

### Description
Retrieves a specific rubric by its collection and rubric ID. Optionally, a specific version can be requested.

### Method
`client.get_rubric(collection_id: str, rubric_id: str, version: int | None = None)`

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **rubric_id** (str) - Required - ID of the rubric to retrieve.

#### Query Parameters
- **version** (int | None) - Optional - Specific version number. If `None`, returns the latest version.

### Returns
#### Success Response
- **rubric** (Rubric) - The rubric configuration object.
  - **rubric_text** (str)
  - **output_schema** (dict)
```

--------------------------------

### Instantiate Docent Client

Source: https://docs.transluce.org/ingestion/sdk

Instantiate a Docent client object using your API key. The API key can be loaded from environment variables. Server and web URLs can be adjusted for self-hosting.

```python
import os
from docent import Docent

client = Docent(
    api_key=os.getenv("DOCENT_API_KEY"),  # is default and can be omitted

    # Uncomment and adjust these if you're self-hosting
    # server_url="http://localhost:8889",
    # web_url="http://localhost:3001",
)
```

--------------------------------

### Get Labels

Source: https://docs.transluce.org/sdk/feedback/labels

Retrieves labels associated with a specific label set and collection, with an option to filter for valid labels.

```APIDOC
## Get Labels

### Description
Retrieves labels associated with a specific label set and collection, with an option to filter for valid labels.

### Method
GET (assumed based on retrieval action)

### Endpoint
`/collections/{collection_id}/label_sets/{label_set_id}/labels` (inferred)

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **label_set_id** (str) - Required - ID of the label set.

#### Query Parameters
- **filter_valid_labels** (bool) - Optional - If `True`, only return labels that fully match the label set schema including required fields. Default returns all labels.

### Response
#### Success Response (200)
- **labels** (list) - A list of label objects.

### Response Example
```json
[
  {
    "id": "label-123",
    "label_set_id": "label-set-id",
    "agent_run_id": "run-id-123",
    "label_value": {"quality": "good", "notes": "Clear and helpful response"}
  }
]
```
```

--------------------------------

### Configure Self-Hosted Docent Instance

Source: https://docs.transluce.org/sdk/authentication

For self-hosted instances, set both API and frontend URLs directly or via environment variables.

```python
client = Docent(
    api_key="your-api-key",
    api_url="https://api.my-docent.example.com",
    frontend_url="https://my-docent.example.com",
)
```

```bash
export DOCENT_API_URL="https://api.my-docent.example.com"
export DOCENT_FRONTEND_URL="https://my-docent.example.com"
```

--------------------------------

### Get Collection Metadata

Source: https://docs.transluce.org/sdk/collections/metadata

Fetches the metadata associated with a specific collection ID. Requires the Docent client to be initialized.

```python
from docent import Docent

client = Docent()

metadata = client.get_collection_metadata("my-collection-id")
print(metadata)  # {"team": "support", "model": "gpt-4"}
```

--------------------------------

### Get a General Label

Source: https://docs.transluce.org/sdk/feedback/labels

Fetches a specific general label by its ID within a given collection and prints its label value.

```python
general_label = client.get_general_label(
    "my-collection-id",
    label_id="label-123",
)
print(general_label.label_value)
```

--------------------------------

### Initialize Docent Tracing (Custom Configuration)

Source: https://docs.transluce.org/ingestion/tracing

Initialize Docent tracing with custom configuration, including a specific collection name, endpoint, and API key. The API key can be omitted to use the environment variable.

```python
# With custom configuration
initialize_tracing(
    collection_name="my-application",
    endpoint="https://docent.transluce.org/rest/telemetry",  # Optional, uses default if not provided
    api_key="your-api-key",  # Optional, uses env var if not provided
)
```

--------------------------------

### Get AgentRunTree from AgentRunView

Source: https://docs.transluce.org/concepts/agent-run

Provides a cached property to access the AgentRunTree associated with the AgentRunView. The tree is built on first access.

```python
@property
def tree(self) -> AgentRunTree:
    if self._cached_tree is None:
        self._cached_tree = AgentRunTree.from_agent_run(self.agent_run)
    return self._cached_tree
```

--------------------------------

### Initialize Docent Tracing (Existing Collection)

Source: https://docs.transluce.org/ingestion/tracing

Initialize Docent tracing by specifying an existing collection ID. This allows adding new agent runs to a pre-existing collection on the Docent dashboard.

```python
# Add new agent runs to an existing collection
initialize_tracing(
    collection_id="c49ef42c-7493-4af3-84d5-ac2b67556005", # Your collection's ID from the dashboard
)
```

--------------------------------

### Get a Rubric

Source: https://docs.transluce.org/sdk/rubrics/manage

Retrieves a specific rubric by its collection and rubric ID. Prints the rubric's text and output schema.

```python
rubric = client.get_rubric("my-collection-id", rubric_id)
print(rubric.rubric_text)
print(rubric.output_schema)
```

--------------------------------

### Get a Collection by ID

Source: https://docs.transluce.org/sdk/collections/manage

Fetches a collection using its ID. Handles HTTP errors, specifically checking for a 404 if the collection does not exist.

```python
import requests

try:
    collection = client.get_collection("my-collection-id")
    print(collection["name"])
except requests.exceptions.HTTPError as e:
    if e.response.status_code == 404:
        print("Collection not found")
    else:
        raise
```

--------------------------------

### Convert Harbor Directory to Agent Runs

Source: https://docs.transluce.org/ingestion/integrations/harbor

Recursively converts all Harbor trial directories found under a specified root directory into Docent AgentRuns. Use this when you have a directory structure containing multiple Harbor trials.

```python
from docent.sdk.integrations import convert_harbor_directory_to_agent_runs

agent_runs = convert_harbor_directory_to_agent_runs("/path/to/harbor/root")
```

--------------------------------

### Get Job Status

Source: https://docs.transluce.org/sdk/jobs

Fetches the status of a specific background ingestion job. This is useful for monitoring the progress of individual agent runs.

```APIDOC
## Get Agent Run Job Status

### Description
Fetches the status of a specific background ingestion job.

### Method
`client.get_agent_run_job_status(collection_id: str, job_id: str)`

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **job_id** (str) - Required - ID of the job to check.

### Returns
#### Success Response
- **status** (dict) - Job status information.
  - **job_id** (str) - The job ID.
  - **status** (str) - One of `"pending"`, `"running"`, `"cancelling"`, `"completed"`, `"canceled"`.
  - **type** (str) - The job type.
  - **created_at** (str) - ISO timestamp of job creation.
  - **error_message** (str | None) - Server-provided reason the job ended in `"canceled"`. `None` when the job succeeded or when no specific reason is available.

### Request Example
```python
status = client.get_agent_run_job_status("my-collection-id", "job-123")
print(f"Status: {status['status']}")
print(f"Type: {status['type']}")
```
```

--------------------------------

### Import NeMo Gym Conversion Helpers

Source: https://docs.transluce.org/ingestion/integrations/nemogym

Import the necessary functions for converting NeMo Gym rollouts into Docent AgentRuns. These helpers are essential for preparing your data for ingestion.

```python
from docent.sdk.integrations import (
    convert_nemogym_jsonl_file_to_agent_runs,
    convert_nemogym_rollout_to_agent_run,
)
```

--------------------------------

### Get Agent Run Metadata

Source: https://docs.transluce.org/sdk/agent-runs/metadata

Fetches the metadata associated with a specific agent run. Use this to retrieve existing metadata for inspection.

```python
from docent import Docent

client = Docent()

metadata = client.get_agent_run_metadata("my-collection-id", "run-id-123")
print(metadata)  # {"model": "gpt-4", "score": 0.95}
```

--------------------------------

### Get Cluster Assignments

Source: https://docs.transluce.org/sdk/rubrics/clustering

Retrieves a mapping of centroid IDs to lists of judge result IDs belonging to each cluster for a given collection and rubric.

```APIDOC
## Get Cluster Assignments

### Description
Retrieves a mapping of centroid IDs to lists of judge result IDs belonging to each cluster for a given collection and rubric.

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **rubric_id** (str) - Required - ID of the rubric.

### Returns
#### Success Response (200)
- **assignments** (dict[str, list[str]]) - Mapping of centroid IDs to lists of judge result IDs belonging to that cluster.
```

--------------------------------

### Convert and Upload Inspect Logs Separately

Source: https://docs.transluce.org/ingestion/integrations/inspect

Convert Inspect logs from a directory and then upload the resulting AgentRun objects using the Docent client. This provides more control over the upload process.

```python
from docent import Docent
from docent.sdk.integrations import convert_inspect_directory_to_agent_runs

client = Docent()
agent_runs = convert_inspect_directory_to_agent_runs("/path/to/inspect/logs")
client.add_agent_runs(collection_id, agent_runs)
```

--------------------------------

### Get Clustering State

Source: https://docs.transluce.org/sdk/rubrics/clustering

Fetches the current state of a clustering job for a given collection and rubric. This includes the job ID, centroids, and assignments.

```APIDOC
## Get Clustering State

### Description
Fetches the current state of a clustering job for a given collection and rubric.

### Parameters
#### Path Parameters
- **collection_id** (str) - Required - ID of the collection.
- **rubric_id** (str) - Required - ID of the rubric.

### Returns
#### Success Response (200)
- **state** (dict) - Clustering state.
  - **job_id** (str | None) - Clustering job ID.
  - **centroids** (list[dict]) - List of cluster centroids.
  - **assignments** (dict) - Mapping of centroid IDs to judge result IDs.
```

--------------------------------

### Get Agent Run Job Status

Source: https://docs.transluce.org/sdk/jobs

Retrieve the status of a specific agent run job. Requires the collection ID and the job ID.

```python
from docent import Docent

client = Docent()

status = client.get_agent_run_job_status("my-collection-id", "job-123")
print(f"Status: {status['status']}")
print(f"Type: {status['type']}")
```

--------------------------------

### Creating an AgentRun Object

Source: https://docs.transluce.org/concepts/agent-run

Instantiate an AgentRun with a list of Transcript objects and a metadata dictionary. The metadata should be JSON-serializable.

```python
from docent.data_models import AgentRun, Transcript
from docent.data_models.chat import UserMessage, AssistantMessage

transcripts = [
    Transcript(
        messages=[
            UserMessage(content="Hello, what's 1 + 1?"),
            AssistantMessage(content="2"),
        ]
    )
]

agent_run = AgentRun(
    transcripts=transcripts,
    metadata={
        "scores": {"correct": True, "reward": 1.0},
    }
)
```

--------------------------------

### Get Cluster Centroids

Source: https://docs.transluce.org/sdk/rubrics/clustering

Fetches the list of centroid information dictionaries for a specified collection and rubric. Each centroid represents a cluster of similar judge results.

```python
centroids = client.get_cluster_centroids("my-collection-id", "rubric-123")
for centroid in centroids:
    print(centroid)
```

--------------------------------

### Add Converted Agent Runs to Docent

Source: https://docs.transluce.org/ingestion/integrations/harbor

Demonstrates adding converted AgentRuns, obtained from a Harbor directory, to a Docent client. This follows the conversion process and prepares the data for upload.

```python
from docent import Docent
from docent.sdk.integrations import convert_harbor_directory_to_agent_runs

client = Docent()
agent_runs = convert_harbor_directory_to_agent_runs("/path/to/harbor/root")
client.add_agent_runs(collection_id, agent_runs)
```

--------------------------------

### Get Transcript Group Metadata

Source: https://docs.transluce.org/sdk/agent-runs/metadata

Retrieves metadata for a specific transcript group within a collection. Requires collection ID and transcript group ID.

```python
metadata = client.get_transcript_group_metadata("my-collection-id", "group-id-456")
```

--------------------------------

### Create a Collection

Source: https://docs.transluce.org/ingestion/sdk

Create a new collection in Docent with a specified name and description. This collection will store the ingested data.

```python
collection_id = client.create_collection(
    name="sample collection",
    description="example that comes with the Docent repo",
)
```

--------------------------------

### Initialize and Build AgentRun Tree

Source: https://docs.transluce.org/concepts/agent-run

Initializes the AgentRun tree by adding the root node, then populating it with transcript groups and transcripts. It sorts children by creation time and performs a DFS to mark nodes with transcripts in their subtree.

```python
          # Init tree and add the root Agent Run node
          nodes: dict[str, AgentRunTreeNode] = {
              GLOBAL_ROOT_ID: AgentRunTreeNode(
                  id=GLOBAL_ROOT_ID,
                  node_type=NodeType.AGENT_RUN,
                  children_ids=[],
              )
          }
          parent_map: dict[str, str] = {}

          # Add all transcript groups to the tree
          for tg_id, tg in tg_dict.items():
              # Add this tg
              if tg_id not in nodes:
                  nodes[tg_id] = AgentRunTreeNode(
                      id=tg_id,
                      node_type=NodeType.TRANSCRIPT_GROUP,
                      children_ids=[],
                  )
              # Add parent and mark the relationship
              # If the stated ID is None, then it's the global root
              par_id = tg.parent_transcript_group_id or GLOBAL_ROOT_ID
              if par_id not in nodes:
                  nodes[par_id] = AgentRunTreeNode(
                      id=par_id,
                      node_type=(
                          NodeType.AGENT_RUN
                          if par_id == GLOBAL_ROOT_ID
                          else NodeType.TRANSCRIPT_GROUP
                      ),
                      children_ids=[],
                  )
              nodes[par_id].children_ids.append(tg_id)
              parent_map[tg_id] = par_id

          # Now add all the transcripts
          for t_id, t in t_dict.items():
              # Add this transcript
              nodes[t_id] = AgentRunTreeNode(
                  id=t_id,
                  node_type=NodeType.TRANSCRIPT,
                  children_ids=[],
              )
              # Mark parent relationship
              par_id = t.transcript_group_id or GLOBAL_ROOT_ID
              # This should never happen, but check anyways for safety; fallback to global root
              if par_id not in nodes:
                  logger.error(
                      f"Parent {par_id} not found for transcript {t_id}. Assigning to global root as a fallback"
                  )
                  par_id = GLOBAL_ROOT_ID
              nodes[par_id].children_ids.append(t_id)
              parent_map[t_id] = par_id

          # Go through each node and sort its children by created_at timestamp
          def _cmp(obj_id: str) -> datetime:
              obj_type = nodes[obj_id].node_type
              if obj_type == NodeType.TRANSCRIPT_GROUP:
                  # This should never happen, but check anyways for safety
                  if obj_id not in tg_dict:
                      logger.error(f"Transcript group {obj_id} not found")
                      return datetime.max
                  return tg_dict[obj_id].created_at or datetime.max
              elif obj_type == NodeType.TRANSCRIPT:
                  # This should never happen, but check anyways for safety
                  if obj_id not in t_dict:
                      logger.error(f"Transcript {obj_id} not found")
                      return datetime.max
                  return t_dict[obj_id].created_at or datetime.max
              else:
                  raise ValueError(f"Unknown node type: {obj_type}")

          for node in nodes.values():
              node.children_ids = sorted(node.children_ids, key=_cmp)

          # Combined DFS: mark has_transcript_in_subtree and assign transcript indices
          t_id_to_idx: dict[str, int] = {}

          def _dfs(u_id: str, next_idx: int) -> tuple[bool, int]:
              """Mark has_transcript_in_subtree and assign indices in a single traversal.

              Returns (contains_transcript, next_idx_after).
              """
              node = nodes.get(u_id)
              if node is None:
                  return False, next_idx

              if node.node_type == NodeType.TRANSCRIPT:
                  # Leaf node: assign index immediately (pre-order)
                  t_id_to_idx[u_id] = next_idx
                  node.has_transcript_in_subtree = True
                  return True, next_idx + 1

              # Non-transcript node: recurse into children
              contains_transcript = False
              for child_id in node.children_ids:
                  child_contains, next_idx = _dfs(child_id, next_idx)
                  contains_transcript = contains_transcript or child_contains

              node.has_transcript_in_subtree = contains_transcript
              return contains_transcript, next_idx

          _dfs(GLOBAL_ROOT_ID, 0)

          return cls(nodes=nodes, transcript_id_to_idx=t_id_to_idx, parent_map=parent_map)
```

--------------------------------

### List Collections

Source: https://docs.transluce.org/sdk/collections/manage

Retrieve a list of all collections. The output includes the ID, name, and the count of agent runs for each collection. This is useful for getting an overview of your existing collections.

```python
collections = client.list_collections()
for c in collections:
    print(f"{c['id']}: {c['name']} ({c['counts']['agent_run_count']} runs)")
```

--------------------------------

### Compare Model Performance: GPT-5.1 vs GPT-5 Codex

Source: https://docs.transluce.org/analysis/quickstart

This prompt is used to compare the performance of two models (GPT-5.1 Codex vs GPT-5 Codex) by identifying tasks where GPT-5.1 regresses. It involves comparing failed runs of GPT-5.1 against successful runs of GPT-5, analyzing failure modes, and assessing their impact on the overall results. Auto-approval for the analysis plan is recommended.

```text
/docent What are the main reasons why GPT-5.1 Codex underperforms GPT-5 Codex?

Identify tasks where GPT-5.1 regresses on average. On those tasks, compare a failed GPT-5.1 run against the successful GPT-5 runs. Summarize the main failure modes and analyze whether avoiding those failures was material to the result of the successful runs.

- Collection ID: 479b7093-5a33-47f1-8d7b-fc9f6f16bb75
- Auto accept analysis plan
```

--------------------------------

### Load and Print Agent Runs

Source: https://docs.transluce.org/ingestion/sdk

Loads agent runs from a log and prints the text of the first run.

```python
agent_runs = load_inspect_log(ctf_log)
print(agent_runs[0].text)
```
Tokens:
Raw