### Use Custom Scorers

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Instantiate and use custom scorers for evaluating agent trajectories. This example demonstrates using both `CustomEfficiencyScorer` and `CustomLLMScorer`.

```python
# Use custom scorers
efficiency_scorer = CustomEfficiencyScorer(weight=0.5)
custom_llm_scorer = CustomLLMScorer(weight=1.5, model='gpt-4o')

result = efficiency_scorer(trajectory)
print(f"Efficiency: {result.score:.2f}")
```

--------------------------------

### Get Embedding Store Statistics

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Retrieve statistics from the created embedding store, including total and unique verbs, total pairs, and embedding dimension. Requires an initialized VerbNounEmbeddingStore object.

```python
# Get statistics
stats = store.get_statistics()
print(f"Total verbs: {stats['total_verbs']}")
print(f"Unique verbs: {stats['unique_verbs']}")
print(f"Total pairs: {stats['total_pairs']}")
print(f"Embedding dimension: {stats['embedding_dimension']}")
```

--------------------------------

### Quick Launch Web Dashboard

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Use the `create_web_dashboard` helper function for a quick launch with common configurations. Alternatively, instantiate the `WebDashboard` class directly for finer control over server settings and token management.

```python
dashboard = create_web_dashboard(
    results=results,
    trajectories=trajectories,
    port=5000,
    host='127.0.0.1',
    open_browser=True,
    enable_tunnel=True
)
```

```python
dashboard = WebDashboard(
    port=8080,
    host='0.0.0.0',
    enable_tunnel=True,
    tunnel_token='your-cloudflare-token'  # optional
)

dashboard.load_results(results, trajectories)
dashboard.start_server(
    open_browser=True,
    debug=False,
    production=True
)
```

--------------------------------

### CLI: Launch Web Dashboard

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Launch a web dashboard for visualizing evaluation results. Specify host and port for the dashboard server.

```bash
python evaluate_trajectories.py \
    --input examples/sample_trajectories \
    --scorers reasoning_quality objective_quality \
    --output-json results.json \
    --web-dashboard \
    --dashboard-host 0.0.0.0 \
    --dashboard-port 8080
```

--------------------------------

### Generate Reasoning Tag Clouds via CLI

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Command-line interface for generating reasoning tag clouds. Supports generating all n-grams or a specific range, with options for input/output directories and frequency thresholds.

```bash
# Generate all n-gram tag clouds for reasoning
python generate_tag_cloud.py \
    --input-dir examples/labeled_trajectories \
    --output-dir tag_cloud \
    --tag-source reasoning \
    --all-ngrams
```

```bash
# Generate specific n-gram range
python generate_tag_cloud.py \
    --input-dir examples/labeled_trajectories \
    --output-dir tag_cloud \
    --tag-source reasoning \
    --ngram-range 2-3 \
    --min-frequency 3 \
    --max-tags 100
```

--------------------------------

### Load and Use Existing Embedding Store

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Load a previously saved embedding store from disk using VerbNounEmbeddingStore and perform similarity searches.

```python
# Load existing embedding store
loaded_store = VerbNounEmbeddingStore()
loaded_store.load_from_disk('embeddings_output')

# Use loaded store for similarity search
results = loaded_store.search_similar_pairs('navigate page', k=10)
```

--------------------------------

### CLI: Dry Run for Token Estimation

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Perform a dry run to estimate token usage and associated costs without making actual API calls. Useful for cost planning.

```bash
python evaluate_trajectories.py \
    --input examples/sample_trajectories \
    --scorers reasoning_quality objective_quality \
    --output-json results.json \
    --dry-run
```

--------------------------------

### Generate Action Phrases Tag Clouds via CLI

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Command-line interface for generating action phrases tag clouds. Specify the tag source and n-gram options.

```bash
# Generate action phrases tag clouds (verbs, nouns, pairs)
python generate_tag_cloud.py \
    --input-dir examples/labeled_trajectories \
    --output-dir tag_cloud \
    --tag-source action_phrases \
    --all-ngrams
```

--------------------------------

### Generate Reasoning Tag Cloud

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Create a tag cloud from trajectories with specified n-gram ranges and frequency thresholds. Access and print the top tags and their associated data.

```python
tag_data = create_reasoning_tag_cloud(
    trajectories=trajectories,
    ngram_range=(1, 4),      # unigrams through 4-grams
    min_frequency=2,          # minimum occurrence threshold
    max_frequency_rate=0.8,   # exclude terms in >80% of docs
    max_tags=200
)

# Access tag data
for tag in tag_data['tags'][:10]:
    print(f"{tag['text']}: weight={tag['weight']:.4f}, freq={tag['raw_frequency']}")
    print(f"  Used in trajectories: {tag['trajectories_using']}")
```

--------------------------------

### Evaluate Trajectories with Scorers in Python

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Shows how to use pluggable scorers to assess agent trajectories. Includes listing available scorers, creating scorers via a factory function or direct instantiation, and accessing score results and details. Requires setting an LLM API key for LLM-based scorers.

```python
from evaluator.scorers import create_scorer, get_available_scorers
from evaluator.scorers.reasoning_quality import ReasoningQualityScorer
from evaluator.scorers.objective_quality import ObjectiveQualityScorer
from evaluator.scorers.navigation_path_scorer import NavigationPathScorer
from evaluator.trajectory import Trajectory
import os

# Set API key for LLM-based scorers
os.environ['LLM_API_KEY'] = 'your-api-key-here'

# List available scorers
print(get_available_scorers())
# Output: ['navigation_path', 'objective_quality', 'reasoning_quality']

# Create scorers using factory function
reasoning_scorer = create_scorer('reasoning_quality', model='gemini-2.5-pro-exp-03-25')
objective_scorer = create_scorer('objective_quality', model='gemini-2.0-flash')
nav_scorer = create_scorer('navigation_path')

# Or instantiate directly with custom parameters
reasoning_scorer = ReasoningQualityScorer(
    model='gemini-1.5-pro',
    temperature=0.0,
    weight=1.0
)

# Load and score a trajectory
trajectory = Trajectory.from_json(json.load(open('trajectory.json')))

# Get reasoning quality score
result = reasoning_scorer(trajectory)
print(f"Score: {result.score:.2f}")
print(f"Confidence: {result.confidence:.2f}")
print(f"Details: {result.details}")
# Details include: strategic_backtracking, task_decomposition,
#                  observation_reading, self_verification (each 0-1 scale)

# Get navigation path metrics (no LLM required)
nav_result = nav_scorer(trajectory)
print(f"Path length: {nav_result.details['overall_length']}")
print(f"Backtrack count: {nav_result.details['backtrack_count']}")
print(f"Unique URLs: {nav_result.details['unique_urls_visited']}")
print(f"Domain transitions: {nav_result.details['domain_transitions']}")
```

--------------------------------

### Configure LLM API Key

Source: https://github.com/oootttyyy/agentdiagnose/blob/main/README.md

Set the LLM_API_KEY environment variable to authenticate with the LLM service. This is required for the toolkit to function.

```bash
export LLM_API_KEY="your-api-key-here"
```

--------------------------------

### Load and Parse Trajectories in Python

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Demonstrates loading agent trajectories from JSON files, BrowserGym pickles, or CUGA records. Accesses trajectory properties like objective, actions, and reward. Exports trajectories back to JSON.

```python
import json
from evaluator.trajectory import Trajectory, Action

# Load trajectory from JSON file
with open('trajectory_0001.json', 'r', encoding='utf-8') as f:
    data = json.load(f)

trajectory = Trajectory.from_json(data)

# Access trajectory properties
print(f"Objective: {trajectory.objective}")
print(f"Website: {trajectory.task_website}")
print(f"Total actions: {len(trajectory.actions)}")
print(f"Reward: {trajectory.reward}")

# Iterate through actions
for action in trajectory.actions:
    print(f"Step {action.nth_step}: {action.action_type}")
    print(f"  Action: {action.action}")
    print(f"  Reasoning: {action.reasoning[:100]}..." if action.reasoning else "  No reasoning")
    print(f"  URL: {action.url}")

# Create trajectory from BrowserGym experiment
trajectory = Trajectory.from_browsergym_pickle('/path/to/experiment_dir')

# Create trajectory from CUGA record
trajectory = Trajectory.from_cuga_record('/path/to/cuga_record.json')

# Export trajectory back to JSON
trajectory_json = trajectory.to_json()
with open('output_trajectory.json', 'w') as f:
    json.dump(trajectory_json, f, indent=2)
```

--------------------------------

### All-In-One Pipeline Script

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

This bash script automates the complete analysis pipeline, including data extraction, tag cloud generation, embedding creation, trajectory evaluation, and launching the web dashboard. Ensure all Python scripts are executable and paths are correctly configured.

```bash
#!/bin/bash
# launch_dashboard.sh - Complete pipeline execution

# Configuration
INPUT_DIR="examples/sample_trajectories"
LABELED_DIR="examples/sample_trajectories_labeled"
TAG_CLOUD_DIR="tag_cloud"
EMBEDDINGS_DIR="embeddings_output"
RESULTS_FILE="examples/results.json"

# Step 1: Extract verb-noun pairs
python generate_verb_nouns.py \
    --input-dir "$INPUT_DIR" \
    --output-dir "$LABELED_DIR"

# Step 2: Generate reasoning tag clouds (all n-gram types)
python generate_tag_cloud.py \
    --input-dir "$LABELED_DIR" \
    --output-dir "$TAG_CLOUD_DIR" \
    --tag-source reasoning \
    --all-ngrams

# Step 3: Generate action phrases tag clouds
python generate_tag_cloud.py \
    --input-dir "$LABELED_DIR" \
    --output-dir "$TAG_CLOUD_DIR" \
    --tag-source action_phrases \
    --all-ngrams

# Step 4: Generate embeddings for similarity search
python generate_embeddings.py \
    --input-dir "$LABELED_DIR" \
    --output-dir "$EMBEDDINGS_DIR"

# Step 5: Evaluate trajectories and launch dashboard
python evaluate_trajectories.py \
    --input "$INPUT_DIR" \
    --scorers reasoning_quality objective_quality \
    --output-json "$RESULTS_FILE" \
    --web-dashboard \
    --dashboard-host 0.0.0.0 \
    --dashboard-port 8080

# Or simply run:
./launch_dashboard.sh
```

--------------------------------

### CLI: Basic Evaluation

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Perform basic trajectory evaluation using the command-line interface. Specify input trajectory directory and output JSON file.

```bash
python evaluate_trajectories.py \
    --input examples/sample_trajectories \
    --output-json results.json
```

--------------------------------

### CLI: Web Dashboard with Cloudflare Tunnel

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Enable remote access to the web dashboard using a Cloudflare tunnel. Requires a Cloudflare tunnel token.

```bash
python evaluate_trajectories.py \
    --input examples/sample_trajectories \
    --web-dashboard \
    --enable-tunnel \
    --tunnel-token "your-cloudflare-token"
```

--------------------------------

### Load Judge from Configuration

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Instantiate a TrajectoryJudge from a JSON configuration file. The configuration specifies the judge's name, aggregation method, and scorers.

```python
judge = TrajectoryJudge.from_config('judge_config.json')
```

--------------------------------

### Launch AgentDiagnose Dashboard

Source: https://github.com/oootttyyy/agentdiagnose/blob/main/README.md

Execute this script to run the complete AgentDiagnose pipeline, including data extraction, analysis, and launching the interactive web dashboard.

```bash
./launch_dashboard.sh
```

--------------------------------

### Load Labeled Trajectories for Tag Cloud Generation

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Load trajectories that have already been processed for verb-noun pairs from a directory. This is a prerequisite for generating tag clouds.

```python
from generate_tag_cloud import (
    load_trajectories_from_directory,
    generate_reasoning_tag_cloud,
    generate_action_phrases_tag_cloud
)
from visualization.reasoning_tag_cloud import create_reasoning_tag_cloud
from visualization.action_phrases_tag_cloud import create_action_phrases_tag_cloud

trajectories = load_trajectories_from_directory('examples/labeled_trajectories')
```

--------------------------------

### CLI: Evaluate with Specific Scorers

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Evaluate trajectories using a specified subset of scorers. Requires input directory and output JSON file.

```bash
python evaluate_trajectories.py \
    --input examples/sample_trajectories \
    --scorers reasoning_quality objective_quality navigation_path \
    --output-json results.json
```

--------------------------------

### CLI: Batch Verb-Noun Pair Extraction

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Process multiple trajectories from an input directory, label verb-noun pairs, and save them to an output directory. Includes options to print statistics.

```bash
python generate_verb_nouns.py \
    --input-dir examples/sample_trajectories \
    --output-dir examples/labeled_trajectories \
    --print-stats
```

--------------------------------

### Generate Action Phrases Tag Cloud

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Generate a tag cloud for action phrases (verbs, nouns, or pairs) from trajectories. This is useful for understanding common actions within the data.

```python
# Generate action phrases tag cloud (verbs, nouns, or pairs)
action_tag_data = create_action_phrases_tag_cloud(
    trajectories=trajectories,
    phrase_type='pairs',      # 'verbs', 'nouns', or 'pairs'
    min_frequency=2,
    max_frequency_rate=0.8,
    max_tags=200
)
```

--------------------------------

### Generate Embeddings via CLI

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Command-line interface for generating embeddings. Specify input and output directories. The output includes pickle files for metadata and numpy files for vectors, along with FAISS indices.

```bash
# Generate embeddings from labeled trajectories
python generate_embeddings.py \
    --input-dir examples/labeled_trajectories \
    --output-dir embeddings_output
```

--------------------------------

### Create Embedding Store for Similarity Search

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Generate vector embeddings for verb-noun pairs using sentence transformers and build FAISS indices. Configure model, device, and index type. Supports various kwargs for model and tokenizer.

```python
from generate_embeddings import VerbNounEmbeddingStore, create_embedding_store

# Create embedding store from trajectory directory
store = create_embedding_store(
    data_dir='examples/labeled_trajectories',
    output_dir='embeddings_output',
    model_name='Qwen/Qwen3-Embedding-0.6B',
    faiss_index_type='flat',  # or 'ivf' for large datasets
    model_kwargs={'device_map': 'auto'},
    tokenizer_kwargs={'padding_side': 'left'},
    prompt_name='query',
    print_stats=True
)
```

--------------------------------

### Label Verb-Noun Pairs in Trajectory

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Load a trajectory and label verb-noun pairs within its actions using NLP parsing. Prints statistics and allows access to extracted pairs.

```python
from evaluator.trajectory import Trajectory, label_verb_noun
import json

with open('trajectory.json', 'r') as f:
    trajectory = Trajectory.from_json(json.load(f))

trajectory.label_verb_noun_pairs(print_stats=True)

for action in trajectory.actions:
    print(f"Step {action.nth_step}:")
    print(f"  Root verb: {action.output_root_verb}")
    print(f"  Root noun: {action.output_root_noun}")
    print(f"  All pairs: {action.output_verb_noun_pairs}")
```

--------------------------------

### Prepare Evaluation Results for Web Dashboard

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Structure evaluation results, including reasoning and objective quality scores, into a dictionary format suitable for the web dashboard. Requires importing ScorerResult.

```python
from dashboard.web_dashboard import WebDashboard, create_web_dashboard
from evaluator.scorers.base import ScorerResult

# Prepare evaluation results
results = {
    '0001': {
        'reasoning_quality': ScorerResult(
            score=0.85, name='ReasoningQuality',
            description='Reasoning quality assessment',
            details={'task_decomposition': 0.9, 'observation_reading': 0.8},
            confidence=0.9, weight=1.0
        ),
        'objective_quality': ScorerResult(
            score=0.75, name='ObjectiveQuality',
            description='Objective quality assessment',
            details={'actionability': 0.75},
            confidence=0.9, weight=1.0
        )
    }
}

trajectories = {'0001': trajectory}
```

--------------------------------

### Batch Evaluate Trajectories

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Evaluate multiple trajectories loaded from JSON files in a batch. Results are saved to a specified output directory.

```python
trajectories = [
    Trajectory.from_json(json.load(open(f'trajectory_{i:04d}.json')))
    for i in range(10)
]

results = judge.evaluate_batch(
    trajectories=trajectories,
    output_dir='./evaluation_results',
    metadata={'batch': 'experiment_1'}
)
```

--------------------------------

### Search Similar Verbs, Nouns, and Pairs

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Perform similarity searches within the embedding store for verbs, nouns, or verb-noun pairs. Specify the query term and the number of results (k). Results include the term, score, and metadata.

```python
# Search for similar verbs
results = store.search_similar_verbs('click', k=5)
for verb, score, metadata in results:
    print(f"  {verb} (score: {score:.3f}) - {metadata['trajectory_file']}")

# Search for similar nouns
results = store.search_similar_nouns('button', k=5)
for noun, score, metadata in results:
    print(f"  {noun} (score: {score:.3f})")

# Search for similar verb-noun pairs
results = store.search_similar_pairs('submit form', k=5)
for pair, score, metadata in results:
    print(f"  {pair} (score: {score:.3f})")
```

--------------------------------

### Export Tag Data for Visualization

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Export generated tag data into formats suitable for D3.js and WordCloud2 visualizations. Requires importing the ReasoningTagCloudGenerator.

```python
# Export for D3.js visualization
from visualization.reasoning_tag_cloud import ReasoningTagCloudGenerator
generator = ReasoningTagCloudGenerator()
d3_json = generator.export_tag_data(tag_data, format_type='d3')
wordcloud2_json = generator.export_tag_data(tag_data, format_type='wordcloud2')
```

--------------------------------

### Batch Evaluate Trajectories with TrajectoryJudge in Python

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Utilizes the `TrajectoryJudge` class for high-level batch evaluation of multiple trajectories. Configures scorers and specifies an aggregation method for combining scores.

```python
from evaluator.evaluator import TrajectoryJudge, EvaluationResult
from evaluator.scorers import create_scorer
from evaluator.trajectory import Trajectory
import json

# Create scorers
scorers = [
    create_scorer('reasoning_quality', model='gemini-1.5-pro'),
    create_scorer('objective_quality', model='gemini-2.0-flash'),
    create_scorer('navigation_path')
]

# Initialize judge with aggregation method
judge = TrajectoryJudge(
    scorers=scorers,
    name="AgentEvaluator",
    aggregation_method="weighted_average"  # or "min", "max"
)

```

--------------------------------

### CLI: Limit Trajectories for Testing

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Limit the number of trajectories processed during evaluation, useful for testing or quick checks.

```bash
python evaluate_trajectories.py \
    --input examples/sample_trajectories \
    --scorers reasoning_quality \
    --output-json results.json \
    --limit 5
```

--------------------------------

### CLI: Parallel Evaluation

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Execute trajectory evaluation in parallel using a specified number of worker processes to speed up processing.

```bash
python evaluate_trajectories.py \
    --input examples/sample_trajectories \
    --scorers reasoning_quality \
    --output-json results.json \
    --max-workers 8
```

--------------------------------

### Implement Custom Non-LLM Scorer

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Extend `BaseScorer` to create a custom scorer that measures trajectory efficiency based on action count and unique URLs visited. This scorer does not require an LLM.

```python
from evaluator.scorers.base import BaseScorer, LLMScorer, ScorerResult
from evaluator.trajectory import Trajectory
from typing import List, Dict
import os
from litellm import completion

class CustomEfficiencyScorer(BaseScorer):
    """Non-LLM scorer that measures trajectory efficiency."""

    def __init__(self, weight: float = 1.0):
        super().__init__(
            weight=weight,
            name="EfficiencyScore",
            description="Measures trajectory efficiency based on action count and backtracking"
        )

    def score(self, trajectory: Trajectory) -> ScorerResult:
        total_actions = len(trajectory.actions)
        unique_urls = len(set(a.url for a in trajectory.actions if a.url))

        # Calculate efficiency (fewer actions for more unique pages = better)
        if total_actions > 0:
            efficiency = min(1.0, unique_urls / total_actions)
        else:
            efficiency = 0.0

        return ScorerResult(
            score=efficiency,
            name=self.name,
            description=self.description,
            details={
                'total_actions': total_actions,
                'unique_urls': unique_urls,
                'efficiency_ratio': efficiency
            },
            confidence=1.0,
            weight=self.weight
        )
```

--------------------------------

### Implement Custom LLM Scorer

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Extend `LLMScorer` to create a custom scorer that uses an LLM for evaluation. This involves defining prompt generation and response parsing logic. Ensure the `LLM_API_KEY` environment variable is set.

```python
from evaluator.scorers.base import BaseScorer, LLMScorer, ScorerResult
from evaluator.trajectory import Trajectory
from typing import List, Dict
import os
from litellm import completion

class CustomLLMScorer(LLMScorer):
    """LLM-based scorer for custom evaluation criteria."""

    def __init__(self, weight: float = 1.0, model: str = "gpt-4o"):
        super().__init__(
            weight=weight,
            name="CustomLLMScore",
            description="Custom LLM-based evaluation",
            model=model,
            temperature=0.0
        )

    def generate_prompt(self, trajectory: Trajectory) -> List[Dict[str, str]]:
        steps_text = "\n".join([
            f"Step {a.nth_step}: {a.action} - {a.reasoning[:100] if a.reasoning else 'No reasoning'}"
            for a in trajectory.actions
        ])

        return [
            {"role": "system", "content": "Evaluate agent performance on a scale of 1-4."},
            {"role": "user", "content": f"Objective: {trajectory.objective}\n\nSteps:\n{steps_text}"}
        ]

    def parse_response(self, response: str) -> ScorerResult:
        # Parse LLM response and extract score
        import re
        score_match = re.search(r'(\d)', response)
        score = int(score_match.group(1)) / 4.0 if score_match else 0.5

        return ScorerResult(
            score=score,
            name=self.name,
            description=self.description,
            details={'raw_response': response},
            confidence=0.8,
            weight=self.weight
        )

    def score(self, trajectory: Trajectory) -> ScorerResult:
        prompt = self.generate_prompt(trajectory)

        response = completion(
            api_key=os.getenv('LLM_API_KEY'),
            model=self.model,
            messages=prompt
        )

        return self.parse_response(response.choices[0].message.content)
```

--------------------------------

### Evaluate Single Trajectory

Source: https://context7.com/oootttyyy/agentdiagnose/llms.txt

Load a trajectory from a JSON file and evaluate it using a judge. The result summary can be printed and exported to JSON.

```python
trajectory = Trajectory.from_json(json.load(open('trajectory.json')))
result = judge.evaluate(trajectory, metadata={'source': 'webarena'})
result.print_summary()
result.to_json('evaluation_result.json')
```

--------------------------------

### Run AgentDiagnose Evaluator with Custom Options

Source: https://github.com/oootttyyy/agentdiagnose/blob/main/README.md

Use the evaluator script for fine-grained control over trajectory evaluation. Specify input data, desired scorers, output file, and optional flags like --dry-run for cost estimation.

```python
python evaluate_trajectories.py --input examples/sample_trajectories --scorers reasoning_quality objective_quality --output-json results.json
```

```python
python evaluate_trajectories.py --input examples/sample_trajectories --scorers reasoning_quality objective_quality --output-json results.json --dry-run
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.