GEPA

https://gepa-ai.github.io/gepa/guides/
Admin
GEPA is a text evolution engine that optimizes prompts, code, agent architectures, and...
Tokens:34,424
Snippets:415
Trust Score:8.3
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install and Setup uv

Source: https://gepa-ai.github.io/gepa/guides/contributing

Installs uv, clones the GEPA repository, sets up the development environment with uv, and verifies the installation by running tests.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh

git clone https://github.com/gepa-ai/gepa
cd gepa
uv sync --extra dev --python 3.11

uv run pytest tests/
```

--------------------------------

### Build Documentation Locally

Source: https://gepa-ai.github.io/gepa/guides/contributing

Navigates to the docs directory, installs dependencies, generates API documentation, and starts a local development server for viewing the documentation.

```bash
cd docs
pip install -r requirements.txt
python scripts/generate_api_docs.py
mkdocs serve
```

--------------------------------

### Setup with conda and pip

Source: https://gepa-ai.github.io/gepa/guides/contributing

Creates a conda environment for GEPA development, activates it, installs GEPA with development dependencies, and verifies the installation by running tests.

```bash
conda create -n gepa-dev python=3.11
conda activate gepa-dev
pip install -e ".[dev]"

pytest tests/
```

--------------------------------

### Quick Start: Implementing a Custom GEPA Callback

Source: https://gepa-ai.github.io/gepa/guides/callbacks

This example demonstrates how to define a custom callback class with methods for different optimization events and how to pass it to the `gepa.optimize` function.

```python
import gepa

class MyCallback:
    def on_optimization_start(self, event):
        print(f"Starting: {event['trainset_size']} train / {event['valset_size']} val examples")

    def on_iteration_end(self, event):
        status = "✓" if event["proposal_accepted"] else "✗"
        print(f"  {status} iteration {event['iteration']}")

    def on_optimization_end(self, event):
        print(f"Done — {event['total_iterations']} iterations, "
              f"{event['total_metric_calls']} metric calls")

result = gepa.optimize(
    seed_candidate={"system_prompt": "You are a helpful assistant."},
    trainset=trainset,
    valset=valset,
    adapter=adapter,
    reflection_lm="openai/gpt-5",
    max_metric_calls=200,
    callbacks=[MyCallback()],
)


```

--------------------------------

### Full Example: Optimize Q&A Prompt with claude -p

Source: https://gepa-ai.github.io/gepa/guides/claude-cli-as-proposer

An end-to-end script demonstrating optimization using `claude -p` for both task and reflection LMs. Includes setup for `claude_cli_lm`, `claude_cli_chat`, a sample dataset, and `gepa.optimize` call.

```python
import subprocess
import gepa


def claude_cli_lm(prompt: str) -> str:
    result = subprocess.run(
        ["claude", "-p", prompt],
        capture_output=True, text=True, timeout=120,
    )
    if result.returncode != 0:
        raise RuntimeError(f"claude -p failed: {result.stderr}")
    return result.stdout


def claude_cli_chat(messages: list) -> str:
    parts = []
    for m in messages:
        if m["role"] == "system":
            parts.append(f"[System]\n{m['content']}")
        elif m["role"] == "user":
            parts.append(f"[User]\n{m['content']}")
    return claude_cli_lm("\n\n".join(parts))


dataset = [
    {"input": "What is 2+2?", "answer": "4", "additional_context": {}},
    {"input": "What is the capital of France?", "answer": "Paris", "additional_context": {}},
    {"input": "What color is the sky on a clear day?", "answer": "blue", "additional_context": {}},
]

result = gepa.optimize(
    seed_candidate={"instructions": "Answer the question correctly."},
    trainset=dataset,
    task_lm=claude_cli_chat,
    reflection_lm=claude_cli_lm,
    max_metric_calls=20,
    reflection_minibatch_size=1,
    display_progress_bar=True,
)

print(f"Best instructions: {result.best_candidate}")
print(f"Best score: {result.val_aggregate_scores[result.best_idx]:.3f}")

```

--------------------------------

### Install Confidence Adapter Extra

Source: https://gepa-ai.github.io/gepa/guides/confidence-adapter

Install the ConfidenceAdapter by including the 'confidence' extra when installing gepa. This is required to use the confidence-aware features.

```bash
pip install "gepa[confidence]"

```

--------------------------------

### Complete Fitness Function Example

Source: https://gepa-ai.github.io/gepa/guides/gskill

A simplified GEPA fitness function demonstrating setup, agent execution, patch verification, and side info generation. It utilizes a pool of Docker harnesses for concurrent evaluations.

```python
from gepa.optimize_anything import optimize_anything, GEPAConfig, EngineConfig

def create_fitness_fn(model_name: str, n_workers: int):
    """Create a fitness function with a pool of Docker harnesses."""
    harness_pool = [SWEHarness() for _ in range(n_workers)]
    # ... pool management code ...

    def fitness_fn(candidate: dict[str, str], example: dict[str, Any]) -> tuple[float, dict]:
        skills = candidate["skills"]
        harness = get_available_harness()

        try:
            # 1. Setup Docker container with the task
            harness.setup_task(task_instance=example)

            # 2. Run the agent with current skills
            patch, trace, metrics = harness.run_agent(
                example["problem_statement"], skills, model_name=model_name
            )

            # 3. Verify the patch with tests
            passed, test_output = harness.verify_with_patch(patch)
            score = 1.0 if passed else 0.0

            # 4. Return score + diagnostic info for reflection
            side_info = {
                "Input": {"Problem": example["problem_statement"][:200]},
                "Generated Outputs": {"Patch": patch[:500], "Agent Trace": trace},
                "Feedback": {"Status": "passed" if passed else "failed", "Test Output": test_output},
                "scores": {"correctness": score},
            }
            return score, side_info

        finally:
            harness.cleanup()
            release_harness(harness)

    return fitness_fn

# Use it
fitness_fn = create_fitness_fn("gpt-5-mini", n_workers=6)

result = optimize_anything(
    seed_candidate={"skills": ""},
    evaluator=fitness_fn,
    dataset=train_data,
    valset=val_data,
    config=GEPAConfig(
        engine=EngineConfig(parallel=True, max_workers=6, max_metric_calls=600),
    ),
)


```

--------------------------------

### Install GEPA with Optional Dependencies

Source: https://gepa-ai.github.io/gepa/guides/experiment-tracking

Install GEPA with optional dependencies for WandB and MLflow using `gepa[full]`, or install them individually.

```bash
pip install "gepa[full]"         # includes wandb + mlflow
pip install wandb mlflow         # or install individually

```

--------------------------------

### Example Data Split

Source: https://gepa-ai.github.io/gepa/guides/faq

Demonstrates how to split a dataset into training and validation sets using Python slicing. Assumes 'examples' is a list or array.

```python
# Example split
trainset = examples[:80]  # 80% for training
valset = examples[80:]    # 20% for validation

```

--------------------------------

### Install GEPA with All Optional Dependencies

Source: https://gepa-ai.github.io/gepa/guides/quickstart

Install GEPA along with all its optional dependencies for full functionality.

```bash
pip install gepa[full]

```

--------------------------------

### Install and Run Pre-commit Hooks

Source: https://gepa-ai.github.io/gepa/guides/contributing

Installs pre-commit hooks for automated code quality checks and demonstrates how to commit changes or run hooks manually on all files.

```bash
# Install hooks (one-time setup)
uv run pre-commit install

# Hooks run automatically on commit
git add .
git commit -m "your message"

# Or run manually
uv run pre-commit run --all-files
```

--------------------------------

### Install GEPA from PyPI

Source: https://gepa-ai.github.io/gepa/guides/quickstart

Install the GEPA package using pip.

```bash
pip install gepa

```

--------------------------------

### HuggingFace Cookbook: Prompt Optimization

Source: https://gepa-ai.github.io/gepa/guides/use-cases

A comprehensive guide on prompt optimization using DSPy and GEPA. Covers setting up DSPy, processing datasets, building reasoning programs, and error-driven feedback optimization.

```markdown
Comprehensive guide on **prompt optimization with DSPy and GEPA**.

**What's Inside:**
    * Setting up DSPy with language models
    * Processing mathematical problem datasets
    * Building Chain-of-Thought reasoning programs
    * Error-driven feedback optimization

View cookbook
```

--------------------------------

### Install gskill and Dependencies

Source: https://gepa-ai.github.io/gepa/guides/gskill

Install the gskill package along with its dependencies for SWE-smith and mini-swe-agent. Ensure Docker is running and download SWE-smith images for your target repository.

```bash
pip install gepa[gskill]
pip install mini-swe-agent swebench

```

```bash
export OPENAI_API_KEY=<your-key>

# Docker must be running
docker ps

# Download SWE-smith images for your target repo
python -m swesmith.build_repo.download_images

```

--------------------------------

### Install Latest Development Version

Source: https://gepa-ai.github.io/gepa/guides/quickstart

Install the latest development version of GEPA directly from its GitHub repository.

```bash
pip install git+https://github.com/gepa-ai/gepa.git

```

--------------------------------

### Verify claude CLI installation

Source: https://gepa-ai.github.io/gepa/guides/claude-cli-as-proposer

Ensure the `claude` CLI is installed and authenticated by running it in interactive and non-interactive modes.

```bash
claude
claude -p "say hello"

```

--------------------------------

### Quick Start: Confidence Adapter for Classification

Source: https://gepa-ai.github.io/gepa/guides/confidence-adapter

Demonstrates how to initialize and use the ConfidenceAdapter for classifying transactions with a predefined schema and training data. This snippet sets up a classification task using GPT-4.1-mini and optimizes the process with a reflection LLM.

```python
import gepa
from gepa.adapters.confidence_adapter import ConfidenceAdapter

adapter = ConfidenceAdapter(
    model="openai/gpt-4.1-mini",
    field_path="category_name",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "classification",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "category_name": {
                        "type": "string",
                        "enum": [
                            "Bills/Electricity",
                            "Bills/Gas & Oil",
                            "Food & Drinks/Restaurants",
                            "Shopping/Electronics",
                            "Shopping/Video Games",
                        ],
                    }
                },
                "required": ["category_name"],
                "additionalProperties": False,
            },
        },
    },
)

result = gepa.optimize(
    seed_candidate={"system_prompt": "Classify the following transaction."},
    trainset=[
        {"input": "UBER EATS payment", "answer": "Food & Drinks/Restaurants", "additional_context": {}},
        {"input": "LIGHT electricity bill", "answer": "Bills/Electricity", "additional_context": {}},
        {"input": "Steam purchase", "answer": "Shopping/Video Games", "additional_context": {}},
    ],
    adapter=adapter,
    reflection_lm="openai/gpt-4.1",
    max_metric_calls=500,
)

```

--------------------------------

### DefaultAdapter Feedback Example

Source: https://gepa-ai.github.io/gepa/guides/confidence-adapter

Illustrates the feedback provided by the DefaultAdapter when a classification is incorrect. It focuses on the discrepancy between the model's response and the correct answer.

```text
"The generated response is incorrect. The correct answer is 'Bills/Electricity'. Ensure that the correct answer is included in the response exactly as it is."

```

--------------------------------

### Optimize with GEPAConfig and TrackingConfig

Source: https://gepa-ai.github.io/gepa/guides/experiment-tracking

Configure tracking options, including WandB, using nested GEPAConfig and TrackingConfig objects for more structured setup.

```python
from gepa.optimize_anything import optimize_anything, GEPAConfig, TrackingConfig

result = optimize_anything(
    seed_candidate="...",
    evaluator=my_evaluator,
    config=GEPAConfig(
        tracking=TrackingConfig(
            use_wandb=True,
            wandb_init_kwargs={"project": "my-gepa-run"},
        )
    ),
)

```

--------------------------------

### Add Hard Examples Dynamically

Source: https://gepa-ai.github.io/gepa/guides/callbacks

Adds hard examples from the validation set to the training set when a new best program is found. This callback implements active learning by enriching the training data with challenging examples.

```python
class ActiveLearner:
    """Add hard examples from the valset to the trainset when we find a new best."""

    def __init__(self, hard_threshold: float = 0.3):
        self.hard_threshold = hard_threshold
        self._last_best = None

    def on_valset_evaluated(self, event):
        if event["is_best_program"] and event["outputs_by_val_id"]:
            self._last_best = event

    def on_iteration_start(self, event):
        if self._last_best is None:
            return
        scores = self._last_best["scores_by_val_id"]
        hard_ids = [vid for vid, s in scores.items() if s < self.hard_threshold]
        if hard_ids:
            # Add the raw inputs for the hard examples to the training set
            hard_examples = [{"id": vid} for vid in hard_ids]
            event["trainset_loader"].add_items(hard_examples)
            print(f"Added {len(hard_ids)} hard examples to trainset")
        self._last_best = None

```

--------------------------------

### Custom Batch Sampler Implementation

Source: https://gepa-ai.github.io/gepa/guides/batch-sampling

Implement a custom batch sampler by creating a class that adheres to the `BatchSampler` protocol. This example demonstrates a sampler that always picks the hardest examples based on their scores from the Pareto front.

```python
from gepa.strategies.batch_sampler import BatchSampler
from gepa.core.data_loader import DataLoader
from gepa.core.state import GEPAState


class MyBatchSampler:
    """Custom batch sampler that always picks the hardest examples."""

    def __init__(self, minibatch_size: int):
        self.minibatch_size = minibatch_size

    def next_minibatch_ids(
        self, loader: DataLoader, state: GEPAState
    ) -> list:
        # Access the Pareto front to find the hardest examples
        hardest = sorted(
            state.pareto_front_valset.items(),
            key=lambda x: x[1],  # sort by best score (ascending)
        )
        # Return the IDs with the lowest best scores
        ids = [val_id for val_id, _score in hardest[: self.minibatch_size]]

        # Fall back to all IDs if not enough in the Pareto front
        if len(ids) < self.minibatch_size:
            all_ids = list(loader.all_ids())
            ids = all_ids[: self.minibatch_size]

        return ids

```

--------------------------------

### Optimize Anything API Usage

Source: https://gepa-ai.github.io/gepa/guides

Demonstrates how to use the optimize_anything API, starting from an existing artifact or in seedless mode. The evaluator function captures output and errors as Actionable Side Information (ASI).

```python
import gepa.optimize_anything as oa
from gepa.optimize_anything import optimize_anything, GEPAConfig, EngineConfig

def evaluator(candidate, example):
    result = run_my_system(candidate, example)
    oa.log(f"Output: {result.output}")  # captured as Actionable Side Information (ASI)
    oa.log(f"Error: {result.error}")
    return result.score

# Start from an existing artifact…
result = oa.optimize_anything(
    seed_candidate="<your artifact>",
    evaluator=evaluator,
    dataset=my_data,
    objective="Optimize for accuracy and speed.",
    config=GEPAConfig(engine=EngineConfig(max_metric_calls=200)),
)

# … or just describe what you need (seedless mode).
result = oa.optimize_anything(
    evaluator=evaluator,
    objective="Generate a Python function that reverses a string.",
)
```

--------------------------------

### GEPA Optimization Anything with Custom Reflection Minibatch Size

Source: https://gepa-ai.github.io/gepa/guides/batch-sampling

Configure GEPA optimization using `optimize_anything` and specify the reflection minibatch size within the `GEPAConfig`. This allows for fine-grained control over the number of examples per iteration.

```python
from gepa.optimize_anything import optimize_anything, GEPAConfig, ReflectionConfig

result = optimize_anything(
    ...,
    config=GEPAConfig(
        reflection=ReflectionConfig(
            reflection_minibatch_size=5,  # default: 1 (single-task) or 3 (multi-task)
        ),
    ),
)
```

--------------------------------

### Augmenting DSPy Examples with Explanations

Source: https://gepa-ai.github.io/gepa/guides/faq

Use augmented DSPy examples with an 'explanation' field to provide reasoning for the answer. This is recommended for improving GEPA's reflection LLM capabilities.

```python
# Basic example
dspy.Example(question="Is this email urgent?", answer="Yes").with_inputs("question")

# Augmented example (recommended)
dspy.Example(
    question="Is this email urgent?", 
    answer="Yes",
    explanation="The email mentions a deadline of 'end of day today' and uses words like 'critical' and 'ASAP', indicating urgency."
).with_inputs("question")
```

--------------------------------

### Run gskill Smoke Test

Source: https://gepa-ai.github.io/gepa/guides/gskill

Execute a smoke test to validate the setup, including Docker, API keys, and the end-to-end pipeline. This runs a small number of training tasks for quick verification.

```bash
python -m gepa.gskill.train_optimize_anything \
  --smoke-test --model "gpt-5-mini"

```

--------------------------------

### Basic Prompt Optimization with DefaultAdapter

Source: https://gepa-ai.github.io/gepa/guides/quickstart

Use the DefaultAdapter for single-turn LLM tasks. Define training data, a seed prompt, and run optimization.

```python
import gepa

# The DefaultAdapter expects each example as a TypedDict with three fields:
#   input               — the user question
#   additional_context  — dict of extra context (can be empty {})
#   answer              — the expected answer (used for substring match)
trainset = [
    {"input": "What is 2+2?", "additional_context": {}, "answer": "4"},
    {"input": "What is the capital of France?", "additional_context": {}, "answer": "Paris"},
    # ... more examples
]

# Define your seed prompt
seed_prompt = {
    "system_prompt": "You are a helpful assistant. Answer questions concisely."
}

# Run optimization
result = gepa.optimize(
    seed_candidate=seed_prompt,
    trainset=trainset,
    task_lm="openai/gpt-4o-mini",      # Model to optimize
    reflection_lm="openai/gpt-4o",      # Model for reflection
    max_metric_calls=50,                # Budget
)

# Get the optimized prompt and best score
print("Best prompt:", result.best_candidate['system_prompt'])
print("Best score:", result.val_aggregate_scores[result.best_idx])

```

--------------------------------

### Complete SimpleQAAdapter Implementation

Source: https://gepa-ai.github.io/gepa/guides/adapters

A full GEPA adapter for Question Answering tasks. It defines input/output structures, evaluation logic, and a method for creating reflective datasets.

```python
from dataclasses import dataclass
from typing import Any
import litellm
from gepa.core.adapter import GEPAAdapter, EvaluationBatch

@dataclass
class QAInput:
    question: str
    answer: str

@dataclass
class QATrace:
    prompt: str
    response: str

@dataclass
class QAOutput:
    answer: str

class SimpleQAAdapter(GEPAAdapter[QAInput, QATrace, QAOutput]):
    def __init__(self, model: str = "openai/gpt-4o-mini"):
        self.model = model

    def evaluate(
        self,
        batch: list[QAInput],
        candidate: dict[str, str],
        capture_traces: bool = False,
    ) -> EvaluationBatch[QATrace, QAOutput]:
        outputs, scores = [], []
        trajectories = [] if capture_traces else None

        for item in batch:
            # Build prompt
            prompt = f"{candidate['system_prompt']}\n\nQuestion: {item.question}"

            # Call model
            response = litellm.completion(
                model=self.model,
                messages=[{"role": "user", "content": prompt}],
            )
            answer = response.choices[0].message.content

            # Score
            output = QAOutput(answer=answer)
            score = 1.0 if item.answer.lower() in answer.lower() else 0.0

            outputs.append(output)
            scores.append(score)

            if capture_traces:
                trajectories.append(QATrace(prompt=prompt, response=answer))

        return EvaluationBatch(
            outputs=outputs,
            scores=scores,
            trajectories=trajectories,
        )

    def make_reflective_dataset(
        self,
        candidate: dict[str, str],
        eval_batch: EvaluationBatch[QATrace, QAOutput],
        components_to_update: list[str],
    ) -> dict[str, list[dict]]:
        dataset = {"system_prompt": []}

        for i, trace in enumerate(eval_batch.trajectories or []):
            dataset["system_prompt"].append({
                "Inputs": {"question": trace.prompt.split("Question: ")[-1]},
                "Generated Outputs": {"answer": trace.response},
                "Feedback": f"Score: {eval_batch.scores[i]}"
            })

        return dataset

# Usage
adapter = SimpleQAAdapter(model="openai/gpt-4o-mini")
result = gepa.optimize(
    seed_candidate={"system_prompt": "Answer questions accurately."},
    trainset=trainset,
    adapter=adapter,
    reflection_lm="openai/gpt-4o",
    max_metric_calls=50,
)

```

--------------------------------

### Using Custom Batch Sampler with optimize_anything()

Source: https://gepa-ai.github.io/gepa/guides/batch-sampling

Integrate a custom `BatchSampler` within the `GEPAConfig` for `optimize_anything()`. Ensure it's placed under the `reflection` configuration.

```python
from gepa.optimize_anything import optimize_anything, GEPAConfig, ReflectionConfig

result = optimize_anything(
    ...,
    config=GEPAConfig(
        reflection=ReflectionConfig(
            batch_sampler=MyBatchSampler(minibatch_size=5),
        ),
    ),
)
```

--------------------------------

### Custom Acceptance Criterion: Any Example Improved

Source: https://gepa-ai.github.io/gepa/guides/acceptance-criterion

Implements a custom acceptance criterion where a candidate is accepted if at least one example's score improves, even if others regress.

```python
from gepa.strategies.acceptance import AcceptanceCriterion
from gepa.proposer.base import CandidateProposal
from gepa.core.state import GEPAState

class AnyExampleImproved:
    def should_accept(self, proposal: CandidateProposal, state: GEPAState) -> bool:
        old = proposal.subsample_scores_before or []
        new = proposal.subsample_scores_after or []
        return any(n > o for n, o in zip(new, old))

```

--------------------------------

### Implement `make_reflective_dataset` Method

Source: https://gepa-ai.github.io/gepa/guides/adapters

Implements the `make_reflective_dataset` method to construct a dataset for a reflection LLM, formatting input, generated output, and feedback for specified components.

```python
def make_reflective_dataset(
    self,
    candidate: dict[str, str],
    eval_batch: EvaluationBatch[ExecutionTrace, TaskOutput],
    components_to_update: list[str],
) -> dict[str, list[dict]]:
    """Build a reflective dataset for each component."""

    dataset = {}

    for component_name in components_to_update:
        component_data = []

        for i, trace in enumerate(eval_batch.trajectories):
            record = {
                "Inputs": {
                    "prompt": trace.prompt_used,
                },
                "Generated Outputs": {
                    "response": trace.model_response,
                },
                "Feedback": self._generate_feedback(
                    trace,
                    eval_batch.outputs[i],
                    eval_batch.scores[i],
                ),
            }
            component_data.append(record)

        dataset[component_name] = component_data

    return dataset

def _generate_feedback(self, trace, output, score):
    """Generate helpful feedback for the reflection LLM."""
    if score == 1.0:
        return "Correct! The answer matched the expected output."
    else:
        return f"Incorrect. The model answered '{output.answer}' but this was wrong."

```

--------------------------------

### Evaluator Returning Multi-Objective Scores

Source: https://gepa-ai.github.io/gepa/guides/acceptance-criterion

Example of an evaluator function that returns multi-objective scores within the 'side_info' dictionary.

```python
def evaluator(candidate, example):
    ...
    side_info = {
        "scores": {
            "accuracy": accuracy_score,
            "cost": cost_score,
        },
    }
    return score, side_info

```

--------------------------------

### Optimize with Both WandB and MLflow

Source: https://gepa-ai.github.io/gepa/guides/experiment-tracking

Simultaneously enable both Weights & Biases and MLflow tracking for comprehensive logging.

```python
result = gepa.optimize(
    ...
    use_wandb=True,
    use_mlflow=True,
    mlflow_experiment_name="my-gepa-run",
)

```

--------------------------------

### Using 'hybrid' frontier type with gepa.optimize

Source: https://gepa-ai.github.io/gepa/guides/candidate-selection

Sets the `frontier_type` to 'hybrid' when calling `gepa.optimize`. This is a common setting for balancing multiple metrics and examples.

```python
result = gepa.optimize(
    ...,
    candidate_selection_strategy="pareto",
    frontier_type="hybrid",
)
```

--------------------------------

### Using 'hybrid' frontier type with GEPAConfig

Source: https://gepa-ai.github.io/gepa/guides/candidate-selection

Configures the `frontier_type` to 'hybrid' within `GEPAConfig` for `optimize_anything`. This is a common setting for balancing multiple metrics and examples.

```python
config = GEPAConfig(
    engine=EngineConfig(
        candidate_selection_strategy="pareto",
        frontier_type="hybrid",
    ),
)
```

--------------------------------

### Optimize with Weights & Biases

Source: https://gepa-ai.github.io/gepa/guides/experiment-tracking

Enable Weights & Biases tracking during optimization by setting `use_wandb=True` and providing `wandb_init_kwargs` for project and run naming.

```python
import gepa

result = gepa.optimize(
    seed_candidate={"system_prompt": "You are a helpful assistant."},
    trainset=trainset,
    valset=valset,
    adapter=adapter,
    reflection_lm="openai/gpt-5",
    max_metric_calls=200,
    use_wandb=True,
    wandb_init_kwargs={"project": "my-gepa-run", "name": "experiment-1"},
)

```

--------------------------------

### Configuring Candidate Selection in optimize_anything

Source: https://gepa-ai.github.io/gepa/guides/candidate-selection

Sets the candidate selection strategy within the EngineConfig for the optimize_anything function. Requires importing necessary GEPA configuration classes.

```python
from gepa.optimize_anything import optimize_anything, GEPAConfig, EngineConfig

config = GEPAConfig(
    engine=EngineConfig(
        candidate_selection_strategy="pareto",
        max_metric_calls=200,
    ),
)

result = optimize_anything(config=config, ...)
```

--------------------------------

### Preventing Keyword Copying in GEPA Prompts

Source: https://gepa-ai.github.io/gepa/guides/faq

Use a 'Constraint' in the side_info to prevent GEPA from copying specific keywords or phrases from training examples. This helps generalize rules rather than over-indexing on surface patterns.

```python
def evaluator(data, response):
    score = compute_score(response, data["expected"])
    return score, {
        "Input": data["input"],
        "Output": response,
        "Expected": data["expected"],
        "Constraint": (
            "When improving the prompt, do NOT copy specific examples, "
            "keywords, usernames, or verbatim phrases from these examples. "
            "Generalize to rules that apply broadly."
        ),
    }
```

--------------------------------

### Enable Experiment Tracking with Weights & Biases

Source: https://gepa-ai.github.io/gepa/guides/faq

Enable Weights & Biases integration for experiment tracking by setting `use_wandb=True` during optimization. This allows logging of proposed prompts and optimization progress.

```python
result = gepa.optimize(..., use_wandb=True)
```

--------------------------------

### Define gskill Fitness Function Signature

Source: https://gepa-ai.github.io/gepa/guides/gskill

Implement a fitness function for gskill that scores a candidate skill on a given task. The function must accept 'candidate' and 'example' arguments and return a score and diagnostic information.

```python
def fitness_fn(
    candidate: dict[str, str],
    example: dict[str, Any],
) -> tuple[float, dict[str, Any]]:
    """
    Args:
        candidate: Dict of optimizable text parameters (e.g. {"skills": "..."})
        example: A single task instance from the dataset

    Returns:
        score: Float, higher is better. 1.0 for pass, 0.0 for fail.
        side_info: Dict of diagnostic info for the reflection model.
    """


```

--------------------------------

### Perform a Full gskill Training Run

Source: https://gepa-ai.github.io/gepa/guides/gskill

Initiate a comprehensive training run for gskill. This command specifies the target repository, training dataset sizes, models for agent and reflection, parallel workers, and budget.

```bash
python -m gepa.gskill.train_optimize_anything \
  --repo pygments__pygments \
  --train-size 200 --val-size 50 --test-size 100 \
  --model gpt-5-mini --reflection-model gpt-5.2-pro \
  --workers 6 --max-metric-calls 600 \
  --proposer loop --wandb

```

--------------------------------

### View Optimized Prompt Instructions

Source: https://gepa-ai.github.io/gepa/guides/quickstart

Access the generated instructions from the optimized program signature. This is useful for inspecting the output of the optimization process.

```python
print(optimized_program.generate.signature.instructions)
```

--------------------------------

### Using 'pareto' Candidate Selection Strategy

Source: https://gepa-ai.github.io/gepa/guides/candidate-selection

Selects candidates based on their performance across multiple keys in the Pareto frontier. Probability is proportional to the number of keys a candidate is best on.

```python
result = gepa.optimize(
    ...,
    candidate_selection_strategy="pareto",
)
```

--------------------------------

### Penalize Malformed Outputs in Metric for Smaller Models

Source: https://gepa-ai.github.io/gepa/guides/faq

To fix malformed outputs from smaller models, penalize format failures in your metric function. Include parse errors in `side_info` to guide GEPA's reflection LM.

```python
def evaluator(data, response):
    import json
    # Hard penalty for format violations
    try:
        parsed = json.loads(response)
    except json.JSONDecodeError:
        return 0.0, {"Output": response, "Error": "Malformed JSON — failed to parse"}

    score = compute_quality_score(parsed, data)
    return score, {
        "Output": parsed,
        "Expected": data["expected"],
        "Score": score,
    }
```

--------------------------------

### Prompt Optimization with Pydantic AI

Source: https://gepa-ai.github.io/gepa/guides/use-cases

Tutorial demonstrating GEPA integration with Pydantic AI using `Agent.override()` for instruction injection and Pydantic Evals for parallel evaluation. Achieved 97% accuracy in contact extraction.

```markdown
Tutorial demonstrating GEPA integration with Pydantic AI using `Agent.override()` for instruction injection and Pydantic Evals for parallel evaluation.

**Results:** Contact extraction improved from 86% → 97% accuracy

Read the tutorial
View code examples
```

--------------------------------

### OpenAI Cookbook: Self-Evolving Agents

Source: https://gepa-ai.github.io/gepa/guides/use-cases

The official OpenAI Cookbook features GEPA for building autonomous self-healing workflows. Learn to diagnose agent shortcomings, build retraining loops, and combine human review with LLM-as-judge evaluations.

```markdown
The official OpenAI Cookbook (Nov 2025) features GEPA for building **autonomous self-healing workflows**.

**What You'll Learn:**
    * Diagnose why agents fall short of production readiness
    * Build automated LLMOps retraining loops
    * Combine human review, LLM-as-judge evaluations, and GEPA optimization

View cookbook
```

--------------------------------

### Using 'epsilon_greedy' Candidate Selection Strategy

Source: https://gepa-ai.github.io/gepa/guides/candidate-selection

Combines exploration and exploitation. With probability epsilon, selects a random candidate; otherwise, picks the best by aggregate score.

```python
result = gepa.optimize(
    ...,
    candidate_selection_strategy="epsilon_greedy",
)
```

--------------------------------

### Optimize with MLflow

Source: https://gepa-ai.github.io/gepa/guides/experiment-tracking

Enable MLflow tracking by setting `use_mlflow=True` and specifying the `mlflow_tracking_uri` and `mlflow_experiment_name`.

```python
import gepa

result = gepa.optimize(
    seed_candidate={"system_prompt": "You are a helpful assistant."},
    trainset=trainset,
    valset=valset,
    adapter=adapter,
    reflection_lm="openai/gpt-5",
    max_metric_calls=200,
    use_mlflow=True,
    mlflow_tracking_uri="./mlruns",         # local directory
    mlflow_experiment_name="my-gepa-run",
)

```

--------------------------------

### GEPA Batch Proposer CLI

Source: https://gepa-ai.github.io/gepa/guides/gskill

Execute GEPA training with the default 'batch' proposer using the command line. The batch proposer sends all evaluation results to the reflection model at once.

```bash
python -m gepa.gskill.train_optimize_anything --proposer batch


```

--------------------------------

### GEPA Loop Proposer CLI

Source: https://gepa-ai.github.io/gepa/guides/gskill

Execute GEPA training with the 'loop' proposer using the command line. The loop proposer processes each evaluation result individually, merging skills iteratively.

```bash
python -m gepa.gskill.train_optimize_anything --proposer loop


```

--------------------------------

### Evaluate Mini-SWE-agent

Source: https://gepa-ai.github.io/gepa/guides/gskill

Runs the mini-SWE-agent evaluation script. This command executes evaluations for both with-skills and without-skills conditions. Specify the configuration file and the number of workers.

```bash
# Runs both with-skills and without-skills conditions
python -m gepa.gskill.gskill.evaluate.mini_swe_agent \
  --config gepa_results/logs/run_xxx/config.json \
  --workers 16

```

--------------------------------

### Enable Logging and Tracking for Optimization

Source: https://gepa-ai.github.io/gepa/guides/quickstart

Configure GEPA to log optimization progress using Weights & Biases or MLflow, save state to disk, and display a progress bar. Set `use_wandb`, `use_mlflow`, `run_dir`, and `display_progress_bar`.

```python
result = gepa.optimize(
    # ... other args ...
    use_wandb=True,
    use_mlflow=True,
    run_dir="./gepa_runs/my_exp",
    display_progress_bar=True,
)
```

--------------------------------

### Configure Reflection LM with optimize_anything

Source: https://gepa-ai.github.io/gepa/guides/claude-cli-as-proposer

Set the `claude_cli_lm` wrapper within `ReflectionConfig` when using `optimize_anything`.

```python
from gepa.optimize_anything import optimize_anything, GEPAConfig, ReflectionConfig

config = GEPAConfig(
    reflection=ReflectionConfig(
        reflection_lm=claude_cli_lm,
    ),
)

result = optimize_anything(config=config, ...)

```

--------------------------------

### BAML Prompt Optimization

Source: https://gepa-ai.github.io/gepa/guides/use-cases

BAML integrates GEPA into `baml-cli optimize` for test-driven prompt optimization with multi-objective support, including accuracy, latency, and token usage.

```markdown
BAML integrates GEPA into `baml-cli optimize` for test-driven prompt optimization with multi-objective support (accuracy, latency, tokens).

Read the guide
```

--------------------------------

### Customize GEPA Reflection Prompt

Source: https://gepa-ai.github.io/gepa/guides/faq

Customize the GEPA reflection prompt for domain-specific optimization by providing your custom logic for the instruction proposer.

```python
dspy.GEPA(
    metric=metric,
    instruction_proposer=CustomProposer(...)
)
```

--------------------------------

### Resume Optimization from a Previous Run

Source: https://gepa-ai.github.io/gepa/guides/faq

Set the `run_dir` in `EngineConfig` to automatically save and resume GEPA optimization states. Ensure the directory exists and contains `gepa_state.bin` for resumption.

```python
config = GEPAConfig(engine=EngineConfig(
    run_dir="./runs/my_exp",      # Will resume if gepa_state.bin exists
    max_metric_calls=500,
))
result = optimize_anything(..., config=config)
```

--------------------------------

### Using Custom Batch Sampler with gepa.optimize()

Source: https://gepa-ai.github.io/gepa/guides/batch-sampling

Pass a custom `BatchSampler` instance directly to `gepa.optimize()`. Do not set `reflection_minibatch_size` when using a custom sampler.

```python
result = gepa.optimize(
    ...,
    batch_sampler=MyBatchSampler(minibatch_size=5),
)
```

--------------------------------

### Optimize All Signatures in Multi-Module DSPy Programs

Source: https://gepa-ai.github.io/gepa/guides/faq

Use `component_selector='all'` to update all signatures in a multi-module program simultaneously, significantly boosting rollout efficiency. This is useful for complex programs with many distinct components.

```python
optimizer = dspy.GEPA(
    metric=metric,
    component_selector='all'  # Update all signatures at once!
)
```

--------------------------------

### Custom WandB init kwargs

Source: https://gepa-ai.github.io/gepa/guides/experiment-tracking

Pass custom keyword arguments to `wandb.init()` when using `gepa.optimize` with `use_wandb=True`. This allows fine-grained control over WandB run configuration.

```python
gepa.optimize(
    ...
    use_wandb=True,
    wandb_init_kwargs={
        "project": "prompt-optimization",
        "name": f"run-{my_experiment_id}",
        "tags": ["aime", "gpt-5"],
        "notes": "Testing new seed prompt",
        "config": {"dataset": "aime_2025", "model": "gpt-5"},
    },
)

```

--------------------------------

### Prompt Optimization with DSPy and GEPA

Source: https://gepa-ai.github.io/gepa/guides/quickstart

Integrate GEPA with DSPy for complex program optimization. Define a DSPy module, a metric with feedback, and compile with GEPA.

```python
import dspy

# Configure the task LM
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# Define your program
class QAProgram(dspy.Module):
    def __init__(self):
        self.generate = dspy.ChainOfThought("question -> answer")

    def forward(self, question):
        return self.generate(question=question)

# Define a metric WITH feedback - this is key for GEPA!
def metric_with_feedback(example, pred, trace=None):
    correct = example.answer.lower() in pred.answer.lower()
    score = 1.0 if correct else 0.0

    # Provide textual feedback to guide GEPA's reflection
    if correct:
        feedback = f"Correct! The answer '{pred.answer}' matches the expected answer '{example.answer}'."
    else:
        feedback = (
            f"Incorrect. Expected '{example.answer}' but got '{pred.answer}'. "
            f"Think about how to reason more carefully to arrive at the correct answer."
        )

    return dspy.Prediction(score=score, feedback=feedback)

# Prepare data (aim for 30-300 examples for best results)
trainset = [
    dspy.Example(question="What is 2+2?", answer="4").with_inputs("question"),
    dspy.Example(question="What is the capital of France?", answer="Paris").with_inputs("question"),
    # ... more examples
]

# Optimize with GEPA
optimizer = dspy.GEPA(
    metric=metric_with_feedback,
    reflection_lm=dspy.LM("openai/gpt-4o"),  # Strong model for reflection
    auto="light",  # Automatic budget configuration
    num_threads=8,
    track_stats=True,
)

optimized_program = optimizer.compile(QAProgram(), trainset=trainset)

```

--------------------------------

### Stop optimize_anything on Reflection Cost Budget

Source: https://gepa-ai.github.io/gepa/guides/cost-tracking

Configure `optimize_anything` to halt optimization when the reflection LM's cost reaches a defined budget. This is set within the `EngineConfig` of `GEPAConfig`.

```python
config = oa.GEPAConfig(
    engine=oa.EngineConfig(
        max_reflection_cost=10.00,
    ),
)


```

--------------------------------

### GEPA Three-Stage Pipeline Overview

Source: https://gepa-ai.github.io/gepa/guides

Visual representation of the GEPA iteration pipeline, showing the flow from Executor to Reflector to Curator.

```text
┌────────────┐      ┌────────────┐      ┌────────────┐
│  Executor  │ ───▶ │  Reflector │ ───▶ │   Curator  │
│            │      │            │      │            │
│ Run candi- │      │ Analyze    │      │ Generate   │
│ date on    │      │ traces to  │      │ improved   │
│ tasks,     │      │ diagnose   │      │ candidate  │
│ capture    │      │ failure    │      │ from diag- │
│ full       │      │ modes and  │      │ nostic     │
│ traces     │      │ causal     │      │ insights   │
│            │      │ patterns   │      │            │
└────────────┘      └────────────┘      └────────────┘

  1. **Executor** : Runs the candidate on a minibatch of evaluation tasks using the task model, capturing complete execution traces — reasoning chains, intermediate outputs, error messages, performance metrics, and any ASI returned by the evaluator.
  2. **Reflector** : A strong LLM (the `reflection_lm`) analyzes the collected traces to identify failure modes, logic breakdowns, and causal relationships. For example: "The agent used inductive reasoning on problem 3, but proof by contradiction would be more effective given the problem structure." The reflector sees both successes and failures to build a complete picture.
  3. **Curator** : Based on the reflector's diagnosis, generates an improved candidate with concrete modifications to the text, code, agent logic, or system architecture. Each new candidate inherits accumulated lessons from all of its ancestors in the search tree.
```

--------------------------------

### Log LM Calls and Responses

Source: https://gepa-ai.github.io/gepa/guides/callbacks

Logs the prompt sent to the reflection LM and its raw response before instruction extraction. Use this to debug or understand the LM's behavior.

```python
class LMCallLogger:
    def on_proposal_end(self, event):
        for component, prompt in event["prompts"].items():
            print(f"\n=== Reflection LM call — component: {component} ===")
            if isinstance(prompt, str):
                print("PROMPT (last 500 chars):", prompt[-500:])
            print("RAW OUTPUT:", event["raw_lm_outputs"].get(component, "")[:500])
            print("EXTRACTED:", event["new_instructions"].get(component, "")[:200])

```

--------------------------------

### Display Live Progress

Source: https://gepa-ai.github.io/gepa/guides/callbacks

Logs the new best score and the delta when a new best program is found during validation. This callback is useful for monitoring training progress in real-time.

```python
class ProgressCallback:
    def __init__(self):
        self.best_score = 0.0

    def on_valset_evaluated(self, event):
        if event["is_best_program"]:
            delta = event["average_score"] - self.best_score
            self.best_score = event["average_score"]
            print(f"[iter {event['iteration']}] New best: {self.best_score:.4f}  (+{delta:.4f})")

```

--------------------------------

### Configuring GEPA Runtime and Budget

Source: https://gepa-ai.github.io/gepa/guides/faq

Shows how to control GEPA's execution time and resource usage by setting `max_metric_calls` and using various `stop_callbacks`. GEPA will halt execution when any of the specified conditions are met.

```python
from gepa.optimize_anything import GEPAConfig, EngineConfig
from gepa.utils import TimeoutStopCondition, NoImprovementStopper

config = GEPAConfig(
    engine=EngineConfig(max_metric_calls=200),
    stop_callbacks=[
        TimeoutStopCondition(timeout_seconds=3600),
        NoImprovementStopper(max_iterations_without_improvement=10),
    ],
)

```
Tokens:
Raw