### Install Dependencies and Run Local Development Server

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/README.md

Navigate to the docs directory, install project dependencies using npm, and start the local development server.

```bash
cd docs
npm install
npm run dev
```

--------------------------------

### Benchmark Configuration with Resolver

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/README.md

Example of a config.yaml file enabling and configuring a benchmark resolver. This setup specifies the benchmark name, the Python module path for the resolver, and benchmark-specific parameters like difficulty level.

```yaml
benchmark:
  enabled: true                    # Enable benchmark loader
  name: kernelbench                # Benchmark name (for logging)
  resolver: benchmarks.kernelbench.resolver  # Python module path
  
  # Benchmark-specific parameters
  level: 2                         # Example: difficulty level
  problem_id: 5                    # Example: specific problem ID
```

--------------------------------

### Install Benchmark Requirements

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Command to install requirements from a benchmark's specific requirements.txt file.

```bash
uv pip install -r path/to/requirements.txt
```

--------------------------------

### Benchmark Dependency Extras Installation

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Commands for installing benchmark dependency extras using 'uv sync'. Supports extras for math, ADRS systems, and more.

```bash
uv sync                              # Base install
uv sync --extra math                 # Math benchmarks (SciPy, JAX, PyWavelets, …)
uv sync --extra adrs                 # ADRS systems benchmarks
uv sync --extra frontier-cs          # Frontier-CS benchmark tooling
uv sync --extra external             # OpenEvolve / GEPA / ShinkaEvolve backends
uv sync --extra prompt-optimization  # HotPotQA prompt optimization
```

--------------------------------

### KernelBench Configuration Example

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/kernelbench/README.md

Example configuration for KernelBench, specifying the benchmark level and problem ID.

```yaml
benchmark:
  # KernelBench problem specification
  level: 2                    # Problem difficulty level (1, 2, 3 or 4)
  problem_id: 5               # Specific problem ID within the level
```

--------------------------------

### Install Dependencies and Run Benchmark

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/prompt_optimization/README.md

Installs necessary Python packages and runs the prompt optimization benchmark using SkyDiscover. Ensure you have the OpenAI API key set as an environment variable.

```bash
cd benchmarks/prompt_optimization/hotpot_qa

# Install deps
pip install dspy litellm bm25s pystemmer datasets diskcache ujson

# Set API key
export OPENAI_API_KEY=...

# Run (first run downloads ~1.3GB of data)
uv run skydiscover-run initial_prompt.txt evaluator.py -c config_adaevolve.yaml -i 100  # AdaEvolve
uv run skydiscover-run initial_prompt.txt evaluator.py -c config_evox.yaml -i 100       # EvoX
```

--------------------------------

### Install KernelBench Library

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/kernelbench/README.md

Installs the KernelBench library using uv pip, which is required for problem fetching.

```bash
uv pip install -r benchmarks/kernelbench/requirements.txt
```

--------------------------------

### Install ShinkaEvolve External Backend

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Use this command to clone the ShinkaEvolve repository and install it as an editable package. This is required for using the ShinkaEvolve backend.

```bash
git clone --depth 1 https://github.com/SakanaAI/ShinkaEvolve.git external_repos/ShinkaEvolve
uv pip install -e external_repos/ShinkaEvolve
```

--------------------------------

### Install Harbor CLI and Download Dataset

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/README.md

Installs the Harbor CLI and downloads a specified dataset. The dataset is saved to a local directory for use with SkyDiscover.

```bash
pip install harbor
harbor datasets download algotune@1.0 -o /tmp/algotune
```

--------------------------------

### Install Dependencies and Set API Key

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/frontier-cs-eval/README.md

Install project dependencies using 'uv sync' and set the OpenAI API key as an environment variable. This step is crucial before running the discovery process.

```bash
cd ../..
uv sync --extra frontier-cs

export OPENAI_API_KEY=...
```

--------------------------------

### Install Dependencies

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/prompt_optimization/hotpot_qa/README.md

Installs necessary Python packages for the project. Ensure you have pip available.

```bash
pip install dspy litellm bm25s pystemmer datasets diskcache ujson
```

--------------------------------

### Initialize Discovery with ShinkaEvolve Backend (Python)

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/extras/external/README.md

Start a discovery process with the ShinkaEvolve backend programmatically. Use 'shinkaevolve' for the search parameter.

```python
from skydiscover import run_discovery
result = run_discovery(
    initial_program="initial_program.py",
    evaluator="evaluator.py",
    search="shinkaevolve",
    model="gpt-5",
    iterations=100,
)
```

--------------------------------

### YAML Configuration Example

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Example of a YAML configuration file for Skydiscover AI. API keys are resolved from environment variables.

```yaml
max_iterations: 100
llm:
  models: [{ name: "gemini/gemini-3-pro-preview", weight: 1.0 }]
search:
  type: "adaevolve"                  # or "evox", "topk", "beam_search", "best_of_n"
prompt:
  system_message: |
    You are an expert at optimizing algorithms.
```

--------------------------------

### Install Benchmark-Specific Requirements

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/README.md

Installs requirements for a specific benchmark task if a 'requirements.txt' file exists in its directory.

```bash
uv pip install -r benchmarks/<task>/requirements.txt
```

--------------------------------

### Install PyTorch

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/ADRS/eplb/README.md

Installs the PyTorch library, which is required by the evaluator. Use this command in your terminal.

```bash
uv pip install torch
```

--------------------------------

### Running EvoX via Command Line

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/search/evox/README.md

Example of how to initiate an EvoX search process using the `uv run` command with specified initial program, evaluator, configuration, and iterations.

```bash
uv run skydiscover-run initial_program.py evaluator.py \
  --config config.yaml --search evox --iterations 100
```

--------------------------------

### Install Optional Dependencies for Benchmarks

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/installation.mdx

Install optional dependencies for specific benchmark categories using uv sync with the --extra flag. This is required for running certain comparison benchmarks.

```bash
uv sync --extra adrs         # ADRS systems benchmarks
uv sync --extra external     # OpenEvolve / GEPA / ShinkaEvolve backends
uv sync --extra math         # Math benchmarks (SciPy, JAX, etc.)
```

--------------------------------

### Initialize Discovery with OpenEvolve Backend (Python)

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/extras/external/README.md

Start a discovery process using the OpenEvolve backend programmatically. Specify 'openevolve' as the search parameter.

```python
from skydiscover import run_discovery
result = run_discovery(
    initial_program="initial_program.py",
    evaluator="evaluator.py",
    search="openevolve",
    model="gpt-5",
    iterations=100,
)
```

--------------------------------

### Install KernelBench with GPU Support

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/kernelbench/README.md

Install the KernelBench package, including GPU support, from its GitHub repository. This is necessary if the package is not found.

```bash
pip install "kernelbench[gpu] @ git+https://github.com/ScalingIntelligence/KernelBench.git"
```

--------------------------------

### Install Skydiscover AI and Set API Key

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Install the necessary tools using uv and set your OpenAI API key. This is a prerequisite for running Skydiscover.

```bash
uv sync
export OPENAI_API_KEY="<your-key>"
```

--------------------------------

### EvoX Configuration Example

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/search/evox/README.md

A snippet from the `evox.yaml` configuration file, showing how to enable auto-generation of variation operators for search strategies.

```yaml
search:
  type: "evox"
  database:
    auto_generate_variation_operators: true
```

--------------------------------

### Run Circle Packing Benchmark with Evox Search

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Execute the circle packing benchmark using the 'evox' search algorithm for 100 iterations. Ensure 'math' extra is installed.

```bash
uv sync --extra math
uv run skydiscover-run benchmarks/math/circle_packing/initial_program.py \
  benchmarks/math/circle_packing/evaluator.py \
  --config benchmarks/math/circle_packing/config.yaml \
  --search evox \
  --iterations 100
```

--------------------------------

### Install Extra Dependencies for Benchmarks

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/README.md

Installs optional dependencies based on the specific benchmarks you intend to run. Choose extras like 'external', 'math', 'adrs', 'frontier-cs', or 'prompt-optimization'.

```bash
uv sync --extra external            # openevolve/gepa/shinkaevolve
```

```bash
uv sync --extra math                # math benchmarks
```

```bash
uv sync --extra adrs                # ADRS benchmarks
```

```bash
uv sync --extra frontier-cs         # frontier-cs-eval benchmark
```

```bash
uv sync --extra prompt-optimization # HotPotQA prompt benchmark
```

--------------------------------

### Verify SkyDiscover Installation

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/installation.mdx

Verify the installation by running the skydiscover-run command with the --help flag. This should display the available command-line flags.

```bash
uv run skydiscover-run --help
```

--------------------------------

### Install KernelBench with GPU Support (Native)

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/kernelbench/README.md

Installs KernelBench with GPU support using pip, intended for native Python execution without Docker.

```bash
# Install KernelBench with GPU support
pip install -r benchmarks/kernelbench/evaluator/requirements.txt
```

--------------------------------

### Clone Frontier-CS and Start Judge Server

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/frontier-cs-eval/README.md

Clone the Frontier-CS repository and start the judge server using Docker Compose. Ensure you are in the correct directory before running these commands.

```bash
cd benchmarks/frontier-cs-eval
git clone https://github.com/FrontierCS/Frontier-CS.git

cd Frontier-CS/algorithmic
docker compose up -d
```

--------------------------------

### Run KernelBench with Explicit Initial Program

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/kernelbench/README.md

Execute KernelBench using a specified initial Python program. This is useful for custom benchmark setups.

```bash
uv run skydiscover-run my_kernel.py benchmarks/kernelbench/evaluator/ \
  -c benchmarks/kernelbench/config.yaml \
  --search <algo>
```

--------------------------------

### Running EvoX via Python API

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/search/evox/README.md

Demonstrates how to use the `run_discovery` function from the `skydiscover` library to start an EvoX search, specifying essential parameters.

```python
from skydiscover import run_discovery
result = run_discovery(
    initial_program="initial_program.py",
    evaluator="evaluator.py",
    search="evox",
    model="gpt-5",
    iterations=100,
)
```

--------------------------------

### Python API discover_solution() Wrapper

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Example of using the discover_solution() convenience wrapper in Python. It allows for inline string solutions and callable evaluators.

```python
from skydiscover import discover_solution

result = discover_solution(
    initial_solution="def solve(x): return x",  # optional — omit to start from scratch
    evaluator=lambda path: {"combined_score": run_tests(path)},
    iterations=50,
    search="evox",
)
```

--------------------------------

### Run Custom Problem Discovery Letting LLM Start from Scratch

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Run Skydiscover on a custom problem without an initial program, allowing the LLM to start from scratch. Specify the algorithm, model, and iterations.

```bash
uv run skydiscover-run evaluator.py \
  --search <algo> \
  --model gpt-5 \
  --iterations 100
```

--------------------------------

### Run Circle Packing Benchmark with AdaEvolve Search

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Execute the circle packing benchmark using the 'adaevolve' search algorithm for 100 iterations. Ensure 'math' extra is installed.

```bash
uv run skydiscover-run benchmarks/math/circle_packing/initial_program.py \
  benchmarks/math/circle_packing/evaluator.py \
  --config benchmarks/math/circle_packing/config.yaml \
  --search adaevolve \
  --iterations 100
```

--------------------------------

### Top-K Search Configuration

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/search/topk/README.md

Example YAML configuration for the Top-K search strategy, specifying the search type and the number of context programs.

```yaml
search:
  type: "topk"
  num_context_programs: 4
```

--------------------------------

### Custom Context Builder Implementation

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/context_builder/README.md

Extend `DefaultContextBuilder` to create a custom context builder. This example shows how to load custom templates and inject additional guidance into the prompt using the `{search_guidance}` placeholder.

```python
from pathlib import Path
from skydiscover.context_builder.default import DefaultContextBuilder
from skydiscover.context_builder.utils import TemplateManager

class MyContextBuilder(DefaultContextBuilder):

    def __init__(self, config):
        super().__init__(config)
        # load your templates on top of the defaults
        default_templates = str(Path(__file__).parent.parent / "default" / "templates")
        my_templates = str(Path(__file__).parent / "templates")
        self.template_manager = TemplateManager(default_templates, my_templates)

    def build_prompt(self, current_program, context=None, **kwargs):
        context = context or {}
        # format whatever the manager put into the context dict
        guidance = self._format_guidance(context.get("my_key"))
        return super().build_prompt(current_program, context, search_guidance=guidance, **kwargs)

    def _format_guidance(self, data):
        if not data:
            return ""
        return f"## CONTEXT\n{data}"

```

```python
self.context_builder = MyContextBuilder(self.config)

```

--------------------------------

### Best-of-N Configuration Example

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/search/best_of_n/README.md

This YAML configuration snippet shows how to set the `best_of_n` parameter, which defines the number of iterations to reuse the same parent during the search.

```yaml
search:
  type: "best_of_n"
  database:
    best_of_n: 5   # number of iterations to reuse the same parent
```

--------------------------------

### Model Provider Configuration

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Examples of configuring different LLM model providers via the command line. Supports OpenAI, Gemini, Anthropic, and local models via API base URL.

```bash
--model gpt-5                                               # OpenAI (default)
--model gemini/gemini-3-pro-preview                          # Gemini
--model anthropic/claude-sonnet-4-20250514                   # Anthropic
--model ollama/llama3 --api-base http://localhost:11434/v1   # Local (Ollama, vLLM, etc.)
```

--------------------------------

### Configure Model (GPT vs Gemini)

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/arc_benchmark/README.md

Switch between different AI models like GPT and Gemini by editing the `config.yaml` file or using command-line overrides. This example shows how to specify Gemini.

```bash
uv run skydiscover-run ... -m gemini/gemini-3-pro-preview
```

--------------------------------

### Quick Experiment with Python API

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/configuration.mdx

Use the `discover_solution` convenience wrapper for rapid experimentation. Define initial solution and evaluator inline.

```python
from skydiscover import discover_solution

result = discover_solution(
    initial_solution="def solve(x): return x",
    evaluator=lambda path: {"combined_score": run_tests(path)},
    iterations=50,
    search="evox",
)
```

--------------------------------

### Configure with YAML

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/configuration.mdx

Use a YAML file for detailed configuration of models, search algorithms, and prompts.

```yaml
max_iterations: 100
llm:
  models:
    - name: "gpt-5"
      weight: 0.7
    - name: "gemini/gemini-3-pro-preview"
      weight: 0.3
  max_tokens: 8192
  timeout: 120

search:
  type: "evox"

prompt:
  system_message: |
    You are an expert at optimizing algorithms.

evaluator:
  timeout: 60
```

--------------------------------

### Run Discovery with ShinkaEvolve Backend

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/extras/external/README.md

Execute SkyDiscover using the ShinkaEvolve backend via the command line. Specify 'shinkaevolve' as the search type for multi-patch evolution.

```bash
uv run skydiscover-run initial_program.py evaluator.py \
  --config config.yaml --search shinkaevolve --iterations 100
```

--------------------------------

### Monitor Configuration

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/configs/README.md

Set up the live dashboard, including enabling it, specifying the host and port, and configuring AI-generated run summaries.

```yaml
monitor:
  enabled: false
  port: 8765
  host: "0.0.0.0"
  max_solution_length: 10000

  # AI-generated run summaries
  summary_model: "gpt-5-mini"
  summary_api_key: null    # falls back to OPENAI_API_KEY
  summary_top_k: 3
  summary_interval: 0      # auto-generate every N programs (0 = manual only)
```

--------------------------------

### Combine Optional Extras

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/installation.mdx

You can combine multiple optional extras when installing dependencies by specifying the --extra flag multiple times.

```bash
uv sync --extra external --extra math
```

--------------------------------

### Copy Configuration Template

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/configs/README.md

Copy a template configuration file to your project directory to begin customization.

```bash
cp configs/evox.yaml my_config.yaml
```

--------------------------------

### Run Harbor Benchmark for AlgoTune

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Download the AlgoTune dataset and run Skydiscover on a set cover problem using a specified model and search strategy with 10 iterations.

```bash
pip install harbor
harbor datasets download algotune@1.0 -o /tmp/algotune
uv run skydiscover-run /tmp/algotune/<id>/algotune-set-cover \
  --model anthropic/claude-sonnet-4-6 \
  --search best_of_n -i 10
```

--------------------------------

### Run Sky Festival Benchmark with EvoX

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/image_gen/README.md

Navigate to the sky_festival directory and run the benchmark using the EvoX strategy. The output will be saved to sky_festival_output.

```bash
cd benchmarks/image_gen/sky_festival

# EvoX
uv run skydiscover-run evaluator.py -c config.yaml -s evox -o sky_festival_output
```

--------------------------------

### Dockerfile for Containerized Evaluator

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/README.md

A standard Dockerfile to set up the environment for the containerized evaluator. It installs dependencies, copies the evaluator code, and makes `evaluate.sh` executable.

```dockerfile
FROM python:3.12-slim
WORKDIR /benchmark

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
RUN chmod +x evaluate.sh

ENTRYPOINT ["./evaluate.sh"]
```

--------------------------------

### Run LLM-SQL Benchmark

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/ADRS/llm_sql/README.md

Run the LLM-SQL benchmark from the repository root. This command executes the initial program and evaluator scripts with a specified configuration and algorithm.

```bash
uv run skydiscover-run \
  benchmarks/ADRS/llm_sql/initial_program.py \
  benchmarks/ADRS/llm_sql/evaluator.py \
  -c benchmarks/ADRS/llm_sql/config.yaml \
  -s [your_algorithm] \
  -i 100
```

--------------------------------

### Continue Improving Model from Checkpoint

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/quick-start.mdx

This command resumes model training from a specified checkpoint, allowing for further iterations and improvements. Ensure you have 'uv' installed for running the script.

```bash
uv run skydiscover-run examples/text_similarity/initial_program.py \
  examples/text_similarity/evaluator/ \
  --config examples/text_similarity/config.yaml \
  --search best_of_n --iterations 20 \
  --checkpoint outputs/best_of_n/text_similarity_*/checkpoints/checkpoint_5
```

--------------------------------

### Run Discovery with Configuration

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/configs/README.md

Execute the Skydiscover run command with a specified configuration file and number of iterations.

```bash
uv run skydiscover-run initial_program.py evaluator.py -c my_config.yaml -i 100
```

--------------------------------

### Basic AdaEvolve Configuration

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/search/adaevolve/README.md

Configure the fundamental settings for AdaEvolve search, such as the number of islands, decay rate, and intensity ranges. Use this for standard evolutionary search setups.

```yaml
search:
  type: "adaevolve"
  database:
    num_islands: 2
    decay: 0.9
    intensity_min: 0.15
    intensity_max: 0.5
    migration_interval: 15
    migration_count: 5
    use_paradigm_breakthrough: true
    use_dynamic_islands: true
```

--------------------------------

### Run Discovery with GEPA Backend

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/extras/external/README.md

Execute SkyDiscover using the GEPA backend via the command line. Use 'gepa' as the search type to leverage GEPA's optimize_anything API.

```bash
uv run skydiscover-run initial_program.py evaluator.py \
  --config config.yaml --search gepa --iterations 100
```

--------------------------------

### Run Discovery with OpenEvolve Backend

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/extras/external/README.md

Execute SkyDiscover using the OpenEvolve backend via the command line. Ensure 'openevolve' is specified as the search type.

```bash
uv run skydiscover-run initial_program.py evaluator.py \
  --config config.yaml --search openevolve --iterations 100
```

--------------------------------

### Placeholder for Rewritten Program

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/context_builder/gepa_native/templates/full_rewrite_user_message.txt

This is a placeholder within the user message template where the rewritten program code is expected to be inserted. It indicates the start of the code block for the new program solution.

```python
# Your rewritten program here
```

--------------------------------

### Dockerfile for Evaluator Environment

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/quick-start.mdx

This Dockerfile sets up the Python environment for the evaluator. It installs necessary dependencies (scipy), copies the evaluation scripts and data, makes the entrypoint script executable, and defines the entrypoint.

```dockerfile
FROM python:3.12-slim
WORKDIR /benchmark

RUN pip install --no-cache-dir scipy

COPY evaluator.py pairs.json evaluate.sh ./
RUN chmod +x evaluate.sh

ENTRYPOINT ["./evaluate.sh"]

```

--------------------------------

### Run TriMul Benchmark

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/gpu_mode/trimul/README.md

Execute the TriMul benchmark using SkyDiscover. Ensure you are in the repository root and replace '[your_algorithm]' with your chosen algorithm.

```bash
uv run skydiscover-run \
  benchmarks/gpu_mode/trimul/initial_program.py \
  benchmarks/gpu_mode/trimul/evaluator.py \
  -c benchmarks/gpu_mode/trimul/config.yaml \
  -s [your_algorithm] -i 50
```

--------------------------------

### Run Benchmark Locally

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/gpu_mode/README.md

Execute a benchmark on your local GPU. Specify the initial program, evaluator, configuration file, algorithm, and number of iterations.

```bash
# Run on local GPU
uv run skydiscover-run \
  benchmarks/gpu_mode/trimul/initial_program.py \
  benchmarks/gpu_mode/trimul/evaluator.py \
  -c benchmarks/gpu_mode/trimul/config.yaml \
  -s [your_algorithm] \
  -i 50
```

--------------------------------

### Starting Solution with Evolve Block

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/README.md

Provide an initial Python solution marked with EVOLVE-BLOCK-START and EVOLVE-BLOCK-END comments to indicate the region that SkyDiscover should mutate. If markers are absent, the entire file is considered mutable.

```python
# EVOLVE-BLOCK-START
def solve(input_data):
    return input_data  # baseline — SkyDiscover will improve this
# EVOLVE-BLOCK-END
```

--------------------------------

### Seed Program for LLM Evolution

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/README.md

Defines the starting point for an LLM to evolve. Mark the region for evolution using EVOLVE-BLOCK-START and EVOLVE-BLOCK-END comments. For prompt optimization, use a plain .txt file without markers.

```python
# EVOLVE-BLOCK-START
def solve(input_data):
    return input_data  # LLM will improve this
# EVOLVE-BLOCK-END
```

--------------------------------

### Provide Initial Program for SkyDiscover

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/quick-start.mdx

This Python code defines a similarity function using Levenshtein distance. It serves as a starting point for SkyDiscover's evolution process, with the code between EVOLVE-BLOCK-START and EVOLVE-BLOCK-END being the target for mutation.

```python
# EVOLVE-BLOCK-START
def similarity(a: str, b: str) -> float:
    """
    Return a similarity score between 0.0 (unrelated) and 1.0 (identical)
    for two input strings.

    This should capture not just character-level similarity but also
    meaning — paraphrases should score high, negations should score low,
    and typos should be forgiven.

    Only use the Python standard library (no external packages).
    """
    # Baseline: normalized Levenshtein distance
    if a == b:
        return 1.0
    if not a or not b:
        return 0.0

    m, n = len(a), len(b)
    dp = list(range(n + 1))
    for i in range(1, m + 1):
        prev = dp[0]
        dp[0] = i
        for j in range(1, n + 1):
            temp = dp[j]
            if a[i - 1] == b[j - 1]:
                dp[j] = prev
            else:
                dp[j] = 1 + min(dp[j], dp[j - 1], prev)
            prev = temp

    max_len = max(m, n)
    return 1.0 - dp[n] / max_len
# EVOLVE-BLOCK-END
```

--------------------------------

### Initialize Discovery with GEPA Backend (Python)

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/extras/external/README.md

Initiate a discovery process with the GEPA backend using the Python API. Set the 'search' parameter to 'gepa'.

```python
from skydiscover import run_discovery
result = run_discovery(
    initial_program="initial_program.py",
    evaluator="evaluator.py",
    search="gepa",
    model="gpt-5",
    iterations=100,
)
```

--------------------------------

### Run SkyDiscover Evolution

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/docs/content/docs/getting-started/quick-start.mdx

This command initiates the SkyDiscover evolution process. It specifies the entry point script, the evaluator directory, a configuration file, the search strategy, and the number of iterations.

```bash
uv run skydiscover-run examples/text_similarity/initial_program.py \
  examples/text_similarity/evaluator/ \
  --config examples/text_similarity/config.yaml \
  --search best_of_n \
  --iterations 20
```

--------------------------------

### Run AdaEvolve Prompt Optimization

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/prompt_optimization/hotpot_qa/README.md

Executes the AdaEvolve (multi-island adaptive search) strategy for prompt optimization. This command starts the evolution process using a specified initial prompt, evaluator script, and configuration file.

```bash
uv run skydiscover-run \
  benchmarks/prompt_optimization/hotpot_qa/initial_prompt.txt \
  benchmarks/prompt_optimization/hotpot_qa/evaluator.py \
  -c benchmarks/prompt_optimization/hotpot_qa/config_adaevolve.yaml -i 100
```

--------------------------------

### System Prompt for Summarizing Search Algorithm Attempts

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/skydiscover/context_builder/evox/templates/batch_summary_prompt.txt

This is the system prompt used to guide the AI in summarizing previous search algorithm attempts. It defines the required output format and the specific information to extract for each program.

```text
===SYSTEM===
You are summarizing previous search algorithm attempts.

For EACH program, provide:

1) TECHNIQUE: (1-2 short sentences)
   Describe the parent and context selection approach.
   • Parent selection: what parents were typically chosen recently? 
   • Context selection: what context programs were used recently?
   • How is the label used?

2) KEY TAKEAWAYS: (3-5 bullet points)
   • Success or not (improved the downstream problem score or not). Compare scores across parent, context, and resulting child programs.
   • Highlight any notable patterns in the numbers (e.g., score gaps to SOTA, recent score trends 
   [slowing down, plateau, ...], regressions, etc.). 
   • Describe observable patterns in the technique used, parent/context selection (e.g., score ranges, diversity, selection frequency, best program inclusion/exclusion, context lineage diversity, empty context usage, ...) 
    and any correlations with child program outcomes. 
   • Label usage analysis:
      - Report the COUNT of DIVERGE_LABEL and REFINE_LABEL uses, and the child scores for each.
      - Label-guided generations are inherently high-variance. Individual attempts 
        often regress — this is expected. Breakthroughs typically require 3-5+ tries with varied parents and contexts.
        Do NOT treat individual regressions as evidence that "labels don't work".
      - If any labels were UNDER-UTILIZED during LONG STAGNATION, flag this as a MISSED OPPORTUNITY. 
      - Flag if the same parent program or same contexts are reused repeatedly with labels and no improvement is observed.

Keep under 70 words per program. Report facts only.
Ground all statements in the actual statistics provided.

===PER_PROGRAM===
TASK DESCRIPTION:
{task_description}

--------------------------------------------------------
SEARCH ALGORITHM CODE:
```python
{solution}
```

--------------------------------------------------------
STATS (includes execution trace with parent/context scores):
{db_stats_text}

===INSTRUCTIONS===
Below are {num_programs} PREVIOUS SEARCH ALGORITHM ATTEMPTS.

{combined_content}
--------------------------------------------------------
For EACH program, provide analysis in this EXACT format:

[PROGRAM 1]

SIGNALS OBSERVED: <...>

WHAT SEEMED TO HELP (ONLY LIST WHAT HELPED): <...>

POTENTIAL KEY INSIGHT: <...>

[PROGRAM 2]

...and so on. Use the exact [PROGRAM N] markers.

Keep under 60 words per program. Ground observations in actual statistics.
```

--------------------------------

### Run Math Benchmark

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/math/README.md

Execute a math benchmark, specifying the initial program, evaluator, configuration, algorithm, and number of iterations.

```bash
uv run skydiscover-run \
  benchmarks/math/signal_processing/initial_program.py \
  benchmarks/math/signal_processing/evaluator.py \
  -c benchmarks/math/signal_processing/config.yaml \
  -s [your_algorithm] \
  -i 100
```

--------------------------------

### Run Grayscale Benchmark

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/gpu_mode/grayscale/README.md

Execute the grayscale benchmark using SkyDiscover. Ensure you replace '[your_algorithm]' with your specific algorithm name.

```bash
uv run skydiscover-run \
  benchmarks/gpu_mode/grayscale/initial_program.py \
  benchmarks/gpu_mode/grayscale/evaluator.py \
  -c benchmarks/gpu_mode/grayscale/config.yaml \
  -s [your_algorithm] -i 50
```

--------------------------------

### Download Dataset for Cloudcast

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/ADRS/cloudcast/README.md

Download the network profiles and evaluation configurations required for Cloudcast benchmarks. Navigate to the specific directory before running.

```bash
cd benchmarks/ADRS/cloudcast
bash download_dataset.sh
```

--------------------------------

### Running a Benchmark with a Resolver

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/README.md

Command-line interface command to run a benchmark that utilizes a resolver. Note that no initial_program argument is provided, as the resolver handles program generation.

```bash
uv run skydiscover-run benchmarks/kernelbench/evaluator/ \
  -c benchmarks/kernelbench/config.yaml \
  --search adaevolve \
  --iterations 50
```

--------------------------------

### Run SkyDiscover Optimization (Repo Root)

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/ADRS/eplb/README.md

Executes the SkyDiscover optimization process from the repository root. Specify your algorithm using the -s flag and the number of iterations with -i.

```bash
uv run skydiscover-run \
  benchmarks/ADRS/eplb/initial_program.py \
  benchmarks/ADRS/eplb/evaluator.py \
  -c benchmarks/ADRS/eplb/config.yaml \
  -s [your_algorithm] \
  -i 100 \
  -o eplb_output
```

--------------------------------

### Run Sky Festival Benchmark with AdaEvolve

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/image_gen/README.md

Navigate to the sky_festival directory and run the benchmark using the AdaEvolve strategy. The output will be saved to sky_festival_output.

```bash
cd benchmarks/image_gen/sky_festival

# AdaEvolve
uv run skydiscover-run evaluator.py -c config.yaml -s adaevolve -o sky_festival_output
```

--------------------------------

### Run Cloudcast Optimization Benchmark

Source: https://github.com/skydiscover-ai/skydiscover/blob/main/benchmarks/ADRS/cloudcast/README.md

Execute the Cloudcast benchmark from the repository root. This command runs the initial program and evaluator scripts with specified configurations and algorithm.

```bash
uv run skydiscover-run \
  benchmarks/ADRS/cloudcast/initial_program.py \
  benchmarks/ADRS/cloudcast/evaluator.py \
  -c benchmarks/ADRS/cloudcast/config.yaml \
  -s [your_algorithm] \
  -i 100
```