AIDE ML

https://github.com/wecoai/aideml
Admin
AIDE ML is an LLM-driven agent that autonomously writes, evaluates, and improves machine learning...
Tokens:25,578
Snippets:347
Trust Score:8.2
License:MIT
Update:3 months ago
Show doc for...
Tokens:
Raw
### Quick Start CLI Usage

Source: https://github.com/wecoai/aideml/blob/main/README.md

Install the package, configure the API key, and execute an optimization task.

```bash
# 1  Install
pip install -U aideml

# 2  Set an LLM key
export OPENAI_API_KEY=<your‑key>  # https://platform.openai.com/api-keys

# 3  Run an optimisation
aide data_dir="example_tasks/house_prices" \
     goal="Predict the sales price for each house" \
     eval="RMSE between log‑prices"
```

--------------------------------

### Launch Web UI

Source: https://github.com/wecoai/aideml/blob/main/README.md

Install the package with Streamlit dependencies and start the web interface.

```bash
pip install -U aideml   # adds streamlit
cd aide/webui
streamlit run app.py
```

--------------------------------

### Minimal Setup

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/configuration.md

The basic command to initialize a project with a data directory and a goal.

```bash
aide data_dir=/path/to/data goal="Predict prices"
```

--------------------------------

### Development Installation

Source: https://github.com/wecoai/aideml/blob/main/README.md

Clone the repository and install in editable mode for development.

```bash
git clone https://github.com/WecoAI/aideml.git
cd aideml && pip install -e .
```

--------------------------------

### Experiment.__init__

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Experiment.md

Initializes a new experiment run with task configuration and workspace setup.

```APIDOC
## Experiment(data_dir, goal, eval=None)

### Description
Initializes a new experiment run, loads configuration, prepares the workspace, and initializes the agent and interpreter.

### Parameters
- **data_dir** (str) - Required - Path to the directory containing the data files for the task.
- **goal** (str) - Required - Description of the goal of the task in natural language.
- **eval** (str | None) - Optional - Optional description of how the solution should be evaluated.
```

--------------------------------

### Query Usage Examples

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Backend.md

Demonstrates text generation and tool calling using the query function.

```python
from aide.backend import query

# Text generation
response = query(
    system_message="You are a Python expert",
    user_message="Write a function to compute factorial",
    model="gpt-4-turbo",
    temperature=0.7
)

# Function calling
from aide.backend import FunctionSpec
spec = FunctionSpec(
    name="evaluate",
    description="Evaluate code quality",
    json_schema={
        "type": "object",
        "properties": {
            "score": {"type": "number"},
            "feedback": {"type": "string"}
        },
        "required": ["score", "feedback"]
    }
)

result = query(
    system_message="Rate this code",
    user_message="def add(a, b): return a + b",
    model="claude-3-sonnet",
    func_spec=spec
)
# result is now: {"score": 9.0, "feedback": "..."}
```

--------------------------------

### Initialize FunctionSpec example

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Backend.md

Demonstrates creating a FunctionSpec instance with a JSON schema for code review parameters.

```python
from aide.backend import FunctionSpec

review_spec = FunctionSpec(
    name="submit_review",
    description="Submit a review of code execution results",
    json_schema={
        "type": "object",
        "properties": {
            "is_bug": {
                "type": "boolean",
                "description": "true if the code has a bug"
            },
            "metric": {
                "type": "number",
                "description": "the validation metric value"
            }
        },
        "required": ["is_bug", "metric"]
    }
)
```

--------------------------------

### Journal Usage Example

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Journal.md

Demonstrates creating a journal, appending nodes, simulating execution, and querying results.

```python
from aide.journal import Journal, Node
from aide.utils.metric import MetricValue

journal = Journal()

# Create and append initial draft
node1 = Node(code="import pandas as pd\ndf = pd.read_csv('./input/data.csv')")
journal.append(node1)

# Simulate execution
node1.metric = MetricValue(0.85, maximize=True)
node1.is_buggy = False

# Create improvement on node1
node2 = Node(code="...", parent=node1, plan="Add feature engineering")
journal.append(node2)

# Query the journal
print(f"Total nodes: {len(journal)}")
print(f"Draft nodes: {len(journal.draft_nodes)}")
print(f"Good nodes: {len(journal.good_nodes)}")
best = journal.get_best_node()
print(f"Best solution: {best.metric}")
```

--------------------------------

### Install AIDE ML package

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Install the package using pip. Requires Python 3.10 or higher.

```bash
pip install -U aideml
```

--------------------------------

### LLM Model Selection

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/configuration.md

Examples for configuring different LLM providers, including Anthropic and local Ollama instances.

```bash
# Anthropic
aide data_dir=data goal="..." agent.code.model=claude-3-sonnet agent.feedback.model=claude-3-haiku

# Fully local (Ollama)
export OPENAI_BASE_URL=http://localhost:11434/v1
aide data_dir=data goal="..." agent.code.model=qwen2.5 agent.feedback.model=qwen2.5
```

--------------------------------

### Interpreter Usage Example

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Interpreter.md

Complete workflow demonstrating initialization, code execution, result handling, and session cleanup.

```python
from aide.interpreter import Interpreter
from pathlib import Path

interp = Interpreter(
    working_dir="/tmp/workspace",
    timeout=60,  # 1 minute timeout
)

result = interp.run("""
import pandas as pd
df = pd.read_csv("./input/data.csv")
print(f"Loaded {len(df)} rows")
metric = df["target"].mean()
print(f"Mean target: {metric}")
""")

if result.exc_type is None:
    print(f"Success! Output:\n{result.term_out}")
else:
    print(f"Error: {result.exc_type}")
    print(f"Exception info: {result.exc_info}")
    print(f"Traceback: {result.exc_stack}")

interp.cleanup_session()
```

--------------------------------

### Top-Level Exports and Usage

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Public API definitions and a standard usage example for the Experiment class.

```python
from .agent import Agent
from .interpreter import Interpreter
from .journal import Journal, Node
from .utils.config import Config

class Solution:
    code: str
    valid_metric: float

class Experiment:
    def __init__(self, data_dir: str, goal: str, eval: str | None = None)
    def run(self, steps: int) -> Solution
```

```python
import aide

exp = aide.Experiment(
    data_dir="path/to/data",
    goal="Predict churn",
    eval="AUC"
)
result = exp.run(steps=20)
```

--------------------------------

### Assign WorstMetricValue in AIDE ML

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/MetricValue.md

Example of assigning WorstMetricValue to a node when code execution fails or is buggy.

```python
from aide.journal import Node
from aide.utils.metric import WorstMetricValue

# After execution fails or LLM reviews detect a bug:
buggy_node = Node(code="...")
buggy_node.is_buggy = True
buggy_node.metric = WorstMetricValue()

# When comparing for best solution, this node will be ignored
```

--------------------------------

### Initialize and run an Agent

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Demonstrates setting up an Agent with a task description, configuration, and journal, then executing steps using an interpreter callback.

```python
from aide import Experiment, Agent, Journal
from aide.interpreter import Interpreter
from aide.utils.config import load_cfg

cfg = load_cfg()
journal = Journal()
agent = Agent(
    task_desc="Predict house prices given features",
    cfg=cfg,
    journal=journal
)
interpreter = Interpreter(cfg.workspace_dir, timeout=3600)

# Run 5 steps
for _ in range(5):
    agent.step(exec_callback=interpreter.run)
```

--------------------------------

### Execute AIDE ML Experiment

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Experiment.md

Demonstrates initializing an experiment and running it for a specified number of steps.

```python
import aide
import logging

# Configure logging to see agent progress
logging.basicConfig(level=logging.INFO)

# Create an experiment
exp = aide.Experiment(
    data_dir="path/to/data",
    goal="Predict customer churn within 30 days",
    eval="ROC-AUC score on validation set"
)

# Run for 20 optimization steps
best_solution = exp.run(steps=20)

print(f"Best metric: {best_solution.valid_metric}")
print(f"Best code:\n{best_solution.code}")
```

--------------------------------

### Configure AideML with Python

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Initialize configuration using default settings or programmatic overrides.

```python
from aide.utils.config import load_cfg, prep_cfg, _load_cfg

# Load defaults with CLI overrides
cfg = load_cfg()

# Or: Programmatically construct config
raw_cfg = _load_cfg(use_cli_args=False)
raw_cfg.data_dir = "data/titanic"
raw_cfg.goal = "Predict survival"
raw_cfg.eval = "Accuracy"
raw_cfg.agent.steps = 10
cfg = prep_cfg(raw_cfg)
```

--------------------------------

### Initialize AIDE Experiment

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Import the main public API from the top-level package.

```python
import aide
exp = aide.Experiment(...)
```

--------------------------------

### Agent.__init__

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Initializes the agent with a task description, configuration, and a journal instance.

```APIDOC
## Agent.__init__(task_desc, cfg, journal)

### Description
Initializes the agent with task description and configuration.

### Parameters
- **task_desc** (str) - Required - The task description in markdown format.
- **cfg** (Config) - Required - Configuration object from config.yaml and CLI overrides.
- **journal** (Journal) - Required - The Journal instance to record all solutions.
```

--------------------------------

### Prepare configuration with prep_cfg

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Validates and prepares a raw configuration object, including path resolution and directory creation.

```python
from aide.utils.config import _load_cfg, prep_cfg

# Load raw config (no CLI args)
raw_cfg = _load_cfg(use_cli_args=False)
raw_cfg.data_dir = "/path/to/data"
raw_cfg.goal = "Predict prices"

# Prepare for use
cfg = prep_cfg(raw_cfg)
print(cfg.log_dir)  # logs/0-<random-name>
print(cfg.workspace_dir)  # workspaces/0-<random-name>
```

--------------------------------

### Truncate long strings

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Response.md

Truncates strings exceeding a threshold by keeping a specified number of characters from the start and end.

```python
from aide.utils.response import trim_long_string

long_output = "START" + "x" * 10000 + "END"
trimmed = trim_long_string(long_output)
# Returns: "STARTxxxx...xxxx [9990 characters truncated] ...xxxxEND"
```

--------------------------------

### Docker Deployment

Source: https://github.com/wecoai/aideml/blob/main/README.md

Build and run the AIDE container with mounted volumes for logs, workspaces, and data.

```bash
docker build -t aide .
docker run -it --rm \
  -v "${LOGS_DIR:-$(pwd)/logs}:/app/logs" \
  -v "${WORKSPACE_BASE:-$(pwd)/workspaces}:/app/workspaces" \
  -v "$(pwd)/aide/example_tasks:/app/data" \
  -e OPENAI_API_KEY="your-actual-api-key" \
  aide data_dir=/app/data/house_prices goal="Predict price" eval="RMSE"
```

--------------------------------

### Configure AIDE via YAML

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Essential configuration settings for data directories, goals, metrics, and LLM model selection.

```yaml
data_dir: /path/to/data           # Input data directory (required)
goal: "Predict sales price"        # Task goal (required)
eval: "RMSE"                       # Evaluation metric (optional)

agent.steps: 20                    # Optimization iterations
agent.code.model: gpt-4-turbo      # LLM for code generation
agent.feedback.model: gpt-4-mini   # LLM for evaluation
```

--------------------------------

### trim_long_string(string: str, threshold: int = 5100, k: int = 2500) -> str

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Response.md

Truncates long strings while preserving the start and end portions.

```APIDOC
## trim_long_string(string: str, threshold: int = 5100, k: int = 2500) -> str

### Description
Truncates long strings while preserving start and end segments if the length exceeds the threshold.

### Parameters
- **string** (str) - Required - String to potentially truncate.
- **threshold** (int) - Optional - Length at which truncation begins (default: 5100).
- **k** (int) - Optional - Number of characters to keep from start and end (default: 2500).

### Returns
- **str** - Original string if length <= threshold, otherwise first k + last k chars with middle replaced.
```

--------------------------------

### Initialize Agent

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Constructor for the Agent class, requiring task description, configuration, and a journal instance.

```python
class Agent:
    def __init__(
        self,
        task_desc: str,
        cfg: Config,
        journal: Journal,
    )
```

--------------------------------

### Manage Configuration and Workspace

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Handles loading, preparing, and saving configurations for agents and experiments.

```python
class Config: ...
class AgentConfig: ...
class ExecConfig: ...
class StageConfig: ...
class SearchConfig: ...

def load_cfg(path: Path = Path(__file__).parent / "config.yaml") -> Config
def prep_cfg(cfg: Config) -> Config
def _load_cfg(path: Path = ..., use_cli_args: bool = True) -> Config
def load_task_desc(cfg: Config) -> str
def prep_agent_workspace(cfg: Config) -> None
def save_run(cfg: Config, journal: Journal) -> None
def print_cfg(cfg: Config) -> None
```

--------------------------------

### aide.Experiment

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

The primary entry point for running machine learning experiments.

```APIDOC
## class aide.Experiment

### Description
Initializes and executes an experiment to achieve a specific goal using provided data.

### Methods
- **__init__(data_dir: str, goal: str, eval: str | None = None)**: Initializes the experiment.
- **run(steps: int) -> Solution**: Executes the experiment for a specified number of steps and returns the final solution.
```

--------------------------------

### Set Optional API Keys and Base URL

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Environment variables for alternative LLM providers and custom API base URLs.

```bash
export ANTHROPIC_API_KEY="..."
export GEMINI_API_KEY="..."
export OPENROUTER_API_KEY="..."
export OPENAI_BASE_URL="http://localhost:11434/v1"  # For local servers
```

--------------------------------

### Load Configuration

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Configuration can be loaded automatically from files and CLI overrides or initialized manually.

```python
# Automatic
cfg = aide.utils.config.load_cfg()  # Loads config.yaml + CLI overrides

# Manual
from aide.utils.config import Config
cfg = Config(data_dir=..., goal=..., ...)
```

--------------------------------

### prep_cfg(cfg: Config) -> Config

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Validates and prepares the configuration object after loading.

```APIDOC
## prep_cfg

### Description
Validates and prepares the configuration after loading. This includes path resolution, directory creation, and schema validation.

### Parameters
- **cfg** (Config) - Required - The configuration object to prepare.

### Returns
- **Config** - The prepared configuration object.

### Example
```python
from aide.utils.config import _load_cfg, prep_cfg

# Load raw config
raw_cfg = _load_cfg(use_cli_args=False)
raw_cfg.data_dir = "/path/to/data"
raw_cfg.goal = "Predict prices"

# Prepare for use
cfg = prep_cfg(raw_cfg)
```
```

--------------------------------

### Execute AIDE ML via CLI

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Run the agent from the command line by setting the API key and providing configuration parameters.

```bash
export OPENAI_API_KEY=your-key
aide data_dir="data" goal="Predict prices" eval="RMSE" agent.steps=20
```

--------------------------------

### Initialize Experiment Class

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Experiment.md

Defines the constructor signature for the Experiment class.

```python
class Experiment:
    def __init__(self, data_dir: str, goal: str, eval: str | None = None)
```

--------------------------------

### Generate directory preview in Python

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Creates a textual summary of a directory structure and its contents.

```python
def generate(
    base_path: Path | str,
    include_file_details: bool = True,
    simple: bool = False
) -> str
```

```python
from aide.utils.data_preview import generate
from pathlib import Path

preview = generate("./my_data")
print(preview)
# Output:
# ```
# train.csv (150 lines)
# test.csv (50 lines)
# metadata.json (1.2 KB)
# ```
#
# -> train.csv has 1000 rows and 20 columns.
# The columns are: feature_1, feature_2, ..., target
#
# -> test.csv has 500 rows and 19 columns.
# The columns are: feature_1, feature_2, ...
```

--------------------------------

### Execute Run and Visualize Journal

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Entry point for the aide command and utilities for visualizing journals as rich trees.

```python
def run()
def journal_to_rich_tree(journal: Journal) -> Tree
```

--------------------------------

### Define Task Description File

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Use a markdown file to define task goals and data context, then execute AIDE referencing that file.

```markdown
# Task Description

Goal: Predict house prices from features
Data: House sales dataset with 1000 samples
Evaluation: RMSE on held-out test set
```

```bash
aide data_dir=data desc_file=task.md
```

--------------------------------

### Select LLMs via CLI

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Specify different LLM providers and models directly through command-line arguments.

```bash
# Use Claude for code generation
aide data_dir=data goal="..." agent.code.model=claude-3-sonnet

# Use local Ollama
export OPENAI_BASE_URL=http://localhost:11434/v1
aide data_dir=data goal="..." \
  agent.code.model=qwen2.5 \
  agent.feedback.model=qwen2.5
```

--------------------------------

### Interpreter.__init__

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Interpreter.md

Initializes a new Interpreter instance with a specified working directory and execution constraints.

```APIDOC
## Interpreter.__init__

### Description
Initializes the Interpreter, which manages a long-lived child process for code execution.

### Parameters
- **working_dir** (Path | str) - Required - Working directory for code execution.
- **timeout** (int) - Optional - Execution timeout in seconds (default: 3600).
- **format_tb_ipython** (bool) - Optional - Use IPython format for exception tracebacks (default: False).
- **agent_file_name** (str) - Optional - Filename to write code to before execution (default: "runfile.py").
```

--------------------------------

### Package Structure

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

The directory layout of the AIDE ML project.

```text
aide/
├── __init__.py                  # Main entry point
├── agent.py                     # Agent tree-search implementation
├── interpreter.py               # Python code executor
├── journal.py                   # Solution tree data structure
├── journal2report.py            # Report generation from journal
├── run.py                       # CLI entry point
├── backend/
│   ├── __init__.py             # Backend routing and query interface
│   ├── utils.py                # FunctionSpec, prompt compilation
│   ├── backend_openai.py       # OpenAI API client
│   ├── backend_anthropic.py    # Anthropic API client
│   ├── backend_gemini.py       # Google Gemini API client
│   └── backend_openrouter.py   # OpenRouter API client
└── utils/
    ├── __init__.py             # Utility exports
    ├── config.py               # Configuration loading and validation
    ├── config.yaml             # Default configuration
    ├── data_preview.py         # Data preview generation
    ├── metric.py               # Metric value and comparison
    ├── response.py             # Response parsing utilities
    ├── serialize.py            # JSON serialization
    ├── tree_export.py          # HTML tree visualization
    └── viz_templates/          # HTML templates
```

--------------------------------

### generate(base_path, include_file_details=True, simple=False)

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Generates a complete textual preview of a directory structure and its files.

```APIDOC
## generate(base_path, include_file_details=True, simple=False)

### Description
Generates a complete textual preview of a directory structure and its files.

### Parameters
- **base_path** (Path | str) - Required - Root directory to preview.
- **include_file_details** (bool) - Optional (Default: True) - Include detailed previews of individual files.
- **simple** (bool) - Optional (Default: False) - Use simplified previews if content is long.

### Returns
- **str** - Markdown-formatted preview string.
```

--------------------------------

### Run Optimization Experiment

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Execute an optimization experiment with a specified number of steps.

```python
exp = aide.Experiment(data_dir="data", goal="...", eval="...")
result = exp.run(steps=100)  # 100 iterations
```

--------------------------------

### Extended Run Configuration

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/configuration.md

A comprehensive command including evaluation metrics and specific agent parameters.

```bash
aide data_dir=/path/to/data \
     goal="Build churn prediction model" \
     eval="ROC-AUC on validation set" \
     agent.steps=50 \
     agent.search.num_drafts=10 \
     agent.code.model=gpt-4-turbo
```

--------------------------------

### Experiment.run

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Experiment.md

Executes the optimization loop for a specified number of steps.

```APIDOC
## run(steps)

### Description
Runs the optimization loop for the specified number of steps to improve the solution.

### Parameters
- **steps** (int) - Required - Number of improvement iterations to perform.

### Returns
- **Solution** (dataclass) - Contains the best code found (str) and the valid_metric (float).
```

--------------------------------

### Cost-Optimized Configuration

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/configuration.md

Reducing resource usage by selecting smaller models and limiting agent steps and drafts.

```bash
aide data_dir=data goal="..." \
     agent.code.model=gpt-4-mini \
     agent.feedback.model=gpt-4-mini \
     agent.steps=10 \
     agent.search.num_drafts=3
```

--------------------------------

### Local LLM Configuration

Source: https://github.com/wecoai/aideml/blob/main/README.md

Configure AIDE to use a local LLM via Ollama, optionally using it for both code generation and feedback.

```bash
export OPENAI_BASE_URL="http://localhost:11434/v1"
aide agent.code.model="qwen2.5" data_dir=… goal=… eval=…
```

```bash
export OPENAI_BASE_URL="http://localhost:11434/v1"
aide agent.code.model="qwen2.5" agent.feedback.model="qwen2.5" data_dir=… goal=… eval=…
```

--------------------------------

### Define AIDE ML configuration in YAML

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/configuration.md

Use this structure in aide/utils/config.yaml to define experiment parameters, agent settings, and execution constraints.

```yaml
# Data input
data_dir: null              # Required: path to data directory
desc_file: null             # Optional: path to task description file
goal: null                  # Optional: task goal (alternative to desc_file)
eval: null                  # Optional: evaluation metric description

# Directories
log_dir: logs               # Base directory for experiment logs
workspace_dir: workespaces  # Base directory for agent workspaces

# Data handling
preprocess_data: True       # Extract archives (.zip, .tar, etc.)
copy_data: True             # Copy data vs symlink (copy recommended)

exp_name: null              # Auto-generated if not provided

# Code execution settings
exec:
  timeout: 3600             # Execution timeout (seconds)
  agent_file_name: runfile.py  # Filename for agent code
  format_tb_ipython: False   # Use IPython traceback format

# Report generation
generate_report: True
report:
  model: gpt-4.1
  temp: 1.0

# Agent configuration
agent:
  steps: 20                 # Number of iterations
  k_fold_validation: 5      # CV folds (1 to disable)
  expose_prediction: False  # Ask for predict() function
  data_preview: True        # Provide data preview to agent
  
  code:
    model: o4-mini
    temp: 0.5
  
  feedback:
    model: gpt-4.1-mini
    temp: 0.5
  
  search:
    max_debug_depth: 3
    debug_prob: 0.5
    num_drafts: 5
```

--------------------------------

### Configure Performance Strategies

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

CLI flags to adjust agent behavior for speed, quality, or local execution.

```bash
aide ... agent.steps=10 agent.search.num_drafts=3 agent.code.model=gpt-4-mini
```

```bash
aide ... agent.steps=100 agent.search.num_drafts=10 agent.search.debug_prob=0.7
```

```bash
# Requires Ollama running on localhost:11434
export OPENAI_BASE_URL=http://localhost:11434/v1
aide ... agent.code.model=qwen2.5 agent.feedback.model=qwen2.5
```

--------------------------------

### Load configuration with load_cfg

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Loads configuration from a YAML file and merges it with CLI arguments.

```python
from aide.utils.config import load_cfg

# Load default config with CLI overrides
cfg = load_cfg()

# Access settings
print(f"Steps: {cfg.agent.steps}")
print(f"Code model: {cfg.agent.code.model}")
```

--------------------------------

### Advanced CLI Configuration

Source: https://github.com/wecoai/aideml/blob/main/README.md

Customize the coding model and number of optimization steps via command-line arguments.

```bash
aide agent.code.model="claude-4-sonnet" \
     agent.steps=50 \
     data_dir=… goal=… eval=…
```

--------------------------------

### Default Configuration File

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

The default configuration file located at aide/utils/config.yaml defines data directories, execution parameters, and agent settings.

```yaml
data_dir: null  # Required: path to data directory

# Task description (provide one):
desc_file: null  # Path to plaintext task description file
goal: null       # Natural language task goal
eval: null       # Optional: evaluation metric description

# Directories
log_dir: logs
workspace_dir: workspaces

# Data handling
preprocess_data: True   # Extract .zip, .tar, etc.
copy_data: True         # Copy data vs symlink (recommended)

exp_name: null  # Auto-generated if not provided

# Code execution
exec:
  timeout: 3600              # 1 hour
  agent_file_name: runfile.py
  format_tb_ipython: False

generate_report: True

# Report generation (uses LLM)
report:
  model: gpt-4.1
  temp: 1.0

# Agent configuration
agent:
  steps: 20  # Number of optimization iterations
  k_fold_validation: 5
  expose_prediction: False
  data_preview: True

  # Code generation LLM
  code:
    model: o4-mini
    temp: 0.5

  # Result evaluation LLM
  feedback:
    model: gpt-4.1-mini
    temp: 0.5

  # Tree-search algorithm
  search:
    max_debug_depth: 3
    debug_prob: 0.5
    num_drafts: 5
```

--------------------------------

### Agent.plan_and_code_query

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Generates a natural language plan and Python code in a single LLM call.

```APIDOC
## Agent.plan_and_code_query(prompt, retries=3)

### Description
Generates a natural language plan and Python code in a single LLM call.

### Parameters
- **prompt** (Any) - Required - Compiled prompt dict/list (will be converted to markdown)
- **retries** (int) - Optional - Number of retries if extraction fails (default: 3)

### Returns
- **tuple[str, str]** - (natural_language_plan, python_code)
```

--------------------------------

### Access Agent Internals Programmatically

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Retrieve configuration settings, inspect agent state, and manually execute steps within an Experiment.

```python
exp = aide.Experiment(...)

# Access configuration
print(f"Model: {exp.cfg.agent.code.model}")
print(f"Timeout: {exp.cfg.exec.timeout}")

# Access agent state
print(f"Agent data preview: {exp.agent.data_preview}")

# Manually run steps
for _ in range(5):
    exp.agent.step(exec_callback=exp.interpreter.run)
    
# Access results
for node in exp.journal.good_nodes:
    print(f"Node {node.id}: metric={node.metric}, stage={node.stage_name}")
```

--------------------------------

### Define Config

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Defines the root experiment configuration class containing all settings.

```python
@dataclass
class Config(Hashable):
    data_dir: Path
    desc_file: Path | None
    goal: str | None
    eval: str | None
    log_dir: Path
    workspace_dir: Path
    preprocess_data: bool
    copy_data: bool
    exp_name: str
    exec: ExecConfig
    generate_report: bool
    report: StageConfig
    agent: AgentConfig
```

--------------------------------

### preview_csv(path, file_name, simple=True)

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Generates a detailed preview of a CSV file, including row/column counts and column statistics.

```APIDOC
## preview_csv(path, file_name, simple=True)

### Description
Generates a detailed preview of a CSV file. Simple mode shows column names; detailed mode includes statistics per column.

### Parameters
- **path** (Path) - Required - Path to the CSV file.
- **file_name** (str) - Required - File name to display in preview.
- **simple** (bool) - Optional (Default: True) - Use simplified preview (column names only).

### Returns
- **str** - Markdown-formatted preview.
```

--------------------------------

### Specify ML tasks with AIDE

Source: https://github.com/wecoai/aideml/blob/main/README.md

Use the aide command to define a data directory, goal, and evaluation metric for automated machine learning tasks.

```bash
aide data_dir=…  goal="Predict churn"  eval="AUROC"
```

--------------------------------

### Custom Task Description File

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/configuration.md

Defining tasks in an external text file and referencing it during execution.

```text
Task: Predict house prices
Data: Historical house sales with features like square footage, beds, baths
Evaluation: RMSE on test set
```

```bash
aide data_dir=/path/to/data desc_file=task_desc.txt
```

--------------------------------

### Configure Model Optimization Profiles

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/configuration.md

YAML configuration snippets for setting agent models based on performance, cost, or local execution requirements.

```yaml
agent.code.model: gpt-4-turbo
agent.feedback.model: gpt-4.1-mini
```

```yaml
agent.code.model: gpt-4-mini
agent.feedback.model: gpt-4-mini
```

```yaml
agent.code.model: claude-3-opus
agent.feedback.model: claude-3-sonnet
```

```yaml
agent.code.model: qwen2.5          # Via Ollama or local server
agent.feedback.model: qwen2.5
```

--------------------------------

### Generate workspace data preview

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Generates a summary of the workspace input directory for inclusion in agent prompts.

```python
from aide.utils.data_preview import generate

# Agent preview
data_preview = generate(cfg.workspace_dir / "input")

# Included in agent prompt if agent.data_preview=True
```

--------------------------------

### Initialize Interpreter

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Interpreter.md

Constructor for the Interpreter class, defining the working directory and execution constraints.

```python
class Interpreter:
    def __init__(
        self,
        working_dir: Path | str,
        timeout: int = 3600,
        format_tb_ipython: bool = False,
        agent_file_name: str = "runfile.py",
    )
```

--------------------------------

### Run Optimization Loop

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Experiment.md

Defines the method signature for executing the optimization process.

```python
def run(self, steps: int) -> Solution
```

--------------------------------

### Create Interpreter Process

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Interpreter.md

Spawns a new child process and initializes inter-process communication queues.

```python
def create_process(self) -> None
```

--------------------------------

### aide.backend.query

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Interface for querying various LLM providers.

```APIDOC
## function aide.backend.query

### Description
Sends a query to a configured LLM backend.

### Parameters
- **system_message** (PromptType) - Optional
- **user_message** (PromptType) - Optional
- **model** (str) - Required
- **temperature** (float) - Optional
- **max_tokens** (int) - Optional
- **func_spec** (FunctionSpec) - Optional
```

--------------------------------

### Draft Initial Solution

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Generates an initial solution design and code, returning a new Node without a parent.

```python
def _draft(self) -> Node
```

--------------------------------

### load_cfg(path: Path = Path(__file__).parent / "config.yaml") -> Config

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Loads configuration from a YAML file and merges it with CLI arguments.

```APIDOC
## load_cfg

### Description
Loads configuration from a YAML file and merges with CLI arguments. It performs validation and setup via prep_cfg.

### Parameters
- **path** (Path) - Optional - Path to YAML config file. Defaults to aide/utils/config.yaml.

### Returns
- **Config** - Validated and processed configuration object.

### Example
```python
from aide.utils.config import load_cfg

# Load default config with CLI overrides
cfg = load_cfg()

# Access settings
print(f"Steps: {cfg.agent.steps}")
print(f"Code model: {cfg.agent.code.model}")
```
```

--------------------------------

### Compare MetricValue instances

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/MetricValue.md

Demonstrates how MetricValue handles different optimization objectives and mixed types.

```python
from aide.utils.metric import MetricValue

# Accuracy (higher is better)
acc1 = MetricValue(0.85, maximize=True)
acc2 = MetricValue(0.90, maximize=True)
assert acc2 > acc1  # True

# MSE (lower is better)
mse1 = MetricValue(0.1, maximize=False)
mse2 = MetricValue(0.05, maximize=False)
assert mse2 > mse1  # True (lower is better)

# Mixed types
worst = MetricValue(None, maximize=True)
assert acc1 > worst  # True
```

--------------------------------

### Compare WorstMetricValue

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/MetricValue.md

Shows how WorstMetricValue compares against standard MetricValue instances.

```python
from aide.utils.metric import MetricValue, WorstMetricValue

valid = MetricValue(0.85, maximize=True)
invalid = WorstMetricValue()

assert valid > invalid  # True
assert invalid > valid  # False
assert invalid.is_worst  # True
```

--------------------------------

### load_task_desc(cfg: Config) -> str

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Loads the task description from a file or generates it from the configuration goal.

```APIDOC
## load_task_desc

### Description
Loads task description from a file if provided, or constructs it from the goal and evaluation settings in the configuration.

### Parameters
- **cfg** (Config) - Required - Configuration object.

### Returns
- **str** - Task description string.
```

--------------------------------

### Import Public API Modules

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Standard imports for core AideML components and advanced backend utilities.

```python
from aide import Experiment, Solution
from aide import Agent, Journal, Node, Interpreter

# Advanced
from aide.backend import query, FunctionSpec
from aide.utils.config import Config
from aide.utils.metric import MetricValue, WorstMetricValue
```

--------------------------------

### preview_json(path, file_name)

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Generates a preview of a JSON file using an auto-generated JSON Schema.

```APIDOC
## preview_json(path, file_name)

### Description
Generates a preview of a JSON file using auto-generated JSON Schema.

### Parameters
- **path** (Path) - Required - Path to the JSON file.
- **file_name** (str) - Required - File name to display.

### Returns
- **str** - JSON schema in JSON format with indentation.
```

--------------------------------

### Update Data Preview

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Generates and stores a textual preview of the data directory.

```python
def update_data_preview(self)
```

--------------------------------

### Generate file tree representation

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Visualizes a directory structure with file sizes and recursive subdirectories.

```python
def file_tree(path: Path, depth: int = 0) -> str
```

```text
train/
    data.csv (5.2 MB)
    metadata.json (3.4 KB)
    ... and 8 other files
    images/
        img001.jpg (256 KB)
        img002.jpg (241 KB)
        ... and 2998 other files
```

--------------------------------

### OpenAI Backend query()

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Backend.md

Handles OpenAI API calls including GPT-4, GPT-3.5, and o-series models with built-in exponential backoff.

```APIDOC
## query(system_message, user_message, func_spec=None, **model_kwargs)

### Description
Handles OpenAI API calls for models such as GPT-4, GPT-3.5, o1, and o4. Implements exponential backoff for rate limit management.

### Parameters
- **system_message** (str | None) - Optional - The system prompt for the model.
- **user_message** (str | None) - Optional - The user input message.
- **func_spec** (FunctionSpec | None) - Optional - Function specification for tool calling.
- **model_kwargs** (dict) - Optional - Additional keyword arguments passed to the model.

### Returns
- **tuple[OutputType, float, int, int, dict]** - Returns a tuple containing the output, latency, input tokens, output tokens, and metadata.
```

--------------------------------

### Agent.step

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Executes one iteration of the tree search process.

```APIDOC
## Agent.step(exec_callback)

### Description
Executes one iteration of the tree search: selects a node, generates code, executes it, and evaluates the result.

### Parameters
- **exec_callback** (ExecCallbackType) - Required - Callable that executes code: (code: str, reset: bool) -> ExecutionResult
```

--------------------------------

### Python API Integration

Source: https://github.com/wecoai/aideml/blob/main/README.md

Run experiments programmatically by initializing an Experiment object and executing the run method.

```python
import aide
import logging

def main():
    logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    aide_logger = logging.getLogger("aide")
    aide_logger.setLevel(logging.INFO)
    print("Starting experiment...")
    exp = aide.Experiment(
        data_dir="example_tasks/bitcoin_price",  # replace this with your own directory
        goal="Build a time series forecasting model for bitcoin close price.",  # replace with your own goal description
        eval="RMSLE"  # replace with your own evaluation metric
    )

    best_solution = exp.run(steps=2)

    print(f"Best solution has validation metric: {best_solution.valid_metric}")
    print(f"Best solution code: {best_solution.code}")
    print("Experiment finished.")

if __name__ == '__main__':
    main()
```

--------------------------------

### Define recognized file types

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Sets of file extensions categorized by how they are processed by the preview utility.

```python
code_files = {".py", ".sh", ".yaml", ".yml", ".md", ".html", ".xml", ".log", ".rst"}
```

```python
plaintext_files = {".txt", ".csv", ".json", ".tsv"} | code_files
```

--------------------------------

### Tune Agent Parameters

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Adjust agent execution settings like step counts and search depth for complex tasks.

```bash
# Longer runs with more drafts and debugging
aide data_dir=data goal="..." \
  agent.steps=100 \
  agent.search.num_drafts=10 \
  agent.search.debug_prob=0.7 \
  agent.search.max_debug_depth=5
```

--------------------------------

### Generate Data Previews

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Functions for generating file trees and previews for CSV and JSON data files.

```python
def generate(
    base_path: Path | str,
    include_file_details: bool = True,
    simple: bool = False
) -> str

def file_tree(path: Path, depth: int = 0) -> str
def preview_csv(path: Path, file_name: str, simple: bool = True) -> str
def preview_json(path: Path, file_name: str) -> str
def get_file_len_size(f: Path) -> tuple[int, str]
```

--------------------------------

### aide.Interpreter

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Executes Python code in a controlled environment.

```APIDOC
## class aide.Interpreter

### Description
Provides an interface to execute code and manage session state.

### Methods
- **run(code: str, reset_session: bool = True) -> ExecutionResult**: Executes the provided code string.
- **create_process()**: Initializes the execution process.
- **cleanup_session()**: Cleans up the current execution session.
```

--------------------------------

### generate

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/DataPreview.md

Generates a data preview for a specified directory, typically used to provide context to an agent.

```APIDOC
## generate

### Description
Generates a preview of the data within a workspace directory. This preview is often included in agent prompts to provide a 'Data Overview' when configured.

### Signature
`generate(path: Path)`

### Parameters
- **path** (Path) - Required - The directory path to generate a preview for.

### Example
```python
from aide.utils.data_preview import generate
data_preview = generate(cfg.workspace_dir / "input")
```
```

--------------------------------

### aide.Agent

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Manages the tree-search implementation for agent-based task solving.

```APIDOC
## class aide.Agent

### Description
Handles the agent's decision-making process, including searching, planning, and coding.

### Methods
- **step(exec_callback: ExecCallbackType)**: Performs a single agent step.
- **search_policy() -> Node | None**: Returns the current node based on the search policy.
- **plan_and_code_query(prompt: Any, retries: int = 3) -> tuple[str, str]**: Generates a plan and code based on the provided prompt.
- **update_data_preview()**: Refreshes the internal data preview.
```

--------------------------------

### format_code(code: str) -> str

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Response.md

Formats Python code using the Black code formatter.

```APIDOC
## format_code(code: str) -> str

### Description
Formats Python code using the Black code formatter. Returns original code if formatting fails.

### Parameters
- **code** (str) - Required - Python code to format.

### Returns
- **str** - Formatted code, or original code if Black fails.
```

--------------------------------

### Import Extension Modules

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Imports for internal journal management, interpreter execution results, and code formatting utilities.

```python
from aide.journal import Journal, Node, InteractiveSession
from aide.interpreter import ExecutionResult
from aide.utils.response import extract_code, wrap_code, format_code
from aide.utils.data_preview import generate as preview_data
```

--------------------------------

### wrap_code(code: str, lang: str = "python") -> str

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Response.md

Wraps a string of code with markdown code block syntax.

```APIDOC
## wrap_code(code: str, lang: str = "python") -> str

### Description
Wraps code with markdown code block syntax.

### Parameters
- **code** (str) - Required - Code to wrap.
- **lang** (str) - Optional - Language identifier for the code block (default: "python").

### Returns
- **str** - Code wrapped in triple backticks with language identifier.
```

--------------------------------

### compile_prompt_to_md

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Backend.md

Converts prompt dictionaries, lists, or strings into markdown format for improved LLM readability.

```APIDOC
## compile_prompt_to_md(prompt: PromptType, _header_depth: int = 1) -> str

### Description
Converts prompt dictionaries, lists, or strings into markdown format. Dictionaries are converted to headers, lists to bullet points, and strings are returned as-is.

### Parameters
- **prompt** (PromptType) - Required - The input data (str, list, or dict) to convert.
- **_header_depth** (int) - Optional - The starting header level (default: 1).

### Example
```python
from aide.backend import compile_prompt_to_md

prompt = {
    "Introduction": "You are a Python expert",
    "Instructions": ["Write clean code", "Add comments"]
}

md = compile_prompt_to_md(prompt)
```
```

--------------------------------

### Generate Plan and Code

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Performs a single LLM call to generate both a natural language plan and Python code, with built-in retry logic.

```python
def plan_and_code_query(self, prompt: Any, retries: int = 3) -> tuple[str, str]
```

--------------------------------

### AIDE Visualization CSS Styles

Source: https://github.com/wecoai/aideml/blob/main/aide/utils/viz_templates/template.html

Sets the layout for the visualization dashboard with a split-screen design for canvas and text content.

```css
body { margin: 0; padding: 0; box-sizing: border-box; overflow: scroll; } body { background-color: #f2f0e7; } canvas { float: left; height: 100vh; width: 40vw; } #text-container { float: right; height: 100vh; width: 50vw; background-color: #282c34; } #plan { /* border-left: 2px solid #282c34; */ background-color: #282c34; color: #f2f0e7; min-height: 5rem; padding: 1em 0 1em 1em; }
```

--------------------------------

### Define ExecConfig

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Config.md

Defines configuration for code execution environments.

```python
@dataclass
class ExecConfig:
    timeout: int
    agent_file_name: str
    format_tb_ipython: bool
```

--------------------------------

### Configure LLM Models via CLI

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/README.md

Override default LLM models for code generation and feedback using CLI arguments.

```bash
# Anthropic Claude
aide ... agent.code.model=claude-3-sonnet agent.feedback.model=claude-3-haiku

# Google Gemini
aide ... agent.code.model=gemini-2.0-flash

# Local LLM (Ollama)
export OPENAI_BASE_URL=http://localhost:11434/v1
aide ... agent.code.model=qwen2.5 agent.feedback.model=qwen2.5
```

--------------------------------

### Agent.update_data_preview

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Agent.md

Generates a textual preview of the data directory.

```APIDOC
## Agent.update_data_preview()

### Description
Generates a textual preview of the data directory and stores it in self.data_preview. Called automatically on first step if agent.data_preview config is enabled.
```

--------------------------------

### Cite AIDE in BibTeX

Source: https://github.com/wecoai/aideml/blob/main/README.md

Use this BibTeX entry to cite the AIDE project in academic publications.

```bibtex
@article{aide2025,
      title={AIDE: AI-Driven Exploration in the Space of Code}, 
      author={Zhengyao Jiang and Dominik Schmidt and Dhruv Srikanth and Dixing Xu and Ian Kaplan and Deniss Jacenko and Yuxiang Wu},
      year={2025},
      eprint={2502.13138},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2502.13138}, 
}
```

--------------------------------

### Backend Type Definitions

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Backend.md

Core type aliases used for prompt inputs and query return values.

```python
PromptType = str | dict | list
OutputType = str | FunctionCallType
FunctionCallType = dict
```

--------------------------------

### Retrieve OpenAI tool dictionary

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/api-reference/Backend.md

Converts the function specification into the format required by OpenAI.

```python
@property
def as_openai_tool_dict() -> dict
```

--------------------------------

### Backend Module Interface

Source: https://github.com/wecoai/aideml/blob/main/_autodocs/module-map.md

Functions for querying various LLM providers through the backend interface.

```python
def query(
    system_message: PromptType | None,
    user_message: PromptType | None,
    model: str,
    temperature: float | None = None,
    max_tokens: int | None = None,
    func_spec: FunctionSpec | None = None,
    **model_kwargs
) -> OutputType

def determine_provider(model: str) -> str
def compile_prompt_to_md(prompt: PromptType, _header_depth: int = 1) -> str
```
Tokens:
Raw