### Quick Start: Docker Setup and Training

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/webshop/README.md

Use these commands to set up the environment and start GPU training with Docker. VERL manages the Qwen model via vLLM.

```bash
cd examples/vercel_ai_webshop

# 1. Set up environment
make setup

# 2. Run GPU training (VERL manages the Qwen model via vLLM)
make train
```

--------------------------------

### Run Hello 1024 Example (Distributed)

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/tinker/README.md

Launches the 'Hello 1024' example components (store, algorithm, runner) in separate terminals for detailed inspection. The algorithm connects to the store, and the runner starts worker processes.

```bash
agl store --port 4747
```

```bash
dotenv run python hello.py algo
```

```bash
dotenv run python hello.py runner
```

--------------------------------

### Set up Environment with uv

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/rag/README.md

Installs dependencies using uv and activates the virtual environment. Ensure you have uv installed.

```bash
uv sync --frozen --extra apo --group agents --group torch-gpu-stable --extra verl --group rag
source .venv/bin/activate
```

--------------------------------

### Install Dependencies

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/chartqa/README.md

Installs necessary dependencies for the ChartQA example. Ensure you are using vLLM version 0.10.2 to avoid potential issues.

```bash
uv sync --frozen \
    --group dev \
    --group experiment \
    --group image \
    --group langchain \
    --group vllm-0-10-2 \
    --group torch-gpu-stable
```

--------------------------------

### Install All Extras (CPU)

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/installation.md

Installs all dependencies, including algorithm-specific and example-specific ones, for a CPU-only machine using `uv sync`.

```bash
uv sync --frozen \
    --extra apo \
    --extra verl \
    --group dev \
    --group torch-cpu \
    --group torch-stable \
    --group trl \
    --group agents \
    --no-default-groups
```

--------------------------------

### Run Hello 1024 Example (One-Click)

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/tinker/README.md

Executes the 'Hello 1024' fine-tuning example in a single process. This workflow automatically sets up free ports for the LiteLLM proxy and Agent-lightning store.

```bash
dotenv run python hello.py oneclick
```

--------------------------------

### Runner Output Example

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/write-first-algorithm.md

This is an example of the output you might see when the runner process starts successfully and begins its operations.

```text
2025-10-14 22:23:41,339 [INFO] ... [Worker 0] Setting up tracer...
2025-10-14 22:23:41,343 [INFO] ... [Worker 0] Instrumentation applied.
2025-10-14 22:23:41,494 [INFO] ... [Worker 0] AgentOps client initialized.
2025-10-14 22:23:41,494 [INFO] ... [Worker 0] Started async rollouts (max: unlimited).
```

--------------------------------

### Start and Use LightningStoreServer and Client

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/deep-dive/store.md

Demonstrates the minimal lifecycle for setting up a LightningStoreServer in the owner process and connecting to it using LightningStoreClient from the same or a different process. Ensure the server is started before the client attempts to connect.

```python
import agentlightning as agl

# Server (owner process)
in_memory_store = agl.InMemoryLightningStore()
server = agl.LightningStoreServer(store=in_memory_store, host="0.0.0.0", port=4747)
await server.start()      # starts uvicorn in a daemon thread and waits for /health
# or keep your own event loop and stop via await server.stop()
# await server.run_forever()

# Client (same or different process)
client = agl.LightningStoreClient("http://localhost:4747")

print(await client.query_rollouts(status_in=["queuing"]))

await client.close()
await server.stop()
```

--------------------------------

### Install All Extras (GPU)

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/installation.md

Installs all dependencies, including algorithm-specific and example-specific ones, for a CUDA 12.8 compatible GPU machine using `uv sync`.

```bash
uv sync --frozen \
    --extra apo \
    --extra verl \
    --group dev \
    --group torch-gpu-stable \
    --group trl \
    --group agents \
    --no-default-groups
```

--------------------------------

### Minimal Developer Installation

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/installation.md

Clones the repository and installs only essential development dependencies using `uv sync`.

```bash
git clone https://github.com/microsoft/agent-lightning
cd agent-lightning
uv sync --group dev
```

--------------------------------

### Install Agent-Lightning and Agent-OS

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/agentos/README.md

Install the necessary packages for Agent-Lightning and Agent-OS integration.

```bash
pip install agentlightning agent-os
```

--------------------------------

### Start External Store Server

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/chartqa/README.md

Starts the Agent-Lightning managed store server on a specified port. This is recommended for distributed training setups.

```bash
agl store --port 4747
```

--------------------------------

### Install Unsloth and Dependencies

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/unsloth/README.md

Installs PyTorch, vLLM, Unsloth, and related packages required for the SFT example. Ensure your CUDA version is compatible with the PyTorch installation.

```bash
pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128
pip install vllm==0.10.2
pip install unsloth==2025.10.1 unsloth_zoo==2025.10.1 bitsandbytes peft datasets transformers trl kernels

pip install openai-agents mcp
```

--------------------------------

### Install and Run Pre-commit Hooks

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/installation.md

Installs pre-commit hooks for code style and linting enforcement, and then runs them on all files.

```bash
uv run pre-commit install
uv run pre-commit run --all-files --show-diff-on-failure --color=always
```

--------------------------------

### Install Dependencies

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/spider/README.md

Installs the necessary Python packages for the Spider example, including LangGraph, LangChain, and SQL-related libraries. Ensure you have Agent-Lightning and VERL dependencies installed separately.

```bash
pip install "langgraph<1.0" "langchain[openai]<1.0" "langchain-community" "langchain-text-splitters<1.0" "sqlparse" "nltk"
```

--------------------------------

### Setup Logging

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/reference/trainer.md

General function to set up logging.

```APIDOC
## Logging - Setup Logging

::: agentlightning.setup_logging
```

--------------------------------

### Install SWE-bench Harness and Utilities

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/claude_code/README.md

Install necessary Python packages for the SWE-bench harness and utilities. Ensure you are in a Python virtual environment (e.g., uv).

```bash
(uv) pip install swebench transformers datasets python-dotenv
```

--------------------------------

### Start the Lightning Store

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/write-first-algorithm.md

Run this command in a terminal to start the LightningStore server. It listens on port 4747 by default and waits for connections from the algorithm and runner.

```bash
agl store
```

--------------------------------

### Start LLM Proxy Server

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/debug.md

Initialize and start the LLM proxy server. This script sets up an in-memory store and configures the proxy to connect to a local vLLM endpoint.

```python
import asyncio
import aiohttp
import agentlightning as agl

async def serve_llm_proxy():
    store = agl.InMemoryLightningStore()
    store_server = agl.LightningStoreServer(store, "127.0.0.1", 8081)
    await store_server.start()

    llm_proxy = agl.LLMProxy(
        port=8082,
        model_list=[
            {
                "model_name": "Qwen/Qwen2.5-0.5B-Instruct",
                "litellm_params": {
                    "model": "hosted_vllm/Qwen/Qwen2.5-0.5B-Instruct",
                    "api_base": "http://localhost:8080/v1",
                },
            }
        ],
        store=store_server,
    )

    await llm_proxy.start()
    await asyncio.sleep(1000000)
```

--------------------------------

### Azure ML Setup

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/webshop/README.md

Commands for setting up Azure CLI with the ML extension and logging in. This is a prerequisite for submitting Azure ML jobs.

```bash
az extension add -n ml
az login
```

--------------------------------

### Install Pre-commit Hooks

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/community/contributing.md

Installs the pre-commit hooks to enforce code formatting and linting. This should be run once after cloning the repository.

```bash
uv run --no-sync pre-commit install
```

--------------------------------

### Quick Start: Train with Agent-OS Runner and Policy Reward

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/agentos/README.md

Initialize a governed kernel, wrap it in an Agent-OS runner, and train a model using policy-aware rewards.

```python
from agentlightning import Trainer
from agentlightning.contrib.runner.agentos import AgentOSRunner
from agentlightning.contrib.reward.agentos import PolicyReward
from agent_os import KernelSpace
from agent_os.policies import SQLPolicy

# Create governed kernel
kernel = KernelSpace(
    policy=SQLPolicy(
        deny=["DROP", "DELETE"]
    )
)

# Wrap in Agent-OS runner
runner = AgentOSRunner(kernel)

# Train with policy-aware rewards
trainer = Trainer(
    runner=runner,
    reward_fn=PolicyReward(kernel),
    algorithm="GRPO"
)

trainer.train()
```

--------------------------------

### Start Local vLLM Server

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/debug.md

Manually start a local vLLM server for debugging LLM features. Ensure the model is accessible.

```bash
vllm serve Qwen/Qwen2.5-0.5B-Instruct --port 8080
```

--------------------------------

### Copy Environment Template

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/tinker/README.md

Copies the example environment file to be filled with necessary credentials for LiteLLM, Tinker, and optional services like Weights & Biases.

```bash
cp examples/tinker/.env.example examples/tinker/.env
```

--------------------------------

### Start Algorithm Process

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/parallelize.md

Set environment variables to define the host, port, and role for the algorithm process. This allows the strategy to manage the LightningStoreServer.

```bash
export AGL_SERVER_HOST=0.0.0.0
export AGL_SERVER_PORT=4747
export AGL_CURRENT_ROLE=algorithm

python train_calc_agent.py
```

--------------------------------

### Install Agent-Lightning Nightly Build

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/installation.md

Install the latest nightly build from Test PyPI for experimental features. Use with caution as these builds may be unstable.

```bash
pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning
```

--------------------------------

### Install NPU Dependencies

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/train-sql-agent.md

Install specific versions of vLLM, vLLM-Ascend, and VERL required for NPU training. Ensure to use the trusted-host option for packages from huaweicloud.

```bash
pip install vllm==0.10.0 --trusted-host repo.huaweicloud.com
```

```bash
pip install vllm-Ascend==0.10.0rc1 --trusted-host repo.huaweicloud.com
```

```bash
pip install verl==0.5.0
```

--------------------------------

### Configure and Start LLMProxy with vLLM Integration

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/unsloth-sft.md

Set up an LLMProxy to forward requests to a vLLM server, enabling tracing and token ID collection. The proxy automatically starts if not running. Add the proxy as a resource to the store for runner access.

```python
import agentlightning as agl

llm_proxy = agl.LLMProxy(port=port, store=store)
model_list = [
    {
        "model_name": "Qwen3-4B-Instruct",
        "litellm_params": {"model": f"hosted_vllm/{model_path}", "api_base": server_address},
    }
]
llm_proxy.update_model_list(model_list)
# If the proxy is not running, it will start automatically.
await llm_proxy.restart()
# Add the proxy as a resource to the store so that the runners can access it via URL.
resource_update = await store.add_resources({"main_llm": llm_proxy.as_resource()})
```

--------------------------------

### Build Dashboard with npm

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/installation.md

Navigates to the dashboard directory, installs npm dependencies, and builds the dashboard assets using Vite.

```bash
cd dashboard
npm ci
npm run build
```

--------------------------------

### Python Client for LightningStore

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/reference/cli.md

Example of how to instantiate a LightningStore client in Python and use it with the Trainer.

```python
store_client = agl.LightningStoreClient("http://localhost:4747")
trainer = agl.Trainer(store=store_client, ...)
```

--------------------------------

### Manual Training: WebShop Server

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/webshop/README.md

Start the WebShop Flask server manually in a dedicated terminal using Docker Compose.

```bash
cd examples/vercel_ai_webshop
docker compose up webshop --build
```

--------------------------------

### Manual Training: Headless Runners

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/webshop/README.md

Start headless rollout runners manually. Ensure the AGENT_LIGHTNING_STORE_URL is set correctly.

```bash
cd examples/vercel_ai_webshop
export AGENT_LIGHTNING_STORE_URL="http://localhost:4747"
pnpm headless -- --worker-id runner-1
```

--------------------------------

### Provision Tooling with uv sync

Source: https://github.com/microsoft/agent-lightning/blob/main/CLAUDE.md

Use this command to provision the necessary tooling for the development environment. It ensures all dependencies are installed and configured.

```bash
uv sync --group dev
```

--------------------------------

### Example Rollout and Span Output

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/debug.md

Illustrates the expected output format when querying rollouts and spans from the store, including status and captured attributes.

```python
[Rollout(rollout_id='ro-519769241af8', input='Explain why the sky appears blue using principles of light scattering in 100 words.', start_time=1760706315.6996238, ..., status='succeeded')]
[Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...)]
```

--------------------------------

### Setup Module Logging

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/reference/trainer.md

Function to set up logging for a specific module.

```APIDOC
## Logging - Setup Module Logging

::: agentlightning.setup_module_logging
```

--------------------------------

### Install Specific Development Dependencies

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/community/contributing.md

Installs dependencies including GPU extras and optional packages. Use this command when you need specific optional dependencies beyond the default development stack.

```bash
uv sync --frozen \
    --extra apo \
    --extra verl \
    --group dev \
    --group torch-cpu \
    --group torch-stable \
    --group agents \
    --no-default-groups
```

--------------------------------

### LightningStore Server CLI

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/reference/cli.md

Start an independent LightningStore server using the CLI. This server stores data in memory by default.

```text
usage: agl store [-h] [--host HOST] [--port PORT] [--cors-origin CORS_ORIGINS] [--log-level {DEBUG,INFO,WARNING,ERROR}] [--tracker {prometheus,console} [{prometheus,console} ...]] [--n-workers N_WORKERS] [--backend {memory,mongo}]
                 [--mongo-uri MONGO_URI]

Run a LightningStore server

options:
  -h, --help            show this help message and exit
  --host HOST           Host to bind the server to
  --port PORT           Port to run the server on
  --cors-origin CORS_ORIGINS
                        Allowed CORS origin. Repeat for multiple origins. Use '*' to allow all origins.
  --log-level {DEBUG,INFO,WARNING,ERROR}
                        Configure the logging level for the store.
  --tracker {prometheus,console} [{prometheus,console} ...]
                        Enable metrics tracking. Repeat for multiple trackers.
  --n-workers N_WORKERS
                        Number of workers to run in the server. When it's greater than 1, the server will be run using `mp` launch mode. Only applicable for zero-copy stores such as MongoDB backend.
  --backend {memory,mongo}
                        Backend to use for the store.
  --mongo-uri MONGO_URI
                        MongoDB URI to use for the store. Applicable only if --backend is 'mongo'.
```

--------------------------------

### Start vLLM Server for Local Model

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/chartqa/README.md

Starts a vLLM server for local model inference, specifying the model, GPU memory utilization, maximum model length, and allowed media path. This is required for testing the agent with a local vLLM server.

```bash
export CHARTQA_DATA_DIR=<path to chartqa data>
vllm serve Qwen/Qwen2-VL-2B-Instruct \
    --gpu-memory-utilization 0.6 \
    --max-model-len 4096 \
    --allowed-local-media-path $CHARTQA_DATA_DIR \
    --enable-prefix-caching \
    --port 8088
```

--------------------------------

### Start MCP Server

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/rag/README.md

Launches the MCP server for Wikipedia retrieval. This should be run in a separate terminal.

```bash
python wiki_retriever_mcp.py
```

--------------------------------

### Start MongoDB with Docker Compose

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/parallelize.md

Launch a MongoDB instance using Docker Compose for local development and testing.

```bash
docker compose -f compose.mongo.yml up -d
```

--------------------------------

### Manual Training: Training Coordinator

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/webshop/README.md

Set up and run the Agent Lightning training coordinator manually. Requires initial setup and activation of a virtual environment.

```bash
cd examples/vercel_ai_webshop/agl
./setup.sh                    # First time only
source activate.sh
python run_training.py qwen   # Full training
```

--------------------------------

### Docker Training Commands

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/webshop/README.md

Commands for starting GPU training, scaling runners, checking status, and stopping services when using Docker.

```bash
# Start GPU training - VERL manages vLLM, no API key needed
make train

# Run with more runners
N_RUNNERS=3 make train

# Check container status
make status

# Stop all services
make stop
```

--------------------------------

### Install VERL with Dependencies

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/algorithm-zoo/verl.md

Install VERL and its dependencies using pip. It is recommended to follow the installation guide for compatibility.

```bash
pip install agentlightning[verl]
```

--------------------------------

### LitAgentRunner Setup

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/write-first-algorithm.md

Initializes and runs a LitAgentRunner with a specified agent and store client. The runner manages task dequeuing, agent execution, and instrumentation via a tracer.

```python
# Connecting to Store
store = agl.LightningStoreClient("http://localhost:4747")  # or some other address
runner = LitAgentRunner[str](tracer=AgentOpsTracer())
with runner.run_context(agent=simple_agent, store=store):  # <-- where the wrapping and instrumentation happens
    await runner.iter()  # polling for new tasks forever
```

--------------------------------

### Start Ray for Distributed Training

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/search_r1/README.md

Initiates the Ray cluster for distributed training. Ensure WANDB_API_KEY is set if using Weights & Biases for experiment tracking.

```bash
bash ../../scripts/restart_ray.sh
```

--------------------------------

### Serve Documentation with Live Reload

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/community/contributing.md

Build and serve the project documentation locally using MkDocs, with live reloading enabled for immediate feedback on changes.

```bash
uv run --no-sync mkdocs serve --strict
```

--------------------------------

### Initialize Runner, Tracer, and Store

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/debug.md

Set up `OtelTracer`, `LitAgentRunner`, and `InMemoryLightningStore` for debugging. The runner requires a tracer instance.

```python
import agentlightning as agl

tracer = agl.OtelTracer()
runner = agl.LitAgentRunner(tracer)
store = agl.InMemoryLightningStore()
```

--------------------------------

### Download Tiny Dataset Files

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/rag/README.md

Downloads the necessary dataset files (parquet, pkl, index) for the RAG agent example using gdown. Ensure gdown is installed.

```bash
pip install gdown

# tiny training dataset
cd examples/rag
gdown --fuzzy "https://drive.google.com/file/d/1Pq4Ag8zVoN8gUtLu0LcBfY35Dm5zL0hq/view?usp=drive_link" 
    -O dataset_tiny.parquet

# chunks_candidate_tiny.pkl
gdown --fuzzy "https://drive.google.com/file/d/1REXCpRLbeZu1KfWWKhIGEQe_WNHUOBkS/view?usp=drive_link" 
    -O chunks_candidate_tiny.pkl

# index_hnsw_faiss_n32e40_tiny.index
gdown --fuzzy "https://drive.google.com/file/d/1f6P-h_8KSRhe5pqDHWbRQWvUhTygfZ-c/view?usp=drive_link" 
    -O index_hnsw_faiss_n32e40_tiny.index
```

--------------------------------

### Configure and Train an Agent

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/train-first-agent.md

Configure the `Trainer` with an algorithm and initial prompt, load datasets, and start the training process. Ensure `openai >= 1.100.0` is installed for the `TraceToMessages` adapter.

```python
trainer = agl.Trainer(
    algorithm=algo,
    n_runners=8, # Run 8 agents in parallel to try out the prompts
    initial_resources={
        # The initial prompt template to be tuned
        "prompt_template": prompt_template_baseline()
    },
    # This is used to convert the span data into a message format consumable by APO algorithm
    adapter=agl.TraceToMessages(),
)

# 2. Load datasets: They can be list of task objects consumable by `room_selector`.
dataset_train, dataset_val = ...

# 3. Start the training process!
trainer.fit(
    agent=room_selector,
    train_dataset=dataset_train,
    val_dataset=dataset_val
)
```

--------------------------------

### Initialize and Use InMemoryLightningStore

Source: https://context7.com/microsoft/agent-lightning/llms.txt

Demonstrates initializing an in-memory store, adding resources, enqueuing a rollout task, querying rollouts and spans, and fetching statistics. Ensure thread safety if used concurrently.

```python
import asyncio
import agentlightning as agl
from agentlightning.store.memory import InMemoryLightningStore

async def demo_store():
    store = InMemoryLightningStore(thread_safe=True)

    # Add initial resources (LLM endpoint, prompt template, etc.)
    resources: agl.NamedResources = {
        "llm": agl.LLM(endpoint="http://localhost:8000/v1", model="gpt-4o-mini"),
    }
    resource_update = await store.update_resources("default", resources)
    print(f"Resources saved: {resource_update.resources_id}")

    # Enqueue a rollout task
    rollout = await store.enqueue_rollout(
        input={"question": "SELECT users WHERE age > 30"},
        mode="train",
        resources_id=resource_update.resources_id,
    )
    print(f"Rollout queued: {rollout.rollout_id}, status={rollout.status}")

    # Query pending rollouts
    pending = await store.query_rollouts(status_in=["queuing"])
    print(f"Pending rollouts: {len(pending)}")

    # Retrieve spans after execution
    spans = await store.query_spans(rollout_id=rollout.rollout_id)
    print(f"Spans collected: {len(spans)}")

    # Fetch store statistics
    stats = await store.get_statistics()
    print(f"Store stats: {stats}")

asyncio.run(demo_store())
```

--------------------------------

### Install AGL Environments and Dependencies

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/envs/README.md

Run this script to install the necessary environments and AGL dependencies. Ensure you activate the correct conda environment after installation.

```bash
cd contrib/recipes/envs

git clone https://github.com/agent-lightning/agl-envs
mv agl-envs agl_envs

# Install alfworld dependency
bash agl_envs/setup/setup_alfworld.sh
conda activate alfworld

# Install scienceworld dependency
bash agl_envs/setup/setup_sciworld.sh
conda activate sciworld

# Install AGL dependency
bash install_agl.sh
```

--------------------------------

### Example Console Output Log

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/azure/README.md

This log provides a detailed view of the fine-tuning process, including stages like client-server execution, model deployment, rollout completion, fine-tuning job creation and success, and model evaluation. Note the time taken for each stage, especially the fine-tuning queue wait times.

```log
10:13:02,624 Starting client-server execution with 2 runner(s) [role=both, main_process=algorithm]
10:13:02,639 Starting LightningStore server on localhost:4747
10:13:02,749 [AOAI FT 1/3] [Stage 1] Starting fine-tuning iteration with 24 tasks...
10:13:02,750 [AOAI FT 1/3] [Stage 2] Using model deployment: gpt-4.1-mini
10:13:03,428 [Worker 1] Started async rollouts (max: unlimited). 
10:13:03,429 [Worker 0] Started async rollouts (max: unlimited).
10:13:05,279 [Worker 0 | Rollout ro-efab388d2f0e] Completed in 1.83s. Collected 4 span(s). Final reward: 1.0
10:13:05,454 [Worker 1 | Rollout ro-8ba08859ae85] Completed in 2.01s. Collected 4 span(s). Final reward: 1.0
[... 22 more rollouts omitted ...]
10:13:28,430 [AOAI FT 1/3] [Stage 3] Completed rollouts for 24 tasks.
10:13:28,431 [AOAI FT 1/3] Keeping 28 example(s) for fine-tuning after reward-based filtering.
10:13:28,431 [AOAI FT 1/3] [Stage 4] Prepared 28 training examples after filtering.
10:13:28,431 [AOAI FT 1/3] [Stage 5] Starting fine-tuning for model gpt-4.1-mini-2025-04-14...
10:13:29,854 [AOAI FT 1/3] Uploaded training file to Azure OpenAI (file_id=file-0fd6e72151094a0eb0306de7aae4883b).
10:13:41,216 [AOAI FT 1/3] Fine-tuning job ftjob-0ee45c42591b4f4a8bd4f49ef2301dcd created for base model gpt-4.1-mini-2025-04-14.
10:13:41,217 [AOAI FT 1/3] Waiting for fine-tuning job ftjob-0ee45c42591b4f4a8bd4f49ef2301dcd to complete.
12:29:11,444 [AOAI FT 1/3] Fine-tuning job ftjob-0ee45c42591b4f4a8bd4f49ef2301dcd succeeded with new model id gpt-4.1-mini-2025-04-14.ft-0ee45c42591b4f4a8bd4f49ef2301dcd-v01.
12:29:11,444 [AOAI FT 1/3] [Stage 6] Deploying fine-tuned model...
12:29:14,217 [AOAI FT 1/3] Waiting for deployment gpt-4.1-mini-ft_v01 to become ready.
12:29:15,458 [AOAI FT 1/3] Waiting for deployment to be ready. Current provisioning state of gpt-4.1-mini-ft_v01: Creating
[... 7 repetitive deployment status checks omitted ...]
12:32:53,773 [AOAI FT 1/3] Waiting for deployment to be ready. Current provisioning state of gpt-4.1-mini-ft_v01: Succeeded
12:32:53,773 [AOAI FT 1/3] Deployment gpt-4.1-mini-ft_v01 is ready with version 1.
12:32:53,774 [AOAI FT 1/3] [Stage 7] Evaluating on validation dataset...
[... 8 validation rollouts omitted ...]
12:33:03,979 [AOAI FT 1/3] [Stage 7] Evaluation completed. Average reward: 1.0000
12:33:03,979 [AOAI FT 2/3] [Stage 1] Starting fine-tuning iteration with 24 tasks...
12:33:03,979 [AOAI FT 2/3] [Stage 2] Using model deployment: gpt-4.1-mini-ft_v01
[... 24 rollouts omitted ...]
12:33:34,619 [AOAI FT 2/3] [Stage 3] Completed rollouts for 24 tasks.
12:33:34,620 [AOAI FT 2/3] [Stage 4] Prepared 27 training examples after filtering.
12:33:34,620 [AOAI FT 2/3] [Stage 5] Starting fine-tuning for model gpt-4.1-mini-2025-04-14.ft-0ee45c42591b4f4a8bd4f49ef2301dcd-v01...
13:16:43,810 [AOAI FT 2/3] Fine-tuning job ftjob-06366e441ee24a0ea242014fea8fbc3a succeeded with new model id gpt-4.1-mini-2025-04-14.ft-06366e441ee24a0ea242014fea8fbc3a-v02.
13:16:43,810 [AOAI FT 2/3] [Stage 6] Deploying fine-tuned model...
13:16:46,263 [AOAI FT 2/3] Waiting for deployment gpt-4.1-mini-ft_v02 to become ready.
[... 5 repetitive deployment status checks omitted ...]
13:19:23,856 [AOAI FT 2/3] Waiting for deployment to be ready. Current provisioning state of gpt-4.1-mini-ft_v02: Succeeded
13:19:23,857 [AOAI FT 2/3] [Stage 7] Evaluating on validation dataset...
[... 8 validation rollouts omitted ...]
13:19:39,072 [AOAI FT 2/3] [Stage 7] Evaluation completed. Average reward: 1.0000
13:19:39,072 [AOAI FT 3/3] [Stage 1] Starting fine-tuning iteration with 24 tasks...
13:19:39,073 [AOAI FT 3/3] [Stage 2] Using model deployment: gpt-4.1-mini-ft_v02
[... 24 rollouts omitted ...]
13:20:04,721 [AOAI FT 3/3] [Stage 3] Completed rollouts for 24 tasks.
13:20:04,722 [AOAI FT 3/3] [Stage 4] Prepared 27 training examples after filtering.
13:20:04,722 [AOAI FT 3/3] [Stage 5] Starting fine-tuning for model gpt-4.1-mini-2025-04-14.ft-06366e441ee24a0ea242014fea8fbc3a-v02...
14:02:47,241 [AOAI FT 3/3] Fine-tuning job ftjob-2651d3183a4b40679d4c3fc886940c0c succeeded with new model id gpt-4.1-mini-2025-04-14.ft-2651d3183a4b40679d4c3fc886940c0c-v03.
14:02:47,242 [AOAI FT 3/3] [Stage 6] Deploying fine-tuned model...

```

--------------------------------

### Install MongoDB Support for Agent-lightning

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/parallelize.md

Install the necessary package for Agent-lightning to interact with MongoDB.

```bash
pip install agentlightning[mongo]
```

--------------------------------

### Set up MongoDB Data Directory and Permissions

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/parallelize.md

Create a data directory for MongoDB and set appropriate permissions to avoid issues.

```bash
mkdir -p data/mongo-host
chmod 777 data/mongo-host
```

--------------------------------

### Install Agent-Lightning Stable Release

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/installation.md

Use this command to install the latest stable version of Agent-Lightning from PyPI. This is recommended for general use.

```bash
pip install --upgrade agentlightning
```

--------------------------------

### Prepare and Launch Retrieval Server

Source: https://github.com/microsoft/agent-lightning/blob/main/contrib/recipes/search_r1/README.md

Activates the 'retriever' environment and starts a retrieval server at http://127.0.0.1:8000. This server handles document retrieval during training. Keep it running in a separate session.

```bash
bash retrieval_launch.sh
```

--------------------------------

### Configure ClientServerExecutionStrategy for Distributed Training

Source: https://context7.com/microsoft/agent-lightning/llms.txt

Illustrates setting up distributed training by splitting the algorithm and runners into separate processes using ClientServerExecutionStrategy. This includes configurations for the algorithm/server node, runner/worker nodes, and a local setup.

```python
import agentlightning as agl
from agentlightning.execution import ClientServerExecutionStrategy

# On the algorithm/server node:
trainer = agl.Trainer(
    algorithm=agl.APO(openai_client),
    strategy=ClientServerExecutionStrategy(
        role="algorithm",           # this process hosts algorithm + store server
        server_port=8765,
        n_runners=16,               # expect 16 remote runners
        graceful_timeout=30.0,
    ),
    initial_resources={"prompt_template": initial_template},
    adapter=agl.TraceToMessages(),
)
trainer.fit(agent=my_agent, train_dataset=train_data)

# On each runner/worker node (separate script/machine):
runner_trainer = agl.Trainer(
    strategy=ClientServerExecutionStrategy(
        role="runner",
        server_host="10.0.0.1",     # algorithm node IP
        server_port=8765,
        n_runners=2,                # runners in this process
    ),
)
runner_trainer.fit(agent=my_agent)

# Or run everything locally (both roles in one process):
local_trainer = agl.Trainer(
    algorithm=agl.APO(openai_client),
    strategy=ClientServerExecutionStrategy(
        role="both",
        n_runners=8,
        server_port=8765,
    ),
    initial_resources={"prompt_template": initial_template},
)
local_trainer.fit(agent=my_agent, train_dataset=train_data)
```

--------------------------------

### Install Dependencies with Pip

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/calc_x/README.md

Installs necessary Python packages for AutoGen, OpenAI extensions, and the MCP protocol. Ensure this is run before setting up the agent.

```bash
pip install "autogen-agentchat" "autogen-ext[openai]" "mcp>=1.10.0"
```

--------------------------------

### Install Agent Lightning

Source: https://github.com/microsoft/agent-lightning/blob/main/README.md

Install the Agent Lightning package using pip. For the latest nightly build, use the Test PyPI repository.

```bash
pip install agentlightning
```

```bash
pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning
```

--------------------------------

### Setup Azure OpenAI Fine-tuning Environment

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/azure/README.md

Copy the sample environment file, fill in your Azure values, and source it before running any scripts. This sets up the necessary environment variables for the fine-tuning process.

```bash
cp examples/azure_finetune/.env.example examples/azure_finetune/.env
# edit examples/azure_finetune/.env with your keys and identifiers
source examples/azure_finetune/.env
```

--------------------------------

### Initialize and Use LLMProxy

Source: https://context7.com/microsoft/agent-lightning/llms.txt

Shows how to set up an LLMProxy to intercept and trace LLM calls, exporting spans to a LightningStore. The proxy can be launched in a separate process for better performance.

```python
import asyncio
import agentlightning as agl
from agentlightning.store.memory import InMemoryLightningStore

async def demo_proxy():
    store = InMemoryLightningStore(thread_safe=True)

    proxy = agl.LLMProxy(
        port=4000,
        model_list=[
            {
                "model_name": "llama3",
                "litellm_params": {
                    "model": "openai/llama3",
                    "api_base": "http://vllm-server:8000/v1",
                    "api_key": "none",
                },
            }
        ],
        store=store,
        launch_mode="mp",   # recommended: separate process
        num_retries=2,
    )

    await proxy.start()
    print(f"LLMProxy running: {proxy.is_running()}")

    # Create a resource that points agents through the proxy
    proxy_resource = proxy.as_resource(model="llama3")
    print(f"Proxy endpoint: {proxy_resource.endpoint}")

    # Use with Trainer — the proxy auto-starts when algorithm runs
    trainer = agl.Trainer(
        llm_proxy=proxy,
        initial_resources={"llm": proxy_resource},
        n_runners=4,
    )

    await proxy.stop()

asyncio.run(demo_proxy())
```

--------------------------------

### Full Training Setup with Trainer

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/train-sql-agent.md

Use this snippet to initialize the `Trainer` with your agent, algorithm, and datasets for full reinforcement learning training. It configures parallelism and specifies how algorithms interpret agent trace data.

```python
import agentlightning as agl

agent = LitSQLAgent()
algorithm = agl.VERL(verl_config)
trainer = agl.Trainer(
    n_runners=10,
    algorithm=algorithm,
    adapter={"agent_match": active_agent},
)
train_data = pd.read_parquet("data/train_spider.parquet").to_dict("records")
val_data = pd.read_parquet("data/test_dev_500.parquet").to_dict("records")
trainer.fit(agent, train_dataset=train_data, val_dataset=val_data)
```

--------------------------------

### Start Full Fine-Tuning Loop

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/azure/README.md

Execute this command from the repository root to initiate the fine-tuning process. The script divides the dataset, performs fine-tune, deploy, and evaluate cycles, and manages Azure's fine-tuning queue.

```bash
python train_capital_agent.py
```

--------------------------------

### Trainer with Client-Server Strategy using Environment Variables

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/tutorials/parallelize.md

Configure the Trainer with the Client-Server strategy and set execution parameters like server port, current role, and managed store using environment variables.

```python
import os

os.environ["AGL_SERVER_PORT"] = "10000"
os.environ["AGL_CURRENT_ROLE"] = "algorithm"
os.environ["AGL_MANAGED_STORE"] = "0"

trainer = agl.Trainer(algorithm=algorithm, n_runners=8, strategy="cs")
```

--------------------------------

### Install Agent Lightning with APO

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/algorithm-zoo/apo.md

Install the Agent Lightning library with APO support using pip. This command ensures that all necessary dependencies for APO are included.

```bash
pip install agentlightning[apo]
```

--------------------------------

### Launch SQL Agent Training with NPU

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/train-sql-agent.md

Execute the training script with the 'npu' argument to utilize Huawei Ascend NPUs for training the SQL agent.

```bash
python train_sql_agent.py npu
```

--------------------------------

### Install Dependencies with uv

Source: https://github.com/microsoft/agent-lightning/blob/main/examples/tinker/README.md

Installs necessary dependencies for development, agents, and Tinker using the uv package manager. Ensure Tinker, LiteLLM, CrewAI, and Agent-lightning are available.

```bash
uv sync --frozen --extra apo --group dev --group agents --group tinker
```

--------------------------------

### Get Logger Instance

Source: https://github.com/microsoft/agent-lightning/blob/main/CLAUDE.md

Instantiate a logger for a module using the standard Python logging library. It's recommended to use `__name__` to get a logger specific to the current module.

```python
logging.getLogger(__name__)
```

--------------------------------

### Start the Runner Process

Source: https://github.com/microsoft/agent-lightning/blob/main/docs/how-to/write-first-algorithm.md

Execute this command in a separate terminal to start the runner. Ensure your OpenAI API key is set as an environment variable. The runner connects to the store and waits for tasks.

```bash
export OPENAI_API_KEY=sk-... # Your OpenAI API key
python apo_custom_algorithm.py runner
```

--------------------------------

### Define LLM, PromptTemplate, and NamedResources

Source: https://context7.com/microsoft/agent-lightning/llms.txt

Illustrates how to instantiate LLM for direct endpoint access, PromptTemplate for tunable prompts, and group them into NamedResources for agent distribution. ProxyLLM is also shown for routed LLM access.

```python
import agentlightning as agl

# LLM resource for direct endpoint access
llm = agl.LLM(
    endpoint="http://vllm-server:8000/v1",
    model="meta-llama/Llama-3-8B-Instruct",
    api_key="token-abc123",
    sampling_parameters={"temperature": 0.7, "max_tokens": 512, "top_p": 0.95},
)

# PromptTemplate resource (tuned by APO algorithm)
template = agl.PromptTemplate(
    template="You are a SQL expert. Convert this request to SQL:\n{request}\n\nSQL:",
    engine="f-string",
)
formatted = template.format(request="Get all users over 30")
print(formatted)
# → "You are a SQL expert. Convert this request to SQL:\nGet all users over 30\n\nSQL:"

# NamedResources dict — passed to agents during rollout
resources: agl.NamedResources = {
    "main_llm": llm,
    "system_prompt": template,
}

# ProxyLLM — resolves to rollout-scoped endpoint at runtime
proxy_llm = agl.ProxyLLM(
    endpoint="http://localhost:4000/v1",
    model="llama3",
)
# The @rollout decorator automatically strips ProxyLLM → LLM with baked-in rollout ID
# so the agent function always receives a plain LLM resource.
```