Osmosis

https://docs.osmosis.ai
Admin
Osmosis is a comprehensive post-training and reinforcement learning platform that helps companies...
Tokens:49,875
Snippets:704
Trust Score:7.5
Update:2 months ago
Show doc for...
Tokens:
Raw
### Run quickstart wizard

Source: https://docs.osmosis.ai/cli/command-reference

Initiates the interactive setup process for authentication and workspace configuration.

```cli
osmosis quickstart [--workspace NAME]
```

--------------------------------

### osmosis quickstart

Source: https://docs.osmosis.ai/cli/command-reference

The quickstart command initiates an interactive setup wizard to authenticate the CLI, select a workspace, and clone the associated repository.

```APIDOC
## osmosis quickstart

### Description
Starts an interactive wizard to sign in, clone a workspace, and configure the environment. This command is designed for interactive terminal use and is not compatible with --json or --plain flags.

### Usage
`osmosis quickstart [--workspace NAME]`

### Parameters
- **--workspace** (str) - Optional - The name of the workspace to set up. If omitted, the CLI will prompt for selection or detect the workspace from the current git remote.
```

--------------------------------

### Install Combined SDK Extras

Source: https://docs.osmosis.ai/sdk/installation

Demonstrates how to install multiple optional features simultaneously.

```bash
python -m pip install "osmosis-ai[server,strands]>=0.3.0rc1,<0.4"
```

--------------------------------

### Run the Quickstart Wizard

Source: https://docs.osmosis.ai/platform/onboarding

Execute the quickstart wizard to configure the workspace and prepare for AI agent tasks.

```bash
osmosis quickstart
```

--------------------------------

### Install LocalBackend

Source: https://docs.osmosis.ai/sdk/execution-backends/local-backend

Install the base package with the server extra to enable rollout server functionality.

```bash
pip install "osmosis-ai[server]>=0.3.0rc1,<0.4"
```

--------------------------------

### Set up workspace

Source: https://docs.osmosis.ai/skill.md

Initialize the local environment by cloning the repository, installing the CLI, and authenticating.

```bash
# Create or join workspace on platform, connect GitHub
# Clone the workspace repository
git clone https://github.com/your-org/osmosis-workspace.git
cd osmosis-workspace

# Install CLI and authenticate
pip install "osmosis-ai[strands]>=0.3.0rc1,<0.4"
osmosis auth login
osmosis doctor  # Validate structure
```

--------------------------------

### Verify CLI Installation

Source: https://docs.osmosis.ai/cli/installation

Check the installed version of the CLI to confirm successful setup.

```bash
osmosis --version
```

--------------------------------

### Install HarborBackend dependencies

Source: https://docs.osmosis.ai/sdk/execution-backends/harbor-backend

Install the required osmosis-ai packages with server and harbor features enabled.

```bash
pip install "osmosis-ai[server,harbor]>=0.3.0rc1,<0.4"
```

--------------------------------

### Download command examples

Source: https://docs.osmosis.ai/cli/command-reference

Common usage patterns for downloading evaluation data.

```bash
# Default: metrics + trajectories for the whole run
osmosis eval download my-eval-run

# Only artifacts for rows 3, 7, and 10–20
osmosis eval download my-eval-run --type artifacts --rows 3,7,10-20

# Everything, into a custom root, non-interactive
osmosis eval download my-eval-run --type all -o ./eval-outputs/ --yes
```

--------------------------------

### Quick Start Benchmark Commands

Source: https://docs.osmosis.ai/cli/benchmarks

Commands to list benchmarks, view info, set API keys, and submit a benchmark configuration.

```bash
osmosis benchmark list
osmosis benchmark info terminal-bench-2-1
osmosis secret set OPENAI_API_KEY
osmosis benchmark submit configs/benchmark/terminal-bench-smoke.toml
```

--------------------------------

### Configure HarborBackend and rollout server

Source: https://docs.osmosis.ai/sdk/execution-backends/harbor-backend

Setup the Harbor backend with a specific task directory and environment configuration, then initialize the rollout server.

```python
from pathlib import Path

from harbor.models.environment_type import EnvironmentType
from harbor.models.trial.config import EnvironmentConfig as HarborEnvironmentConfig
from harbor.trial.queue import TrialQueue
from osmosis_ai.rollout.backend.harbor import HarborBackend
from osmosis_ai.rollout.server import create_rollout_server

backend = HarborBackend(
    orchestrator=TrialQueue(n_concurrent=4),
    tasks_dir=Path("tasks"),
    task_mode="dataset",
    agent="mini-swe-agent",
    grader=None,
    environment_config=HarborEnvironmentConfig(type=EnvironmentType.SKYPILOT),
    max_queue_depth=8,
)

app = create_rollout_server(
    backend=backend,
    lifespan=backend.prewarm_lifespan(["task-a", "task-b"]),
)
```

--------------------------------

### Install Optional SDK Extras

Source: https://docs.osmosis.ai/sdk/installation

Commands for installing specific feature sets via pip extras.

```bash
python -m pip install "osmosis-ai[server]>=0.3.0rc1,<0.4"
```

```bash
python -m pip install "osmosis-ai[strands]>=0.3.0rc1,<0.4"
```

```bash
python -m pip install "osmosis-ai[openai-agents]>=0.3.0rc1,<0.4"
```

```bash
python -m pip install "osmosis-ai[harbor]>=0.3.0rc1,<0.4"
```

```bash
python -m pip install "osmosis-ai[rubric]>=0.3.0rc1,<0.4"
```

```bash
python -m pip install "osmosis-ai[parquet]>=0.3.0rc1,<0.4"
```

```bash
python -m pip install "osmosis-ai[full]>=0.3.0rc1,<0.4"
```

--------------------------------

### Define and run a minimal rollout server

Source: https://docs.osmosis.ai/sdk/installation

Wires a workflow and grader into a LocalBackend and starts the server using uvicorn.

```python
import os

import uvicorn
from osmosis_ai.rollout import (
    AgentWorkflow,
    AgentWorkflowContext,
    AgentWorkflowOutput,
    Grader,
    GraderContext,
    LocalBackend,
)
from osmosis_ai.rollout.server import create_rollout_server


class EchoWorkflow(AgentWorkflow):
    async def run(self, ctx: AgentWorkflowContext) -> AgentWorkflowOutput:
        return AgentWorkflowOutput(
            messages=[*ctx.prompt, {"role": "assistant", "content": "ready"}]
        )


class ReadyGrader(Grader):
    async def grade(self, ctx: GraderContext) -> None:
        if ctx.sample is None:
            raise ValueError("workflow produced no sample")
        ctx.set_reward(1.0)


def main() -> None:
    backend = LocalBackend(workflow=EchoWorkflow, grader=ReadyGrader)
    app = create_rollout_server(backend=backend)
    port = os.environ.get("_OSMOSIS_ROLLOUT_PORT")
    uvicorn.run(
        app,
        host="0.0.0.0" if port else "127.0.0.1",
        port=int(port or "8000"),
    )


if __name__ == "__main__":
    main()
```

--------------------------------

### Install and authenticate with Osmosis CLI

Source: https://docs.osmosis.ai/cli/overview

Install the required package, verify the version, and perform interactive authentication. CI environments can use the OSMOSIS_TOKEN environment variable instead of the login command.

```bash
python -m pip install "osmosis-ai>=0.3.0rc1,<0.4"
osmosis --version
osmosis auth login
```

--------------------------------

### Install Osmosis Agent Skill

Source: https://docs.osmosis.ai/sdk/ai-tools

Command to install the hosted Agent Skill for Osmosis documentation.

```bash
npx skills add https://docs.osmosis.ai
```

--------------------------------

### Benchmark Configuration Example

Source: https://docs.osmosis.ai/cli/config-files

A complete TOML configuration file for submitting a benchmark, including experiment, task, agent, and execution settings.

```toml
[experiment]
benchmark = "terminal-bench-2-1"               # Benchmark key from `benchmark list`

# Omit [tasks] to run all tasks.
[tasks]
task_names = ["terminal-bench/git-multibranch"] # Explicit task names
# categories = ["software-engineering"]       # Union tasks from these categories
# task_set = "parity"                         # Benchmark-defined parity sample

[[agents]]
harness = "codex"

[agents.model]
type = "provider"
model = "openai/gpt-5.2"
api_key_secret = "OPENAI_API_KEY"             # Platform secret record name

# [agents.env]
# AGENT_MODE = "strict"                       # Literal env for this agent

[execution]
attempts_per_task = 1
max_concurrent_attempts = 4
timeout_multiplier = 1.0
max_retries = 0
# pass_threshold = 1.0
# Judge-enabled benchmarks only:
# judge_model = "openai/gpt-5.2"
# judge_api_key_secret = "OPENAI_API_KEY"

# Registry datasets whose verifier reads its own credentials:
# [verifier]
# required = ["MY_VLM_KEY"]                   # Platform secret record names

# [env]
# LOG_LEVEL = "info"                          # Literal env for every agent
```

--------------------------------

### Submit Configurations with Secrets

Source: https://docs.osmosis.ai/cli/config-files

Examples of submitting benchmark or training configurations using environment variables, secrets files, or piped input.

```bash
# Nothing to pass: the name is the env var name, so an exported value is found
osmosis benchmark submit configs/benchmark/custom.toml

# Many values from a dotenv file
osmosis benchmark submit configs/benchmark/custom.toml --secrets-file .env.local

# Training and eval take the same flag
osmosis train submit configs/training/my-run.toml --secrets-file .env.local

# Piped from a secret manager: secrets.env.tpl holds WEATHER_API_KEY={{ op://vault/weather/key }}, so no value lands on disk
op inject -i secrets.env.tpl | osmosis benchmark submit configs/benchmark/custom.toml --secrets-file - --yes
```

--------------------------------

### Configure Rollout Server Prewarming

Source: https://docs.osmosis.ai/migration-guides/v0-3

Initialize the rollout server with prewarming to prepare task images and agent setup before traffic acceptance.

```python
from osmosis_ai.rollout.server import create_rollout_server

app = create_rollout_server(
    backend=backend,
    lifespan=backend.prewarm_lifespan(),
)
```

```python
app = create_rollout_server(
    backend=backend,
    lifespan=backend.prewarm_lifespan(["task-a", "task-b"]),
)
```

--------------------------------

### Execute eval info commands

Source: https://docs.osmosis.ai/cli/command-reference

Examples of running the command for standard output and JSON-formatted output with a custom directory.

```bash
osmosis eval info my-eval-run
osmosis --json eval info my-eval-run -o ./eval-outputs/
```

--------------------------------

### Verify SDK and CLI Installation

Source: https://docs.osmosis.ai/sdk/installation

Checks the installed version of the Python package and the CLI tool.

```bash
python -c "import osmosis_ai; print(osmosis_ai.__version__)"
osmosis --version
```

--------------------------------

### Initialize a Basic Strands Agent

Source: https://docs.osmosis.ai/sdk/integrations/strands

Demonstrates the minimal setup for an OsmosisStrandsAgent within an AgentWorkflow.

```python
from osmosis_ai.rollout import AgentWorkflow, AgentWorkflowContext
from osmosis_ai.rollout.integrations.agents.strands import (
    OsmosisRolloutModel,
    OsmosisStrandsAgent,
)


class StrandsWorkflow(AgentWorkflow):
    async def run(self, ctx: AgentWorkflowContext) -> None:
        agent = OsmosisStrandsAgent(
            name="assistant",
            model=OsmosisRolloutModel(params={"temperature": 1.0}),
            messages=ctx.prompt,
            callback_handler=None,
        )
        await agent.invoke_async()
```

--------------------------------

### Configure a Cursor CLI Agent

Source: https://docs.osmosis.ai/cli/config-files

Example configuration for a provider-based agent using the cursor-cli harness.

```toml
[[agents]]
harness = "cursor-cli"
harness_api_key_secret = "CURSOR_API_KEY"

[agents.model]
type = "provider"
model = "openai/gpt-5.2"
api_key_secret = "OPENAI_API_KEY"
```

--------------------------------

### Upgrade osmosis-ai package

Source: https://docs.osmosis.ai/migration-guides/v0-3

Install the required v0.3 features for Harbor and server functionality.

```bash
pip install --upgrade "osmosis-ai[server,harbor]>=0.3.0rc1,<0.4"
```

--------------------------------

### Template Mode Directory Structure

Source: https://docs.osmosis.ai/sdk/execution-backends/harbor-backend

Example directory layout for a project configured in template mode.

```text
rollouts/my-rollout/
├── main.py
├── pyproject.toml
└── task-template/
    ├── task.toml
    └── environment/
        └── Dockerfile
```

--------------------------------

### Prompt Mode Dataset Example (JSONL)

Source: https://docs.osmosis.ai/platform/datasets

Use this format when providing user prompts and ground truth answers for model evaluation.

```jsonl
{"system_prompt": "You are a helpful math tutor.", "user_prompt": "What is 15 * 23?", "ground_truth": "345"}
{"system_prompt": "You are a helpful math tutor.", "user_prompt": "Simplify 3/9.", "ground_truth": "1/3"}
```

--------------------------------

### Install Osmosis SDK v0.3 RC

Source: https://docs.osmosis.ai/sdk/installation

Pins the environment to the v0.3 release candidate line to ensure compatibility with pre-release builds.

```bash
python -m pip install "osmosis-ai>=0.3.0rc1,<0.4"
```

--------------------------------

### CLI Command Execution

Source: https://docs.osmosis.ai/cli/overview

Examples of using global flags like --json and --yes for machine-readable output and automated confirmation.

```cli
osmosis --json train info <run-name>
osmosis benchmark submit <config-path> --yes
```

--------------------------------

### Configure a Hosted LoRA Agent

Source: https://docs.osmosis.ai/cli/config-files

Example configuration for an agent using a deployed LoRA model.

```toml
[[agents]]
harness = "codex"

[agents.model]
type = "hosted"
base_model = "Qwen/Qwen3.6-35B-A3B"
lora_model_name = "deep-swe-agent"
```

--------------------------------

### Install Osmosis AI Extras

Source: https://docs.osmosis.ai/migration-guides/v0-3

Upgrade the package by selecting specific feature extras via pip.

```bash
pip install --upgrade "osmosis-ai[strands]>=0.3.0rc1,<0.4"
```

```bash
pip install --upgrade "osmosis-ai[openai-agents]>=0.3.0rc1,<0.4"
```

```bash
pip install --upgrade "osmosis-ai[server]>=0.3.0rc1,<0.4"
```

```bash
pip install --upgrade "osmosis-ai[full]>=0.3.0rc1,<0.4"
```

--------------------------------

### List available starter templates

Source: https://docs.osmosis.ai/sdk/overview

Displays all available templates that can be applied to the current workspace.

```bash
osmosis template list
```

--------------------------------

### Upload and list datasets

Source: https://docs.osmosis.ai/platform/quickstart

Upload the starter dataset and verify its presence in the workspace.

```bash
osmosis dataset upload data/multiply.jsonl
osmosis dataset list
```

--------------------------------

### Manage rollout templates

Source: https://docs.osmosis.ai/cli/workspace/structure-and-config

List available templates and apply a specific starter template to the workspace.

```bash
osmosis template list
osmosis template apply multiply-local-strands
```

--------------------------------

### Install Osmosis CLI

Source: https://docs.osmosis.ai/cli/installation

Install the CLI package using common Python package managers. Python 3.12 or later is required.

```bash
pip install osmosis-ai
```

```bash
pipx install osmosis-ai
```

```bash
uv tool install osmosis-ai
```

--------------------------------

### Execute the rollout server entrypoint

Source: https://docs.osmosis.ai/sdk/installation

Command to run the previously defined main.py script.

```bash
python main.py
```

--------------------------------

### Apply a starter template

Source: https://docs.osmosis.ai/sdk/overview

Applies a specific starter template to the current workspace directory to scaffold rollout code and configurations.

```bash
osmosis template apply multiply-local-strands
```

```bash
osmosis template apply multiply-local-openai
```

--------------------------------

### Initialize a LocalBackend server

Source: https://docs.osmosis.ai/sdk/execution-backends

Instantiate a LocalBackend with specific workflow and grader configurations, then wrap it in a rollout server.

```python
from osmosis_ai.rollout import LocalBackend
from osmosis_ai.rollout.server import create_rollout_server

backend = LocalBackend(
    workflow=MyWorkflow,
    workflow_config=my_workflow_config,
    grader=MyGrader,
    grader_config=my_grader_config,
)
app = create_rollout_server(backend=backend)
```

--------------------------------

### Create or apply a rollout

Source: https://docs.osmosis.ai/skill.md

Use the CLI to list and apply templates or initialize a new rollout project.

```bash
# Option A: Apply a starter template
osmosis template list
osmosis template apply multiply-local-strands

# Option B: Create a new rollout
osmosis rollout init my-rollout
```

--------------------------------

### Initialize LocalBackend

Source: https://docs.osmosis.ai/migration-guides/v0-3

Instantiate the LocalBackend using the same keyword-only constructor as in previous versions.

```python
from osmosis_ai.rollout import LocalBackend

backend = LocalBackend(
    workflow=MyWorkflow,
    workflow_config=my_workflow_config,
    grader=MyGrader,
    grader_config=my_grader_config,
)
```

--------------------------------

### Format Secrets File

Source: https://docs.osmosis.ai/cli/config-files

Example of a dotenv-style secrets file format supported by the --secrets-file flag.

```dotenv
# comment
WEATHER_API_KEY=plain-value
export ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY="sk-...with spaces"
```

--------------------------------

### Initialize a Rollout

Source: https://docs.osmosis.ai/cli/workspace/overview

Scaffold a new rollout within the workspace.

```cli
osmosis rollout init <name>
```

--------------------------------

### Copy Benchmark Template

Source: https://docs.osmosis.ai/platform/benchmarks

Initialize a new benchmark configuration by copying the default template.

```bash
cp configs/benchmark/default.toml configs/benchmark/terminal-bench-smoke.toml
```

--------------------------------

### Training Run Completed Payload

Source: https://docs.osmosis.ai/platform/webhooks

Example payload for a completed training run event, including metrics and run metadata.

```json
{
  "event": "training_run.completed",
  "timestamp": "2026-07-15T22:14:05.123456+00:00",
  "data": {
    "id": "a1b2c3d4-0000-0000-0000-000000000000",
    "name": "my-training-run",
    "status": "finished",
    "started_at": "2026-07-15T20:01:12+00:00",
    "completed_at": "2026-07-15T22:14:03+00:00",
    "duration_seconds": 7971,
    "model_name": "Qwen3-8B",
    "dataset": { "id": "d1e2f3a4-0000-0000-0000-000000000000", "name": "my-dataset" },
    "rollout": { "id": "r1s2t3u4-0000-0000-0000-000000000000", "name": "my-rollout" },
    "platform_url": "https://platform.osmosis.ai/my-workspace/training/a1b2c3d4-0000-0000-0000-000000000000",
    "latest_metrics": {
      "rollout/raw_reward": 0.82,
      "eval/validation/reward": 0.79,
      "train/entropy_loss": 1.24,
      "rollout/response_lengths": 512,
      "rollout/total_lengths": 1930,
      "rollout/truncated_ratio": 0.03
    }
  }
}
```

--------------------------------

### Metadata Mode Dataset Example (JSONL)

Source: https://docs.osmosis.ai/platform/datasets

Use this format to provide per-row metadata objects for AgentWorkflow and Grader processing.

```jsonl
{"metadata": {"task": "multiply", "left": 15, "right": 23}}
{"metadata": {"task": "simplify_fraction", "numerator": 3, "denominator": 9}}
```

--------------------------------

### Submit Training Run

Source: https://docs.osmosis.ai/cli/workspace/overview

Submit a training configuration file to the platform.

```cli
osmosis train submit configs/training/<name>.toml
```

--------------------------------

### Submit training

Source: https://docs.osmosis.ai/skill.md

Submit a training configuration and monitor the lifecycle logs.

```bash
osmosis train submit configs/training/my-rollout.toml
osmosis train info <run-name>  # Monitor training
osmosis train logs <run-name>  # View lifecycle logs
```

--------------------------------

### Configure HarborBackend in Template Mode

Source: https://docs.osmosis.ai/sdk/execution-backends/harbor-backend

Initialization of the HarborBackend using template mode with a specified task directory and environment configuration.

```python
from pathlib import Path

from harbor.models.environment_type import EnvironmentType
from harbor.models.trial.config import EnvironmentConfig as HarborEnvironmentConfig
from harbor.trial.queue import TrialQueue
from osmosis_ai.rollout.backend.harbor import HarborBackend
from osmosis_ai.rollout.server import create_rollout_server

backend = HarborBackend(
    orchestrator=TrialQueue(n_concurrent=4),
    tasks_dir=Path("task-template"),
    task_mode="template",
    agent=MyWorkflow,
    workflow_config=my_workflow_config,
    grader=MyGrader,
    grader_config=my_grader_config,
    environment_config=HarborEnvironmentConfig(type=EnvironmentType.SKYPILOT),
    max_queue_depth=8,
)

app = create_rollout_server(
    backend=backend,
    lifespan=backend.prewarm_lifespan(),
)
```

--------------------------------

### Show benchmark run info

Source: https://docs.osmosis.ai/cli/command-reference

Retrieves configuration, progress, and metric details for a specific benchmark run.

```cli
osmosis benchmark runs info <run-name>
```

--------------------------------

### Submit a training run via CLI

Source: https://docs.osmosis.ai/platform/training-runs

Execute a training run using a specified TOML configuration file.

```bash
osmosis train submit configs/training/default.toml
```

--------------------------------

### Upgrade the CLI

Source: https://docs.osmosis.ai/cli/command-reference

Updates the CLI to the latest version. The command automatically detects whether it was installed via pip, pipx, or uv tool.

```bash
osmosis upgrade
```

--------------------------------

### JSON output structure for model list

Source: https://docs.osmosis.ai/cli/command-reference

Example of the JSON structure returned when using the --json flag with the model list command.

```json
{
  "base_models": { "items": [...], "next_offset": null },
  "lora_models": { "items": [...], "next_offset": null },
  "active_deployments": 2,
  "max_active_deployments": 5
}
```

--------------------------------

### List and Inspect Benchmarks

Source: https://docs.osmosis.ai/platform/benchmarks

Use these commands to view available benchmarks and retrieve metadata for a specific benchmark.

```bash
osmosis benchmark list
osmosis benchmark info terminal-bench-2-1
```

--------------------------------

### List Synced Rollouts

Source: https://docs.osmosis.ai/platform/rollouts

CLI command to display all rollouts currently synced from the repository.

```cli
osmosis rollout list [--branch NAME]
```

--------------------------------

### Benchmark Run Completed Event Payload

Source: https://docs.osmosis.ai/platform/webhooks

Example JSON payload for the benchmark_run.completed event, containing run metadata and agent performance statistics.

```json
{
  "event": "benchmark_run.completed",
  "timestamp": "2026-07-15T22:14:05.123456+00:00",
  "data": {
    "id": "b1c2d3e4-0000-0000-0000-000000000000",
    "name": "terminal-bench-smoke",
    "status": "finished",
    "benchmark_name": "Terminal-Bench 2.1",
    "started_at": "2026-07-15T21:40:00+00:00",
    "completed_at": "2026-07-15T22:14:03+00:00",
    "duration_seconds": 2043,
    "platform_url": "https://platform.osmosis.ai/my-workspace/benchmarks/runs/b1c2d3e4-0000-0000-0000-000000000000",
    "agents": [
      {
        "id": "c1d2e3f4-0000-0000-0000-000000000000",
        "agent_index": 0,
        "label": "codex",
        "model": "openai/gpt-5.2",
        "tasks": 10,
        "attempts": 10,
        "passed": 7,
        "pass_rate": 0.7
      }
    ]
  }
}
```

--------------------------------

### Ask the Agent to Plan

Source: https://docs.osmosis.ai/sdk/create-a-rollout

Provide this prompt to the agent to initiate the planning skill and establish the dataset schema.

```text
I want to train a model for <task> in this Osmosis workspace. Start with the `plan-training` skill: read the workspace instructions, help me settle the dataset plan, and propose the next step before creating rollouts, running evaluation runs, or submitting a training run.
```

--------------------------------

### Submit a training run

Source: https://docs.osmosis.ai/cli/training

Commands to list datasets, push workspace changes, and submit a training configuration.

```bash
osmosis dataset list
git push
osmosis train submit configs/training/my-rollout.toml
```

--------------------------------

### Preview dataset rows via CLI

Source: https://docs.osmosis.ai/platform/datasets

Displays the first few rows of a specified dataset.

```bash
osmosis dataset preview my-dataset --rows 5
```

--------------------------------

### List Base Models via CLI

Source: https://docs.osmosis.ai/platform/models

Retrieves a list of available base models including their name, creation date, and creator.

```bash
osmosis model list --type base
```

--------------------------------

### Rollout directory structure

Source: https://docs.osmosis.ai/cli/workspace/structure-and-config

The expected layout for a rollout subdirectory containing the entrypoint and dependencies.

```text
rollouts/
└── my-rollout/
    ├── main.py          # Entrypoint: defines AgentWorkflow + Grader
    ├── pyproject.toml   # Rollout package dependencies
    └── README.md        # Rollout notes
```

--------------------------------

### Download Benchmark Run Outputs

Source: https://docs.osmosis.ai/cli/benchmarks

Downloads benchmark run data to a local directory. Use the --type flag to include artifacts and logs, and --overwrite to replace existing files.

```bash
osmosis benchmark runs download my-benchmark-run
osmosis benchmark runs download my-benchmark-run --type all -o ./benchmark-outputs/
```

--------------------------------

### List and download datasets via CLI

Source: https://docs.osmosis.ai/platform/datasets

Commands for listing all datasets in the workspace and downloading a specific dataset file.

```bash
# List all datasets in the current workspace
osmosis dataset list

# Download a dataset file
osmosis dataset download my-dataset
```

--------------------------------

### Configure Project Dependencies

Source: https://docs.osmosis.ai/migration-guides/v0-3

Combine multiple extras in project configuration files.

```toml
[project]
dependencies = [
    "osmosis-ai[server,strands]>=0.3.0rc1,<0.4",
]
```

--------------------------------

### Rollout Directory Structure

Source: https://docs.osmosis.ai/sdk/overview

The standard file hierarchy for a rollout, including configuration and data directories.

```text
repository/
├── rollouts/
│   └── my-rollout/
│       ├── main.py
│       └── pyproject.toml
├── configs/
│   ├── eval/
│   │   └── my-rollout.toml
│   └── training/
│       └── my-rollout.toml
└── data/
    └── test.jsonl
```

--------------------------------

### Download evaluation run outputs

Source: https://docs.osmosis.ai/cli/command-reference

Downloads metrics, trajectories, artifacts, and logs from the platform to a local directory.

```cli
osmosis eval download <run-name> [--type TYPES] [--rows ROWS] [--output PATH] [--overwrite] [--yes]
```

--------------------------------

### Verify HarborBackend migration via CLI

Source: https://docs.osmosis.ai/migration-guides/v0-3

Use ripgrep to search for deprecated class names and constructor parameters in the rollouts directory.

```bash
rg "HarborBackendV2|HarborAgentWorkflowContext|OsmosisInstalledAgent|task_dir=|user_code_dir=|workflow=|custom_tests_dir=|prebuild_local_image=|symlink_environment=" rollouts
```

--------------------------------

### Test rollout entrypoint imports

Source: https://docs.osmosis.ai/skill.md

Verify that the rollout entrypoint can be imported without errors using a Python one-liner.

```python
python -c "from rollouts.my_rollout.main import app"
```

--------------------------------

### Implement Custom Sample Source

Source: https://docs.osmosis.ai/migration-guides/v0-3

Define a custom sample source by implementing the singular source contract without setting an ID on the RolloutSample.

```python
from typing import Any

from osmosis_ai.rollout import RolloutSample, SampleSource


class MySampleSource(SampleSource):
    def __init__(self, messages: list[dict[str, Any]]) -> None:
        self.messages = messages

    async def get_sample(self) -> RolloutSample:
        return RolloutSample(messages=self.messages)


rollout_ctx.set_sample_source(MySampleSource(messages))
sample = await rollout_ctx.get_sample()
```

--------------------------------

### Download benchmark run data

Source: https://docs.osmosis.ai/cli/command-reference

Use this command to fetch summary metrics, results, artifacts, or logs for a specific benchmark run.

```cli
osmosis benchmark runs download <run-name> [--type TYPES] [--output PATH] [--overwrite] [--yes]
```

--------------------------------

### Define Sample and Reward JSON Schemas

Source: https://docs.osmosis.ai/migration-guides/v0-3

Use these structures for custom backend execution results and reward files.

```json
{
  "messages": [],
  "reward": null,
  "remove_sample": false,
  "metrics": {},
  "extra_fields": {}
}
```

```json
{
  "reward": 1.0
}
```

--------------------------------

### Inspect Benchmark Runs

Source: https://docs.osmosis.ai/cli/benchmarks

Commands to list, retrieve information, and view logs for specific benchmark runs.

```cli
osmosis benchmark runs list
osmosis benchmark runs info <run-name>
osmosis benchmark runs logs <run-name>
```

--------------------------------

### Apply a template

Source: https://docs.osmosis.ai/cli/command-reference

Applies a specific template by name. Use the --force flag to overwrite existing rollout directories and configuration files.

```cli
osmosis template apply <name> [--force]
```

--------------------------------

### Download benchmark run outputs via CLI

Source: https://docs.osmosis.ai/platform/benchmarks

Use these commands to retrieve run summaries, results, and artifacts. The --type flag allows filtering specific output categories.

```cli
# Default: summary.csv and results.csv
osmosis benchmark runs download <run-name>

# Include every available export and result artifact
osmosis benchmark runs download <run-name> --type all

# Download only selected output types
osmosis benchmark runs download <run-name> --type summary,logs
```

--------------------------------

### Harbor Constructor

Source: https://docs.osmosis.ai/sdk/execution-backends/harbor-backend

The Harbor constructor initializes the execution backend with specific task, agent, and environment configurations.

```APIDOC
## Harbor Constructor

### Description
Initializes the Harbor backend for managing trial execution, task grading, and agent workflows.

### Parameters
- **orchestrator** (TrialQueue) - Required - Harbor queue used to run trials.
- **tasks_dir** (Path) - Required - One task directory in template mode, or a task-root directory in dataset mode.
- **agent** (type | str | None) - Optional - Workflow class, import path, registered native agent name, or None.
- **native_agent_kwargs** (dict[str, Any] | None) - Optional - Extra configuration for a registered native agent.
- **task_mode** (str) - Optional - "template" or "dataset".
- **model_name** (str) - Optional - Model passed to native agents; defaults to "openai/osmosis-rollout".
- **grader** (type | str | None) - Optional - Osmosis grader used when the task does not provide tests/test.sh.
- **workflow_config** (Any) - Optional - Workflow config instance or import path.
- **grader_config** (Any) - Optional - Grader config instance or import path.
- **code_dir** (Path | None) - Optional - Python project to package.
- **bundle** (Path | None) - Optional - Prebuilt Osmosis bundle wheel.
- **environment_config** (EnvironmentConfig | None) - Optional - Harbor runtime and placement configuration.
- **trials_dir** (Path | None) - Optional - Host directory for Harbor trial data.
- **cleanup_successful_trials** (bool) - Optional - Removes successful trial staging after artifacts are archived; defaults to True.
- **patch_dockerfile_with_sdk** (bool | None) - Optional - Preinstalls bundle dependencies into the copied task image.
- **agent_setup_timeout_sec** (float | None) - Optional - Optional Harbor agent setup timeout.
- **max_queue_depth** (int | None) - Optional - Maximum waiting queue depth.
```

--------------------------------

### List All Models

Source: https://docs.osmosis.ai/platform/models

Display both base models and LoRA models side by side.

```bash
osmosis model list
```
Tokens:
Raw