### Build and Run Dashboard

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Builds the dashboard by installing dependencies and running a build script, then starts the Exo server. A delay is included to ensure the server is ready.

```bash
# Build the dashboard (must be done before running exo)
cd dashboard && npm install && npm run build && cd ..

# Start exo (serves the dashboard at http://localhost:52415)
uv run exo &
sleep 8  # Wait for server to start
```

--------------------------------

### Clone and build EXO dashboard

Source: https://github.com/exo-explore/exo/blob/main/CONTRIBUTING.md

Clone the EXO repository, install dashboard dependencies, build the dashboard, and then run the main EXO application. This is the primary setup command for running EXO from source.

```bash
git clone https://github.com/exo-explore/exo.git
cd exo/dashboard
npm install && npm run build && cd ..
uv run exo
```

--------------------------------

### Install uv and Node.js with Homebrew

Source: https://github.com/exo-explore/exo/blob/main/README.md

Installs uv for Python dependency management and Node.js for building the dashboard using Homebrew. This command assumes Homebrew is already installed.

```bash
brew install uv node
```

--------------------------------

### Install Dependencies on Ubuntu/Debian

Source: https://github.com/exo-explore/exo/blob/main/README.md

Installs Node.js, npm, uv, and Rust using apt and curl. Ensure you have the necessary permissions.

```bash
# Install Node.js and npm
sudo apt update
sudo apt install nodejs npm

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install Rust (using rustup)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain install nightly
```

--------------------------------

### Create Instance Request Body Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example JSON request body for creating a new model instance.

```json
{
  "instance": {
    "model_id": "llama-3.2-1b",
    "placement": { }
  }
}
```

--------------------------------

### Install Playwright for Screenshots

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Installs Playwright and its Chromium browser dependency. This is a one-time setup for taking headless screenshots.

```bash
npx --yes playwright install chromium
cd /tmp && npm init -y && npm install playwright
```

--------------------------------

### Install Dependencies using Homebrew on Linux

Source: https://github.com/exo-explore/exo/blob/main/README.md

Installs Homebrew, uv, node, and Rust. This method is an alternative to using the system package manager.

```bash
# Install Homebrew on Linux
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Install dependencies
brew install uv node

# Install Rust (using rustup)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain install nightly
```

--------------------------------

### Install Exo macOS App with Homebrew

Source: https://github.com/exo-explore/exo/blob/main/README.md

Installs the latest build of the Exo macOS app using Homebrew Cask.

```bash
brew install --cask exo
```

--------------------------------

### Install Rust Toolchain (Nightly)

Source: https://github.com/exo-explore/exo/blob/main/README.md

Installs the Rust programming language toolchain, including the nightly version required for building Rust bindings. This script downloads and runs the official Rust installer.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain install nightly
```

--------------------------------

### Build and Run Dashboard

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Builds the dashboard using npm and starts the exo server. Ensure the dashboard is built before running exo.

```bash
cd dashboard && npm install && npm run build && cd ..

uv run exo &
sleep 8  # Wait for server to start
```

--------------------------------

### Inspect System State

Source: https://github.com/exo-explore/exo/blob/main/README.md

Get information about instance IDs and the deployment state of the system.

```bash
curl http://localhost:52415/state
```

--------------------------------

### Install uv with Homebrew

Source: https://github.com/exo-explore/exo/blob/main/CONTRIBUTING.md

Install the uv package manager using Homebrew. This is a prerequisite for managing Python dependencies.

```bash
brew install uv
```

--------------------------------

### Run exo Application

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Starts the exo system, including master and worker processes, with the API accessible at http://localhost:52415.

```bash
# Run exo (starts both master and worker with API at http://localhost:52415)
uv run exo
```

--------------------------------

### Install Rust Toolchain

Source: https://github.com/exo-explore/exo/blob/main/CONTRIBUTING.md

Install the Rust programming language toolchain, including the nightly version required for certain bindings. This is done using the official rustup script.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain install nightly
```

--------------------------------

### Exo Environment Variable Usage Examples

Source: https://github.com/exo-explore/exo/blob/main/README.md

Demonstrates how to set various environment variables to configure Exo's behavior, such as model directories, offline mode, and cluster isolation.

```bash
# Use pre-downloaded models from NFS mount (read-only)
EXO_MODELS_READ_ONLY_DIRS=/mnt/nfs/models:/opt/ai-models uv run exo
```

```bash
# Download models to an external SSD (falls back to default dir if full)
EXO_MODELS_DIRS=/Volumes/ExternalSSD/exo-models uv run exo
```

```bash
# Run in offline mode
EXO_OFFLINE=true uv run exo
```

```bash
# Enable image models
EXO_ENABLE_IMAGE_MODELS=true uv run exo
```

```bash
# Use custom namespace for cluster isolation
EXO_LIBP2P_NAMESPACE=my-dev-cluster uv run exo
```

--------------------------------

### Base URL Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

The base URL for interacting with the EXO API.

```text
http://localhost:52415
```

--------------------------------

### Get Master Node ID Response Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example JSON response when requesting the master node ID.

```json
{
  "node_id": "node-1234"
}
```

--------------------------------

### Run Exo Application

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Starts the exo application, including both master and worker processes, with the API accessible at http://localhost:52415.

```bash
uv run exo
```

--------------------------------

### Ollama Version Response Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

This is an example of the response received when querying the Ollama version endpoint, indicating the compatibility version.

```json
{
  "version": "exo v1.0"
}
```

--------------------------------

### Image Generation Request Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example of a request body for generating an image. Specify prompt, model, and desired output format.

```json
{
  "prompt": "a robot playing chess",
  "model": "exolabs/FLUX.1-dev",
  "n": 1,
  "size": "1024x1024",
  "stream": false,
  "response_format": "b64_json"
}
```

--------------------------------

### Install Homebrew Package Manager

Source: https://github.com/exo-explore/exo/blob/main/README.md

Installs Homebrew, a package manager for macOS, which is a prerequisite for installing other development tools. Run this script in your terminal.

```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

--------------------------------

### Install macmon from Git Repository

Source: https://github.com/exo-explore/exo/blob/main/README.md

Installs a specific fork revision of the macmon tool from its Git repository using Cargo. This is necessary due to a known crash in the Homebrew version on Apple Silicon.

```bash
cargo install --git https://github.com/vladkens/macmon \
  --rev a1cd06b6cc0d5e61db24fd8832e74cd992097a7d \
  macmon \
  --force
```

--------------------------------

### Benchmarked Image Generation Response Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example response for benchmarked image generation, including standard image data and performance metrics.

```json
{
  "created": 1234567890,
  "data": [
    {
      "b64_json": "iVBORw0KGgoAAAANSUhEUgAA...",
      "url": null
    }
  ],
  "generation_stats": {
    "seconds_per_step": 0.123,
    "total_generation_time": 5.456,
    "num_inference_steps": 50,
    "num_images": 1,
    "image_width": 1024,
    "image_height": 1024,
    "peak_memory_usage": 2048
  }
}
```

--------------------------------

### Install macmon with specific Git revision

Source: https://github.com/exo-explore/exo/blob/main/CONTRIBUTING.md

Install the macmon tool from a specific Git revision to ensure compatibility with the EXO project. This is necessary for hardware monitoring on Apple Silicon.

```bash
cargo install --git https://github.com/vladkens/macmon \
  --rev a1cd06b6cc0d5e61db24fd8832e74cd992097a7d \
  macmon \
  --force
```

--------------------------------

### Set RDMA Network Configuration (Source Install)

Source: https://github.com/exo-explore/exo/blob/main/README.md

When running Exo from source, use this script to disable Thunderbolt Bridge and configure DHCP for RDMA ports.

```bash
tmp/set_rdma_network_config.sh
```

--------------------------------

### Run Exo with Nix

Source: https://github.com/exo-explore/exo/blob/main/README.md

Use this command to run Exo directly if you have Nix installed on macOS. Ensure Nix is configured with experimental features and trusted users.

```bash
nix run .#exo
```

--------------------------------

### Get Instance

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Retrieves detailed information about a specific model instance in the EXO cluster.

```APIDOC
## GET /instance/{instance_id}

### Description
Returns details of a specific instance.

### Method
GET

### Endpoint
/instance/{instance_id}

### Parameters
#### Path Parameters
- **instance_id** (string) - Required - The ID of the instance to retrieve.

### Response
#### Success Response (200)
- (JSON) - JSON description of the instance.

### Response Example
```json
{
  "instance_id": "...",
  "model_id": "...",
  "status": "ready",
  "placement": { ... }
}
```
```

--------------------------------

### Image Editing Request Example (Multipart)

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Illustrates the multipart/form-data request structure for image editing, including the input image and prompt.

```text
Request (multipart/form-data):

* `image`: file, required - Input image to edit
* `prompt`: string, required - Text description of desired changes
* `model`: string, required - Image editing model ID (e.g., `exolabs/FLUX.1-Kontext-dev`)
* `n`: integer (default: 1) - Number of edited images to generate
* `size`: string (optional) - Output image dimensions
* `response_format`: string (default: "b64_json") - Either `url` or `b64_json`
* `input_fidelity`: string (default: "low") - Either `low` or `high` - Controls how closely the output follows the input image
* `stream`: string (default: "false") - Enable streaming
* `partial_images`: string (default: "0") - Number of partial images to stream
* `quality`: string (default: "medium") - Either `high`, `medium`, or `low`
* `output_format`: string (default: "png") - Either `png`, `jpeg`, or `webp`
* `advanced_params`: string (optional) - JSON-encoded advanced parameters
```

--------------------------------

### GET /images

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

List all stored images.

```APIDOC
## GET /images

### Description
List all stored images.

### Method
GET

### Endpoint
/images

### Response
#### Success Response (200)
Array of image metadata including URLs and expiration times.
```

--------------------------------

### OpenAI-Compatible Chat Completions Request

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example request body for the OpenAI-compatible chat completions endpoint. Use this for standard chat interactions.

```json
{
  "model": "llama-3.2-1b",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello" }
  ],
  "stream": false
}
```

--------------------------------

### Cancel Command Response

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example response when a command is successfully cancelled via the API.

```json
{
  "message": "Command cancelled.",
  "command_id": "cmd-abc-123"
}
```

--------------------------------

### Await Instance SSE Stream Messages

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example Server-Sent Events (SSE) messages for the Await Instance endpoint, indicating instance readiness or timeout.

```text
data: {"type": "ready", "instance": {...}}

data: {"type": "timeout", "message": "No instance found for model ..."}
```

--------------------------------

### Ollama Show Request Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Send a request to the Ollama show endpoint to retrieve detailed information about a specific model. The 'name' field is required.

```json
{
  "name": "llama-3.2-1b"
}
```

--------------------------------

### Ollama Generate Request Example

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

This snippet demonstrates how to make a text generation request using the Ollama API format. Provide the model and prompt details.

```json
{
  "model": "llama-3.2-1b",
  "prompt": "Hello",
  "stream": false
}
```

--------------------------------

### Example TOML Model Card

Source: https://github.com/exo-explore/exo/blob/main/CONTRIBUTING.md

A TOML file structure for defining model metadata and capabilities. This includes essential fields like model ID, layers, size, and supported tasks.

```toml
model_id = "mlx-community/Llama-3.2-1B-Instruct-4bit"
n_layers = 16
hidden_size = 2048
supports_tensor = true
tasks = ["TextGeneration"]
family = "llama"
quantization = "4bit"
base_model = "Llama 3.2 1B"
capabilities = ["text"]

[storage_size]
in_bytes = 729808896
```

--------------------------------

### Take Headless Screenshots with Playwright

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Launches a headless Chromium browser, navigates to the Exo dashboard, optionally injects test data into localStorage, and takes a screenshot. Ensure you are in the /tmp directory where Playwright was installed.

```javascript
// Run from /tmp where playwright is installed: cd /tmp && node -e "..."
const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch({ headless: true });
  const page = await browser.newPage({ viewport: { width: 1280, height: 800 } });
  await page.goto('http://localhost:52415', { waitUntil: 'networkidle' });
  await page.waitForTimeout(2000);

  // Inject test data into localStorage if needed (e.g., recent models)
  await page.evaluate(() => {
    localStorage.setItem('exo-recent-models', JSON.stringify([
      { modelId: 'mlx-community/Qwen3-30B-A3B-4bit', launchedAt: Date.now() },
    ]));
  });
  await page.reload({ waitUntil: 'networkidle' });
  await page.waitForTimeout(2000);

  // Interact with UI elements
  await page.locator('text=SELECT MODEL').click();
  await page.waitForTimeout(1000);

  // Take screenshot
  await page.screenshot({ path: '/tmp/screenshot.png', fullPage: false });
  await browser.close();
})();
```

--------------------------------

### Ollama API Chat and Tags Endpoints

Source: https://github.com/exo-explore/exo/blob/main/README.md

Exo supports Ollama API endpoints for compatibility with tools like OpenWebUI. Examples include sending a chat request and listing available models.

```bash
# Ollama chat
curl -X POST http://localhost:52415/ollama/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "Hello"}
    ],
    "stream": false
  }'

# List models (Ollama format)
curl http://localhost:52415/ollama/api/tags
```

--------------------------------

### Basic Benchmarking with exo-bench

Source: https://github.com/exo-explore/exo/blob/main/README.md

Run a basic benchmark for a specified model, testing different prompt sizes and generation lengths. Ensure nodes are running with `uv run exo` beforehand.

```bash
uv run bench/exo_bench.py \
  --model Llama-3.2-1B-Instruct-4bit \
  --pp 128,256,512 \
  --tg 128,256
```

--------------------------------

### Preview Model Instance Placements

Source: https://github.com/exo-explore/exo/blob/main/README.md

Use the `/instance/previews` endpoint to see all valid placements for a given model. This helps in selecting an optimal placement before creating an instance.

```bash
curl "http://localhost:52415/instance/previews?model_id=llama-3.2-1b"
```

--------------------------------

### Clone Repo, Build Dashboard, and Run Exo

Source: https://github.com/exo-explore/exo/blob/main/README.md

Clones the Exo repository, builds the dashboard using npm, and then runs Exo using uv.

```bash
# Clone exo
git clone https://github.com/exo-explore/exo

# Build dashboard
cd exo/dashboard && npm install && npm run build && cd ..

# Run exo
uv run exo
```

--------------------------------

### Run Benchmark for Qwen3.5-27B-4bit Model

Source: https://github.com/exo-explore/exo/blob/main/bench/METHODOLOGY.md

Execute the benchmark script for a specified model, instance meta, sharding strategy, and node configuration. Use --help for a full list of flags.

```bash
cd bench && uv run python exo_bench.py \
  --model "mlx-community/Qwen3.5-27B-4bit" \
  --instance-meta jaccl \
  --sharding tensor \
  --min-nodes 2 --max-nodes 2 \
  --pp 512 4096 --tg 128 \
  --repeat 3 \
  --warmup 1
```

--------------------------------

### Filter and Select First Valid Instance Placement

Source: https://github.com/exo-explore/exo/blob/main/README.md

This command pipes the output of the `/instance/previews` endpoint to `jq` to filter for valid placements and then uses `head -n1` to select the first available instance placement.

```bash
curl "http://localhost:52415/instance/previews?model_id=llama-3.2-1b" | jq -c '.previews[] | select(.error == null) | .instance' | head -n1
```

--------------------------------

### Place Instance

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Initiates the placement of an instance for a model using the server's internal placement logic.

```APIDOC
## POST /place_instance

### Description
Places an instance for a model using the server's placement logic. The instance may not be ready immediately.

### Method
POST

### Endpoint
/place_instance

### Parameters
#### Request Body
- (JSON) - JSON describing the instance to be placed.
  - **model_id** (string) - Required - The ID of the model.
  - **placement_constraints** (object) - Optional - Constraints for placement.

### Request Example
```json
{
  "model_id": "llama-3.2-1b",
  "placement_constraints": { "min_replicas": 2 }
}
```

### Response
#### Success Response (200)
- (JSON) - Command acknowledgement. Clients should use `/instance/await` to confirm instance readiness.
```

--------------------------------

### Create a Model Instance

Source: https://github.com/exo-explore/exo/blob/main/README.md

Send a POST request to the `/instance` endpoint with the desired instance placement details in the JSON payload. This operation is asynchronous.

```bash
curl -X POST http://localhost:52415/instance \
  -H 'Content-Type: application/json' \
  -d '{
    "instance": {...}
  }'
```

--------------------------------

### GET /images/{image_id}

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Retrieve a stored image by ID.

```APIDOC
## GET /images/{image_id}

### Description
Retrieve a stored image by ID.

### Method
GET

### Endpoint
/images/{image_id}

### Parameters
#### Path Parameters
- **image_id** (string) - required - ID of the image

### Response
#### Success Response (200)
Image file with appropriate content type.
```

--------------------------------

### List Downloaded Models

Source: https://github.com/exo-explore/exo/blob/main/README.md

Fetch a list of models that have been successfully downloaded and are available for use.

```bash
curl http://localhost:52415/models?status=downloaded
```

--------------------------------

### Await Model Instance Readiness

Source: https://github.com/exo-explore/exo/blob/main/README.md

Use the `/instance/await` endpoint to wait for a model instance to become ready. It returns an SSE stream, emitting 'ready' on success or 'timeout' if a timeout is set.

```bash
curl -N "http://localhost:52415/instance/await?model_id=mlx-community/Llama-3.2-1B-Instruct-4bit"
```

--------------------------------

### Get Master Node ID

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Retrieves the identifier of the current master node in the EXO cluster.

```APIDOC
## GET /node_id

### Description
Returns the identifier of the current master node.

### Method
GET

### Endpoint
/node_id

### Response
#### Success Response (200)
- **node_id** (string) - The identifier of the master node.

### Response Example
```json
{
  "node_id": "node-1234"
}
```
```

--------------------------------

### OpenAI-Compatible Non-Streaming Response

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example of a non-streaming response from the chat completions endpoint, including usage statistics.

```json
{
  "id": "...",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "llama-3.2-1b",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I help you?"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 8,
    "total_tokens": 23
  }
}
```

--------------------------------

### Get Events

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Retrieves a list of internal events recorded by the EXO master, useful for debugging and observability.

```APIDOC
## GET /events

### Description
Returns the list of internal events recorded by the master (mainly for debugging and observability).

### Method
GET

### Endpoint
/events

### Response
#### Success Response (200)
- (Array of event objects) - A list of internal events.

### Response Example
```json
[
  { "timestamp": "...", "level": "info", "message": "..." },
  { "timestamp": "...", "level": "error", "message": "..." }
]
```
```

--------------------------------

### List All Models

Source: https://github.com/exo-explore/exo/blob/main/README.md

Retrieve a list of all available models, including those added from HuggingFace and locally managed ones.

```bash
curl http://localhost:52415/models
```

--------------------------------

### Get Cluster State

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Fetches the current state of the EXO cluster, including node and active instance information.

```APIDOC
## GET /state

### Description
Returns the current state of the cluster, including nodes and active instances.

### Method
GET

### Endpoint
/state

### Response
#### Success Response (200)
- (JSON object) - Describes the cluster topology, nodes, and instances.

### Response Example
```json
{
  "topology": { ... },
  "nodes": [ ... ],
  "instances": [ ... ]
}
```
```

--------------------------------

### Non-streaming Image Generation Response

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example of a non-streaming response for image generation, containing base64 encoded image data.

```json
{
  "created": 1234567890,
  "data": [
    {
      "b64_json": "iVBORw0KGgoAAAANSUhEUgAA...",
      "url": null
    }
  ]
}
```

--------------------------------

### Build Dashboard

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Builds the dashboard component of the exo project. This must be done before running exo.

```bash
# Build the dashboard (required before running exo)
cd dashboard && npm install && npm run build && cd ..
```

--------------------------------

### Build Dashboard

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Builds the dashboard component of the exo project. This must be done before running exo.

```bash
cd dashboard && npm install && npm run build && cd ..
```

--------------------------------

### Preview Instance Placements

Source: https://github.com/exo-explore/exo/blob/main/README.md

Previews all valid placements for a given model. This helps in selecting an appropriate placement before creating an instance.

```APIDOC
## GET /instance/previews

### Description
Previews all valid placements for a given model.

### Method
GET

### Endpoint
/instance/previews

### Parameters
#### Query Parameters
- **model_id** (string) - Required - The ID of the model to preview placements for.

### Response
#### Success Response (200)
- **previews** (array) - A list of available instance placements.
  - **model_id** (string) - The ID of the model.
  - **sharding** (string) - The sharding strategy for the placement.
  - **instance_meta** (string) - Metadata about the instance placement.
  - **instance** (object) - Details about the instance placement.
  - **memory_delta_by_node** (object) - Memory delta by node.
  - **error** (string or null) - Any error associated with the placement.

### Response Example
```json
{
  "previews": [
    {
      "model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit",
      "sharding": "Pipeline",
      "instance_meta": "MlxRing",
      "instance": {},
      "memory_delta_by_node": {"local": 729808896},
      "error": null
    }
  ]
}
```
```

--------------------------------

### Claude Messages API Request

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example request body for the Claude Messages API endpoint. Use this for Claude-specific chat interactions.

```json
{
  "model": "llama-3.2-1b",
  "messages": [
    { "role": "user", "content": "Hello" }
  ],
  "max_tokens": 1024,
  "stream": false
}
```

--------------------------------

### Streaming Image Generation Response

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example of a streaming response format for image generation, using Server-Sent Events for partial and final image data.

```text
data: {"type":"partial","image_index":0,"partial_index":1,"total_partials":5,"format":"png","data":{"b64_json":"..."}}

data: {"type":"final","image_index":0,"format":"png","data":{"b64_json":"..."}}

data: [DONE]
```

--------------------------------

### Await Instance Ready

Source: https://github.com/exo-explore/exo/blob/main/README.md

Waits for a model instance to be ready. Emits an SSE stream with readiness status.

```APIDOC
## GET /instance/await

### Description
Waits for a model instance to be ready. Emits an SSE stream with readiness status.

### Method
GET

### Endpoint
/instance/await

### Parameters
#### Query Parameters
- **model_id** (string) - Required - The ID of the model instance to wait for.
- **timeout_seconds** (integer) - Optional - The maximum time in seconds to wait. Defaults to indefinite.

### Response
#### Success Response (SSE Stream)
- Emits messages with a `"type"` field. Ready state emits `"type": "ready"` with instance details. Timeout state emits `"type": "timeout"`.
```

--------------------------------

### OpenAI Responses API Request

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Example request body for the OpenAI Responses API endpoint. Use this for interactions requiring that specific response format.

```json
{
  "model": "llama-3.2-1b",
  "messages": [
    { "role": "user", "content": "Hello" }
  ],
  "stream": false
}
```

--------------------------------

### Run Exo as Legacy SysV-style Daemon

Source: https://github.com/exo-explore/exo/blob/main/README.md

Launches Exo as a legacy background daemon using double-fork. Intended for legacy init scripts; systemd and launchd should run Exo in the foreground.

```bash
uv run exo --legacy-daemon
```

--------------------------------

### Run All Pre-Commit Checks

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Executes all required pre-commit checks sequentially: type checking, linting, formatting, and tests.

```bash
uv run basedpyright && uv run ruff check && nix fmt && uv run pytest
```

--------------------------------

### Create Instance

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Creates a new model instance within the EXO cluster. Instance creation is asynchronous.

```APIDOC
## POST /instance

### Description
Creates a new model instance in the cluster. Instance creation is asynchronous.

### Method
POST

### Endpoint
/instance

### Parameters
#### Request Body
- **instance** (object) - Required - Details of the instance to create.
  - **model_id** (string) - Required - The ID of the model to instantiate.
  - **placement** (object) - Optional - Placement configuration for the instance.

### Request Example
```json
{
  "instance": {
    "model_id": "llama-3.2-1b",
    "placement": { }
  }
}
```

### Response
#### Success Response (200)
- (JSON) - Command acknowledgement. Clients should use `/instance/await` to confirm instance readiness.
```

--------------------------------

### Run Tests (Default)

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Executes the project's tests, excluding slow tests by default.

```bash
uv run pytest
```

--------------------------------

### POST /bench/chat/completions

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Same as /v1/chat/completions, but also returns performance and generation statistics.

```APIDOC
## POST /bench/chat/completions

### Description
Same as `/v1/chat/completions`, but also returns performance and generation statistics.

### Method
POST

### Endpoint
/bench/chat/completions

### Request Body
Same schema as `/v1/chat/completions`.

### Response
Chat completion plus benchmarking metrics including:
* `prompt_tps` - Tokens per second during prompt processing
* `generation_tps` - Tokens per second during generation
* `prompt_tokens` - Number of prompt tokens
* `generation_tokens` - Number of generated tokens
* `peak_memory_usage` - Peak memory used during generation
```

--------------------------------

### Run Tests (Default)

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Executes the project's tests. By default, this command excludes slow-running tests.

```bash
# Run tests (excludes slow tests by default)
uv run pytest
```

--------------------------------

### Pre-Commit Formatting

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Ensures code formatting is applied correctly using nix before committing. This is a mandatory pre-commit check.

```bash
# 3. Formatting - MUST be applied
nix fmt
```

--------------------------------

### Create Model Instance

Source: https://github.com/exo-explore/exo/blob/main/README.md

Creates a model instance with the specified placement. This is an asynchronous operation.

```APIDOC
## POST /instance

### Description
Creates a model instance with the specified placement. This is an asynchronous operation.

### Method
POST

### Endpoint
/instance

### Parameters
#### Request Body
- **instance** (object) - Required - The desired instance placement details.

### Request Example
```json
{
  "instance": {}
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message.
- **command_id** (string) - The ID of the asynchronous command.

### Response Example
```json
{
  "message": "Command received.",
  "command_id": "e9d1a8ab-...."
}
```
```

--------------------------------

### Advanced Benchmarking with Filters

Source: https://github.com/exo-explore/exo/blob/main/README.md

Execute a benchmark with specific filters for maximum nodes, sharding strategy, and repetition count, outputting results to a custom JSON file. This allows for targeted performance analysis.

```bash
uv run bench/exo_bench.py \
  --model Llama-3.2-1B-Instruct-4bit \
  --pp 128,512 \
  --tg 128 \
  --max-nodes 2 \
  --sharding tensor \
  --repeat 3 \
  --json-out my-results.json
```

--------------------------------

### Clone Exo Repository and Build Dashboard

Source: https://github.com/exo-explore/exo/blob/main/README.md

Clones the Exo source code from GitHub and builds the dashboard application using npm. Navigate into the cloned directory before running these commands.

```bash
# Clone exo
git clone https://github.com/exo-explore/exo

# Build dashboard
cd exo/dashboard && npm install && npm run build && cd ..
```

--------------------------------

### Run exo with Verbose Logging

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Executes the exo application with increased logging verbosity for debugging purposes. Use -v for standard verbose logging or -vv for more detailed output.

```bash
# Run with verbose logging
uv run exo -v   # or -vv for more verbose
```

--------------------------------

### Format Code

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Applies code formatting rules to the project using the nix formatter.

```bash
nix fmt
```

--------------------------------

### Preview Placements

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Generates possible placement previews for a given model without creating an instance.

```APIDOC
## GET /instance/previews

### Description
Returns possible placement previews for a given model.

### Method
GET

### Endpoint
/instance/previews

### Parameters
#### Query Parameters
- **model_id** (string) - Required - The ID of the model for which to preview placements.

### Response
#### Success Response (200)
- (Array of placement preview objects) - Possible placements for the model.

### Response Example
```json
[
  { "node_id": "node-1", "sharding": "..." },
  { "node_id": "node-2", "sharding": "..." }
]
```
```

--------------------------------

### Format Code

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Applies code formatting using nix. Any changes made by this command must be staged before committing.

```bash
# Format code (using nix)
nix fmt
```

--------------------------------

### Run Exo with Verbose Logging

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Executes the exo application with increased logging verbosity for debugging purposes.

```bash
uv run exo -v   # or -vv for more verbose
```

--------------------------------

### Perform Linting

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Checks the codebase for style and potential errors using the linter.

```bash
uv run ruff check
```

--------------------------------

### POST /bench/images/generations

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Same as /v1/images/generations, but also returns generation statistics.

```APIDOC
## POST /bench/images/generations

### Description
Same as `/v1/images/generations`, but also returns generation statistics.

### Method
POST

### Endpoint
/bench/images/generations

### Parameters
#### Request Body
Same schema as `/v1/images/generations`.

### Response
#### Success Response (200)
Image generation plus benchmarking metrics including:
- **seconds_per_step** (float) - Average time per denoising step
- **total_generation_time** (float) - Total generation time
- **num_inference_steps** (integer) - Number of inference steps used
- **num_images** (integer) - Number of images generated
- **image_width** (integer) - Output image width
- **image_height** (integer) - Output image height
- **peak_memory_usage** (integer) - Peak memory used during generation
```

--------------------------------

### Await Instance

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Waits for a model instance to become available in the cluster, providing updates via an SSE stream.

```APIDOC
## GET /instance/await

### Description
Waits until API state contains an instance for the requested model. The response is an SSE stream.

### Method
GET

### Endpoint
/instance/await

### Parameters
#### Query Parameters
- **model_id** (string) - Required - The ID of the model to wait for.
- **timeout_seconds** (float) - Optional, default `0`. `0` waits indefinitely; positive values time out after that many seconds. Maximum positive value: `300`.

### Response
#### Success Response (200)
- (SSE Stream) - Provides `data` messages indicating instance readiness or timeout.

### Stream Messages Example
```text
data: {"type": "ready", "instance": {...}}

data: {"type": "timeout", "message": "No instance found for model ..."}
```
```

--------------------------------

### POST /bench/images/edits

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Same as /v1/images/edits, but also returns generation statistics.

```APIDOC
## POST /bench/images/edits

### Description
Same as `/v1/images/edits`, but also returns generation statistics.

### Method
POST

### Endpoint
/bench/images/edits

### Parameters
#### Request Body
Same schema as `/v1/images/edits`.

### Response
#### Success Response (200)
Same format as `/bench/images/generations`, including `generation_stats`.
```

--------------------------------

### Run All Tests (Including Slow)

Source: https://github.com/exo-explore/exo/blob/main/AGENTS.md

Executes all project tests, including those categorized as slow.

```bash
uv run pytest -m ""
```

--------------------------------

### Perform Linting

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Checks the codebase for style and potential errors using ruff. This check must pass.

```bash
# Linting
uv run ruff check
```

--------------------------------

### Send Chat Completion Request (OpenAI Compatible)

Source: https://github.com/exo-explore/exo/blob/main/README.md

Make a POST request to the `/v1/chat/completions` endpoint, using the same format as OpenAI's API, to send a chat message and receive a response. Supports streaming.

```bash
curl -N -X POST http://localhost:52415/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "What is Llama 3.2 1B?"}
    ],
    "stream": true
  }'
```

--------------------------------

### List Models

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Retrieves a list of all available models and their associated metadata. Supports filtering by download status.

```APIDOC
## GET /models

### Description
Returns the list of available models and their metadata.

### Method
GET

### Endpoint
/models

### Query Parameters
- **status** (string) - Optional - Filter by `downloaded` to show only downloaded models

### Response
#### Success Response
Array of model descriptors including `is_custom` field for custom HuggingFace models.
```

--------------------------------

### Ollama PS

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Returns a list of running models (active instances).

```APIDOC
## GET /ollama/api/ps

### Description
Returns a list of running models (active instances).

### Method
GET

### Endpoint
/ollama/api/ps

### Response
#### Success Response
- **instances** (array) - An array of active model instances.
```

--------------------------------

### Compute Placement

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Computes a potential placement configuration for a model without actually creating an instance.

```APIDOC
## GET /instance/placement

### Description
Computes a placement for a potential instance without creating it.

### Method
GET

### Endpoint
/instance/placement

### Parameters
#### Query Parameters
- **model_id** (string) - Optional - The ID of the model.
- **sharding** (string or config) - Optional - Sharding configuration.
- **instance_meta** (string) - Optional - JSON-encoded metadata for the instance.
- **min_nodes** (integer) - Optional - Minimum number of nodes required for placement.

### Response
#### Success Response (200)
- (JSON object) - Describes the proposed placement and instance configuration.

### Response Example
```json
{
  "placement": { "node_id": "node-1", "sharding": "..." },
  "instance_config": { ... }
}
```
```

--------------------------------

### Instance Management Endpoints

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Endpoints for managing instances, including creation, retrieval, and deletion.

```APIDOC
## Instance Management Endpoints

### POST /instance

### Description
Creates a new instance.

### GET /instance/await

### Description
Waits for instance availability.

### GET /instance/previews

### Description
Retrieves instance previews.

### GET /instance/placement

### Description
Retrieves instance placement information.

### GET /{instance_id}

### Description
Retrieves details for a specific instance.

### DELETE /{instance_id}

### Description
Deletes a specific instance.

### POST /place_instance

### Description
Places an instance.
```

--------------------------------

### Send Responses Request (OpenAI Compatible)

Source: https://github.com/exo-explore/exo/blob/main/README.md

Interact with the API using the OpenAI Responses API format via the `/v1/responses` endpoint. This endpoint also supports streaming.

```bash
curl -N -X POST http://localhost:52415/v1/responses \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "Hello"}
    ],
    "stream": true
  }'
```

--------------------------------

### Search HuggingFace Models

Source: https://github.com/exo-explore/exo/blob/main/README.md

Search for models on the HuggingFace hub using a query string and limit the number of results.

```bash
curl "http://localhost:52415/models/search?query=llama&limit=10"
```

--------------------------------

### Run All Tests

Source: https://github.com/exo-explore/exo/blob/main/CLAUDE.md

Executes all project tests, including those that are typically excluded due to their duration.

```bash
# Run all tests including slow tests
uv run pytest -m ""
```

--------------------------------

### POST /v1/chat/completions

Source: https://github.com/exo-explore/exo/blob/main/docs/api.md

Executes a chat completion request using an OpenAI-compatible schema. Supports streaming and non-streaming modes.

```APIDOC
## POST /v1/chat/completions

### Description
Executes a chat completion request using an OpenAI-compatible schema. Supports streaming and non-streaming modes.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Request Body
* `model` (string) - required - Model ID to use
* `messages` (array) - required - Conversation messages
* `stream` (boolean) - optional - Enable streaming responses (default: false)
* `max_tokens` (integer) - optional - Maximum tokens to generate
* `temperature` (float) - optional - Sampling temperature
* `top_p` (float) - optional - Nucleus sampling parameter
* `top_k` (integer) - optional - Top-k sampling parameter
* `stop` (string or array) - optional - Stop sequences
* `seed` (integer) - optional - Random seed for reproducibility
* `enable_thinking` (boolean) - optional - Enable thinking mode for capable models (DeepSeek V3.1, Qwen3, GLM-4.7)
* `tools` (array) - optional - Tool definitions for function calling
* `logprobs` (boolean) - optional - Return log probabilities
* `top_logprobs` (integer) - optional - Number of top log probabilities to return

### Request Example
```json
{
  "model": "llama-3.2-1b",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello" }
  ],
  "stream": false
}
```

### Response
OpenAI-compatible chat completion response.

#### Success Response (200)
When `stream=true`, returns Server-Sent Events (SSE) with format:
```
data: {"id":"...","object":"chat.completion","created":...,"model":"...","choices":[...]}

data: [DONE]
```

When `stream=false`, returns:
```json
{
  "id": "...",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "llama-3.2-1b",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I help you?"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 8,
    "total_tokens": 23
  }
}
```
```

--------------------------------

### OpenAI Chat Completions

Source: https://github.com/exo-explore/exo/blob/main/README.md

Sends a chat completions request in OpenAI's format.

```APIDOC
## POST /v1/chat/completions

### Description
Sends a chat completions request in OpenAI's format. Supports streaming responses.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Request Body
- **model** (string) - Required - The model to use for the chat completion.
- **messages** (array) - Required - An array of message objects representing the conversation history.
  - **role** (string) - Required - The role of the message sender (e.g., "user", "system", "assistant").
  - **content** (string) - Required - The content of the message.
- **stream** (boolean) - Optional - Whether to stream the response. Defaults to true.

### Request Example
```json
{
  "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "messages": [
    {"role": "user", "content": "What is Llama 3.2 1B?"}
  ],
  "stream": true
}
```
```