### Install and serve LTX-2.5

Source: https://rapidmlx.com/docs/models/families/ltx

Install the video extras, install ffmpeg, and start the local server for the LTX-2.5 model. The server runs on http://localhost:8000/v1.

```bash
pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve ltx-2.5-mlx-q8
```

--------------------------------

### Install and submit a text-to-video job with curl

Source: https://rapidmlx.com/docs/models/families/video

Install the video extras, ffmpeg, and start the server. Then submit a text-to-video job with curl, poll for the result, and download the MP4. The job ID is returned in the first response.

```bash
pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve ltx-2.5-mlx-q8

# Submit a text-to-video job — the MP4 comes back with synchronized audio
curl http://127.0.0.1:8000/v1/videos \
  -F model=ltx-2.5-mlx-q8 \
  -F 'prompt=a fox running through fresh snow, cinematic tracking shot' \
  -F seconds=4 \
  -F size=1280x720 \
  -F seed=42

# Poll it, then download the MP4
curl http://127.0.0.1:8000/v1/videos/video_ID
curl http://127.0.0.1:8000/v1/videos/video_ID/content --output result.mp4
```

--------------------------------

### Install rapid-mlx with guided extra

Source: https://rapidmlx.com/docs/frameworks/langchain.html

Install the guided extra required for structured output. Without it, the server returns guided_extra_required.

```bash
$ pip install "rapid-mlx[guided]"
```

--------------------------------

### Install rapid-mlx[all] extra

Source: https://rapidmlx.com/docs/extras.html

Installs the all extra, which is a union of vision, chat, embeddings, and audio extras. It is expanded inline to avoid the self-dependency pitfall that breaks editable installs. Note that [all] does not include [mtp], [dflash], [guided], [image], [video], [audio-desktop], [mirror], [test], or [dev].

```bash
$ pip install "rapid-mlx[all]==0.13.4"
```

--------------------------------

### Install and serve Z-Image-Turbo with rapid-mlx

Source: https://rapidmlx.com/docs/models/families/z-image.html

Install the image extra and start the local server. The server exposes an OpenAI-compatible endpoint at http://localhost:8000/v1.

```bash
pip install 'rapid-mlx[image]'
rapid-mlx serve z-image-turbo
```

--------------------------------

### Serve model and configure DSH with dry-run

Source: https://rapidmlx.com/docs/agents/deepseek-harness.html

Start a rapid-mlx server with a model, then preview and apply the DSH setup. The setup discovers the running model and its context window, patches `$DSH_HOME/settings.yaml` with a timestamped backup, preserves unrelated providers, writes atomically, and refuses to apply if the file changed after the preview.

```shell
$ rapid-mlx serve qwen3.6-35b-8bit

# in another terminal
$ rapid-mlx agents dsh --setup --dry-run   # preview
$ rapid-mlx agents dsh --setup             # apply

```

--------------------------------

### Install rapid-mlx[guided] extra

Source: https://rapidmlx.com/docs/extras.html

Installs the guided extra for llguidance-based structured JSON output. As of 0.10.15, llguidance is a core dependency, so this extra is retained for backward compatibility.

```bash
$ pip install "rapid-mlx[guided]==0.13.4"
```

--------------------------------

### List and configure agent clients with rapid-mlx agents

Source: https://rapidmlx.com/docs/cli.html

Use the agents subcommand to list available agent clients, view setup guides, auto-configure with --setup, run integration tests with --test, and specify agent versions. Flags include --model (auto-detected), --base-url (default http://localhost:8000/v1), and --agent-version.

```bash
$ rapid-mlx agents                          # list
$ rapid-mlx agents codex                    # setup guide
$ rapid-mlx agents aider --setup            # auto-configure
$ rapid-mlx agents hermes --test            # run integration flow
$ rapid-mlx agents aider --agent-version 0.90.0 --setup
```

--------------------------------

### Install rapid-mlx with video support

Source: https://rapidmlx.com/docs/models/families/cogvideox.html

Install the necessary package to enable video generation capabilities.

```bash
pip install 'rapid-mlx[video]'
```

--------------------------------

### Install OpenCode

Source: https://rapidmlx.com/docs/agents/opencode.html

Install the OpenCode TUI using Homebrew or npm.

```bash
$ brew install sst/tap/opencode
# or
$ npm install -g opencode-ai
```

--------------------------------

### Install rapid-mlx[video] extra

Source: https://rapidmlx.com/docs/extras.html

Installs the video extra for local video generation with LTX-2.5/2.3, Wan, and CogVideoX-Fun models. Requires Apple Silicon and Python >= 3.11. The video startup preflight gives 3.10 users an explicit upgrade diagnostic.

```bash
$ pip install "rapid-mlx[video]==0.13.4"
```

--------------------------------

### Install and Bootstrap launchd Service

Source: https://rapidmlx.com/docs/headless-macos-service

Installs the configured plist to the system launch daemon directory and bootstraps the service.

```bash
$ sudo install -o root -g wheel -m 644 \
  /tmp/com.rapidmlx.server.plist \
  /Library/LaunchDaemons/com.rapidmlx.server.plist
$ sudo launchctl bootstrap system \
  /Library/LaunchDaemons/com.rapidmlx.server.plist
$ sudo launchctl print system/com.rapidmlx.server
```

--------------------------------

### Install Qwen Code CLI

Source: https://rapidmlx.com/docs/agents/qwen-code.html

Install the Qwen Code CLI globally via npm.

```bash
$ npm install -g @qwenlm/qwen-code
```

--------------------------------

### Start rapid-mlx server

Source: https://rapidmlx.com/docs/agents/continue-dev.html

Start the server with a specific model to be used by the Continue extension.

```bash
$ rapid-mlx serve qwen3-coder-30b-4bit
```

--------------------------------

### Run rapid-mlx serve and qwen-code

Source: https://rapidmlx.com/docs/agents/qwen-code.html

Start the rapid-mlx server with a Qwen model and then launch the qwen-code CLI.

```bash
$ rapid-mlx serve qwen3-coder-30b-4bit
```

```bash
$ qwen-code
```

--------------------------------

### Install via Homebrew

Source: https://rapidmlx.com/docs/getting-started.html

Install Rapid-MLX using the Homebrew package manager.

```bash
$ brew install rapid-mlx
```

--------------------------------

### Install Kilo Code

Source: https://rapidmlx.com/docs/agents/kilo-code.html

Installation commands for Kilo Code using Homebrew or npm.

```bash
$ brew install kilo-org/kilo/kilo-code
# or
$ npm install -g kilo-code
```

--------------------------------

### Install langchain-openai

Source: https://rapidmlx.com/docs/frameworks/langchain.html

Install the langchain-openai package to use ChatOpenAI with rapid-mlx.

```bash
$ pip install langchain-openai
```

--------------------------------

### Prepare the service account and verify server

Source: https://rapidmlx.com/docs/headless-macos-service

Install Rapid-MLX, create a log directory, and verify the server runs in the foreground before setting up the daemon.

```bash
$ curl -fsSL https://rapidmlx.com/install.sh | bash
$ mkdir -p "$HOME/Library/Logs/Rapid-MLX"
$ chmod 750 "$HOME/Library/Logs/Rapid-MLX"
$ rapid-mlx serve qwen3.5-4b-4bit --host 127.0.0.1 --port 8000
```

--------------------------------

### Launch IDE clients

Source: https://rapidmlx.com/docs/getting-started.html

Use the launch command to configure and start the server for specific IDE clients.

```bash
$ rapid-mlx launch cursor      # Cursor
$ rapid-mlx launch claude-code # Claude Code
$ rapid-mlx launch aider       # Aider
```

--------------------------------

### Install [chat] extra

Source: https://rapidmlx.com/docs/extras.html

Adds the Gradio-based rapid-mlx-chat front-end.

```bash
$ pip install "rapid-mlx[chat]==0.13.4"
```

--------------------------------

### Install development runtime

Source: https://rapidmlx.com/docs/extras.html

Installs the test runtime plus additional tools for linting and type-checking, such as Ruff and mypy.

```bash
$ pip install "rapid-mlx[dev]==0.13.4"
```

--------------------------------

### Serve Tmax models with rapid-mlx

Source: https://rapidmlx.com/docs/models/tmax.html

Starts the Rapid-MLX server with the tmax-9b model, or tmax-27b for the larger variant. Run this before using the chat completion or tool-call examples.

```bash
$ rapid-mlx serve tmax-9b
# or, for the larger model:
$ rapid-mlx serve tmax-27b
```

--------------------------------

### Install via pip

Source: https://rapidmlx.com/docs/getting-started.html

Install Rapid-MLX within a Python virtual environment.

```bash
$ python3 -m venv .venv && source .venv/bin/activate
$ pip install rapid-mlx
```

--------------------------------

### Start a model server

Source: https://rapidmlx.com/docs/getting-started.html

Launch the server using a model alias. The server will download weights on the first run.

```bash
$ rapid-mlx serve qwen3.5-4b-4bit
# downloads the MLX weights on first run, then:
INFO  uvicorn running on http://127.0.0.1:8000
```

--------------------------------

### Install rapid-mlx[image] extra

Source: https://rapidmlx.com/docs/extras.html

Installs the image extra for local image generation with FLUX.1-schnell and Qwen-Image models. Requires Apple Silicon and Python >= 3.11. The core package still supports Python 3.10, but the image-lane preflight provides an upgrade diagnostic for 3.10 users.

```bash
$ pip install "rapid-mlx[image]==0.13.4"
```

--------------------------------

### Start the OpenAI-compatible HTTP server

Source: https://rapidmlx.com/docs/cli.html

Commands to launch the server with various model configurations and network settings.

```bash
$ rapid-mlx serve qwen3.5-4b-4bit
$ rapid-mlx serve qwen3.6-35b-8bit --port 9000 --host 0.0.0.0
$ rapid-mlx serve gpt-oss-20b-mxfp4-q8 --enable-auto-tool-choice \
    --tool-call-parser harmony
```

--------------------------------

### Run rapid-mlx server and Copilot suggest

Source: https://rapidmlx.com/docs/agents/copilot.html

Start the rapid-mlx server with a model and then use gh copilot suggest to query it.

```bash
$ rapid-mlx serve qwen3.6-35b-4bit
$ gh copilot suggest "find the largest file in this directory"
```

--------------------------------

### Start rapid-mlx Server and Claude Code

Source: https://rapidmlx.com/docs/agents/claude-code.html

Commands to start the server and connect the Claude Code CLI.

```bash
$ rapid-mlx serve qwen3.6-35b-4bit
# in another shell:
$ ANTHROPIC_BASE_URL=http://localhost:8000 ANTHROPIC_API_KEY=not-needed claude
```

--------------------------------

### Install OpenAI Python SDK

Source: https://rapidmlx.com/docs/agents/gpt-oss.html

Install the OpenAI Python SDK to interact with rapid-mlx's /v1/chat/completions endpoint.

```bash
$ pip install openai
```

--------------------------------

### Install Aider

Source: https://rapidmlx.com/docs/agents/aider.html

Install the aider-chat package via pip.

```bash
$ pip install aider-chat
```

--------------------------------

### Install rapid-mlx extras via pip

Source: https://rapidmlx.com/docs/extras.html

Use bracket syntax to install specific optional dependencies. Quote the package string when using zsh.

```bash
$ pip install "rapid-mlx[vision]==0.13.4"
$ pip install "rapid-mlx[vision,mtp]==0.13.4"    # combine
$ pip install "rapid-mlx[all]==0.13.4"           # vision + chat + embeddings + audio
```

--------------------------------

### Install OpenHands

Source: https://rapidmlx.com/docs/agents/openhands.html

Install the OpenHands package. Ensure Docker Desktop or dockerd is running as it requires a sandbox-runtime container image.

```bash
$ pip install openhands
# Docker Desktop / dockerd must be running — OpenHands ships a
# sandbox-runtime container image and a docker-in-docker sock passthrough.
```

--------------------------------

### Install via curl

Source: https://rapidmlx.com/docs/getting-started.html

Install Rapid-MLX using the provided shell script.

```bash
$ curl -fsSL https://rapidmlx.com/install.sh | bash
```

--------------------------------

### Install Anthropic Python SDK

Source: https://rapidmlx.com/docs/agents/claude-code.html

Install the Python SDK to interact with the rapid-mlx server programmatically.

```bash
$ pip install anthropic
```

--------------------------------

### Run rapid-mlx serve and Codex CLI

Source: https://rapidmlx.com/docs/agents/codex-cli.html

Run the local server and start Codex CLI. Start `rapid-mlx serve` with a model alias, then in another shell run `codex` for interactive use or `codex exec` for a one-shot command. Recommended aliases per `rapid-mlx agents codex` are `qwen3.6-35b-4bit`, `qwen3.5-9b-4bit`, and `qwen3-coder-30b-4bit`.

```bash
$ rapid-mlx serve qwen3.6-35b-4bit
# in another shell:
$ codex
# or one-shot exec:
$ codex exec "read main.py and summarize what it does"
```

--------------------------------

### Install PydanticAI

Source: https://rapidmlx.com/docs/frameworks/pydantic-ai.html

Install the PydanticAI package using pip.

```bash
$ pip install pydantic-ai
```

--------------------------------

### Serve and generate video with CogVideoX-Fun

Source: https://rapidmlx.com/docs/models/families/cogvideox.html

Start the model server and use curl to request a video generation task.

```bash
rapid-mlx serve cogvideox-fun-5b-q4

curl http://127.0.0.1:8000/v1/videos \
  -F model=cogvideox-fun-5b-q4 \
  -F 'prompt=a fox running through fresh snow, cinematic tracking shot' \
  -F seconds=1 \
  -F size=672x384
```

--------------------------------

### Serve a model via CLI

Source: https://rapidmlx.com/docs/frameworks/openai-sdk.html

Command to start the rapid-mlx server with a specific model.

```bash
$ rapid-mlx serve qwen3.5-9b-4bit
⚡ serving on http://localhost:8000/v1
```

--------------------------------

### Install [audio] extra

Source: https://rapidmlx.com/docs/extras.html

Required for TTS and STT functionality. Includes specific pins for mlx-audio and requires a phonemizer-fork.

```bash
$ pip install "rapid-mlx[audio]==0.13.4"
```

--------------------------------

### Install smolagents

Source: https://rapidmlx.com/docs/frameworks/smolagents.html

Install the smolagents package via pip.

```bash
$ pip install smolagents
```

--------------------------------

### Install Hermes Agent

Source: https://rapidmlx.com/docs/agents/hermes.html

Install the package via pip or from source if the build fails.

```bash
$ pip install hermes-agent
# If the pip build fails on your platform, clone and install from source in a venv:
#   git clone https://github.com/NousResearch/hermes-agent && cd hermes-agent
#   python -m venv .venv && .venv/bin/pip install -e .
```

--------------------------------

### Serve Gemma 4 26B on 32-63 GB Mac

Source: https://rapidmlx.com/docs/hardware-tiers.html

Recommended for the 32-63 GB tier. The flags are not optional: --no-mllm drops the vision path, --kv-cache-dtype bf16 and --cache-memory-mb 512 cap the cache. The installer prints the full command; the app applies the same flags when it starts this pick on this tier.

```bash
rapid-mlx serve gemma-4-26b-4bit --no-mllm --kv-cache-dtype bf16 --cache-memory-mb 512
```

--------------------------------

### Install Claude Code CLI

Source: https://rapidmlx.com/docs/agents/claude-code.html

Install the official Anthropic CLI via npm.

```bash
$ npm install -g @anthropic-ai/claude-code
# or download the binary from anthropic.com/claude-code
```

--------------------------------

### Run a simple invocation

Source: https://rapidmlx.com/docs/frameworks/langchain.html

Start the rapid-mlx server and invoke the model with a simple prompt using ChatOpenAI.

```bash
$ rapid-mlx serve qwen3.5-9b-4bit
$ python -c "from langchain_openai import ChatOpenAI; \
  llm = ChatOpenAI(model='default', base_url='http://localhost:8000/v1', api_key='not-needed'); \
  print(llm.invoke('Say hello').content)"
```

--------------------------------

### Install GitHub Copilot CLI extension

Source: https://rapidmlx.com/docs/agents/copilot.html

Install the gh-copilot extension. Requires an authenticated gh session (gh auth login).

```bash
$ gh extension install github/gh-copilot
# Requires an authenticated ``gh`` session (``gh auth login``).
```

--------------------------------

### Run OpenCode with rapid-mlx

Source: https://rapidmlx.com/docs/agents/opencode.html

Commands to start the rapid-mlx server and launch the OpenCode TUI.

```bash
$ rapid-mlx serve qwen3-coder-30b-4bit
$ opencode
```

--------------------------------

### Install DeepSeek Harness: check Node version and npm install

Source: https://rapidmlx.com/docs/agents/deepseek-harness.html

Verify Node is 22.15 or newer (needed for the Zstd stream API; the manifest does not declare this requirement) and install the `@deepseek-ai/dsh` package globally.

```shell
$ node --version   # must be >= 22.15
$ npm install -g @deepseek-ai/dsh

```

--------------------------------

### Upgrade rapid-mlx installation

Source: https://rapidmlx.com/docs/cli.html

Detects the install method (brew, pip, install.sh) and runs the appropriate upgrade command. --dry-run prints what would be done without executing; -y skips confirmation.

```bash
$ rapid-mlx upgrade
$ rapid-mlx upgrade --dry-run
$ rapid-mlx upgrade -y
```

--------------------------------

### Sample session_start telemetry payload JSON

Source: https://rapidmlx.com/docs/telemetry.html

Example output from the preview command on an M3 Ultra 256 GB machine. The engine field is a schema v1 back-compat slot with no runtime information.

```json
{
  "schema_version": 1,
  "client_id": "b2292c64-de82-495d-b2b8-877d488f6537",
  "session_id": "preview-0000000000000000",
  "rapid_mlx_version": "0.12.17",
  "platform": {
    "os": "darwin",
    "os_version": "25.5",
    "arch": "arm64",
    "chip": "Apple M3 Ultra",
    "memory_gb": 256,
    "python_version": "3.12"
  },
  "event": "session_start",
  "timestamp": "2026-07-07T19:42:11Z",
  "session": {
    "subcommand": "serve",
    "duration_seconds": null,
    "models_loaded": [
      "mlx-community/Qwen3.5-9B-4bit"
    ],
    "engine": "",
    "flag_names": [
      "port",
      "host"
    ]
  }
}
```

--------------------------------

### Run rapid-mlx bench with freeform, submit, and tier flags

Source: https://rapidmlx.com/docs/cli.html

Shows four example invocations of the bench command. The first two run freeform benchmarks; --submit runs the canonical B=1 community bench and opens a PR to community-benchmarks/. The last two use --tier speed or all to run a validation ladder (smoke, speed, harness).

```shell
$ rapid-mlx bench qwen3.5-4b-4bit
$ rapid-mlx bench qwen3.6-35b-8bit --submit                # canonical B=1
$ rapid-mlx bench gpt-oss-20b-mxfp4-q8 --tier speed
$ rapid-mlx bench gpt-oss-20b-mxfp4-q8 --tier all          # smoke → speed → harness
```

--------------------------------

### Serve UI-TARS 1.5 7B 4-bit with rapid-mlx

Source: https://rapidmlx.com/docs/models/ui-tars.html

Starts a rapid-mlx server for the ui-tars-1.5-7b-4bit model on port 8000, bound to localhost. The alias config defaults to temperature=0.0 and top_p=1.0 for deterministic behavior; override only for exploration or data-collection workloads.

```bash
rapid-mlx serve ui-tars-1.5-7b-4bit \
  --port 8000 \
  --host 127.0.0.1
```

--------------------------------

### Run OpenHands with rapid-mlx

Source: https://rapidmlx.com/docs/agents/openhands.html

Start the rapid-mlx server and execute OpenHands with the appropriate model configuration.

```bash
$ rapid-mlx serve qwen3.6-35b-4bit
$ LLM_BASE_URL=http://localhost:8000/v1 \
  LLM_API_KEY=not-needed \
  LLM_MODEL=openai/qwen3.6-35b-4bit \
  openhands
```

--------------------------------

### Install R2 mirror seeding tools

Source: https://rapidmlx.com/docs/extras.html

Installs dependencies for maintainers to seed the Cloudflare R2 mirror. This is not required for standard client-side downloads.

```bash
$ pip install "rapid-mlx[mirror]==0.13.4"
```

--------------------------------

### Install [vision] extra

Source: https://rapidmlx.com/docs/extras.html

Required for models with vision input, adding dependencies like mlx-vlm and torch.

```bash
$ pip install "rapid-mlx[vision]==0.13.4"
```

--------------------------------

### Enable SuffixDecoding with rapid-mlx serve

Source: https://rapidmlx.com/docs/perf-flags.html

Example command to launch a model with SuffixDecoding enabled, specifying the method and the number of speculative tokens.

```bash
$ rapid-mlx serve gemma-4-12b-4bit \
    --speculative-config '{"method":"suffix","num_speculative_tokens":8}' --force-spec-decode
```

--------------------------------

### Serve DiffusionGemma via CLI

Source: https://rapidmlx.com/docs/models/diffusiongemma.html

Command to start the Rapid-MLX server with the 4-bit quantized DiffusionGemma model.

```bash
$ rapid-mlx serve diffusion-gemma-26b-4bit
```

--------------------------------

### Install [mtp] extra

Source: https://rapidmlx.com/docs/extras.html

Enables MTP speculative decoding. Requires pillow to avoid ModuleNotFoundError during assistant-drafter loading.

```bash
$ pip install "rapid-mlx[mtp]==0.13.4"
```

--------------------------------

### Install Codex CLI with brew or npm

Source: https://rapidmlx.com/docs/agents/codex-cli.html

Install the OpenAI Codex CLI via Homebrew or npm. The vendor binary is upstream OpenAI; use either command.

```bash
$ brew install codex
# or
$ npm install -g @openai/codex
```

--------------------------------

### Run Aider with rapid-mlx

Source: https://rapidmlx.com/docs/agents/aider.html

Start the rapid-mlx server and run aider with a specific model.

```bash
$ rapid-mlx serve qwen3.5-9b-4bit
$ aider --openai-api-base http://localhost:8000/v1 \
        --openai-api-key not-needed \
        --model openai/qwen3.5-9b-4bit \
        main.py
```

--------------------------------

### Install [audio-desktop] extra

Source: https://rapidmlx.com/docs/extras.html

A trimmed audio stack for macOS desktop sidecar usage. For the pip engine, the full [audio] extra is recommended instead.

```bash
$ pip install "rapid-mlx[audio-desktop]==0.12.17"
```

--------------------------------

### Configure ~/.hermes/config.yaml

Source: https://rapidmlx.com/docs/agents/hermes.html

Example configuration file for the Hermes Agent, specifying the model provider, base URL, and context length.

```yaml
# ~/.hermes/config.yaml
model:
  provider: "custom"
  default: "default"
  base_url: "http://localhost:8000/v1"
  context_length: 262144
  max_tokens: 4096
platform_toolsets:
  cli: [terminal, file, code_execution, web, browser, skills]
```

--------------------------------

### OpenAI SDK configuration for gpt-oss

Source: https://rapidmlx.com/docs/agents/gpt-oss.html

Configure the OpenAI Python SDK to connect to rapid-mlx's local server. Use base_url pointing to /v1 and api_key='not-needed'. The example creates a chat completion and prints the response content.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",
)

resp = client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "Say hello"}],
)
print(resp.choices[0].message.content)
```

--------------------------------

### Install minimum pytest runtime

Source: https://rapidmlx.com/docs/extras.html

Installs the core dependencies required to import and collect tests, excluding linter toolchains.

```bash
$ pip install "rapid-mlx[test]==0.13.4"
```

--------------------------------

### Run Hermes Agent

Source: https://rapidmlx.com/docs/agents/hermes.html

Commands to start the rapid-mlx server and initiate the Hermes chat interface.

```bash
$ rapid-mlx serve qwen3.5-9b-4bit
$ hermes chat
```

--------------------------------

### Install [embeddings] extra

Source: https://rapidmlx.com/docs/extras.html

Unlocks the /v1/embeddings endpoint and preloading of embedding models.

```bash
$ pip install "rapid-mlx[embeddings]==0.13.4"
```

--------------------------------

### Activate argcomplete for rapid-mlx in zsh/bash

Source: https://rapidmlx.com/docs/troubleshooting.html

Run these commands once to enable tab completion for rapid-mlx when installed via pip or install.sh. Homebrew installs activate argcomplete automatically. Add the first line to ~/.zshrc or ~/.bashrc for persistent activation.

```bash
# zsh / bash — add to ~/.zshrc or ~/.bashrc
$ eval "$(register-python-argcomplete rapid-mlx)"

# Or system-wide:
$ activate-global-python-argcomplete
```

--------------------------------

### rapid-mlx benchmark subcommands overview

Source: https://rapidmlx.com/docs/cli.html

Shows the six benchmark subcommands with example arguments. Each command supports --json for scripts. The run command saves locally and prints a run id; share uploads only after showing the payload and asking for confirmation.

```bash
$ rapid-mlx benchmark catalog                     # models with a protocol + fit for this Mac
$ rapid-mlx benchmark plan qwen3.5-9b-4bit        # exact workload; runs nothing
$ rapid-mlx benchmark run qwen3.5-9b-4bit         # run, save locally, print run id
$ rapid-mlx benchmark results --limit 5           # local runs + share receipts
$ rapid-mlx benchmark inspect RUN_ID              # one saved run, in full
$ rapid-mlx benchmark share --preview RUN_ID      # exact payload, nothing sent
$ rapid-mlx benchmark share RUN_ID                # show payload, ask [y/N], upload
```

--------------------------------

### GET /docs

Source: https://rapidmlx.com/docs/api.html

Provides an interactive Swagger UI for exploring the API.

```APIDOC
## GET /docs

### Description
Swagger UI (interactive).

### Method
GET

### Endpoint
/docs

### Response
#### Success Response (200)
- **html** (string) - The Swagger UI HTML page.

#### Response Example
<!DOCTYPE html>...
```

--------------------------------

### Run Kilo Code

Source: https://rapidmlx.com/docs/agents/kilo-code.html

Commands to start the rapid-mlx server and launch the Kilo Code agent.

```bash
$ rapid-mlx serve qwen3-coder-30b-4bit
$ kilo-code
```

--------------------------------

### Check optional extras with rapid-mlx doctor

Source: https://rapidmlx.com/docs/extras.html

Run `rapid-mlx doctor` to check which optional extras are installed. The output lists missing packages and their install commands, including a version requirement for the dflash extras.

```bash
rapid-mlx doctor
```

```text
◆ Optional Packages
  ⚠ mlx-vlm (vision extras) not installed (`pip install 'rapid-mlx[vision]'`)
  ⚠ mlx-audio (audio extras) not installed (`pip install 'rapid-mlx[audio]'`)
  ⚠ mlx-embeddings (embeddings extras) not installed (`pip install 'rapid-mlx[embeddings]'`)
  ⚠ mlx-vlm 0.5.0+ (dflash extras) not installed or too old
```

--------------------------------

### Use Anthropic Python SDK with rapid-mlx

Source: https://rapidmlx.com/docs/agents/claude-code.html

Example of initializing the Anthropic client and handling reasoning model responses by iterating through content blocks.

```python
# Note the base_url is the ROOT, not /v1 — the SDK appends /v1/messages itself.
# The test file strips /v1 with ``_BASE.rstrip('/').removesuffix('/v1')``.
from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:8000",
    api_key="not-needed",
)

message = client.messages.create(
    model="default",
    max_tokens=256,
    messages=[{"role": "user", "content": "Say hello"}],
)
# Reasoning models emit a ThinkingBlock first — walk content and find the
# first block with type == "text" (see _first_text in the test file).
for block in message.content:
    if getattr(block, "type", None) == "text":
        print(block.text)
        break
```

--------------------------------

### Serve VibeThinker with rapid-mlx

Source: https://rapidmlx.com/docs/models/vibethinker.html

Command to start a rapid-mlx server for the VibeThinker 1.5B 4-bit model on port 8000, bound to localhost. The alias bakes in temperature=1.0 and top_p=0.95 from the model card.

```bash
rapid-mlx serve vibethinker-1.5b-4bit \
  --port 8000 \
  --host 127.0.0.1
```

--------------------------------

### Install [dflash] extra

Source: https://rapidmlx.com/docs/extras.html

Enables DFlash speculative decoding for specific dense 8bit models. Does not include vision-related dependencies.

```bash
$ pip install "rapid-mlx[dflash]==0.13.4"
```

--------------------------------

### Serve and Query Ornith-1.5

Source: https://rapidmlx.com/docs/models/families/ornith.html

Commands to start the model server and perform a chat completion request using curl.

```bash
rapid-mlx serve ornith-1.5-9b-bf16

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"ornith-1.5-9b-bf16","messages":[{"role":"user","content":"Hello"}]}'
```

--------------------------------

### Serve Hunyuan 3 4-bit alias

Source: https://rapidmlx.com/docs/models/families/hunyuan.html

Starts the OpenAI-compatible server for the `hy3-preview-4bit` alias. Pulls weights automatically on first use.

```bash
rapid-mlx serve hy3-preview-4bit
```

--------------------------------

### Serve and generate an image with Qwen Image

Source: https://rapidmlx.com/docs/models/families/qwen-image.html

Starts the rapid-mlx server for qwen-image and sends a curl request to the OpenAI-compatible /v1/images/generations endpoint. The server runs on localhost:8000/v1.

```bash
rapid-mlx serve qwen-image

curl http://localhost:8000/v1/images/generations \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen-image","prompt":"an ink-wash mountain village at dawn","size":"1024x1024"}'

```

--------------------------------

### Structured output with with_structured_output

Source: https://rapidmlx.com/docs/frameworks/langchain.html

Use with_structured_output with a Pydantic model to get structured JSON output. Requires the guided extra.

```python
from pydantic import BaseModel, Field

class Person(BaseModel):
    name: str = Field(description="The person's name")
    age: int = Field(description="The person's age in years")

structured_llm = llm.with_structured_output(Person)
r = structured_llm.invoke([HumanMessage(content="Extract: 'Bob is 42 years old'")])
# r == Person(name="Bob", age=42)
```

--------------------------------

### Run rapid-mlx server

Source: https://rapidmlx.com/docs/agents/cursor.html

Start the rapid-mlx server with a specific model. After running, select 'default' or the served alias in Cursor's model dropdown.

```bash
$ rapid-mlx serve qwen3.6-35b-4bit
```

--------------------------------

### Safe update: backup, bootout, reinstall, re-assert extras, doctor, bootstrap

Source: https://rapidmlx.com/docs/headless-macos-service

Perform a safe upgrade of the headless Rapid-MLX service: check version, freeze current packages, back up the launchd plist, boot out the daemon, run the installer, re-assert optional extras (e.g., rapid-mlx[vision]), run doctor, bootstrap the daemon, and run the smoke test. Always boot out the daemon before maintenance so KeepAlive cannot race the installer. A runtime rebuild does not preserve optional extras; record and re-assert them. Investigate every Doctor issue, but do not use Doctor alone to accept or reject the update because some 0.13.3 runtime layouts can report false import failures. Treat the API smoke as the actionable gate.

```shell
$ sudo -u YOUR_SERVICE_ACCOUNT -H \
  /Users/YOUR_SERVICE_ACCOUNT/.local/bin/rapid-mlx --version
$ sudo -u YOUR_SERVICE_ACCOUNT -H /bin/bash -c \
  '/Users/YOUR_SERVICE_ACCOUNT/.rapid-mlx/bin/python -m pip freeze \
  > "$HOME/rapid-mlx-before-upgrade.txt"'
$ sudo cp -p /Library/LaunchDaemons/com.rapidmlx.server.plist \
  /Library/LaunchDaemons/com.rapidmlx.server.plist.pre-upgrade
$ sudo launchctl bootout system/com.rapidmlx.server
$ curl -fsSL https://rapidmlx.com/install.sh | \
  sudo -u YOUR_SERVICE_ACCOUNT -H /bin/bash

# Re-assert every optional extra used by this appliance.
$ sudo -u YOUR_SERVICE_ACCOUNT -H \
  /Users/YOUR_SERVICE_ACCOUNT/.rapid-mlx/bin/python \
  -m pip install --upgrade 'rapid-mlx[vision]'

$ sudo -u YOUR_SERVICE_ACCOUNT -H \
  /Users/YOUR_SERVICE_ACCOUNT/.local/bin/rapid-mlx doctor || \
  echo 'Doctor reported issues; keep the daemon stopped and continue validation.'
$ sudo launchctl bootstrap system \
  /Library/LaunchDaemons/com.rapidmlx.server.plist
$ ./headless_service_smoke.sh
```

--------------------------------

### Share the model via public URL

Source: https://rapidmlx.com/docs/cli.html

Starts the server and exposes it via a Cloudflare-fronted URL using a built-in reverse tunnel.

```bash
$ rapid-mlx share qwen3.5-4b-4bit
# prints URL + share key + one-click chat link, Ctrl-C stops
```

--------------------------------

### GET /metrics

Source: https://rapidmlx.com/docs/api.html

Returns Prometheus-format metrics for monitoring.

```APIDOC
## GET /metrics

### Description
Prometheus-format counters and histograms.

### Method
GET

### Endpoint
/metrics

### Response
#### Success Response (200)
- **body** (string) - Prometheus metrics in text format.

#### Response Example
# HELP ...
```

--------------------------------

### Serve gpt-oss model

Source: https://rapidmlx.com/docs/agents/gpt-oss.html

Start the rapid-mlx server with the gpt-oss-20b-mxfp4-q8 alias. Since 0.10.14, bare serve auto-selects harmony parsers and enables grammar-constrained tool calling by default.

```bash
$ rapid-mlx serve gpt-oss-20b-mxfp4-q8
```

--------------------------------

### GET /v1/models

Source: https://rapidmlx.com/docs/api.html

Lists the available models with their aliases, parser bindings, and capability flags.

```APIDOC
## GET /v1/models

### Description
List available models — alias, parser bindings, capability flags.

### Method
GET

### Endpoint
/v1/models

### Response
#### Success Response (200)
- **models** (array) - List of available models with alias, parser bindings, and capability flags.

#### Response Example
{
  "models": []
}
```

--------------------------------

### Serve and generate video with Wan

Source: https://rapidmlx.com/docs/models/families/wan.html

Start the rapid-mlx server with a specific Wan model and perform text-to-video generation using curl. The image-to-video mode can be enabled by adding an input reference frame.

```bash
rapid-mlx serve wan2.2-ti2v-5b-q8

# Text-to-video at the checkpoint's native 24 fps
curl http://127.0.0.1:8000/v1/videos \
  -F model=wan2.2-ti2v-5b-q8 \
  -F 'prompt=a fox running through fresh snow, cinematic tracking shot' \
  -F seconds=1 \
  -F size=832x512 \
  -F seed=42

# Image-to-video: add a conditioning first frame
#   -F input_reference=@start.png
```

--------------------------------

### Serve and generate with flux2-klein-4b

Source: https://rapidmlx.com/docs/models/families/flux.html

Start the local OpenAI-compatible server for the FLUX.2 klein 4B model, then send a text-to-image request to the /v1/images/generations endpoint. The curl command specifies the model, prompt, and output size; the server must be running first.

```bash
rapid-mlx serve flux2-klein-4b

curl http://localhost:8000/v1/images/generations \
  -H 'Content-Type: application/json' \
  -d '{"model":"flux2-klein-4b","prompt":"a paper crane on wet slate, studio light","size":"1024x1024"}'

```

--------------------------------

### Serve TTS and synthesize speech with curl

Source: https://rapidmlx.com/docs/api.html

Starts a TTS server with the kokoro-tts model and sends a speech synthesis request to /v1/audio/speech, saving the output to speech.wav. Requires the 'rapid-mlx[audio]' extra.

```bash
# first install: pip install 'rapid-mlx[audio]'
$ rapid-mlx serve kokoro-tts --port 8002 &
$ curl http://localhost:8002/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{"model": "kokoro-tts", "voice": "af_heart", "input": "hello"}' \
    -o speech.wav
```

--------------------------------

### Serve DeepSeek V4 Flash with rapid-mlx

Source: https://rapidmlx.com/docs/models/deepseek-v4-flash.html

Starts a rapid-mlx server for the deepseek-v4-flash-2bit alias on port 8000, bound to localhost. The alias includes the deepseek tool parser and the deepseek_v4 reasoning parser (since 0.11.6) for structured reasoning_content.

```bash
rapid-mlx serve deepseek-v4-flash-2bit \
  --port 8000 \
  --host 127.0.0.1
```

--------------------------------

### Add prefill-step-size to serve command

Source: https://rapidmlx.com/docs/troubleshooting.html

Add --prefill-step-size 8192 (or 16384) to the serve command to mitigate cold prefill on long prompts, making subsequent turns hit the prefix cache and get 10-30× faster.

```bash
--prefill-step-size 8192
```

--------------------------------

### rapid-mlx chat: interactive REPL usage examples

Source: https://rapidmlx.com/docs/cli.html

Shows various invocations of the `rapid-mlx chat` command, including default model, specifying a model with `--think`, connecting to an existing server with `--port`, providing MCP tools with `--mcp-config`, and the Ollama-style `run` alias. The REPL defaults to `qwen3.5-4b-4bit` when no model is given.

```bash
$ rapid-mlx chat
$ rapid-mlx chat qwen3.5-9b-4bit --think
$ rapid-mlx chat --port 8000     # connect to existing server
$ rapid-mlx chat --mcp-config ~/mcp.json  # give the REPL MCP tools
$ rapid-mlx run gemma-4-12b-4bit # Ollama-style alias
```

--------------------------------

### Serve embeddings and test with curl

Source: https://rapidmlx.com/docs/api.html

Starts an embeddings server with the embeddinggemma-300m-6bit model and sends a test request to the /v1/embeddings endpoint. Requires the 'rapid-mlx[embeddings]' extra.

```bash
# first install: pip install 'rapid-mlx[embeddings]'
$ rapid-mlx serve embeddinggemma-300m-6bit --port 8001 &
$ curl http://localhost:8001/v1/embeddings \
    -H "Content-Type: application/json" \
    -d '{"model": "embeddinggemma-300m-6bit", "input": "ping"}'
```

--------------------------------

### GET /v1/videos/capabilities

Source: https://rapidmlx.com/docs/api.html

Returns the capabilities of the loaded video model, including supported modes, native fps, size rule, and frame-count rule.

```APIDOC
## GET /v1/videos/capabilities

### Description
What the loaded video model accepts — modes, native fps, size rule and frame-count rule. Same contract the route validates against.

### Method
GET

### Endpoint
/v1/videos/capabilities

### Response
#### Success Response (200)
- **modes** (array) - Supported modes.
- **native_fps** (number) - Native frames per second.
- **size_rule** (string) - Size rule.
- **frame_count_rule** (string) - Frame count rule.

#### Response Example
{
  "modes": [],
  "native_fps": 24,
  "size_rule": "...",
  "frame_count_rule": "..."
}
```

--------------------------------

### GET /v1/videos/capabilities

Source: https://rapidmlx.com/docs/models/families/video

Reports the live model's capabilities, including family, native fps, supported modes, size rule, and frame-count rule. This is the same contract the route validates against.

```APIDOC
## GET /v1/videos/capabilities

### Description
Reports the live model's capabilities, including family, native fps, supported modes, size rule, and frame-count rule. This is the same contract the route validates against.

### Method
GET

### Endpoint
/v1/videos/capabilities

### Request Example
```
curl http://localhost:8000/v1/videos/capabilities
```

### Response
#### Success Response (200)
- **family** (string) - The model family (e.g., LTX, Wan, CogVideoX-Fun).
- **native_fps** (number) - The model's native frame rate.
- **supported_modes** (array) - Supported modes, e.g., text-to-video, image-to-video.
- **size_rule** (object) - Size constraints: fixed list for CogVideoX-Fun, 64-pixel-aligned range plus maximum pixel area for Wan.
- **frame_count_rule** (object) - Frame count rule as a `minimum`/`step`/`offset` triple.

#### Response Example
```
{
  "family": "ltx",
  "native_fps": 24,
  "supported_modes": ["text-to-video", "image-to-video"],
  "size_rule": {
    "type": "range",
    "min": 512,
    "max": 1920,
    "alignment": 64
  },
  "frame_count_rule": {
    "minimum": 9,
    "step": 8,
    "offset": 1
  }
}
```
```

--------------------------------

### Stream chat completions with OpenAI client

Source: https://rapidmlx.com/docs/models/diffusiongemma.html

Example of using the OpenAI Python client to interact with the DiffusionGemma model via the Rapid-MLX local endpoint.

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-used")

stream = client.chat.completions.create(
    model="diffusion-gemma-26b-4bit",
    messages=[
        {"role": "user", "content": "Write a haiku about Apple Silicon."}
    ],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
```

--------------------------------

### Run rapid-mlx doctor to verify installation

Source: https://rapidmlx.com/docs/hardware-tiers.html

Use the doctor subcommand to check your machine's Apple Silicon variant, macOS version, free disk, and HF cache size, and to flag missing optional extras. The sample output shows a successful check on an M3 Ultra 256 GB system.

```bash
$ rapid-mlx doctor
```

```text
◆ System
  ✓ Apple Silicon (Apple M3 Ultra, 256 GB)
  ✓ macOS 26.5.1 (Darwin 25.5.0)
  ✓ Free disk: 114 GB
◆ Required Packages
  ✓ mlx 0.32.0
  ✓ mlx-lm 0.31.3
  ✓ rapid-mlx 0.12.17
```

--------------------------------

### curl TTS and STT requests to OpenAI-compatible endpoints

Source: https://rapidmlx.com/docs/models/families/audio

Shows how to call the audio endpoints with curl: a text-to-speech POST to /v1/audio/speech with a JSON body and output file, and a speech-to-text multipart upload to /v1/audio/transcriptions. The model names 'kokoro' and 'whisper-large-v3-turbo' are used as examples.

```bash
# Text-to-speech
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model": "kokoro", "input": "hello from rapid-mlx"}' \
  --output hello.wav

# Speech-to-text (multipart upload)
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -H "Content-Type: multipart/form-data" \
  -F file=@hello.wav \
  -F model=whisper-large-v3-turbo
```

--------------------------------

### Configure ~/.codex/config.toml for rapid-mlx

Source: https://rapidmlx.com/docs/agents/codex-cli.html

Configuration block that rapid-mlx emits for Codex CLI. The `[model_providers.rapid-mlx]` block is resolved after reading `model_provider = "rapid-mlx"`; `base_url` must include the trailing `/v1`. No auth field is included because rapid-mlx is unauthed by default and Codex `--strict-config` rejects an inline literal like `api_key = "..."`; if the server starts with `--api-key`, add `env_key = "RAPID_MLX_API_KEY"` and export that variable.

```toml
# ~/.codex/config.toml
model = "default"
model_provider = "rapid-mlx"

[model_providers.rapid-mlx]
name = "Rapid-MLX (local)"
base_url = "http://localhost:8000/v1"
# No auth field — rapid-mlx is unauthed by default and Codex
# --strict-config rejects an inline literal like `api_key = "..."`.
# If you start the server with --api-key, add:
#     env_key = "RAPID_MLX_API_KEY"
# and `export RAPID_MLX_API_KEY=<your-key>` in your shell.
```

--------------------------------

### Configure Wan process-level tuning

Source: https://rapidmlx.com/docs/models/families/wan.html

Set environment variables for steps, scheduler, and VAE tiling before launching the rapid-mlx server.

```bash
RAPID_MLX_WAN_STEPS=8 \
RAPID_MLX_WAN_SCHEDULER=unipc \
RAPID_MLX_WAN_TILING=auto \
rapid-mlx serve wan2.2-ti2v-5b-q8
```

--------------------------------

### Configure Aider via Environment Variables

Source: https://rapidmlx.com/docs/agents/aider.html

Set the API base and key environment variables to point to the local rapid-mlx server.

```bash
$ export OPENAI_API_BASE=http://localhost:8000/v1
$ export OPENAI_API_KEY=not-needed
```

--------------------------------

### Download and Configure launchd Template

Source: https://rapidmlx.com/docs/headless-macos-service

Downloads the service template, updates the service account placeholder, and validates the plist file structure.

```bash
$ curl -fsSLo /tmp/com.rapidmlx.server.plist \
  https://raw.githubusercontent.com/raullenchai/Rapid-MLX/main/examples/launchd/com.rapidmlx.server.plist
$ sed -i '' 's/serveuser/YOUR_SERVICE_ACCOUNT/g' \
  /tmp/com.rapidmlx.server.plist
$ plutil -lint /tmp/com.rapidmlx.server.plist
$ plutil -p /tmp/com.rapidmlx.server.plist
```

--------------------------------

### Configure OpenAIServerModel

Source: https://rapidmlx.com/docs/frameworks/smolagents.html

Initialize the OpenAIServerModel to connect smolagents to the rapid-mlx server.

```python
from smolagents import CodeAgent, OpenAIServerModel, ToolCallingAgent, tool

model = OpenAIServerModel(
    model_id="default",
    api_base="http://localhost:8000/v1",
    api_key="not-needed",
)
```

--------------------------------

### GET /openapi.json

Source: https://rapidmlx.com/docs/api.html

Returns the OpenAPI 3 schema for the entire API.

```APIDOC
## GET /openapi.json

### Description
OpenAPI 3 schema for everything above.

### Method
GET

### Endpoint
/openapi.json

### Response
#### Success Response (200)
- **schema** (object) - The OpenAPI schema.

#### Response Example
{
  "openapi": "3.0.0"
}
```