Rapid-MLX

https://rapidmlx.com/docs
Admin
Rapid-MLX is a high-performance, OpenAI-compatible inference server for Apple Silicon, enabling fast...
Tokens:102,658
Snippets:867
Trust Score:5
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install and serve LTX-2.5

Source: https://rapidmlx.com/docs/models/families/ltx

Install the video extras, install ffmpeg, and start the local server for the LTX-2.5 model. The server runs on http://localhost:8000/v1.

```bash
pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve ltx-2.5-mlx-q8
```

--------------------------------

### Install and submit a text-to-video job with curl

Source: https://rapidmlx.com/docs/models/families/video

Install the video extras, ffmpeg, and start the server. Then submit a text-to-video job with curl, poll for the result, and download the MP4. The job ID is returned in the first response.

```bash
pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve ltx-2.5-mlx-q8

# Submit a text-to-video job — the MP4 comes back with synchronized audio
curl http://127.0.0.1:8000/v1/videos \
  -F model=ltx-2.5-mlx-q8 \
  -F 'prompt=a fox running through fresh snow, cinematic tracking shot' \
  -F seconds=4 \
  -F size=1280x720 \
  -F seed=42

# Poll it, then download the MP4
curl http://127.0.0.1:8000/v1/videos/video_ID
curl http://127.0.0.1:8000/v1/videos/video_ID/content --output result.mp4
```

--------------------------------

### Install rapid-mlx with guided extra

Source: https://rapidmlx.com/docs/frameworks/langchain.html

Install the guided extra required for structured output. Without it, the server returns guided_extra_required.

```bash
$ pip install "rapid-mlx[guided]"
```

--------------------------------

### Install rapid-mlx[all] extra

Source: https://rapidmlx.com/docs/extras.html

Installs the all extra, which is a union of vision, chat, embeddings, and audio extras. It is expanded inline to avoid the self-dependency pitfall that breaks editable installs. Note that [all] does not include [mtp], [dflash], [guided], [image], [video], [audio-desktop], [mirror], [test], or [dev].

```bash
$ pip install "rapid-mlx[all]==0.13.4"
```

--------------------------------

### Install and serve Z-Image-Turbo with rapid-mlx

Source: https://rapidmlx.com/docs/models/families/z-image.html

Install the image extra and start the local server. The server exposes an OpenAI-compatible endpoint at http://localhost:8000/v1.

```bash
pip install 'rapid-mlx[image]'
rapid-mlx serve z-image-turbo
```

--------------------------------

### Serve model and configure DSH with dry-run

Source: https://rapidmlx.com/docs/agents/deepseek-harness.html

Start a rapid-mlx server with a model, then preview and apply the DSH setup. The setup discovers the running model and its context window, patches `$DSH_HOME/settings.yaml` with a timestamped backup, preserves unrelated providers, writes atomically, and refuses to apply if the file changed after the preview.

```shell
$ rapid-mlx serve qwen3.6-35b-8bit

# in another terminal
$ rapid-mlx agents dsh --setup --dry-run   # preview
$ rapid-mlx agents dsh --setup             # apply

```

--------------------------------

### Install rapid-mlx[guided] extra

Source: https://rapidmlx.com/docs/extras.html

Installs the guided extra for llguidance-based structured JSON output. As of 0.10.15, llguidance is a core dependency, so this extra is retained for backward compatibility.

```bash
$ pip install "rapid-mlx[guided]==0.13.4"
```

--------------------------------

### List and configure agent clients with rapid-mlx agents

Source: https://rapidmlx.com/docs/cli.html

Use the agents subcommand to list available agent clients, view setup guides, auto-configure with --setup, run integration tests with --test, and specify agent versions. Flags include --model (auto-detected), --base-url (default http://localhost:8000/v1), and --agent-version.

```bash
$ rapid-mlx agents                          # list
$ rapid-mlx agents codex                    # setup guide
$ rapid-mlx agents aider --setup            # auto-configure
$ rapid-mlx agents hermes --test            # run integration flow
$ rapid-mlx agents aider --agent-version 0.90.0 --setup
```

--------------------------------

### Install rapid-mlx with video support

Source: https://rapidmlx.com/docs/models/families/cogvideox.html

Install the necessary package to enable video generation capabilities.

```bash
pip install 'rapid-mlx[video]'
```

--------------------------------

### Install OpenCode

Source: https://rapidmlx.com/docs/agents/opencode.html

Install the OpenCode TUI using Homebrew or npm.

```bash
$ brew install sst/tap/opencode
# or
$ npm install -g opencode-ai
```

--------------------------------

### Install and Bootstrap launchd Service

Source: https://rapidmlx.com/docs/headless-macos-service

Installs the configured plist to the system launch daemon directory and bootstraps the service.

```bash
$ sudo install -o root -g wheel -m 644 \
  /tmp/com.rapidmlx.server.plist \
  /Library/LaunchDaemons/com.rapidmlx.server.plist
$ sudo launchctl bootstrap system \
  /Library/LaunchDaemons/com.rapidmlx.server.plist
$ sudo launchctl print system/com.rapidmlx.server
```

--------------------------------

### Install Qwen Code CLI

Source: https://rapidmlx.com/docs/agents/qwen-code.html

Install the Qwen Code CLI globally via npm.

```bash
$ npm install -g @qwenlm/qwen-code
```

--------------------------------

### Start rapid-mlx server

Source: https://rapidmlx.com/docs/agents/continue-dev.html

Start the server with a specific model to be used by the Continue extension.

```bash
$ rapid-mlx serve qwen3-coder-30b-4bit
```

--------------------------------

### Install Kilo Code

Source: https://rapidmlx.com/docs/agents/kilo-code.html

Installation commands for Kilo Code using Homebrew or npm.

```bash
$ brew install kilo-org/kilo/kilo-code
# or
$ npm install -g kilo-code
```

--------------------------------

### Serve VibeThinker with rapid-mlx

Source: https://rapidmlx.com/docs/models/vibethinker.html

Command to start a rapid-mlx server for the VibeThinker 1.5B 4-bit model on port 8000, bound to localhost. The alias bakes in temperature=1.0 and top_p=0.95 from the model card.

```bash
rapid-mlx serve vibethinker-1.5b-4bit \
  --port 8000 \
  --host 127.0.0.1
```

--------------------------------

### Prepare the service account and verify server

Source: https://rapidmlx.com/docs/headless-macos-service

Install Rapid-MLX, create a log directory, and verify the server runs in the foreground before setting up the daemon.

```bash
$ curl -fsSL https://rapidmlx.com/install.sh | bash
$ mkdir -p "$HOME/Library/Logs/Rapid-MLX"
$ chmod 750 "$HOME/Library/Logs/Rapid-MLX"
$ rapid-mlx serve qwen3.5-4b-4bit --host 127.0.0.1 --port 8000
```

--------------------------------

### Run rapid-mlx serve and qwen-code

Source: https://rapidmlx.com/docs/agents/qwen-code.html

Start the rapid-mlx server with a Qwen model and then launch the qwen-code CLI.

```bash
$ rapid-mlx serve qwen3-coder-30b-4bit
```

```bash
$ qwen-code
```

--------------------------------

### Install rapid-mlx[video] extra

Source: https://rapidmlx.com/docs/extras.html

Installs the video extra for local video generation with LTX-2.5/2.3, Wan, and CogVideoX-Fun models. Requires Apple Silicon and Python >= 3.11. The video startup preflight gives 3.10 users an explicit upgrade diagnostic.

```bash
$ pip install "rapid-mlx[video]==0.13.4"
```

--------------------------------

### Launch IDE clients

Source: https://rapidmlx.com/docs/getting-started.html

Use the launch command to configure and start the server for specific IDE clients.

```bash
$ rapid-mlx launch cursor      # Cursor
$ rapid-mlx launch claude-code # Claude Code
$ rapid-mlx launch aider       # Aider
```

--------------------------------

### Structured output with with_structured_output

Source: https://rapidmlx.com/docs/frameworks/langchain.html

Use with_structured_output with a Pydantic model to get structured JSON output. Requires the guided extra.

```python
from pydantic import BaseModel, Field

class Person(BaseModel):
    name: str = Field(description="The person's name")
    age: int = Field(description="The person's age in years")

structured_llm = llm.with_structured_output(Person)
r = structured_llm.invoke([HumanMessage(content="Extract: 'Bob is 42 years old'")])
# r == Person(name="Bob", age=42)
```

--------------------------------

### Start a model server

Source: https://rapidmlx.com/docs/getting-started.html

Launch the server using a model alias. The server will download weights on the first run.

```bash
$ rapid-mlx serve qwen3.5-4b-4bit
# downloads the MLX weights on first run, then:
INFO  uvicorn running on http://127.0.0.1:8000
```

--------------------------------

### Serve Tmax models with rapid-mlx

Source: https://rapidmlx.com/docs/models/tmax.html

Starts the Rapid-MLX server with the tmax-9b model, or tmax-27b for the larger variant. Run this before using the chat completion or tool-call examples.

```bash
$ rapid-mlx serve tmax-9b
# or, for the larger model:
$ rapid-mlx serve tmax-27b
```

--------------------------------

### Serve Gemma 4 26B on 32-63 GB Mac

Source: https://rapidmlx.com/docs/hardware-tiers.html

Recommended for the 32-63 GB tier. The flags are not optional: --no-mllm drops the vision path, --kv-cache-dtype bf16 and --cache-memory-mb 512 cap the cache. The installer prints the full command; the app applies the same flags when it starts this pick on this tier.

```bash
rapid-mlx serve gemma-4-26b-4bit --no-mllm --kv-cache-dtype bf16 --cache-memory-mb 512
```

--------------------------------

### Install PydanticAI

Source: https://rapidmlx.com/docs/frameworks/pydantic-ai.html

Install the PydanticAI package using pip.

```bash
$ pip install pydantic-ai
```

--------------------------------

### Run rapid-mlx server and Copilot suggest

Source: https://rapidmlx.com/docs/agents/copilot.html

Start the rapid-mlx server with a model and then use gh copilot suggest to query it.

```bash
$ rapid-mlx serve qwen3.6-35b-4bit
$ gh copilot suggest "find the largest file in this directory"
```

--------------------------------

### Install smolagents

Source: https://rapidmlx.com/docs/frameworks/smolagents.html

Install the smolagents package via pip.

```bash
$ pip install smolagents
```

--------------------------------

### Install Aider

Source: https://rapidmlx.com/docs/agents/aider.html

Install the aider-chat package via pip.

```bash
$ pip install aider-chat
```

--------------------------------

### Install [chat] extra

Source: https://rapidmlx.com/docs/extras.html

Adds the Gradio-based rapid-mlx-chat front-end.

```bash
$ pip install "rapid-mlx[chat]==0.13.4"
```

--------------------------------

### Install langchain-openai

Source: https://rapidmlx.com/docs/frameworks/langchain.html

Install the langchain-openai package to use ChatOpenAI with rapid-mlx.

```bash
$ pip install langchain-openai
```
Tokens:
Raw