### Help Command Examples

Source: https://github.com/blaizzy/mlx-vlm/blob/main/docs/cli_reference.md

Examples of how to get help for specific MLX-VLM subcommands.

```bash
python -m mlx_vlm convert --help
```

```bash
python -m mlx_vlm generate --help
```

--------------------------------

### Installation

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

The easiest way to get started is to install the `mlx-vlm` package using pip.

```sh
pip install -U mlx-vlm
```

--------------------------------

### Install PyTorch for Layout+OCR

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/falcon_ocr/README.md

Install PyTorch if you plan to use the layout detector for Layout + OCR functionality.

```bash
pip install torch
```

--------------------------------

### Start mlx-vlm Server

Source: https://github.com/blaizzy/mlx-vlm/blob/main/docs/usage.md

Command to start the mlx-vlm server using FastAPI.

```bash
python -m mlx_vlm.server
```

--------------------------------

### Server Command Line Interface

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of starting the mlx_vlm server with specific model configurations.

```sh
mlx_vlm.server --model mlx-community/gemma-4-31B-it-bf16 \
  --draft-model mlx-community/gemma-4-31B-it-assistant-bf16 \
  --draft-kind mtp --draft-block-size 4
```

--------------------------------

### CLI Examples for Dynamic Resolution

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/deepseekocr_2/README.md

Provides command-line interface examples for running OCR tasks, including passing complex processor configurations via the --processor-kwargs flag.

```bash
# Default: dynamic resolution with 1-6 patches
mlx_vlm.generate \
    --model mlx-community/DeepSeek-OCR-2-bf16 \
    --image document.png \
    --prompt "<|grounding|>OCR this image." \
    --max-tokens 1000

# Global view only (faster, 257 tokens)
mlx_vlm.generate \
    --model mlx-community/DeepSeek-OCR-2-bf16 \
    --image document.png \
    --prompt "<|grounding|>OCR this image." \
    --max-tokens 1000 \
    --processor-kwargs '{"cropping": false}'

# Limit to 3 patches max
mlx_vlm.generate \
    --model mlx-community/DeepSeek-OCR-2-bf16 \
    --image document.png \
    --prompt "<|grounding|>OCR this image." \
    --max-tokens 1000 \
    --processor-kwargs '{"cropping": true, "max_patches": 3}'
```

--------------------------------

### Install MLX-VLM in Editable Mode

Source: https://github.com/blaizzy/mlx-vlm/blob/main/docs/contributing.md

Install the project in editable mode to make local changes reflected immediately. This is typically done before starting development.

```bash
pip install -e .
```

--------------------------------

### Install mlx-vlm

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/falcon_perception/README.md

Install the mlx-vlm library using pip. This is the first step to use Falcon-Perception models.

```bash
pip install mlx-vlm
```

--------------------------------

### Distributed Inference Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/docs/usage.md

Example command for running distributed inference with a large language model (Kimi K2.6) across multiple computers using mlx.launch and JACCL protocol.

```bash
mlx.launch \
    --hostfile ring-thunderbolt.json \
    --backend jaccl \
    --hostfile /path/to/hosts.json \
    --env MLX_METAL_FAST_SYNCH=1 \
    -- \
    mlx-vlm/examples/sharded_generate.py \
    --model moonshotai/Kimi-K2.6 \
    --prompt "Describe this image" \
    --image mx-vlm/examples/images/scene_1.jpg
```

--------------------------------

### CLI Examples for Dynamic Resolution with mlx-vlm

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/deepseekocr/README.md

These bash commands illustrate how to control dynamic resolution for OCR using the mlx-vlm command-line interface. Examples include default settings, using only a global view, and limiting the maximum number of patches. These commands require the mlx_vlm CLI tool to be installed.

```bash
# Default: dynamic resolution with 1-6 patches
mlx_vlm.generate \
    --model mlx-community/DeepSeek-OCR-bf16 \
    --image document.png \
    --prompt "<|grounding|>OCR this image." \
    --max-tokens 1000

# Global view only (faster, 257 tokens)
mlx_vlm.generate \
    --model mlx-community/DeepSeek-OCR-bf16 \
    --image document.png \
    --prompt "<|grounding|>OCR this image." \
    --max-tokens 1000 \
    --processor-kwargs '{"cropping": false}'

# Limit to 3 patches max
mlx_vlm.generate \
    --model mlx-community/DeepSeek-OCR-bf16 \
    --image document.png \
    --prompt "<|grounding|>OCR this image." \
    --max-tokens 1000 \
    --processor-kwargs '{"cropping": true, "max_patches": 3}'
```

--------------------------------

### Install MLX-VLM

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/gemma4/README.md

Install the MLX-VLM library using pip. This is a prerequisite for using the library.

```sh
pip install -U mlx-vlm
```

--------------------------------

### Quick Start

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/sam3d_body/README.md

Initialize the SAM3DPredictor by loading pre-trained weights.

```python
from mlx_vlm.models.sam3d_body.generate import SAM3DPredictor

predictor = SAM3DPredictor.from_pretrained("/path/to/sam3d-mlx-weights")
```

--------------------------------

### Server with MTP Drafter

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/speculative/drafters/qwen3_5_mtp/README.md

Example command to run the server with the MTP drafter.

```bash
uv run mlx_vlm.server \
  --model Qwen/Qwen3.5-4B \
  --draft-model ./Qwen3.5-4B-mtp \
  --draft-kind mtp \
  --draft-block-size 4
```

--------------------------------

### Pre-load a model at startup

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of pre-loading a model at server startup with a specified port.

```sh
mlx_vlm.server --port 8080 --model mlx-community/Qwen2.5-VL-3B-Instruct-4bit
```

--------------------------------

### Python API Example with DFlash Windowing

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Python API example demonstrating DFlash draft-cache windowing.

```python
from mlx_vlm import load
from mlx_vlm.generate import generate
from mlx_vlm.speculative.drafters import load_drafter

model, processor = load("Qwen/Qwen3.5-4B")
draft_model, draft_kind = load_drafter("z-lab/Qwen3.5-4B-DFlash")
draft_model.config.draft_window_size = 256  # None disables windowing

result = generate(
    model,
    processor,
    "Write a quicksort in Python.",
    max_tokens=512,
    temperature=0,
    draft_model=draft_model,
    draft_kind=draft_kind,
)
```

--------------------------------

### Uniform 8-bit KV cache quantization

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example command to start the mlx_vlm server with uniform 8-bit KV cache quantization enabled.

```shell
mlx_vlm.server --model google/gemma-4-26b-a4b-it --kv-bits 8
```

--------------------------------

### Install mlx-vlm Package

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/dots_ocr/README.md

Install the mlx-vlm package using uv pip.

```bash
uv pip install mlx-vlm
```

--------------------------------

### Set top_logprobs_k via command line

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example command to start the mlx_vlm server with a specific cap for top log probabilities.

```shell
mlx_vlm.server --model mlx-community/Qwen2-VL-2B-Instruct-4bit --top-logprobs-k 5
```

--------------------------------

### TurboQuant 3.5-bit KV cache quantization

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example command to start the mlx_vlm server with TurboQuant 3.5-bit KV cache quantization enabled.

```shell
mlx_vlm.server --model google/gemma-4-26b-a4b-it --kv-bits 3.5 --kv-quant-scheme turboquant
```

--------------------------------

### Audio Support Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of sending audio files along with text to the generate endpoint.

```shell
curl -X POST "http://localhost:8080/generate" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/gemma-3n-E2B-it-4bit",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Describe what you hear in these audio files" },
          { "type": "input_audio", "input_audio": "/path/to/audio1.wav" },
          { "type": "input_audio", "input_audio": "https://example.com/audio2.mp3" }
        ]
      }
    ],
    "stream": true,
    "max_tokens": 500
  }'
```

--------------------------------

### Inference Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/rt_detr_v2/README.md

Example of how to load a pre-converted MLX checkpoint and perform inference on an image.

```python
from pathlib import Path
from PIL import Image
from huggingface_hub import snapshot_download
from transformers import AutoProcessor
from mlx_vlm.utils import load_model
from mlx_vlm.models.rt_detr_v2.generate import RTDetrV2Predictor
import mlx_vlm.models.rt_detr_v2  # registers the processor with AutoProcessor

path = Path(snapshot_download("mlx-community/docling-layout-heron-mlx-bf16"))
model = load_model(path)
processor = AutoProcessor.from_pretrained(path)
predictor = RTDetrV2Predictor(model, processor, threshold=0.3)

result = predictor.predict(Image.open("page.png"))
for name, score, box in zip(result.class_names, result.scores, result.boxes):
    print(f"{name:20s} {score:.3f} {box.tolist()}")
```

--------------------------------

### Chat UI Command Line Interface

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of launching the chat interface using Gradio.

```sh
mlx_vlm.chat_ui --model mlx-community/Qwen2-VL-2B-Instruct-4bit
```

--------------------------------

### Install MLX VLM and Matplotlib

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/falcon_perception_demo.ipynb

Installs the necessary libraries for using MLX VLM and plotting results. Run this command in your environment before proceeding.

```python
!pip install -U mlx-vlm matplotlib
```

--------------------------------

### Install MLX-VLM and Matplotlib

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/sam3_demo.ipynb

Installs the necessary libraries for running MLX-VLM and plotting results. This should be run in a compatible environment.

```python
!uv pip install -U mlx-vlm matplotlib
```

--------------------------------

### Generate with MTP Drafter

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/speculative/drafters/qwen3_5_mtp/README.md

Example command to generate text using the MTP drafter.

```bash
uv run mlx_vlm.generate \
  --model Qwen/Qwen3.5-4B \
  --draft-model ./Qwen3.5-4B-mtp \
  --draft-kind mtp \
  --draft-block-size 4 \
  --prompt "Make a program to find pi" \
  --max-tokens 256 --temperature 0
```

--------------------------------

### Video Understanding - Command Line

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Command line example for video analysis.

```shell
mlx_vlm.video_generate --model mlx-community/Qwen2-VL-2B-Instruct-4bit --max-tokens 100 --prompt "Describe this video" --video path/to/video.mp4 --max-pixels 224 224 --fps 1.0
```

--------------------------------

### CLI for TurboQuant KV Cache

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Examples of using the command line interface with TurboQuant KV cache quantization.

```sh
# 3.5-bit KV cache quantization (3-bit keys + 4-bit values)
mlx_vlm generate \
  --model mlx-community/Qwen3.5-4B-4bit \
  --kv-bits 3.5 \
  --kv-quant-scheme turboquant \
  --prompt "Your long prompt here..."

# Server with TurboQuant
mlx_vlm server \
  --model google/gemma-4-26b-a4b-it \
  --kv-bits 3.5 \
  --kv-quant-scheme turboquant
```

--------------------------------

### Real-Time Camera Detection (CLI)

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/sam3/README.md

Perform real-time object detection using a webcam feed. Press 'q' to quit the preview window. Lower resolutions increase FPS.

```bash
python -m mlx_vlm.models.sam3.generate --task realtime --prompt "a person" --resolution 224
```

--------------------------------

### Server with Speculative Decoding (DFlash)

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of starting the mlx_vlm server with DFlash for speculative decoding.

```sh
# Server with speculative decoding
mlx_vlm.server --model Qwen/Qwen3.5-4B \
  --draft-model z-lab/Qwen3.5-4B-DFlash
```

--------------------------------

### Server Command Line Interface with EAGLE3

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of starting the mlx_vlm server with the EAGLE3 draft model.

```sh
mlx_vlm.server --model mlx-community/gemma-4-31B-it-bf16 \
  --draft-model RedHatAI/gemma-4-31B-it-speculator.eagle3
```

--------------------------------

### Programmatic Training with Qwen3-VL (Python)

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/LORA.MD

Provides a Python example of how to initiate training programmatically by calling the `mlx_vlm.lora.main` function. It demonstrates creating an `argparse.Namespace` object with all necessary training arguments.

```python
import argparse
from mlx_vlm.lora import main

# Create arguments namespace
args = argparse.Namespace(
    model_path="mlx-community/Qwen3-VL-2B-Instruct-bf16",
    dataset="your-huggingface-dataset-id",
    split="train",
    dataset_config=None,
    batch_size=2,
    epochs=2,
    learning_rate=2e-5,
    iters=1000,
    steps_per_report=10,
    steps_per_eval=200,
    steps_per_save=100,
    val_batches=25,
    max_seq_length=2048,
    lora_rank=8,
    lora_alpha=16,
    lora_dropout=0.0,
    output_path="./qwen3-lora-adapter.safetensors",
    adapter_path=None,
    full_finetune=False,
    train_vision=False,
    grad_checkpoint=False,
    grad_clip=None,
    train_on_completions=False,
    gradient_accumulation_steps=1,
    assistant_id=77091,
    image_resize_shape=None,
    custom_prompt_format=None
)

# Run training
main(args)
```

--------------------------------

### Call Local Server with OpenAI Python Client

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of calling the local server with the OpenAI Python client to get structured output.

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")

response = client.chat.completions.create(
    model="mlx-community/Qwen3.5-4B-MLX-4bit",
    messages=[
        {"role": "user", "content": "Return a dog object."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "AnimalResult",
            "strict": True,
            "schema": schema,
        },
    },
)

result = AnimalResult.model_validate_json(response.choices[0].message.content)
print(result)
```

--------------------------------

### Define model and output paths

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Sets up constants for the model name, maximum tokens, and directories for output artifacts. Ensures output directories are created.

```python
MODEL = "rednote-hilab/dots.mocr"
MAX_TOKENS = 10_000
OUTPUT_ROOT = Path.home() / ".cache" / "mlx_vlm" / "dots_mocr_examples"
PROJECT_ROOT = Path.cwd()
ASSET_ROOT = PROJECT_ROOT / "examples" / "images"
if not ASSET_ROOT.exists():
    ASSET_ROOT = PROJECT_ROOT / "images"
if not ASSET_ROOT.exists():
    raise FileNotFoundError("Expected bundled assets under examples/images")
OUTPUTS_DIR = OUTPUT_ROOT / "outputs"
OVERLAYS_DIR = OUTPUT_ROOT / "overlays"
RENDERED_DIR = OUTPUT_ROOT / "rendered"

for directory in (OUTPUTS_DIR, OVERLAYS_DIR, RENDERED_DIR):
    directory.mkdir(parents=True, exist_ok=True)

PROMPTS = {
    "layout_all": """Please output the layout information from the PDF image, including each layout element's bbox, its category, and the corresponding text content within the bbox.

1. Bbox format: [x1, y1, x2, y2]

2. Layout Categories: The possible categories are ['Caption', 'Footnote', 'Formula', 'List-item', 'Page-footer', 'Page-header', 'Picture', 'Section-header', 'Table', 'Text', 'Title'].

3. Text Extraction & Formatting Rules:
    - Picture: For the 'Picture' category, the text field should be omitted.
    - Formula: Format its text as LaTeX.
    - Table: Format its text as HTML.
    - All Others (Text, Title, etc.): Format their text as Markdown.

4. Constraints:
    - The output text must be the original text from the image, with no translation.
    - All layout elements must be sorted according to human reading order.

5. Final Output: The entire output must be a single JSON object."",
    "layout_only": "Please output the layout information from this PDF image, including each layout's bbox and its category. The bbox should be in the format [x1, y1, x2, y2]. The layout categories for the PDF document include ['Caption', 'Footnote', 'Formula', 'List-item', 'Page-footer', 'Page-header', 'Picture', 'Section-header', 'Table', 'Text', 'Title']. Do not output the corresponding text. The layout result should be in JSON format.",
    "ocr": "Extract the text content from this image.",
    "web": "Parsing the layout info of this webpage image with format json:\n",
    "scene": "Detect and recognize the text in the image.",
    "svg": "Please generate the SVG code based on the image.viewBox=\"0 0 {width} {height}"",
    "general": "You are a helpful assistant.\n\nPlease describe the content of this image.",
}

ALIASES = {
    "demo_hf_layout": "demo_document_parsing",
    "parser_image_default": "demo_document_parsing",
}
```

--------------------------------

### Set top_logprobs_k via environment variable

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example command to start the mlx_vlm server with a specific cap for top log probabilities using an environment variable.

```shell
TOP_LOGPROBS_K=5 mlx_vlm.server --model mlx-community/Qwen2-VL-2B-Instruct-4bit
```

--------------------------------

### Execute QLoRA Training via CLI

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/LORA.MD

Example command-line interface usage for running QLoRA training using a quantized model checkpoint.

```bash
python lora.py \
    --model-path mlx-community/Qwen3-VL-2B-Instruct-4bit \
    --dataset your-dataset-id \
    --batch-size 4 \
    --epochs 2
```

--------------------------------

### Server Command Line Interface (FastAPI)

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Examples of starting the mlx_vlm server with FastAPI, including options for port, preloading models, adapters, and trust remote code.

```sh
mlx_vlm.server --port 8080

# Preload a model at startup (Hugging Face repo or local path)
mix_vlm.server --model <hf_repo_or_local_path>

# Preload a model with adapter
mix_vlm.server --model <hf_repo_or_local_path> --adapter-path <adapter_path>

# With trust remote code enabled (required for some models)
mix_vlm.server --trust-remote-code

# Enable thinking mode by default for requests that do not override it
mix_vlm.server --model Qwen/Qwen3.5-4B --enable-thinking
```

--------------------------------

### Full Fine-tuning with Vision Training (Bash)

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/LORA.MD

Performs a full fine-tuning of Qwen3-VL, including its vision modules. This example uses the `--full-finetune` and `--train-vision` flags, along with specific settings for batch size, epochs, learning rate, and gradient checkpointing.

```bash
python lora.py \
    --model-path mlx-community/Qwen3-VL-2B-Instruct-bf16 \
    --dataset your-dataset-id \
    --full-finetune \
    --train-vision \
    --batch-size 1 \
    --epochs 1 \
    --learning-rate 5e-6 \
    --grad-checkpoint \
    --output-path ./qwen3-full-finetune.safetensors
```

--------------------------------

### CLI for SAM 3.1 Real-time Webcam Tracking

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/sam3_1/README.md

Command-line interface command to enable real-time object tracking from a webcam using SAM 3.1 with optimized settings.

```bash
python -m mlx_vlm.models.sam3_1.generate --task realtime --prompt "a person" --model mlx-community/sam3.1-bf16 --resolution 224
```

--------------------------------

### Responses Endpoint Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of using the responses endpoint with an image and text.

```shell
curl -X POST "http://localhost:8080/responses" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Qwen2-VL-2B-Instruct-4bit",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "input_text", "text": "What is in this image?"},
          {"type": "input_image", "image_url": "/path/to/image.jpg"}
        ]
      }
    ],
    "max_tokens": 100
  }'
```

--------------------------------

### Run General Image QA Scenario

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Executes the general visual question-answering scenario for 'demo_general'. This uses a visual QA prompt on a specified image.

```python
demo_general = run_and_show("demo_general")
```

--------------------------------

### Run Parser PDF Default Scenario

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Executes the layout prompt on the first page of 'demo_pdf1.pdf' for the 'parser_pdf_default' scenario.

```python
parser_pdf_default = run_and_show("parser_pdf_default")
```

--------------------------------

### Basic LoRA Training with Qwen3-VL (Bash)

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/LORA.MD

Demonstrates basic LoRA training for the Qwen3-VL model using a Hugging Face dataset. This example sets the model path, dataset ID, batch size, epochs, learning rate, and output path for the adapter.

```bash
python lora.py \
    --model-path mlx-community/Qwen3-VL-2B-Instruct-bf16 \
    --dataset your-huggingface-dataset-id \
    --batch-size 2 \
    --epochs 2 \
    --learning-rate 2e-5 \
    --output-path ./qwen3-lora-adapter.safetensors
```

--------------------------------

### Multi-Modal (Image + Audio) Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of sending both an image and audio file to the generate endpoint.

```shell
curl -X POST "http://localhost:8080/generate" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/gemma-3n-E2B-it-4bit",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "input_image", "image_url": "/path/to/image.jpg"},
          {"type": "input_audio", "input_audio": "/path/to/audio.wav"}
        ]
      }
    ],
    "max_tokens": 100
  }'
```

--------------------------------

### Initialize Qwen2-VL Model and Processor

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/text_extraction.ipynb

Loads the Qwen2-VL-7B-Instruct-4bit model and its associated processor for vision-language tasks. This setup is required before performing any image inference.

```python
from mlx_vlm import load, apply_chat_template, generate
from mlx_vlm.utils import load_image

# Load model and processor
qwen_vl_model, qwen_vl_processor = load("mlx-community/Qwen2-VL-7B-Instruct-4bit")
qwen_vl_config = qwen_vl_model.config
```

--------------------------------

### Configure and Execute Model Training in MLX-VLM

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/LORA.MD

Demonstrates how to use argparse.Namespace to configure training parameters for different scenarios such as LoRA, full fine-tuning, and QLoRA. Each example invokes the main function from mlx_vlm.lora to initiate the training process.

```python
import argparse
from mlx_vlm.lora import main

args = argparse.Namespace(
    model_path="mlx-community/Llama-3.2-11B-Vision-Instruct-4bit",
    dataset="your-dataset-id",
    batch_size=1,
    epochs=3,
    learning_rate=1e-5,
    lora_rank=8,
    output_path="./mllama-lora-adapter.safetensors",
    grad_checkpoint=True,
    gradient_accumulation_steps=4
)
main(args)
```

```python
import argparse
from mlx_vlm.lora import main

args = argparse.Namespace(
    model_path="mlx-community/Qwen3-VL-2B-Instruct-bf16",
    full_finetune=True,
    train_vision=True,
    output_path="./qwen3-full-finetune.safetensors"
)
main(args)
```

```python
import argparse
from mlx_vlm.lora import main

args = argparse.Namespace(
    model_path="mlx-community/Qwen3-VL-2B-Instruct-4bit",
    lora_rank=16,
    output_path="./qwen3-qlora-adapter.safetensors"
)
main(args)
```

--------------------------------

### Install MLX-VLM

Source: https://github.com/blaizzy/mlx-vlm/blob/main/agents/grounded_reasoning/README.md

Install the mlx-vlm package using pip. Ensure you have Python 3.10+ and macOS on Apple Silicon.

```bash
uv pip install -U mlx-vlm
```

--------------------------------

### CLI Image Understanding with Gemma 4

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/gemma4/README.md

Understand images using Gemma 4 models via the command-line interface. Provide the image path and a descriptive prompt.

```sh
python -m mlx_vlm.generate \
  --model google/gemma-4-e4b-it \
  --image path/to/image.jpg \
  --prompt "Describe this image." \
  --max-tokens 500 \
  --temperature 1.0 --top-p 0.95 --top-k 64
```

--------------------------------

### Install mlx-vlm Package

Source: https://github.com/blaizzy/mlx-vlm/blob/main/agents/grounded_reasoning/demo.ipynb

Install the mlx-vlm package using pip. Ensure you are using the latest version for optimal performance.

```python
!pip install -U mlx-vlm
```

--------------------------------

### Load and Display Image for Shelf Analysis

Source: https://github.com/blaizzy/mlx-vlm/blob/main/agents/grounded_reasoning/demo.ipynb

Loads an image from a URL and displays it for visual analysis. This is the initial step before running any queries on the image content.

```python
image_3 = load_image(
    "https://globalenterprises.online/wp-content/uploads/2024/06/pexels-photo-20155362.jpeg"
)
display(image_3)
```

--------------------------------

### Image Input Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example cURL request for image input to the chat completions endpoint with streaming enabled.

```sh
curl -X POST "http://localhost:8080/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Qwen2.5-VL-32B-Instruct-8bit",
    "messages":
    [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "This is today's chart for energy demand in California. Can you provide an analysis of the chart and comment on the implications for renewable energy in California?"
          },
          {
            "type": "input_image",
            "image_url": "/path/to/repo/examples/images/renewables_california.png"
          }
        ]
      }
    ],
    "stream": true,
    "max_tokens": 1000
  }'
```

--------------------------------

### Text Input Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example cURL request for text input to the chat completions endpoint with streaming enabled.

```sh
curl -X POST "http://localhost:8080/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Qwen2-VL-2B-Instruct-4bit",
    "messages": [
      {
        "role": "user",
        "content": "Hello, how are you"
      }
    ],
    "stream": true,
    "max_tokens": 100
  }'
```

--------------------------------

### Install mlx-vlm dependencies

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/glm_ocr/README.md

Commands to install the mlx-vlm package using standard pip or the fast uv package manager.

```sh
pip install mlx-vlm
```

```sh
uv pip install mlx-vlm
```

--------------------------------

### Run Web Parsing Scenario

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Executes the web page layout parsing scenario for 'demo_web_parsing'. This processes a given webpage image to extract layout information.

```python
demo_web_parsing = run_and_show("demo_web_parsing")
```

--------------------------------

### Install MLX-VLM Library

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/object_detection.ipynb

Install the required MLX-VLM package via pip to enable vision-language model capabilities.

```bash
pip install -U mlx-vlm
```

--------------------------------

### Run Scene Spotting Scenario

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Executes the scene spotting scenario for 'demo_scene_spotting'. This detects and labels text instances within a scene image.

```python
demo_scene_spotting = run_and_show("demo_scene_spotting")
```

--------------------------------

### Thinking Budget Example

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of how to use the --thinking-budget flag to limit tokens in the thinking block for models like Qwen3.5.

```sh
mlx_vlm.generate --model mlx-community/Qwen3.5-2B-4bit \
  --thinking-budget 50 \
  --thinking-start-token "<think>" \
  --thinking-end-token "</think>" \
  --enable-thinking \
  --prompt "Solve 2+2"
```

--------------------------------

### Render SVG Preview using qlmanage

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Renders a PNG preview of an SVG file using the macOS 'qlmanage' tool. Requires 'qlmanage' to be installed. Raises an error if 'qlmanage' is not found or if the preview fails to generate.

```python
def render_preview(input_path: Path, output_dir: Path, size: int):
    if shutil.which("qlmanage") is None:
        raise RuntimeError("qlmanage is required on macOS to render PDF and SVG previews")
    output_dir.mkdir(parents=True, exist_ok=True)
    subprocess.run(
        ["qlmanage", "-t", "-s", str(size), "-o", str(output_dir), str(input_path)],
        check=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
    )
    preview_path = output_dir / f"{input_path.name}.png"
    if not preview_path.exists():
        raise FileNotFoundError(f"Quick Look did not render {input_path}")
    return preview_path
```

--------------------------------

### Install MLX-VLM dependencies

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/video_understanding.ipynb

Installs the required mlx-vlm package via pip to enable model loading and inference capabilities.

```bash
!pip install -U mlx-vlm
```

--------------------------------

### Run Document Parsing Scenario

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Executes the document parsing scenario for 'demo_document_parsing'. This involves full layout extraction and saving structured output with an overlay.

```python
demo_document_parsing = run_and_show("demo_document_parsing")
```

--------------------------------

### Generate Summary Files and Contact Sheet

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

After running all example sections, this code generates summary files (`results.json`, `summary.md`) and a contact sheet (`overlay_contact_sheet.png`) in the specified output directory. It includes checks to ensure all scenario sections have been run.

```python
missing = [key for key in SCENARIO_ORDER if key not in result_map]
if missing:
    raise RuntimeError(f"Run all example sections before the summary cell: {missing}")

ordered_results = [result_map[key] for key in SCENARIO_ORDER]

summary_lines = [
    "# dots.mocr README examples via MLX-VLM",
    "",
    f"Output root: {OUTPUT_ROOT}",
    "",
    "| example | kind | tokens | capped | parse_ok | overlay | output |",
    "| --- | --- | ---: | --- | --- | --- | --- |",
]
for item in ordered_results:
    summary_lines.append(
        f"| {item['id']} | {item['kind']} | {item['generation_tokens']} | {item['capped']} | {item.get('parse_ok')} | {Path(item['overlay']).name} | {Path(item['output']).name} |"
    )
summary_md = "\n".join(summary_lines) + "\n"

(OUTPUT_ROOT / "results.json").write_text(
    json.dumps(ordered_results, ensure_ascii=False, indent=2),
    encoding="utf-8",
)
(OUTPUT_ROOT / "summary.md").write_text(summary_md, encoding="utf-8")
make_contact_sheet(ordered_results, OUTPUT_ROOT / "overlay_contact_sheet.png")

display(Markdown(summary_md))
display(Image.open(OUTPUT_ROOT / "overlay_contact_sheet.png"))
print(OUTPUT_ROOT)
```

--------------------------------

### Initialize Model and Load Image

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/object_pointing.ipynb

Sets up the Molmo-7B-D-0924 model and loads a target image for analysis using MLX-VLM utilities.

```python
from mlx_vlm import load, apply_chat_template, generate
from mlx_vlm.utils import load_image
from utils import parse_points, plot_locations

# Load model and processor
model, processor = load("mlx-community/Molmo-7B-D-0924-4bit")
config = model.config

# Load image
image = load_image("images/desktop_setup.png")
```

--------------------------------

### List Available Models

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example cURL command to list available models.

```sh
curl "http://localhost:8080/models"
```

--------------------------------

### Real-Time Detection with Bounding Boxes (CLI)

Source: https://github.com/blaizzy/mlx-vlm/blob/main/mlx_vlm/models/sam3/README.md

Overlay bounding boxes and labels on detected objects in the real-time camera feed. This helps visualize the detection results.

```bash
python -m mlx_vlm.models.sam3.generate --task realtime --prompt "a cup" --resolution 224 --show-boxes
```

--------------------------------

### Define Schema with Pydantic

Source: https://github.com/blaizzy/mlx-vlm/blob/main/README.md

Example of defining a Pydantic model for structured outputs.

```python
from typing import Literal

from pydantic import BaseModel, ConfigDict, Field


class AnimalResult(BaseModel):
    model_config = ConfigDict(extra="forbid")

    animal: Literal["dog", "cat", "bird", "unknown"]
    species: str = Field(max_length=60)
    description: str = Field(max_length=200)


schema = AnimalResult.model_json_schema()
```

--------------------------------

### Run Image to SVG Scenario

Source: https://github.com/blaizzy/mlx-vlm/blob/main/examples/dots_mocr_demo.ipynb

Executes the image to SVG conversion scenario for 'demo_svg'. This generates an SVG from an image, creates a comparison render, and displays the final overlay.

```python
demo_svg = run_and_show("demo_svg")
```