### Install mlx-optiq

Source: https://mlx-optiq.pages.dev/llms.txt

Install the mlx-optiq package using pip. Optional extras can be included for specific functionalities like conversion, evaluation, or serving.

```bash
pip install mlx-optiq
```

--------------------------------

### Serve Model with Vision Support

Source: https://mlx-optiq.pages.dev/llms.txt

Starts a server that automatically enables image support if a vision sidecar is present. Use this to serve models capable of processing image and text prompts.

```bash
optiq serve --model mlx-community/gemma-4-e2b-it-OptiQ-4bit
# POST messages with content: [{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}},
#                              {"type":"text","text":"What is in this image?"}]
```

--------------------------------

### Prepare and Serve KV-Cache with OptiQ

Source: https://mlx-optiq.pages.dev/llms.txt

Use `optiq kv-cache` to prepare a model for mixed-precision KV-cache serving. Then, use `optiq serve` to start the API endpoint with the prepared KV cache configuration.

```bash
optiq kv-cache mlx-community/Qwen3.5-9B-OptiQ-4bit \
    --target-bits 5.0 --candidate-bits 4,8 \
    -o ./kv/qwen35_9b

optiq serve --model mlx-community/Qwen3.5-9B-OptiQ-4bit \
    --kv-config ./kv/qwen35_9b/kv_config.json \
    --port 8080
```

--------------------------------

### Get LoRA Adapter Information

Source: https://mlx-optiq.pages.dev/llms.txt

Retrieves information about a trained LoRA adapter. This command is used after fine-tuning to inspect the adapter's configuration.

```bash
optiq lora info ./my_adapter
```

--------------------------------

### Run Full Benchmark Suite Evaluation

Source: https://mlx-optiq.pages.dev/llms.txt

Perform a comprehensive benchmark evaluation covering MMLU, GSM8K, IFEval, BFCL, HumanEval, and HashHop. Results can be scored and output to a JSON file.

```bash
optiq eval ./optiq_mixed --task all --score --output-json ./bench.json
```

--------------------------------

### Run Single Task Evaluation (BFCL)

Source: https://mlx-optiq.pages.dev/llms.txt

Evaluate the model on the BFCL task, consisting of 200 simple function-call questions.

```bash
optiq eval ./optiq_mixed --task bfcl
```

--------------------------------

### Load and Generate with Pre-built Quantized Model

Source: https://mlx-optiq.pages.dev/llms.txt

Loads a pre-built quantized model and generates a response to a given prompt. Ensure the model name is valid and available in the mlx-community repository.

```python
from mlx_lm import load, generate

model, tok = load("mlx-community/Qwen3.5-9B-OptiQ-4bit")

prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Explain mixed-precision quantization."}],
    tokenize=False,
    add_generation_prompt=True,
)
out = generate(model, tok, prompt=prompt, max_tokens=300)
print(out)
```

--------------------------------

### OpenAI Client for OptiQ API

Source: https://mlx-optiq.pages.dev/llms.txt

Connect to the OptiQ server using the OpenAI Python client to interact with the chat completions endpoint. Ensure the `base_url` points to your local OptiQ server.

```python
# OpenAI client
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-used")
resp = client.chat.completions.create(
    model="mlx-community/Qwen3.5-9B-OptiQ-4bit",
    messages=[{"role": "user", "content": "hi"}],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="")
```

--------------------------------

### Hot-swap Mounted LoRA Adapters in Python

Source: https://mlx-optiq.pages.dev/llms.txt

Use this Python primitive for in-process, multi-adapter hot-swapping. It requires loading the base model and preparing it for mounted LoRA, then mounting individual adapters. Activation is managed via `AdapterActivation` context manager.

```python
from mlx_lm import load, generate
from optiq.adapters.mount import (
    prepare_model_for_mounted_lora,
    mount_adapter_on_model,
    AdapterActivation,
)

model, tok = load("mlx-community/Qwen3.5-9B-OptiQ-4bit")
prepare_model_for_mounted_lora(model)
mount_adapter_on_model(model, "agent-A", "./adapter_a")
mount_adapter_on_model(model, "agent-B", "./adapter_b")

with AdapterActivation("agent-A"):
    out_a = generate(model, tok, prompt=p, max_tokens=100)

with AdapterActivation("agent-B"):
    out_b = generate(model, tok, prompt=p, max_tokens=100)
```

--------------------------------

### Run Single Task Evaluation (HashHop)

Source: https://mlx-optiq.pages.dev/llms.txt

Evaluate the model on the HashHop task, which involves 25x4 multi-hop key/value retrieval within a context of approximately 12k tokens.

```bash
optiq eval ./optiq_mixed --task hashhop
```

--------------------------------

### Run Single Task Evaluation (IFEval)

Source: https://mlx-optiq.pages.dev/llms.txt

Evaluate the model on the full 540-prompt IFEval task.

```bash
optiq eval ./optiq_mixed --task ifeval
```

--------------------------------

### Configure Claude Code with Local OptiQ via Environment Variables

Source: https://mlx-optiq.pages.dev/llms.txt

Set environment variables to direct Claude Code to use your locally running OptiQ quant. This allows Claude Code to be driven by your local model.

```bash
export ANTHROPIC_BASE_URL="http://localhost:8080"
export ANTHROPIC_API_KEY="not-used"
claude    # now driven by your local quant
```

--------------------------------

### Quantize Custom Model with Auto Reference Mode

Source: https://mlx-optiq.pages.dev/llms.txt

Converts a model to a specified bits-per-weight (bpw) using the 'auto' reference mode, which selects between bf16 and uniform_4bit based on available RAM. Outputs to a specified directory.

```bash
# auto-routes between bf16 and uniform_4bit reference based on RAM
optiq convert Qwen/Qwen3.5-9B \
    --target-bpw 4.5 \
    --candidate-bits 4,8 \
    --reference auto \
    -o ./optiq_output/Qwen3.5-9B
```

--------------------------------

### Streaming Generation with MLX-LM

Source: https://mlx-optiq.pages.dev/llms.txt

Performs streaming generation, printing tokens as they are generated. Useful for interactive applications where immediate feedback is desired. Configurable with samplers for controlling output randomness.

```python
from mlx_lm import load, stream_generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("mlx-community/Qwen3.5-9B-OptiQ-4bit")
sampler = make_sampler(temp=0.6, top_p=0.95)

for response in stream_generate(model, tok, prompt="...", max_tokens=200, sampler=sampler):
    print(response.text, end="", flush=True)
```

--------------------------------

### Run Single Task Evaluation (HumanEval)

Source: https://mlx-optiq.pages.dev/llms.txt

Evaluate the model on the HumanEval task, which involves 164 problems requiring sandboxed code execution. This ensures untrusted model-generated code cannot escape.

```bash
optiq eval ./optiq_mixed --task humaneval
```

--------------------------------

### Anthropic Client for OptiQ API

Source: https://mlx-optiq.pages.dev/llms.txt

Connect to the OptiQ server using the Anthropic Python client to interact with the messages endpoint. The `base_url` should point to your local OptiQ server.

```python
# Anthropic client (same server)
from anthropic import Anthropic
client = Anthropic(base_url="http://localhost:8080", api_key="not-used")
resp = client.messages.create(
    model="mlx-community/Qwen3.5-9B-OptiQ-4bit",
    max_tokens=300,
    messages=[{"role": "user", "content": "hi"}],
)
print(resp.content[0].text)
```

--------------------------------

### LoRA Fine-tuning with Custom Data

Source: https://mlx-optiq.pages.dev/llms.txt

Performs sensitivity-aware LoRA fine-tuning on a specified model using custom training data. The `--rank-scaling by_bits` option adjusts adapter rank based on quantization bits.

```bash
optiq lora train mlx-community/Qwen3.5-9B-OptiQ-4bit \
    --data ./my_training_data \
    --max-seq-length 1400 \
    --rank 8 --rank-scaling by_bits \
    --num-layers 16 --iters 1000 \
    -o ./my_adapter
```

--------------------------------

### OpenAI Chat Completions API

Source: https://mlx-optiq.pages.dev/llms.txt

This endpoint provides OpenAI-compatible chat completions, supporting streaming Server-Sent Events (SSE). It allows users to interact with served models using the standard OpenAI client library.

```APIDOC
## POST /v1/chat/completions

### Description
Provides OpenAI-compatible chat completions with streaming support.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Query Parameters
- **model** (string) - Required - The model to use for completions.

#### Request Body
- **messages** (array) - Required - The conversation messages.
  - **role** (string) - Required - The role of the message (e.g., "user", "assistant").
  - **content** (string) - Required - The content of the message.
- **stream** (boolean) - Optional - Whether to stream the response.

### Request Example
```json
{
  "model": "mlx-community/Qwen3.5-9B-OptiQ-4bit",
  "messages": [
    {"role": "user", "content": "hi"}
  ],
  "stream": true
}
```

### Response
#### Success Response (200)
- **choices** (array) - Description of the response choices.
  - **delta** (object) - The delta content of the message.
    - **content** (string) - The streamed content of the message.

#### Response Example
```json
{
  "choices": [
    {
      "delta": {
        "content": "Hello!"
      }
    }
  ]
}
```
```

--------------------------------

### Run Fast Triage Evaluation

Source: https://mlx-optiq.pages.dev/llms.txt

Execute a quick evaluation using the 'smoketest' task, which includes KL divergence and GSM8K-50. This is suitable for fast triage and takes approximately 5 minutes on a 27B model.

```bash
optiq eval ./optiq_mixed --task smoketest
```

--------------------------------

### Anthropic Messages API

Source: https://mlx-optiq.pages.dev/llms.txt

This endpoint provides Anthropic-compatible messages, supporting streaming Server-Sent Events (SSE). It allows users to interact with served models using the official Anthropic Python SDK or Claude Code.

```APIDOC
## POST /v1/messages

### Description
Provides Anthropic-compatible messages with streaming support.

### Method
POST

### Endpoint
/v1/messages

### Parameters
#### Query Parameters
- **model** (string) - Required - The model to use for messages.
- **max_tokens** (integer) - Required - The maximum number of tokens to generate.

#### Request Body
- **messages** (array) - Required - The conversation messages.
  - **role** (string) - Required - The role of the message (e.g., "user", "assistant").
  - **content** (string) - Required - The content of the message.

### Request Example
```json
{
  "model": "mlx-community/Qwen3.5-9B-OptiQ-4bit",
  "max_tokens": 300,
  "messages": [
    {"role": "user", "content": "hi"}
  ]
}
```

### Response
#### Success Response (200)
- **content** (array) - The content of the response.
  - **text** (string) - The text content of the response.

#### Response Example
```json
{
  "content": [
    {
      "type": "text",
      "text": "Hello there!"
    }
  ]
}
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.