MiMo-V2-Flash

https://github.com/xiaomimimo/mimo-v2-flash
Admin
MiMo-V2-Flash is a Mixture-of-Experts (MoE) language model with 309B total parameters and 15B active...
Tokens:25,082
Snippets:332
Trust Score:7.6
License:Apache-2.0
Update:2 months ago
Show doc for...
Tokens:
Raw
### Enable and Start Systemd Service

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Commands to enable and start the configured systemd service.

```bash
sudo systemctl enable mimo-server
sudo systemctl start mimo-server
```

--------------------------------

### Launch MiMo-V2-Flash Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Start the server using the specified number of GPUs.

```bash
SGLANG_ENABLE_SPEC_V2=1 python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --tp-size 8 \
    --port 9001 \
    --trust-remote-code
```

```bash
SGLANG_ENABLE_SPEC_V2=1 python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --tp-size 4 \
    --port 9001 \
    --trust-remote-code
```

--------------------------------

### Install SGLang and Launch Inference Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/README.md

Installs the specific SGLang version required for MiMo-V2-Flash and launches the server with recommended performance flags.

```bash
pip install sglang==0.5.6.post2.dev8005+pr.15207.g39d5bd57a \
  --index-url https://sgl-project.github.io/whl/pr/ \
  --extra-index-url https://pypi.org/simple

#Launch the server
SGLANG_ENABLE_SPEC_V2=1 python3 -m sglang.launch_server \
        --model-path XiaomiMiMo/MiMo-V2-Flash \
        --served-model-name mimo-v2-flash \
        --pp-size 1 \
        --dp-size 2 \
        --enable-dp-attention \
        --tp-size 8 \
        --moe-a2a-backend deepep \
        --page-size 1 \
        --host 0.0.0.0 \
        --port 9001 \
        --trust-remote-code \
        --mem-fraction-static 0.75 \
        --max-running-requests 128 \
        --chunked-prefill-size 16384 \
        --reasoning-parser qwen3 \
        --tool-call-parser mimo \
        --context-length 262144 \
        --attention-backend fa3 \
        --speculative-algorithm EAGLE \
        --speculative-num-steps 3 \
        --speculative-eagle-topk 1 \
        --speculative-num-draft-tokens 4 \
        --enable-mtp
```

--------------------------------

### Launch Inference Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Commands to start the inference server in basic or production modes.

```bash
SGLANG_ENABLE_SPEC_V2=1 python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --port 9001
```

```bash
SGLANG_ENABLE_SPEC_V2=1 python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --pp-size 1 \
    --dp-size 2 \
    --enable-dp-attention \
    --tp-size 8 \
    --moe-a2a-backend deepep \
    --page-size 1 \
    --host 0.0.0.0 \
    --port 9001 \
    --trust-remote-code \
    --mem-fraction-static 0.75 \
    --max-running-requests 128 \
    --chunked-prefill-size 16384 \
    --reasoning-parser qwen3 \
    --tool-call-parser mimo \
    --context-length 262144 \
    --attention-backend fa3 \
    --speculative-algorithm EAGLE \
    --speculative-num-steps 3 \
    --speculative-eagle-topk 1 \
    --speculative-num-draft-tokens 4 \
    --enable-mtp
```

--------------------------------

### Verify System Prerequisites

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Check for NVIDIA GPU availability, Python version, and pip installation.

```bash
# Check NVIDIA GPU availability
nvidia-smi

# Verify Python version (3.10+)
python --version

# Verify pip is installed
pip --version
```

--------------------------------

### Verify SGLang Installation

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Confirm the installed version of SGLang.

```bash
python -c "import sglang; print(sglang.__version__)"
```

--------------------------------

### Launch Inference Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/model-specification.md

Command to start the SGLang inference server with optimized parameters for MiMo-V2-Flash.

```bash
SGLANG_ENABLE_SPEC_V2=1 python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --pp-size 1 \
    --dp-size 2 \
    --enable-dp-attention \
    --tp-size 8 \
    --moe-a2a-backend deepep \
    --page-size 1 \
    --host 0.0.0.0 \
    --port 9001 \
    --trust-remote-code \
    --mem-fraction-static 0.75 \
    --max-running-requests 128 \
    --chunked-prefill-size 16384 \
    --reasoning-parser qwen3 \
    --tool-call-parser mimo \
    --context-length 262144 \
    --attention-backend fa3 \
    --speculative-algorithm EAGLE \
    --speculative-num-steps 3 \
    --speculative-eagle-topk 1 \
    --speculative-num-draft-tokens 4 \
    --enable-mtp
```

--------------------------------

### Install SGLang

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Install the specific SGLang version required for MiMo-V2-Flash.

```bash
pip install sglang==0.5.6.post2.dev8+pr.15207.g39d5bd57a \
  --index-url https://sgl-project.github.io/whl/pr/ \
  --extra-index-url https://pypi.org/simple
```

--------------------------------

### Multi-turn Tool Use Flow

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/model-specification.md

Example requests and responses demonstrating the reasoning mode and tool call sequence.

```json
{
  "messages": [
    {
      "role": "user",
      "content": "What is the weather tomorrow?"
    }
  ],
  "model": "mimo-v2-flash",
  "chat_template_kwargs": {
    "enable_thinking": true
  }
}
```

```json
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "I'll check the weather for you.",
      "reasoning_content": "[Thinking: User asked about tomorrow's weather. I should use the weather_forecast tool.]"
    },
    "tool_calls": [{
      "id": "call_1",
      "type": "function",
      "function": {
        "name": "weather_forecast",
        "arguments": "{\"location\": \"current\", \"days\": 1}"
      }
    }]
  }]
}
```

```json
{
  "messages": [
    {
      "role": "user",
      "content": "What is the weather tomorrow?"
    },
    {
      "role": "assistant",
      "content": "I'll check the weather for you.",
      "reasoning_content": "[Thinking: User asked about tomorrow's weather. I should use the weather_forecast tool.]"
    },
    {
      "role": "user",
      "content": "The weather tomorrow is sunny, 25°C"
    }
  ],
  "model": "mimo-v2-flash"
}
```

--------------------------------

### Interact with Server via Python

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Client implementation examples using the standard requests library and the OpenAI Python SDK.

```python
import requests
import json

url = "http://localhost:9001/v1/chat/completions"

payload = {
    "messages": [
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
    "model": "mimo-v2-flash",
    "max_tokens": 512,
    "temperature": 0.8,
    "top_p": 0.95
}

headers = {"Content-Type": "application/json"}

response = requests.post(url, json=payload, headers=headers)
result = response.json()

print(json.dumps(result, indent=2))
```

```python
from openai import OpenAI

client = OpenAI(
    api_key="not-needed",
    base_url="http://localhost:9001/v1"
)

response = client.chat.completions.create(
    model="mimo-v2-flash",
    messages=[
        {
            "role": "user",
            "content": "Explain quantum computing"
        }
    ],
    max_tokens=2048,
    temperature=0.8,
    top_p=0.95
)

print(response.choices[0].message.content)
```

--------------------------------

### Specify GPU Devices

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Example configuration for restricting the server to specific GPU indices.

```bash
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
```

--------------------------------

### Launch Optimized 8-GPU Production Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Optimized production setup featuring speculative decoding, MTP, and specific parsers.

```bash
export SGLANG_ENABLE_SPEC_V2=1
python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --tp-size 8 \
    --dp-size 1 \
    --enable-dp-attention \
    --mem-fraction-static 0.75 \
    --max-running-requests 64 \
    --chunked-prefill-size 16384 \
    --attention-backend fa3 \
    --context-length 262144 \
    --enable-mtp \
    --speculative-algorithm EAGLE \
    --speculative-num-steps 3 \
    --speculative-eagle-topk 1 \
    --speculative-num-draft-tokens 4 \
    --reasoning-parser qwen3 \
    --tool-call-parser mimo \
    --port 9001 \
    --host 0.0.0.0 \
    --trust-remote-code
```

--------------------------------

### GET /v1/models

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Retrieves a list of available models deployed on the server.

```APIDOC
## GET /v1/models

### Description
Returns a list of models currently loaded and available on the server.

### Method
GET

### Endpoint
http://localhost:9001/v1/models

### Response
#### Success Response (200)
- **data** (array) - List of model objects
- **object** (string) - Type of the response object

#### Response Example
{
  "data": [
    {
      "id": "mimo-v2-flash",
      "object": "model",
      "owned_by": "XiaomiMiMo",
      "permission": []
    }
  ],
  "object": "list"
}
```

--------------------------------

### GET /metrics

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Retrieves performance metrics from the SGLang dashboard.

```APIDOC
## GET /metrics

### Description
Accesses performance metrics for the running model instance.

### Method
GET

### Endpoint
http://localhost:9001/metrics
```

--------------------------------

### Nginx Rate Limiting Configuration

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/api-reference.md

Example configuration for implementing rate limiting at the reverse proxy level using Nginx.

```nginx
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;

location /v1/chat/completions {
    limit_req zone=api burst=20 nodelay;
    proxy_pass http://localhost:9001;
}
```

--------------------------------

### Interact with Server via curl

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Examples for sending chat completion requests using curl, including streaming and reasoning modes.

```bash
curl -i http://localhost:9001/v1/chat/completions \
    -H 'Content-Type:application/json' \
    -d '{
        "messages": [{
            "role": "user",
            "content": "Hello MiMo"
        }],
        "model": "mimo-v2-flash",
        "max_tokens": 4096,
        "temperature": 0.8,
        "top_p": 0.95,
        "stream": false
    }'
```

```bash
curl -i http://localhost:9001/v1/chat/completions \
    -H 'Content-Type:application/json' \
    -d '{
        "messages": [{
            "role": "user",
            "content": "Write a poem about AI"
        }],
        "model": "mimo-v2-flash",
        "max_tokens": 2048,
        "temperature": 0.8,
        "top_p": 0.95,
        "stream": true
    }'
```

```bash
curl -i http://localhost:9001/v1/chat/completions \
    -H 'Content-Type:application/json' \
    -d '{
        "messages": [{
            "role": "user",
            "content": "Solve: If a train travels at 60 mph for 3 hours, how far does it go?"
        }],
        "model": "mimo-v2-flash",
        "max_tokens": 2048,
        "temperature": 0.8,
        "top_p": 0.95,
        "chat_template_kwargs": {
            "enable_thinking": true
        }
    }'
```

--------------------------------

### Define Dockerfile for MiMo v2 Flash

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Base image configuration and dependency installation for the MiMo v2 Flash container.

```dockerfile
FROM pytorch/pytorch:2.4.0-cuda12.1-cudnn8-devel

RUN pip install -U pip

# Install SGLang with MiMo support
RUN pip install sglang==0.5.6.post2.dev8+pr.15207.g39d5bd57a \
    --index-url https://sgl-project.github.io/whl/pr/ \
    --extra-index-url https://pypi.org/simple

# Copy startup script
COPY start_server.sh /app/
RUN chmod +x /app/start_server.sh

WORKDIR /app

EXPOSE 9001

CMD ["./start_server.sh"]
```

--------------------------------

### Launch Minimal Development Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Use for testing purposes only. Requires at least 40GB VRAM on a single GPU.

```bash
python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --port 9001
```

--------------------------------

### Verify Server Startup

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Check for the startup confirmation message.

```text
INFO:     Application startup complete
```

--------------------------------

### Deploy with Docker

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Builds the container image and runs the inference server with GPU support.

```bash
# Build image
docker build -t mimo-v2-flash .

# Run with GPUs
docker run --gpus all -p 9001:9001 mimo-v2-flash
```

--------------------------------

### GET /health

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Checks the current health status of the server.

```APIDOC
## GET /health

### Description
Verifies that the server is running and responsive.

### Method
GET

### Endpoint
http://localhost:9001/health
```

--------------------------------

### Launch Long-Context Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Configuration optimized for long-context tasks with 262k context length support.

```bash
export SGLANG_ENABLE_SPEC_V2=1
python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --tp-size 8 \
    --dp-size 1 \
    --enable-dp-attention \
    --mem-fraction-static 0.6 \
    --chunked-prefill-size 8192 \
    --context-length 262144 \
    --attention-backend fa3 \
    --max-running-requests 16 \
    --enable-mtp \
    --port 9001 \
    --host 0.0.0.0 \
    --trust-remote-code
```

--------------------------------

### Launch Standard 4-GPU Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Standard configuration for 4 GPUs with static memory allocation.

```bash
export SGLANG_ENABLE_SPEC_V2=1
python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --tp-size 4 \
    --dp-size 1 \
    --mem-fraction-static 0.75 \
    --max-running-requests 32 \
    --port 9001 \
    --trust-remote-code
```

--------------------------------

### Configure Attention Backend and Context Length

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Recommended settings for achieving maximum performance and supporting large context windows.

```bash
--attention-backend fa3 --context-length 262144
```

--------------------------------

### Example chat completion response schema

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/model-specification.md

The structure of the JSON object returned by the chat completion API.

```json
{
  "id": "chatcmpl-...",
  "object": "text_completion",
  "created": 1234567890,
  "model": "mimo-v2-flash",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "...",
        "reasoning_content": "..."
      },
      "tool_calls": [...],
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}
```

--------------------------------

### Configure English System Prompt

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Initializes the MiMo assistant with English system instructions and sends a user query.

```python
system_message = """You are MiMo, an AI assistant developed by Xiaomi.

Today's date: 2026-08-19 Tuesday. Your knowledge cutoff date is December 2024."""

response = client.chat.completions.create(
    model="mimo-v2-flash",
    messages=[
        {"role": "system", "content": system_message},
        {"role": "user", "content": "Your question here"}
    ]
)
```

--------------------------------

### Set Attention Backend

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Recommended configuration to enable Flash Attention v3 for maximum speed.

```bash
export SGLANG_ATTENTION_BACKEND=fa3
```

--------------------------------

### Configure MoE Backend

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Required setting for optimal performance with MoE models.

```bash
--moe-a2a-backend deepep
```

--------------------------------

### Few-Shot Learning Message Structure

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/features-and-capabilities.md

Define model behavior by providing a sequence of example interactions within the messages array.

```json
{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant..."
    },
    {
      "role": "user",
      "content": "Example 1 question"
    },
    {
      "role": "assistant",
      "content": "Example 1 answer"
    },
    {
      "role": "user",
      "content": "Actual question"
    }
  ]
}
```

--------------------------------

### Configure Memory Management by Hardware Constraint

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Recommended environment variable settings for static memory reservation and chunked prefill size based on GPU cluster size.

```bash
export SGLANG_MEM_FRACTION_STATIC=0.6
export SGLANG_CHUNKED_PREFILL_SIZE=8192
```

```bash
export SGLANG_MEM_FRACTION_STATIC=0.75
export SGLANG_CHUNKED_PREFILL_SIZE=16384
```

```bash
export SGLANG_MEM_FRACTION_STATIC=0.85
export SGLANG_CHUNKED_PREFILL_SIZE=32768
```

--------------------------------

### Configure Production Server Binding

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Settings for network access and SSL/TLS security in production environments.

```bash
--host 0.0.0.0 --port 9001 --ssl-certfile /path/to/cert.pem --ssl-keyfile /path/to/key.pem
```

--------------------------------

### Performance Configuration Presets

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Recommended parameter settings for different operational priorities.

```json
# Enable thinking mode for hard problems
"chat_template_kwargs": {"enable_thinking": true}

# Use balanced temperature
"temperature": 0.8
```

```json
# Use low temperature for deterministic output
"temperature": 0.3

# Reduce token limit
"max_tokens": 512
```

```json
# Standard recommendation
"temperature": 0.8,
"top_p": 0.95,
"max_tokens": 2048
```

--------------------------------

### Configure Environment Variables

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Set the required environment variables for SGLang and GPU visibility.

```bash
export SGLANG_ENABLE_SPEC_V2=1
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
```

--------------------------------

### Configure Reasoning Mode

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/performance-benchmarks.md

Enable reasoning mode to improve output quality.

```yaml
enable_thinking: true
```

--------------------------------

### Launch High-Throughput 16-GPU Server

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

High-throughput configuration utilizing 16 GPUs with deepEP backend for MoE.

```bash
export SGLANG_ENABLE_SPEC_V2=1
python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --tp-size 8 \
    --dp-size 2 \
    --enable-dp-attention \
    --moe-a2a-backend deepep \
    --mem-fraction-static 0.85 \
    --max-running-requests 256 \
    --chunked-prefill-size 32768 \
    --attention-backend fa3 \
    --context-length 262144 \
    --enable-mtp \
    --speculative-algorithm EAGLE \
    --speculative-num-steps 3 \
    --reasoning-parser qwen3 \
    --tool-call-parser mimo \
    --port 9001 \
    --host 0.0.0.0 \
    --trust-remote-code
```

--------------------------------

### Chat Interface with OpenAI SDK

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Initializes the client and performs a basic chat completion request.

```python
from openai import OpenAI

client = OpenAI(
    api_key="not-needed",
    base_url="http://localhost:9001/v1"
)

response = client.chat.completions.create(
    model="mimo-v2-flash",
    messages=[
        {"role": "user", "content": "Explain quantum computing"}
    ],
    max_tokens=1024,
    temperature=0.8
)

print(response.choices[0].message.content)
```

--------------------------------

### Pre-download Model

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Use the Python transformers library to cache the model locally before server startup.

```python
python -c "from transformers import AutoTokenizer, AutoModelForCausalLM; \
AutoTokenizer.from_pretrained('XiaomiMiMo/MiMo-V2-Flash'); \
AutoModelForCausalLM.from_pretrained('XiaomiMiMo/MiMo-V2-Flash', \
trust_remote_code=True, device_map='auto')"
```

--------------------------------

### System Prompt Templates

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/features-and-capabilities.md

Recommended system prompts for English and Chinese interactions, including placeholders for date and week context.

```text
You are MiMo, an AI assistant developed by Xiaomi.

Today's date: {date} {week}. Your knowledge cutoff date is December 2024.
```

```text
你是MiMo（中文名称也是MiMo），是小米公司研发的AI智能助手。

今天的日期：{date} {week}，你的知识截止日期是2024年12月。
```

--------------------------------

### Configure for Long-Context Applications

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Use these settings to support extended context lengths.

```bash
# Server config
--context-length 262144
--mem-fraction-static 0.6
--chunked-prefill-size 8192
--max-running-requests 16

# Request config
{"temperature": 0.8, "max_tokens": 4096}
```

--------------------------------

### Configure Tool Call Parsers

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Specify parsers during server launch to enable tool call functionality.

```bash
--reasoning-parser qwen3 \
--tool-call-parser mimo
```

--------------------------------

### Configure Parsing for MiMo Tools

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Set parsers for reasoning and tool calling functionality.

```bash
--reasoning-parser qwen3 --tool-call-parser mimo
```

--------------------------------

### Test Server with cURL

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Send a chat completion request to the running server.

```bash
curl -X POST http://localhost:9001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "Hello MiMo"}
    ],
    "model": "mimo-v2-flash",
    "max_tokens": 512
  }'
```

--------------------------------

### Integrate Mimo-v2-flash with OpenAI SDK

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/INDEX.md

Use the OpenAI client to interact with the Mimo-v2-flash model by pointing the base_url to the local inference server.

```python
from openai import OpenAI
client = OpenAI(api_key="not-needed", base_url="http://localhost:9001/v1")
response = client.chat.completions.create(
    model="mimo-v2-flash",
    messages=[{"role": "user", "content": "Hello"}]
)
```

--------------------------------

### Login to HuggingFace

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Authenticate with the HuggingFace CLI to access gated models.

```bash
huggingface-cli login
```

--------------------------------

### Perform Server Health Check

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Verify the model server is running and listing available models.

```bash
curl http://localhost:9001/v1/models
```

--------------------------------

### Integrate Mimo-v2-flash with LangChain

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/INDEX.md

Configure the ChatOpenAI class to use the Mimo-v2-flash model by specifying the custom base URL.

```python
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(model="mimo-v2-flash", openai_api_base="http://localhost:9001/v1")
```

--------------------------------

### Configure for Maximum Throughput

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Use these settings for batch processing workloads.

```bash
# Server config
--max-running-requests 256
--mem-fraction-static 0.85
--dp-size 2
--chunked-prefill-size 32768

# Request config
{"temperature": 0.8, "max_tokens": 2048}
```

--------------------------------

### Enable extended reasoning processing

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/features-and-capabilities.md

Configure the chat template to activate extended reasoning capabilities.

```json
chat_template_kwargs: {"enable_thinking": true}
```

--------------------------------

### Configure for Reasoning-Heavy Tasks

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Enable thinking capabilities for complex reasoning tasks.

```json
{
  "temperature": 0.8,
  "top_p": 0.95,
  "chat_template_kwargs": {"enable_thinking": true}
}
```

--------------------------------

### Manually Download Model to Cache

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Use this Python command to force a download of the MiMo-V2-Flash model to the local cache.

```python
python -c "from transformers import AutoTokenizer; AutoTokenizer.from_pretrained('XiaomiMiMo/MiMo-V2-Flash')"
```

--------------------------------

### Configure for Maximum Latency

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Use these settings for real-time applications requiring low latency.

```bash
# Server config
--max-running-requests 8
--mem-fraction-static 0.75
--tp-size 8

# Request config
{"temperature": 0.3, "top_p": 0.9, "max_tokens": 512}
```

--------------------------------

### Simple Chat Completion

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/api-reference.md

Perform a basic chat request with the MiMo v2 Flash model.

```bash
curl -X POST http://localhost:9001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "Hello MiMo"}
    ],
    "model": "mimo-v2-flash",
    "temperature": 0.8,
    "top_p": 0.95
  }'
```

```json
{
  "id": "chatcmpl-abc123",
  "object": "text_completion",
  "created": 1692893821,
  "model": "mimo-v2-flash",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! I'm MiMo, an AI assistant developed by Xiaomi. How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 22,
    "total_tokens": 33
  }
}
```

--------------------------------

### Enable Latency Optimizations

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Apply specific backend and algorithm flags to improve response times.

```bash
--enable-mtp \
--speculative-algorithm EAGLE \
--attention-backend fa3
```

--------------------------------

### Enable Speed Optimization Flags

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/performance-benchmarks.md

Use these command-line flags to prioritize inference speed.

```bash
--enable-mtp
```

```bash
--speculative-algorithm EAGLE
```

--------------------------------

### Access Performance Metrics

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Retrieve server performance metrics via the metrics endpoint.

```bash
# Access metrics at:
# http://localhost:9001/metrics
curl http://localhost:9001/metrics
```

--------------------------------

### Streaming Request

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/api-reference.md

How to initiate a streaming request by setting the stream parameter to true.

```APIDOC
## Streaming Request

### Description
To enable streaming responses, include the `stream` parameter set to `true` in your request body.

### Request Body
- **messages** (array) - Required - List of message objects.
- **model** (string) - Required - The model identifier, e.g., "mimo-v2-flash".
- **stream** (boolean) - Required - Set to `true` to enable SSE streaming.

### Request Example
{
  "messages": [
    {"role": "user", "content": "Count to 5"}
  ],
  "model": "mimo-v2-flash",
  "stream": true
}
```

--------------------------------

### Define Required Chat Completion Parameters

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Minimum configuration required to initiate a chat completion request.

```json
{
  "messages": [
    {"role": "user", "content": "Hello"}
  ],
  "model": "mimo-v2-flash"
}
```

--------------------------------

### Sampling Presets for Common Use Cases

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Pre-configured sampling parameters for specific tasks like code generation, creative writing, or reasoning.

```json
{
  "temperature": 0.2,
  "top_p": 0.9
}
```

```json
{
  "temperature": 0.8,
  "top_p": 0.95
}
```

```json
{
  "temperature": 1.0,
  "top_p": 0.95
}
```

```json
{
  "temperature": 0.3,
  "top_p": 0.9
}
```

```json
{
  "temperature": 0.8,
  "top_p": 0.95,
  "chat_template_kwargs": {"enable_thinking": true}
}
```

--------------------------------

### Enable Full Inference Optimization

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Configuration for Multi-Token Prediction and EAGLE speculative decoding.

```bash
--enable-mtp \
--speculative-algorithm EAGLE \
--speculative-num-steps 3 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 4
```

--------------------------------

### Inspect Server Logs

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

View the last 100 lines of the server log file to diagnose errors.

```bash
tail -100 /tmp/sglang.log
```

--------------------------------

### Verify Server Status

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Check if the server is listening on the expected port and verify the process is running.

```bash
netstat -tlnp | grep 9001
ps aux | grep sglang
```

--------------------------------

### Deploy Kubernetes StatefulSet and Service

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Orchestration manifest for deploying the application as a StatefulSet with a LoadBalancer service.

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mimo-v2-flash
spec:
  serviceName: mimo-v2-flash
  replicas: 1
  selector:
    matchLabels:
      app: mimo-v2-flash
  template:
    metadata:
      labels:
        app: mimo-v2-flash
    spec:
      containers:
      - name: mimo-server
        image: mimo-v2-flash:latest
        ports:
        - containerPort: 9001
          name: api
        env:
        - name: SGLANG_ENABLE_SPEC_V2
          value: "1"
        resources:
          limits:
            nvidia.com/gpu: "8"
          requests:
            nvidia.com/gpu: "8"
        volumeMounts:
        - name: huggingface-cache
          mountPath: /root/.cache/huggingface
      volumes:
      - name: huggingface-cache
        persistentVolumeClaim:
          claimName: huggingface-cache-pvc

---
apiVersion: v1
kind: Service
metadata:
  name: mimo-v2-flash
spec:
  selector:
    app: mimo-v2-flash
  ports:
  - port: 9001
    targetPort: 9001
  type: LoadBalancer
```

--------------------------------

### POST /v1/chat/completions

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/model-specification.md

Generates a chat completion using the Mimo v2 Flash model.

```APIDOC
## POST /v1/chat/completions

### Description
Generates a chat completion response based on the provided message history and model configuration.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Request Body
- **messages** (Array[Message]) - Required - Conversation message history
- **model** (string) - Required - Model identifier (e.g., mimo-v2-flash)
- **max_tokens** (integer) - Optional - Maximum tokens to generate
- **temperature** (number) - Optional - Sampling temperature (0.0 to 2.0, default 0.8)
- **top_p** (number) - Optional - Nucleus sampling parameter (default 0.95)
- **stream** (boolean) - Optional - Enable streaming responses (default false)
- **chat_template_kwargs** (object) - Optional - Additional template parameters (e.g., enable_thinking: boolean)

### Response
#### Success Response (200)
- **id** (string) - Unique completion identifier
- **object** (string) - Object type: text_completion
- **created** (integer) - Unix timestamp of creation
- **model** (string) - Model identifier used for generation
- **choices** (Array[Choice]) - List of generated completions
- **usage** (Usage) - Token usage statistics
```

--------------------------------

### Set Environment Variables

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Configure environment variables for speculative decoding and performance optimizations.

```bash
export SGLANG_ENABLE_SPEC_V2=1
```

```bash
# Enable specific optimization backends
export SGLANG_ENABLE_FA3=1

# Set GPU memory fraction
export SGLANG_MEM_FRACTION_STATIC=0.75

# Enable multi-turn optimization
export SGLANG_ENABLE_PREFIX_CACHE=1
```

--------------------------------

### Configure Systemd Service

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Defines the systemd service unit file for managing the inference server process.

```ini
[Unit]
Description=MiMo-V2-Flash Inference Server
After=network.target

[Service]
Type=simple
User=nvidia
WorkingDirectory=/home/nvidia
Environment="SGLANG_ENABLE_SPEC_V2=1"
Environment="CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
ExecStart=/usr/bin/python3 -m sglang.launch_server \
    --model-path XiaomiMiMo/MiMo-V2-Flash \
    --served-model-name mimo-v2-flash \
    --tp-size 8 \
    --port 9001 \
    --host 0.0.0.0 \
    --trust-remote-code
Restart=always

[Install]
WantedBy=multi-user.target
```

--------------------------------

### Verify HuggingFace Access

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/quick-start.md

Authenticate and verify credentials if the model is not found.

```bash
huggingface-cli login
huggingface-cli whoami
```

--------------------------------

### Reasoning Mode

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/api-reference.md

Enable internal reasoning by setting the enable_thinking flag in chat_template_kwargs.

```bash
curl -X POST http://localhost:9001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "If x + 3 = 7, what is x?"}
    ],
    "model": "mimo-v2-flash",
    "chat_template_kwargs": {
      "enable_thinking": true
    }
  }'
```

```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "The answer is x = 4.",
        "reasoning_content": "[Thinking: I need to solve for x. Given x + 3 = 7, I subtract 3 from both sides. x = 7 - 3 = 4.]"
      },
      "finish_reason": "stop"
    }
  ]
}
```

--------------------------------

### Check GPU Memory

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Use this command to inspect current GPU memory usage.

```bash
nvidia-smi
```

--------------------------------

### Enable Speculative Decoding v2

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/configuration-reference.md

Required setting to enable EAGLE-based speculative decoding for optimal performance.

```bash
export SGLANG_ENABLE_SPEC_V2=1
```

--------------------------------

### Configure Docker Compose for MiMo v2 Flash

Source: https://github.com/xiaomimimo/mimo-v2-flash/blob/main/_autodocs/deployment-guide.md

Service definition for running the container with GPU resource reservations and environment variables.

```yaml
version: '3.8'

services:
  mimo-server:
    image: mimo-v2-flash:latest
    container_name: mimo-v2-flash
    environment:
      - SGLANG_ENABLE_SPEC_V2=1
      - NVIDIA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
    ports:
      - "9001:9001"
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 8
              capabilities: [gpu]
```
Tokens:
Raw