### Run Python Example Script

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/AGENTS.md

Execute a Python example script located in the examples/python/ directory.

```bash
python3 examples/python/<script>.py
```

--------------------------------

### Full MCP Configuration Example

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/mcp-config-schema.md

An example demonstrating a complete MCP client configuration with multiple servers using different transport types and custom settings.

```json
{
  "servers": [
    {
      "name": "filesystem",
      "tool_prefix": "fs",
      "source": {
        "type": "Process",
        "command": "npx",
        "args": ["-y", "@modelcontextprotocol/server-filesystem", "/srv/agent-workspace"]
      }
    },
    {
      "name": "github",
      "source": {
        "type": "Http",
        "url": "https://mcp.github.example.com",
        "headers": {
          "Authorization": "Bearer xxx"
        }
      }
    }
  ],
  "auto_register_tools": true,
  "max_concurrent_calls": 8,
  "tool_timeout_secs": 45
}
```

--------------------------------

### Install mistralrs Python Package

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/tutorials/03-python-sdk.md

Install the Python package that matches your hardware accelerator. Only one wheel should be installed.

```bash
pip install mistralrs             # CPU, or Intel CPU with MKL
pip install mistralrs-cuda        # NVIDIA GPUs
pip install mistralrs-metal       # Apple Silicon
pip install mistralrs-mkl         # Intel CPU, MKL wheel with symbols pinned
pip install mistralrs-accelerate  # macOS, Accelerate framework
```

--------------------------------

### Serve Example with Sandbox Enabled

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/sandbox.md

Example of running the mistralrs serve command with code execution enabled and a configured sandbox. Network access is blocked, and memory is capped.

```bash
```bash
mistralrs serve \
  -m mistralrs-community/gemma-4-E4B-it-UQFF \
  --from-uqff 8 \
  --enable-code-execution \
  --sandbox on \
  --sandbox-network none \
  --sb-max-memory-mb 2048 \
  --code-exec-workdir . \
  --enable-search
```
```

--------------------------------

### Install CLI Binary

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/CLAUDE.md

Install the mistralrs CLI binary. Replace <features> with the desired features.

```bash
# Install CLI binary
cargo install --path mistralrs-cli --features <features>
```

--------------------------------

### Install OpenAI Python Client

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/tutorials/02-serve-an-api.md

Install the official `openai` Python package to interact with the API server.

```bash
pip install openai
```

--------------------------------

### Mistral.rs Agent Executor with Tools

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/server/phi3_duckduckgo_mistral.rs.ipynb

Example of setting up an agent executor in Rust using Mistral.rs, integrating with tools and a prompt template. This snippet shows the agent's setup and execution flow.

```rust
use langchain_rust::chains::llm_chain::LLMChain;
use langchain_rust::chains::sequential::SimpleSequentialChain;
use langchain_rust::chat_models::ChatModel;
use langchain_rust::chat_models::Message;
use langchain_rust::chat_models::MessageType;
use langchain_rust::llms::llm::LLM;
use langchain_rust::prompt::PromptTemplate;
use langchain_rust::tools::Tool;
use langchain_rust::tools::ToolDescription;
use langchain_rust::tools::ToolInputSchema;
use langchain_rust::tools::ToolOutputSchema;
use serde::{Deserialize, Serialize};

// Define a simple tool
#[derive(Debug, Clone)]
struct EchoTool;

#[derive(Debug, Clone, Serialize, Deserialize)]
struct EchoInput {
    text: String,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
struct EchoOutput {
    text: String,
}

impl ToolInputSchema for EchoTool {
    type Input = EchoInput;
}

impl ToolOutputSchema for EchoTool {
    type Output = EchoOutput;
}

impl Tool for EchoTool {
    fn name(&self) -> &str {
        "echo"
    }

    fn description(&self) -> &str {
        "Echoes the input text."
    }

    fn run(&self, input: <Self as ToolInputSchema>::Input) -> <Self as ToolOutputSchema>::Output {
        EchoOutput { text: input.text }
    }
}

#[tokio::main]
async fn main() {
    // Initialize the LLM (replace with your actual Mistral.rs model)
    let llm = LLM::new("mistralai/Mistral-7B-Instruct-v0.1");

    // Define a prompt template
    let prompt = PromptTemplate::from("What is {{input}}?");

    // Create an LLMChain
    let llm_chain = LLMChain::new(llm.clone(), prompt);

    // Create a tool description
    let echo_tool_description = ToolDescription {
        name: "echo".to_string(),
        description: "Echoes the input text.".to_string(),
        input_schema: EchoInput::schema().to_string(),
        output_schema: EchoOutput::schema().to_string(),
    };

    // Create a tool instance
    let echo_tool = EchoTool;

    // Create an agent executor
    let mut agent_executor = AgentExecutor::new(llm, vec![echo_tool.into()]);

    // Define the input for the agent
    let input = "Mistral.rs";

    // Execute the agent
    match agent_executor.invoke(input).await {
        Ok(output) => println!("Agent output: {:?}", output),
        Err(e) => eprintln!("Agent error: {:?}", e),
    }
}

```

--------------------------------

### Install mistralrs-cli Globally from Source

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/install/from-source.md

Install the mistralrs-cli binary globally from a local source checkout, enabling custom features.

```bash
cargo install --path mistralrs-cli --features "cuda flash-attn cudnn"
```

--------------------------------

### Start mistral.rs API Server with Web UI

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/README.md

Starts an API server and makes the built-in web UI accessible. The UI is typically found at http://localhost:1234/ui.

```bash
mistralrs serve -m google/gemma-4-E4B-it
```

--------------------------------

### Install mistral.rs on Windows (PowerShell)

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/README.md

Installs the mistral.rs CLI tool using a PowerShell command. Review scripts before execution.

```powershell
irm https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.ps1 | iex
```

--------------------------------

### Example API Call to Use MCP Tools

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/MCP_QUICK_START.md

This example demonstrates how to use MCP tools by sending a chat completion request to the Mistral.rs server. The model automatically uses the configured tools to list files and create a file.

```bash
# File operations example
curl -X POST http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-4B",
    "messages": [
      {"role": "user", "content": "List the files in the current directory and create a hello.txt file"}
    ]
  }'
```

--------------------------------

### Install mistral.rs CLI with Metal and Accelerate Features

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/install/macos-metal.md

Installs the mistral.rs command-line interface from crates.io, enabling Metal GPU support and Accelerate framework for CPU-based computations. Ensure Rust and necessary build tools are installed.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.sh | sh
```

```bash
cargo install mistralrs-cli --features "metal accelerate"
```

--------------------------------

### Install mistral.rs CLI from Source with Metal and Accelerate

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/install/macos-metal.md

Installs the mistral.rs command-line interface from its source repository, enabling Metal GPU support and Accelerate framework. This method requires cloning the repository and building from source.

```bash
git clone https://github.com/EricLBuehler/mistral.rs.git
cd mistral.rs
cargo install --path mistralrs-cli --features "metal accelerate"
```

--------------------------------

### Run Server

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/CLAUDE.md

Start the mistralrs server on a specified port with a given model ID.

```bash
# Run server
mistralrs serve -p 1234 -m <model_id>
```

--------------------------------

### Basic Rust SDK Chatbot Example

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/README.md

Demonstrates building a multimodal model with specific quantization and logging, then sending a chat request.

```rust
use anyhow::Result;
use mistralrs::{IsqType, TextMessageRole, TextMessages, MultimodalModelBuilder};

#[tokio::main]
async fn main() -> Result<()> {
    let model = MultimodalModelBuilder::new("google/gemma-4-E4B-it")
        .with_isq(IsqType::Q4K)
        .with_logging()
        .build()
        .await?;

    let messages = TextMessages::new().add_message(
        TextMessageRole::User,
        "Hello!",
    );

    let response = model.send_chat_request(messages).await?;

    println!("{:?}", response.choices[0].message.content);

    Ok(())
}
```

--------------------------------

### Start Mistral.rs Server with Phi-3

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/server/phi3_duckduckgo_mistral.rs.ipynb

Launches the Mistral.rs server with the Phi-3 model. Ensure the model is downloaded and accessible.

```bash
mistralrs serve -p 8099 --quant q4k -m microsoft/Phi-3-mini-128k-instruct
```

--------------------------------

### Install Ubuntu on WSL2

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/install/windows.md

Execute this command in an administrator PowerShell to install the Ubuntu distribution for WSL2. This is a prerequisite for using mistral.rs within WSL2.

```powershell
wsl --install -d Ubuntu
```

--------------------------------

### Install mistral.rs

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/index.mdx

Installs the mistral.rs binary. Supports Linux, macOS, and Windows. Auto-detects CUDA, Metal, or MKL on compatible systems.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.sh | sh
```

```powershell
irm https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.ps1 | iex
```

--------------------------------

### Install mistral.rs on Linux/macOS

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/README.md

Installs the mistral.rs CLI tool using a curl script. Ensure you review scripts before execution.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.sh | sh
```

--------------------------------

### Check FFmpeg installation

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/video-setup.md

Verify if the FFmpeg binary is installed and accessible in the server's PATH. This is crucial for processing non-GIF video files.

```bash
ffmpeg -version
```

--------------------------------

### Build and Install CLI Binary

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/AGENTS.md

Builds and installs the mistralrs-cli binary with specified features. This is useful for deploying the command-line interface.

```bash
cargo build --release --package mistralrs-cli --features "<features>"
cargo install --path mistralrs-cli --features "<features>"
```

--------------------------------

### Start Agentic Server

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/agents/agentic-runtime.md

Starts a Mistral.rs server with agent capabilities enabled, including search and code execution. The UI is enabled by default.

```bash
mistralrs serve --agent -m google/gemma-4-E4B-it
```

--------------------------------

### Start mistral.rs with MCP config

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/agents/connect-mcp-server.md

Launch mistral.rs as an MCP client using a configuration file or environment variable.

```bash
mistralrs serve --mcp-config mcp.json -m <model>
```

--------------------------------

### Start mistralrs Server with TOML Configuration

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/serve/multiple-models.md

Launch the mistralrs server using a TOML configuration file to load multiple models.

```bash
mistralrs from-config -f models.toml
```

--------------------------------

### Run mistral.rs with MCP Configuration

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/mcp-config-schema.md

Command-line instruction to start the mistral.rs server with a specified MCP configuration file and model.

```bash
mistralrs serve --mcp-config mcp.json -m Qwen/Qwen3-4B
```

--------------------------------

### Run Agent with Ask Permission

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/agents/agentic-runtime.md

Starts the mistral.rs server in agent mode, prompting for approval before each agent action.

```bash
mistralrs run --agent -m google/gemma-4-E4B-it --agent-permission ask
```

--------------------------------

### Serve Voxtral Model via CLI

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/use-speech-models.md

Starts the mistral.rs server with the Voxtral model. Ensure the model path is correct.

```bash
mistralrs serve -m mistralai/Voxtral-Mini-3B-2507
```

--------------------------------

### Serve FLUX diffusion model

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/use-image-generation.md

Start the mistral.rs server with the FLUX model. Ensure you accept any required licenses before running.

```bash
mistralrs serve -m black-forest-labs/FLUX.1-schnell
```

--------------------------------

### Start mistral.rs Server with Basic Configuration

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/deploy/production-checklist.md

Launches the mistral.rs server with specified host, port, quantization level, and model path. Ensure the model path is correctly set.

```bash
mistralrs serve --host 127.0.0.1 --port 8080 --quant 4 -m <model>
```

--------------------------------

### Run mistral.rs for Interactive Chat

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/README.md

Starts an interactive chat session with a specified model. No special setup is required beyond installation.

```bash
mistralrs run -m Qwen/Qwen3-4B
```

--------------------------------

### Send video to a multimodal model

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/python/multimodal-input.md

Send video files to models like Gemma 4 by providing a `video_url`. The engine decodes frames and passes them to the model for analysis. Setup and FFmpeg installation are detailed in a separate guide.

```python
response = runner.send_chat_completion_request(
    ChatCompletionRequest(
        model="google/gemma-4-E4B-it",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "video_url", "video_url": {"url": "file:///path/to/clip.mp4"}},
                    {"type": "text", "text": "Describe what happens in this video."}
                ]
            }
        ],
    )
)
```

--------------------------------

### List Models Response

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/http-api.md

Example JSON response when listing loaded models via the GET /v1/models endpoint.

```json
{
  "object": "list",
  "data": [
    {
      "id": "default",
      "object": "model",
      "created": 1234567890,
      "owned_by": "local",
      "status": "loaded",
      "tools_available": true,
      "mcp_tools_count": 5,
      "mcp_servers_connected": 1
    }
  ]
}
```

--------------------------------

### Run Server with Default UI

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/CLAUDE.md

Start the mistralrs server with the built-in web UI enabled by default. Pass --no-ui to disable.

```bash
# Run server (built-in web UI is on by default at /ui; pass --no-ui to disable)
mistralrs serve -m <model_id>
```

--------------------------------

### Initialize MCP Client and Discover Tools

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/mistralrs-mcp/README.md

Demonstrates how to configure and initialize an McpClient with HTTP server details and automatic tool registration. Ensure the bearer token and URL are correctly set for your MCP server.

```rust
use mistralrs_mcp::{McpClient, McpClientConfig, McpServerConfig, McpServerSource};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = McpClientConfig {
        servers: vec![
            McpServerConfig {
                id: "web_search".to_string(),
                name: "Web Search MCP".to_string(),
                source: McpServerSource::Http {
                    url: "https://api.example.com/mcp".to_string(),
                    timeout_secs: Some(30),
                    headers: None,
                },
                enabled: true,
                tool_prefix: Some("web".to_string()),
                resources: None,
                bearer_token: Some("your-api-token".to_string()),
            },
        ],
        auto_register_tools: true,
        tool_timeout_secs: Some(30),
        max_concurrent_calls: Some(10),
    };
    
    let mut client = McpClient::new(config);
    client.initialize().await?;
    
    let tools = client.get_tools();
    println!("Discovered {} tools", tools.len());
    
    Ok(())
}
```

--------------------------------

### Verify mistral.rs Installation

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/tutorials/01-install-and-run.md

Checks if the mistralrs binary is installed and accessible in your PATH. Run this in a new shell after installation.

```bash
mistralrs --version
```

--------------------------------

### Install FFmpeg on macOS with Homebrew

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/video-setup.md

Install FFmpeg on macOS using the Homebrew package manager. This command simplifies the installation process for Mac users.

```bash
brew install ffmpeg
```

--------------------------------

### Install Mistral.rs CUDA package

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/python/cookbook.ipynb

Install the necessary Python package for CUDA support.

```python
# First, install Rust: https://rustup.rs/
%pip install mistralrs-cuda -v
```

--------------------------------

### Serve Model with Topology

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/perf/topology.md

Command to start the mistralrs server with a specified topology configuration file. Ensure the model path is correctly provided.

```bash
mistralrs serve --topology topology.yaml -m <model>
```

--------------------------------

### Run CLI with Qwen3-VL Model

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/use-vision-input.md

Starts the CLI in interactive mode, automatically detecting file paths in prompts. Ensure the specified model is available.

```bash
mistralrs run -m Qwen/Qwen3-VL-4B-Instruct
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/server/phi3_duckduckgo_mistral.rs.ipynb

Installs necessary Python libraries for Langchain, OpenAI compatibility, and DuckDuckGo search.

```bash
python -m pip install openai duckduckgo-search langchain langchain_community langchain_openai
```

--------------------------------

### Egg.js Quickstart Development Commands

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/calibration_data/calibration_datav3.txt

These commands are used to set up and run an Egg.js application in development mode. Ensure Node.js version is >= 8.0 for Egg2.x.

```shell
$ npm install
$ npm run dev
$ open http://localhost:7001/
```

--------------------------------

### AnyMoE Configuration Initialization

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/python/anymoe.md

Initializes an AnyMoE configuration. Requires essential details like hidden size and dataset path. Optional parameters control training behavior and learning rate.

```python
__init__(
    hidden_size: int,
    dataset_json: str,
    prefix: str,
    mlp: str,
    model_ids: list[str],
    expert_type: AnyMoeExpertType,
    layers: list[int] = [],
    lr: float = 0.001,
    epochs: int = 100,
    batch_size: int = 4,
    gate_model_id: str | None = None,
    training: bool = True,
    loss_csv_path: str | None = None,
) -> None
```

--------------------------------

### Tune mistral.rs for Hardware and Emit Config

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/README.md

Benchmarks your system to find optimal settings for quantization and device mapping, then saves these to a TOML configuration file.

```bash
mistralrs tune -m Qwen/Qwen3-4B --emit-config config.toml
```

--------------------------------

### Mistral.rs Python API Example

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/server/phi3_duckduckgo_mistral.rs.ipynb

Demonstrates how to integrate mistral.rs into your Python application. It offers out-of-the-box support with documentation and examples.

```python
from langchain_community.llms import MistralAI

llm = MistralAI(
    model="mistral-tiny",
    # You can also pass parameters directly to the API, like:
    # temperature=0.3,
    # max_tokens=100,
    # top_p=0.9,
)

print(llm.invoke("What is LangChain?"))
```

--------------------------------

### Non-Streaming Response Example

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/http-api.md

Example JSON response for a non-streaming request, including a 'files' array with file metadata and 'agentic_tool_calls'.

```json
{
  "files": [
    {
      "id": "file_abc_r0_0",
      "name": "plot.png",
      "format": "png",
      "mime_type": "image/png",
      "bytes": 14823,
      "source": {"tool": "mistralrs_execute_python", "round": 0, "turn": 0},
      "data_base64": "iVBORw0KGgo..."
    }
  ],
  "agentic_tool_calls": [
    {
      "round": 0,
      "name": "mistralrs_execute_python",
      "file_ids": ["file_abc_r0_0"]
    }
  ]
}
```

--------------------------------

### Ensure UI is Enabled

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/troubleshooting.md

The UI is enabled by default. Verify that `--no-ui` was not passed during startup and that no reverse proxy is interfering with the `/ui` path.

```bash
--no-ui
```

--------------------------------

### Install FFmpeg on Fedora/RHEL

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/video-setup.md

Install FFmpeg on Fedora or RHEL-based systems using the dnf package manager. This ensures video decoding capabilities.

```bash
sudo dnf install ffmpeg
```

--------------------------------

### Set up Runner and Initial Messages

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/python/tool_calling.ipynb

Initializes the `Runner` with a specified model and architecture, and defines the initial user message for the conversation.

```python
messages = [
    {
        "role": "user",
        "content": "What is the value of the area of a circle with radius 4?",
    }
]

runner = Runner(
    which=Which.Plain(
        model_id="lamm-mit/Bioinspired-Llama-3-1-8B-128k-gamma",
        arch=Architecture.Llama,
    ),
)
```

--------------------------------

### Install with Flash Attention v3

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/perf/use-flash-attention.md

Install mistralrs-cli with Flash Attention v3 support for Hopper (H100) GPUs. Requires CUDA and cuDNN.

```bash
cargo install --path mistralrs-cli --features "cuda flash-attn flash-attn-v3 cudnn"
```

--------------------------------

### Rust SDK: Agent with Web Search and Code Execution

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/tutorials/05-build-an-agent.md

Set up and execute an agent with web search and code execution capabilities using the Rust SDK. This example configures the model and sends a request to find population data, calculate a share, and generate a chart.

```rust
use anyhow::Result;
use futures::StreamExt;
use mistralrs חיצוני:: (
    CodeExecutionConfig,
    IsqBits,
    RequestBuilder,
    SearchEmbeddingModel,
    TextMessageRole,
    TextMessages,
    TextModelBuilder,
    WebSearchOptions,
);

#[tokio::main] async fn main() -> Result<()> {
    let model = TextModelBuilder::new("Qwen/Qwen3-4B")
        .with_auto_isq(IsqBits::Four)
        .with_search(SearchEmbeddingModel::default())
        .with_code_execution(CodeExecutionConfig::default())
        .build()
        .await?;

    let messages = TextMessages::new().add_message(
        TextMessageRole::User,
        "Find recent population figures for Tokyo and Japan, calculate Tokyo's share of Japan's population, and save a bar chart as tokyo-population.png.",
    );

    let request = RequestBuilder::from(messages)
        .with_web_search_options(WebSearchOptions::default())
        .with_code_execution()
        .with_session_id("tokyo-demo")
        .set_max_tool_rounds(6)
        .require_file("tokyo-population.png");

    let mut stream = model.stream_chat_request(request).await?;
    while let Some(event) = stream.next().await {
        match event {
            mistralrs::Response::Chunk(chunk) => {
                if let Some(text) = chunk
                    .choices
                    .first()
                    .and_then(|choice| choice.delta.content.as_deref()) {
                    print!("{text}");
                }
            }
            mistralrs::Response::AgenticToolCallProgress { tool_name, .. } => {
                eprintln!("tool progress: {tool_name}");
            }
            _ => {}
        }
    }

    Ok(())
}
```

--------------------------------

### Install FFmpeg in a Dockerfile

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/video-setup.md

Integrate FFmpeg installation into a Docker image using a RUN command. This ensures video processing capabilities within containers.

```dockerfile
RUN apt-get update \
    && apt-get install -y --no-install-recommends ffmpeg \
    && rm -rf /var/lib/apt/lists/*
```

--------------------------------

### Install FFmpeg on Debian/Ubuntu

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/video-setup.md

Install FFmpeg on Debian-based systems like Ubuntu using the apt package manager. This is a prerequisite for handling video files.

```bash
sudo apt update
sudo apt install ffmpeg
```

--------------------------------

### Initialize Tools and Schemas

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/python/tool_calling.ipynb

Defines the available tools, including their names, descriptions, and parameter schemas. The `run_python` tool is defined to execute Python code and return its printed output.

```python
from io import StringIO
import json
import sys
from mistralrs import Runner, ToolChoice, Which, ChatCompletionRequest, Architecture

tools = [
    json.dumps(
        {
            "type": "function",
            "function": {
                "name": "run_python",
                "description": "Run some Python code",
                "parameters": {
                    "type": "string",
                    "properties": {
                        "code": {
                            "type": "string",
                            "description": "The Python code to evaluate. The return value whatever was printed out from `print`.",
                        },
                    },
                    "required": ["code"],
                },
            },
        }
    )
]


def custom_serializer(obj):
    try:
        res = json.dumps(obj)
    except Exception:
        # Handle serializing, for example, an imported module
        res = None
    return res


def run_python(code: str) -> str:
    lcls = dict()
    # No opening of files
    glbls = {"open": None}

    print(f"Running:\n```py\n{code}\n```")

    old_stdout = sys.stdout
    out = StringIO()
    sys.stdout = out
    exec(code, glbls, lcls)
    sys.stdout = old_stdout

    return out.getvalue()


functions = {
    "run_python": run_python,
}
```

--------------------------------

### Agentic Run with File Output Example

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/reference/http-api.md

Example JSON request for an agentic run that includes enabling code execution and specifying a file output.

```json
{
  "model": "default",
  "messages": [
    {"role": "user", "content": "Plot sin(x) and save as plot.png."} 
  ],
  "enable_code_execution": true,
  "files": [{"name": "plot.png", "format": "png"}]
}
```

--------------------------------

### Egg framework development and deployment commands

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/calibration_data/calibration_datav3_small.txt

Standard shell commands for installing dependencies, running the development server, and deploying the Egg application.

```shell
$ npm install
$ npm run dev
$ open http://localhost:7001/
```

```shell
$ EGG_SERVER_ENV=prod npm start
```

--------------------------------

### Install mistralrs Python SDK

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/mistralrs-pyo3/README.md

Install the mistralrs Python SDK based on your hardware. Use 'mistralrs-cuda' for NVIDIA GPUs, 'mistralrs-metal' for Apple Silicon, or 'mistralrs' for CPU-only.

```bash
pip install mistralrs-cuda   # NVIDIA GPUs
pip install mistralrs-metal  # Apple Silicon
pip install mistralrs        # CPU only
```

--------------------------------

### Initialize Plain Runner with configuration

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/python/cookbook.ipynb

Load a model using the Plain architecture with specific tokenizer and repetition penalty settings.

```python
runner = Runner(
    which=Which.Plain(
        model_id="mistralai/Mistral-7B-Instruct-v0.1",
        arch=Architecture.Mistral,
        tokenizer_json=None,
        repeat_last_n=64,
    )
)
```

--------------------------------

### Start mistralrs as an MCP server

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/agents/expose-as-mcp.md

Use this command to start the mistralrs server with MCP enabled on a specific port. The MCP endpoint uses JSON-RPC 2.0 over HTTP.

```bash
mistralrs serve --mcp-port 3030 -m Qwen/Qwen3-4B
```

--------------------------------

### Build with Metal Support (macOS)

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/CLAUDE.md

Build the project with Metal support for macOS.

```bash
# With Metal support (macOS)
cargo build --release --features metal
```

--------------------------------

### Create MCP Configuration File (Filesystem Server)

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/MCP_QUICK_START.md

This JSON configuration enables the filesystem server for MCP. It specifies the server name, its type as a 'Process' command, and the arguments to run the filesystem server. Set 'auto_register_tools' to true to automatically register available tools.

```json
{
  "servers": [
    {
      "name": "Filesystem Tools",
      "source": {
        "type": "Process",
        "command": "npx",
        "args": ["@modelcontextprotocol/server-filesystem", "."]
      }
    }
  ],
  "auto_register_tools": true
}
```

--------------------------------

### Install mistral.rs Python Package

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/README.md

Install the mistralrs Python package. Choose the appropriate variant based on your hardware acceleration needs (CUDA, Metal, MKL, or general accelerate).

```bash
pip install mistralrs  # or mistralrs-cuda, mistralrs-metal, mistralrs-mkl, mistralrs-accelerate
```

--------------------------------

### Serve a model with the web UI

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/serve/with-web-ui.md

Run the mistral.rs server with a specified model to enable the built-in web UI. Access the UI by navigating to http://localhost:1234/ui in your browser.

```bash
mistralrs serve -m Qwen/Qwen3-4B
```

--------------------------------

### Run and Serve Text Models

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/docs/src/content/docs/guides/models/text-model-walkthroughs.md

Start with auto-detection for models unless a specific architecture is required. Use `mistralrs run` for direct execution and `mistralrs serve` to start an OpenAI-compatible server.

```bash
mistralrs run --quant 4 -m Qwen/Qwen3-4B
mistralrs serve --quant 4 -p 1234 -m Qwen/Qwen3-4B
```

--------------------------------

### Execute Tool and Get Final Response

Source: https://github.com/ericlbuehler/mistral.rs/blob/master/examples/python/tool_calling.ipynb

Executes the called tool if it's recognized, appends the tool call and its result to the message history, and sends a new request to the model to get the final answer.

```python
if tool_called.name in functions:
    args = json.loads(tool_called.arguments)
    result = functions[tool_called.name](**args)
    print(f"Called tool `{tool_called.name}`")

    messages.append(
        {
            "role": "assistant",
            "content": json.dumps({"name": tool_called.name, "parameters": args}),
        }
    )

    messages.append({"role": "tool", "content": result})

    res = runner.send_chat_completion_request(
        ChatCompletionRequest(
            model="llama-3.1",
            messages=messages,
            max_tokens=256,
            presence_penalty=1.0,
            top_p=0.1,
            temperature=0.1,
            tool_schemas=tools,
            tool_choice=ToolChoice.Auto,
        )
    )
    # print(completion.usage)
    print(res.choices[0].message.content)
```