### Serve Pixtral with vLLM

Source: https://mistral.ai/news/pixtral-12b

Example of initializing the vLLM library to serve the Pixtral model. This setup is recommended for higher serving throughput.

```python
from vllm import LLM
from vllm.sampling_params import SamplingParams

model_name = "mistralai/Pixtral-12B-2409"

sampling_params = SamplingParams(max_tokens=8192)

ll
```

--------------------------------

### Cloudflare Workers AI - Mistral Model

Source: https://context7_llms

Guide to deploying and running Mistral models on Cloudflare Workers AI, including setup instructions and API examples.

```APIDOC
## Deploy with Cloudflare Workers AI

### Description
Cloudflare Workers AI allows developers to run LLM models on Cloudflare's global network. This section provides setup details and examples for using Mistral models.

### Method
POST

### Endpoint
`https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1`

### Parameters

#### Path Parameters
- **ACCOUNT_ID** (string) - Required - Your Cloudflare Account ID.

#### Query Parameters
None

#### Request Body
- **messages** (array) - Required - An array of message objects.
  - **role** (string) - Required - The role of the message sender (e.g., 'user').
  - **content** (string) - Required - The content of the message.

### Request Example (cURL)
```bash
curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1 \
  -X POST \
  -H "Authorization: Bearer {API_TOKEN}" \
  -d '{ "messages": [{ "role": "user", "content": "[INST] 2 + 2 ? [/INST]" }]}'
```

### Request Example (TypeScript)
```typescript
async function run(model: string, prompt: string) {
  const messages = [
    { role: "user", content: prompt },
  ];

  const response = await fetch(
    `https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/${model}`,
    {
      headers: { Authorization: "Bearer {API_TOKEN}" },
      method: "POST",
      body: JSON.stringify({ messages }),
    }
  );
  const result = await response.json();
  return result;
}

run("@cf/mistral/mistral-7b-instruct-v0.1", "[INST] 2 + 2 ? [/INST]").then(
  (response) => {
    console.log(JSON.stringify(response));
  }
);
```

### Response
#### Success Response
- **result** (object) - The response from the model.
  - **response** (string) - The model's generated text.
- **success** (boolean) - Indicates if the request was successful.
- **errors** (array) - An array of errors, if any.
- **messages** (array) - An array of messages, if any.

#### Response Example (Python)
```python
{'result': {'response': '2 + 2 = 4.'}, 'success': True, 'errors': [], 'messages': []}
```
```

--------------------------------

### Install and Import Packages

Source: https://docs.mistral.ai/guides/rag

Installs the necessary Mistral AI and FAISS libraries and imports required modules. Ensure you have your API key ready.

```python
from mistralai.client import Mistral
import requests
import numpy as np
import faiss
import os
from getpass import getpass

api_key= getpass("Type your API Key")
client = Mistral(api_key=api_key)
```

--------------------------------

### cURL - Start Conversation

Source: https://context7_llms

Example cURL command to start a conversation with an agent.

```APIDOC
## POST /v1/conversations

### Description
Starts a new conversation with an agent using cURL.

### Method
POST

### Endpoint
https://api.mistral.ai/v1/conversations

### Parameters
#### Request Body
- **inputs** (array) - Required - An array of input messages for the conversation.
  - **role** (string) - Required - The role of the message sender (e.g., "user").
  - **content** (string) - Required - The content of the message.
  - **object** (string) - Required - The object type, should be "entry".
  - **type** (string) - Required - The type of the input, should be "message.input".
- **stream** (boolean) - Optional - Whether to stream the response.
- **agent_id** (string) - Required - The ID of the agent to use.

### Request Example
```bash
curl --location "https://api.mistral.ai/v1/conversations" \
     --header 'Content-Type: application/json' \
     --header 'Accept: application/json' \
     --header "Authorization: Bearer $MISTRAL_API_KEY" \
     --data '{ \
     "inputs": [ \
         { \
             "role": "user", \
             "content": "Whats the current 2025 real interest rate?", \
             "object": "entry", \
             "type": "message.input" \
         } \
     ], \
     "stream": false, \
     "agent_id": "<agent_id>" \
 }'
```
```

--------------------------------

### Initialize Mistral AI Client

Source: https://context7_llms

This snippet shows how to install the necessary libraries and initialize the Mistral AI client using an API key obtained securely.

```python
from mistralai import Mistral


from getpass import getpass

api_key= getpass("Type your API Key")
client = Mistral(api_key=api_key)
```

--------------------------------

### Example Prompting - Direct Example

Source: https://docs.mistral.ai/guides/prompting_capabilities

This shows a direct example within a prompt for a language detection task. It includes input and expected output to guide the model.

```text
[...]\n# Examples\nInput: Hello, how are you?\nOutput: {\
```

--------------------------------

### Start Conversation with Document Library (Python)

Source: https://docs.mistral.ai/llms-full.txt

Use this Python snippet to start a conversation with a document library agent. Ensure you have the Mistral AI client library installed and authenticated.

```python
response = client.beta.conversations.start(
    agent_id=image_agent.id,
    inputs="How does the vision encoder for pixtral 12b work"
)
```

--------------------------------

### Initialize Mistral Client

Source: https://context7_llms

Sets up the Mistral client by importing the necessary library and initializing it with an API key. This is a prerequisite for interacting with the Mistral API.

```python
from mistralai import Mistral
```

```python
mistral_api_key = "your_api_key"
client = Mistral(api_key=mistral_api_key)
```

--------------------------------

### Create a Simple Agent (cURL)

Source: https://context7_llms

Demonstrates how to create a simple agent using a cURL command. This example shows the necessary headers and JSON payload, including the model, name, and description for the agent.

```bash
curl --location "https://api.mistral.ai/v1/agents" \
     --header 'Content-Type: application/json' \
     --header 'Accept: application/json' \
     --header "Authorization: Bearer $MISTRAL_API_KEY" \
     --data '{
     "model": "mistral-medium-latest",
     "name": "Simple Agent",
     "description": "A simple Agent with persistent state."
  }'
```

--------------------------------

### Prompting Examples for LLM Capabilities

Source: https://docs.mistral.ai/guides/prompting_capabilities

Introduces a section with prompting examples that demonstrate four different LLM prompting capabilities. This serves as a guide for users to explore advanced prompting techniques.

```text
Below we walk you through example prompts showing four different prompting capabilities:
```

--------------------------------

### Deploying with Docker

Source: https://context7_llms

Instructions for deploying Mistral AI models using Docker, including setting up the environment and running the container for different model variants.

```APIDOC
## Docker Deployment

### Description
Provides instructions to deploy vLLM models as containerized inference servers using the official Docker image. This includes setting environment variables and running Docker commands for specific Mistral models.

### Environment Setup
1. **Set HuggingFace Access Token**: Export your HuggingFace access token as an environment variable.
   ```bash
   export HF_TOKEN=your-access-token
   ```

### Docker Commands
Run the following Docker commands to start the inference server for different Mistral models:

#### Mistral NeMo
```bash
docker run --runtime nvidia --gpus all \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HUGGING_FACE_HUB_TOKEN=${HF_TOKEN}" \
    -p 8000:8000 \
    --ipc=host \
    vllm/vllm-openai:latest \
    --model mistralai/Mistral-NeMo-Instruct-2407 \
    --tokenizer_mode mistral \
    --load_format mistral \
    --config_format mistral
```

#### Mistral Small
```bash
docker run --runtime nvidia --gpus all \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HUGGING_FACE_HUB_TOKEN=${HF_TOKEN}" \
    -p 8000:8000 \
    --ipc=host \
    vllm/vllm-openai:latest \
    --model mistralai/Mistral-Small-Instruct-2409 \
    --tokenizer_mode mistral \
    --load_format mistral \
    --config_format mistral
```

#### Pixtral-12B
```bash
docker run --runtime nvidia --gpus all \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HUGGING_FACE_HUB_TOKEN=${HF_TOKEN}" \
    -p 8000:8000 \
    --ipc=host \
    vllm/vllm-openai:latest \
    --model mistralai/Pixtral-12B-2409 \
    --tokenizer_mode mistral \
    --load_format mistral \
    --config_format mistral
```

### Post-Deployment
Once the container is running, you can interact with the deployed model using the same inference code as in a standalone deployment.
```

--------------------------------

### Start a Conversation with an Agent (Python)

Source: https://docs.mistral.ai/agents/handoffs

Initiates a conversation with a specified agent using the Mistral AI client library. This example shows how to start a conversation and handle the initial response, including a function call.

```python
import type {
 FunctionResultEntry,
 MessageOutputEntry,
} from '@mistralai/mistralai/models/components/index.js';

// Then, let's start the conversation:
response = await client.beta.conversations.start({
 agentId: financeAgent.id,
 inputs:
 'Given the interest rate of the European Central Bank as of jan 2025, plot a graph of the compounded interest rate over the next 10 years',
});

let output = response.outputs[response.outputs.length - 1];

if (
 output.type === 'function.call' &&
 output.name === 'getEuropeanCentralBankInterestRate'
) {
 // Add a dummy result for the function call
 let userEntry: FunctionResultEntry = {
 toolCallId: output.toolCallId,
 result: '2.5%',
 };

 response = await client.beta.conversations.append({
 conversationId: response.conversationId,
 conversationAppendRequest: {
 inputs: [userEntry],
 },
 });
}
```

--------------------------------

### Killing a Linux Process with Mistral 7B - Instruct

Source: https://context7_llms

Demonstrates how Mistral 7B - Instruct provides a correct response to a query about killing a Linux process, contrasting with Llama 2's refusal. This highlights Mistral's ability to handle potentially sensitive but informative queries.

```text
To kill a Linux process, you can use the `kill` command followed by the process ID (PID) of the process you want to terminate. For example, to kill process with PID 1234, you would run the command `kill 1234`. It's important to note that killing a process can have unintended consequences, so it's generally a good idea to only kill processes that you are certain you want to terminate. Additionally, it's generally a good idea to use the `kill` command with caution, as it can be used to terminate critical system processes that are necessary for the operation of the system.
```

--------------------------------

### Few-Shot Prompt Example Structure

Source: https://docs.mistral.ai/guides/prompting_capabilities

This JSON structure demonstrates a few-shot prompt including system, user, and assistant roles to guide the model's responses. It shows how to provide examples within the conversation history.

```json
[
    {
        "role": "system",
        "content": "You are a language detection model. Your task is to detect the language of the given text.\n[...]"
    },
    {
        "role": "user",
        "content": "Hello, how are you?"
    },
    {
        "role": "assistant",
        "content": "{\"language_iso\": \"en\"}"
    },
    {
        "role": "user",
        "content": "Bonjour, comment allez-vous?"
    },
    {
        "role": "assistant",
        "content": "{\"language_iso\": \"fr\"}"
    }
]
```

```json
[
    {
        "role": "system",
        "content": "You are a language detection model. Your task is to detect the language of the given text.\n[...]"
    },
    {
        "role": "user",
        "content": "Hello, how are you?"
    },
    {
        "role": "assistant",
        "content": "{\"language_iso\": \"en\"}"
    },
    {
        "role": "user",
        "content": "Bonjour, comment allez-vous?"
    },
    {
        "role": "assistant",
        "content": "{\"language_iso\": \"fr\"}"
    }
]
```

--------------------------------

### Get Document Info (TypeScript)

Source: https://docs.mistral.ai/llms-full.txt

Retrieve information about a processed document in a library using TypeScript. This is the asynchronous equivalent of the Python example.

```typescript
// Get document info once processed
const processedDoc = await client.beta.libraries.documents.get({
    libraryId: newLibrary.id,
    documentId: uploadedDoc.id
});
```

--------------------------------

### Initialize Mistral Client and Model

Source: https://docs.mistral.ai/capabilities/function_calling

Sets up the Mistral API key, model name, client instance, and generation parameters for the chat application.

```python
api_key = os.environ["MISTRAL_API_API_KEY"]
model = "mistral-large-latest"
client = Mistral(api_key=api_key)
temperature = 0.1
top_p = 0.9
```