### Install vLLM and Start Server

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/vllm.md

Install the vLLM library and start a vLLM server. Ensure the API key is set for authentication.

```shell
pip install vllm

vllm serve microsoft/Phi-3-mini-4k-instruct \
  --dtype auto \
  --api-key token-abc123
```

--------------------------------

### Install Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/models_playing_chess.md

Installs the necessary libraries: outlines, chess, transformers, accelerate, and einops. This is a prerequisite for running the example.

```python
%pip install outlines -q
%pip install chess -q
%pip install transformers accelerate einops -q
```

--------------------------------

### Install uv and Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/installation.md

Installs the uv dependency manager, creates a virtual environment, and installs Outlines using uv.

```shell
curl -LsSf https://astral.sh/uv/install.sh | sh

uv venv
source .venv/bin/activate
uv pip install outlines
```

--------------------------------

### Clone Repository and Setup Virtual Environment

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/community/contribute.md

Clone the Outlines repository and set up a virtual environment using uv, venv, or conda. Activate the environment before proceeding with installations.

```shell
git clone git@github.com/YourUserName/outlines.git
cd outlines
```

```shell
uv venv
source .venv/bin/activate
alias pip="uv pip"
```

```shell
python -m venv .venv
source .venv/bin/activate
```

```shell
conda env create -f environment.yml
```

--------------------------------

### Install Documentation Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/community/contribute.md

Install the necessary dependencies for working with and building the project documentation.

```shell
pip install -r requirements-doc.txt
```

--------------------------------

### Install Optional Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/community/contribute.md

Install optional dependencies for specific backends, such as vLLM. Refer to the installation guide for a full list of supported optional dependencies.

```shell
pip install "'.[vllm]'"
```

--------------------------------

### Set up Python virtual environment

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/deploy-using-modal.md

Installs modal and outlines in a virtual environment. Activate the environment before installing packages.

```shell
python -m venv venv
source venv/bin/activate
pip install modal outlines
```

--------------------------------

### Install Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/vlm.md

Install Outlines, transformers, and torch for vision-language model integration.

```shell
pip install outlines transformers torch pillow
```

--------------------------------

### Install Outlines with pip

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/getting_started.md

Use pip for installing Outlines with transformer support. This is the classic installation method.

```shell
pip install 'outlines[transformers]'
```

--------------------------------

### Install Outlines with uv

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/getting_started.md

Use uv for installing Outlines with transformer support. Ensure uv is installed separately.

```shell
uv pip install 'outlines[transformers]'
```

--------------------------------

### Install SGLang and Launch Server

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/sglang.md

Install SGLang with all dependencies and launch a server instance. Ensure the `--grammar-backend outlines` flag is used if you want Outlines to handle structured generation.

```shell
pip install "sglang[all]"

python -m sglang.launch_server \
  --model-path NousResearch/Meta-Llama-3-8B-Instruct \
  --host 0.0.0.0 \
  --port 30000
```

--------------------------------

### Install Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/README.md

Install the Outlines library using pip. This is the first step to using Outlines for structured generation.

```shell
pip install outlines
```

--------------------------------

### Install Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/deploy-using-bentoml.md

Install necessary libraries for BentoML and model serving. It's recommended to do this within a virtual environment.

```shell
pip install -r requirements.txt
```

--------------------------------

### Install Modal Client

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/deploy-using-modal.md

Install the Modal client library using pip. This is a prerequisite for using Modal's cloud features.

```shell
pip install modal
```

--------------------------------

### Install Dependencies and Pre-commit Hooks

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/community/contribute.md

Install the project dependencies in editable mode, including test dependencies. Set up pre-commit hooks for code style checks. Optionally install GPU test dependencies if a GPU is available.

```shell
pip install -e ".[test]"
pre-commit install
```

```shell
pip install -e ".[test-gpu]"
```

--------------------------------

### Install Outlines from main branch

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/installation.md

Installs the latest version of Outlines directly from its GitHub main branch.

```shell
pip install git+https://github.com/dottxt-ai/outlines.git@main
```

--------------------------------

### Install Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/fastapi_vllm_deployment.md

Install the necessary Python packages for building the FastAPI application.

```shell
pip install fastapi uvicorn outlines openai pydantic
```

--------------------------------

### Few-Shot Examples for Dating Profile Generation

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/dating_profiles.md

Provides a list of `Example` objects, each containing a client's description and a corresponding `DatingProfile`. These serve as few-shot examples for the language model.

```python
samples: list[Example] = [
    Example(
        description="I'm an author and former professional soccer player living in Seattle who publishes popular fiction books. A typical day for me starts by hanging out with my cat, drinking a coffee, and reading as much as I can in a few hours. Then, I'll prepare a quick smoothie before starting to write for a few hours, take a break with soccer or running a few miles, and finally meet friends for dinner at a new, hip restaurant in the evening. Sometimes we go axe-throwing afterwards, or play poker, or watch a comedy show, or visit a dive bar. On my vacations, I travel extensively to countries South America, Europe, and Asia, with the goal of visiting them all!",
        profile=DatingProfile(
            bio="Adventurer, dreamer, author, and soccer enthusiast. Life’s too short to waste time so I make the most of each day by exploring new places and playing with my friends on the pitch. What’s your favorite way to get out and have fun?",
            job="Famous Soccer Player -> Famous Author",
            interests=["Soccer", "Travel", "Friends", "Books", "Fluffy Animals"],
            qna1=QuestionAnswer(
                question=QuestionChoice.B, answer="swim in all seven oceans!"
            ),
            qna2=QuestionAnswer(
                question=QuestionChoice.E,
                answer="fun-loving, adventurous, and a little bit crazy",
            ),
        ),
    ),
    Example(
        description="I run my company and build houses for a living. I'm a big fan of the outdoors and love to go hiking, camping, and fishing. I don't like video games, but do like to watch movies. My love language is home-cooked food, and I'm looking for someone who isn't afraid to get their hands dirty.",
        profile=DatingProfile(
            bio="If you're looking for a Montana man who loves to get outdoors and hunt, and who's in-tune with his masculinity then I'm your guy!",
            job="House Construction Manager / Entrepreneur",
            interests=["Hunting", "Hiking", "The outdoors", "Home-cooked food"],
            qna1=QuestionAnswer(question=QuestionChoice.A, answer="food made at home"),
            qna2=QuestionAnswer(
                question=QuestionChoice.C,
                answer="having a man in your life who can fix anything",
            ),
        ),
    ),
    Example(
        description="I run my own Youtube channel with 10M subscribers. I love working with kids, and my audience skews pretty young too. In my free time, I play Fortnite and Roblox. I'm looking for someone who is also a gamer and likes to have fun. I'm learning Japanese in my free time as well as how to cook.",
        profile=DatingProfile(
            bio="Easy on the eyes (find me on Youtube!) and great with kids. What more do you need?",
            job="Youtuber 10M+ subscribers",
            interests=["Kids", "Gaming", "Japanese"],
            qna1=QuestionAnswer(question=QuestionChoice.D, answer="anime and gaming!"),
            qna2=QuestionAnswer(question=QuestionChoice.F, answer="Fortnite, gg ez"),
        ),
    ),
]
```

--------------------------------

### Access Specific Example Data

Source: https://github.com/dottxt-ai/outlines/blob/main/examples/simulation_based_inference.ipynb

This snippet shows how to access and display the details of a specific example from the `example_set` by its index. This is useful for inspecting the content of selected examples.

```python
example_set[0]
```

```python
example_set[1]
```

```python
example_set[2]
```

```python
example_set[6]
```

```python
example_set[9]
```

--------------------------------

### Install Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/receipt-digitization.md

Install the necessary Python packages for Outlines, PyTorch, Transformers, Pillow, and Rich.

```shell
pip install outlines torch==2.4.0 transformers accelerate pillow rich
```

--------------------------------

### Setup Modal Token

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/deploy-using-modal.md

Run this command to obtain and set up your Modal authentication token. This is required for authenticating with the Modal service.

```shell
modal setup
```

--------------------------------

### Example of a Generated Prompt

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/receipt-digitization.md

This is an example of the final prompt structure sent to the model, showing how system, user, image, and assistant roles are demarcated, along with the embedded instructions and image placeholders.

```text
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
<|vision_start|><|image_pad|><|vision_end|>
You are an expert at extracting information from receipts.
Please extract the information from the receipt. Be as detailed as
possible -- missing or misreporting information is a crime.

Return the information in the following JSON schema:

<JSON SCHEMA OMITTED>
<|im_end|>
<|im_start|>assistant

```

--------------------------------

### Load and use template from file for few-shot learning

Source: https://github.com/dottxt-ai/outlines/blob/main/README.md

Load a template from a file and use it with examples for few-shot learning. This allows for more complex prompt engineering by providing context and examples.

```python
example_template = outlines.Template.from_file("templates/few_shot.txt")

# Use with examples for few-shot learning
examples = [
    ("The food was cold", "Negative"),
    ("The staff was friendly", "Positive")
]
few_shot_prompt = example_template(examples=examples, query="Service was slow")
print(few_shot_prompt)
```

--------------------------------

### Initialize MLXLM Model

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/mlxlm.md

Use `outlines.from_mlxlm` with the output of `mlx_lm.load` to create a model instance. Ensure `mlx` and `mlx-lm` libraries are installed.

```python
import outlines
import mlx_lm

# Create the model
model = outlines.from_mlxlm(
    *mlx_lm.load("mlx-community/TinyLlama-1.1B-Chat-v1.0-4bit")
)
```

--------------------------------

### Example PGN Output

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/models_playing_chess.md

An example of the game's progress in Portable Game Notation (PGN) format, showing the sequence of moves made.

```pgn
e4 e5 1.Nf3 Ne7 3.b4 Nf5 5.Nc3 Ne7 7.Bb5 a6 9.Na4 b6 11.c3 Nec6 13.c4 a5 15.d4 Qg5 17.Nd2 Bb7 19.dxe5
```

--------------------------------

### Install Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/read-pdfs.md

Install necessary Python libraries including outlines, pillow, transformers, torch, and pdf2image. The 'rich' library is optional for enhanced output formatting.

```shell
pip install outlines pillow transformers torch==2.4.0 pdf2image

# Optional, but makes the output look nicer
pip install rich
```

--------------------------------

### Start vLLM Server

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/fastapi_vllm_deployment.md

Command to start a vLLM server with a specified model. This is a prerequisite for running the FastAPI application if vLLM is used as the backend.

```shell
vllm serve Qwen/Qwen2.5-VL-7B-Instruct
```

--------------------------------

### Initialize Chess Board

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/models_playing_chess.md

Initializes a standard chessboard using the `chess` library. The board is set to its starting position.

```python
import chess

board = chess.Board("rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1")
```

--------------------------------

### Load Qwen2-VL Model

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/read-pdfs.md

Configure and load the Qwen2-VL-7B-Instruct model and its processor using Hugging Face's transformers library. This setup is for using the Qwen2-VL model.

```python
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
model_name = "Qwen/Qwen2-VL-7B-Instruct"
model_class = Qwen2VLForConditionalGeneration
processor_class = AutoProcessor
```

--------------------------------

### OpenAI-Compatible Setup with Custom Headers

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/openai_compatible.md

Some providers require custom headers for authentication or specific functionalities. Pass these using the `default_headers` argument when initializing the OpenAI client.

```python
import openai
import outlines

# Some providers need custom headers
client = openai.OpenAI(
    base_url="https://api.your-provider.com/v1",
    api_key="your-api-key",
    default_headers={"Custom-Header": "value"}
)

model = outlines.from_openai(client, "provider-model-name")
```

--------------------------------

### Start SGLang Server

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/fastapi_vllm_deployment.md

Command to launch an SGLang server with a specified model and port. This is used when SGLang is chosen as the inference backend.

```shell
python -m sglang.launch_server \
    --model-path meta-llama/Llama-2-7b-chat-hf \
    --port 30000
```

--------------------------------

### Serve BentoML Service Locally

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/deploy-using-bentoml.md

This shell command starts a local BentoML server. Ensure you are in the directory containing your 'service.py' file.

```shell
bentoml serve .
```

--------------------------------

### Define Unstructured Prompt

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/structured_generation_workflow.md

A simple prompt to guide the LLM in generating a phone number. This prompt is used in both unstructured and structured generation examples.

```python
prompt_phone = """
    Please generate a realistic phone number for Washington State in the following format
    (555) 555-5555
"""
```

--------------------------------

### Start TGI Server with Docker

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/fastapi_vllm_deployment.md

This shell command starts a Text Generation Inference server using Docker. It maps port 8080 on your host to port 80 in the container and specifies the model to be used (`meta-llama/Llama-2-7b-chat-hf`). Ensure you have Docker installed and the necessary GPU drivers.

```shell
docker run --gpus all -p 8080:80 \
    ghcr.io/huggingface/text-generation-inference:latest \
    --model-id meta-llama/Llama-2-7b-chat-hf
```

--------------------------------

### Prepare example and training sets

Source: https://github.com/dottxt-ai/outlines/blob/main/examples/simulation_based_inference.ipynb

Parses the downloaded dataset, creating an 'example_set' of 10 problems for sampling and a 'train_set' of 500 problems for inference. Extracts the final numerical answer from each problem.

```python
example_set = []
for _ in range(10):
    line = json.loads(next(lines))
    answer = re.findall(r"\d+", line["answer"])[-1]
    example_set.append({"question": line["question"], "answer": answer})

train_set = []
for _ in range(500):
    line = json.loads(next(lines))
    answer = re.findall(r"\d+", line["answer"])[-1]
    train_set.append({"question": line["question"], "answer": answer})
```

--------------------------------

### Async Structured Generation Example

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/vllm.md

This snippet shows how to generate a structured user profile asynchronously using Outlines with a vLLM backend. Ensure you have the necessary libraries (openai, outlines, pydantic) installed and a vLLM server running.

```python
import asyncio
import openai
import outlines
from pydantic import BaseModel

class User(BaseModel):
    name: str
    email: str
    age: int

async def generate_user():
    async_client = openai.AsyncOpenAI(base_url="http://0.0.0.0:8000/v1", api_key="token-abc123")
    async_model = outlines.from_vllm(async_client, "microsoft/Phi-3-mini-4k-instruct")

    result = await async_model("Generate a random user profile.", output_type=User)
    user = User.model_validate_json(result)
    print(f"Name: {user.name}, Email: {user.email}, Age: {user.age}")

asyncio.run(generate_user())
```

--------------------------------

### Basic OpenAI-Compatible Setup

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/openai_compatible.md

Configure the OpenAI client with a generic base URL and API key for a compatible provider. This is the most straightforward way to integrate with services that mimic the OpenAI API.

```python
import openai
import outlines

# Generic OpenAI-compatible setup
client = openai.OpenAI(
    base_url="https://api.your-provider.com/v1",
    api_key="your-api-key"
)

model = outlines.from_openai(client, "provider-model-name")
```

--------------------------------

### Define Prompt Template for Order Extraction

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/extraction.md

Creates a prompt template using Outlines' `Template` class. This template guides the LLM to extract pizza name and quantity from customer orders, providing an example and specifying JSON output format.

```python
from outlines import Template

take_order = Template.from_string(
    """You are the owner of a pizza parlor. Customers \ 
    send you orders from which you need to extract:

    1. The pizza that is ordered
    2. The number of pizzas

    # EXAMPLE

    ORDER: I would like one Margherita pizza
    RESULT: {"pizza": "Margherita", "number": 1}

    # OUTPUT INSTRUCTIONS

    Answer in valid JSON. Here are the different objects relevant for the output:

    Order:
        pizza (str): name of the pizza
        number (int): number of pizzas

    Return a valid JSON of type "Order"

    # OUTPUT

    ORDER: {{ order }}
    RESULT: """)
```

--------------------------------

### Build and Serve Documentation Locally

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/community/contribute.md

Build the documentation and serve it locally using mkdocs. The documentation will update automatically as changes are made.

```shell
mkdocs serve
```

--------------------------------

### Create and Use Application with Template

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/utility/application.md

Demonstrates creating an Application instance using a Template object and then calling it with a model and variables. Ensure necessary libraries like 'typing', 'transformers', and 'outlines' are imported.

```python
from typing import Literal
import transformers
from outlines import Application, Template, from_transformers

# Create a template
template_str = "Is {{ name }} a boy or a girl name?"
template = Template.from_string(template_str)

# Create a model
model = from_transformers(
    transformers.AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct"),
    transformers.AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
)

# Create the application and call it to generate text
application = Application(template, Literal["boy", "girl"])
response = application(model, {"name": "Alice"}, max_new_tokens=10)

print(response) # "girl"
```

--------------------------------

### Install and Login to Cerebrium

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/deploy-using-cerebrium.md

Install the Cerebrium package and authenticate your account to begin.

```shell
pip install cerebrium
cerebrium login
```

--------------------------------

### Sample Completions with gpt-4o-mini using Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/examples/sampling.ipynb

This snippet shows how to generate multiple samples from the gpt-4o-mini model using the Outlines library for a given question. It requires setting up an OpenAI client and defining the prompt.

```python
model = outlines.from_openai(openai.OpenAI(), "gpt-4o")

question = "When I was 6, my sister was half the age of my brother. When I was 14, my sister was 3 years younger than my brother. Now I'm 70, how old is my sister now?"
prompt = gsm8k_prompt(question)
answers = model(prompt, n=20, max_tokens=512)
```

--------------------------------

### Generate Training Examples for LLM Prompting

Source: https://github.com/dottxt-ai/outlines/blob/main/examples/simulation_based_inference.ipynb

This function selects random examples from a dataset and generates a prompt for an LLM. It then analyzes the raw answers to identify which examples were most effective in eliciting the correct answer.

```python
def one_train_example(problem, example_set):
    example_ids = random.choices(range(0, len(example_set)), k=5)
    examples = [example_set[i] for i in example_ids]
    prompt = few_shots(question=problem["question"], examples=examples)
    answers_raw = model(prompt, samples=20)

    samples = []
    for answer_raw in answers_raw:
        try:
            answer = re.findall(r"\d+", answer_raw)[-1]
            if answer == problem["answer"]:
                samples += example_ids
            else:
                continue
        except IndexError:
            pass

    return samples
```

--------------------------------

### Initialize Ollama Sync and Async Models

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/ollama.md

Demonstrates how to create synchronous and asynchronous Ollama model instances using the `outlines.from_ollama` function. Requires an `ollama.Client` or `ollama.AsyncClient` instance and a model name.

```python
import ollama
import outlines

# Create the client or async client
client = ollama.Client()
async_client = ollama.AsyncClient()

# Create a sync model
model = outlines.from_ollama(
    client,
    "qwen2.5vl:3b",
)

# Create an async model
model = outlines.from_ollama(
    async_client,
    "qwen2.5vl:3b",
)
```

--------------------------------

### Install llama-cpp-python

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/chain_of_thought.md

Install the llama-cpp-python library, which is required for using llama.cpp models with Outlines.

```shell
pip install llama-cpp-python
```

--------------------------------

### Initialize SGLang Models (Sync and Async)

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/sglang.md

Create synchronous and asynchronous OpenAI clients pointing to the SGLang server. Then, use `outlines.from_sglang` to create corresponding Outlines model instances. Check the type of the created model.

```python
import openai
import outlines

# Create the OpenAI client
sync_openai_client = openai.OpenAI(base_url="http://localhost:11434")
async_openai_client = openai.AsyncOpenAI(base_url="http://localhost:11434")

# Create a sync model
sync_model = outlines.from_sglang(sync_openai_client)
print(type(sync_model)) # <class 'outlines.models.sglang.SGLang'>

# Create an async model
async_model = outlines.from_sglang(async_openai_client)
print(type(async_model)) # <class 'outlines.models.sglang.AsyncSGLang'>
```

--------------------------------

### Visualize Example Sampling Counts with Matplotlib

Source: https://github.com/dottxt-ai/outlines/blob/main/examples/simulation_based_inference.ipynb

This code visualizes the distribution of how often each example was sampled. It uses `numpy.unique` to count occurrences and `matplotlib.pylab` to create a bar chart, which helps in identifying the most effective examples.

```python
import numpy as np
import matplotlib.pylab as plt

example_ids, counts = np.unique(samples, return_counts=True)

fig = plt.figure(figsize=(12, 8))
ax = fig.add_subplot(111)
ax.bar(example_ids, counts)

ax.spines[["top", "right"]].set_visible(False)

ax.set_xticks(range(10))
ax.set_xlabel("Example #")
ax.set_ylabel("Counts")
```

--------------------------------

### Initialize Anthropic Client and Outlines Model

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/anthropic.md

Instantiate an Anthropic client and then use `outlines.from_anthropic` to create an Outlines model. Ensure your `ANTHROPIC_API_KEY` environment variable is set or passed to the `Anthropic` constructor.

```python
from anthropic import Anthropic
import outlines

# Create the Anthropic client
client = Anthropic()

# Create the model
model = outlines.from_anthropic(
    client,
    "claude-3-5-sonnet-latest"
)
```

--------------------------------

### Initialize vLLM Model with Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/getting_started.md

Create an Outlines model using a vLLM server. Requires a separate vLLM server running and an OpenAI client configured for its endpoint.

```python
import outlines
from openai import OpenAI

# You must have a separate vLLM server running
# Create an OpenAI client with the base URL of the VLLM server
openai_client = OpenAI(base_url="http://localhost:11434/v1")

# Create an Outlines model
model = outlines.from_vllm(openai_client, "microsoft/Phi-3-mini-4k-instruct")
```

--------------------------------

### Initialize Mistral Client and Sync/Async Models

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/mistral.md

Instantiate a Mistral client and create both synchronous and asynchronous Outlines model instances. Ensure the MISTRAL_API_KEY environment variable is set or passed to the Mistral client.

```python
import mistralai
import outlines

# Create the Mistral client
client = mistralai.Mistral()

# Create a sync model
model = outlines.from_mistral(
    client,
    "mistral-large-latest"
)

# Create aa async model
model = outlines.from_mistral(
    client,
    "mistral-large-latest",
    True
)
```

--------------------------------

### Setup Outlines Model and JSON Schema

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/deploy-using-cerebrium.md

Initialize an Outlines generator with a pre-trained Transformers model and a JSON schema for structured output. The model is downloaded on first deploy and cached for subsequent calls.

```python
import outlines
import transformers
from outlines.types import JsonSchema


model = outlines.from_transformers(
    transformers.AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct"),
    transformers.AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
)

schema = """
{
    "title": "Character",
    "type": "object",
    "properties": {
        "name": {
            "title": "Name",
            "maxLength": 10,
            "type": "string"
        },
        "age": {
            "title": "Age",
            "type": "integer"
        },
        "armor": {"$ref": "#/definitions/Armor"},
        "weapon": {"$ref": "#/definitions/Weapon"},
        "strength": {
            "title": "Strength",
            "type": "integer"
        }
    },
    "required": ["name", "age", "armor", "weapon", "strength"],
    "definitions": {
        "Armor": {
            "title": "Armor",
            "description": "An enumeration.",
            "enum": ["leather", "chainmail", "plate"],
            "type": "string"
        },
        "Weapon": {
            "title": "Weapon",
            "description": "An enumeration.",
            "enum": ["sword", "axe", "mace", "spear", "bow", "crossbow"],
            "type": "string"
        }
    }
}
"""

generator = outlines.Generator(model, JsonSchema(schema))
```

--------------------------------

### Initialize OpenAI Model (v1)

Source: https://github.com/dottxt-ai/outlines/blob/main/outlines/release_note.md

Use `outlines.from_openai` to initialize OpenAI models in v1. Pass the OpenAI client instance and the model name directly.

```python
from openai import OpenAI
from pydantic import BaseModel
from outlines import from_openai

class Character(BaseModel):
	name: str

model = from_openai(OpenAI(), "gpt-4o")
result = model("Create a character", Character)
```

--------------------------------

### Define Example Data Structure

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/dating_profiles.md

A dataclass is used to structure the input description and the desired dating profile output for few-shot examples.

```python
from dataclasses import dataclass

@dataclass
class Example:
    description: str
    profile: DatingProfile
```

--------------------------------

### Initialize Dottxt Client (Synchronous)

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/dottxt.md

Instantiate a synchronous Dottxt client and an Outlines model using the client. Ensure the API key is provided either as an argument or an environment variable.

```python
from dottxt.client import DotTxt
import outlines

client = DotTxt(api_key="...")
model = outlines.from_dottxt(client, "dottxt/dottxt-v1-alpha")
```

--------------------------------

### Example JSON Dating Profile 2

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/dating_profiles.md

A second example of a generated JSON dating profile, showcasing variations in interests and Q&A answers.

```json
{
    "bio": "I’m a sexy lawyer with time on my hands. I love to game and
    play ping pong, but the real reason you should swipe to the right
    is because I look great in a suit. Who doesn’t love a man in a
    suit? Just saying. Send me a message if you think it’s time to take
    your dating life to the next level.",
    "job": "Lawyer",
    "interests":
    [
        "Gaming",
        "Ping Pong",
        "Tailored Suits",
        "Weddings",
        "Streaming Services"
    ],
    "qna1":
    {
        "question": "The first item on my bucket list is",
        "answer": "simulate space but stay alive for as long as possible"
    },
    "qna2":
    {
        "question": "People would describe me as",
        "answer": "easy-going, a little nerdy but with a mature essence"
    }
}
```

--------------------------------

### Example JSON Dating Profile 1

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/dating_profiles.md

An example of a generated JSON dating profile, including bio, job, interests, and Q&A sections.

```json
{
    "bio": "I'm an ambitious lawyer with a casual and fashionable style. I love
    games and sports, but my true passion is preparing refreshing cocktails at
    home and dressing to the nines at weddings. I'm currently looking for a woman
    to show a good time to and get a kiss on the opulent suit I just had made.
    Send resume to this inbox.",
    "job": "Lawyer",
    "interests":
    [
        "Stylish guys",
        "Gaming",
        "Ping pong",
        "Cocktails",
        "Weddings"
    ],
    "qna1":
    {
        "question": "The first item on my bucket list is",
        "answer": "be married and have a family."
    },
    "qna2":
    {
        "question": "People would describe me as",
        "answer": "charming, stylish, and funny."
    }
}
```

--------------------------------

### Initialize llama.cpp Model with Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/getting_started.md

Create an Outlines model using llama.cpp. The model will be downloaded from the HuggingFace hub if not found locally.

```python
import outlines
from llama_cpp import Llama

# Model to use, it will be downloaded from the HuggingFace hub
repo_id = "TheBloke/Llama-2-13B-chat-GGUF"
file_name = "llama-2-13b-chat.Q4_K_M.gguf"

# Create a Llama.cpp model
llama_cpp_model = Llama.from_pretrained(repo_id, file_name)

# Create an Outlines model
model = outlines.from_llamacpp(llama_cpp_model)
```

--------------------------------

### Install Outlines and Dependencies

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/earnings-reports.md

Install the necessary Python packages for Outlines, Pandas, Transformers, and PyTorch. It's recommended to use PyTorch version 2.4.0.

```shell
pip install outlines pandas transformers torch==2.4.0 accelerate
```

--------------------------------

### Initialize Gemini Model with Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/getting_started.md

Create an Outlines model using Google's Gemini. Requires a GenerativeModel client instance.

```python
import outlines
from google.generativeai import GenerativeModel

# Create a Gemini client
gemini_client = GenerativeModel()

# Create an Outlines model
model = outlines.from_gemini(gemini_client, "gemini-1-5-flash")
```

--------------------------------

### Initialize OpenAI Model with Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/getting_started.md

Create an Outlines model using the OpenAI API. Requires an OpenAI client instance.

```python
import outlines
from openai import OpenAI

# Create an OpenAI client instance
openai_client = OpenAI()

# Create an Outlines model
model = outlines.from_openai(openai_client, "gpt-4o")
```

--------------------------------

### Validate Regex Against Example

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/structured_generation_workflow.md

Uses Python's `re` module to check if the defined regex pattern matches the example phone number. This is a crucial step before structured generation.

```python
import re

re.match(phone_regex_1.pattern, phone_number)
# <re.Match object; span=(0, 14), match='(206) 386-4636'>
```

--------------------------------

### Process Logits with Guide

Source: https://github.com/dottxt-ai/outlines/blob/main/llm.txt

This function processes logits by masking invalid tokens based on the current FSM state and a guide. It's used in custom logits processors.

```python
# Simplified logits processing
def process_logits(logits, current_state, guide):
    valid_tokens = guide.get_valid_tokens(current_state)
    mask = torch.full_like(logits, -float('inf'))
    mask[valid_tokens] = 0
    return logits + mask
```

--------------------------------

### Validate Improved Regex Against Example

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/structured_generation_workflow.md

Validates the more sophisticated regex against the real example phone number to ensure it still matches. This confirms the refined structure is compatible with actual data.

```python
re.match(phone_regex_2.pattern, phone_number)[0] == phone_number
```

--------------------------------

### OpenAI Model Initialization Comparison (v0 vs v1)

Source: https://github.com/dottxt-ai/outlines/blob/main/outlines/release_note.md

In v1, `outlines.from_openai` replaces direct initialization with `OpenAI` and `OpenAIConfig`. Inference arguments are now passed during model calls.

```python
# v0
from outlines.models.openai import OpenAI, OpenAIConfig
from openai import OpenAI as OpenAIClient

model = OpenAI(
	OpenAIClient(),
	OpenAIConfig(model="gpt-4o", stop=["."])
)
```

```python
# v1
import outlines
from openai import OpenAI

model = outlines.from_openai(OpenAIClient(), "gpt-4o")
```

--------------------------------

### Define SimToM Prompt Templates with Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/simtom.md

Load prompt templates from files using Outlines' Template class. Ensure the specified file paths are correct.

```python
from outlines import Template

perspective_taking = Template.from_file("prompt_templates/simtom_prospective_taking.txt")
simulation = Template.from_file("prompt_templates/simtom_simulation.txt")
```

--------------------------------

### Initialize OpenAI Models in Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/openai.md

Instantiate synchronous and asynchronous OpenAI models using the `outlines.from_openai` function. Ensure your OpenAI API key is set as an environment variable or passed during client instantiation.

```python
import outlines
import openai

# Create the client or async client
client = openai.OpenAI()
async_client = openai.AsyncOpenAI()

# Create a sync model
model = outlines.from_openai(
    client,
    "gpt-4o"
)

# Create aa async model
model = outlines.from_openai(
    async_client,
    "gpt-4o"
)
```

--------------------------------

### Example Structured JSON Output

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/guide/vlm.md

This is an example of the structured JSON output generated by the vision multi-modal model, containing tags, a short caption, and a dense caption describing the image content.

```json
{"tags_list": [
  {
    "tag": "astronaut",
    "category": <TagType.ENTITY: "Entity">
    "confidence": 0.99
  },
  {"tag": "moon", "category": <TagType.ENTITY: "Entity">, "confidence": 0.98},
  {
    "tag": "space suit",
    "category": <TagType.ATTRIBUTE: "Attribute">
    "confidence": 0.97
  },
  {
    "tag": "lunar module",
    "category": <TagType.ENTITY: "Entity">
    "confidence": 0.95
  },
  {
    "tag": "shadow of astronaut",
    "category": <TagType.COMPOSITION: "Composition">
    "confidence": 0.95
  },
  {
    "tag": "footprints in moon dust",
    "category": <TagType.CONTEXTUAL: "Contextual">
    "confidence": 0.93
  },
  {
    "tag": "low angle shot",
    "category": <TagType.TECHNICAL: "Technical">
    "confidence": 0.92
  },
  {
    "tag": "human first steps on the moon",
    "category": <TagType.SEMANTIC: "Semantic">
    "confidence": 0.95
  }],
  "short_caption": "First man on the Moon",
  "dense_caption": "The figure clad in a pristine white space suit, emblazoned with the American flag, stands powerfully on the moon's desolate and rocky surface. The lunar module, a workhorse of space engineering, looms in the background, its metallic legs sinking slightly into the dust where footprints and tracks from the mission's journey are clearly visible. The photograph captures the astronaut from a low angle, emphasizing his imposing presence against the desolate lunar backdrop. The stark contrast between the blacks and whiteslicks of lost light and shadow adds dramatic depth to this seminal moment in human achievement."
}
```

--------------------------------

### Initialize Gemini Model with Outlines

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/features/models/gemini.md

Instantiate a Gemini model using the `outlines.from_gemini` function. Ensure you have a `google.genai.Client` instance and specify the desired model name.

```python
import outlines
from google import genai

# Create the client
client = genai.Client()

# Create the model
model = outlines.from_gemini(
    client,
    "gemini-1.5-flash-latest"
)
```

--------------------------------

### Define Real Example Phone Number

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/structured_generation_workflow.md

A concrete example of a real phone number used for validating generated structures. This helps ensure the regex or schema accurately models the desired output.

```python
phone_number = "(206) 386-4636"
```

--------------------------------

### Outlines Application for Model Flexibility

Source: https://github.com/dottxt-ai/outlines/blob/main/outlines/release_note.md

Use the Application class to abstract prompt templates and output types, allowing easy switching between models. Initialize with a template and an output type, then call with model and variables.

```python
from pydantic import BaseModel
from outlines import Application, Template

class Character(BaseModel):
	name: str

template = Template.from_string("Create a {{ gender }} character.")
app = Application(template, Character)
response = app(model, {"gender": "female"})
```

--------------------------------

### Example Output with Roast Field

Source: https://github.com/dottxt-ai/outlines/blob/main/docs/examples/receipt-digitization.md

This example shows the output when the `roast` field is included in the `ReceiptSummary` model. The model generates a comment based on the receipt's contents, such as the number of items and bag fees.

```json
{
    ...
    "roast": "You must be a fan of Trader Joe's because you bought enough
    items to fill a small grocery bag and still had to pay for a bag fee.
    Maybe you should start using reusable bags to save some money and the
    environment."
}
```