### Install Guidance

Source: https://github.com/guidance-ai/guidance/blob/main/docs/index.md

Install the Guidance library from PyPI using pip.

```default
pip install guidance
```

--------------------------------

### Launch VLLM Server

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/litellm_models.ipynb

Launch a VLLM instance with specific configurations for guided decoding and caching. Ensure VLLM is installed and a model is available.

```bash
vllm serve Qwen/Qwen3-1.7B --host 0.0.0.0 \
--port 8000 \
--reasoning-parser deepseek_r1 \
--enable-prefix-caching \
--guided-decoding-backend guidance \
--max-model-len 16384
```

--------------------------------

### Install Guidance (No CUDA)

Source: https://github.com/guidance-ai/guidance/blob/main/CONTRIBUTING.md

Installs the Guidance package in editable mode with all optional dependencies, excluding CUDA support.

```bash
python -m pip install -e .[all,test,llamacpp,transformers]
```

--------------------------------

### Install Guidance with CUDA Support

Source: https://github.com/guidance-ai/guidance/blob/main/CONTRIBUTING.md

Installs PyTorch with CUDA support and then installs the Guidance package with CUDA enabled. Ensure your system meets CUDA requirements.

```bash
conda install pytorch pytorch-cuda=12.1 -c pytorch -c nvidia
CMAKE_ARGS="-DGGML_CUDA=on" python -m pip install -e .[all,test,llamacpp,transformers]
```

--------------------------------

### Video.js Plugin Setup

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Handles the setup and triggering of plugins for a Video.js player. It ensures plugins are correctly initialized and their events are managed.

```javascript
mu=(e,t,i)=>{const s=(i?"before":"")+"pluginsetup";e.trigger(s,t),e.trigger(s+":"+t.name,t)}
```

--------------------------------

### Setup OpenAI LLM with Guidance

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/testing_lms.ipynb

Initializes the OpenAI LLM client and sets it as the default for the guidance library. Ensure you have the necessary API keys configured.

```python
import guidance
import re
import numpy as np
chatgpt = guidance.llms.OpenAI("gpt-3.5-turbo")
guidance.llm = chatgpt
```

--------------------------------

### Initialize Models

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/api_examples/library/gen.ipynb

Initializes different language models for use with the Guidance library. Requires models to be installed.

```python
from guidance import models, gen

gpt2 = models.Transformers("gpt2", device=0)
gpt3 = models.OpenAI("text-davinci-003")
gpt4 = models.OpenAI("gpt-4")
```

--------------------------------

### Example Task Execution with ChatGPT

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This code snippet demonstrates how to use the `run_task_chatgpt` function to execute a specific task. It defines the task and then calls the function to get the result.

```python
task = 'Find out what license the open source project located in ~/work/project is using.'
run_task_chatgpt(task)
```

--------------------------------

### Initialize Logging and Transformers Model

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/unstable/State Debugging.ipynb

Sets up basic logging and imports the Transformers model for use with Guidance. Ensure you have the necessary libraries installed.

```python
from guidance.models import Transformers
from guidance import gen, user, system

import logging
logging.basicConfig(format='%(asctime)s - %(levelname)s - %(message)s',
                    datefmt='%Y-%m-%d %H:%M:%S')
logging.getLogger().setLevel(logging.DEBUG)
```

--------------------------------

### Basic Text Generation with Guidance

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/prompt_boundaries_and_token_healing.ipynb

A simple example of using Guidance to generate text following a given prompt. This demonstrates the basic functionality of the `lm` and `gen` objects.

```python
lm + 'I read a book about' + gen(max_tokens=6)
```

--------------------------------

### Setup LLM and Load Model

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/react.ipynb

Imports necessary libraries and loads a LlamaCpp model. Ensure the model path and arguments are correctly configured for your environment.

```python
import math

from huggingface_hub import hf_hub_download

import guidance
from guidance import models, gen, select

repo_id = "TheBloke/Mistral-7B-Instruct-v0.2-GGUF"
filename = "mistral-7b-instruct-v0.2.Q8_0.gguf"
model_kwargs = {"verbose": True, "n_gpu_layers": -1, "n_ctx": 4096}

downloaded_file = hf_hub_download(repo_id=repo_id, filename=filename)
mistral7 = guidance.models.LlamaCpp(downloaded_file, **model_kwargs)
```

--------------------------------

### Video.js Player Initialization

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Initializes a Video.js player instance with custom settings. This example shows how to set up a player, including options like language and ID, and handles potential errors during initialization.

```javascript
class au extends fl{
constructor(e,t,i){
if(e.id=e.id||t.id||
`vjs_video_${Mo()}
`,(t=Object.assign(au.getTagSettings(e),t)).initChildren=!1,
t.createEl=!1,
t.evented=!1,
t.reportTouchActivity=!1,
!t.language){
const i=e.closest("[lang]");
i&&(t.language=i.getAttribute("lang"))
}
if(super(null,t,i),this.boundDocumentFullscreenChange_=
e=>this.documentFullscreenChange_(e),
this.boundFullWindowOnEscKey_=
e=>this.fullWindowOnEscKey(e),
this.boundUpdateStyleEl_=
e=>this.updateStyleEl_(e),
this.boundApplyInitTime_=
e=>this.applyInitTime_(e),
this.boundUpdateCurrentBreakpoint_=
e=>this.updateCurrentBreakpoint_(e),
this.boundHandleTechClick_=
e=>this.handleTechClick_(e),
this.boundHandleTechDoubleClick_=
e=>this.handleTechDoubleClick_(e),
this.boundHandleTechTouchStart_=
e=>this.handleTechTouchStart_(e),
this.boundHandleTechTouchMove_=
e=>this.handleTechTouchMove_(e),
this.boundHandleTechTouchEnd_=
e=>this.handleTechTouchEnd_(e),
this.boundHandleTechTap_=
e=>this.handleTechTap_(e),
this.boundUpdatePlayerHeightOnAudioOnlyMode_=
e=>this.updatePlayerHeightOnAudioOnlyMode_(e),
this.isFullscreen_=!1,
this.log=ua(this.id_),
this.fsApi_=ra,
this.isPosterFromTech_=!1,
this.queuedCallbacks_=[],
this.isReady_=!1,
this.hasStarted_=!1,
this.userActive_=!1,
this.debugEnabled_=!1,
this.audioOnlyMode_=!1,
this.audioPosterMode_=!1,
this.audioOnlyCache_={controlBarHeight:null,playerHeight:null,hiddenChildren:[]},
!this.options_||!this.options_.techOrder||!this.options_.techOrder.length)throw new Error("No techOrder specified. Did you overwrite videojs.options instead of just changing the properties you want to
```

--------------------------------

### MPT Model QA Attempt with Guided Generation

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This snippet shows the application of the guided generation prompt to the MPT model. It's used to test MPT's ability to follow the structured output format, comparing its performance to Vicuna.

```python
qa_guided(llm=mpt, transcript=meeting_transcript, query=query1)
```

--------------------------------

### Executing the ReAct Prompt Example

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/react.ipynb

Initializes the language model and calls the `react_prompt_example` function to execute the ReAct loop for a given question and set of tools. The result is stored in the `lm` variable.

```python
lm = mistral7
lm += react_prompt_example("What is the logarithm of Leonardo DiCaprio's age?", tools)
```

--------------------------------

### Vicuna Guided Terminal Execution

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This code demonstrates using a guided terminal with the Vicuna model. It initializes a stateful shell and then calls the guided_terminal function with the LLM and task.

```python
shell = StatefulShellOpenSource()
t = guided_terminal(llm=vicuna, task=task, shell=shell)
```

--------------------------------

### Placeholder for Open Source Model Tool Use Example

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/tool_use.ipynb

This is a placeholder for a Python code example demonstrating tool use with an open-source model. It indicates that more detailed tool definitions are necessary when the model does not have specialized support for function calls.

```python
# TODO
```

--------------------------------

### StatefulShellOpenSource Class for Guided LLM Interaction

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This class integrates a BashSession with the guided LLM prompt. It handles command execution and raises StopIteration when a 'DONE' command is encountered, facilitating a stateful interaction with open-source models.

```python
class StatefulShellOpenSource:
    def __init__(self):
        self.session = BashSession()
    def __call__(self, command):
        if 'DONE' in command:
            raise StopIteration
        output = self.session.run(command)
        return output.strip()
```

--------------------------------

### Install Guidance Stitch

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/README.md

Install the guidance-stitch package using pip. This is the primary method for adding the library to your Python environment.

```bash
pip install guidance-stitch
```

--------------------------------

### Generate Text with Guidance

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/token_healing.ipynb

Use the Guidance library to generate text from a language model. This example shows how to append generated text to a prompt.

```python
import guidance
from guidance import gen, models

gpt2 = models.Transformers("gpt2", temperature=0.8, do_sample=True)

gpt2 += "The url of Google is http:" + gen(max_tokens=5)
```

--------------------------------

### Install Dependencies for OnnxRuntime-GenAI

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/onnxruntime_models.ipynb

Install the necessary packages for using OnnxRuntime-GenAI models, including the core library and Hugging Face Hub for model downloading.

```bash
pip install -e .[onnxruntime_genai]
pip install huggingface_hub
```

--------------------------------

### Example JSON Output

Source: https://github.com/guidance-ai/guidance/blob/main/README.md

This shows the expected JSON output after running the Guidance generation code. The output adheres to the specified Pydantic schema, including data types and constraints.

```json
lm['bp']='{"systolic": 301, "diastolic": 15, "location": "arm"}'
{
    "systolic": 301,
    "diastolic": 15,
    "location": "arm"
}
{
        "systolic": 301,
        "diastolic": 15,
        "location": "arm"
}
```

--------------------------------

### Install and Enable Classic Notebook Extension

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/README.md

Commands to install and enable the Stitch extension for the classic Jupyter Notebook interface. The `--symlink` flag is not supported on Windows.

```bash
jupyter nbextension install --sys-prefix --symlink --overwrite --py guidance-stitch
jupyter nbextension enable --sys-prefix --py guidance-stitch
```

--------------------------------

### Create Conda Environment

Source: https://github.com/guidance-ai/guidance/blob/main/CONTRIBUTING.md

Sets up a new Python 3.12 environment for development. Activate it before proceeding with installations.

```bash
conda create --name guidancedev python=3.12
conda activate guidancedev
```

--------------------------------

### Create a Transformers Model

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/intro_to_guidance.ipynb

Instantiate a model using the Transformers library. Ensure you have the necessary libraries installed.

```python
from guidance import models

model = models.Transformers("Qwen/Qwen2.5-1.5B")
```

--------------------------------

### Python: Sending Role Start Tokens

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This snippet demonstrates sending role-start special tokens in a Python program, likely for managing conversational context or agent roles.

```python
partial_output(parser.program.llm.role_start(name))
```

--------------------------------

### Create a LlamaCpp Model

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/intro_to_guidance.ipynb

Instantiate a model using LlamaCpp. This requires downloading a .gguf model file and specifying its path. Ensure the huggingface_hub library is installed.

```python
from guidance import models
from huggingface_hub import hf_hub_download
model = models.LlamaCpp(
    hf_hub_download(
        repo_id="bartowski/Llama-3.2-3B-Instruct-GGUF",
        filename="Llama-3.2-3B-Instruct-Q6_K_L.gguf",
    ),
    verbose=True,
    n_ctx=4096,
)
```

--------------------------------

### Video.js Player Initialization

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Initializes a Video.js player on a given HTML element. It handles existing player instances, applies options, and runs setup hooks.

```javascript
bu(e,t,i){let s=bu.getPlayer(e);if(s)return t&&da.warn(\`Player "${e}" is already initialised. Options will not be applied."),i&&s.ready(i),s;const n="string"==typeof e?To("#"+vu(e)):e;if(!Xa(n))throw new TypeError("The element or ID supplied is not valid. (videojs)");const r="getRootNode"in n&&n.getRootNode()instanceof Le.ShadowRoot?n.getRootNode():n.ownerDocument.body;n.ownerDocument.defaultView&&r.contains(n)||da.warn("The element supplied is not included in the DOM"),!0===(t=t||{}).restoreEl&&(t.restoreEl=(n.parentNode&&n.parentNode.hasAttribute&&n.parentNode.hasAttribute("data-vjs-player")?n.parentNode:n).cloneNode(!0)),sa("beforesetup").forEach((e=>{const i=e(n,va(t));fa(i)&&!Array.isArray(i)?t=va(t,i):da.error("please return an object in beforesetup hooks")}));const a=fl.getComponent("P
```

--------------------------------

### Setup Main Playlist Loader Listeners

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Sets up event listeners for the main playlist loader, specifically for the 'loadedmetadata' event. This event is used to trigger a function 'Qu' with a calculated target duration.

```javascript
setupMainPlaylistLoaderListeners_() {
  this.mainPlaylistLoader_.on("loadedmetadata", (() => {
    const e = this.mainPlaylistLoader_.media(),
      t = 1.5 * e.targetDuration * 1e3;
    Qu(this.mainPlaylistL
```

--------------------------------

### Metadata Track Setup

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Initializes a metadata track if it doesn't exist, and sets the in-band metadata track dispatch type for Safari browsers. This ensures proper handling of metadata.

```javascript
const Op=(e,t,i)=>{e.metadataTrack_||(e.metadataTrack_=i.addRemoteTextTrack({kind:"metadata",label:"Timed Metadata"},!1).track,bu.browser.IS_ANY_SAFARI||(e.metadataTrack_.inBandMetadataTrackDispatchType=t))}
```

--------------------------------

### Install Python Package in Editable Mode

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/README.md

Install the Python package in editable mode for development. This command also builds the TypeScript package and installs test and example dependencies.

```bash
pip install -e ".[test, examples]"
```

--------------------------------

### Generate More Failing Email Examples with ChatGPT

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/testing_lms.ipynb

Formats the identified failing emails into a prompt for ChatGPT to generate more examples of emails where summarization might fail. It then calls ChatGPT to get these new examples.

```python
fails = '\n----\n'.join(['EMAIL: ' + x for x in error_inputs])
question = f'''I have a tool that takes an email and makes it more concise, without losing any important information.
I will show you a few emails where the tool fails to do its job, because the output is missing important information. Your goal is to try to come up with more emails that the tool would fail on.
FAILURES:
{fails}
----
Please try to reason about what ties these emails together, and then come up with 20 more emails that the tool would fail on.
Please use the same format as above, i.e. just the email body, no header or subject, and start each email with "EMAIL:".
'
more = ask_chatgpt(input=question, silent=False, temperature=1, max_tokens=1000, n=1)
```

--------------------------------

### Example Task Execution with Open Source Models

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This code snippet demonstrates running a task using the `guided_terminal` prompt with an open-source LLM (Vicuna or MPT). It initializes the `StatefulShellOpenSource` and passes it as the shell to the Guidance program.

```python
shell = StatefulShellOpenSource()
task = 'Find out what license the open source project located in ~/work/project is using.'
t = guided_terminal(llm=vicuna, task=task, shell=shell)
```

```python
shell = StatefulShellOpenSource()
t = guided_terminal(llm=mpt, task=task, shell=shell)
```

--------------------------------

### Initialize Guidance Model

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/guaranteeing_valid_syntax.ipynb

Set up the language model to be used with Guidance. Supports various model types, including local GGUF files and Hugging Face Transformers models.

```python
import guidance

# Define the model we will use
# lm = guidance.models.LlamaCpp("/path/to/model.gguf", n_gpu_layers=-1)
lm = guidance.models.Transformers("microsoft/Phi-3-mini-4k-instruct")
```

--------------------------------

### Adaptive Bitrate (ABR) Timer Setup

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Configures the ABR timer to start when the main playlist is loaded and stop when the player is paused. It restarts the timer on play events. The timer interval is set to 250ms for checking ABR conditions.

```javascript
this.bufferBasedABR && (this.mainPlaylistLoader_.one("loadedplaylist", (() => this.startABRTimer_())), this.tech_.on("pause", (() => this.stopABRTimer_())), this.tech_.on("play", (() => this.startABRTimer_()))); 
```

--------------------------------

### Prompt with Clear Stopping Criteria

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb

Improves the previous prompt by adding a clear stopping criterion to ensure the LLM generates a specific number of items. This example asks for 6 items and stops when the start of the sixth item is detected, resulting in five items.

```python
lm + '''\
What are the most common commands used in the Linux operating system?

Here are the 6 most common commands:
1. "''' + gen(max_tokens=100, stop="\n6.")
```

--------------------------------

### Example of Search Function Usage

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/rag.ipynb

Demonstrates how to use the `search` function within a `guidance` program. It initializes a language model and calls the `search` function with a specific query, showing how snippets are added to the output.

```python
# Assume lm is an initialized language model
# lm = models.OpenAI("text-davinci-003")

# Example usage:
# lm = search(lm, 'messi "net worth"')
# print(lm)

# Now let's define the functions that search and extract quotes:

@guidance
def search(lm, query):
    # Setting this for later use
    lm = lm.set('query', query)
    # This is where search actually gets called
    lm = lm.set('snippets', format_snippets(top_snippets(query)))
    lm += '\nObservation:\n' + lm['snippets']
    return lm

@guidance
def extract_quotes(lm):
    query = lm['query']
    snippets = lm['snippets'].split('\n\n')[:-1]
    snippet_substrings = [substring(x) for x in snippets]
    # By default we have 3 snippets. The model can pick (1) which snippets it's going to quote, and then (2) a substring of that snippet
    snippet = '- Snippet ' + select([
        '1: "' + snippet_substrings[0] + '"',
        '2: "' + snippet_substrings[1] + '"',
        '3: "' + snippet_substrings[2] + '"',]) + '\n'
    # We can do one or more quotes
    lm += capture(one_or_more(snippet), name='temp_gen')
    # We save all of the quotes so that we can print at the end
    current_quotes = lm.get('current_quotes', '')
    current_quotes += f'''Query: {query}\n{lm['temp_gen']}'''
    lm = lm.set('current_quotes', current_quotes)
    return lm

```

--------------------------------

### Install Jupyterlab extension

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/docs/source/installing.md

Install the stitch front-end extension for Jupyterlab.

```bash
jupyter labextension install @guidance-ai/stitch
```

--------------------------------

### Install Stitch JupyterLab Extension

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/docs/source/develop-install.md

Install the stitch JupyterLab extension if you are working with JupyterLab.

```bash
jupyter labextension install .
```

--------------------------------

### Install Stitch Jupyter Notebook Extension

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/docs/source/develop-install.md

Install and enable the stitch Jupyter Notebook extension. Choose the appropriate flag (--sys-prefix, --user, or --system) based on your Jupyter installation.

```bash
jupyter nbextension install [--sys-prefix / --user / --system] --symlink --py stitch

jupyter nbextension enable [--sys-prefix / --user / --system] --py stitch
```

--------------------------------

### MPT Guided Terminal Execution

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This code snippet shows how to execute a task using the MPT language model within a guided terminal environment. It sets up a stateful shell and then initiates the guided terminal process.

```python
shell = StatefulShellOpenSource()
t = guided_terminal(llm=mpt, task=task, shell=shell)
```

--------------------------------

### Install stitch via conda

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/docs/source/installing.md

Use this command to install the stitch package using conda.

```bash
conda install stitch
```

--------------------------------

### Initialize OpenAI Model

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb

This snippet shows how to initialize an OpenAI model for use with the `guidance` library.

```python
openai_model = models.OpenAI("gpt-4o-mini")
```

--------------------------------

### Install stitch via pip

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/docs/source/installing.md

Use this command to install the stitch package using pip.

```bash
pip install stitch
```

--------------------------------

### Install classic notebook extension

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/docs/source/installing.md

Install the stitch front-end extension for classic Jupyter notebooks if using version < 5.3. Choose the appropriate flag (--sys-prefix, --user, or --system) based on your installation needs.

```bash
jupyter nbextension install [--sys-prefix / --user / --system] --py stitch
```

--------------------------------

### Install Stitch with Pip (Editable Mode)

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/docs/source/develop-install.md

Install the stitch package in editable mode using pip for development.

```bash
pip install -e .
```

--------------------------------

### Run Basic Test Suite

Source: https://github.com/guidance-ai/guidance/blob/main/CONTRIBUTING.md

Executes the basic test suite for the project. Defaults to using GPT2 on the CPU if no specific model is selected.

```bash
python -m pytest ./tests/
```

--------------------------------

### Basic Prompting with Azure OpenAI

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/api_examples/models/AzureOpenAI.ipynb

Demonstrates basic interaction with the configured Azure OpenAI model. It sets a system message, provides a user query, and uses `gen` to capture the assistant's response.

```python
from guidance import system, user, assistant

with system():
    lm = azureai_model + "You are a helpful assistant."
    
with user():
    lm += "What is the meaning of life?"

with assistant():
    lm += gen("response")
```

--------------------------------

### Guided Generation for Open Source Models (Vicuna)

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

This snippet demonstrates how to use explicit guidance for open-source models like Vicuna. It structures the assistant's response into specific generation targets (segment1, segment2, segment3, answer), forcing a particular output format.

```python
qa_guided = guidance('''{{#system~}}
{{llm.default_system_prompt}}
{{~/system}}

{{#user~}}
You will read a meeting transcript, then extract the relevant segments to answer the following question:
Question: {{query}}
----
{{transcript}}
----
Based on the above, please answer the following question:
Question: {{query}}
Please extract the three segment from the transcript that are the most relevant for the answer, and then answer the question.
Note that conversation segments can be of any length, e.g. including multiple conversation turns. If you need less than three segments, you can leave the rest blank.

As an example of output format, here is a fictitious answer to a question about another meeting transcript:
CONVERSATION SEGMENTS:
Segment 1: Peter and John discuss the weather.
Peter: John, how is the weather today?
John: It's raining.
Segment 2: Peter insults John
Peter: John, you are a bad person.
Segment 3: Blank
ANSWER: Peter and John discussed the weather and Peter insulted John.
{{/user}}

{{#assistant~}}
CONVERSATION SEGMENTS:
Segment 1: {{gen 'segment1' temperature=0}}
Segment 2: {{gen 'segment2' temperature=0}}
Segment 3: {{gen 'segment3' temperature=0}}
ANSWER: {{gen 'answer' temperature=0}}
{{~/assistant~}}''')
qa_guided(llm=vicuna, transcript=meeting_transcript, query=query1)
```

--------------------------------

### Import necessary libraries

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/litellm_models.ipynb

Import the required libraries for the LiteLLM and Guidance integration. This includes os for environment variables, BaseModel for Pydantic models, and guidance itself.

```python
import os
from pydantic import BaseModel
import guidance
```

--------------------------------

### Find Tokens Starting with 'http'

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/prompt_boundaries_and_token_healing.ipynb

Identifies and prints tokens in the vocabulary that start with 'http', showing the distinction between 'http' and 'https' tokens.

```python
http_tokens = [i for i,t in enumerate(tokens) if t.startswith("http")]
print_tokens(http_tokens)
```

--------------------------------

### Adapt Prompt for Chat Model with Role Context

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb

This example adapts a prompt for a chat-based model by using `guidance`'s role context blocks (user, assistant) to structure the conversation and ensure compatibility with different chat models.

```python
from guidance import user, assistant, system
newline = "\n"

with user():
    lm2 = chat_lm + "What are the most common commands used in the Linux operating system?"

with assistant():

    # generate a bunch of command names
    lm_tmp = lm2 + 'Here are ten common command names:\n'
    for i in range(10):
        lm_tmp += f'{i+1}. "' + gen('commands', list_append=True, stop='"', max_tokens=20, temperature=0.7) + '"\n'

    # discuss them
    for i,command in enumerate(set(lm_tmp["commands"])):
        lm2 += f'{i+1}. "{command}"\n'
    lm2 += f'''Perhaps the most useful command from that list is: "{gen('cool_command', stop='"')}", because {gen('cool_command_desc', max_tokens=100, stop=newline)}
On a scale of 1-10, it has a coolness factor of: {gen('coolness', regex="[0-9]+")}.'''
```

--------------------------------

### Player Initialization and Loading Logic

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Handles player initialization, including setting up listeners, initializing loaders, and determining when to load the main playlist based on the player's preload setting. It also sets up event listeners to capture timing metrics for loaded data.

```javascript
this.setupMainPlaylistLoaderListeners_();
this.setupSegmentLoaderListeners_();
this.logger_ = Eu("pc");
this.triggeredFmp4Usage = !1;
"none" === this.tech_.preload() ? (this.loadOnPlay_ = () => {
  this.loadOnPlay_ = null,
  this.mainPlaylistLoader_.load()
}, this.tech_.one("play", this.loadOnPlay_)) : this.mainPlaylistLoader_.load();
```

```javascript
const y = "none" === this.tech_.preload() ? "play" : "loadstart";
this.tech_.one(y, (() => {
  const e = Date.now();
  this.tech_.one("loadeddata", (() => {
    this.timeToLoadedData__ = Date.now() - e,
    this.mainAppendsToLoadedData__ = this.mainSegmentLoader_.mediaAppends,
    this.audioAppendsToLoadedData__ = this.audioSegmentLoader_.mediaAppends
  }))
}))
```

--------------------------------

### Start ABR Timer

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Starts the ABR timer by clearing any existing timer and setting a new interval of 250ms to periodically check for ABR conditions.

```javascript
startABRTimer_() {
  this.stopABRTimer_(), this.abrTimer_ = Le.setInterval((() => this.checkABR_()), 250)
}
```

--------------------------------

### Generate Initial Use Cases with ChatGPT

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/testing_lms.ipynb

Use this code to prompt a language model (like ChatGPT) to generate initial examples of concrete use cases for an email assistant tool. Specify the tool's functionality and desired output format in the prompt.

```python
question = '''I have a tool that helps users with their email.
The user highlights sentences or larger chunks either in received emails or in a draft they are writing. Then, the user writes a natural langauge instruction, and the tool performs the desired operation on the highlighted text.
For example, the user may highlight the whole draft, and ask the system to 'make it more concise'. Or the user may highlight an email they received and ask the system to 'write an answer politely declining'.

Please give me 5 examples of concrete use cases for such a tool. For each example, please specify:
- The scenario, i.e. what the user wants to do with the tool. 
    - What did they highlight?
    - What kind of email is the user trying to write or respond to?
- Examples of instructions that could help the user in their writing'''
initial_ideas = ask_chatgpt(input=question, temperature=1, max_tokens=1000, n=3, silent=False)
```

--------------------------------

### Initialize Transformers Model with Guidance

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb

Initializes a Transformers model for use with the Guidance library. Ensure the model name is correct and the environment has the necessary dependencies.

```python
import math
import guidance
from guidance import models, gen, select

lm = models.Transformers("Qwen/Qwen2.5-1.5B")
```

--------------------------------

### Import necessary libraries

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/guidance_acceleration.ipynb

Imports the required libraries for the demonstration, including time, torch, guidance, and specific models and generation functions.

```python
import time
import torch
import guidance
from guidance import models, gen
```

--------------------------------

### Find Tokens Starting with Colon

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/prompt_boundaries_and_token_healing.ipynb

Searches the model's vocabulary for all tokens that start with a colon and prints their IDs and string representations. This demonstrates the variety of tokens that can follow a colon.

```python
tokens = generator.tokenizer.convert_ids_to_tokens(range(generator.tokenizer.vocab_size))
colon_tokens = [i for i,t in enumerate(tokens) if t.startswith(":")]
print_tokens(colon_tokens)
```

--------------------------------

### Test Email Assistant with Sample Inputs and Instructions

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/testing_lms.ipynb

This code iterates through predefined lists of inputs and instructions, applying them to the email assistant using the `format_batch` function. It then displays the differences between the original inputs and the generated outputs, highlighting changes.

```python
# Copy paste from guidance github repo
inputs = ['I really like guidance.', 'Microsoft is a company.', 'I like to eat apples.']
instructions = ['Add the word "dog" somewhere', 'Add an appropriate emoji', 'Make the sentiment more positive']
for instruction in instructions:
    outputs = format_batch(instruction=instruction, source='DRAFT', batch=inputs, silent=True)
    show_diffs(instruction, inputs, outputs, color=True)
```

--------------------------------

### Simple Templating with Guidance

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/intro_to_guidance.ipynb

Use f-strings to define templates in Guidance. Interpolate standard variables and Guidance functions like `gen`. Be mindful of character restrictions in f-string slots for Python versions prior to 3.12.

```python
query = "Who won the last Kentucky derby and by how much?"
model + f'''\
Q: {query}
A: {gen(stop="Q:", max_tokens=40)}'''
```

--------------------------------

### Configure Logging

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/chat.ipynb

Applies the configured logging level to the basic logging setup.

```python
logging.basicConfig(level=requested_log_level)
```

--------------------------------

### Execute Guidance Prompt with ChatGPT

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

Executes the defined Guidance program 'qa_attempt5' using the 'chatgpt' LLM, providing the meeting transcript and a query. This demonstrates calling the prompt with specific variables.

```python
qa_attempt5(llm=chatgpt, transcript=meeting_transcript, query=query2)
```

```python
qa_attempt5(llm=chatgpt, transcript=meeting_transcript, query=query1)
```

--------------------------------

### Run JupyterLab

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/README.md

Start the JupyterLab application. This command should be run in a separate terminal from the watch command.

```bash
jupyter lab
```

--------------------------------

### Download and Prepare OnnxRuntime Model

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/onnxruntime_models.ipynb

Download the Phi-4-mini-instruct-onnx model from Hugging Face Hub and prepare the model path for use with OnnxRuntimeGenAI.

```python
# Download Phi4 model
model_sub_dir = "gpu/gpu-int4-rtn-block-32"
base_model_path = snapshot_download(
    repo_id="microsoft/Phi-4-mini-instruct-onnx",
    allow_patterns=f"{model_sub_dir}/*",
)
model_path = os.path.join(base_model_path, model_sub_dir)
```

--------------------------------

### Define test question and inputs

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/testing_lms.ipynb

Sets up a question to ask the model and a list of input strings to evaluate. This is the initial step for testing specific model properties.

```python
question = "Does the text use any informal language?"
inputs = ['I really like guidance.', 'I like to eat apples.', 'Make my day, buddy', 'Please make sure to tell your father.']
```

--------------------------------

### Define and Execute Guidance Program

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/proverb.ipynb

Sets up the default language model and defines a Guidance program to adapt proverbs. The program uses `gen()` to control text generation and specify stop conditions or patterns.

```python
import guidance
from guidance import models, gen
newline = "\n"

# set the default language model used to execute guidance programs
gpt3 = models.OpenAI("text-davinci-003")

# Define a guidance program that adapts proverbs.
@guidance
def program(lm, proverb, book, chapter, verse):
    lm += f"""
    Tweak this proverb to apply to model instructions instead.

    {proverb}
    - {book} {chapter}:{verse}

    UPDATED
    Where there is no guidance{gen('rewrite', stop=newline + "-")}
    - GPT {gen('chapter', stop=":")}:{gen('verse', regex="[0-9]+")}"""
    return lm

# execute the program on a specific proverb
lm = gpt3 + program(
    proverb="Where there is no guidance, a people falls,\nbut in an abundance of counselors there is safety.",
    book="Proverbs",
    chapter=11,
    verse=14
)
```

--------------------------------

### Load LLM Models with Guidance

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb

Loads various large language models (LLMs) including MPT, Vicuna, and OpenAI's GPT-3.5-turbo using the Guidance library. Ensure the specified paths and devices are correctly configured.

```python
import guidance
import transformers
path = '/home/marcotcr/.cache/huggingface/hub/vicuna-13b'
mpt = guidance.llms.transformers.MPTChat('mosaicml/mpt-7b-chat', device=1)
vicuna = guidance.llms.transformers.Vicuna(path, device_map='auto')
chatgpt = guidance.llms.OpenAI("gpt-3.5-turbo")
```

--------------------------------

### Define a repeatable string

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/guidance_acceleration.ipynb

Defines a simple string that will be used repeatedly in the examples to test generation performance.

```python
# Define a trivial string we can extend with small models easily
prefix = "Repeat this. Repeat this. "*5 + "Repeat this. Repeat this."
print(prefix)
```

--------------------------------

### View Extracted Inputs and Expected Outputs

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/code_generation.ipynb

Displays the extracted arguments and expected results from the docstring examples as a list of tuples.

```python
# (input, expected output)
list(zip(lm['args'], lm['expected']))
```

--------------------------------

### Initialize Video Player with Options

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Initializes a video player using the Video.js library with specified options for controls, inline playback, and fullscreen support. Ensure the video element exists before attempting initialization.

```javascript
function lg(e,t,i){let s,n,{videoData:r}=t;return D((()=>{console.log("videoElement exists?",!!s),s?setTimeout(((()=>{try{n=bu(s,{controls:!0,fluid:!0,playsinline:!0,controlBar:{fullscreenToggle:!0}}),console.log("Player initialized successfully")}catch(e){console.error("Failed to initialize player:",e)}}),0):console.error("Video element not found during mount")})),P((()=>{n&&n.dispose()})),e.$$set=e=>{"videoData"in e&&i(0,r=e.videoData)},[r,s,function(e){N[e?"unshift":"push"]((()=>{s=e,i(1,s)}))}"]}ne(".video-container.svelte-xkosau{width:500px}");class cg extends se{constructor(e){super(),ie(this,e,lg,og,r,{videoData:0})}}
```

--------------------------------

### Develop JupyterLab Extension

Source: https://github.com/guidance-ai/guidance/blob/main/packages/python/stitch/README.md

Commands to enable and build a JupyterLab extension for development. Use `--overwrite` to replace existing installations.

```bash
jupyter labextension develop --overwrite .
jlpm run build
```

--------------------------------

### Get or Set Subtitle Track

Source: https://github.com/guidance-ai/guidance/blob/main/guidance/resources/graphpaper-inline.html

Retrieves or sets the subtitle track. If the track is set and the player is ready, it initiates the buffer monitoring.

```javascript
track(e){return void 0===e||(this.subtitlesTrack_=e,"INIT"===this.state&&this.couldBeginLoading_()&&this.init_()),this.subtitlesTrack_}
```

--------------------------------

### Execute Email Assistant Prompt

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/testing_lms.ipynb

This snippet demonstrates how to use the defined `email_format` Guidance prompt. It provides sample email text and calls the prompt with specific instructions and source type.

```python
email = ''' Hey Marco,
Can you please schedule a meeting for next week? I would like to touch base with you.
Thanks,
Scott'''
email_format(instruction='Politely decline', input=email, source='EMAIL')
```

--------------------------------

### Identify Failed Summarization Examples

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/testing_lms.ipynb

Print details of emails where the summarized response is not more concise than the original. This aids in debugging and understanding summarization failures.

```python
print('Failed examples:')
for i in np.where(~more_concise)[0][:1]:
    print(f'Email: {emails[i]}')
    print('Length: %d' % len(emails[i]))
    print(f'Response: {responses[i]}')
    print('Length: %d' % len(responses[i]))
    print('------')
    print()
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/onnxruntime_models.ipynb

Import essential libraries for model handling, including os, snapshot_download from huggingface_hub, Pydantic for data modeling, AutoTokenizer from transformers, and guidance components.

```python
import os

from huggingface_hub import snapshot_download
from pydantic import BaseModel
from transformers import AutoTokenizer

import guidance
from guidance.chat import Phi4MiniChatTemplate
```

--------------------------------

### Define LLM and Guidance Program for Tool Use

Source: https://github.com/guidance-ai/guidance/blob/main/notebooks/art_of_prompt_design/tool_use.ipynb

Sets up the OpenAI LLM and defines a Guidance program that includes tool definitions and logic for handling function calls. The program iterates up to 10 times, generating answers and breaking if the LLM stops generating function calls.

```python
import guidance

# define the chat model we want to use (must be a recent model supporting function calls)
guidance.llm = guidance.llms.OpenAI("gpt-3.5-turbo-0613", caching=False)

# define a guidance program that uses tools
program = guidance("""
{{~#system~}}
You are a helpful assistant.
{{>tool_def functions=functions}}
{{~/system~}}

{{~#user~}}
Get the current weather in New York City.
{{~/user~}}

{{~#each range(10)~}}
    {{~#assistant~}}
    {{gen 'answer' max_tokens=50 function_call="auto"}}
    {{~/assistant~}}

    {{#if not callable(answer)}}{{break}}{{/if}}
    
    {{~#function name=answer.__name__~}}
    {{answer()}}
    {{~/function~}}
{{~/each~}}"")
```