### Start LMQL Playground with Nix

Source: https://github.com/eth-sri/lmql/blob/main/scripts/flake.d/README.md

Use this command to launch a local LMQL playground instance. Ensure Nix is installed and configured.

```console
$ nix run github:eth-sri/lmql#playground
```

--------------------------------

### Build LMQL Documentation Locally

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/documentation.md

Use these commands to install dependencies and run a live server for local documentation development.

```bash
cd docs
yarn 
yarn docs:dev
```

--------------------------------

### Install LMQL with Hugging Face GPU Support

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/installation.md

Install LMQL with dependencies for local GPU model execution. Ensure you have a compatible PyTorch installation.

```bash
pip install lmql[hf]
```

--------------------------------

### Launch LMQL Playground

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/installation.md

Run this command to start a local instance of the web-based LMQL Playground IDE for experimentation.

```bash
lmql playground
```

--------------------------------

### Install Latest Development Version of LMQL

Source: https://github.com/eth-sri/lmql/blob/main/README.md

Installs the `lmql` package directly from the main branch of the GitHub repository. Use with caution as this version may be less stable than PyPI releases.

```bash
pip install git+https://github.com/eth-sri/lmql
```

--------------------------------

### LMQL Simple Syntax Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/blog/posts/release-0.0.6.5.md

Demonstrates the simplified LMQL syntax where defaults are assumed for model and decoding strategy.

```lmql
name::simple-syntax

"One line is all it takes [CONTINUATION]"
```

--------------------------------

### Example Sequences for foo-bar Constraint

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/constraints/custom-constraints.md

Illustrates valid and invalid sequences for the 'foo'-'bar' constraint. These examples clarify the expected behavior of the custom constraint.

```python
"Hello foo!" # invalid
"Hello foo bar!" # valid
"Hello foo bar foo bar!" # valid
"Hello foo bar foo!" # invalid
```

--------------------------------

### Enter Development Shell with LMQL Dependencies

Source: https://github.com/eth-sri/lmql/blob/main/scripts/flake.d/README.md

This command starts a shell environment with all necessary LMQL dependencies available. Useful for development and testing.

```console
$ nix develop github:eth-sri/lmql#
```

--------------------------------

### Setup LMQL Path and API Key

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/lib/output.ipynb

This snippet sets up the LMQL path and loads the OpenAI API key. It's typically hidden in documentation.

```python
# setup lmql path (not shown in documentation, metadata has nbshpinx: hidden)
import sys 
sys.path.append("../../../src/")
# load and set OPENAI_API_KEY
import os 
os.environ["OPENAI_API_KEY"] = open("../../../api.env").read().split("\n")[1].split(": ")[1].strip()

%load_ext autoreload
%autoreload 2

# disable logit bias logging
import lmql.runtime.bopenai.batched_openai as batched_openai
batched_openai.set_logit_bias_logging(False)
```

--------------------------------

### LMQL Query Builder Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Demonstrates using the LMQL QueryBuilder for a programmatic approach to constructing and running queries.

```python
import lmql

query = (
    lmql.QueryBuilder()
          .set_decoder('argmax')
          .set_prompt('What is the capital of France? [ANSWER]')
          .set_model('gpt2')
          .set_distribution('ANSWER', '["A", "B"]')
          .build()
)

query.run_sync()
# You can also run it asynchronously with query.run_async() and await the result
```

--------------------------------

### LMQL Simple Query Strings Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Demonstrates a basic LMQL program using two simple query strings to interact with an LLM.

```lmql
"Hello [WHO]"
"Goodbye [WHO]"
```

--------------------------------

### Start LMQL Docker Container with GPU and Local Models

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/docker-setup.md

Starts an LMQL Docker container with access to all host GPUs. This command assumes the image was built with GPU support.

```bash
docker run --gpus all -d -p 3000:3000 -p 3004:3004 lmql-docker:cuda11.8
```

--------------------------------

### Start LMTP Interactive Client

Source: https://github.com/eth-sri/lmql/blob/main/src/lmql/models/lmtp/README.md

Launches the interactive client to connect to the running LMTP server. Specify the model name to load.

```bash
python src/lmql/models/lmtp/lmtp_client.py  gpt2-medium
```

--------------------------------

### Install LMQL via Pip

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/installation.md

Use this command to install the base LMQL package in a Python environment.

```bash
pip install lmql
```

--------------------------------

### Serve Model with Custom Transformers Arguments

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/models/hf.md

Starts an LMQL inference server with custom arguments passed directly to HuggingFace's `AutoModelForCausalLM.from_pretrained`. This example sets `trust_remote_code` to `True` and specifies port 9999.

```bash
lmql serve-model gpt2-medium --cuda --port 9999 --trust_remote_code True
```

--------------------------------

### Send Prompt with Custom Parameters

Source: https://github.com/eth-sri/lmql/blob/main/src/lmql/models/lmtp/README.md

Example of sending a prompt to the interactive client with custom generation parameters like temperature.

```bash
>>>> Hello there temperature=1.2
```

--------------------------------

### Chatbot Interaction Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/6-chat.md

This example shows how to represent user and assistant messages in a chat interface using promptdown syntax. It demonstrates a user asking a question and an assistant providing a detailed answer about LMQL.

```promptdown
![bubble:user|What is the best way to interact with LLMs?]

![bubble:assistant|[ANSWER|] The best way to interact with LLMs (Language Model Models) is through a query language like LMQL. LMQL allows you to easily and efficiently query large language models and retrieve the information you need. With LMQL, you can specify the input text, the output format, and the model you want to use , all in a single query. This makes it easy to integrate LLMs into your applications and workflows, and to get the most out of these powerful language models. Additionally, LMQL provides a standardized way of interacting with LLMs, which makes it easier for developers and data scientists to collaborate and share their work .]
```

--------------------------------

### Setup LMQL Anaconda Environment with GPU

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Use these commands to create and activate a conda environment for LMQL development with GPU support. This requires a CUDA-enabled setup.

```bash
# prepare conda environment
conda env create -f scripts/conda/requirements.yml -n lmql
conda activate lmql

# registers the `lmql` command in the current shell
source scripts/activate-dev.sh
```

--------------------------------

### Promptdown Meta Prompting Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/4-meta-prompting.md

This example illustrates meta-prompting using Promptdown syntax. It defines a question, suggests an expert persona with a default fallback, and then constructs a response leveraging the chosen or default expert.

```promptdown
Q: What are Large Language Models?⏎

A good person to answer this question would be [EXPERT| a data scientist or a machine learning engineer.]

For instance, (a data scientist or a machine learning engineer) would answer [ANSWER| this question by explaining that large language models are a type of artificial intelligence (AI) model that uses deep learning algorithms to process large amounts of natural language data.]

```

--------------------------------

### LMQL Prompt Construction Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Demonstrates how LMQL constructs prompts by concatenating query strings and substituting variables during execution. Shows the prompt state at different execution points.

```lmql
numbers::

"Hello, [NAME] and [NAME2]\n" 
        ⬆ 1       ⬆ 2

"How are you doing?"
⬆ 3

" [FEELINGS]"
⬆ 4        ⬆ 5
```

--------------------------------

### Promptdown Example with Constraints

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/2.5-data-types.md

Demonstrates the equivalent of LMQL's type and regex constraints within the Promptdown syntax, showing inline value specification.

```promptdown
Q: It's the last day of June. What day is it?
A: Today is [RESPONSE| 30/06]

Q: What's the month number?
A: [ANSWER| 6]
```

--------------------------------

### Start LMQL Docker Container with Environment Variables

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/docker-setup.md

Starts an LMQL Docker container and overrides default environment variables, such as the OpenAI API key. Ensure the API key is kept secure.

```bash
docker run -d -e OPENAI_API_KEY=<your openai api key> -p 3000:3000 -p 3004:3004 lmql-docker:latest
```

--------------------------------

### Vue.js Script Setup for Blog Posts

Source: https://github.com/eth-sri/lmql/blob/main/docs/blog/index.md

This script setup block imports post data and defines helper functions to extract version information and clean titles for display. It's used within a Vue.js component to process post metadata.

```javascript
import { data as posts } from './posts.data.js'

function getTitleVersion(title) {
    const version = title.match(/v\d+\.\d+(\.\d+)?/)
    if (version) {
        return version[0]
    }
}

function getTitleText(title) {
    // strip version from title
    return title
} 

```

--------------------------------

### LMQL FollowMap Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/lib/integrations/lmql.txt

Demonstrates the construction of a FollowMap for a TEXT constraint in LMQL. The FollowMap determines future valid tokens based on the current partial trace.

```lmql
Follow[TEXT in ["Stephen Hawking"]]("Steph", t) = {
  fin(⊤) if t = "en Hawking"
  fin(⊥) else 
}
```

--------------------------------

### Setup LMQL Anaconda Environment without GPU

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Use these commands to create and activate a conda environment for LMQL development without local GPU support. This setup is suitable for API-integrated models.

```bash
# prepare conda environment
conda env create -f scripts/conda/requirements-no-gpu.yml -n lmql-no-gpu
conda activate lmql-no-gpu

# registers the `lmql` command in the current shell
source scripts/activate-dev.sh
```

--------------------------------

### Build LMQL Docker Image with GPU Support

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/docker-setup.md

Builds a Docker image with GPU support. Requires CUDA installed on the host and Docker GPU support. PyTorch with CUDA 11.8 support will be installed.

```bash
docker build --build-arg GPU_ENABLED=true -t lmql-docker:cuda11.8 .
```

--------------------------------

### LMQL Query String Examples

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Illustrates various ways to construct query strings in LMQL, including basic strings, variables, constraints, decorators, type annotations, nested calls, and string interpolation.

```lmql
"Hello, world!"
```

```lmql
"Hello, [NAME] and [NAME2]!"
```

```lmql
"Hello, [NAME]!" where len(TOKENS(NAME)) < 10
```

```lmql
"Hello, [@fct(a=12) NAME]!"
```

```lmql
"Your Age: [AGE:int]!"
```

```lmql
"Q: What is 2x2? A: [ANSWER: chain_of_thought(shots=2)]"
```

```lmql
NAME = "Alice"
"Hello, {NAME}, how old are you: [AGE:int]?"
```

--------------------------------

### LMQL Standalone Query: Full Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

A comprehensive LMQL standalone query combining decoder, prompt, from, distribution, and where clauses.

```lmql
argmax 
    "What is the capital of France? [ANSWER]" 
from
    "gpt2"
distribution 
    ANSWER in ["A", "B"]
```

--------------------------------

### Start LMQL Docker Container with API Environment File

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/docker-setup.md

Mounts an api.env file from the host machine into the Docker container to configure environment variables. This is an alternative to using the -e flag.

```bash
docker run -d -v $(PWD)/api.env:/lmql/.lmql/api.env -p 3000:3000 -p 3004:3004 lmql-docker:latest
```

--------------------------------

### LMQL Grammar Notation Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Illustrates the EBNF-like grammar notation used for LMQL, including terminals, non-terminals, and operators.

```grammar
<SENTENCE> := "The quick" ["brown"]? <SUBJECT> <VERB> "over the lazy dog."
<SUBJECT> := "fox" | "dog"
<VERB> := "jumps" | "hops" | [.*]{4}
```

--------------------------------

### Start LMQL Docker Container

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/docker-setup.md

Runs a Docker container from the built LMQL image. Ports 3000 and 3004 are mapped to the host for accessing the playground and backend.

```bash
docker run -d -p 3000:3000 -p 3004:3004 lmql-docker:latest
```

--------------------------------

### Define and Run an LMQL Query

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/1-code.md

This example demonstrates defining a simple LMQL query function that asks a question, constrains the answer's length and termination, and returns a typed integer. It also shows how to call this function from Python.

```lmql
controls::

@lmql.query
def meaning_of_life():
    '''lmql
    # top-level strings are prompts
    "Q: What is the answer to life, the \ 
     universe and everything?"

    # generation via (constrained) variables
    "A: [ANSWER]" where \
        len(ANSWER) < 120 and STOPS_AT(ANSWER, ".")

    # results are directly accessible
    print("LLM returned", ANSWER)

    # use typed variables for guaranteed 
    # output format
    "The answer is [NUM: int]"

    # query programs are just functions 
    return NUM
    '''

# so from Python, you can just do this
meaning_of_life() # 42
```

--------------------------------

### Start llama.cpp Model Server

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/models/llama.cpp.md

Launches an LMTP inference endpoint for llama.cpp models. This endpoint can be used by LMQL via an lmql.model(...) object.

```bash
lmql serve-model llama.cpp:<PATH TO WEIGHTS>.gguf
```

--------------------------------

### Serve GPTQ Quantized Model

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/models/hf.md

Serve a GPTQ-quantized model using the 'lmql serve-model' command with the '--loader gptq' flag. Ensure AutoGPTQ is installed.

```bash
lmql serve-model TheBloke/Arithmo-Mistral-7B-GPTQ --loader gptq
```

--------------------------------

### Serve AWQ Quantized Model

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/models/hf.md

Serve an AWQ-quantized model using the 'lmql serve-model' command with the '--loader awq' flag. Ensure AutoAWQ is installed.

```bash
lmql serve-model TheBloke/Mistral-7B-OpenOrca-AWQ --loader awq
```

--------------------------------

### LMQL Multi-Part Program with Nested Queries

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/nestedqueries.md

Chain multiple nested queries to inject instructions sequentially without affecting the overall prompt flow. This example uses a date formatting query multiple times.

```lmql
@lmql.query
def dateformat():
    '''lmql
    "(respond in DD/MM/YYYY) [ANSWER]"
    return ANSWER.strip()
    '''

"Q: When was Obama born? [ANSWER: dateformat]\n"
"Q: When was Bruno Mars born? [ANSWER: dateformat]\n"
"Q: When was Dua Lipa born? [ANSWER: dateformat]\n"

"Out of these, who was born last?[LAST]"
```

--------------------------------

### Promptdown Representation of Long Constraint

Source: https://github.com/eth-sri/lmql/blob/main/docs/blog/posts/release-0.0.6.md

Shows the promptdown representation for the long-form constraint example, illustrating how the LLM output is guided and potentially short-circuited by the specified options.

```promptdown
If we have the choice we choose [OPTION|Option A with a whole lot of extra context]
```

--------------------------------

### Server Initialization Log

Source: https://github.com/eth-sri/lmql/blob/main/src/lmql/models/lmtp/README.md

Shows the server's startup log, including model loading and the network address it's running on.

```log
> lmql serve-model --cuda gpt2-medium
[Loading gpt2-medium with AutoModelForCausalLM.from_pretrained(gpt2-medium, 'device_map': 'auto')]
======== Running on http://localhost:8080 ========
(Press CTRL+C to quit)
[gpt2-medium ready on device cuda:0]
GENERATE {"model": "gpt2-medium", "prompt": [15496, 612, 220], "stream_id": 1}
```

--------------------------------

### Promptdown Example with Constraints

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/2-constraining.md

This example shows how to define constraints directly within a Promptdown block. It specifies that the JOKE should be the answer to the question and the PUNCHLINE should be 'Dam!'.

```promptdown
Tell me a joke:

Q: [JOKE| What did the fish say when it hit the wall?]
A: [PUNCHLINE| Dam!]
```

--------------------------------

### Test Web Build of LMQL

Source: https://github.com/eth-sri/lmql/blob/main/CONTRIBUTING.md

Use this script to build and serve the WebAssembly/Pyodide version of LMQL for testing web playground features. Access it at `http://localhost:8081/playground/`.

```bash
bb
```

--------------------------------

### LMQL Grammar Examples of Valid Sentences

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Provides examples of valid sentence derivations according to the defined LMQL grammar, showcasing optional elements and choices.

```lmql
"The quick fox jumps over the lazy dog."
```

```lmql
"The quick brown fox jumps over the lazy dog."
```

```lmql
"The quick dog jumps over the lazy dog."
```

```lmql
"The quick brown dog hops over the lazy dog."
```

```lmql
"The quick dog A9_D over the lazy dog."
```

--------------------------------

### Promptdown Model Output Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/overview.md

This is an example of the expected model output in Promptdown format for the LMQL sentiment analysis program. It shows how the generated analysis and classification are presented.

```promptdown
# Model Output
Review: We had a great stay. Hiking in the mountains was fabulous and the food is really good.
Q: What is the underlying sentiment of this review and why?
A: [ANALYSIS|The underlying sentiment of this review is positive because the reviewer had a great stay, enjoyed the hiking and found the food to be good.]
Based on this, the overall sentiment of the message can be 
considered to be [CLS| positive]
```

--------------------------------

### Enter LMQL Development Shell with Nix

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Enter a Nix development shell for LMQL. This command downloads all optional dependencies for maximum flexibility. Replace `github:eth-sri/lmql` with a local path if working within the source tree.

```bash
nix develop github:eth-sri/lmql#lmql
```

--------------------------------

### Implement Key-Value Store with LMQL and Python

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/tools.md

Demonstrates using LMQL to interact with Python's interpreter state for a simple key-value store. Useful for maintaining context and state during complex reasoning tasks.

```lmql
# implement a simple key value storage
# with two operations
storage = {}
def assign(key, value):
    # store a value
    storage[key] = value; return f'{{{key}: "{value}"}}
def get(key):
    # retrieve a value
    return storage.get(key)

# instructive prompt, instructing the model to how use the storage
"""In your reasoning you can use actions. You do this as follows:
`action_name(<args>) # result: <inserted result>`
To remember things, you can use 'assign'/'get':
- To remember something:
`assign("Alice", "banana") # result: "banana"`
- To retrieve a stored value:
`get("Alice") # result: "banana"`
Always tail calls with " # result". Using these actions, let's solve
the following question.\n"""

# actual problem statement
"""
Q: Alice, Bob, and Claire are playing a game. At the start 
of the game, they are each holding a ball: Alice has a black 
ball, Bob has a brown ball, and Claire has a blue ball. 

As the game progresses, pairs of players trade balls. First, 
Bob and Claire swap balls. Then, Alice and Bob swap balls. 
Finally, Claire and Bob swap balls. At the end of the game, 
what ball does Alice have?
A: Let's think step by step.
"""

# core reasoning loop
for i in range(32):
    "[REASONING]" where STOPS_AT(REASONING, "# result") and \
                        STOPS_AT(REASONING, "Therefore, ")
    
    if REASONING.endswith("# result"):
        cmd = REASONING.rsplit("`",1)[-1]
        cmd = cmd[:-len("# result")]
        "{eval(cmd)}`\n"
    else:
        break

# generate final answer
"Therefore at the end of the game, Alice has the[OBJECT]" \
    where STOPS_AT(OBJECT, ".") and STOPS_AT(OBJECT, ",")
```

```promptdown
# Model Output
(...)
A: Let's think step by step

[REASONING()| At the start of the game:
`assign('Alice', 'black') # result] {Alice: 'black'}
[REASONING()| `assign('Bob', 'brown') # result] {Bob: 'brown'}
[REASONING()| `assign('Claire', 'blue') # result] {Claire: 'blue'}

[REASONING()| After Bob and Claire swap balls:
`assign('Bob', 'blue') # result] {Bob: 'blue'}
[REASONING()| `assign('Claire', 'brown') # result] {Claire: 'brown'}

[REASONING()| After Alice and Bob swap balls:
`assign('Alice', 'blue') # result] {Alice: 'blue'}
[REASONING()| `assign('Bob', 'black') # result] {Bob: 'black'}

[REASONING()| After Claire and Bob swap balls:
`assign('Claire', 'black') # result] {Claire: 'black'}
[REASONING()| `assign('Bob', 'brown') # result] {Bob: 'brown'}

[REASONING()| At the end of the game, Alice has a blue ball:
`get('Alice') # result] blue`
Therefore at the end of the game, Alice has the [OBJECT| blue ball.]
```

--------------------------------

### Run LMQL Playground with Nix (Basic)

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Invoke the LMQL playground using Nix with the '-basic' target, which only supports OpenAI models. Replace `github:eth-sri/lmql` with a local path if working within the source tree.

```bash
nix run github:eth-sri/lmql#playground-basic
```

--------------------------------

### Write a Basic LMQL Query

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/index.md

This is a simple 'Hello World' LMQL query. It consists of a prompt statement and a constraint clause that limits the output length.

```lmql
"Say 'this is a test':[RESPONSE]" where len(TOKENS(RESPONSE)) < 25
```

--------------------------------

### Run LMQL Playground with Nix (llama.cpp)

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Invoke the LMQL playground using Nix with the '-llamaCpp' target, which supports llama.cpp. Replace `github:eth-sri/lmql` with a local path if working within the source tree.

```bash
nix run github:eth-sri/lmql#playground-llamaCpp
```

--------------------------------

### Run LMQL Playground with Nix

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Invoke the LMQL playground using Nix. This command downloads all optional dependencies for maximum flexibility. Replace `github:eth-sri/lmql` with a local path if working within the source tree.

```bash
nix run github:eth-sri/lmql#playground
```

--------------------------------

### LMQL Standalone Query: Decoder Clause + From

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Example of an LMQL query using the 'argmax' decoder and specifying the model with the 'from' clause.

```lmql
argmax "What is the capital of France? [ANSWER]" from "gpt2"
```

--------------------------------

### Run LMTP Server with Nix

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Invoke the LMTP server using Nix. This command downloads all optional dependencies for maximum flexibility. Replace `github:eth-sri/lmql` with a local path if working within the source tree.

```bash
nix run github:eth-sri/lmql#lmtp-server
```

--------------------------------

### LMQL Token Constraints

Source: https://github.com/eth-sri/lmql/blob/main/docs/blog/posts/release-0.0.6.3.md

Use TOKENS(...) to specify the exact number of tokens for a variable. This example enforces a 10-token response.

```lmql
name::token_constraints
argmax 
    "A 10 token response[WHO]" 
from 
    "openai/text-ada-001" 
where 
    len(TOKENS(WHO)) == 10
```

--------------------------------

### Run LMQL Playground with Nix (Hugging Face)

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Invoke the LMQL playground using Nix with the '-hf' target, which supports OpenAI and Hugging Face models. Replace `github:eth-sri/lmql` with a local path if working within the source tree.

```bash
nix run github:eth-sri/lmql#playground-hf
```

--------------------------------

### Execute LangChain Chain for Company Name

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/lib/integrations/langchain.ipynb

Runs the previously defined LangChain `LLMChain` to get a company name for a given product.

```python
chain.run("colorful socks")
```

--------------------------------

### LMQL Standalone Query: Decoder Clause + Where

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Example of an LMQL query using 'argmax' decoder and a 'where' clause to constrain the answer length.

```lmql
argmax "What is the capital of France? [ANSWER]" \
    where len(TOKENS(ANSWER)) < 10
```

--------------------------------

### Setup LMQL Path and Environment

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/lib/python.ipynb

This code configures the Python environment to use the LMQL library. It appends the LMQL source directory to sys.path, enables autoreload for development, and disables logit bias logging for cleaner output. Required for using LMQL within Python scripts or notebooks.

```python
# setup lmql path
import sys 
sys.path.append("../../../src/")

%load_ext autoreload
%autoreload 2

# disable logit bias logging
import lmql.runtime.bopenai.batched_openai as batched_openai
batched_openai.set_logit_bias_logging(False)

import lmql
```

--------------------------------

### Run LMQL Playground with Nix (Replicate)

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/dev-setup.md

Invoke the LMQL playground using Nix with the '-replicate' target, which supports Hugging Face models remoted via Replicate. Replace `github:eth-sri/lmql` with a local path if working within the source tree.

```bash
nix run github:eth-sri/lmql#playground-replicate
```

--------------------------------

### Promptdown Sentiment Analysis Distribution

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/3.5-distributions.md

This Promptdown example performs sentiment analysis, generating an analysis and then measuring the probability distribution for the sentiment classification.

```promptdown
Review: We had a great stay. Hiking in the mountains was fabulous and the food is really good.

Q: What is the underlying sentiment of this review and why?

A: [ANALYSIS|Positive, because the reviewer enjoyed their stay and had positive experiences with both the activities and food.]

Based on this, the overall sentiment of the message 
can be considered to be [_CLS(color='ablue')|\[CLASSIFICATION\]]





```
<div class="distribution">
   <i>P(<b>CLASSIFICATION</b>) =</i> 
   <div>
   -  <b>positive 0.9998711120293567</b><br/>
   -  neutral 0.00012790777085508993<br/>
   -  negative   9.801997880775052e-07
   </div>
</div>
```

--------------------------------

### Promptdown Representation of Dynamic Prompt

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/overview.md

This is the Promptdown representation of the LMQL dynamic prompting example, showing the structure of the input and expected model outputs.

```promptdown
# Model Output
Review: We had a great stay. Hiking in the mountains was fabulous and the food is really good.

Q: What is the underlying sentiment of this review and why?
A: [ANALYSIS|The underlying sentiment of this review is positive because the reviewer had a great stay, enjoyed the hiking and found the food to be good.]

Based on this, the overall sentiment of the message can be considered to be [CLASSIFICATION|positive]

What is it that they liked about their stay?
[FURTHER_ANALYSIS|The reviewer liked the hiking in the mountains and the food.]

```

--------------------------------

### Post-Processing Decorator Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/decorators.md

Applies a decorator to post-process a generated variable value, converting it to uppercase. Decorators are applied after the variable has finished decoding.

```lmql
name::decorators

def screaming(value):
    """Decorator to convert a string to uppercase"""
    return value.upper()

"Say 'this is a test':[@screaming TEST]"
```

```promptdown
Say 'this is a test': [TEST|THIS IS A TEST]
```

--------------------------------

### Multi-Part Prompting with Promptdown

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/3-multi-part.md

Illustrates multi-part prompting using Promptdown syntax, including a reasoning step with a specific output and constrained answer extraction. This is useful for structured prompts where intermediate reasoning is explicitly defined.

```promptdown
Q: It was Sept. 1st, 2021 a week ago. What is the date 10 days ago in MM/DD/YYYY?
Answer Choices: (A) 08/29/2021 (B) 08/28/2021 (C) 08/29/1925 (D) 08/30/2021 (E) 05/25/2021 (F) 09/19/2021

A: Let's think step by step.
[REASONING(color='red')| Sept. 1st, 2021 was a week ago, so 10 days ago would be 8 days before that, which is August 23rd, 2021, so the answer is (A) 08/29/2021.]

Therefore, the answer is [ANSWER| A]
```

--------------------------------

### LMQL Standalone Query: Decoder Clause + Distribution

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/reference.md

Example of an LMQL query using 'argmax' decoder and a 'distribution' clause to limit possible answers.

```lmql
argmax "What is the capital of France? [ANSWER]" \
    distribution ANSWER in ["A", "B"]
```

--------------------------------

### Initialize WebSocket Connection and Event Handlers

Source: https://github.com/eth-sri/lmql/blob/main/src/lmql/lib/chat/assets/index.html

Sets up the WebSocket connection on page load, updates connection status, and defines handlers for message reception, connection opening, and closing.

```javascript
let messageId = 0;
let websocket = null;
window.onload = function() {
  setThinking(true);
  document.getElementById('connection-status').innerText = 'Connecting...';
  host = window.location.host;
  websocket = new WebSocket("ws://" + host + "/chat");
  document.getElementById('address').innerText = websocket.url;
  websocket.onmessage = function(event) {
    let payload = JSON.parse(event.data);
    if (payload.type === 'response') {
      let id = payload.message_id;
      let object = payload.data;
      // show prompt on backside
      let prompt = object.prompt;
      // replace all <lmql:*/> tags with <span class="lmql-tag">...</span>
      prompt = prompt.replace(/<lmql:.\*?\/>/g, (match) => {
        // replace angled brackets with html entities
        let tag_type = match.substring(6, match.length - 2);
        return `<span class="lmql-tag ${tag_type}">${tag_type}</span>`;
      });
      // escape all << and >>
      prompt = prompt.replace(/<</g, '&lt;&lt;');
      prompt = prompt.replace(/>>/g, '&gt;&gt;');
      document.getElementById("raw-prompt").innerHTML = prompt;
      addResponse(object.variables["ANSWER"].trim(), 'assistant', "response-" + id);
    }
  }
  websocket.onopen = function(event) {
    document.getElementById('connection-status').innerText = 'Connected';
    websocket.send(JSON.stringify({ type: 'init' }));
    setThinking(false);
  }
  websocket.onclose = function(event) {
    document.getElementById('connection-status').innerText = 'Disconnected';
  }
  // get send and focus
  document.querySelector('.input textarea').focus();
}
```

--------------------------------

### Streaming Decorator Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/language/decorators.md

Apply a streaming decorator to process intermediate variable values during decoding. This is useful for real-time output, such as displaying partial responses.

```lmql
name::streaming-decorators
from lmql.runtime.program_state import ProgramState

@lmql.decorators.streaming
def stream(value: str, context: ProgramState):
    """Decorator to stream the variable value"""
    print("VALUE", [value])

"Enumerate the alphabet without spaces:[@stream TEST]"
```

--------------------------------

### LMQL Meta Prompting with Beam Search

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/4-meta-prompting.md

This snippet demonstrates meta-prompting in LMQL using the 'beam' decoding algorithm for multi-branch exploration. It shows how to specify decoding strategies, pose questions, use multi-part meta-prompts with expert identification, and process intermediate results in Python before generating a final response.

```lmql
# specify a decoding algorithm (e.g. beam, sample, best_k)
# to enable multi-branch exploration of your program
beam(n=2)

# pose a question
"Q: What are Large Language Models?\n\n"

# use multi-part meta prompting for improved reasoning
"A good person to answer this question would be[EXPERT]\n\n" where STOPS_AT(EXPERT, ".") and STOPS_AT(EXPERT, "\n")

# process intermediate results in Python
expert_name = EXPERT.rstrip(".\n")

# generate the final response by leveraging the expert
"For instance,{expert_name} would answer [ANSWER]" \
    where STOPS_AT(ANSWER, ".") 

```

--------------------------------

### LMQL INT Constraint Example

Source: https://github.com/eth-sri/lmql/blob/main/docs/blog/posts/release-0.0.5.md

Demonstrates the use of the INT constraint to ensure generated numbers are valid integers and are converted to the `int` type for further processing.

```lmql
argmax
   "My favorite number is: [NUM]\n"
   print(type(NUM), NUM * 2) # <class 'int'> 4
   "Number times two is {NUM * 2}"
from
   'openai/text-ada-001'
where
   INT(NUM) 

```

--------------------------------

### Build LMQL Docker Image

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/development/docker-setup.md

Use this command to create a Docker image with the latest stable version of LMQL. Ensure the Dockerfile is in the current directory.

```bash
docker build -t lmql-docker:latest .
```

--------------------------------

### Invoke Wikipedia Tool During Promptdown Generation

Source: https://github.com/eth-sri/lmql/blob/main/docs/features/examples/5-wikipedia.md

Demonstrates calling an external tool like Wikipedia within a Promptdown structure. The example shows how to define the action, provide a placeholder for the tool's output, and then use that output to formulate a final answer.

```promptdown
Q: From which countries did the Norse originate?

Action: Let's search Wikipedia for the term [TERM| 'Norse']
Result: (Norse is a demonym for Norsemen, a Medieval North Germanic ethnolinguistic group ancestral to modern Scandinavians, defined as speakers of Old Norse from about the 9th to the 13th centuries.)

Final Answer: [ANSWER| The Norse originated from Scandinavia.]
```

--------------------------------

### Generate Text with lmql.generate

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/lib/generations.md

Generates text completions based on a prompt. Uses the default model if none is specified.

```python
async def lmql.generate(
    prompt: str, 
    max_tokens: Optional[int] = None, 
    model: Optional[Union[LLM, str]] = None, 
    **kwargs
) -> Union[str, List[str]]
```

--------------------------------

### LMQL Query with Sample Decoding

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/lib/integrations/lmql.txt

Demonstrates an LMQL query with control flow and sample decoding. The sample(n=2) method generates two possible outputs.

```lmql
A list of things not to forget when travelling: 
- keys 
- passport 
The most important of these is sun screen. 

A list of things not to forget when travelling: 
- watch 
- hat 
The most important of these is keys.
```

--------------------------------

### Load and Set OpenAI API Key

Source: https://github.com/eth-sri/lmql/blob/main/docs/docs/lib/integrations/llama_index.ipynb

Loads necessary libraries and sets the OPENAI_API_KEY environment variable for use with OpenAI models. Ensure llama_index is installed.

```python
#notebooks.js:hidden
%load_ext autoreload
%autoreload 2

import sys 
sys.path.append("../../../../src/")
# load and set OPENAI_API_KEY
import os 
from lmql.runtime.openai_secret import openai_secret
os.environ["OPENAI_API_KEY"] = openai_secret
```