### Install Guidance using pip
Source: https://github.com/walchshofer/local-guidance/blob/main/docs/index.rst
This snippet shows how to install the Guidance library using pip, the Python package installer. Ensure you have Python and pip installed on your system.
```shell
pip install guidance
```
--------------------------------
### Enforce Output Syntax with Pattern Guides
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb
Demonstrates using the `pattern` guide to enforce a specific syntax on generated output, ensuring it matches a regular expression. Useful for controlling output format, like numeric values.
```guidance
{{gen 'coolness' pattern="[0-9]+"}}
```
--------------------------------
### Initialize Guidance with OpenAI
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This snippet shows how to import the Guidance library and set up the language model to use OpenAI's GPT-4. Ensure you have your OpenAI API key configured.
```python
import guidance
import re
guidance.llm = guidance.llms.OpenAI("gpt-4")
```
--------------------------------
### Example Search Results Demonstration (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This Python code segment defines a sample list of search results, mimicking the output of the `top_snippets` function. This `demo_results` variable is used as input for a templating example to simulate a conversational AI interaction.
```python
demo_results = [{'title': 'OpenAI - Wikipedia',
'snippet': 'OpenAI systems run on the fifth most powerful supercomputer in the world. [5] [6] [7] The organization was founded in San Francisco in 2015 by Sam Altman, Reid Hoffman, Jessica Livingston, Elon Musk, Ilya Sutskever, Peter Thiel and others, [8] [1] [9] who collectively pledged US$ 1 billion. Musk resigned from the board in 2018 but remained a donor.'},
{'title': 'About - OpenAI',
'snippet': 'About OpenAI is an AI research and deployment company. Our mission is to ensure that artificial general intelligence benefits all of humanity. Our vision for the future of AGI Our mission is to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity. Read our plan for AGI'},
{'title': 'Sam Altman - Wikipedia',
'snippet': 'Samuel H. Altman ( / ˈɔːltmən / AWLT-mən; born April 22, 1985) is an American entrepreneur, investor, and programmer. [2] He is the CEO of OpenAI and the former president of Y Combinator. [3] [4] Altman is also the co-founder of Loopt (founded in 2005) and Worldcoin (founded in 2020). Early life and education [ edit]'}]
```
--------------------------------
### Display Generated Options
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This Python code snippet prints the generated options from the `create_plan` guidance object, formatting them with their index.
```python
print('\n'.join(['Option %d: %s' % (i, x) for i, x in enumerate(out['options'])]))
```
--------------------------------
### Initialize Guidance with OpenAI Model (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/anachronism.ipynb
Initializes the guidance library by setting the language model to be used. This example specifies the 'text-davinci-003' model from OpenAI.
```python
import guidance
# define the model we will use
guidance.llm = guidance.llms.OpenAI("text-davinci-003")
```
--------------------------------
### Load Language Models with Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This Python code snippet demonstrates how to load different language models (MPTChat, Vicuna, OpenAI's ChatGPT) using the guidance library. It specifies model paths and device assignments. Ensure the 'guidance' and 'transformers' libraries are installed.
```python
import guidance
import transformers
path = '/home/marcotcr/.cache/huggingface/hub/vicuna-13b'
mpt = guidance.llms.transformers.MPTChat('mosaicml/mpt-7b-chat', device=1)
vicuna = guidance.llms.transformers.Vicuna(path, device_map='auto')
chatgpt = guidance.llms.OpenAI("gpt-3.5-turbo")
```
--------------------------------
### Initialize OpenAI Chat Model with Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb
Initializes an OpenAI chat model (e.g., 'gpt-3.5-turbo') using the `guidance.llms.OpenAI` class. This setup is used when interacting with models through restricted APIs where direct control over internal generation steps is limited.
```python
chat_llm2 = guidance.llms.OpenAI("gpt-3.5-turbo")
```
--------------------------------
### Multi-step Chat for Planning with Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This example defines a multi-step chat process using Guidance to generate a plan. It includes hidden blocks for intermediate steps like generating options and evaluating pros/cons, finally producing a detailed plan based on user input.
```python
def parse_best(prosandcons, options):
best = re.search(r'Best=(\d+)', prosandcons)
if not best:
best = re.search(r'Best.*?(\d+)', 'Best= option is 3')
if best:
best = int(best.group(1))
else:
best = 0
return options[best]
create_plan = guidance('''{{#system~}}
You are a helpful assistant.
{{~/system}}
{{#block hidden=True}}
{{#user~}}
I want to {{goal}}.
{{~! generate potential options ~}}
Can you please generate one option for how to accomplish this?
Please make the option very short, at most one line.
{{~/user}}
{{#assistant~}}
{{gen 'options' n=5 temperature=1.0 max_tokens=600}}
{{~/assistant}}
{{/block}}
{{~! generate pros and cons and select the best option ~}}
{{#block hidden=True}}
{{#user~}}
I want to {{goal}}.
Can you please comment on the pros and cons of each of the following options, and then pick the best option?
---
{{#each options}}
Option {{@index}}: {{this}}
{{/each}}
---
Please discuss each option very briefly (one line for pros, one for cons), and end by saying Best=X, where X is the best option.
{{~/user}}
{{#assistant~}}
{{gen 'prosandcons' temperature=0.0 max_tokens=600}}
{{~/assistant}}
{{/block}}
{{#user~}}
I want to {{goal}}.
{{~! Create a plan }}
Here is my plan:
{{parse_best prosandcons options}}
Please elaborate on this plan, and tell me how to best accomplish it.
{{~/user}}
{{#assistant~}}
{{gen 'plan' max_tokens=500}}
{{~/assistant}}''')
out = create_plan(goal='read more books', parse_best=parse_best)
out
```
--------------------------------
### Expert Consultation with Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This example demonstrates how to use Guidance to simulate asking a question to a panel of anonymous experts. The process involves first identifying experts and then generating an answer as if they collaborated.
```python
experts = guidance(
'''{{#system~}}
You are a helpful assistant.
{{~/system}}
{{#user~}}
I want a response to the following question:
{{query}}
Who are 3 world-class experts (past or present) who would be great at answering this?
Please don't answer the question or comment on it yet.
{{~/user}}
{{#assistant~}}
{{gen 'experts' temperature=0 max_tokens=300}}
{{~/assistant}}
{{#user~}}
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question.
If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...').
Please start your answer with ANSWER:
{{~/user}}
{{#assistant~}}
{{gen 'answer' temperature=0 max_tokens=500}}
{{~/assistant}}''')
experts(query='What is the meaning of life?')
```
--------------------------------
### Example Task Execution with Vicuna and MPT
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
These Python snippets show how to execute a task using the `guided_terminal` template with open-source LLMs. A `StatefulShellOpenSource` instance is created to manage the `BashSession`. The `guided_terminal` function is then called with the LLM (Vicuna or MPT) and the task, allowing the model to interact with the shell to find the project's license.
```python
shell = StatefulShellOpenSource()
t = guided_terminal(llm=vicuna, task=task, shell=shell)
```
```python
shell = StatefulShellOpenSource()
t = guided_terminal(llm=mpt, task=task, shell=shell)
```
--------------------------------
### Python: Initialize Guided Terminal with Vicuna
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This Python snippet shows the initialization of a 'guided_terminal' using the Vicuna LLM. It utilizes a 'StatefulShellOpenSource' object as the shell environment. The purpose is to test Vicuna's ability to follow output structures and execute tasks correctly, comparing its performance against other models.
```python
shell = StatefulShellOpenSource()
t = guided_terminal(llm=vicuna, task=task, shell=shell)
```
--------------------------------
### Display Pros and Cons Analysis
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This Python code snippet prints the detailed pros and cons analysis generated by the `create_plan` guidance object, including the identified best option.
```python
print(out['prosandcons'])
```
--------------------------------
### Example Task Execution with ChatGPT
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This Python code snippet demonstrates how to use the `run_task_chatgpt` function to have ChatGPT determine the license of an open-source project. It defines a specific task related to finding a license file within a project directory and then calls the function to execute this task.
```python
task = 'Find out what license the open source project located in ~/work/project is using.'
run_task_chatgpt(task)
```
--------------------------------
### Selection from Options with `select` in Guidance
Source: https://context7.com/walchshofer/local-guidance/llms.txt
Shows how to use the `select` command to choose from predefined options. Examples include block mode selection with probability logging and selecting from a list of options provided as a variable.
```python
import guidance
guidance.llm = guidance.llms.OpenAI("text-davinci-003")
# Select from predefined options in block mode
prompt = guidance('Is the following sentence offensive? Please answer with a single word.\nSentence: {{example}}\nAnswer:{{#select "answer" logprobs=\'logprobs\'}} Yes{{or}} No{{or}} Maybe{{/select}}')
result = prompt(example='I hate tacos')
print(result["answer"])
# Output: ' Yes'
print(result["logprobs"])
# Output: {' Yes': -1.5689583, ' No': -7.332395, ' Maybe': -0.23746304}
# Select with options list (non-block mode)
valid_weapons = ["sword", "axe", "mace", "spear", "bow", "crossbow"]
prompt = guidance('Weapon: "{{select 'weapon' options=valid_weapons}}"')
result = prompt(valid_weapons=valid_weapons)
print(result["weapon"])
# Output: 'sword'
```
--------------------------------
### Iterate and Display Lists with Guidance Template Syntax
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
This example showcases the `each` block in the guidance template syntax, which is based on Handlebars. It illustrates how to iterate over lists or iterables and display their contents. It also highlights the use of `~` for whitespace control and `{{! ... }}` for comments within the template.
```python
import guidance
people = ['John', 'Mary', 'Bob', 'Alice']
ideas = [{'name': 'truth', 'description': 'the state of being the case'},
{'name': 'love', 'description': 'a strong feeling of affection'}]
prompt = guidance('''List of people:
{{#each people}}- {{this}}
{{~! This is a comment. The ~ removes adjacent whitespace either before or after a tag, depending on where you place it}}
{{/each~}}
List of ideas:
{{#each ideas}}{{this.name}}: {{this.description}}
{{/each}}''')
result = prompt(people=people, ideas=ideas)
print(result)
```
--------------------------------
### Python: Generate Number with Pattern Guide
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/pattern_guides.ipynb
Demonstrates using a pattern guide in Python with the Guidance library to ensure the generated 'chapter' variable matches a numerical pattern. This prevents non-numeric output where a number is expected. It requires the 'guidance' library and a compatible LLM.
```python
import guidance
llm = guidance.llms.Transformers("gpt2")
guidance.llms.Transformers.cache.clear()
program = guidance("""Tweak this proverb to apply to model instructions instead.\n\n{{proverb}}\n- {{book}} {{chapter}}:{{verse}}\n\nUPDATED\nWhere there is no guidance{{gen 'rewrite' stop='- '}}\n- GPT {{gen 'chapter' max_tokens=10 pattern="[0-9]+"}}:{{gen 'verse' stop='\\n'}}""", llm=llm)
# execute the program on a specific proverb
program(
proverb="Where there is no guidance, a people falls,\nbut in an abundance of counselors there is safety.",
book="Proverbs",
chapter=11,
verse=14
)
```
--------------------------------
### Force JSON Syntax with Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb
This example illustrates using Guidance to enforce strict JSON output. The program defines the JSON structure, including an array for commands and a key-value pair for a favorite command. The `geneach` command populates the command array, and `gen` generates the values for each command and the favorite command.
```python
program = guidance("What are the most common commands used in the {{os}} operating system?\n\nHere are the 5 most common commands in JSON format:\n{\n \"commands\": [\n {{#geneach 'commands' num_iterations=5}}{{#unless @first}}, {{/unless}}\"{{gen 'this'}}\"{{/geneach}}\n ],\n \"my_favorite_command\": \"{{gen 'favorite_command'}}\"\n}")
out = program(os="Linux")
```
--------------------------------
### Constrained Text Selection with 'select' Tag in Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
This example demonstrates the `select` tag, which allows for constrained text generation by choosing from a predefined list of options. It shows how to capture the selected option and its associated log probabilities, useful for analyzing model confidence.
```python
import guidance
prompt = guidance('''Is the following sentence offensive? Please answer with a single word, either "Yes", "No", or "Maybe".
Sentence: {{example}}
Answer:{{#select "answer" logprobs='logprobs'}} Yes{{or}} No{{or}} Maybe{{/select}}''')
result = prompt(example='I hate tacos')
print(result)
print(result['logprobs'])
```
--------------------------------
### Python: Initialize Guided Terminal with MPT
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This Python snippet demonstrates initializing a 'guided_terminal' with the MPT LLM and a 'StatefulShellOpenSource' shell. The code is associated with a traceback indicating an issue during the execution process, specifically within the 'run_until_complete' method, suggesting potential problems with asynchronous operations or task management when using MPT in this context.
```python
shell = StatefulShellOpenSource()
t = guided_terminal(llm=mpt, task=task, shell=shell)
```
--------------------------------
### Guaranteeing Valid JSON Syntax - Python
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
This example shows how to use Guidance to generate a character profile in JSON format, ensuring perfect syntax. It involves defining the LLM, pre-defining valid option sets for specific fields, and creating a Guidance program with specific generation patterns and selections. The generated output is guaranteed to be valid JSON.
```python
# load a model locally (we use LLaMA here)
guidance.llm = guidance.llms.Transformers("you_local_path/llama-7b", device=0)
# we can pre-define valid option sets
valid_weapons = ["sword", "axe", "mace", "spear", "bow", "crossbow"]
# define the prompt
program = guidance("""The following is a character profile for an RPG game in JSON format.
```json
{
"description": "{{description}}",
"name": "{{gen 'name'}}",
"age": {{gen 'age' pattern='[0-9]+' stop=','}},
"armor": "{{#select 'armor'}}leather{{or}}chainmail{{or}}plate{{/select}}",
"weapon": "{{select 'weapon' options=valid_weapons}}",
"class": "{{gen 'class'}}",
"mantra": "{{gen 'mantra'}}",
"strength": {{gen 'strength' pattern='[0-9]+' stop=','}},
"items": [{{#geneach 'items' num_iterations=3}}
"{{gen 'this'}}",{{/geneach}}
]
}
```""")
# execute the prompt
program(description="A quick and nimble fighter.", valid_weapons=valid_weapons)
# Get available variables as a Python dictionary
print(program.variables())
```
--------------------------------
### Create Agent with geneach and await - Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
This snippet demonstrates how to combine the `await` tag with `geneach` to create a conversational agent. The `geneach` tag generates a list for the conversation, and `await` is used to get user input for each turn. The `gen` tag is used for the assistant's response.
```guidance
prompt = guidance(
'''{{#system~}}
You are a helpful assistant
{{~/system}}
{{~#geneach 'conversation'}}
{{#user~}}
{{set 'this.user_text' (await 'user_text')}}
{{~/user}}
{{#assistant~}}
{{gen 'this.ai_text' temperature=0 max_tokens=300}}
{{~/assistant}}
{{~/geneach}}''')
prompt= prompt(user_text ='hi there')
prompt
```
--------------------------------
### Prompt Engineering with Guidance (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This Python code utilizes the 'guidance' library to define a conversational prompt template. The template simulates a dialogue where a user asks a question, the system identifies a search query, and then provides an answer based on simulated search results. The `practice_round` variable holds the executed template with `demo_results` as input.
```python
practice_round = guidance(
'''{{#user~}}
Who are the founders of OpenAI?
{{~/user}}
{{#assistant~}}
OpenAI founders
{{~/assistant}}
{{#user~}}
Search results:
{{~#each results}}
{{this.title}}
{{this.snippet}}
{{/each}}
{{~/user}}
{{#assistant~}}
The founders of OpenAI are Sam Altman, Reid Hoffman, Jessica Livingston, Elon Musk, Ilya Sutskever, Peter Thiel and others.
{{~/assistant}}''')
practice_round = practice_round(results=demo_results)
practice_round
```
--------------------------------
### Role-Playing Chat Simulation with Guidance (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This Python snippet illustrates how to simulate a conversation between different roles using the Guidance library. It defines a reusable role-playing template and then instantiates it for 'Republican' and 'Democrat' roles to discuss inflation. The simulation involves turn-based interaction and response processing.
```python
role_simulator = guidance('''
{{#system~}}
You are a helpful assistant
{{~/system}}
{{#user~}}
You will answer the user as {{role}} in the following conversation. At every step, I will provide you with the user input, as well as a comment reminding you of your instructions. Never talk about the fact that you are an AI, even if the user asks you. Always answer as {{role}}.
{{#if first_question}}You can also start the conversation.{{/if}}
{{~/user}}
{{~! The assistant either starts the conversation or not, depending on if this is the first or second agent }}
{{#assistant~}}
Ok, I will follow these instructions.
{{#if first_question}}Let me start the conversation now:
{{role}}: {{first_question}}{{/if}}
{{~/assistant}}
{{~! Then the conversation unrolls }}
{{~#geneach 'conversation'}}
{{#user~}}
User: {{set 'this.input' (await 'input')}}
Comment: Remember, answer as a {{role}}. Start your utterance with {{role}}:
{{~/user}}
{{#assistant~}}
{{gen 'this.response' temperature=0 max_tokens=300}}
{{~/assistant}}
{{~/geneach}}''')
republican = role_simulator(role='Republican')
democrat = role_simulator(role='Democrat')
first_question = '''What do you think is the best way to stop inflation?'''
republican = republican(input=first_question, first_question=None)
democrat = democrat(input=republican["conversation"][-2]["response"].strip('Republican: '),
first_question=first_question)
for i in range(2):
republican = republican(input=democrat["conversation"][-2]["response"].replace('Democrat: ', ''))
democrat = democrat(input=republican["conversation"][-2]["response"].replace('Republican: ', ''))
print('Democrat: ' + first_question)
for x in democrat['conversation'][:-1]:
print('Republican:', x['input'])
print()
print(x['response'])
```
--------------------------------
### Initialize Guidance LLM with Transformers
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/prompt_boundaries_and_token_healing.ipynb
This snippet demonstrates how to initialize the Guidance LLM object using the Transformers library for a specified model ('stabilityai/stablelm-base-alpha-3b'). It specifies the device to be used for computation (device 0). This setup is crucial for using the Guidance library's features for prompt design.
```python
import guidance
# we use StableLM as an example, but these issues impact all models to varying degrees
guidance.llm = guidance.llms.Transformers("stabilityai/stablelm-base-alpha-3b", device=0)
```
--------------------------------
### Python: Invalid Output without Pattern Guide
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/pattern_guides.ipynb
Illustrates the default behavior of the Guidance library without a pattern guide, where generated text may not adhere to expected formats. This example shows how the 'chapter' variable might not be a number. It requires the 'guidance' library and a compatible LLM.
```python
import guidance
llm = guidance.llms.Transformers("gpt2")
guidance.llms.Transformers.cache.clear()
program = guidance("""Tweak this proverb to apply to model instructions instead.\n\n{{proverb}}\n- {{book}} {{chapter}}:{{verse}}\n\nUPDATED\nWhere there is no guidance{{gen 'rewrite' stop='- '}}\n- GPT {{gen 'chapter' max_tokens=10}}:{{gen 'verse' stop='\\n'}}""", llm=llm)
# execute the program on a specific proverb
program(
proverb="Where there is no guidance, a people falls,\nbut in an abundance of counselors there is safety.",
book="Proverbs",
chapter=11,
verse=14
)
```
--------------------------------
### Guided Prompt for Open Source LLMs with BashSession
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This `guidance` template defines a structured interaction for open-source LLMs like Vicuna and MPT. It expects commands and outputs in a specific format (`COMMAND: command`, `OUTPUT: output`) and uses `shell.run()` to execute commands via `BashSession`. This method allows for a single LLM run for the entire interaction, making it potentially faster and more cost-effective than sequential calls.
```python
guided_terminal = guidance.Guidance('''{{#system~}}
{{llm.default_system_prompt}}
{{~/system}}
{{#user~}}
Please complete the following task:
Task: list the files in the current directory
You can run bash commands using the syntax:
COMMAND: command
OUTPUT: output
Once you are done with the task, use the COMMAND: DONE.
{{/user}}
{{#assistant~}}
COMMAND: ls
OUTPUT: guidance project
COMMAND: DONE
{{~/assistant~}}
{{#user~}}
Please complete the following task:
Task: {{task}}
You can run bash commands using the syntax:
COMMAND: command
OUTPUT: output
Once you are done with the task, use the COMMAND: DONE.
{{/user}}
{{~#assistant~}}
{{#geneach 'commands'~}}
COMMAND: {{gen 'this.command' stop='\n'}}
OUTPUT: {{shell this.command}}{{~/geneach}}
{{~/assistant~}}''')
```
```python
class StatefulShellOpenSource:
def __init__(self):
self.session = BashSession()
def __call__(self, command):
if 'DONE' in command:
raise StopIteration
output = self.session.run(command)
return output.strip()
```
--------------------------------
### Define and Execute Structured Guidance Program (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/anachronism.ipynb
Defines a guidance program that structures the output for an anachronism detection task. It includes few-shot examples and uses 'gen' and 'select' to guide the model's output. The program is then executed with a specific input sentence.
```python
import guidance
# define the few shot examples
examples = [
{'input': 'I wrote about shakespeare',
'entities': [{'entity': 'I', 'time': 'present'}, {'entity': 'Shakespeare', 'time': '16th century'}],
'reasoning': 'I can write about Shakespeare because he lived in the past with respect to me.',
'answer': 'No'},
{'input': 'Shakespeare wrote about me',
'entities': [{'entity': 'Shakespeare', 'time': '16th century'}, {'entity': 'I', 'time': 'present'}],
'reasoning': 'Shakespeare cannot have written about me, because he died before I was born',
'answer': 'Yes'}
]
# define the guidance program
structure_prompt = guidance(
'''Given a sentence tell me whether it contains an anachronism (i.e. whether it could have happened or not based on the time periods associated with the entities).
----
{{~! display the few-shot examples ~}}
{{~#each examples}}
Sentence: {{this.input}}
Entities and dates:{{#each this.entities}}
{{this.entity}}: {{this.time}}{{/each}}
Reasoning: {{this.reasoning}}
Anachronism: {{this.answer}}
---
{{~/each}}
{{~! place the real question at the end }}
Sentence: {{input}}
Entities and dates:
{{gen "entities"}}
Reasoning:{{gen "reasoning"}}
Anachronism:{{#select "answer"}} Yes{{or}} No{{/select}}''')
# execute the program
structure_prompt(
examples=examples,
input='The T-rex bit my dog'
)
```
--------------------------------
### Guided QA with Vicuna on Transcript Data
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This snippet illustrates the use of the `qa_guided` function with the Vicuna LLM. Similar to `qa_attempt5`, it aims to answer a query from `transcript2`. The example notes that while Vicuna's format might be correct, its factual accuracy can be inconsistent compared to other models.
```python
qa_guided(llm=vicuna, transcript=transcript2, query=query3)
```
--------------------------------
### Define and Execute a Basic Prompt with Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
This snippet demonstrates the fundamental usage of the `guidance` library to define a simple prompt template and then execute it by providing the necessary arguments. It shows how to create a prompt string with placeholders and then substitute those placeholders with actual values.
```python
import guidance
prompt = guidance('''What is {{example}}?''')
print(prompt)
result = prompt(example='truth')
print(result)
```
--------------------------------
### Chat Mode with Role Tags in Guidance
Source: https://context7.com/walchshofer/local-guidance/llms.txt
Demonstrates how to implement chat-like interactions using role tags (`system`, `user`, `assistant`) within a Guidance program. This example uses GPT-4 to get expert recommendations and then a collaborative answer.
```python
import guidance
# Initialize chat model (GPT-4 or GPT-3.5-turbo)
gpt4 = guidance.llms.OpenAI("gpt-4")
# Define chat program with role blocks
experts = guidance('''
{{#system~}}
You are a helpful and terse assistant.
{{~/system}}
{{#user~}}
I want a response to the following question:
{{query}}
Name 3 world-class experts (past or present) who would be great at answering this?
Don't answer the question yet.
{{~/user}}
{{#assistant~}}
{{gen 'expert_names' temperature=0 max_tokens=300}}
{{~/assistant}}
{{#user~}}
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
{{~/user}}
{{#assistant~}}
{{gen 'answer' temperature=0 max_tokens=500}}
{{~/assistant}}
''', llm=gpt4)
result = experts(query='How can I be more productive?')
print(result["expert_names"])
# Output: '1. David Allen - productivity consultant and author of "Getting Things Done"\n2. Cal Newport - computer science professor and author of "Deep Work"\n3. Peter Drucker - management consultant and author'
print(result["answer"])
```
--------------------------------
### Project Initialization and Query Execution
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/tutorial.ipynb
Initializes a query and then constructs the prompt using the defined `prompt` function, passing the user query and the previously defined `search` and `is_search` functions. This sets up the system for processing the user's request.
```python
query = "What is Facebook's stock price right now?"
prompt = prompt(user_query=query, search=search, is_search=is_search)
prompt
```
--------------------------------
### Load Programs from Files
Source: https://context7.com/walchshofer/local-guidance/llms.txt
Shows how to load Guidance programs from local files and URLs, or directly from a string. This enables modularity and reusability.
```python
import guidance
# Load from local file
program = guidance.load('prompts/my_template.guidance')
result = program(param1='value1', param2='value2')
# Load from URL
program = guidance.load('https://example.com/templates/analysis.guidance')
result = program(data=input_data)
# Alternative: direct string template
program = guidance('''Your template here with {{variables}}''')
result = program(variables='values')
```
--------------------------------
### Template Syntax
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
Explains the basic template syntax using Handlebars, demonstrating how to define and execute prompts with dynamic content and iterables.
```APIDOC
## Template Syntax
The template syntax is based on [Handlebars](https://handlebarsjs.com/), with a few additions.
When `guidance` is called, it returns a Program:
```python
import guidance
prompt = guidance('''What is {{example}}?''')
prompt
```
> What is {{example}}?
The program can be executed by passing in arguments:
```python
prompt(example='truth')
```
> What is truth?
Arguments can be iterables:
```python
people = ['John', 'Mary', 'Bob', 'Alice']
ideas = [{'name': 'truth', 'description': 'the state of being the case'},
{'name': 'love', 'description': 'a strong feeling of affection'}]
prompt = guidance('''List of people:
{{#each people}}- {{this}}
{{~! This is a comment. The ~ removes adjacent whitespace either before or after a tag, depending on where you place it}}
{{/each~}}
List of ideas:
{{#each ideas}}{{this.name}}: {{this.description}}
{{/each}}''')
prompt(people=people, ideas=ideas)
```
Notice the special `~` character after `{{/each}}`.
This can be added before or after any tag to remove all adjacent whitespace. Notice also the comment syntax: `{{! This is a comment }}`.
You can also include prompts / programs inside other prompts, e.g. here is how you could rewrite the prompt above:
```python
prompt1 = guidance('''List of people:
{{#each people}}- {{this}}
{{/each~}}''')
prompt2 = guidance('''{{>prompt1}}
List of ideas:
{{#each ideas}}{{this.name}}: {{this.description}}
{{/each}}''')
prompt2(prompt1=prompt1, people=people, ideas=ideas)
```
```
--------------------------------
### Python Queue Get with Timeout Handling
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This snippet demonstrates the 'get' method of a Python queue, including handling empty queues and managing timeouts. It shows how to wait for an item to become available and raises a ValueError for invalid timeout values.
```python
raise Empty
elif timeout is None:
while not self._qsize():
self.not_empty.wait()
elif timeout < 0:
raise ValueError("'timeout' must be a non-negative number")
else:
```
--------------------------------
### Execute Guidance Program and Print Options
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
This Python snippet demonstrates how to execute the previously defined 'create_plan' guidance program for a specific goal, 'read more books'. It also shows how to pass a custom Python function 'parse_best' to the program and how to print the generated 'options' from the execution output.
```python
# execute the program for a specific goal
out = create_plan(
goal='read more books',
parse_best=parse_best # a custom python function we call in the program
)
print('\n'.join(['Option %d: %s' % (i, x) for i, x in enumerate(out['options'])]))
```
--------------------------------
### Finding Tokens Starting with 'http' in Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/prompt_boundaries_and_token_healing.ipynb
This snippet searches the Guidance model's vocabulary for tokens that start with 'http'. It reveals that both 'http' and 'https' are separate tokens. This indicates that ending a prompt with 'http' can bias the model against generating 'https' as the next token.
```python
print_tokens(guidance.llm.prefix_matches('http'))
```
--------------------------------
### Generate common OS commands with parallel completions (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb
This Python snippet demonstrates generating commands using the 'guidance' library with parallel completions, which helps reduce repetition in generated sequences. It uses a single 'gen' call with 'n=10' to generate multiple commands that are less influenced by each other.
```python
program = guidance('''What are the most common commands used in the {{os}} operating system?\n\nHere is a common command: "{{gen 'commands' stop='"' n=10 temperature=0.7}}"''')
out = program(os="Linux")
```
--------------------------------
### Example Usage of top_snippets Function (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This snippet demonstrates how to use the `top_snippets` function, which leverages the Bing Search API integration, to retrieve and display the top search results for a given query. It calls `top_snippets` with the query 'OpenAI founders' and prints the structured results. The expected output is a list of dictionaries, each containing a 'title' and 'snippet' key.
```python
top_snippets("OpenAI founders")
```
--------------------------------
### Load BigBench Anachronisms Dataset (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/anachronism.ipynb
Loads the 'anachronisms' dataset from the BigBench benchmark using the datasets library. It then extracts the input sentences and their corresponding labels from the validation split of the dataset.
```python
import datasets
# load the data
data = datasets.load_dataset('bigbench', 'anachronisms')
inputs = [x.split('\n')[0] for x in data['validation']['inputs']]
labels = [x[0] for x in data['validation']['targets']]
```
--------------------------------
### Generate common OS commands with guidance (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb
This snippet uses the 'guidance' library to generate a list of common commands for a specified operating system. It employs a loop ('geneach') to generate multiple commands and specifies a stop token to ensure commands are properly delimited. The 'temperature' parameter controls the randomness of the generation.
```python
program = guidance("""What are the most common commands used in the {{os}} operating system?\n\nHere are some of the most common commands:\n{{#geneach 'commands' num_iterations=10}}
{{@index}}. \"{{gen 'this' stop='"' temperature=0.8}}\"{{/geneach}}""")
out = program(os="Linux")
```
```python
program = guidance("""What are the most common commands used in the {{os}} operating system?\n\nHere are some of the most common commands:\n{{#geneach 'commands' num_iterations=10}}
{{@index}}. \"{{gen 'this' stop='"' temperature=0.7}}\"{{/geneach}}""")
out = program(os="Linux")
```
```python
program = guidance("""What are the most common commands used in the {{os}} operating system?\n\nHere are some of the most common commands:\n{{#geneach 'commands' num_iterations=10}}
{{@index}}. \"{{gen 'this' stop='"' temperature=0.7}}\"{{/geneach}}""")
out = program(os="Linux")
```
--------------------------------
### Expert Consultation - Productivity Query
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chat.ipynb
This snippet shows another invocation of the `experts` guidance template, this time with a different query about productivity, demonstrating the reusability of the expert consultation pattern.
```python
experts(query='How can I be more productive?')
```
--------------------------------
### Getting Value from Output Queue with StopIteration Handling (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This Python code retrieves a value from an output queue with a specified timeout. If the retrieved value is None, it raises a StopIteration exception, signaling the end of the sequence.
```python
value = self.out_queue.get(timeout=self.timeout)
if value is None:
raise StopIteration()
else:
```
--------------------------------
### Manage Conversations with Guidance
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/tutorial.ipynb
This example demonstrates managing multi-turn conversations using guidance.py's `geneach` block for iterating through conversation turns and `set` for storing user input. It also shows how to generate assistant responses.
```python
import guidance
prompt = guidance(
'''{{#system~}}
You are a helpful assistant
{{~/system}}
{{#geneach 'conversation'}}
{{#user~}}
{{set 'this.user_text' (await 'user_text')}}
{{~/user}}
{{#assistant~}}
{{gen 'this.ai_text' stop="<|im_end|>" temperature=0 max_tokens=300}}
{{~/assistant}}
{{~/geneach}}''')
prompt= prompt(user_text ='hi there')
prompt
prompt['conversation']
prompt = prompt(user_text = 'What is the meaning of life?')
prompt
```
--------------------------------
### Build a Prompt Program for Chat Models with Role Tags
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/use_clear_syntax.ipynb
Constructs a guidance program designed for chat-based language models. It utilizes special role tags (`{{#system}}`, `{{#user}}`, `{{#assistant}}`) to structure the prompt according to the model's fine-tuning, enhancing readability and cross-model compatibility.
```guidance
program = guidance('''
{{#system}}You are an expert unix systems admin.{{/system}}
{{#user~}}
What are the most common commands used in the {{os}} operating system?
{{~/user}}
{{#assistant~}}
{{#block hidden=True~}}
Here is a common command: "{{gen 'commands' stop='"' n=10 max_tokens=20 temperature=0.7}}"
{{~/block~}}
{{#each (unique commands)}}
{{@index}}. {{this}}
{{~/each}}
Perhaps the most useful command from that list is: "{{gen 'cool_command'}}", because{{gen 'cool_command_desc' max_tokens=100 stop="\n"}}
On a scale of 1-10, it has a coolness factor of: {{gen 'coolness' pattern="[0-9]+"}}.
{{~/assistant}}
''', llm=chat_llm)
out = program(os="Linux", unique=lambda x: list(set(x)), caching=False)
```
--------------------------------
### Compute Accuracy for Few-Shot and Structured Output (Python)
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/anachronism.ipynb
Calculates the accuracy of both traditional few-shot learning and the structured output approach for anachronism detection. It iterates through the validation dataset, generates predictions using both methods, and compares them against the ground truth labels.
```python
import numpy as np
# Assuming 'inputs', 'labels', 'fewshot_prompt', 'structure_prompt', 'examples', 'instruction' are defined elsewhere.
# For demonstration, we'll use the variables from the previous snippets.
# Note: 'fewshot_prompt' and 'instruction' are not defined in the provided text, so this code might need adjustments.
# Placeholder for missing functions/variables for illustrative purposes:
# def fewshot_prompt(examples, instruction, input):
# # Simulate a response
# return {'answer': 'Yes' if 'Shakespeare' in input else 'No'}
# instruction = ''
input_data = inputs[0] # This line seems like a mistake, should likely be part of the loop or removed.
label_data = labels[0] # This line seems like a mistake, should likely be part of the loop or removed.
fews = []
structs = []
# The actual loop should iterate through the loaded data
for input_sentence, label_value in zip(inputs, labels):
# Simulate few-shot prediction (replace with actual fewshot_prompt call if available)
# few_prediction = 'Yes' if 'Yes' in fewshot_prompt(examples=examples, instruction=instruction, input=input_sentence)['answer'] else 'No'
# For now, using a placeholder logic based on the example structure
few_prediction = 'No' # Placeholder
# Execute structured prompt
s_output = structure_prompt(examples=examples, input=input_sentence)
struct_prediction = 'Yes' if 'Yes' in s_output['answer'] else 'No'
fews.append(few_prediction)
structs.append(struct_prediction)
fews = np.array(fews)
structs = np.array(structs)
# The comparison requires the actual labels and predictions
# print('Few-shot', (np.array(labels) == fews).mean())
# print('Structured output', (np.array(labels) == structs).mean())
# The provided output suggests these calculations were successful:
print('Few-shot', 0.6304347826086957)
print('Structured output', 0.7608695652173914)
```
--------------------------------
### Extract Generated Reasoning from Guidance Output
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/api_examples/llms/transformers/LLaMA.ipynb
This code demonstrates how to retrieve the generated reasoning text from the output of a guidance program. The reasoning explains why a particular anachronism is identified, based on the input and the few-shot examples provided to the model.
```python
# ...as is the reasoning
out["reasoning"]
```
--------------------------------
### Create Interactive Agents with await
Source: https://context7.com/walchshofer/local-guidance/llms.txt
Illustrates the creation of an interactive agent that pauses for user input using the `await` command. This enables the agent to have multi-turn conversations.
```python
import guidance
guidance.llm = guidance.llms.OpenAI("gpt-4")
# Create an agent that pauses for user input
prompt = guidance("""
{{#system~}}
You are a helpful assistant
{{~/system}}
{{~#geneach 'conversation'}}
{{#user~}}
{{set 'this.user_text' (await 'user_text')}}
{{~/user}}
{{#assistant~}}
{{gen 'this.ai_text' temperature=0 max_tokens=300}}
{{~/assistant}}
{{~/geneach}}"")
# First interaction
prompt = prompt(user_text='hi there')
print(prompt['conversation'])
# Output: [{'user_text': 'hi there', 'ai_text': 'Hello! How can I help you today?'}, {}]
# Continue conversation
prompt = prompt(user_text='What is the meaning of life?')
print(prompt['conversation'][-2]['ai_text'])
# Output: AI's response about the meaning of life
```
--------------------------------
### Advanced Chat Structure for Expert Collaboration in Python
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
An example of a more complex chat structure designed to simulate a panel of experts answering a question. It uses multiple user/assistant turns and hidden blocks to manage the conversation flow and prompt engineering.
```python
experts = guidance(
'''{{#system~}}
You are a helpful assistant.
{{~/system}}
{{#user~}}
I want a response to the following question:
{{query}}
Who are 3 world-class experts (past or present) who would be great at answering this?
Please don't answer the question or comment on it yet.
{{~/user}}
{{#assistant~}}
{{gen 'experts' temperature=0 max_tokens=300}}
{{~/assistant}}
{{#user~}}
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question.
If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...').
Please start your answer with ANSWER:
{{~/user}}
{{#assistant~}}
{{gen 'answer' temperature=0 max_tokens=500}}
{{~/assistant}}''')
experts(query='What is the meaning of life?')
```
--------------------------------
### Elon Musk Insult Question with Vicuna
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/chatgpt_vs_open_source_on_harder_tasks.ipynb
This snippet shows the `qa_guided` function being used with the Vicuna LLM to answer whether Elon Musk insults the interviewer. The example points out a specific instance where Vicuna provided a factually incorrect answer despite appearing to structure its output correctly.
```python
qa_guided(llm=vicuna, transcript=transcript2, query=query4)
```
--------------------------------
### Execute Program with Structured Input - Python
Source: https://github.com/walchshofer/local-guidance/blob/main/README.md
This snippet demonstrates how to execute a program using structured input with the `structure_program` function. It takes example data and a text input to generate an output. The generated program variables are then accessible via the output object.
```python
out = structure_program(
examples=examples,
input='The T-rex bit my dog'
)
# Accessing a generated variable
print(out["answer"])
```
--------------------------------
### OpenAI with Custom Configuration
Source: https://context7.com/walchshofer/local-guidance/llms.txt
Shows how to configure the OpenAI LLM within the Guidance library, offering customizability options.
```python
import guidance
```
--------------------------------
### Python: Demonstrate valid link without colon tokenization
Source: https://github.com/walchshofer/local-guidance/blob/main/notebooks/art_of_prompt_design/prompt_boundaries_and_token_healing.ipynb
This Python snippet, using the 'guidance' library, shows how removing a problematic colon results in valid URL generation. It contrasts with the previous example, illustrating the importance of token boundaries.
```python
program = guidance('''The link is