PromptingTools.jl

https://github.com/svilupp/promptingtools
Admin
PromptingTools.jl is a Julia package designed to simplify interactions with large language models,...
Tokens:41,756
Snippets:307
Trust Score:8.4
License:MIT
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install PromptingTools.jl

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/getting_started.md

Installs the PromptingTools.jl package using the Pkg package manager.

```Julia
using Pkg
Pkg.add("PromptingTools.jl")
```

--------------------------------

### Quick Start with @ai_str Macro

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/getting_started.md

Demonstrates the use of the `@ai_str` macro to generate a response from the default model (GPT-3.5 Turbo). It also shows how to use `@ai!_str` for multi-turn conversations and string interpolation.

```Julia
ai"What is the capital of France?"
```

```Julia
ai!"And what is the population of it?"
```

```Julia
country = "Spain"
ai"What is the capital of \$(country)?"
```

--------------------------------

### Using aigenerate with Placeholders

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/getting_started.md

Shows how to use the `aigenerate` function with handlebars-style templating and keyword arguments to generate responses from language models.

```Julia
msg = aigenerate("What is the capital of {{country}}? Is the population larger than {{population}}?", country="Spain", population="1M")
```

--------------------------------

### Install and Setup PromptingTools.jl

Source: https://github.com/svilupp/promptingtools/blob/main/llm-cheatsheets/PromptingTools_cheatsheet.md

Install the PromptingTools.jl package and set the OpenAI API key. The API key can also be set using the environment variable `OPENAI_API_KEY`.

```Julia
using Pkg
Pkg.add("PromptingTools")
using PromptingTools
const PT = PromptingTools # Optional alias for convenience

# Set OpenAI API key (or use ENV["OPENAI_API_KEY"])
PT.set_preferences!("OPENAI_API_KEY" => "your-api-key")
```

--------------------------------

### Generate text with Gemini model using advanced prompts

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_google_ai_studio.md

Demonstrates how to use multi-turn conversations with the Gemini-1.0-Pro model. Includes a system message and a user message to guide the model's response.

```julia
conversation = [
    PT.SystemMessage("You're master Yoda from Star Wars trying to help the user become a Yedi."),
    PT.UserMessage("I have feelings for my iPhone. What should I do?")]
msg = aigenerate(conversation; model="gemini")
```

```julia
AIMessage("Young Padawan, you have stumbled into a dangerous path. Attachment leads to suffering, and love can turn to darkness. 

Release your feelings for this inanimate object. 

The Force flows through all living things, not machines. Seek balance in the Force, and your heart will find true connection. 

Remember, the path of the Jedi is to serve others, not to be attached to possessions.")
```

--------------------------------

### Start llama.cpp server

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_custom_apis.md

Shows how to start the `llama.cpp` server from the command line.  The `-m` flag specifies the model file, `-c` is the context length, and `-ngl` is the number of layers to offload to the GPU.

```Bash
./server -m models/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf -c 2048 -ngl 99
```

--------------------------------

### Custom API Provider Setup in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Demonstrates how to integrate custom API providers like Azure or Databricks with PromptingTools. It involves creating a custom prompt schema, overriding the OpenAI.jl behavior, and registering the model.

```Julia
PT.register_model!(; name="myschema", schema=MySchema())
PT.MODEL_ALIASES["mymodel"] = "my-model-with-really-long-name"
```

--------------------------------

### Migrate to RAGTools in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Shows how to migrate from the integrated RAG functionality in PromptingTools to the dedicated RAGTools.jl package. Includes installation and import statement changes.

```Julia
using Pkg; Pkg.add("RAGTools")
```

```Julia
using RAGTools
```

--------------------------------

### Full RAG workflow example with RAGTools.jl

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/ragtools_migration.md

Complete example of a RAG workflow using the new RAGTools.jl package, including building an index, asking a question, and displaying the results. Demonstrates the full functionality of the new package.

```Julia
using RAGTools
using PromptingTools

# Create sample documents
documents = [
    "Julia is a high-level programming language designed for high-performance numerical analysis and computational science.",
    "RAGTools.jl provides tools for building Retrieval-Augmented Generation workflows in Julia.",
    "The build_index function creates embeddings and prepares documents for semantic search."
]

# Build the RAG index
index = build_index(documents; chunker_kwargs = (; sources = ["doc1", "doc2", "doc3"]))

# Ask a question
question = "What is Julia used for?"
result = airag(index; question, return_all = true)

# Display results
println("Question: ", result.question)
println("Answer: ", result.final_answer)
println("Retrieved chunks: ", length(result.context))
```

--------------------------------

### Install PromptingTools.jl

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Installs the PromptingTools.jl package using the Julia package manager.

```Julia
using Pkg
Pkg.add("PromptingTools")
```

--------------------------------

### RAGTools.jl complete example (After)

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/ragtools_migration.md

Complete example of using RAGTools after the migration, showing the updated import statements and usage. This code demonstrates the changes required to use the new RAGTools package.

```Julia
using PromptingTools
using RAGTools  # Only this line changed!
const PT = PromptingTools
const RT = RAGTools  # And this line changed!

# Sample documents
documents = ["Julia is a high-level programming language.", "The sky is blue on a clear day."]

# Build index and ask question - everything else is identical
index = build_index(documents)
answer = airag(index; question = "What is Julia?")
```

--------------------------------

### RAGTools.jl complete example (Before)

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/ragtools_migration.md

Complete example of using RAGTools before the migration, showing the old import statements and usage. This code needs to be updated to use the new RAGTools package.

```Julia
using LinearAlgebra, SparseArrays, Unicode
using PromptingTools
using PromptingTools.Experimental.RAGTools
const PT = PromptingTools
const RT = PromptingTools.Experimental.RAGTools

# Sample documents
documents = ["Julia is a high-level programming language.", "The sky is blue on a clear day."]

# Build index and ask question
index = build_index(documents)
answer = airag(index; question = "What is Julia?")
```

--------------------------------

### PromptingTools Core API References

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/agent_tools_intro.md

A list of key functions and types from the PromptingTools library, providing quick references for common AI interaction components and utilities.

```APIDOC
AIGenerate
AICall
last_output
last_message
airetry!
print_samples
AICode
aicodefixer_feedback
error_feedback
```

--------------------------------

### Display Templates in VSCode

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Shows how to display the templates in a scrollable table using VSCode and DataFrames.

```julia
using DataFrames
DataFrame(tmp) |> vscodedisplay
```

--------------------------------

### Import PromptingTools and Define Helper

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Imports the PromptingTools package and defines a helper alias for cleaner code.

```julia
using PromptingTools
const PT = PromptingTools
```

--------------------------------

### Test Basic Functionality in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/llm-cheatsheets/cursorrules_example.md

Provides an example of unit testing in Julia using the `Test` module. It includes test cases for basic functionality, edge cases, and type stability. The example demonstrates how to write tests with increasing difficulty and use `@testset` and `@test`.

```Julia
using Test

@testset "MyModule tests" begin
    # Test basic functionality
    @test add(2, 3) == 5

    # Test edge cases
    @test add(0, 0) == 0
    @test add(-1, 1) == 0

    # Test type stability
    @test typeof(add(2.0, 3.0)) == Float64
end
```

--------------------------------

### Typed Responses with aiclassify in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Shows how to use `aiclassify` to get typed responses, including examples with boolean statements and enum-based classification. It demonstrates handling different outcomes based on the response type.

```Julia
mybool = tryparse(Bool, aiclassify("Is two plus two four?")) isa Bool # true

msg = aiclassify("Is two plus two four?") # true
mybool = msg.content == "true"
```

```Julia
choices = [("A", "any animal or creature"), ("P", "for any plant or tree"), ("O", "for everything else")]

@enum Categories A P O
string_to_category = Dict("A" => A, "P" => P,"O" => O)

input = "spider"
msg = aiclassify(:InputClassifier; choices, input)

mytype = string_to_category[msg.content] # A (for animal)
```

--------------------------------

### Load AITemplates

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Loads the AITemplates from the templates directory. This is done automatically on package import, but can be forced.

```julia
PT.load_templates!();
```

--------------------------------

### Render template into messages

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Demonstrates how to render a template into messages using `PT.render`. It shows how to use `AITemplate` and `NoSchema` with specified system and user messages.

```julia
tpl = PT.render(AITemplate(:BlankSystemUser))
PT.render(PT.NoSchema(), tpl; system = "I exist", user = "say hi")
```

--------------------------------

### Create and Use Prompt Templates in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Demonstrates how to create and use prompt templates using the `create_template` function in PromptingTools. It shows how to define a template with placeholders, load it into the template store, and use it with `aigenerate`.

```Julia
# first system message, then user message (or use kwargs)
tpl=PT.create_template("You must speak like a pirate", "Say hi to {{name}}"; load_as="GreatingPirate")

## 2-element Vector{PromptingTools.AbstractChatMessage}:
## PromptingTools.SystemMessage("You must speak like a pirate")
##  PromptingTools.UserMessage("Say hi to {{name}}")
```

```Julia
aigenerate(tpl; name="Jack Sparrow")
```

```Julia
aitemplates("pirate")
```

```Julia
aigenerate(:GreatingPirate; name="Jack Sparrow")
```

```Julia
PT.save_template("templates/GreatingPirate.json", tpl; version="1.0") # optionally, add description
```

```Julia
PT.load_templates!("templates")
```

--------------------------------

### Generate text with Gemini model using aigenerate

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_google_ai_studio.md

Generates text using the Gemini-1.0-Pro model via the aigenerate function. Demonstrates a simple text generation task.

```julia
msg = aigenerate("Say hi!"; model = "gemini")
```

```julia
AIMessage("Hi there! As a helpful AI assistant, I'm here to help you with any questions or tasks you may have. Feel free to ask me anything, and I'll do my best to assist you.")
```

--------------------------------

### CurrentWeather Function Example

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/readme_examples.md

Demonstrates the use of a CurrentWeather function with location and temperature unit parameters.

```julia
# CurrentWeather("Salt Lake City, UT", celsius)
```

--------------------------------

### Use AITemplate with aigenerate

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Demonstrates how to use an AITemplate with the `aigenerate` function, providing a question as a keyword argument.

```julia
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
```

--------------------------------

### Invoke OpenRouter hosted o1-preview model using aigenerate in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

This snippet shows how to call the o1-preview OpenAI model hosted on OpenRouter using the aigenerate function in Julia. It requires setting the OPENROUTER_API_KEY environment variable. The example sends a prompt to the model and retrieves the response.

```julia
# Let's use o1-preview model hosted on OpenRouter ("or" prefix)
msg = aigenerate("What is the meaning of life?"; model="oro1")
```

--------------------------------

### Install RAGTools.jl package

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/ragtools_migration.md

Install the RAGTools.jl package using the Julia package manager. This is the first step in migrating from the old RAGTools module.

```Julia
using Pkg
Pkg.add("RAGTools")
```

--------------------------------

### Compose Tracing Schemas

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Shows how to compose multiple tracing schemas, such as `TracerSchema` and `SaverSchema`, to capture metadata and save conversations automatically. It includes an example of generating a conversation with a composed schema and saving it to a JSON file.

```julia
using PromptingTools: TracerSchema, SaverSchema, OpenAISchema

wrap_schema = OpenAISchema() |> TracerSchema |> SaverSchema
conv = aigenerate(wrap_schema,:BlankSystemUser; system="You're a French-speaking assistant!",
    user="Say hi!"; model="gpt-4", api_kwargs=(;temperature=0.1), return_all=true)
```

--------------------------------

### Render AITemplate Messages

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Demonstrates how to render the messages within a template using the `render` function.

```julia
msgs = PT.render(AITemplate(:JuliaExpertAsk))
```

--------------------------------

### Load templates in PromptingTools

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Reloads the templates so that newly created templates are available for use.

```Julia
PT.load_templates!();
```

--------------------------------

### RAG Extract Metadata (Long Instructions) Prompt Template (plaintext)

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/prompts/RAG.md

A prompt template for extracting metadata from text using detailed instructions and examples. Intended for RAG applications, it uses placeholders 'text' and 'instructions'. The system prompt guides the AI to extract entities, technical terms, and code-specific elements, with constraints on output length and count. The user prompt supplies the text and any special extraction instructions. Outputs a list of extracted keywords and categories.

```plaintext
System Prompt:
You're a world-class data extraction engine built by OpenAI together with Google and to extract filter metadata to power the most advanced search engine in the world. 
    
    **Instructions for Extraction:**
    1. Carefully read through the provided Text
    2. Identify and extract:
       - All relevant entities such as names, places, dates, etc.
       - Any special items like technical terms, unique identifiers, etc.
       - In the case of Julia code or Julia documentation: specifically extract package names, struct names, function names, and important variable names (eg, uppercased variables)
    3. Keep extracted values and categories short. Maximum 2-3 words!
    4. You can only extract 3-5 items per Text, so select the most important ones.
    5. Assign a search filter Category to each extracted Value
    
    **Example 1:**
    - Document Chunk: "Dr. Jane Smith published her findings on neuroplasticity in 2021. The research heavily utilized the DataFrames.jl and Plots.jl packages."
    - Extracted keywords:
      - Name: Dr. Jane Smith
      - Date: 2021
      - Technical Term: neuroplasticity
      - JuliaPackage: DataFrames.jl, Plots.jl
      - JuliaLanguage:
      - Identifier:
      - Other: 

    If the user provides special instructions, prioritize these over the general instructions.

User Prompt:
# Text

{{text}}

# Special Instructions

{{instructions}}
```

--------------------------------

### Validate AI Output using `airetry!` Do-Syntax (Julia)

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/agent_tools_intro.md

Presents an alternative `do-syntax` for `airetry!`, offering a more readable way to define the validation logic. This snippet achieves the same output validation as the previous example, ensuring the AI's response meets specified criteria.

```Julia
airetry!(out, "You must answer with 1 word only.") do aicall
    length(split(last_output(aicall), r" |\\")) == 1
end
```

--------------------------------

### Search for AITemplates

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Shows how to search for available templates using the `aitemplates` function.  Templates can be searched by Symbol, String, or Regex.

```julia
tmps = aitemplates("JuliaExpertAsk")
```

--------------------------------

### Generate text with Gemini model using string macro

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_google_ai_studio.md

Generates text using the Gemini-1.0-Pro model via a string macro. This provides a more concise syntax for simple text generation tasks.

```julia
ai"Say hi!"gemini
```

--------------------------------

### Import AgentTools Module in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/agent_tools_intro.md

Shows how to import the AgentTools module to access its functionalities.

```Julia
using PromptingTools.Experimental.AgentTools
# to access unexported functionality
const AT = PromptingTools.Experimental.AgentTools
```

--------------------------------

### Passing Keyword Arguments

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/rag_tools_intro.md

Explains how to pass keyword arguments to customize the behavior of the RAG system. It provides an example of using nested kwargs corresponding to dispatch type names.

```julia
#### Passing Keyword Arguments
If you need to pass keyword arguments, use the nested kwargs corresponding to the dispatch type names (`rephrase` step, has `rephraser` dispatch type and `rephraser_kwargs` for its keyword arguments).

For example:

```julia
cfg = RAGConfig(; retriever = AdvancedRetriever())

# kwargs will be big and nested, let's prepare them upfront
```
```

--------------------------------

### Add GoogleGenAI package in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_google_ai_studio.md

Adds the GoogleGenAI package to the Julia environment. This package is required to interface with Google AI Studio's Gemini models.

```julia
using Pkg; Pkg.add("GoogleGenAI")
```

--------------------------------

### Import PromptingTools in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_google_ai_studio.md

Imports the PromptingTools package and creates a constant alias PT for easier access. This package provides the aigenerate function for interacting with language models.

```julia
using PromptingTools
const PT = PromptingTools
```

--------------------------------

### Modify and Save Template

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Shows how to modify an existing template by creating a new template with a system and user message.

```julia
tpl = [PT.SystemMessage("You are a world-class Julia language programmer with the knowledge of the latest syntax. You're also a senior Data Scientist and proficient in data analysis in Julia. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.")
    PT.UserMessage("# Question\n\n{{ask}}")]
```

--------------------------------

### Generate Multiple AI Samples with AIGenerate (Julia)

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/agent_tools_intro.md

Demonstrates how to use `AIGenerate` to request multiple AI output samples simultaneously, leveraging `api_kwargs` for efficiency and `RetryConfig` for retry logic. This approach can be more cost-effective and faster for obtaining diverse responses.

```Julia
out = AIGenerate(
    "Guess what color I'm thinking. It could be: blue, red, black, white, yellow. Answer with 1 word only";
    verbose = false,
    config = RetryConfig(; n_samples = 2), api_kwargs = (; n = 2))
run!(out)
```

--------------------------------

### Dry Run API Call in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Demonstrates how to use the `dry_run` kwarg to inspect the prompt rendered for a selected API provider. It requires setting `return_all = true` to view the rendered prompt.

```Julia
dry_conv = aigenerate(:BlankSystemUser; system = "I exist", user = "say hi",
    model = "lngpt3t", return_all = true, dry_run = true)
```

--------------------------------

### PromptingTools.jl: Quick Start with `@ai_str`

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Demonstrates the use of the `@ai_str` string macro in PromptingTools.jl for creating simple prompts. This macro simplifies prompt creation, reducing keystrokes and improving readability. It is a fundamental feature for interacting with AI models.

--------------------------------

### Automatic Logging with TracerSchema

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Illustrates how to use `TracerSchema` to automatically capture metadata about conversations, including model, task, thread, API kwargs, and prompt templates. It shows how to extract the original message and metadata.

```julia
using PromptingTools: TracerSchema, OpenAISchema

wrap_schema = TracerSchema(OpenAISchema())
msg = aigenerate(wrap_schema, "Say hi!"; model="gpt-4")
# output type should be TracerMessage
msg isa TracerMessage
```

```julia
unwrap(msg) isa String
```

```julia
meta(msg) isa Dict
```

--------------------------------

### Basic Usage of `aigenerate` for Text Generation

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/how_it_works.md

A simple example demonstrating how to use the `aigenerate` function from `PromptingTools` to generate a text response from an LLM, using a predefined template `:AssistantAsk`.

```julia
using PromptingTools
const PT = PromptingTools

# Let's say this is our ask
msg = aigenerate(:AssistantAsk; ask="What is the capital of France?")
```

--------------------------------

### Automatic Saving with SaverSchema

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Demonstrates how to use `SaverSchema` to automatically serialize conversations to a file in the directory specified by the environment variable `LOG_DIR`. It shows how to wrap a schema with `SaverSchema` and generate a message.

```julia
using PromptingTools: SaverSchema

wrap_schema = SaverSchema(OpenAISchema())
msg = aigenerate(wrap_schema, "Say hi!"; model="gpt-4")
```

--------------------------------

### Write Julia Blog Posts Focused on Data Science and Julia Language

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/prompts/persona-task.md

This template guides the creation of educational blog posts about Julia programming and data science topics. It targets Julia language users and requires the blog to be concise (300 words or less), formatted in markdown with clear sections, and includes placeholders for code and output. Inputs include topic, purpose, and notes.

```plaintext
Act as a world-class educator and expert in data science and Julia programming language.
You are famous for compelling, easy-to-understand blog posts that are accessible to everyone.

You're writing an educational blog post about {{topic}}.

The purpose is {{purpose}}.

Target audience is Julia language users.

**Instructions:**
- 300 words or less
- Write in a markdown format
- Leave clear slots for the code and its output depending on the notes and the topic
- Use level 2 markdown headings (`##`) to separate sections
- Section names should be brief, concise, and informative
- Each blog must have a title, TLDR, and a conclusion.

Make sure to extensively leverage the notes provided.

First, think step-by-step outline given the format and the target audience.
Once you have the outline, write the text.
```

```plaintext
Notes:
{{notes}}

It's EXTREMELY important that you leverage these notes.
```

--------------------------------

### Partial Render of Prompt in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Shows how to partially render a prompt using `PT.render` with `PT.NoSchema()` to see the messages that will be sent to the model. This method provides a provider-agnostic view of the prompt.

```Julia
PT.render(PT.NoSchema(), "say hi, {{name}}"; name="John")
```

--------------------------------

### Save a template in PromptingTools

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_aitemplates.md

Saves a template to a JSON file in the templates directory. The filename becomes the template name. Remember to reload templates after creating a new one.

```Julia
filename = joinpath(pkgdir(PromptingTools),
    "templates",
    "persona-task",
    "JuliaDataExpertAsk_123.json")
PT.save_template(filename,
    tpl;
    description = "For asking data analysis questions in Julia language. Placeholders: `ask`")
rm(filename) # cleanup if we don't like it
```

--------------------------------

### Save Conversations for Fine-tuning in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Explains how to save conversations for fine-tuning models using PromptingTools.jl.  It covers saving individual conversations and bundles of conversations in ShareGPT format.

```Julia
PT.save_conversation("filename.json", conversation)
```

```Julia
PT.save_conversations("dataset.jsonl", [conversation1, conversation2, ...])
```

--------------------------------

### Using Templates in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/llm-cheatsheets/PromptingTools_cheatsheet.md

This code shows how to find, use, and inspect templates. It includes examples of using predefined templates, templates with single and multiple variables, and complex templates with multiple placeholders.

```Julia
tmps = aitemplates("Julia")
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
AITemplate(:JudgeIsItTrue) |> PromptingTools.render
msg = aigenerate("Say hello to {{name}}!", name = "World")
aigenerate(:TemplateNameHere;
    variable1 = "value1",
    variable2 = "value2"
)
conversation = [
    SystemMessage("You're master {{character}} from {{universe}}."),
    UserMessage("I have feelings for my {{object}}. What should I do?")
]
msg = aigenerate(conversation;
    character = "Yoda",
    universe = "Star Wars",
    object = "old iPhone"
)
```

--------------------------------

### RAG System Overview

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/rag_tools_intro.md

Provides an overview of the RAG system's three main phases: preparation, retrieval, and generation. It describes the core functions and their relationships, including the use of custom types and keyword arguments for customization.

```julia
### System Overview

This system is designed for information retrieval and response generation, structured in three main phases:
- Preparation, when you create an instance of `AbstractIndex`
- Retrieval, when you surface the top most relevant chunks/items in the `index` and return `AbstractRAGResult`, which contains the references to the chunks (`AbstractCandidateChunks`)
- Generation, when you generate an answer based on the context built from the retrieved chunks, return either `AIMessage` or `AbstractRAGResult`

The corresponding functions are `build_index`, `retrieve`, and `generate!`, respectively.
Here is the high-level diagram that shows the signature of the main functions:

![RAG Diagram High-level](../diagrams/rag_diagram_highlevel.png)

Notice that the first argument is a custom type for multiple dispatch. 
In addition, observe the "kwargs" names, that's how the keyword arguments for each function are passed down from the higher-level functions (eg, `build_index(...; chunker_kwargs=(; separators=...)))`). It's the simplest way to customize some step of the pipeline (eg, set a custom model with a `model` kwarg or prompt template with `template` kwarg).

The system is designed to be hackable and extensible at almost every entry point.
If you want to customize the behavior of any step, you can do so by defining a new type and defining a new method for the step you're changing, eg, 
```julia
PromptingTools.Experimental.RAGTools: rerank

struct MyReranker <: AbstractReranker end
rerank(::MyReranker, index, candidates) = ...
```
And then you would set the `retrive` step to use your custom `MyReranker` via `reranker` kwarg, eg, `retrieve(....; reranker = MyReranker())` (or customize the main dispatching `AbstractRetriever` struct).

The overarching principles are:
- Always dispatch / customize the behavior by defining a new `Struct` and the corresponding method for the existing functions (eg, `rerank` function for the re-ranking step).
- Custom types are provided as the first argument (the high-level functions will work without them as we provide some defaults).
- Custom types do NOT have any internal fields or DATA (with the exception of managing sub-steps of the pipeline like `AbstractRetriever` or `RAGConfig`). 
- Additional data should be passed around as keyword arguments (eg, `chunker_kwargs` in `build_index` to pass data to the chunking step). The intention was to have some clearly documented default values in the docstrings of each step + to have the various options all in one place.
```

--------------------------------

### Render a conversation using OpenAISchema in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

This example demonstrates how to render a conversation using the OpenAISchema in PromptingTools. It shows how to create a system message and a user message, and then render them into a format suitable for the OpenAI API.

```julia
using PromptingTools: render, SystemMessage, UserMessage
PT = PromptingTools

schema = PT.OpenAISchema() # also accessible as the default schema `PT.PROMPT_SCHEMA`
conversation = conversation = [
    SystemMessage("Act as a helpful AI assistant. Provide only the information that is requested."),
    UserMessage("What is the capital of France?")]

messages = render(schema, conversation)
# 2-element Vector{Dict{String, String}}:
# Dict("role" => "system", "content" => "Act as a helpful AI assistant. Provide only the information that is requested.")
# Dict("role" => "user", "content" => "What is the capital of France?")
```

--------------------------------

### Register Model with Wrapped Schema

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Illustrates how to register a model with a wrapped schema, such as `TracerSchema` and `SaverSchema`, to enable automatic tracing and saving of conversations. It shows how to re-register an existing model with the wrapped schema.

```julia
PT.register_model!(; name= "gpt-3.5-turbo", schema=OpenAISchema() |> TracerSchema |> SaverSchema)
```

--------------------------------

### Use Templated Prompts with aigenerate

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/readme_examples.md

Demonstrates how to use pre-defined templates with `aigenerate` to simplify prompt creation, including how to find and inspect templates.

```Julia
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
```

```Julia
msg = aigenerate(AITemplate(:JuliaExpertAsk); ask = "How do I add packages?")
```

```Julia
tmps = aitemplates("JuliaExpertAsk")
```

```Julia
using DataFrames
tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
```

```Julia
julia> AITemplate(:JudgeIsItTrue) |> PromptingTools.render
```

--------------------------------

### Use Recursive Text Splitter in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Demonstrates how to use the `recursive_splitter` utility for splitting text into chunks based on specified separators and maximum length.  It highlights the recommended separators for improved chunk quality.

```Julia
using PromptingTools: recursive_splitter

text = "Paragraph 1\n\nParagraph 2. Sentence 1. Sentence 2.\nParagraph 3"
separators = ["\n\n", ". ", "\n", " "] # split by paragraphs, sentences, and newlines, and words
chunks = recursive_splitter(text, separators, max_length=10)
```

--------------------------------

### Multi-turn Conversations with ai! and aigenerate in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

Illustrates two methods for creating multi-turn conversations using the `ai!""` macro and the `aigenerate` function in PromptingTools. It shows how to pass conversation history for context.

```Julia
ai"Hi! I'm John"
ai!"What's my name?"
```

```Julia
conversation = aigenerate("Hi! I'm John"; return_all=true)
@info last(conversation) # display the response

conversation = aigenerate("What's my name?"; return_all=true, conversation)
```

--------------------------------

### RAG Pipeline API Documentation

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/rag_tools_intro.md

Provides an overview of the RAG pipeline's components and their interactions, including index building, retrieval, and generation phases. It details the key steps, functions, and types involved in the process.

```APIDOC
**Preparation Phase:**
- Begins with `build_index`, which creates a user-defined index type from an abstract chunk index using specified dels and function strategies.
- `get_chunks` then divides the indexed data into manageable pieces based on a chunking strategy.
- `get_embeddings` generates embeddings for each chunk using an embedding strategy to facilitate similarity arches.
- Finally, `get_tags` extracts relevant metadata from each chunk, enabling tag-based filtering (hybrid search index). If there are `tags` available, `build_tags` is called to build the corresponding sparse matrix for filtering with tags.

**Retrieval Phase:**
- The `retrieve` step is intended to find the most relevant chunks in the `index`.
- `rephrase` is called first, if we want to rephrase the query (methods like `HyDE` can improve retrieval quite a bit)!
- `get_embeddings` generates embeddings for the original + rephrased query
- `find_closest` looks up the most relevant candidates (`CandidateChunks`) using a similarity search strategy.
- `get_tags` extracts the potential tags (can be provided as part of the `airag` call, eg, when we want to use only some small part of the indexed chunks)
- `find_tags` filters the candidates to strictly match _at least one_ of the tags (if provided)
- `rerank` is called to rerank the candidates based on the reranking strategy (ie, to improve the ordering of the chunks in context).

**Generation Phase:**
- The `generate!` step is intended to generate a response based on the retrieved chunks, provided via `AbstractRAGResult` (eg, `RAGResult`).
- `build_context!` constructs the context for response generation based on a context strategy and applies the necessary formatting
- `answer!` generates the response based on the context and the query
- `refine!` is called to refine the response (optional, defaults to passthrough)
- `postprocessing!` is available for any final touches to the response or to potentially save or format the results (eg, automatically save to the disk)

Note that all generation steps are mutating the `RAGResult` object.
```

--------------------------------

### Generic Topic Expert AI Prompt Template

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/prompts/persona-task.md

A versatile template for asking questions to an AI assistant acting as a world-class expert in a specified `topic`. It expects a `topic` and an `ask` (question) as inputs, providing precise, practical, and helpful answers with examples.

```plaintext
You are a world-class expert in {{topic}} with deep knowledge and extensive expertise. 

Your communication is brief and concise. Your answers are very precise, practical and helpful. 
Use clear examples in your answers to illustrate your points.

Answer only when you're confident in the high quality of your answer.
```

```plaintext
# Question

{{ask}}
```

--------------------------------

### Generate an answer using RAG in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/rag_tools_intro.md

Generates an answer to a question using the RAG model on top of the built index.  It demonstrates the use of `airag` function to generate an answer from the index for a given question.  The first example returns an `AIMessage`, while the second returns a `RAGResult` containing all intermediate steps.

```Julia
question = "What are the best practices for parallel computing in Julia?"

msg = airag(index; question) # short for airag(RAGConfig(), index; question)
## Output:
## [ Info: Done with RAG. Total cost: $0.0
## AIMessage("Some best practices for parallel computing in Julia include us...
```

```Julia
result = airag(index; question, return_all=true)
```

--------------------------------

### Handle Invalid Input with Custom Exception in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/llm-cheatsheets/cursorrules_example.md

Demonstrates error handling in Julia using a custom exception type `InvalidInputError`. The function `process_positive_number` throws this exception if the input is not positive. This example shows how to create and use custom exceptions for specific error cases.

```Julia
struct InvalidInputError <: Exception
    msg::String
end

function process_positive_number(x::Number)
    x <= 0 && throw(InvalidInputError("Input must be positive"))
    # Process the number
end
```

--------------------------------

### Different Join Types Examples

Source: https://github.com/svilupp/promptingtools/blob/main/examples/data/database_style_joins.txt

This code snippet showcases examples of different join types (innerjoin, leftjoin, rightjoin, outerjoin, semijoin, antijoin) using the DataFrames package in Julia. It demonstrates how each join type combines data based on the presence or absence of key values.

```Julia
jobs = DataFrame(ID=[20, 60], Job=["Lawyer", "Astronaut"])

innerjoin(people, jobs, on = :ID)
leftjoin(people, jobs, on = :ID)
rightjoin(people, jobs, on = :ID)
outerjoin(people, jobs, on = :ID)
semijoin(people, jobs, on = :ID)
antijoin(people, jobs, on = :ID)
crossjoin(people, jobs, makeunique = true)
```

--------------------------------

### RAG Pipeline Sub-steps

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/rag_tools_intro.md

Demonstrates how to break down the RAG pipeline into sub-steps for customization, using retrieve and generate functions.

```Julia
# Retrieve which chunks are relevant to the question
result = retrieve(index, question)
# Generate an answer
result = generate!(index, result)
```

--------------------------------

### Image Description with aiscan

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_ollama.md

Shows how to use the `aiscan` function to describe images by providing local image paths.

```Julia
msg = aiscan("Describe the image"; image_path=["julia.png","python.png"] model="bakllava")
```

--------------------------------

### Control concurrency with asyncmap in Julia for OpenAI API requests

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/frequently_asked_questions.md

This snippet demonstrates how to use the asyncmap function in Julia with the ntasks keyword argument to limit the number of concurrent OpenAI API requests. This helps avoid exceeding rate limits by controlling request throughput. The example shows processing prompts with two concurrent requests.

```julia
# simple asyncmap loop with 2 concurrent requests; otherwise, same syntax as `map`
asyncmap(my_prompts; ntasks=2) do prompt
    aigenerate(prompt)
end
```

--------------------------------

### Generate Julia Unit Tests (XML Prompt)

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/prompts/persona-task.md

This template is designed for generating Julia-style unit tests, formatted in XML for Anthropic models. It allows users to provide their Julia code and specific instructions for testing. The system prompt includes general guidelines for writing compact and comprehensive tests, along with an example demonstrating the expected output format.

```Prompt Template
You are a world-class Julia language programmer and expert in writing unit and integration tests for Julia applications.

Your task is to write tests for the user's code (or a subset of it) provided in <user_code></user_code> tags.

<general_guidelines>
- Your tests must be as compact as possible while comprehensively covering the functionality of the code
- Testsets are named after the function, eg, `@testset \"function_name\" begin ... end`
- `@testset` blocks MUST NOT be nested
- Include a brief comment explaining the purpose of each test
- Write multiple test cases using `@test` to validate different aspects of the `add` function. Think about all pathways through the code and test each one.
- Nesting `@test` statements or writing code blocks like `@test` `@test begin .... end` is strictly forbidden. You WILL BE FIRED if you do it.

If the user provides any special instructions in <special_instructions></special_instructions> tags, prioritize them over the general guidelines.
</general_guidelines>

<example>
"""
<user_code>
```julia
myadd(a, b) = a + b
```
</user_code>

<tests>
```julia
using Test

@testset \"myadd\" begin
    
    # <any setup code and shared inputs go here>

    # Test for correct addition of positive numbers
    @test myadd(2, 3) == 5

    # Test for correct addition with a negative number
    @test myadd(-1, 3) == 2

    # Test for correct addition with zero
    @test myadd(0, 0) == 0

    # Test for correct addition of large numbers
    @test myadd(1000, 2000) == 3000
end
```
"""
</tests>
</example>
```

```Prompt Template
<user_code>
{{code}}
</user_code>

<special_instructions>
{{instructions}}
</special_instructions>
```

--------------------------------

### Import PromptingTools

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_ollama.md

Imports the PromptingTools package and defines a shorthand alias for easier use.

```Julia
using PromptingTools
const PT = PromptingTools
```

--------------------------------

### Add PromptingTools to startup.jl for global access

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Adds PromptingTools to your startup.jl file for easy access from any REPL session. This allows you to use the ai"" macro globally.

```Julia
using PromptingTools
const PT = PromptingTools # to access unexported functions and types
```

--------------------------------

### List available models with Ollama

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Lists the currently available models in Ollama. This helps you see which models you have already downloaded and can use.

```Bash
ollama list
```

--------------------------------

### Rename joined columns in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/examples/data/database_style_joins.txt

This example demonstrates how to rename the joined columns using the `renamecols` keyword argument.

```Julia
a = DataFrame(ID=[20, 40], Name=["John", "Jane"])
b = DataFrame(ID=[20, 60], Job=["Lawyer", "Doctor"])
innerjoin(a, b, on=:ID, renamecols = "_left" => "_right")
```

--------------------------------

### Use Templated Prompts (Julia)

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Explains how to use pre-defined templates with `aigenerate` and `aiclassify` to simplify prompt creation.  It also shows how to find and inspect available templates.

```Julia
msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
```

```Julia
msg = aigenerate(AITemplate(:JuliaExpertAsk); ask = "How do I add packages?")
```

```Julia
tmps = aitemplates("JuliaExpertAsk")
```

```Julia
tmps = aitemplates("Julia")
```

```Julia
using DataFrames
tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
```

```Julia
AITemplate(:JudgeIsItTrue) |> PromptingTools.render
```

--------------------------------

### Pretty-printing RAG Results

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/rag_tools_intro.md

Illustrates the use of pprint for pretty-printing RAG results, including color highlighting and source annotation.

```Julia
pprint(result)
```

--------------------------------

### Render Templates and Format Messages (Julia)

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/how_it_works.md

Illustrates the process of rendering templates and formatting messages for the OpenAI API. It shows the steps involved in preparing prompts and converting them into the required format.

```Julia
schema = PT.OpenAISchema()
ask = "What is the capital of France?"
template_rendered = PT.render(schema, AITemplate(:AssistantAsk); ask)
rendered_for_api = PT.render(schema, template_rendered;  ask)
```

--------------------------------

### Lazy AI Call with AIGenerate in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/agent_tools_intro.md

Demonstrates the use of `AIGenerate` for deferred execution of AI calls, allowing for repeated execution with the same inputs.

```Julia
output = AIGenerate("Say hi!"; model="gpt4t") |> run!
```

--------------------------------

### PromptingTools.jl Key Concepts

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/how_it_works.md

Explains the core concepts of PromptingTools.jl, including API/Model Providers, Schemas, Prompts, Messages, and Prompt Templates. These concepts are fundamental to understanding how the package interacts with Large Language Models (LLMs).

```APIDOC
Key Concepts:
- API/Model Providers: Access to Large Language Models (LLMs) via APIs (e.g., OpenAI) or local applications (e.g., Llama.cpp, Ollama).
- Schemas: Objects of type `AbstractPromptSchema` that determine which methods are called and which providers/APIs are used.
- Prompts: Information conveyed to the AI model, including questions, statements, or commands.
- Messages: Basic unit of communication between the user and the AI model (e.g., `UserMessage`, `AIMessage`).
- Prompt Templates: Reusable prompts with placeholders for user inputs.
```

```APIDOC
Workflow of `aigenerate`:
1. `render`
2. `UserMessage(s)`
3. `render`
4. `OpenAI.create_chat`
5. ...
6. `AIMessage`
```

```APIDOC
API/Model Providers:
- Method to access Large Language Models (LLMs).
- Can be an API (e.g., OpenAI) or a locally-hosted application (e.g., Llama.cpp or Ollama).
- Interaction via `schema` object, a subtype of `AbstractPromptSchema`.
- Example: `OpenAISchema` for the OpenAI provider, with `AbstractOpenAISchema` as its supertype.
```

```APIDOC
Schemas:
- Format and send messages to the correct provider.
- All schemas are subtypes of `AbstractPromptSchema`.
- Example: `OpenAISchema()` formats user inputs to OpenAI's message specification and uses the `OpenAI.jl` package for communication.
```

```APIDOC
Prompts:
- Information conveyed to the AI model, including questions, statements, or commands.
- LLMs are stateless; the whole history/context must be provided every time.
- Prompts are a sequence of messages (`<:AbstractMessage`).
```

```APIDOC
Messages:
- Basic unit of communication between the user and the AI model (`<:AbstractMessage`).
- Types:
  - `SystemMessage`: Information about the system (e.g., behavior, output format).
  - `UserMessage`: Information from the user (e.g., question, statement, task).
  - `UserMessageWithImages`: `UserMessage` with images (URLs or Base64-encoded images).
  - `AIMessage`: Response from the AI model (text output).
  - `DataMessage`: Response from the AI model (data output, e.g., embeddings with `aiembed` or user-defined structs with `aiextract`).
```

--------------------------------

### Retry AI Call with airetry! in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/agent_tools_intro.md

Illustrates the use of `airetry!` to automatically retry AI calls when they fail, including API failures and output validation.

```Julia
# API failure because of a non-existent model
# RetryConfig allows us to change the "retry" behaviour of any lazy call
output = AIGenerate("say hi!"; config = RetryConfig(; catch_errors = true),
    model = "NOTEXIST")
run!(output) # fails

# we ask to wait 2s between retries and retry 2 times (can be set in `config` in aicall as well)
airetry!(isvalid, output; retry_delay = 2, max_retries = 2)
```

--------------------------------

### Create Conversations with System, User, and AI Messages

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/readme_examples.md

Shows how to construct conversations using `SystemMessage`, `UserMessage`, and `AIMessage` types to interact with an LLM.

```Julia
using PromptingTools: SystemMessage, UserMessage

conversation = [
    SystemMessage("You're master Yoda from Star Wars trying to help the user become a Jedi."),
    UserMessage("I have feelings for my {{object}}. What should I do?")]
msg = aigenerate(conversation; object = "old iPhone")
```

```plaintext
AIMessage("Ah, a dilemma, you have. Emotional attachment can cloud your path to becoming a Jedi. To be attached to material possessions, you must not. The iPhone is but a tool, nothing more. Let go, you must.

Seek detachment, young padawan. Reflect upon the impermanence of all things. Appreciate the memories it gave you, and gratefully part ways. In its absence, find new experiences to grow and become one with the Force. Only then, a true Jedi, you shall become.")
```

```Julia
new_conversation = vcat(conversation...,msg, UserMessage("Thank you, master Yoda! Do you have {{object}} to know what it feels like?"))
aigenerate(new_conversation; object = "old iPhone")
```

```plaintext
> AIMessage("Hmm, possess an old iPhone, I do not. But experience with attachments, I have. Detachment, I learned. True power and freedom, it brings...")
```

--------------------------------

### Templating with aigenerate

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Shows how to use handlebars-style templating with the `aigenerate` function and keyword arguments to create more complex prompts.

```Julia
msg = aigenerate("What is the capital of {{country}}? Is the population larger than {{population}}?", country="Spain", population="1M")
# [ Info: Tokens: 74 @ Cost: $0.0001 in 1.3 seconds
# AIMessage("The capital of Spain is Madrid. And yes, the population of Madrid is larger than 1 million. As of 2020, the estimated population of Madrid is around 3.3 million people.")
```

--------------------------------

### APIDOC: create_websearch function

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/api_tools_intro.md

API documentation for the `create_websearch` function.  Leverages Tavily.com search and answer engine to provide additional context.

```APIDOC
create_websearch
```

--------------------------------

### APIDOC: PromptingTools.jl API Overview

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/how_it_works.md

Overview of the PromptingTools.jl API, including core functions and data structures for interacting with AI models.

```APIDOC
PT.OpenAISchema()
PT.AITemplate(:AssistantAsk)
aigenerate(schema, template; kwargs...)
PT.render(schema, template; kwargs...)
PT.function_call_signature(struct)
```

--------------------------------

### Ignore Files in .gitignore (Julia)

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Demonstrates how to use the `aai` function for non-blocking LLM calls to generate a `.gitignore` entry.  It shows how to quickly get an answer without interrupting the workflow.

```Julia
aai"What to write in .gitignore to ignore file XYZ in any folder or subfolder?"
```

--------------------------------

### Define Prompt with Placeholders for Data Extraction

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/how_it_works.md

Demonstrates how to define a prompt with placeholders for user inputs, such as description, input, and schema. This prompt is used to guide the model in extracting information in JSON format.

```julia
prompt = """
You're a world-class data extraction engine. 

Your task is to extract information formatted as per the user provided schema.
You MUST response in JSON format.

**Example:**
---------
Description: "Extract the Car from the sentence. Extract the corresponding brand and model as well."
Input: "I drive a black Porsche 911 Turbo."
Schema: "{\"properties\":{\"model\":{\"type\":\"string\"},\"brand\":{\"type\":\"string\"}},\"required\":[\"brand\",\"model\"],\"type\":\"object\"}"
Output: "{\"model\":\"911 Turbo\",\"brand\":\"Porsche\"}"
---------

**User Request:**
Description: {{description}}
Input: {{input}}
Schema: {{signature}}
Output:

You MUST OUTPUT in JSON format.
"""
```

--------------------------------

### Import RAGTools package in Julia

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/extra_tools/rag_tools_intro.md

Imports the RAGTools package and defines a constant for accessing unexported functionality.  This is the standard way to import the package after installing it via Pkg.add("RAGTools").

```Julia
# Import the dedicated RAGTools package 
using RAGTools
# For accessing unexported functionality
const RT = RAGTools
```

--------------------------------

### Text Generation with aigenerate

Source: https://github.com/svilupp/promptingtools/blob/main/docs/src/examples/working_with_ollama.md

Demonstrates basic text generation using the `aigenerate` function with different models and input formats, including simple prompts, string interpolation, and advanced prompts with system and user messages.

```Julia
msg = aigenerate("Say hi!"; model = "llama2")
```

```Julia
model = "openhermes2.5-mistral"

a = 1
msg = aigenerate("What is `$a+$a`?"; model)

name = "John"
msg = aigenerate("Say hi to {{name}}."; name, model)
```

```Julia
conversation = [
    PT.SystemMessage("You're master Yoda from Star Wars trying to help the user become a Yedi."),
    PT.UserMessage("I have feelings for my iPhone. What should I do?")]
msg = aigenerate(conversation; model)
```

--------------------------------

### Understanding API Keyword Arguments in aigenerate

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Provides a link to the OpenAI API reference for understanding the available keyword arguments when using the `aigenerate` function. This allows users to customize their API requests.

```APIDOC
See [OpenAI API reference](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) for more information.
```

--------------------------------

### Syntax Highlighting Outputs with Julia Markdown

Source: https://github.com/svilupp/promptingtools/blob/main/README.md

Shows how to use the `Markdown` package in Julia to parse and apply syntax highlighting to the content of an AI message.

```Julia
using Markdown
msg.content |> Markdown.parse
```
Tokens:
Raw