### Setup Go Environment and Install txtai.go

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

Installs Go and fetches the txtai.go library. This is a prerequisite for running Go examples.

```python
%%capture
os.chdir("/content")
!apt install golang-go
!go get "github.com/neuml/txtai.go"
```

--------------------------------

### Agent Quickstart Example

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

This is a basic example demonstrating the quickstart for txtai agents. It serves as an entry point for understanding agent functionality.

```python
# examples/agent_quickstart.py
```

--------------------------------

### Install and Start PostgreSQL

Source: https://github.com/neuml/txtai/blob/master/examples/49_External_database_integration.ipynb

Installs the PostgreSQL server and starts the service, setting a password for the default 'postgres' user. This prepares a local PostgreSQL instance for txtai.

```bash
%%capture

# Install and start Postgres
!apt-get update && apt-get install postgresql
!service postgresql start
!sudo -u postgres psql -U postgres -c "ALTER USER postgres PASSWORD 'postgres';"
```

--------------------------------

### RAG Quick Start Example

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

This Python script demonstrates a RAG quick start using txtai. It includes steps for file extraction, text chunking, building an embeddings database, and creating a RAG pipeline. Ensure txtai and pipeline-data are installed.

```python
# pylint: disable=C0103
import os

from txtai import Embeddings, RAG
from txtai.pipeline import Textractor

# Step 1: Collect files from local directory
#
# Defaults to "data". Set to whereever your files are.
path = "data"
files = |os.path.join(path, f) for f in os.listdir(path) if os.path.isfile(os.path.join(path, f))|

# Step 2: Text Extraction / Chunking
#
# Using section based chunking here. More complex options available such as semantic chunking, iterative chunking 
etc.
# Documentation: https://neuml.github.io/txtai/pipeline/data/textractor
# Supports Chonkie chunking as well: https://docs.chonkie.ai/oss/chunkers/overview
textractor = Textractor(backend="docling", sections=True)
chunks = |]
for f in files:
 for chunk in textractor(f):
 chunks.append((f, chunk))

# Step 3: Build an embeddings database
#
# The `path` parameter sets the vector embeddings model. Supports Hugging Face models, llama.cpp, Ollama, vLLM and 
more.
# Documentation: https://neuml.github.io/txtai/embeddings/
embeddings = Embeddings(content=True, path="Qwen/Qwen3-Embedding-0.6B", maxlength=2048)
embeddings.index(chunks)

# Step 4: Create RAG pipeline
#
# Combines an embeddings database and an LLM.
# Supports Hugging Face models, llama.cpp, Ollama, vLLM and more
# Documentation: https://neuml.github.io/txtai/pipeline/text/rag

```

--------------------------------

### Install MLflow Plugin and Start Server

Source: https://github.com/neuml/txtai/blob/master/docs/observability.md

Install the MLflow plugin for txtai and start a local MLflow service. This is a prerequisite for enabling tracing.

```bash
# Install MLflow plugin for txtai
pip install mlflow-txtai

# Start a local MLflow service
mlflow server --host 127.0.0.1 --port 8000
```

--------------------------------

### Install txtai and Datasets

Source: https://github.com/neuml/txtai/blob/master/examples/02_Build_an_Embeddings_index_with_Hugging_Face_Datasets.ipynb

Installs the necessary libraries for the example. Use this to set up your environment.

```python
%%capture
!pip install git+https://github.com/neuml/txtai
!pip install datasets
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/79_RAG_is_more_than_Vector_Search.ipynb

Installs the txtai library and necessary dependencies, including downloading an example SQL database. Use this to set up your environment for the examples.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline-data]

# Download example SQL database
!wget https://huggingface.co/NeuML/txtai-wikipedia-slim/resolve/main/documents
```

--------------------------------

### Get Started with txtai Embeddings

Source: https://github.com/neuml/txtai/blob/master/README.md

This snippet shows how to quickly initialize txtai embeddings, index sample data, and perform a search. It's a basic example for initial setup.

```python
import txtai

embeddings = txtai.Embeddings()
embeddings.index(["Correct", "Not what we hoped"])
embeddings.search("positive", 1)
#[(0, 0.29862046241760254)]
```

--------------------------------

### RAG Quick Start Example

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

A comprehensive example demonstrating the RAG quick start process, including file collection, text extraction, chunking, building an embeddings database, and creating a RAG pipeline. This code is intended to be run from a file.

```python
# pylint: disable=C0103
import os

from txtai import Embeddings, RAG
from txtai.pipeline import Textractor

# Step 1: Collect files from local directory
#
# Defaults to "data". Set to whereever your files are.
path = "data"
files = [os.path.join(path, f) for f in os.listdir(path) if os.path.isfile(os.path.join(path, f))]

# Step 2: Text Extraction / Chunking
#
# Using section based chunking here. More complex options available such as semantic chunking, iterative chunking etc.
# Documentation: https://neuml.github.io/txtai/pipeline/data/textractor
# Supports Chonkie chunking as well: https://docs.chonkie.ai/oss/chunkers/overview
textractor = Textractor(backend="docling", sections=True)
chunks = []
for f in files:
 for chunk in textractor(f):
  chunks.append((f, chunk))

# Step 3: Build an embeddings database
#
# The `path` parameter sets the vector embeddings model. Supports Hugging Face models, llama.cpp, Ollama, vLLM and more.
# Documentation: https://neuml.github.io/txtai/embeddings/
embeddings = Embeddings(content=True, path="Qwen/Qwen3-Embedding-0.6B", maxlength=2048)
embeddings.index(chunks)

# Step 4: Create RAG pipeline
#
# Combines an embeddings database and an LLM.
# Supports Hugging Face models, llama.cpp, Ollama, vLLM and more
# Documentation: https://neuml.github.io/txtai/pipeline/text/rag

# User prompt template
template = """
 Answer the following question using the provided context.

 Question:
 {question}

 Context:
 {context}
"""

rag = RAG(
 embeddings,
 "Qwen/Qwen3-0.6B",
 system="You are a friendly assistant",
 template=template,
 output="flatten",
)

question = "Summarize the main advancements made by BERT"
print(rag(question, maxlength=2048, stripthink=True))
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/56_External_vectorization.ipynb

Installs the txtai library and its dependencies. This is a prerequisite for running the examples.

```python
from txtai import Embeddings

# Load dataset
wikipedia = Embeddings()
wikipedia.load(provider="huggingface-hub", container="neuml/txtai-wikipedia")

# Query for Top 10,000 most popular articles
query = """
SELECT id, text FROM txtai
order by percentile desc
LIMIT 10000
"""

data = wikipedia.search(query)

# Encode vectors using same vector model as Wikipedia
vectors = wikipedia.batchtransform(x["text"] for x in data)

# Build dataset of id, text, embeddings
dataset = []
for i, row in enumerate(data):
  dataset.append({"id": row["id"], "article": row["text"], "embeddings": vectors[i]})

```

--------------------------------

### Configure and Run txtai API

Source: https://github.com/neuml/txtai/blob/master/README.md

This example demonstrates how to configure the txtai API using a YAML file and start the server. It also shows how to query the API for search results.

```yaml
# app.yml
embeddings:
    path: sentence-transformers/all-MiniLM-L6-v2
```

```bash
CONFIG=app.yml uvicorn "txtai.api:app"
curl -X GET "http://localhost:8000/search?query=positive"
```

--------------------------------

### Install and Configure Postgres

Source: https://github.com/neuml/txtai/blob/master/examples/61_Integrate_txtai_with_Postgres.ipynb

Installs Postgres, the pgvector extension for vector search, and starts the database instance. Sets a password for the postgres user.

```bash
%%capture

# Install Postgres and pgvector
!apt-get update && apt install postgresql postgresql-server-dev-14
!git clone --branch v0.6.2 https://github.com/pgvector/pgvector.git
!cd pgvector && make && make install

# Start database
!service postgresql start
!sudo -u postgres psql -U postgres -c "ALTER USER postgres PASSWORD 'pass';"
```

--------------------------------

### Build and Run JavaScript Labels Example

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

Installs Node.js dependencies, builds the JavaScript project, and executes the labels example. This step is required to run the JavaScript code.

```bash
os.chdir("txtai.js/examples/node")
!npm install
!npm run build
!node dist/labels.js
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/32_Model_explainability.ipynb

Installs txtai with pipeline support and the SHAP library. This is a prerequisite for running the examples.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline] shap
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/59_Whats_new_in_txtai_7_0.ipynb

Installs the txtai library with API, graph, and pipeline-training extras, along with the datasets library. This is a prerequisite for running the examples.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[api,graph,pipeline-train] datasets
```

--------------------------------

### Install and Build txtai.js Project

Source: https://github.com/neuml/txtai/blob/master/examples/15_Distributed_embeddings_cluster.ipynb

Installs project dependencies and builds the JavaScript project. This is a prerequisite for running the cluster example.

```python
%%capture
os.chdir("txtai.js/examples/node")
!npm install
!npm run build
```

--------------------------------

### Install Rust and Clone txtai.rs Repository

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

Installs the Rust compiler and clones the txtai.rs GitHub repository for example usage.

```python
%%capture
os.chdir("/content")
!apt-get install rustc
!git clone https://github.com/neuml/txtai.rs
```

--------------------------------

### Install txtai.js

Source: https://github.com/neuml/txtai/blob/master/examples/15_Distributed_embeddings_cluster.ipynb

Installs the txtai JavaScript library using npm. This is a prerequisite for running the JavaScript example that interacts with the txtai API cluster.

```bash
npm install txtai
```

--------------------------------

### Build and Run Rust Labels Example

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

Builds the Rust demo project and then runs the labels example. Ensure you are in the 'txtai.rs/examples/demo' directory.

```python
%%capture
os.chdir("txtai.rs/examples/demo")
!cargo build
```

```python
!cargo run labels
```

--------------------------------

### Install txtai with Optional Dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/faq.md

Example of installing txtai with optional dependencies, using quotes to escape special characters in the shell.

```bash
pip install 'txtai[pipeline]'
```

--------------------------------

### Install txtai.go Library

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

Add this import statement to use the txtai.go library. For module users, it installs automatically. Otherwise, use 'go get'.

```golang
import "github.com/neuml/txtai.go"
```

--------------------------------

### RAG Quick Start Script

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

This script demonstrates a complete RAG pipeline using txtai. It requires installing txtai with pipeline-data and setting up a 'data' directory with files. The script performs text extraction, builds an embeddings index, and initializes a RAG pipeline for question answering.

```python
RAG Quick Start
Easy to use way to get started with RAG using YOUR data

For a complete application see this: https://github.com/neuml/rag

TxtAI has many example notebooks covering everything the framework provides
Examples: https://neuml.github.io/txtai/examples

Install TxtAI
 pip install txtai[pipeline-data]

# pylint: disable=C0103
import os

from txtai import Embeddings, RAG
from txtai.pipeline import Textractor

# Step 1: Collect files from local directory
#
# Defaults to "data". Set to whereever your files are.
path = "data"
files = [os.path.join(path, f) for f in os.listdir(path) if os.path.isfile(os.path.join(path, f))]

# Step 2: Text Extraction / Chunking
#
# Using section based chunking here. More complex options available such as semantic chunking, iterative chunking etc.
# Documentation: https://neuml.github.io/txtai/pipeline/data/textractor
# Supports Chonkie chunking as well: https://docs.chonkie.ai/oss/chunkers/overview
textractor = Textractor(backend="docling", sections=True)
chunks = []
for f in files:
 for chunk in textractor(f):
  chunks.append((f, chunk))

# Step 3: Build an embeddings database
#
# The `path` parameter sets the vector embeddings model. Supports Hugging Face models, llama.cpp, Ollama, vLLM and more.
# Documentation: https://neuml.github.io/txtai/embeddings/
embeddings = Embeddings(content=True, path="Qwen/Qwen3-Embedding-0.6B", maxlength=2048)
embeddings.index(chunks)

# Step 4: Create RAG pipeline
#
# Combines an embeddings database and an LLM.
# Supports Hugging Face models, llama.cpp, Ollama, vLLM and more
# Documentation: https://neuml.github.io/txtai/pipeline/text/rag

# User prompt template
template = """
 Answer the following question using the provided context.

 Question:
 {question}

 Context:
 {context}
"""

rag = RAG(
 embeddings,
 "Qwen/Qwen3-0.6B",
 system="You are a friendly assistant",
 template=template,
 output="flatten",
)

question = "Summarize the main advancements made by BERT"
print(rag(question, maxlength=2048, stripthink=True))
```

--------------------------------

### Install txtai with Agent and Graph Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/71_Analyzing_LinkedIn_Company_Posts_with_Graphs_and_Agents.ipynb

Installs the txtai library along with agent and graph dependencies, as well as the datasets library. This is a prerequisite for running the notebook's examples.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[agent,graph] datasets
```

--------------------------------

### Install txtai from source

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs the latest, unreleased features of txtai directly from its GitHub repository.

```bash
pip install git+https://github.com/neuml/txtai
```

--------------------------------

### Build and Run Java Labels Example

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

Builds and executes the Java labels example using Gradle. Ensure you are in the 'txtai.java/examples' directory.

```python
os.chdir("txtai.java/examples")
!../gradlew -q --console=plain labels 2> /dev/null
```

--------------------------------

### Install txtai and OpenCode Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/81_OpenCode_as_a_txtai_LLM.ipynb

Installs txtai with API and LLM pipeline support, then downloads and runs the OpenCode server.

```bash
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[api,pipeline-llm]

# Install OpenCode if not already installed and run serve
!curl -fsSL https://opencode.ai/install | bash
!opencode serve &
```

--------------------------------

### Install txtai and Outlines

Source: https://github.com/neuml/txtai/blob/master/examples/60_Advanced_RAG_with_guided_generation.ipynb

Installs the necessary libraries for the advanced RAG process. This includes txtai and the outlines library.

```python
!pip install git+https://github.com/neuml/txtai#egg=txtai outlines
```

--------------------------------

### Install txtai with extras from source

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with specific extras directly from its GitHub repository by appending the extra name to the URL.

```bash
pip install git+https://github.com/neuml/txtai#egg=txtai[<name-of-extra>]
```

--------------------------------

### Setup Rust Environment for ONNX

Source: https://github.com/neuml/txtai/blob/master/examples/18_Export_and_run_models_with_ONNX.ipynb

Prepares the Rust environment by creating directories, copying ONNX models, and installing Rust and Cargo. This is a prerequisite for running Rust ONNX inference.

```shell
%%capture
import os

os.chdir("/content")
!mkdir rust
os.chdir("/content/rust")

# Copy ONNX models
!cp ../text-classify.onnx .
!cp ../embeddings.onnx .

# Install Rust
!apt-get install rustc cargo

!mkdir -p src
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/05_Extractive_QA_with_txtai.ipynb

Installs the txtai library and its required dependencies using pip. The %%capture magic command suppresses output during installation.

```python
%%capture
!pip install git+https://github.com/neuml/txtai
```

--------------------------------

### Install txtai with Pipeline Training

Source: https://github.com/neuml/txtai/blob/master/examples/20_Extractive_QA_to_build_structured_data.ipynb

Installs the txtai library with necessary dependencies for pipeline training. Use this command to set up your environment.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline-train]
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/80_Distilling_Knowledge_into_Tiny_LLMs.ipynb

Installs the txtai library with pipeline training capabilities and the datasets library. This is a prerequisite for running the notebook.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline-train] datasets
```

--------------------------------

### Install txtai with all optional dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai along with all available optional dependencies for full functionality.

```bash
pip install txtai[all]
```

--------------------------------

### Install txtai with Database dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with additional dependencies for content storage options.

```bash
pip install txtai[database]
```

--------------------------------

### Install txtai with API, Pipeline, and Workflow extras

Source: https://github.com/neuml/txtai/blob/master/examples/22_Transform_tabular_data_with_composable_workflows.ipynb

Installs the txtai library along with the api, pipeline, and workflow optional packages. Also installs sacremoses for text processing.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[api,pipeline,workflow] sacremoses
```

--------------------------------

### Install txtai with Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/76_Whats_new_in_txtai_9_0.ipynb

Installs the txtai library along with ANN and vector dependencies. Use this command to set up the environment for txtai.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[ann,vectors]
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/39_Classic_Topic_Modeling_with_BM25.ipynb

Installs the txtai library with graph support and the datasets library. This is a prerequisite for running the notebook.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[graph] datasets
```

--------------------------------

### Install txtai with API dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with dependencies required to serve the library via a web API.

```bash
pip install txtai[api]
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/29_Anatomy_of_a_txtai_index.ipynb

Installs the txtai library and necessary system dependencies. This is a prerequisite for creating and managing txtai indexes.

```python
%%capture
!pip install git+https://github.com/neuml/txtai
!apt-get update && apt-get install -y file xxd
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/51_Custom_API_Endpoints.ipynb

Installs the txtai library with API support and the datasets library. This is a prerequisite for running the API extensions.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[api] datasets
```

--------------------------------

### Install Python Packages

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

Installs sounddevice, scipy, and numpy packages using pip. This is a prerequisite for audio playback.

```bash
pip install sounddevice scipy numpy
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/41_Train_a_language_model_from_scratch.ipynb

Installs the txtai library with pipeline training capabilities, along with datasets, sentence-transformers, and ONNX runtimes.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline-train] datasets sentence-transformers onnxruntime onnx
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/73_Chunking_your_data_for_RAG.ipynb

Installs the txtai library with pipeline and data dependencies. This command is typically run in a notebook environment.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline-data]
```

--------------------------------

### Setup for Benchmarking

Source: https://github.com/neuml/txtai/blob/master/examples/48_Benefits_of_hybrid_search.ipynb

Downloads the benchmarks script and a subset of BEIR datasets, then prepares the output directory and cleans up existing benchmark data.

```python
import os

# Get benchmarks script
os.system("wget https://raw.githubusercontent.com/neuml/txtai/master/examples/benchmarks.py")

# Create output directory
os.makedirs("beir", exist_ok=True)

# Download subset of BEIR datasets
datasets = ["nfcorpus", "fiqa", "arguana", "scidocs", "scifact"]
for dataset in datasets:
  url = f"https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/{dataset}.zip"
  os.system(f"wget {url}")
  os.system(f"mv {dataset}.zip beir")
  os.system(f"unzip -d beir beir/{dataset}.zip")

  # Remove existing benchmark data
if os.path.exists("benchmarks.json"):
  os.remove("benchmarks.json")
```

--------------------------------

### Create and Run AudioStream Pipeline

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/audio/audiostream.md

A simple example demonstrating how to instantiate and use the AudioStream pipeline with provided data.

```python
from txtai.pipeline import AudioStream

# Create and run pipeline
audio = AudioStream()
audio(data)
```

--------------------------------

### Basic Text To Audio Example

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/audio/texttoaudio.md

Instantiate and run the TextToAudio pipeline with a simple text prompt.

```python
from txtai.pipeline import TextToAudio

# Create and run pipeline
tta = TextToAudio()
tta("Describe the audio to generate here")
```

--------------------------------

### Setup Agent for College Search

Source: https://github.com/neuml/txtai/blob/master/examples/82_Agentic_College_Search.ipynb

Initializes the Agent with a specific LLM model, web search and web view tools, memory, instructions from a file, and a maximum step count. The `run` function is defined to execute agent queries and display results.

```python
from txtai import Agent
from IPython.display import display, Markdown

def run(query, reset=False):
    answer = agent(query, maxlength=60000, reset=reset)
    display(Markdown(answer))

agent = Agent(model="Qwen/Qwen3-4B-Instruct-2507", tools=["websearch", "webview"], memory=5, instructions="agents.md", max_steps=50)
```

--------------------------------

### Basic RAG Pipeline Setup and Usage

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/llm/rag.md

Demonstrates the basic setup of a RAG pipeline using txtai Embeddings and a specified LLM. It includes indexing data, creating the RAG instance with a custom template, and running a query.

```python
from txtai import Embeddings, RAG

# Input data
data = [
  "US tops 5 million confirmed virus cases",
  "Canada's last fully intact ice shelf has suddenly collapsed, " +
  "forming a Manhattan-sized iceberg",
  "Beijing mobilises invasion craft along coast as Taiwan tensions escalate",
  "The National Park Service warns against sacrificing slower friends " +
  "in a bear attack",
  "Maine man wins $1M from $25 lottery ticket",
  "Make huge profits without work, earn up to $100,000 a day"
]

# Build embeddings index
embeddings = Embeddings(content=True)
embeddings.index(data)

# Create the RAG pipeline
rag = RAG(embeddings, "Qwen/Qwen3-0.6B", template="""
  Answer the following question using the provided context.

  Question:
  {question}

  Context:
  {context}
""")

# Run RAG pipeline
rag("What was won?")
```

--------------------------------

### Install Dependencies for Image Similarity Search

Source: https://github.com/neuml/txtai/blob/master/examples/13_Similarity_search_with_images.ipynb

Installs txtai with similarity extras and downloads test data. This setup is required before creating an embeddings index.

```python
%%capture
!pip install torchvision ipyplot git+https://github.com/neuml/txtai#egg=txtai[similarity]

# Get test data
!wget -N https://github.com/neuml/txtai/releases/download/v3.5.0/tests.tar.gz
!tar -xvzf tests.tar.gz
```

--------------------------------

### Install Mamba Support and Link CUDA Libraries

Source: https://github.com/neuml/txtai/blob/master/examples/53_Integrate_LLM_Frameworks.ipynb

Installs the mamba-ssm library and links CUDA libraries for environment setup. This is a prerequisite for using Mamba models.

```python
%%capture
!pip install mamba-ssm

# Link CUDA libraries into environment
!export LC_ALL="en_US.UTF-8"
!export LD_LIBRARY_PATH="/usr/lib64-nvidia"
!export LIBRARY_PATH="/usr/local/cuda/lib64/stubs"
!ldconfig /usr/lib64-nvidia
```

--------------------------------

### Build and Run Go Labels Example

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

Compiles and executes the Go program to demonstrate text labeling functionality.

```python
!go run labels.go
```

--------------------------------

### Create and Run Summary Pipeline

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/text/summary.md

Instantiate the Summary pipeline and use it to summarize text. Ensure the txtai library is installed.

```python
from txtai.pipeline import Summary

# Create and run pipeline
summary = Summary()
summary("Enter long, detailed text to summarize here")
```

--------------------------------

### Get Embedding Vector

Source: https://github.com/neuml/txtai/blob/master/examples/74_OpenAI_Compatible_API.ipynb

Retrieves the embedding vector for a given text. This example shows the raw output including the index, vector, and object type.

```text
2, -0.02980010211467743, 0.02040782757103443, 0.06390708684921265, -0.0006256934138946235, -0.03723321482539177, -0.013057096861302853, 0.04114076867699623, -0.017182866111397743, -0.05549640208482742, 0.02064032293856144, 0.01683172956109047, -0.008635859936475754, 0.03218064829707146, 0.04564550518989563, -0.021377939730882645, 0.021940747275948524, 0.020410453900694847, 0.017982320860028267, 0.02150171808898449, 0.05921953544020653, -0.042486630380153656, -0.017924992367625237, -0.0114266537129879, -0.02765769325196743, 0.02116318792104721, -0.0008785029058344662, 0.00839359499514103, 0.007519723381847143, -0.07929962873458862, 0.01306573860347271, 0.00335461413487792, -0.013990496285259724, 0.00019492211868055165, -0.017358528450131416, -0.03889889642596245, -0.008545472286641598, 0.01378809567540884, 0.06300467997789383, 0.05205303058028221, 0.029774265363812447, 0.05180739611387253, -0.04484200477600098, -0.03888325020670891, -0.056330904364585876, 0.004683728329837322, 0.016883134841918945, -0.03816996142268181, 0.01605170965194702, 0.0022271168418228626, 0.0010828975355252624, 0.038834843784570694, 0.019416887313127518, 0.00031489337561652064, 0.05024728924036026, -0.05813521891832352, -0.006695288233458996, 0.042213670909404755, -0.012247920036315918, 0.028528228402137756, -0.02632697857916355, -0.05482589080929756, 0.00981950294226408, 0.02605678141117096, 0.06638345867395401, -0.018992368131875992, 0.04858163744211197, -0.014409814961254597, -0.0310173612087965, -0.05839765444397926, 0.08313969522714615, 0.05511852726340294, 0.047723494470119476, -0.033163223415613174, -0.040427759289741516, 0.011779758147895336, -0.05743969976902008, -0.021088508889079094, -0.018184570595622063, 0.022849485278129578, -0.010282794013619423, -0.010582848452031612, -0.038172293454408646, -0.02383989654481411, -0.047329485416412354, -0.02541566826403141, 0.027357304468750954, -0.06858660280704498, -0.06362185627222061, -0.0027012284845113754, -0.035492997616529465, -0.06344638019800186, 0.03718043491244316, 0.012817914597690105, 0.018238751217722893, -0.007895039394497871, 0.042976900935173035, -0.06253521889448166, 0.02173938974738121, 0.01422695629298687, 0.06118226796388626], index=0, object='embedding'}
```

--------------------------------

### File Task Configuration Example

Source: https://github.com/neuml/txtai/blob/master/docs/workflow/task/file.md

Shows how to configure the FileTask using a YAML workflow definition.

```yaml
workflow:
  tasks:
    - task: file
```

--------------------------------

### Initialize and Run Explainability

Source: https://github.com/neuml/txtai/blob/master/examples/32_Model_explainability.ipynb

Initializes a txtai application with embeddings, adds data, indexes it, and then calls the explain method to get explanations for a query. Ensure data is loaded before indexing.

```python
from txtai.app import Application

app = Application("""
writable: true
embeddings:
  path: sentence-transformers/nli-mpnet-base-v2
  content: true
""")

app.add([{"id": uid, "text": text} for uid, text in enumerate(data)])
app.index()

app.explain("feel good story", limit=1)
```

--------------------------------

### Index Data with High-Level Embeddings API

Source: https://github.com/neuml/txtai/blob/master/examples/78_Accessing_Low_Level_Vector_APIs.ipynb

Demonstrates the high-level Embeddings API to index a subset of text data and perform a similarity search. This is a convenient way to get started with txtai.

```python
from txtai import Embeddings

embeddings = Embeddings()
embeddings.index(ds["text"][:10000])
for uid, score in embeddings.search("nasa", 1):
    print(score, ds[uid]["text"][:100])
```

--------------------------------

### Train a QA Model with Few-Shot Learning

Source: https://github.com/neuml/txtai/blob/master/examples/20_Extractive_QA_to_build_structured_data.ipynb

Fine-tunes a pre-trained QA model using a small set of examples to guide its learning. This helps the model understand the specific types of questions and answers required for your data.

```python
import pandas as pd
from txtai.pipeline import HFTrainer, Questions, Labels

# Training data for few-shot learning
data = [
    {"question": "What is the url?",
     "context": "Faiss (https://github.com/facebookresearch/faiss) is a library for efficient similarity search.",
     "answers": "https://github.com/facebookresearch/faiss"},
    {"question": "What is the url", "context": "The last release was Wed Sept 25 2021", "answers": None},
    {"question": "What is the date?", "context": "The last release was Wed Sept 25 2021", "answers": "Wed Sept 25 2021"},
    {"question": "What is the date?", "context": "The order total comes to $44.33", "answers": None},
    {"question": "What is the amount?", "context": "The order total comes to $44.33", "answers": "$44.33"},
    {"question": "What is the amount?", "context": "The last release was Wed Sept 25 2021", "answers": None},
]

# Fine-tune QA model
trainer = HFTrainer()
model, tokenizer = trainer("distilbert-base-cased-distilled-squad", data, task="question-answering")
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/40_Text_to_Speech_Generation.ipynb

Installs txtai with pipeline extras for audio and data, ONNX runtime for GPU, and librosa. Also downloads NLTK data for English POS tagging.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline-audio,pipeline-data] onnxruntime-gpu librosa

# Install NLTK
import nltk
nltk.download('averaged_perceptron_tagger_eng')
```

--------------------------------

### Basic Workflow Example

Source: https://github.com/neuml/txtai/blob/master/docs/workflow/index.md

Creates a simple workflow that multiplies each input value by 2. Output must be consumed for execution.

```python
from txtai.workflow import Workflow, Task

workflow = Workflow([Task(lambda x: [y * 2 for y in x])])
list(workflow([1, 2, 3]))
```

--------------------------------

### Parallel Workflows with Multi-Output Tasks

Source: https://github.com/neuml/txtai/blob/master/examples/23_Tensor_workflows.ipynb

Demonstrates building a workflow with a task that has multiple actions, each applying a sentiment classifier. This setup is useful for ensemble methods or weighted classification. Ensure necessary libraries like `datasets` and `transformers` are installed.

```python
import time

from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

class Tokens:
    def __init__(self, texts):
        tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english")
        tokens = tokenizer(texts, padding=True, return_tensors="pt").to("cuda:0")

        self.inputs, self.attention = tokens["input_ids"], tokens["attention_mask"]

    def __len__(self):
        return len(self.inputs)

    def __getitem__(self, value):
        return (self.inputs[value], self.attention[value])

class Classify:
    def __init__(self, model):
        self.model = model

    def __call__(self, tokens):
        with torch.no_grad():
            inputs, attention = tokens
            outputs = self.model(input_ids=inputs, attention_mask=attention)
            outputs = outputs["logits"]

        return outputs

```

--------------------------------

### Audio Stream Pipeline for Music Playback

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

This example demonstrates playing 'Twinkle Twinkle Little Star' using txtai's AudioStream pipeline. It converts musical notes to frequencies and generates sine waves for playback. Ensure sounddevice, scipy, and numpy are installed.

```python
import numpy as np
from scipy.io.wavfile import write
from scipy.signal import chirp

# Note frequencies
NOTES = {
    "C4": 261.63,
    "D4": 293.66,
    "E4": 329.63,
    "F4": 349.23,
    "G4": 392.00,
    "A4": 440.00,
    "B4": 493.88,
    "C5": 523.25
}

# Melody: note(duration)
melody = [
    ("C4", 0.5), ("C4", 0.5), ("G4", 0.5), ("G4", 0.5), ("A4", 0.5), ("A4", 0.5), ("G4", 1.0),
    ("F4", 0.5), ("F4", 0.5), ("E4", 0.5), ("E4", 0.5), ("D4", 0.5), ("D4", 0.5), ("C4", 1.0)
]

SAMPLE_RATE = 44100  # samples per second
AMPLITUDE = np.iinfo(np.int16).max * 0.5

# Create audio stream
audio = AudioStream()

# Generate and play notes
for note, duration in melody:
    frequency = NOTES[note]
    num_samples = int(duration * SAMPLE_RATE)
    t = np.linspace(0, duration, num_samples, endpoint=False)
    wave = AMPLITUDE * np.sin(2 * np.pi * frequency * t)
    audio.write(wave.astype(np.int16))
    print(f"Playing {note} for {duration}s")

# Wait for playback to finish
audio.wait_done()
print("Playback finished.")
```

--------------------------------

### Install txtai with Pipeline dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with dependencies for all pipeline categories. The default install includes most common pipelines.

```bash
pip install txtai[pipeline]
```

--------------------------------

### Install txtai with Workflow dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with dependencies for all workflow tasks. The default install includes most common workflow tasks.

```bash
pip install txtai[workflow]
```

--------------------------------

### Basic Transcription Example

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/audio/transcription.md

Create and run the Transcription pipeline with a local audio file.

```python
from txtai.pipeline import Transcription

# Create and run pipeline
transcribe = Transcription()
transcribe("path to wav file")
```

--------------------------------

### Basic Segmentation Example

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/data/segmentation.md

Demonstrates creating and running the Segmentation pipeline to split text into sentences.

```python
from txtai.pipeline import Segmentation

# Create and run pipeline
segment = Segmentation(sentences=True)
segment("This is a test. And another test.")
```

--------------------------------

### Basic Labels Pipeline Usage

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/text/labels.md

Demonstrates the basic instantiation and execution of the Labels pipeline with sample text and labels.

```python
from txtai.pipeline import Labels

# Create and run pipeline
labels = Labels()
labels(
    ["Great news", "That's rough"],
    ["positive", "negative"]
)
```

--------------------------------

### Minimal install of txtai with default extras

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs the minimal txtai package with default extras, similar to a standard txtai installation but with a lighter base.

```bash
pip install txtai_minimal[default]
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/36_Run_txtai_in_native_code.ipynb

Installs the txtai library with pipeline dependencies, removes the unused TensorFlow package, and installs necessary development tools for Python and assembly.

```python
%%capture
!pip install git+https://github.com/neuml/txtai#egg=txtai[pipeline] sacremoses

# Remove tensorflow as it's not used and prints noisy log messages
!pip uninstall -y tensorflow

# Install python3.7-dev and nasm
!apt-get install python3.7-dev nasm
```

--------------------------------

### Setup TxtAI Agent with Tools and Memory

Source: https://github.com/neuml/txtai/blob/master/examples/83_TxtAI_got_skills.ipynb

Initializes a TxtAI Agent with a specified model, tools (websearch, webview, skill.md), and memory. The `run` function is defined to execute queries and display results.

```Python
from txtai import Agent
from IPython.display import display, Markdown

def run(query, reset=False):
    answer = agent(query, maxlength=50000, reset=reset)
    display(Markdown(answer))

agent = Agent(
    model="Qwen/Qwen3-4B-Instruct-2507",
    tools=["websearch", "webview", "skill.md"],
    memory=2,
    verbosity_level=0
)
```

--------------------------------

### Install txtai with Vectors dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with dependencies for additional vector methods.

```bash
pip install txtai[vectors]
```

--------------------------------

### Install txtai with Scoring dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with dependencies for additional scoring methods.

```bash
pip install txtai[scoring]
```

--------------------------------

### Install txtai with Model dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with dependencies for additional non-standard models.

```bash
pip install txtai[model]
```

--------------------------------

### Install PostgreSQL and pgvector

Source: https://github.com/neuml/txtai/blob/master/examples/61_Integrate_txtai_with_Postgres.ipynb

Install PostgreSQL and the pgvector extension to enable vector storage and search.

```shell
sudo apt install postgresql postgresql-contrib
CREATE EXTENSION vector;
```

--------------------------------

### Create a Task with a Pre-defined Action

Source: https://github.com/neuml/txtai/blob/master/docs/workflow/task/index.md

This example demonstrates creating a task using a pre-defined action, such as the 'Summary' action. This task will summarize each input element.

```python
summary = Summary()
Task(summary)
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/48_Benefits_of_hybrid_search.ipynb

Installs txtai, pytrec_eval, rank-bm25, and elasticsearch. Also uninstalls tensorflow.

```python
%%capture
!pip install txtai pytrec_eval rank-bm25 elasticsearch
!pip uninstall -y tensorflow
```

--------------------------------

### Install txtai

Source: https://github.com/neuml/txtai/blob/master/examples/26_Entity_extraction_workflows.ipynb

Installs the txtai library and its dependencies. This is a prerequisite for using the entity extraction pipeline.

```python
from txtai.pipeline import Entity

data = ["US tops 5 million confirmed virus cases",
        "Canada's last fully intact ice shelf has suddenly collapsed, forming a Manhattan-sized iceberg",
        "Beijing mobilises invasion craft along coast as Taiwan tensions escalate",
        "The National Park Service warns against sacrificing slower friends in a bear attack",
        "Maine man wins $1M from $25 lottery ticket",
        "Make huge profits without work, earn up to $100,000 a day"]

entity = Entity()

for x, e in enumerate(entity(data)):
  print(data[x])
  print(f"  {e}", "\n")
```

--------------------------------

### Install txtai with combined dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with multiple optional dependencies specified simultaneously.

```bash
pip install txtai[pipeline,workflow]
```

--------------------------------

### Create labels.rs Example

Source: https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb

A Rust program demonstrating text labeling using the txtai.rs API. It connects to a running txtai instance and labels sample text.

```rust
use std::error::Error;

use txtai::labels::Labels;

pub async fn labels() -> Result<(), Box<dyn Error>> {
    let labels = Labels::new("http://localhost:8000");

    let data = ["Wears a red suit and says ho ho",
                "Pulls a flying sleigh",
                "This is cut down and decorated",
                "Santa puts these under the tree",
                "Best way to spend the holidays"];

    println!("{:<40} {}", "Text", "Label");
    println!("{}", "-".repeat(75));

    for text in data.iter() {
        let tags = vec!["🎅 Santa Clause", "🦌 Reindeer", "🍪 Cookies", "🎄 Christmas Tree", "🎁 Gifts", "👪 Family"];
        let label = labels.label(text, &tags).await?[0].id;

        println!("{:<40} {}", text, tags[label]);
    }

    Ok(())
}
```

--------------------------------

### Basic Similarity Example

Source: https://github.com/neuml/txtai/blob/master/docs/pipeline/text/similarity.md

Instantiate and run the Similarity pipeline with a query and a list of texts.

```python
from txtai.pipeline import Similarity

# Create and run pipeline
similarity = Similarity()
similarity("feel good story", [
    "Maine man wins $1M from $25 lottery ticket", 
    "Don't sacrifice slower friends in a bear attack"
])
```

--------------------------------

### Install txtai with Cloud dependencies

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai with dependencies for interfacing with cloud compute services.

```bash
pip install txtai[cloud]
```

--------------------------------

### RAG Quick Start

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

This snippet shows how to set up and use a RAG pipeline for question answering. It covers data collection, text extraction, embedding database creation, and RAG pipeline configuration.

```python
import os

from txtai import Embeddings, RAG
from txtai.pipeline import Textractor

# Step 1: Collect files from local directory
# 
# Defaults to "data". Set to whereever your files are.
path = "data"
files = [os.path.join(path, f) for f in os.listdir(path) if os.path.isfile(os.path.join(path, f))]

# Step 2: Text Extraction / Chunking
# 
# Using section based chunking here. More complex options available such as semantic chunking, iterative chunking  
etc. 
# Documentation: https://neuml.github.io/txtai/pipeline/data/textractor 
# Supports Chonkie chunking as well: https://docs.chonkie.ai/oss/chunkers/overview
textractor = Textractor(backend="docling", sections=True)
chunks = []
for f in files:
 for chunk in textractor(f):
  chunks.append((f, chunk))

# Step 3: Build an embeddings database
# 
# The `path` parameter sets the vector embeddings model. Supports Hugging Face models, llama.cpp, Ollama, vLLM and  
more.
# Documentation: https://neuml.github.io/txtai/embeddings/
embeddings = Embeddings(content=True, path="Qwen/Qwen3-Embedding-0.6B", maxlength=2048)
embeddings.index(chunks)

# Step 4: Create RAG pipeline
# 
# Combines an embeddings database and an LLM.
# Supports Hugging Face models, llama.cpp, Ollama, vLLM and more
# Documentation: https://neuml.github.io/txtai/pipeline/text/rag

# User prompt template
template = """
 Answer the following question using the provided context. 

 Question:
 {question}

 Context:
 {context}
"""

rag = RAG(
 embeddings,
 "Qwen/Qwen3-0.6B",
 system="You are a friendly assistant",
 template=template,
 output="flatten",
)

question = "Summarize the main advancements made by BERT"
print(rag(question, maxlength=2048, stripthink=True))
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/61_Integrate_txtai_with_Postgres.ipynb

Installs the txtai library with necessary dependencies for ann, database, and graph functionalities.

```python
%%capture

# Install txtai and dependencies
!pip install git+https://github.com/neuml/txtai#egg=txtai[ann,database,graph]
```

--------------------------------

### RAG Pipeline Setup and Query

Source: https://github.com/neuml/txtai/blob/master/examples/84_Agent_Tools.ipynb

Sets up a RAG pipeline with custom prompt templates and executes a query. Requires embeddings and a language model.

```python
template = """
 Answer the following question using the provided context.

 Question:
 {question}

 Context:
 {context}
"""

rag = RAG(
 embeddings,
 "Qwen/Qwen3-0.6B",
 system="You are a friendly assistant",
 template=template,
 output="flatten",
)

question = "Summarize the main advancements made by BERT"
print(rag(question, maxlength=2048, stripthink=True))
```

--------------------------------

### Install txtai and Dependencies

Source: https://github.com/neuml/txtai/blob/master/examples/21_Export_and_run_other_machine_learning_models.ipynb

Installs the txtai library with pipeline and similarity extras, along with the datasets library.

```python
from datasets import load_dataset

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline

ds = load_dataset("emotion")

# Train the model
pipeline = Pipeline([
    ('tfidf', TfidfVectorizer()),
    ('lr', LogisticRegression(max_iter=250))
])

pipeline.fit(ds["train"]["text"], ds["train"]["label"])

# Determine accuracy on validation set
results = pipeline.predict(ds["validation"]["text"])
labels = ds["validation"]["label"]

results = [results[x] == label for x, label in enumerate(labels)]
print("Accuracy =", sum(results) / len(ds["validation"]))

```

--------------------------------

### Install txtai via conda

Source: https://github.com/neuml/txtai/blob/master/docs/install.md

Installs txtai using the conda package manager from the conda-forge channel.

```bash
conda install -c conda-forge txtai
```