===============
LIBRARY RULES
===============
From library maintainers:
- Infino is an embedded engine that runs in-process — there is no server or daemon to start. Open a catalog with connect(uri).
- Infino is a Rust engine with first-party Python and Node.js bindings — NOT a Python-only library. Install: 'pip install infino' (Python), 'npm install @infino-ai/infino' (Node.js), or 'cargo add infino' (Rust).
- Use memory:// for an in-process, ephemeral catalog; use a filesystem path or an s3://bucket/prefix (also az://container/prefix or gs://bucket/prefix) URI to persist to object storage (Amazon S3, Azure Blob, GCS, or local disk).
- Create a table from a schema plus an IndexSpec that declares which columns are indexed: .fts(column) for BM25 full-text and .vector(column, dim, ...) for vector search. An _id column is added automatically — do not define it yourself.
- Provide your own embedding vectors for vector columns and vector queries; the core engine stores and searches vectors but does not generate embeddings.
- Append rows with the table's append/add method. Writes become durable only when a commit succeeds — nothing is persisted before that, and readers see either the pre-commit or post-commit state, never a partial one.
- Run keyword search with bm25_search(column, query, k); it returns rows ranked by BM25 relevance with a score.
- Run vector search with vector_search(column, query_vector, k); it returns the nearest rows by the index's metric (cosine, L2, or dot) with a score.
- Hybrid search is first-class: hybrid_search(text_column, text_query, vector_column, query_vector, k) fuses BM25 and vector rankings with reciprocal-rank fusion — one engine over one copy of the data. It is also a SQL table function.
- Reach SQL through the connection's query_sql. The search functions bm25_search, vector_search, hybrid_search, token_match, and exact_match are SQL table functions, so search composes into a query as a relation you can JOIN, filter, and aggregate.
- Search methods take a projection naming the output columns. With no projection they return just _id and score (no row data is decoded); name the columns you want materialized.
- Updates and deletes are by id and are applied through tombstones over an append-only store — Infino is built for retrieval over largely append-only corpora, not high-rate row-level OLTP.
- Each superfile is a valid Apache Parquet file, so any Parquet reader (DataFusion, DuckDB, pyarrow) can read its columns directly; the embedded BM25 and vector index regions are skipped by generic readers.
- Infino ships as a Rust crate and as Python and Node.js bindings; the bindings mirror the same connect / create-table / append / search / query_sql surface.

### Install Dependencies

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/rag/02_hybrid_rag.ipynb

Installs the necessary libraries for running the hybrid search example, including infino, pyarrow, sentence-transformers, and datasets.

```python
# %pip install infino pyarrow sentence-transformers datasets
```

--------------------------------

### Infino Quickstart: Connect, Create Table, and Append Data

Source: https://github.com/infino-ai/infino/blob/main/infino-python/README.md

Connect to a catalog (local path, S3 URI, or ephemeral memory), define a schema with indexing specifications, and append data. Appends are atomic commits. The example uses a stand-in embedding function.

```python
import infino
import pyarrow as pa

# Connect to a catalog. Use a local path or an S3 URI for durable storage;
# "memory://" is ephemeral and handy for tests.
db = infino.connect("./data")

# Tiny stand-in for your embedding model so this runs as-is — a 16-dim
# one-hot by topic. Real embeddings are dense and higher-dimensional.
def embed(topic):
    v = [0.0] * 16
    v[topic] = 1.0
    return v

# Declare a schema and which columns to index. An "_id" column is added
# automatically — you don't define it. A vector column's dim must be in
# [16, 4096]; here we use 16, the floor.
schema = pa.schema([
    pa.field("source", pa.large_utf8(), nullable=False),
    pa.field("body", pa.large_utf8(), nullable=False),
    pa.field("embedding", pa.list_(pa.float32(), 16), nullable=False),
])
docs = db.create_table(
    "docs", schema, infino.IndexSpec().fts("body").vector("embedding", 16, 1, "cosine")
)

# Append rows. One append is one atomic commit.
docs.append([
    {"source": "help-center", "body": "To cancel a subscription, open Settings then Billing.", "embedding": embed(0)},
    {"source": "help-center", "body": "Refunds return to the original payment method.",         "embedding": embed(0)},
    {"source": "blog",        "body": "Enable dark mode under Settings then Appearance.",        "embedding": embed(1)},
])
```

--------------------------------

### Run Infino Demo Example

Source: https://github.com/infino-ai/infino/blob/main/README.md

Execute the demo example to perform an end-to-end tour, including building, BM25 + vector search, and reading back data as Parquet.

```bash
cargo run --example demo
```

--------------------------------

### Setup Python Virtual Environment and Install Requirements

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/README.md

This snippet shows the standard commands to create a Python virtual environment and activate it, followed by installing project dependencies from a requirements file. Ensure you are in the project's root directory.

```sh
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt
```

--------------------------------

### Setup and Initialization

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/crewai/03_knowledge_crew.ipynb

Initializes the environment, sets up the database directory, and connects to an Infino database. It also initializes the LLM, printing its status.

```python
import shutil
import sys
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path

sys.path.insert(0, str(Path.cwd().parent))  # examples/ root, where _shared/ lives

import infino
from crewai import Agent, Crew, Process, Task
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource
from crewai_infino import InfinoKnowledgeStorage

from _shared.crew import crew_llm
from _shared.embedding import DIM, embed, embed_query
from _shared.sql import query

DB_DIR = "./knowledge_data"
shutil.rmtree(DB_DIR, ignore_errors=True)  # start fresh each run

db = infino.connect(DB_DIR)
llm = crew_llm()
print("LLM:", "on" if llm else "off (build + search still run; crew is skipped)")
```

--------------------------------

### Setup and Initialization

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/01_live_data_rag.ipynb

Imports necessary libraries and sets up the environment for the Infino vector store. Initializes embeddings and a chat model, and configures the database directory.

```python
import sys
import shutil
from pathlib import Path

sys.path.insert(0, str(Path.cwd().parent))  # examples/ root, where _shared/ lives

import infino
import pyarrow as pa
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_infino import InfinoVectorStore

from _shared.lc import MiniLMEmbeddings, chat_model
from _shared.embedding import DIM, METRIC
from _shared.loaders import load_amazon
from _shared.sql import query

DB_DIR = "./live_inventory_data"
TABLE = "products"
shutil.rmtree(DB_DIR, ignore_errors=True)  # start fresh each run

embeddings = MiniLMEmbeddings()
llm = chat_model()  # None when no key is set; the demo still runs (prints context)
print("LLM answers:", "on" if llm else "off (retrieval only)")
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/crewai/README.md

Sets up a virtual environment and installs necessary Python packages for the project. Ensure you are in the project's root directory.

```shell
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt
```

--------------------------------

### Install Infino with npm

Source: https://github.com/infino-ai/infino/blob/main/README.md

Install the Infino Node.js package using npm.

```sh
npm install @infino-ai/infino
```

--------------------------------

### Running the Hybrid Search API

Source: https://github.com/infino-ai/infino/blob/main/infino-node/examples/hybrid-search-api/README.md

Install dependencies and start the Infino hybrid search server. The server will download the embedding model and catalog data on first run.

```sh
npm install
node index.mjs
```

--------------------------------

### Install Infino and Dependencies

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/inventory/01_live_inventory.ipynb

Installs the necessary libraries for using Infino and PyArrow, along with datasets for loading sample data.

```python
# %pip install infino pyarrow datasets
```

--------------------------------

### Python Quickstart: Connect and create table

Source: https://github.com/infino-ai/infino/blob/main/README.md

Connect to an in-memory database and create a table with specified schema and indexes. The schema includes text and vector fields.

```python
import infino
import pyarrow as pa

# A knowledge base your agent retrieves over. "memory://" is in-process;
# use "./data" or "s3://bucket/prefix" to persist.
db = infino.connect("memory://")

# Tiny stand-in for your embedding model so this runs as-is — a 16-dim
# one-hot by topic. Real embeddings are dense and higher-dimensional.
def embed(topic):                       # 0 = billing, 1 = appearance
    v = [0.0] * 16
    v[topic] = 1.0
    return v

schema = pa.schema([
    pa.field("source", pa.large_utf8(), nullable=False),
    pa.field("body", pa.large_utf8(), nullable=False),
    pa.field("embedding", pa.list_(pa.float32(), 16), nullable=False),
])
docs = db.create_table(
    "docs", schema,
    infino.IndexSpec().fts("body").vector("embedding", 16, 1, "cosine"),
)
```

--------------------------------

### Install Infino with uv

Source: https://github.com/infino-ai/infino/blob/main/infino-python/README.md

Install the Infino library using uv, a fast Python package installer and resolver. Use 'uv add' for uv-managed projects or 'uv pip install' for the active environment.

```sh
uv add infino            # add to a uv-managed project
```

```sh
uv pip install infino    # install into the active environment
```

--------------------------------

### Node.js Quickstart: Connect and create table

Source: https://github.com/infino-ai/infino/blob/main/README.md

Connect to an in-memory database and create a table with specified schema and indexes using JavaScript. The schema defines text and vector fields.

```javascript
import { connect, IndexSpec } from "@infino-ai/infino";

// A knowledge base your agent retrieves over. "memory://" is in-process;
// use "./data" or "s3://bucket/prefix" to persist.
const db = connect("memory://");

// Tiny stand-in for your embedding model so this runs as-is — a 16-dim
// one-hot by topic. Real embeddings are dense and higher-dimensional.
const embed = (topic) => { const v = Array(16).fill(0.0); v[topic] = 1.0; return v; };

const docs = db.createTable(
  "docs",
  { source: "large_utf8", body: "large_utf8", embedding: { vector: 16 } },
  new IndexSpec().fts("body").vector("embedding", 16, 1, "cosine"),
);
```

--------------------------------

### Run infino Demo

Source: https://github.com/infino-ai/infino/blob/main/CONTRIBUTING.md

Execute the demo example to see the end-to-end functionality of the infino stack, including building, querying, and reading data.

```bash
cargo run --example demo
```

--------------------------------

### Build Infino from Source

Source: https://github.com/infino-ai/infino/blob/main/infino-python/README.md

Compile the Infino extension and install it into a virtual environment using maturin. Requires a Rust toolchain. Includes steps for environment setup, dependency installation, and running tests.

```sh
python3 -m venv .venv && source .venv/bin/activate
pip install maturin pytest pyarrow
maturin develop          # compile the extension and install it into the venv
pytest tests/
```

--------------------------------

### Install Infino with pip

Source: https://github.com/infino-ai/infino/blob/main/README.md

Install the Infino Python package using pip. For faster installation, uv can be used.

```sh
pip install infino
```

```sh
uv pip install infino
```

--------------------------------

### Install Infino with pip

Source: https://github.com/infino-ai/infino/blob/main/infino-python/README.md

Install the Infino library using pip. Requires Python 3.9 or newer.

```sh
pip install infino
```

--------------------------------

### Setup and Initialization for Infino Agent

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/03_support_ops_agent.ipynb

Initializes the environment, imports necessary libraries, and sets up the Infino database connection and chat model. This code is required before building the Infino store and agent.

```python
import sys
import shutil
import time
from pathlib import Path

sys.path.insert(0, str(Path.cwd().parent))  # examples/ root, where _shared/ lives

import infino
import pyarrow as pa
from langchain.agents import create_agent
from langchain_core.globals import set_llm_cache
from langchain_core.tools import tool
from langgraph.checkpoint.memory import MemorySaver

from langchain_infino import InfinoSemanticCache, InfinoVectorStore

from _shared.lc import MiniLMEmbeddings, chat_model
from _shared.embedding import DIM, METRIC
from _shared.loaders import load_amazon
from _shared.sql import query

DB_DIR = "./agent_data"
TABLE = "catalog"
shutil.rmtree(DB_DIR, ignore_errors=True)  # start fresh each run

embeddings = MiniLMEmbeddings()
llm = chat_model()
db = infino.connect(DB_DIR)
if llm is None:
    print("This example needs an LLM key (tool-calling agent). See the README.")
```

--------------------------------

### Infino Quickstart: Connect, Schema, and Search

Source: https://github.com/infino-ai/infino/blob/main/infino-node/README.md

Connect to a catalog, define a table schema with full-text and vector indexes, append data, and perform various types of searches (keyword, semantic, hybrid, SQL). Supports CommonJS require syntax as well.

```javascript
import { connect, IndexSpec } from "@infino-ai/infino";

// Connect to a catalog. Use a local path or an S3 URI for durable storage;
// "memory://" is ephemeral and handy for tests.
const db = connect("./data");

// Tiny stand-in for your embedding model so this runs as-is — a 16-dim
// one-hot by topic. Real embeddings are dense and higher-dimensional.
const embed = (topic) => { const v = Array(16).fill(0.0); v[topic] = 1.0; return v; };

// Declare a schema and which columns to index. An `_id` column is added
// automatically — you don't define it.
const docs = db.createTable(
  "docs",
  { source: "large_utf8", body: "large_utf8", embedding: { vector: 16 } },
  new IndexSpec().fts("body").vector("embedding", 16, 1, "cosine"),
);

// Append rows. One append is one atomic commit.
docs.append([
  { source: "help-center", body: "To cancel a subscription, open Settings then Billing.", embedding: embed(0) },
  { source: "help-center", body: "Refunds return to the original payment method.",         embedding: embed(0) },
  { source: "blog",        body: "Enable dark mode under Settings then Appearance.",        embedding: embed(1) },
]);

// Retrieve context to ground an agent's next answer — keyword, vector,
// hybrid (BM25 + vector fused in one pass), or SQL:
const keyword  = docs.bm25Search("body", "cancel subscription", 5);                          // BM25
const semantic = docs.vectorSearch("embedding", embed(0), 5);                                // vector kNN
const hybrid   = docs.hybridSearch("body", "cancel subscription", "embedding", embed(0), 5); // fused
const billing  = db.querySql("SELECT body FROM docs WHERE source = 'help-center'");          // SQL filter

```

```javascript
const { connect, IndexSpec } = require("@infino-ai/infino");
```

--------------------------------

### Node.js Bindings: Verify Installation

Source: https://github.com/infino-ai/infino/blob/main/CONTRIBUTING.md

Verify that the Node.js package ships correctly by packing it, installing it in a temporary project, and running a roundtrip test.

```bash
make node-verify
```

--------------------------------

### Infino Quickstart: Perform Searches

Source: https://github.com/infino-ai/infino/blob/main/infino-python/README.md

Retrieve context for agent grounding using keyword (BM25), vector (kNN), hybrid (BM25 + vector), or SQL searches. Each search type returns a pyarrow.Table.

```python
# Retrieve context to ground an agent's next answer — keyword, vector,
# hybrid (BM25 + vector fused in one pass), or SQL. Each returns a pyarrow.Table.
keyword  = docs.bm25_search("body", "cancel subscription", k=5)                           # BM25
semantic = docs.vector_search("embedding", embed(0), k=5)                                 # vector kNN
hybrid   = docs.hybrid_search("body", "cancel subscription", "embedding", embed(0), k=5)  # fused
billing  = db.query_sql("SELECT body FROM docs WHERE source = 'help-center'")             # SQL filter
```

--------------------------------

### Install Pre-commit Hooks

Source: https://github.com/infino-ai/infino/blob/main/README.md

Install pre-commit hooks to automatically catch formatting and linting issues before committing code.

```bash
pre-commit install
```

--------------------------------

### Clone and Build infino

Source: https://github.com/infino-ai/infino/blob/main/CONTRIBUTING.md

Clone the infino repository and build the project using Cargo. This is the initial setup step for development.

```bash
git clone git@github.com:infino-ai/infino.git
cd infino
cargo build
```

--------------------------------

### Setup Infino and Langchain Environment

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/02_research_assistant.ipynb

Initializes the Infino database connection, embeddings, and chat model. Ensures the environment is set up for subsequent operations. The chat model is optional and only needed for the self-querying part.

```python
import sys
import shutil
from pathlib import Path

sys.path.insert(0, str(Path.cwd().parent))  # examples/ root, where _shared/ lives

import infino
import pyarrow as pa
from langchain_infino import InfinoVectorStore, InfinoTranslator

from _shared.lc import MiniLMEmbeddings, chat_model
from _shared.embedding import DIM, METRIC
from _shared.loaders import load_ms_marco, load_arxiv

DB_DIR = "./research_data"
shutil.rmtree(DB_DIR, ignore_errors=True)  # start fresh each run

embeddings = MiniLMEmbeddings()
llm = chat_model()  # None when no key is set; only part 3 needs it
db = infino.connect(DB_DIR)
print("LLM (part 3):", "on" if llm else "off")
```

--------------------------------

### Install Infino Node.js Package

Source: https://github.com/infino-ai/infino/blob/main/infino-node/README.md

Install the Infino Node.js package using npm. A prebuilt native binary is automatically selected, eliminating the need for a Rust toolchain. Requires Node.js version 18 or higher.

```sh
npm install @infino-ai/infino
```

--------------------------------

### Infino Codebase Navigation Guide

Source: https://github.com/infino-ai/infino/blob/main/AGENTS.md

A guide to help developers locate specific functionalities within the Infino codebase. It maps features to their corresponding file paths.

```markdown
| If the change is about…                     | Edit here                                                             |
| ------------------------------------------- | --------------------------------------------------------------------- |
| BM25 scoring                                | `src/superfile/fts/bm25.rs`                                           |
| Posting list iteration / skip table         | `src/superfile/fts/posting.rs`                                        |
| Vector quantization codec                   | `src/superfile/vector/quant.rs`                                       |
| Vector distance kernel (incl. SIMD)         | `src/superfile/vector/distance.rs`                                    |
| Tokenizer                                   | `src/superfile/fts/tokenize.rs`                                       |
| Partition strategy                          | `src/supertable/manifest/partition.rs`                                |
| Skip pruning (Bloom / min-max / term range) | `src/supertable/manifest/{bloom,aggregates,term_range,list_prune}.rs` |
| Commit / writer slot / handle               | `src/supertable/writer.rs` + `src/supertable/handle.rs`               |
| Tombstones (delete-path / query-filter)     | `src/supertable/{wal,tombstones}/`                                    |
| New storage backend                         | `src/storage/`                                                        |
| File-format byte layout                     | `src/superfile/format/`                                               |
| Catalog / `connect` / `Connection`          | `src/catalog/`                                                        |
| Public error mapping                        | `src/error.rs`                                                        |
```

--------------------------------

### Python Quickstart: Append data and search

Source: https://github.com/infino-ai/infino/blob/main/README.md

Append data to the 'docs' table and perform various search operations including BM25, semantic vector search, and filtered vector search.

```python
docs.append([
    {"source": "help-center", "body": "To cancel a subscription, open Settings then Billing.", "embedding": embed(0)},
    {"source": "help-center", "body": "Refunds return to the original payment method.",         "embedding": embed(0)},
    {"source": "blog",        "body": "Enable dark mode under Settings then Appearance.",        "embedding": embed(1)},
])

# Retrieve context to ground the agent's next answer:
keyword  = docs.bm25_search("body", "cancel subscription", 5)               # BM25
semantic = docs.vector_search("embedding", embed(0), 5)                     # vector kNN
# vector kNN, restricted to rows whose body matches a keyword (pushdown filter):
filtered = docs.vector_search("embedding", embed(0), 5, filter_column="body", filter_query="billing")
billing  = db.query_sql("SELECT body FROM docs WHERE source = 'help-center'")  # SQL filter
```

--------------------------------

### Example Agent Interaction: Keyword Search

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/03_support_ops_agent.ipynb

Demonstrates asking the agent about a specific product to trigger keyword search. The output confirms product availability and price.

```python
ask("do you carry the NovaSound earbuds, and what's the price?")  # -> keyword

```

--------------------------------

### Setup and Data Ingestion with Infino

Source: https://github.com/infino-ai/infino/blob/main/crate-docs.md

Connects to an in-memory database, defines a schema with text and vector fields, creates a table with FTS and vector indexes, and appends data. This sets up the environment for subsequent retrieval operations.

```rust
use std::sync::Arc;

use infino::arrow_array::{FixedSizeListArray, Float32Array, LargeStringArray, RecordBatch};
use infino::arrow_schema::{DataType, Field, Schema};
use infino::{connect, BoolMode, IndexSpec, Metric, VectorFilter, VectorSearchOptions};

// Tiny stand-in for your embedding model so this runs as-is — a 16-dim
// one-hot by topic. Real embeddings are dense and higher-dimensional.
fn embed(topic: usize) -> Vec<f32> {
    let mut v = vec![0.0_f32; 16];
    v[topic] = 1.0;
    v
}

# fn main() -> Result<(), Box<dyn std::error::Error>> {
// A knowledge base your agent retrieves over. "memory://" is in-process;
// use "./data" or "s3://bucket/prefix" to persist.
let db = connect("memory://")?;

let item = Arc::new(Field::new("item", DataType::Float32, true));
let schema = Arc::new(Schema::new(vec![
    Field::new("source", DataType::LargeUtf8, false),
    Field::new("body", DataType::LargeUtf8, false),
    Field::new("embedding", DataType::FixedSizeList(item.clone(), 16), false),
]));
let docs = db.create_table(
    "docs",
    schema.clone(),
    IndexSpec::new().fts("body").vector("embedding", 16, 1, Metric::Cosine),
)?;

let flat: Vec<f32> = [0usize, 0, 1].iter().flat_map(|&t| embed(t)).collect();
docs.append(&RecordBatch::try_new(
    schema,
    vec![
        Arc::new(LargeStringArray::from(vec!["help-center", "help-center", "blog"])),
        Arc::new(LargeStringArray::from(vec![
            "To cancel a subscription, open Settings then Billing.",
            "Refunds return to the original payment method.",
            "Enable dark mode under Settings then Appearance.",
        ])),
        Arc::new(FixedSizeListArray::new(item, 16, Arc::new(Float32Array::from(flat)), None)),
    ],
)?)?;

// Retrieve context to ground the agent's next answer:
let keyword = docs.bm25_search("body", "cancel subscription", 5, BoolMode::Or, None)?;
let semantic = docs.vector_search("embedding", &embed(0), 5, VectorSearchOptions::new(), None, None)?;
// hybrid: BM25 + vector, fused with reciprocal-rank fusion:
let hybrid = docs.hybrid_search(
    "body", "cancel subscription", BoolMode::Or,
    "embedding", &embed(0), VectorSearchOptions::new(), 5, None,
)?; 
// vector kNN, restricted to rows whose body matches a keyword (pushdown filter):
let filtered = docs.vector_search(
    "embedding", &embed(0), 5, VectorSearchOptions::new(),
    Some(VectorFilter { column: "body", query: "billing", mode: BoolMode::Or }), None,
)?; 
let billing = db.query_sql("SELECT body FROM docs WHERE source = 'help-center'")?;
assert_eq!(keyword.iter().map(|b| b.num_rows()).sum::<usize>(), 1);   // BM25
assert!(semantic.iter().map(|b| b.num_rows()).sum::<usize>() >= 1);   // vector kNN
assert!(hybrid.iter().map(|b| b.num_rows()).sum::<usize>() >= 1);     // hybrid (BM25 + vector)
assert_eq!(filtered.iter().map(|b| b.num_rows()).sum::<usize>(), 1);  // vector + keyword filter
assert_eq!(billing.iter().map(|b| b.num_rows()).sum::<usize>(), 2);   // SQL filter
# Ok(())
# }

```

--------------------------------

### Setup and Indexing Code in Infino

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/code_search/01_code_search.ipynb

This snippet sets up the database directory, connects to Infino, defines a schema with FTS and vector fields, creates an index specification, and appends data to the 'functions' table. It's used for initializing and populating the code search index.

```python
DB_DIR = "./code_data"
shutil.rmtree(DB_DIR, ignore_errors=True)

db = infino.connect(DB_DIR)
schema = pa.schema([
    pa.field("func_name", pa.large_utf8(), nullable=False),
    pa.field("code", pa.large_utf8(), nullable=False),
    pa.field("docstring", pa.large_utf8(), nullable=False),
    pa.field("summary", pa.large_utf8(), nullable=False),
    pa.field("repo", pa.large_utf8(), nullable=False),
    pa.field("url", pa.large_utf8(), nullable=False),
    pa.field("language", pa.large_utf8(), nullable=False),
    pa.field("emb", pa.list_(pa.float32(), DIM), nullable=False),
])
spec = (
    infino.IndexSpec()
    .fts("code")
    .fts("func_name")
    .vector("emb", DIM, n_cent=128, metric=METRIC)
)
table = db.create_table("functions", schema, spec)

# Embed the docstrings: natural-language search matches a query against these.
embeddings = embed([f["docstring"] for f in funcs])


def column(key):
    return pa.array([f[key] for f in funcs], type=pa.large_utf8())


table.append(pa.record_batch([
    column("func_name"), column("code"), column("docstring"), column("summary"),
    column("repo"), column("url"), column("language"),
    as_vector_column(embeddings),
], schema=schema))
print("indexed", db.query_sql("SELECT COUNT(*) AS n FROM functions").to_pydict()["n"][0], "functions")
```

--------------------------------

### Initialize Infino and CrewAI Environment

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/crewai/01_support_triage_crew.ipynb

Sets up the necessary imports, database directory, and connects to the Infino database. It also initializes the LLM for CrewAI agents. The database directory is cleared to ensure a fresh start.

```python
import shutil
import sys
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path

sys.path.insert(0, str(Path.cwd().parent))  # examples/ root, where _shared/ lives

import infino
import pyarrow as pa
from crewai import Agent, Crew, Process, Task
from crewai_infino import InfinoIndex

from _shared.crew import crew_llm
from _shared.embedding import DIM, METRIC, embed, embed_query
from _shared.loaders import load_amazon
from _shared.sql import query

DB_DIR = "./support_data"
TABLE = "support_kb"
shutil.rmtree(DB_DIR, ignore_errors=True)  # start fresh each run

db = infino.connect(DB_DIR)
llm = crew_llm()
print("LLM:", "on" if llm else "off (build + search still run; crew is skipped)")
```

--------------------------------

### Node.js Quickstart: Append data and search

Source: https://github.com/infino-ai/infino/blob/main/README.md

Append data to the 'docs' table and perform various search operations including BM25, semantic vector search, and filtered vector search using JavaScript.

```javascript
docs.append([
  { source: "help-center", body: "To cancel a subscription, open Settings then Billing.", embedding: embed(0) },
  { source: "help-center", body: "Refunds return to the original payment method.",         embedding: embed(0) },
  { source: "blog",        body: "Enable dark mode under Settings then Appearance.",        embedding: embed(1) },
]);

// Retrieve context to ground the agent's next answer:
const keyword  = docs.bm25Search("body", "cancel subscription", 5);            // BM25
const semantic = docs.vectorSearch("embedding", embed(0), 5);                  // vector kNN
// vector kNN, restricted to rows whose body matches a keyword (pushdown filter):
const filtered = docs.vectorSearch("embedding", embed(0), 5, { filter: { column: "body", query: "billing" } });
const billing  = db.querySql("SELECT body FROM docs WHERE source = 'help-center'");  // SQL filter
```

--------------------------------

### Build Infino from Source

Source: https://github.com/infino-ai/infino/blob/main/infino-node/README.md

Build the Infino Node.js binding from source using npm, requiring a Rust toolchain. This process includes installation, building, and testing.

```sh
cd infino-node
npm install && npm run build && npm test
```

--------------------------------

### Example Agent Interaction: SQL Query

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/03_support_ops_agent.ipynb

Demonstrates asking the agent a question that requires data aggregation, triggering a SQL query. The output provides the count of products meeting the criteria.

```python
ask("how many products are rated 4.5 or higher?")                 # -> SQL

```

--------------------------------

### Example Agent Interaction: Semantic Search

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/03_support_ops_agent.ipynb

Demonstrates asking the agent a vague question to trigger semantic search. The output shows the agent's response, including a product suggestion and a note on search quality.

```python
ask("I want wireless earbuds for running, budget around $50")     # -> semantic

```

--------------------------------

### Initialize Infino Database and Table

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/inventory/01_live_inventory.ipynb

Sets up a local Infino database directory, defines a table schema with full-text search on the 'text' column, and creates the 'products' table.

```python
DB_DIR = "./inventory_data"
shutil.rmtree(DB_DIR, ignore_errors=True)

db = infino.connect(DB_DIR)
schema = pa.schema([
    pa.field("title", pa.large_utf8(), nullable=False),
    pa.field("text", pa.large_utf8(), nullable=False),
    pa.field("price", pa.float64(), nullable=False),
    pa.field("rating", pa.float64(), nullable=False),
    pa.field("category", pa.large_utf8(), nullable=False),
    pa.field("store", pa.large_utf8(), nullable=False),
])
spec = infino.IndexSpec().fts("text")
table = db.create_table("products", schema, spec)
```

--------------------------------

### Code Search Notebook Example

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/code_search/README.md

References the main notebook for code search examples. This notebook covers exact_match, vector_search, bm25_search, and hybrid_search functionalities.

```text
`01_code_search.ipynb`
```

--------------------------------

### Build and Index Product Catalog

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/01_live_data_rag.ipynb

Loads product data, prepares texts, metadata, and IDs, then creates an Infino vector store. Promotes 'price' and 'rating' to metadata columns for SQL filtering and attaches other metadata to documents.

```python
products = load_amazon(n=600)
ids = [f"p-{i}" for i in range(len(products))]
texts = [p["text"] for p in products]
metadatas = [
    {"title": p["title"], "category": p["category"],
     "price": p["price"], "rating": p["rating"]}
    for p in products
]
metadata_columns = [
    pa.field("price", pa.float64(), nullable=False),
    pa.field("rating", pa.float64(), nullable=False),
]

db = infino.connect(DB_DIR)
store = InfinoVectorStore.from_texts(
    texts, embeddings, metadatas,
    connection=db, table_name=TABLE, dim=DIM, metric=METRIC,
    ids=ids, metadata_columns=metadata_columns,
)
print("indexed", query(db, f"SELECT COUNT(*) AS n FROM {TABLE}")["n"][0], "products")
```

--------------------------------

### Get Table Schema

Source: https://github.com/infino-ai/infino/blob/main/docs/architecture/overview.md

Retrieves the schema of a Supertable.

```python
schema = table.schema()
```

--------------------------------

### Add Infino Crate

Source: https://github.com/infino-ai/infino/blob/main/crate-docs.md

Installs the Infino crate using Cargo. This is the standard way to add Infino as a dependency to your Rust project.

```sh
cargo add infino
```

--------------------------------

### Connect, Create Table, Index, and Search with Infino AI

Source: https://github.com/infino-ai/infino/blob/main/README.md

Demonstrates connecting to an in-memory database, defining a schema, creating a table with Full-Text Search (FTS) and vector indexes, appending data, and performing BM25 and vector searches. Includes an example of a filtered vector search and SQL query.

```rust
use std::sync::Arc;

use infino::arrow_array::{FixedSizeListArray, Float32Array, LargeStringArray, RecordBatch};
use infino::arrow_schema::{DataType, Field, Schema};
use infino::{connect, BoolMode, IndexSpec, Metric, VectorFilter, VectorSearchOptions};

// Tiny stand-in for your embedding model so this runs as-is — a 16-dim
// one-hot by topic. Real embeddings are dense and higher-dimensional.
fn embed(topic: usize) -> Vec<f32> {
    let mut v = vec![0.0_f32; 16];
    v[topic] = 1.0;
    v
}

# fn main() -> Result<(), Box<dyn std::error::Error>> {
// A knowledge base your agent retrieves over. "memory://" is in-process;
// use "./data" or "s3://bucket/prefix" to persist.
let db = connect("memory://")?;

let item = Arc::new(Field::new("item", DataType::Float32, true));
let schema = Arc::new(Schema::new(vec![
    Field::new("source", DataType::LargeUtf8, false),
    Field::new("body", DataType::LargeUtf8, false),
    Field::new("embedding", DataType::FixedSizeList(item.clone(), 16), false),
]));
let docs = db.create_table(
    "docs",
    schema.clone(),
    IndexSpec::new().fts("body").vector("embedding", 16, 1, Metric::Cosine),
)?;

let flat: Vec<f32> = [0usize, 0, 1].iter().flat_map(|&t| embed(t)).collect();
docs.append(&RecordBatch::try_new(
    schema,
    vec![
        Arc::new(LargeStringArray::from(vec!["help-center", "help-center", "blog"])),
        Arc::new(LargeStringArray::from(vec![
            "To cancel a subscription, open Settings then Billing.",
            "Refunds return to the original payment method.",
            "Enable dark mode under Settings then Appearance.",
        ])),
        Arc::new(FixedSizeListArray::new(item, 16, Arc::new(Float32Array::from(flat)), None)),
    ],
)?)?;

// Retrieve context to ground the agent's next answer:
let keyword = docs.bm25_search("body", "cancel subscription", 5, BoolMode::Or, None)?;
let semantic = docs.vector_search("embedding", &embed(0), 5, VectorSearchOptions::new(), None, None)?;
// vector kNN, restricted to rows whose body matches a keyword (pushdown filter):
let filtered = docs.vector_search(
    "embedding", &embed(0), 5, VectorSearchOptions::new(),
    Some(VectorFilter { column: "body", query: "billing", mode: BoolMode::Or }), None,
)?;
let billing = db.query_sql("SELECT body FROM docs WHERE source = 'help-center'")?;
assert_eq!(keyword.iter().map(|b| b.num_rows()).sum::<usize>(), 1);   // BM25
assert!(semantic.iter().map(|b| b.num_rows()).sum::<usize>() >= 1);   // vector kNN
assert_eq!(filtered.iter().map(|b| b.num_rows()).sum::<usize>(), 1);  // vector + keyword filter
assert_eq!(billing.iter().map(|b| b.num_rows()).sum::<usize>(), 2);   // SQL filter
# Ok(())
# }

```

--------------------------------

### Performing a Hybrid Search

Source: https://github.com/infino-ai/infino/blob/main/infino-node/examples/hybrid-search-api/README.md

Send a GET request to the search endpoint with a query string. The 'k' parameter specifies the number of results to return.

```sh
curl 'http://localhost:3000/search?q=gift+for+someone+who+loves+cooking'
```

```sh
curl 'http://localhost:3000/search?q=something+to+keep+skin+moisturized&k=10'
```

--------------------------------

### Build Infino Index with Product Data and Support Articles

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/crewai/01_support_triage_crew.ipynb

Creates an Infino index, promoting product price, rating, and category to scalar columns for SQL filtering. It then adds product data and specific support articles, including error codes and how-tos, to the index. The number of indexed documents is printed.

```python
products = load_amazon(n=200)
index = InfinoIndex.create(
    db, TABLE,
    embed_documents=embed, embed_query=embed_query, dim=DIM, metric=METRIC,
    metadata_columns=[
        pa.field("price", pa.float64(), nullable=False),
        pa.field("rating", pa.float64(), nullable=False),
        pa.field("category", pa.large_utf8(), nullable=False),
    ],
)
index.add_texts(
    [p["text"] for p in products],
    metadatas=[{"price": p["price"], "rating": p["rating"], "category": p["category"]} 
               for p in products],
)

# Seeded support articles with a known error code and a reset how-to.
ARTICLES = [
    "NovaSound Z9 earbuds: reset by holding both earbuds in the case for 10 seconds "
    "until the LED blinks white.",
    "Error E-507 on the NovaSound Z9 means a stale Bluetooth cache. Forget the device "
    "on your phone, then re-pair to clear it.",
    "HydroPeak 32oz bottle: the lid is dishwasher safe on the top rack; do not "
    "microwave the insulated body.",
]
index.add_texts(
    ARTICLES,
    metadatas=[{"price": 0.0, "rating": 0.0, "category": "Support"} for _ in ARTICLES],
    ids=[f"kb-{i}" for i in range(len(ARTICLES))],
)
print("indexed", query(db, f"SELECT COUNT(*) AS n FROM {TABLE}")["n"][0], "docs")
```

--------------------------------

### Query Baseline Inventory

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/inventory/01_live_inventory.ipynb

Executes a SQL query to get the total count and price range of products. Use this to establish a baseline before making changes.

```python
base = query(db, "SELECT COUNT(*) n, MIN(price) cheapest, MAX(price) priciest FROM products")
print(f"products: {base['n'][0]}   price range: ${base['cheapest'][0]:.2f} - ${base['priciest'][0]:.2f}")
```

--------------------------------

### Generate and Open Infino Documentation

Source: https://github.com/infino-ai/infino/blob/main/README.md

Generate the full API documentation locally, excluding dependencies, and open it in a web browser. This is equivalent to the docs.rs rendering.

```bash
make doc
cargo doc --no-deps --open
```

--------------------------------

### Example Agent Interaction: Memory and Filtering

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/03_support_ops_agent.ipynb

Demonstrates asking a follow-up question that relies on previous context (memory) and filtering. The agent suggests a gift based on prior criteria.

```python
ask("of the highly rated ones, suggest a gift under $40")         # -> memory

```

--------------------------------

### Connect to Infino with Options

Source: https://github.com/infino-ai/infino/blob/main/public-api.txt

Establishes a connection to Infino with specified connection options.

```APIDOC
## connect_with

### Description
Establishes a connection to Infino using a connection string and custom connection options.

### Function Signature
`pub fn infino::connect_with(impl core::convert::AsRef<str>, infino::ConnectOptions) -> core::result::Result<infino::Connection, infino::InfinoError>`

### Parameters
- `connection_string` (impl core::convert::AsRef<str>) - The string representing the connection details.
- `options` (infino::ConnectOptions) - The connection options to use.

### Returns
- `core::result::Result<infino::Connection, infino::InfinoError>` - A result type that is either a successful `infino::Connection` or an `infino::InfinoError`.
```

--------------------------------

### Add Infino Crate without Mimalloc

Source: https://github.com/infino-ai/infino/blob/main/crate-docs.md

Installs the Infino crate while disabling the default mimalloc global allocator. Use this if your project already sets a global allocator to avoid conflicts.

```toml
infino = { version = "0.1", default-features = false }
```

--------------------------------

### Import Libraries

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/rag/02_hybrid_rag.ipynb

Imports required libraries for the hybrid search example, including sys, pathlib, shutil, infino, pyarrow, and custom modules for embedding, data loading, and SQL operations.

```python
import sys
from pathlib import Path

# Make the repo's shared helpers importable.
sys.path.insert(0, str(Path.cwd().parent))

import shutil

import infino
import pyarrow as pa

from _shared.embedding import DIM, METRIC, embed, embed_query, as_vector_column
from _shared.loaders import load_ms_marco
from _shared.sql import sql_lit, query
```

--------------------------------

### Focus Locomo Recall Report on Specific Question Case

Source: https://github.com/infino-ai/infino/blob/main/examples/locomo-recall/README.md

Run the locomo-recall example to focus the report on a single question identified by its index. Useful for isolating problems with a specific query.

```sh
cargo run --example locomo-recall -- --case=42     # focus one question by index
```

--------------------------------

### Build and Index Infino Catalog Store

Source: https://github.com/infino-ai/infino/blob/main/infino-python/examples/langchain/03_support_ops_agent.ipynb

Loads product data, creates an InfinoVectorStore with specified metadata columns for price and rating, and adds featured items. This prepares the data store for agent retrieval.

```python
products = load_amazon(n=300)
store = InfinoVectorStore.from_texts(
    [p["text"] for p in products], embeddings,
    metadatas=[{"title": p["title"], "category": p["category"],
                "price": p["price"], "rating": p["rating"]} for p in products],
    connection=db, table_name=TABLE, dim=DIM, metric=METRIC,
    metadata_columns=[pa.field("price", pa.float64(), nullable=False),
                      pa.field("rating", pa.float64(), nullable=False)],
)

FEATURED = [
    ("NovaSound Z9 wireless running earbuds, sweatproof, 30h battery",
     "NovaSound Z9", 49.0, 4.7),
    ("HydroPeak insulated steel water bottle, 32oz, keeps drinks cold 24h",
     "HydroPeak 32oz Bottle", 24.0, 4.6),
    ("ZenFlow non-slip yoga mat, 6mm cushioned eco TPE",
     "ZenFlow Yoga Mat", 33.0, 4.8),
    ("LumaDesk LED desk lamp, adjustable warmth, USB-C charging",
     "LumaDesk LED Lamp", 39.0, 4.4),
]
store.add_texts(
    [f[0] for f in FEATURED],
    metadatas=[{"title": f[1], "category": "Featured", "price": f[2],
                "rating": f[3]} for f in FEATURED],
    ids=[f"sku-{i}" for i in range(len(FEATURED))],
)
print("indexed", query(db, f"SELECT COUNT(*) AS n FROM {TABLE}")["n"][0], "products")
```