### Install Project Dependencies Source: https://github.com/sqliteai/sqlite-vector/blob/main/examples/semantic_search/README.md Set up a virtual environment and install the required Python packages. ```bash $ python -m venv venv $ source venv/bin/activate $ pip install -r requirements.txt ``` -------------------------------- ### Install @sqliteai/sqlite-vector Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Install the package using npm. It automatically downloads the correct native extension for your platform. ```bash npm install @sqliteai/sqlite-vector ``` -------------------------------- ### Install Python SQLite Vector Package Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Install the sqlite-vector package for Python using pip. This command fetches and installs the package and its dependencies from PyPI. ```bash pip install sqliteai-vector ``` -------------------------------- ### Initialize and Load Vector Extension in Swift Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Example demonstrating how to initialize an SQLite database, enable extension loading, load the vector extension, and query its version in a Swift project. ```swift import vector ... var db: OpaquePointer? sqlite3_open(":memory:", &db) sqlite3_enable_load_extension(db, 1) var errMsg: UnsafeMutablePointer? = nil sqlite3_load_extension(db, vector.path, nil, &errMsg) var stmt: OpaquePointer? sqlite3_prepare_v2(db, "SELECT vector_version()", -1, &stmt, nil) defer { sqlite3_finalize(stmt) } sqlite3_step(stmt) log("vector_version(): \(String(cString: sqlite3_column_text(stmt, 0)))") sqlite3_close(db) ``` -------------------------------- ### Insert Vector Data into Table Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Provides examples for inserting vector data into a table, both as BLOBs and JSON strings. ```sql -- Insert a BLOB vector (Float32, 384 dimensions) using bindings INSERT INTO images (embedding, label) VALUES (?, 'cat'); -- Insert a JSON vector (Float32, 384 dimensions) INSERT INTO images (embedding, label) VALUES (vector_as_f32('[0.3, 1.0, 0.9, 3.2, 1.4,...]'), 'dog'); ``` -------------------------------- ### Get SQLite Vector Extension Information Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Obtain detailed information about the installed SQLite Vector extension, including platform, package name, binary name, and path. Throws an error if the extension is not found. ```typescript import { getExtensionInfo } from '@sqliteai/sqlite-vector'; const info = getExtensionInfo(); console.log(`Running on ${info.platform}`); console.log(`Extension path: ${info.path}`); ``` -------------------------------- ### Initialize SQLite Database with Vector Extension on Android Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Example of initializing an SQLite database on Android with the custom vector extension. This requires specifying the native library directory and configuring the database settings. ```java SQLiteCustomExtension vectorExtension = new SQLiteCustomExtension(getApplicationInfo().nativeLibraryDir + "/vector", null); SQLiteDatabaseConfiguration config = new SQLiteDatabaseConfiguration( getCacheDir().getPath() + "/vector_test.db", SQLiteDatabase.CREATE_IF_NECESSARY | SQLiteDatabase.OPEN_READWRITE, Collections.emptyList(), Collections.emptyList(), Collections.singletonList(vectorExtension) ); SQLiteDatabase db = SQLiteDatabase.openDatabase(config, null, null); ``` -------------------------------- ### Install sqlite-vector package Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/flutter/README.md Add the sqlite-vector dependency to your Dart or Flutter project. ```bash dart pub add sqlite_vector ``` -------------------------------- ### Load SQLite Vector Extension in Python Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/python/README.md Connect to a SQLite database, enable extension loading, load the sqlite-vector extension using its binary path, and then disable extension loading. This setup is necessary before using any vector functions. ```python import importlib.resources import sqlite3 # Connect to your SQLite database conn = sqlite3.connect("example.db") # Load the sqlite-vector extension # pip will install the correct binary package for your platform and architecture ext_path = importlib.resources.files("sqlite_vector.binaries") / "vector" conn.enable_load_extension(True) conn.load_extension(str(ext_path)) conn.enable_load_extension(False) # Now you can use sqlite-vector features in your SQL queries print(conn.execute("SELECT vector_version();").fetchone()) ``` -------------------------------- ### Get TurboQuant Backend Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Retrieve the active backend used by TurboQuant for lookup-table scans. This helps in verifying that TurboQuant is leveraging the expected SIMD path on the target runtime. ```sql SELECT vector_turboquant_backend(); -- e.g., 'NEON' ``` -------------------------------- ### Load SQLite Vector Extension in Node.js Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Load the SQLite Vector extension into a better-sqlite3 database instance. Ensure the extension is installed correctly. ```typescript import { getExtensionPath } from '@sqliteai/sqlite-vector'; import Database from 'better-sqlite3'; const db = new Database(':memory:'); db.loadExtension(getExtensionPath()); // Ready to use const version = db.prepare('SELECT vector_version()').pluck().get(); console.log('Vector extension version:', version); ``` -------------------------------- ### Initialize SQLite Vector Extension in Dart/Flutter Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Load the SQLite Vector extension and open an in-memory database using the `sqlite3` and `sqlite_vector` packages. This example demonstrates loading the extension and querying its version. ```dart import 'package:sqlite3/sqlite3.dart'; import 'package:sqlite_vector/sqlite_vector.dart'; sqlite3.loadSqliteVectorExtension(); final db = sqlite3.openInMemory(); print(db.select('SELECT vector_version()')); ``` -------------------------------- ### Get SQLite Vector Extension Path Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Retrieve the absolute path to the SQLite Vector extension binary for the current platform. This is useful for manually loading the extension. ```typescript import { getExtensionPath } from '@sqliteai/sqlite-vector'; const path = getExtensionPath(); // => '/path/to/node_modules/@sqliteai/sqlite-vector-darwin-arm64/vector.dylib' ``` -------------------------------- ### Get Quantized Data Memory Usage Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Returns the memory in bytes required to preload quantized data for a specific table and column. Use this to estimate memory needs. ```sql SELECT vector_quantize_memory('documents', 'embedding'); -- e.g., 28490112 ``` -------------------------------- ### Get Current Platform Identifier Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Determine the current platform identifier for which the SQLite Vector extension is running. This can be used for conditional logic or debugging. ```typescript import { getCurrentPlatform } from '@sqliteai/sqlite-vector'; const platform = getCurrentPlatform(); // Example output: 'darwin-arm64' ``` -------------------------------- ### Get Active Vector Computation Backend Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md This function returns the active backend used for vector computations, indicating the type of SIMD or hardware acceleration utilized. This is useful for understanding performance characteristics on different systems. ```sql SELECT vector_backend(); -- e.g., 'AVX2' ``` -------------------------------- ### Get SQLite Vector Extension Version Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Use this function to retrieve the current version of the SQLite Vector Extension. The output is a text string representing the version number. ```sql SELECT vector_version(); -- e.g., '1.0.0' ``` -------------------------------- ### Initialize Vector Extension with Configuration Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Demonstrates how to initialize the vector extension for a table, specifying vector type and dimension. Supports different distance metrics like L1, COSINE, DOT, SQUARED_L2, and HAMMING. ```sql -- Initialize the vector. By default, the distance function is L2. -- To use a different metric, specify one of the following options: -- distance=L1, distance=COSINE, distance=DOT, distance=SQUARED_L2, or distance=HAMMING. SELECT vector_init('images', 'embedding', 'type=FLOAT32,dimension=384'); ``` -------------------------------- ### Load SQLite Vector Extension Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Demonstrates how to load the SQLite Vector extension using the SQLite CLI or SQL commands. ```sql -- In SQLite CLI .load ./vector -- In SQL SELECT load_extension('./vector'); ``` -------------------------------- ### Run Semantic Search Interactive Mode Source: https://github.com/sqliteai/sqlite-vector/blob/main/examples/semantic_search/README.md Launch the interactive REPL to index documents and perform similarity queries. ```bash python semsearch.py --repl # Index a directory of documents semsearch> index ./samples # Search for similar documents semsearch> search "neural network architectures for image recognition" ``` -------------------------------- ### Check Vector and TurboQuant Backends Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Verify the active vector and TurboQuant backend implementations using `vector_backend()` and `vector_turboquant_backend()`. ```sql SELECT vector_backend(), vector_turboquant_backend(); ``` -------------------------------- ### Create Table for Vector Embeddings Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Shows how to create a SQL table to store vector embeddings along with associated data. ```sql -- Create a regular SQLite table CREATE TABLE images ( id INTEGER PRIMARY KEY, embedding BLOB, -- store Float32/UInt8/etc. label TEXT ); ``` -------------------------------- ### Quantize Vector Data Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Shows how to quantize vector data for efficient storage and retrieval. Supports TurboQuant for compact 2/3/4-bit quantization. ```sql -- Quantize vector SELECT vector_quantize('images', 'embedding'); -- Or use TurboQuant for compact 2/3/4-bit quantization SELECT vector_quantize('images', 'embedding', 'qtype=TURBO,qbits=4'); ``` -------------------------------- ### Quantize Vectors with TurboQuant Source: https://github.com/sqliteai/sqlite-vector/blob/main/QUANTIZATION.md Apply TurboQuant quantization with specified bit width for further compression. Use 'qtype=TURBO2' as a shorthand for 2 bits per dimension. ```sql SELECT vector_quantize('my_table', 'my_column', 'qtype=TURBO,qbits=4'); ``` ```sql SELECT vector_quantize('my_table', 'my_column', 'qtype=TURBO2'); ``` -------------------------------- ### getExtensionInfo() Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Returns detailed metadata about the loaded extension for the current platform. ```APIDOC ## getExtensionInfo() ### Description Returns detailed information about the extension for the current platform. ### Returns - **ExtensionInfo** - Object containing platform, packageName, binaryName, and path. ### Throws - **ExtensionNotFoundError** - If the extension binary cannot be found ### Example ```typescript import { getExtensionInfo } from '@sqliteai/sqlite-vector'; const info = getExtensionInfo(); ``` ``` -------------------------------- ### Use sqlite-vector with sqlite3 Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/flutter/README.md Load the extension into an sqlite3 database, create a table for vectors, and perform a nearest neighbor search. ```dart import 'package:sqlite3/sqlite3.dart'; import 'package:sqlite_vector/sqlite_vector.dart'; void main() { // Load once at startup. sqlite3.loadSqliteVectorExtension(); final db = sqlite3.openInMemory(); // Create a regular table with a BLOB column for vectors. db.execute('CREATE TABLE items (id INTEGER PRIMARY KEY, embedding BLOB)'); // Insert vectors. final stmt = db.prepare('INSERT INTO items (embedding) VALUES (vector_as_f32(?))'); stmt.execute(['[1.0, 2.0, 3.0, 4.0]']); stmt.dispose(); // Initialize the vector index. db.execute("SELECT vector_init('items', 'embedding', 'type=FLOAT32,dimension=4')"); // Find the 2 nearest neighbors. final results = db.select(''' SELECT e.id, v.distance FROM items AS e JOIN vector_full_scan('items', 'embedding', vector_as_f32('[1.0, 2.0, 3.0, 4.0]'), 2) AS v ON e.id = v.rowid '''); db.dispose(); } ``` -------------------------------- ### vector_init Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Initializes the vector extension for a given table and column. This is mandatory before performing any vector search or quantization. It must be called in every database connection that needs to perform vector operations. ```APIDOC ## `vector_init(table, column, options)` ### Description Initializes the vector extension for a given table and column. This is **mandatory** before performing any vector search or quantization. `vector_init` must be called in every database connection that needs to perform vector operations. The target table must have a **`rowid`** (an integer primary key, either explicit or implicit). If the table was created using `WITHOUT ROWID`, it must have **exactly one primary key column of type `INTEGER`**. This ensures that each vector can be uniquely identified and efficiently referenced during search and quantization. ### Parameters * `table` (TEXT): Name of the table containing vector data. * `column` (TEXT): Name of the column containing the vector embeddings (stored as BLOBs). * `options` (TEXT): Comma-separated key=value string. ### Options * `dimension` (required): Integer specifying the length of each vector. * `type`: Vector data type. Options: * `FLOAT32` (default) * `FLOAT16` * `FLOATB16` * `INT8` * `UINT8` * `1BIT` * `distance`: Distance function to use. Options: * `L2` (default) * `SQUARED_L2` * `COSINE` * `DOT` * `L1` * `HAMMING` ### Example: ```sql SELECT vector_init('documents', 'embedding', 'dimension=384,type=FLOAT32,distance=cosine'); ``` ``` -------------------------------- ### getCurrentPlatform() Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Returns the current platform identifier string. ```APIDOC ## getCurrentPlatform() ### Description Returns the current platform identifier. ### Returns - **Platform** - One of: 'darwin-arm64', 'darwin-x86_64', 'linux-arm64', 'linux-arm64-musl', 'linux-x86_64', 'linux-x86_64-musl', 'win32-x86_64' ### Throws - **Error** - If the platform is unsupported ``` -------------------------------- ### Top-k Nearest Neighbor Search with vector_quantize_scan Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Use this mode to retrieve a fixed number of nearest neighbors, sorted by distance. The query planner optimizes for pre-sorted output. ```sql SELECT rowid, distance FROM vector_quantize_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]'), 10); ``` -------------------------------- ### Initialize Vector Extension Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Initializes the vector extension for a table and column. This is mandatory before any vector operations. Ensure the table has a rowid or a single INTEGER primary key. ```sql SELECT vector_init('documents', 'embedding', 'dimension=384,type=FLOAT32,distance=cosine'); ``` -------------------------------- ### Estimate Memory for Quantized Vectors Source: https://github.com/sqliteai/sqlite-vector/blob/main/QUANTIZATION.md Estimate the memory required to load quantized vectors into memory before preloading. This helps in capacity planning. ```sql SELECT vector_quantize_memory('my_table', 'my_column'); ``` -------------------------------- ### Use sqlite-vector with drift Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/flutter/README.md Configure the drift NativeDatabase to load the sqlite-vector extension upon database creation. ```dart import 'package:sqlite3/sqlite3.dart'; import 'package:sqlite_vector/sqlite_vector.dart'; import 'package:drift/native.dart'; Sqlite3 loadExtensions() { sqlite3.loadSqliteVectorExtension(); return sqlite3; } // Use when creating the database: NativeDatabase.createInBackground( File(path), sqlite3: loadExtensions, ); ``` -------------------------------- ### getExtensionPath() Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Retrieves the absolute path to the SQLite Vector extension binary for the current platform. ```APIDOC ## getExtensionPath() ### Description Returns the absolute path to the SQLite Vector extension binary for the current platform. ### Returns - **string** - Absolute path to the extension file (.so, .dylib, or .dll) ### Throws - **ExtensionNotFoundError** - If the extension binary cannot be found for the current platform ### Example ```typescript import { getExtensionPath } from '@sqliteai/sqlite-vector'; const path = getExtensionPath(); ``` ``` -------------------------------- ### vector_quantize_preload Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Loads the quantized representation for the specified table and column into memory. Should be used at startup to ensure optimal query performance. ```APIDOC ## `vector_quantize_preload(table, column)` ### Description Loads the quantized representation for the specified table and column into memory. Should be used at startup to ensure optimal query performance. `vector_quantize_preload` should be called once after `vector_quantize`. The preloaded data is also shared across all database connections, so they do not need to call it again. ### Returns `NULL` ### Example ```sql SELECT vector_quantize_preload('documents', 'embedding'); ``` ``` -------------------------------- ### vector_turboquant_backend() Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Returns the active backend used by TurboQuant lookup-table scans, useful for validating SIMD path usage. ```APIDOC ## vector_turboquant_backend() ### Description Returns the active backend used by TurboQuant lookup-table scans. This is useful when validating that TurboQuant is using the expected SIMD path on a target runtime. ### Returns TEXT ### Example ```sql SELECT vector_turboquant_backend(); -- e.g., 'NEON' ``` ``` -------------------------------- ### Preload Quantized Vectors in Memory Source: https://github.com/sqliteai/sqlite-vector/blob/main/QUANTIZATION.md Preload quantized vectors into memory for faster access. This can significantly speed up nearest neighbor queries. ```sql SELECT vector_quantize_preload('my_table', 'my_column'); ``` -------------------------------- ### Full Scan Streaming Mode with LIMIT Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Perform a brute-force nearest neighbor search in streaming mode, allowing SQL clauses like LIMIT to control the results. Rows are returned progressively. ```sql SELECT rowid, distance FROM vector_full_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]')) LIMIT 5; ``` -------------------------------- ### Measure Recall of Quantized Search Source: https://github.com/sqliteai/sqlite-vector/blob/main/QUANTIZATION.md Compare the recall of quantized search against exact nearest neighbor search. Input the query vector as BLOB and the number of neighbors k. ```sql WITH exact_knn AS ( SELECT e.rowid FROM vec_examples AS e JOIN vector_full_scan('vec_examples', 'embedding', ?1, ?2) AS v ON e.rowid = v.rowid ), approx_knn AS ( SELECT e.rowid FROM vec_examples AS e JOIN vector_quantize_scan('vec_examples', 'embedding', ?1, ?2) AS v ON e.rowid = v.rowid ), matches AS ( SELECT COUNT(*) AS match_count FROM exact_knn WHERE rowid IN (SELECT rowid FROM approx_knn) ), total AS ( SELECT COUNT(*) AS total_count FROM exact_knn ) SELECT (SELECT match_count FROM matches) AS match_count, (SELECT total_count FROM total) AS total_count, CAST((SELECT match_count FROM matches) AS FLOAT) / CAST((SELECT total_count FROM total) AS FLOAT) AS recall; ``` -------------------------------- ### Streaming Nearest Neighbor Query with Filtering Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Demonstrates a streaming nearest neighbor query that allows for progressive row retrieval and SQL-based filtering and limiting. ```sql -- Streaming mode: omit k to get rows progressively, use SQL to filter and limit SELECT e.id, v.distance FROM images AS e JOIN vector_quantize_scan('images', 'embedding', ?) AS v ON e.id = v.rowid WHERE e.label = 'cat' LIMIT 10; ``` -------------------------------- ### Insert UInt8 Vector using Raw BLOB Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Use `vector_as_u8` to insert a vector represented as a raw BLOB. Ensure the BLOB is correctly formatted. ```sql INSERT INTO compressed_vectors(embedding) VALUES(vector_as_u8(X'010203')); ``` -------------------------------- ### vector_backend() Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Identifies the active backend used for vector computation, indicating available SIMD or hardware acceleration. ```APIDOC ## vector_backend() ### Description Returns the active backend used for vector computation. This indicates the SIMD or hardware acceleration available on the current system. ### Returns TEXT ### Possible Values * `CPU` – Generic fallback * `SSE2` – SIMD on Intel/AMD * `AVX2` – Advanced SIMD on modern x86 CPUs * `AVX512` – Wide SIMD on supported x86 CPUs * `NEON` – SIMD on ARM (e.g., mobile) * `RVV` – SIMD on supported RISC-V CPUs ### Example ```sql SELECT vector_backend(); -- e.g., 'AVX2' ``` ``` -------------------------------- ### Quantize Vectors in a Table Source: https://github.com/sqliteai/sqlite-vector/blob/main/QUANTIZATION.md Use this function to quantize vectors in a specified table and column. This is the default quantization method. ```sql SELECT vector_quantize('my_table', 'my_column'); ``` -------------------------------- ### isMusl() Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Detects if the current system uses musl libc. ```APIDOC ## isMusl() ### Description Detects if the system uses musl libc (e.g., Alpine Linux). ### Returns - **boolean** - true if musl is detected, false otherwise ``` -------------------------------- ### Nearest Neighbor Query on Quantized Data Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Performs a nearest neighbor search on the quantized vector data, returning the top 20 closest vectors. ```sql -- Run a nearest neighbor query on the quantized version (returns top 20 closest vectors) SELECT e.id, v.distance FROM images AS e JOIN vector_quantize_scan('images', 'embedding', ?, 20) AS v ON e.id = v.rowid; ``` -------------------------------- ### Streaming Mode Search with vector_quantize_scan Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Ideal for progressive result consumption or combining vector similarity with SQL filters. Results are returned in scan order, requiring explicit ORDER BY and LIMIT clauses if needed. ```sql SELECT rowid, distance FROM vector_quantize_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]')) LIMIT 10; ``` -------------------------------- ### vector_quantize Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Performs quantization on the specified table and column, precomputing internal data structures to support fast approximate nearest neighbor (ANN) search. If a quantization already exists, it is replaced. This should be called once after data insertion. ```APIDOC ## `vector_quantize(table, column, options)` ### Description Returns the total number of successfully quantized rows. Performs quantization on the specified table and column. This precomputes internal data structures to support fast approximate nearest neighbor (ANN) search. Read more about quantization [here](https://github.com/sqliteai/sqlite-vector/blob/main/QUANTIZATION.md). If a quantization already exists for the specified table and column, it is replaced. If it was previously loaded into memory using `vector_quantize_preload`, the data is automatically reloaded. `vector_quantize` should be called once after data insertion. If called multiple times, the previous quantized data is replaced. The resulting quantization is shared across all database connections, so they do not need to call it again. ### Parameters * `table` (TEXT): Name of the table. * `column` (TEXT): Name of the column containing vector data. * `options` (TEXT, optional): Comma-separated key=value string. ### Available options: * `max_memory`: Max memory to use for quantization (default: 30MB) * `qtype`: Quantization type: `UINT8`, `INT8`, `1BIT`, `TURBO`, `TURBO2`, `TURBO3`, or `TURBO4` * `qbits`: TurboQuant bit width (`2`, `3`, or `4`). Defaults to `4` when `qtype=TURBO`. ### Example: ```sql SELECT vector_quantize('documents', 'embedding', 'max_memory=50MB'); SELECT vector_quantize('documents', 'embedding', 'qtype=BIT'); SELECT vector_quantize('documents', 'embedding', 'qtype=TURBO,qbits=4'); SELECT vector_quantize('documents', 'embedding', 'qtype=TURBO2'); ``` ``` -------------------------------- ### Insert Float32 Vector using JSON Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Use `vector_as_f32` to insert a vector represented as a JSON array into the database. ```sql INSERT INTO documents(embedding) VALUES(vector_as_f32('[0.1, 0.2, 0.3]')); ``` -------------------------------- ### Quantize Vector Data Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Performs quantization on vector data to enable fast approximate nearest neighbor search. This precomputes data structures and can replace existing quantizations. The resulting quantization is shared across connections. ```sql SELECT vector_quantize('documents', 'embedding', 'max_memory=50MB'); ``` ```sql SELECT vector_quantize('documents', 'embedding', 'qtype=BIT'); ``` ```sql SELECT vector_quantize('documents', 'embedding', 'qtype=TURBO,qbits=4'); ``` ```sql SELECT vector_quantize('documents', 'embedding', 'qtype=TURBO2'); ``` -------------------------------- ### Full Scan Top-K Mode Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Perform a brute-force nearest neighbor search to find the top-k results. This mode is suitable for smaller datasets or validation. ```sql SELECT rowid, distance FROM vector_full_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]'), 5); ``` -------------------------------- ### vector_version() Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Retrieves the current version of the SQLite Vector Extension. ```APIDOC ## vector_version() ### Description Returns the current version of the SQLite Vector Extension. ### Returns TEXT ### Example ```sql SELECT vector_version(); -- e.g., '1.0.0' ``` ``` -------------------------------- ### Full Scan Streaming Mode with JOIN and Filtering Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Combine `vector_full_scan` in streaming mode with a JOIN and WHERE clause to filter results based on original table columns. This allows access to additional columns and custom ranking. ```sql SELECT v.rowid, row_number() OVER (ORDER BY v.distance) AS rank_number, v.distance FROM vector_full_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]')) AS v JOIN documents ON documents.rowid = v.rowid WHERE documents.category = 'science' LIMIT 10; ``` -------------------------------- ### Preload Quantized Vector Data Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Loads the quantized representation of vector data into memory for a specified table and column. Call this once after vector_quantize for optimal query performance. The preloaded data is shared across all database connections. ```sql SELECT vector_quantize_preload('documents', 'embedding'); ``` -------------------------------- ### vector_as_bf16, vector_as_i8, vector_as_u8, vector_as_bit Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Encodes a vector into the required internal BLOB format. Use these functions in INSERT, UPDATE, and DELETE statements to properly format vector values. Input can be TEXT (JSON array) or BLOB. An optional dimension parameter can enforce stricter sanity checks. ```APIDOC ## `vector_as_bf16(value, dimension)` ## `vector_as_i8(value, dimension)` ## `vector_as_u8(value, dimension)` ## `vector_as_bit(value, dimension)` ### Description Encodes a vector into the required internal BLOB format to ensure correct storage and compatibility with the system’s vector representation. A real conversion is performed ONLY in case of JSON input. When input is a BLOB, it is assumed to be already properly formatted. Functions in the `vector_as_` family should be used in all `INSERT`, `UPDATE`, and `DELETE` statements to properly format vector values. However, they are *not* required when specifying input vectors for the `vector_full_scan` or `vector_quantize_scan` virtual tables. ### Parameters: * `value` (TEXT or BLOB): * If `TEXT`, it must be a JSON array (e.g., `"[0.1, 0.2, 0.3]"`). * If `BLOB`, no check is performed; the user must ensure the format matches the specified type and dimension. * `dimension` (INT, optional): Enforce a stricter sanity check, ensuring the input vector has the expected dimensionality. ### Usage by format: ```sql -- Insert a Float32 vector using JSON INSERT INTO documents(embedding) VALUES(vector_as_f32('[0.1, 0.2, 0.3]')); -- Insert a UInt8 vector using raw BLOB (ensure correct formatting!) INSERT INTO compressed_vectors(embedding) VALUES(vector_as_u8(X'010203')); ``` ``` -------------------------------- ### Check for Musl libc Source: https://github.com/sqliteai/sqlite-vector/blob/main/packages/node/README.md Detect if the current system is using musl libc, commonly found in Alpine Linux distributions. This can be important for compatibility checks. ```typescript import { isMusl } from '@sqliteai/sqlite-vector'; if (isMusl()) { console.log('Running on a musl-based system.'); } else { console.log('Running on a glibc-based system or non-Linux.'); } ``` -------------------------------- ### vector_as_f32, vector_as_f16 Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Functions to convert vector data to f32 or f16 formats. ```APIDOC ## `vector_as_f32(value)` ## `vector_as_f16(value)` ``` -------------------------------- ### vector_quantize_cleanup Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Releases memory previously allocated by a `vector_quantize_preload` call and removes all quantization entries associated with the specified table and column. ```APIDOC ## `vector_quantize_cleanup(table, column)` ### Description Releases memory previously allocated by a `vector_quantize_preload` call and removes all quantization entries associated with the specified table and column. Use this function when quantization is no longer required. In some cases, running VACUUM may be necessary to reclaim the freed space from the database. If the data changes and you invoke `vector_quantize`, the existing quantization data is automatically replaced. In that case, calling this function is unnecessary. ### Returns `NULL` ### Example ```sql SELECT vector_quantize_cleanup('documents', 'embedding'); ``` ``` -------------------------------- ### vector_quantize_memory Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Returns the amount of memory (in bytes) required to preload quantized data for the specified table and column. ```APIDOC ## `vector_quantize_memory(table, column)` ### Description Returns the amount of memory (in bytes) required to preload quantized data for the specified table and column. ### Returns `INTEGER` ### Example ```sql SELECT vector_quantize_memory('documents', 'embedding'); -- e.g., 28490112 ``` ``` -------------------------------- ### Clean Up Quantized Vector Data Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Releases memory allocated by vector_quantize_preload and removes quantization entries for a specified table and column. This is useful when quantization is no longer needed. Automatic replacement occurs if vector_quantize is called again. ```sql SELECT vector_quantize_cleanup('documents', 'embedding'); ``` -------------------------------- ### vector_quantize_scan Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Performs a fast approximate nearest neighbor search using pre-quantized vector data. This function is recommended for large datasets due to its speed/recall/memory trade-off. It returns rowid and distance, requiring a SELF JOIN for additional columns. Ensure `vector_quantize()` is run beforehand. ```APIDOC ## `vector_quantize_scan(table, column, vector [, k])` ### Description Performs a fast approximate nearest neighbor search using the pre-quantized data. This is the recommended query method for large datasets due to its excellent speed/recall/memory trade-off. Since this interface only returns rowid and distance, if you need to access additional columns from the original table, you must use a SELF JOIN. You must run `vector_quantize()` before using `vector_quantize_scan()` and when data initialized for vectors changes. ### Parameters #### Path Parameters * `table` (TEXT) - Required - Name of the target table. * `column` (TEXT) - Required - Column containing vectors. * `vector` (BLOB or JSON) - Required - The query vector. * `k` (INTEGER) - Optional - Number of nearest neighbors to return. When provided, the module collects the top-k results sorted by distance. When omitted, the module operates in streaming mode — rows are returned progressively, enabling standard SQL clauses such as `WHERE` and `LIMIT`. ### Request Example ```sql -- Top-k mode: return the 10 nearest neighbors, sorted by distance SELECT rowid, distance FROM vector_quantize_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]'), 10); -- Streaming mode: progressively scan using quantized data SELECT rowid, distance FROM vector_quantize_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]')) LIMIT 10; ``` ### Response #### Success Response (200) * `rowid` (INTEGER) - The row identifier from the original table. * `distance` (REAL) - The calculated distance between the query vector and the stored vector. ### Usage Notes * In **top-k mode** (with `k`), results are sorted by distance. The query planner knows the output is pre-sorted, so no additional `ORDER BY` is needed. * In **streaming mode** (without `k`), rows are returned in scan order. Use `ORDER BY distance` and `LIMIT` as needed. * Streaming mode is ideal for combining vector similarity with additional SQL-level filters or progressive result consumption. ``` -------------------------------- ### Add Android SQLite Vector Dependency Source: https://github.com/sqliteai/sqlite-vector/blob/main/README.md Add this dependency to your Gradle build file to include the SQLite Vector extension in your Android application. Ensure the native library path is correctly configured. ```gradle implementation 'ai.sqlite:vector:0.9.80' ``` -------------------------------- ### vector_full_scan Source: https://github.com/sqliteai/sqlite-vector/blob/main/API.md Performs a brute-force nearest neighbor search using the given vector. Optimized for small datasets or validation. Can operate in top-k mode or streaming mode. ```APIDOC ## `vector_full_scan(table, column, vector [, k])` ### Description Performs a brute-force nearest neighbor search using the given vector. Despite its brute-force nature, this function is highly optimized and useful for small datasets (rows < 1000000) or validation. Since this interface only returns rowid and distance, if you need to access additional columns from the original table, you must use a SELF JOIN. ### Parameters: * `table` (TEXT): Name of the target table. * `column` (TEXT): Column containing vectors. * `vector` (BLOB or JSON): The query vector. * `k` (INTEGER, optional): Number of nearest neighbors to return. When provided, the module collects the top-k results sorted by distance. When omitted, the module operates in **streaming mode** — rows are returned progressively as they are scanned, enabling standard SQL clauses such as `WHERE` and `LIMIT` to control filtering and result count. ### Examples: ```sql -- Top-k mode: return the 5 nearest neighbors, sorted by distance SELECT rowid, distance FROM vector_full_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]'), 5); ``` ```sql -- Streaming mode: progressively scan all rows, apply SQL filters SELECT rowid, distance FROM vector_full_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]')) LIMIT 5; ``` ```sql -- Streaming mode with JOIN and filtering SELECT v.rowid, row_number() OVER (ORDER BY v.distance) AS rank_number, v.distance FROM vector_full_scan('documents', 'embedding', vector_as_f32('[0.1, 0.2, 0.3]')) AS v JOIN documents ON documents.rowid = v.rowid WHERE documents.category = 'science' LIMIT 10; ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.