### Install sqlite-vss Go Bindings Source: https://github.com/asg017/sqlite-vss/blob/main/site/using/go.md Installs the official sqlite-vss Go bindings using the go get command. Requires pre-compiled static library files of sqlite-vss, which can be compiled manually or downloaded from GitHub Releases. ```go go get -u github.com/asg017/sqlite-vss/bindings/go ``` -------------------------------- ### Run Datasette with Vector Extensions (Alternative Shell) Source: https://github.com/asg017/sqlite-vss/blob/main/examples/headlines/README.md This command starts a Datasette server for 'test.db' on port 8001. It configures the server to use local vector extensions and a specified plugins directory. ```shell datasette test.db -p 8001 -h 0.0.0.0 \ --plugins-dir=plugins \ --load-extension ./vss0 \ --load-extension ./vector0 ``` -------------------------------- ### Run Datasette with Vector Search Extensions (Shell) Source: https://github.com/asg017/sqlite-vss/blob/main/examples/headlines/README.md This command starts a Datasette server to serve the SQLite database 'full.db'. It loads the necessary VSS and vector extensions and specifies a plugin directory. The server will be accessible at http://0.0.0.0:8001. ```shell datasette full.db --host 0.0.0.0 --load-extension=../../build/vss0.so --load-extension=./vector0.so --plugins-dir=./plugins ``` -------------------------------- ### Installation Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Instructions on how to install the sqlite-vss NPM package for Node.js developers. ```APIDOC ## Installation To install the `sqlite-vss` package on supported platforms, run the following command: ```bash npm install sqlite-vss ``` The package is intended for use with Node.js SQLite clients such as `better-sqlite3` and `node-sqlite3`. ``` -------------------------------- ### Initialize and Use sqlite-vss with Exqlite in Elixir Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/elixir/README.md Example demonstrating how to initialize Exqlite, load the sqlite-vss extensions (vector0 and vss0), and execute a query to get the vss_version. This snippet shows the core integration steps. ```elixir Mix.install([ {:sqlite_vss, path: "../"}, {:exqlite, "~> 0.13.0"} ], verbose: true) Mix.Task.run("sqlite_vss.install") alias Exqlite.Basic {:ok, conn} = Basic.open("example.db") :ok = Exqlite.Basic.enable_load_extension(conn) Exqlite.Basic.load_extension(conn, SqliteVss.loadable_path_vector0()) Exqlite.Basic.load_extension(conn, SqliteVss.loadable_path_vss0()) {:ok, [[version]], [_]} = Basic.exec(conn, "select vss_version()") |> Basic.rows() IO.puts("version: #{version}") ``` -------------------------------- ### Static Linking Flags Example Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Example of linker and compiler flags used for statically linking sqlite-vss into other projects. These flags specify include paths and libraries. ```bash -I./dist/debug -lsqlite_vector0 -lsqlite_vss0 -llfaiss_avx2 ``` -------------------------------- ### Install sqlite-vss Package Source: https://github.com/asg017/sqlite-vss/blob/main/site/getting-started.md Instructions for installing the sqlite-vss package across various programming languages and platforms. Includes options for pip, npm, Deno, RubyGems, Elixir, Cargo, Go, Datasette, and sqlite-utils. ```bash pip install sqlite-vss ``` ```bash npm install sqlite-vss ``` ```typescript import * as sqlite_vss from "https://deno.land/x/sqlite_vss@v{{VERSION}}/mod.ts"; ``` ```bash gem install sqlite-vss ``` ```elixir {:sqlite_vss, "~> {{ VERSION }}"} ``` ```bash cargo add sqlite-vss ``` ```bash go get -u github.com/asg017/sqlite-vss/bindings/go ``` ```bash datasette install datasette-sqlite-vss ``` ```bash sqlite-utils install sqlite-utils-sqlite-vss ``` -------------------------------- ### Install and Load sqlite-vss in Node.js Source: https://github.com/asg017/sqlite-vss/blob/main/site/using/nodejs.md Demonstrates how to install the `sqlite-vss` NPM package and load it into an in-memory SQLite database using the `better-sqlite3` library. It also shows how to retrieve the VSS extension version. ```bash npm install sqlite-vss ``` ```javascript import Database from "better-sqlite3"; import * as sqlite_vss from "sqlite-vss"; const db = new Database(":memory:"); sqlite_vss.load(db); const version = db.prepare("select vss_version()").pluck().get(); console.log(version); ``` -------------------------------- ### Install Development Libraries on Linux Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Installs necessary development libraries on Linux systems using apt-get. These include OpenMP implementation, BLAS, LAPACK, and SQLite development files required by sqlite-vss and its dependencies like Faiss. ```bash sudo apt-get update sudo apt-get install libgomp1 libatlas-base-dev liblapack-dev libsqlite3-dev ``` -------------------------------- ### Install datasette-sqlite-vss Plugin during Datasette Publish Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/datasette/README.md This command publishes a Datasette instance to Cloud Run while also installing the datasette-sqlite-vss plugin. This is useful for deploying Datasette with the sqlite-vss extension pre-installed. ```bash datasette publish cloudrun data.db --service=my-service --install=datasette-sqlite-vss ``` -------------------------------- ### Install sqlite-vss NPM Package Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Installs the sqlite-vss package using npm. This package is intended for Node.js environments and requires a supported platform. ```bash npm install sqlite-vss ``` -------------------------------- ### Install sqlite-vss Python Package Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/python/README.md Installs the sqlite-vss Python package using pip. This command fetches and installs the package and its dependencies from the Python Package Index. ```bash pip install sqlite-vss ``` -------------------------------- ### Install and Load sqlite-vss in Python Source: https://github.com/asg017/sqlite-vss/blob/main/site/using/python.md Installs the sqlite-vss package using pip and loads the extension into a Python SQLite connection. It then verifies the installation by fetching the extension version. ```bash pip install sqlite-vss ``` ```python import sqlite3 import sqlite_vss db = sqlite3.connect(':memory:') db.enable_load_extension(True) sqlite_vss.load(db) db.enable_load_extension(False) version, = db.execute('select vss_version()').fetchone() print(version) ``` -------------------------------- ### Docker Build and Run Commands Source: https://github.com/asg017/sqlite-vss/blob/main/examples/headlines/README.md These Docker commands build a Docker image tagged 'x' and then run it. The second `docker run` command mounts a test database and runs it, while the third command mounts Python scripts and data to build and process the database within the container. ```shell docker build -t x . docker run -p 8001:8001 -v $PWD/test.db:/test.db --rm -it x docker run -p 8001:8001 --rm -it \ -v $PWD/build.py:/build.py \ -v $PWD/test.db:/test.db \ -v $PWD/News_Category_Dataset_v3.json:/News_Category_Dataset_v3.json \ x python3 /build.py /test.db /News_Category_Dataset_v3.json ``` -------------------------------- ### Install LLVM and Libomp on macOS Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Installs LLVM and libomp using Homebrew on macOS, which may be required for compiling certain projects. These libraries provide compiler toolchains and OpenMP support. ```bash brew install llvm libomp ``` -------------------------------- ### Database Initialization and Embedding Generation (Shell) Source: https://github.com/asg017/sqlite-vss/blob/main/examples/headlines/README.md These commands initialize the SQLite database by reading SQL scripts and then generate embeddings for articles using a Python script. Ensure 'import_articles.sql' and 'add_embeddings.py' are available and correctly configured. ```shell sqlite3 full.db '.read import_articles.sql' python3 add_embeddings.py full.db ``` -------------------------------- ### Usage with better-sqlite3 Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Example demonstrating how to use the sqlite-vss package with the better-sqlite3 Node.js client. ```APIDOC ## Usage with better-sqlite3 This example shows how to integrate `sqlite-vss` with `better-sqlite3`. ```js import Database from "better-sqlite3"; import * as sqlite_vss from "sqlite-vss"; const db = new Database(":memory:"); sqlite_vss.load(db); const version = db.prepare("select vss_version() ").pluck().get(); console.log(version); // "v0.2.0" ``` ``` -------------------------------- ### Install datasette-sqlite-vss Plugin with pip Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/datasette/README.md This command installs the datasette-sqlite-vss plugin using pip. It allows you to add the sqlite-vss extension to your Datasette instances for vector search capabilities. ```bash datasette install datasette-sqlite-vss ``` -------------------------------- ### Publish Datasette with sqlite-vss Plugin Source: https://github.com/asg017/sqlite-vss/blob/main/site/using/datasette.md Publishes a Datasette instance to Cloud Run, including the datasette-sqlite-vss plugin. The `--install` flag ensures the plugin is available in the deployed environment. ```bash datasette publish cloudrun data.db \ --service=my-service \ --install=datasette-sqlite-vss ``` -------------------------------- ### Get sqlite-vss Debug Information Source: https://github.com/asg017/sqlite-vss/blob/main/site/api-reference.md Returns a debug string containing comprehensive information about the sqlite-vss installation, including version, build date, and commit hash. ```sqlite select vss_debug(); ``` -------------------------------- ### Install LLVM for macOS Compilation Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Installs the LLVM compiler suite using Homebrew, often required for compiling on macOS, especially for certain architectures. ```shell brew install llvm ``` -------------------------------- ### Install sqlite-vss via Package Managers Source: https://github.com/asg017/sqlite-vss/blob/main/README.md Provides installation commands for sqlite-vss using common package managers across different programming languages. This allows for easy integration into projects. ```python pip install sqlite-vss ``` ```nodejs npm install sqlite-vss ``` ```ruby gem install sqlite-vss ``` ```go go get -u github.com/asg017/sqlite-vss/bindings/go ``` ```rust cargo add sqlite-vss ``` -------------------------------- ### Usage with node-sqlite3 Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Example demonstrating how to use the sqlite-vss package with the node-sqlite3 Node.js client. ```APIDOC ## Usage with node-sqlite3 This example shows how to integrate `sqlite-vss` with `node-sqlite3`. ```js import sqlite3 from "sqlite3"; import * as sqlite_vss from "sqlite-vss"; const db = new sqlite3.Database(":memory:"); db.loadExtension(sqlite_vss.getLoadablePath()); db.get("select vss_version()", (err, row) => { console.log(row); // {vss_version(): "v0.2.0"} }); ``` ``` -------------------------------- ### Go: Integrate sqlite-vss for Vector Search Source: https://context7.com/asg017/sqlite-vss/llms.txt Provides a Go example for using sqlite-vss. It shows how to open a SQLite database, load the extension, create a virtual table, insert sample data, and execute nearest neighbor queries. ```go package main import ( "database/sql" "encoding/json" "fmt" "log" _ "github.com/asg017/sqlite-vss/bindings/go" _ "github.com/mattn/go-sqlite3" ) func main() { db, err := sql.Open("sqlite3", ":memory:") if err != nil { log.Fatal(err) } defer db.Close() // Verify version var version string err = db.QueryRow("SELECT vss_version()").Scan(&version) if err != nil { log.Fatal(err) } fmt.Printf("version=%s\n", version) // Create table and insert data _, err = db.Exec(` CREATE VIRTUAL TABLE vss_demo USING vss0(a(2)); INSERT INTO vss_demo(rowid, a) VALUES (1, '[1.0, 2.0]'), (2, '[2.0, 2.0]'), (3, '[3.0, 2.0]') `) if err != nil { log.Fatal(err) } // Query nearest neighbors rows, err := db.Query(` SELECT rowid, distance FROM vss_demo WHERE vss_search(a, '[1.0, 2.0]') LIMIT 3 `) if err != nil { log.Fatal(err) } defer rows.Close() for rows.Next() { var rowid int64 var distance float32 err = rows.Scan(&rowid, &distance) if err != nil { log.Fatal(err) } fmt.Printf("rowid=%d, distance=%f\n", rowid, distance) } } ``` -------------------------------- ### Install sqlite-vss Package in Elixir Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/elixir/README.md Instructions for adding the sqlite-vss package as a dependency in an Elixir project's `mix.exs` file. This allows for easy installation using Mix. ```elixir def deps do [ {:sqlite_vss, "~> 0.0.0"} ] end ``` -------------------------------- ### Add Vector Search to SQLite (Shell) Source: https://github.com/asg017/sqlite-vss/blob/main/examples/headlines/README.md This command integrates the Vector Search extension into the SQLite database by reading the 'add_vector_search.sql' script. This enables vector search functionalities on the database. ```shell sqlite3 full.db '.read add_vector_search.sql' ``` -------------------------------- ### GET /getLoadablePath Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Retrieves the full path to the `sqlite-vss` extension file, useful for loading it into SQLite clients. ```APIDOC ## GET /getLoadablePath ### Description Returns the full path to where the `sqlite-vss` extension should be installed. This path can be directly used with SQLite clients like `better-sqlite3` and `node-sqlite3` to load the extension. ### Method `GET` (Implicitly via function call) ### Endpoint `sqlite_vss.getLoadablePath()` ### Parameters This function does not accept any parameters. ### Request Example ```js import * as sqlite_vss from "sqlite-vss"; const extensionPath = sqlite_vss.getLoadablePath(); console.log(extensionPath); ``` ### Response #### Success Response (string) - **extensionPath** (string) - The absolute path to the compiled `sqlite-vss` extension file. #### Response Example ```json "/path/to/your/node_modules/sqlite-vss/lib/vss0.node" ``` ### Error Handling This function throws an `Error` if: - `sqlite-vss` is installed and run on an unsupported platform. - The platform-specific optional dependency for `sqlite-vss` is not installed (e.g., when using `--no-optional`). Additionally, the `db.loadExtension()` function itself may throw an error if the compiled extension is incompatible with your SQLite connection due to system package issues, glib version conflicts, or other misconfigurations. ``` -------------------------------- ### Load sqlite-vss with node-sqlite3 Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Shows how to load the sqlite-vss extension with the node-sqlite3 client. This involves getting the loadable path from sqlite-vss and using the db.loadExtension method. ```javascript import sqlite3 from "sqlite3"; import * as sqlite_vss from "sqlite-vss"; const db = new sqlite3.Database(":memory:"); db.loadExtension(sqlite_vss.getLoadablePath()); db.get("select vss_version()", (err, row) => { console.log(row); // {vss_version(): "v0.2.0"} }); ``` -------------------------------- ### Install sqlite-vss Gem for Ruby Source: https://github.com/asg017/sqlite-vss/blob/main/site/using/ruby.md Installs the sqlite-vss Ruby Gem, which provides bindings for the sqlite-vss SQLite extension. This is the primary step for Ruby developers to begin using vector search functionality in SQLite. ```bash gem install sqlite-vss ``` -------------------------------- ### Install Linux Dependencies for sqlite-vss Source: https://github.com/asg017/sqlite-vss/blob/main/README.md Installs necessary system packages on Linux distributions using apt-get. These packages are required for certain sqlite-vss functionalities to work correctly on Linux. ```bash sudo apt-get update sudo apt-get install -y libgomp1 libatlas-base-dev liblapack-dev ``` -------------------------------- ### Configure Ecto Repo to Load sqlite-vss Extensions Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/elixir/README.md Configuration snippet for `runtime.exs` to load sqlite-vss extension files for an Ecto Repo. This ensures the extensions are available when the application starts. ```elixir # global config :exqlite, load_extensions: [SqliteVss.loadable_path_vector0(), SqliteVss.loadable_path_vss0()] # per connection in a Phoenix app config :my_app, MyApp.Repo, load_extensions: [SqliteVss.loadable_path_vector0(), SqliteVss.loadable_path_vss0()] ``` -------------------------------- ### Get Loadable Path for node-sqlite3 Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Obtains the file system path for the sqlite-vss extension, designed for use with the node-sqlite3 library. It ensures the correct extension binary is located for the current environment. ```javascript import sqlite3 from "sqlite3"; import * as sqlite_vss from "sqlite-vss"; const db = new sqlite3.Database(":memory:"); db.loadExtension(sqlite_vss.getLoadablePath()); ``` -------------------------------- ### Install sqlite-vss for Deno Source: https://github.com/asg017/sqlite-vss/blob/main/README.md Shows how to import and use the sqlite-vss library within a Deno project. This is the standard method for integrating external modules in Deno. ```typescript import { sqlite_vss } from "https://deno.land/x/sqlite_vss/mod.ts"; ``` -------------------------------- ### SQL Query for Similar Article Matches Source: https://github.com/asg017/sqlite-vss/blob/main/examples/headlines/README.md This SQL query demonstrates how to find articles similar to 'charlie brown' using the vss_search function. It retrieves similar matches based on headline embeddings and joins them with the articles table. ```sql with similar_matches as ( select rowid from vss_articles where vss_search( headline_embedding, vss_search_params(json(st_encode('charlie brown')), 20) ) ), final as ( select articles.* from similar_matches left join articles on articles.rowid = similar_matches.rowid ) select * from final; ``` -------------------------------- ### Load and Use sqlite-vss in Deno Source: https://github.com/asg017/sqlite-vss/blob/main/README.md Provides an example of using the 'deno.land/x/sqlite_vss' module with Deno. It loads the extension into a SQLite database and fetches the 'vss_version'. Requires specific permissions (-A) and the --unstable flag. ```typescript // Requires all permissions (-A) and the --unstable flag import { Database } from "https://deno.land/x/sqlite3@0.8.0/mod.ts"; import * as sqlite_vss from "https://deno.land/x/sqlite_vss/mod.ts"; const db = new Database(":memory:"); db.enableLoadExtension = true; sqlite_vss.load(db); const [version] = db.prepare("select vss_version()").value<[string]>()!; console.log(version); ``` -------------------------------- ### Create vss0 Virtual Table Source: https://github.com/asg017/sqlite-vss/blob/main/site/getting-started.md Example of creating a `vss0` virtual table in SQLite to store vector data. The table is named `vss_demo` and has a single vector column `a` with 2 dimensions. ```sqlite create virtual table vss_demo using vss0( a(2) ); ``` -------------------------------- ### Load and Use sqlite-vss in Node.js Source: https://github.com/asg017/sqlite-vss/blob/main/README.md Illustrates how to install the 'sqlite-vss' npm package and load the extension into a 'better-sqlite3' database instance. It then retrieves and logs the 'vss_version'. Requires 'better-sqlite3' and 'sqlite-vss' npm packages. ```javascript import Database from "better-sqlite3"; // also compatible with node-sqlite3 import * as sqlite_vss from "sqlite-vss"; const db = new Database(":memory:"); sqlite_vss.load(db); const version = db.prepare("select vss_version()").pluck().get(); console.log(version); ``` -------------------------------- ### Load and Use sqlite-vss in Python Source: https://github.com/asg017/sqlite-vss/blob/main/README.md Shows how to install the 'sqlite-vss' Python package and use it to load the extension into an in-memory SQLite database. It then queries the 'vss_version' and prints it. Requires the 'sqlite-vss' package. ```python import sqlite3 import sqlite_vss db = sqlite3.connect(':memory:') db.enable_load_extension(True) sqlite_vss.load(db) version, = db.execute('select vss_version()').fetchone() print(version) ``` -------------------------------- ### Basic VSS Table Query (SQL) Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md A simple `SELECT` statement to retrieve all data from a VSS virtual table. This serves as a basic query example before utilizing the specialized search functions. ```sql select * from vss_xyz; ``` -------------------------------- ### SQL: Call vss_fvec_add Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Demonstrates calling the 'vss_fvec_add' function, likely for element-wise addition of two vectors. The example shows a simple function call without parameters. ```sql select vss_fvec_add(); -- ``` -------------------------------- ### sqlite-vss Python API Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/python/README.md The sqlite-vss Python package provides two main functions: `vss_loadable_path()` to get the path to the extension and `load(conn)` to load the extension into an SQLite connection. ```APIDOC ## sqlite-vss Python API Documentation ### Description This section details the functions provided by the `sqlite-vss` Python package for interacting with the SQLite VSS extension. ### `vss_loadable_path()` #### Description Returns the full path to the locally installed `sqlite-vss` extension, without the filename. #### Method `vss_loadable_path()` #### Parameters None #### Request Example ```python import sqlite_vss print(sqlite_vss.vss_loadable_path()) # Expected output: '/.../venv/lib/python3.9/site-packages/sqlite_vss/vss0' ``` #### Response * **return_value** (string) - The absolute path to the `sqlite-vss` extension directory. ### `load(connection)` #### Description Loads the `sqlite-vss` extension into the given `sqlite3.Connection` object. #### Method `load(connection)` #### Parameters * **connection** (sqlite3.Connection) - Required - The SQLite connection object to load the extension into. #### Request Example ```python import sqlite_vss import sqlite3 conn = sqlite3.connect(':memory:') conn.enable_load_extension(True) sqlite_vss.load(conn) conn.enable_load_extension(False) # Example usage after loading result = conn.execute('select vss_version()').fetchone() print(result[0]) # Expected output: 'v0.1.0' ``` #### Response None (modifies the connection object in-place). #### Error Handling * If the extension cannot be loaded, an `sqlite3.OperationalError` might be raised. ``` -------------------------------- ### Get Loadable Path for better-sqlite3 Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Retrieves the absolute path to the sqlite-vss extension file, suitable for loading with better-sqlite3. This function handles platform-specific paths and throws errors for unsupported environments. ```javascript import Database from "better-sqlite3"; import * as sqlite_vss from "sqlite-vss"; const db = new Database(":memory:"); db.loadExtension(sqlite_vss.getLoadablePath()); ``` -------------------------------- ### Search Vectors in vss0 Table (Query 1) Source: https://github.com/asg017/sqlite-vss/blob/main/site/getting-started.md Example of searching the `vss_demo` table for vectors similar to `[2.0, 2.0]`. It uses the `vss_search` function and returns the `rowid` and `distance` of the top 3 closest matches. ```sqlite select rowid, distance from vss_lookup where vss_search(a, json('[2.0, 2.0]')) limit 3; ``` -------------------------------- ### Load sqlite-vss Modules and Get Version (SQL) Source: https://github.com/asg017/sqlite-vss/blob/main/README.md This snippet demonstrates how to load the necessary sqlite-vss modules (`vector0` and `vss0`) and then retrieve the version of the extension using the `vss_version()` function. This is typically the first step when using sqlite-vss in an SQL environment. ```sql .load ./vector0 .load ./vss0 select vss_version(); -- 'v0.0.1' ``` -------------------------------- ### Check sqlite-vss Version and Faiss Configuration Source: https://context7.com/asg017/sqlite-vss/llms.txt Executes the vss_debug() function to retrieve the current version of sqlite-vss, its Faiss version, and compile options. This is useful for verifying installation and compatibility. ```sql -- Check version and Faiss configuration SELECT vss_debug(); -- Returns: version, Faiss version, compile options ``` -------------------------------- ### Search Vectors in vss0 Table (Query 2) Source: https://github.com/asg017/sqlite-vss/blob/main/site/getting-started.md Example of searching the `vss_demo` table for vectors similar to `[-2.0, -2.0]`. It uses the `vss_search` function and returns the `rowid` and `distance` of the top 3 closest matches. ```sqlite select rowid, distance from vss_lookup where vss_search(a, json('[-2.0, -2.0]')) limit 3; ``` -------------------------------- ### Get sqlite-vss Loadable Path in Python Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/python/README.md Retrieves the absolute path to the locally installed sqlite-vss extension file using the `vss_loadable_path()` function. This path can be used with `sqlite3.Connection.load_extension()`. ```python import sqlite_vss print(sqlite_vss.vss_loadable_path()) # '/.../venv/lib/python3.9/site-packages/sqlite_vss/vss0' ``` -------------------------------- ### Running Deno with SQLite-vss Permissions (Bash) Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/deno/README.md This command shows the necessary Deno runtime flags to execute code that uses the x/sqlite_vss module. It requires network and filesystem access to download and cache the pre-compiled SQLite extension. The flags `--allow-ffi` and `--unstable` are needed, and `--allow-all` is a convenient shorthand. ```bash deno run -A --unstable ``` -------------------------------- ### Build SQLite Database Source: https://github.com/asg017/sqlite-vss/blob/main/examples/mcu-wiki/README.md This command reads a SQL script to build or initialize the SQLite database. It's the first step in setting up the project environment. ```shell sqlite3 test.db ".read build.sql" ``` -------------------------------- ### Supported Platforms Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Information about the operating systems and CPU architectures supported by the sqlite-vss NPM package. ```APIDOC ## Supported Platforms The `sqlite-vss` NPM package pre-compiles the `vss0` SQLite extension, making it available on the following platforms: - `darwin-x64` (MacOS x86_64) - `linux-x64` (Linux x86_64) To determine your machine's platform, you can check the `process.arch` and `process.platform` values in Node.js: ```bash $ node -e 'console.log([process.platform, process.arch])' [ 'darwin', 'x64' ] ``` During installation, the correct platform-specific package (e.g., `sqlite-vss-darwin-x64`) is automatically downloaded from npm's optional dependencies. More platforms may be supported in the future. ``` -------------------------------- ### Build Static sqlite-vss (Release) Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Compiles release static library files for sqlite-vss. These files offer optimized performance and smaller sizes for static linking. The output files will be in `dist/release`. ```bash make static-release ``` -------------------------------- ### Clone sqlite-vss Repository Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Clones the sqlite-vss GitHub repository and its submodules, which is the initial step for building from source. This command ensures all necessary components are downloaded. ```bash git clone --recurse-submodules https://github.com/asg017/sqlite-vss.git cd sqlite-vss ``` -------------------------------- ### Build SQLite Dependency Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Configures and builds the vendored SQLite library. This step is necessary to compile the sqlite-vss extension. ```bash cd vendor/sqlite ./configure && make ``` -------------------------------- ### Link sqlite-vss with Go using CGO Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/go/README.md This Go code snippet demonstrates how to import the `sqlite-vss` Go bindings and configure CGO for linking against the `sqlite-vss` libraries. It requires the `go-sqlite3` driver and the `sqlite-vss` Go bindings. The `CGO_LDFLAGS` directive is crucial for specifying the library path and dynamic lookup behavior. ```go package main import ( "database/sql" _ "github.com/asg017/sqlite-vss/bindings/go" _ "github.com/mattn/go-sqlite3" ) // #cgo LDFLAGS: -Lpath/to/sqlite-vss-libs/ -Wl,-undefined,dynamic_lookup import "C" func main() { /* */ } ``` -------------------------------- ### Build Go project with sqlite-vss using CGO_LDFLAGS Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/go/README.md This bash command shows how to set the `CGO_LDFLAGS` environment variable to link the `sqlite-vss` libraries during the Go build process. This method avoids modifying Go source files directly and is useful for CI/CD pipelines or simpler build scripts. Ensure the path to your `sqlite-vss` libraries is correctly specified. ```bash CGO_LDFLAGS="-Lpath/to/sqlite-vss-libs/ -Wl,-undefined,dynamic_lookup" go build . ``` -------------------------------- ### Load sqlite-vss with better-sqlite3 Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Demonstrates how to load the sqlite-vss extension into a better-sqlite3 database instance. It requires importing both libraries and calling the load function. ```javascript import Database from "better-sqlite3"; import * as sqlite_vss from "sqlite-vss"; const db = new Database(":memory:"); sqlite_vss.load(db); const version = db.prepare("select vss_version()").pluck().get(); console.log(version); // "v0.2.0" ``` -------------------------------- ### Get sqlite-vss library version Source: https://github.com/asg017/sqlite-vss/blob/main/site/api-reference.md Retrieves the version string of the sqlite-vector library. This function does not require any input parameters. ```sqlite select vector_version(); ``` -------------------------------- ### Get sqlite-vss Version Source: https://github.com/asg017/sqlite-vss/blob/main/site/api-reference.md Retrieves the version string of the sqlite-vss library. This function is useful for checking compatibility and debugging. ```sqlite select vss_version(); "{{VERSION}}" ``` -------------------------------- ### Build Loadable sqlite-vss (Release) Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Compiles a release version of the sqlite-vss extension as a loadable module. This version is slower to compile but offers better runtime performance and smaller binaries. The output files will be in `dist/release`. ```bash # Build a release version of `sqlite-vss`. # Slower to compile, but faster at runtime make loadable-release ``` -------------------------------- ### SQL: Call vss_distance_l1 Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Executes the 'vss_distance_l1' function, which calculates the L1 distance between two vectors. This function is a placeholder and does not take arguments in this example. ```sql select vss_distance_l1(); -- ``` -------------------------------- ### Build Static sqlite-vss (Debug) Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Compiles debug static library files for sqlite-vss. These `.a` and `.h` files can be used to link sqlite-vss directly into other projects. The output files will be in `dist/debug`. ```bash # Build a debug static archive files of `sqlite-vss`, ".a" and ".h" files make static ``` -------------------------------- ### SQL: Call vss_debug Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Invokes the 'vss_debug' function, which is likely used for debugging purposes within the VSS extension. The example shows a basic call. ```sql select vss_debug(); -- ``` -------------------------------- ### SQL: Call vss_fvec_sub Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Shows an example of calling the 'vss_fvec_sub' function, presumably for element-wise subtraction of two vectors. The SQL snippet is a basic invocation. ```sql select vss_fvec_sub(); -- ``` -------------------------------- ### Build Loadable sqlite-vss (Debug) Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Compiles a debug version of the sqlite-vss extension as a loadable module. This version is faster to compile but slower at runtime. The output files will be in `dist/debug`. ```bash # Build a debug version of `sqlite-vss`. # Faster to compile, slower at runtime make loadable ``` -------------------------------- ### SQL: Call vss_distance_linf Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Invokes the 'vss_distance_linf' function for calculating the L-infinity (maximum absolute difference) distance between vectors. The example shows a call without parameters. ```sql select vss_distance_linf(); -- ``` -------------------------------- ### Fetch SQLite Amalgamation Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Downloads the SQLite amalgamation source files required for building the sqlite-vss extension. This script fetches the necessary SQLite version. ```bash ./vendor/get_sqlite.sh ``` -------------------------------- ### Create VSS Table Constructor Syntax (SQL) Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Defines the syntax for creating a virtual table using the `vss0` module in SQLite. It specifies vector columns with their dimensions and optional Faiss factory strings for index creation. Dependencies include the `vss0` module. Inputs are column definitions and optional factory/metric type configurations. Outputs a virtual table optimized for vector search. ```sql create virtual table vss_xyz using vss0( headline_embedding(384), description_embedding(384) factory="IVF4096,Flat,IDMap2" ); ``` -------------------------------- ### Get sqlite-vss library debug information Source: https://github.com/asg017/sqlite-vss/blob/main/site/api-reference.md Returns debug information for the sqlite-vector library. This function is useful for diagnosing issues and understanding the library's internal state. It does not take any arguments. ```sqlite select vector_debug(); ``` -------------------------------- ### Check Supported Platform with Node.js Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/node/sqlite-vss/README.md Determines the current operating system and architecture using Node.js process properties. This is used to verify if the platform is supported by the sqlite-vss package. ```bash $ node -e 'console.log([process.platform, process.arch])' [ 'darwin', 'x64' ] ``` -------------------------------- ### Prepare Training Data for IVF Index (SQL) Source: https://github.com/asg017/sqlite-vss/blob/main/README.md For Faiss indexes that require training (like IVF indexes), this SQL command demonstrates how to prepare the training data. It inserts data into the virtual table with a special `operation='training'` constraint. This process is necessary before the index can be effectively used for querying. ```sql insert into vss_ivf_articles(operation, headline_embedding, description_embedding) select 'training', headline_embedding, description_embedding from articles; ``` -------------------------------- ### Load sqlite-vss Extension in Deno (JavaScript) Source: https://github.com/asg017/sqlite-vss/blob/main/bindings/deno/README.md This snippet demonstrates how to load the sqlite-vss extension within a Deno environment using the x/sqlite3 and x/sqlite_vss modules. It initializes an in-memory SQLite database, loads the extension, and then queries for the extension's version. Dependencies include 'https://deno.land/x/sqlite3@0.8.0/mod.ts' and 'https://deno.land/x/sqlite_vss@v0.1.2/mod.ts'. ```javascript import { Database } from "https://deno.land/x/sqlite3@0.8.0/mod.ts"; import * as sqlite_vss from "https://deno.land/x/sqlite_vss@v0.1.2/mod.ts"; const db = new Database(":memory:"); db.enableLoadExtension = true; sqlite_vss.load(db); const [version] = db .prepare("select vss_version()") .value<[string]>()!; console.log(version); ``` -------------------------------- ### Training vss0 Indexes Source: https://github.com/asg017/sqlite-vss/blob/main/site/api-reference.md Explains how to train Faiss indexes for vss0 virtual tables when a specific factory string requires it. ```APIDOC ## Training vss0 Indexes ### Operation ```sqlite insert into vss_table_name(operation, vector_column_name) select 'training', vector_column_name from source_table; ``` **Description** Used to train Faiss indexes that require training data, such as those using factory strings like `"IVF4096,Flat,IDMap2"`. Training data is read into memory, so caution is advised for large datasets. A `LIMIT` clause can be used to control the amount of training data. **Parameters** * **`operation`** (string) - Set to `'training'` to initiate the training process. * **`vector_column_name`** (string) - The name of the vector column to be trained. **Example** ```sqlite insert into vss_documents(operation, content_embedding) select 'training', content_embedding from documents; ``` ``` -------------------------------- ### Build sqlite-vss Loadable Extension Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Builds the sqlite-vss extension as a loadable module for SQLite. Supports both debug and release versions, with release offering better runtime performance. ```shell # build a debug version of `sqlite-vss`. Faster to compile, slow at runtime make loadable # build a release version of `sqlite-vss`. Slow to compile, but fast at runtime make loadable-release ``` -------------------------------- ### Python: Integrate sqlite-vss for Vector Search Source: https://context7.com/asg017/sqlite-vss/llms.txt Shows how to connect to an in-memory SQLite database, load the sqlite-vss extension, create a virtual table, insert vector embeddings, and perform nearest neighbor searches using Python. ```python import sqlite3 import sqlite_vss # Connect and load extensions db = sqlite3.connect(':memory:') db.enable_load_extension(True) sqlite_vss.load(db) # Verify version version = db.execute('SELECT vss_version()').fetchone()[0] print(f"sqlite-vss version: {version}") # Create virtual table db.execute(""" CREATE VIRTUAL TABLE vss_items USING vss0( embedding(384) ) """) # Insert vectors embeddings = [ (1, [0.1, 0.2, 0.3] + [0.0] * 381), (2, [0.4, 0.5, 0.6] + [0.0] * 381), (3, [0.7, 0.8, 0.9] + [0.0] * 381) ] import json for rowid, vec in embeddings: db.execute( "INSERT INTO vss_items(rowid, embedding) VALUES (?, ?)", (rowid, json.dumps(vec)) ) db.commit() # Query nearest neighbors query_vector = json.dumps([0.1, 0.2, 0.3] + [0.0] * 381) results = db.execute(""" SELECT rowid, distance FROM vss_items WHERE vss_search(embedding, ?) LIMIT 3 """, (query_vector,)).fetchall() for rowid, distance in results: print(f"rowid={rowid}, distance={distance}") ``` -------------------------------- ### Get vector value at a specific index Source: https://github.com/asg017/sqlite-vss/blob/main/site/api-reference.md Retrieves the float value at a specific position within a vector. The exact usage and input parameters are not fully detailed in the provided snippet, but it's implied to take vector identifiers and indices. ```sqlite select vector_value_at(); ``` -------------------------------- ### Integrate SQLite VSS with C Source: https://context7.com/asg017/sqlite-vss/llms.txt This C code snippet demonstrates integrating the SQLite VSS extension. It includes registering auto-extensions for vector and VSS functionality, opening an in-memory database, executing SQL commands to create a virtual table and insert data, and performing a nearest neighbor search query. ```c #include "sqlite3.h" #include "sqlite-vector.h" #include "sqlite-vss.h" #include int main(int argc, char *argv[]) { int rc; sqlite3 *db; sqlite3_stmt *stmt; // Register auto-extensions rc = sqlite3_auto_extension((void (*)())sqlite3_vector_init); if (rc != SQLITE_OK) { fprintf(stderr, "Error loading vector extension\n"); return 1; } rc = sqlite3_auto_extension((void (*)())sqlite3_vss_init); if (rc != SQLITE_OK) { fprintf(stderr, "Error loading vss extension\n"); return 1; } // Open database rc = sqlite3_open(":memory:", &db); if (rc != SQLITE_OK) { fprintf(stderr, "Cannot open database: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } // Create table and insert data char *error_message; rc = sqlite3_exec(db, "CREATE VIRTUAL TABLE vss_demo USING vss0(a(2)); " "INSERT INTO vss_demo(rowid, a) " "VALUES " " (1, '[1.0, 2.0]'), " " (2, '[2.0, 2.0]'), " " (3, '[3.0, 2.0]') ", NULL, NULL, &error_message); if (rc != SQLITE_OK) { fprintf(stderr, "Error: %s\n", error_message); sqlite3_free(error_message); sqlite3_close(db); return 1; } // Query nearest neighbors rc = sqlite3_prepare_v2(db, "SELECT rowid, distance " "FROM vss_demo " "WHERE vss_search(a, '[1.0, 2.0]') " "LIMIT 10", -1, &stmt, NULL); if (rc != SQLITE_OK) { fprintf(stderr, "Error: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } while ((rc = sqlite3_step(stmt)) == SQLITE_ROW) { long rowid = sqlite3_column_int64(stmt, 0); double distance = sqlite3_column_double(stmt, 1); printf("rowid=%ld distance=%f\n", rowid, distance); } sqlite3_finalize(stmt); sqlite3_close(db); return 0; } ``` -------------------------------- ### Run Deno script Source: https://github.com/asg017/sqlite-vss/blob/main/site/using/deno.md This is a bash command to execute a Deno script. It requires the '--allow-net' and '--unstable' flags. ```bash deno run -A --unstable main.ts ``` -------------------------------- ### Configure Project and Version Source: https://github.com/asg017/sqlite-vss/blob/main/CMakeLists.txt Sets up the CMake project, defines versions, and configures file generation for headers based on project version. It ensures compatibility with specified CMake versions. ```cmake cmake_minimum_required(VERSION 3.17 FATAL_ERROR) project(sqlite-vss VERSION $ENV{SQLITE_VSS_CMAKE_VERSION}) project(sqlite-vss-static VERSION $ENV{SQLITE_VSS_CMAKE_VERSION}) project(sqlite-vector VERSION $ENV{SQLITE_VSS_CMAKE_VERSION}) project(sqlite-vector-static VERSION $ENV{SQLITE_VSS_CMAKE_VERSION}) if(PROJECT_VERSION_TWEAK) set(SQLITE_VSS_VERSION "v${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH}-alpha.${PROJECT_VERSION_TWEAK}") else() set(SQLITE_VSS_VERSION "v${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH}") endif() configure_file(src/sqlite-vss.h.in sqlite-vss.h) configure_file(src/sqlite-vector.h.in sqlite-vector.h) ``` -------------------------------- ### CMake Build Options and Dependencies Source: https://github.com/asg017/sqlite-vss/blob/main/CMakeLists.txt Configures build options such as FAISS GPU/Python support and testing, and includes external dependencies like FAISS and nlohmann_json via subdirectories and include paths. ```cmake set(CMAKE_CXX_STANDARD 17) set(CMAKE_CXX_STANDARD_REQUIRED ON) option(FAISS_ENABLE_GPU "" OFF) option(FAISS_ENABLE_PYTHON "" OFF) option(BUILD_TESTING "" OFF) add_subdirectory(./vendor/faiss) # vendor in SQLite amalgammation include_directories(vendor/sqlite) link_directories(BEFORE vendor/sqlite) # Adding nlohmann_json for json parsing set(JSON_BuildTests OFF CACHE INTERNAL "") add_subdirectory(vendor/json) ``` -------------------------------- ### Set LLVM Compiler Flags on macOS (M1/M2 ARM) Source: https://github.com/asg017/sqlite-vss/blob/main/site/building-source.md Configures environment variables to explicitly use LLVM compilers and specify library/include paths for macOS on M1/M2 ARM architecture. Note the different paths for LLVM and libomp compared to x86_64. ```bash export CC="/opt/homebrew/opt/llvm/bin/clang" export CXX="/opt/homebrew/opt/llvm/bin/clang++" export LDFLAGS="-L/opt/homebrew/opt/libomp/lib" export CPPFLAGS="-I/opt/homebrew/opt/libomp/include" ``` -------------------------------- ### Node.js: Integrate sqlite-vss for Vector Search Source: https://context7.com/asg017/sqlite-vss/llms.txt Demonstrates Node.js integration with sqlite-vss using the 'better-sqlite3' library. It covers connecting to the database, loading the extension, creating a table, inserting data, and querying for nearest neighbors. ```javascript import Database from "better-sqlite3"; import * as sqlite_vss from "sqlite-vss"; // Connect and load extensions const db = new Database(":memory:"); sqlite_vss.load(db); // Verify version const version = db.prepare("SELECT vss_version()").pluck().get(); console.log(`sqlite-vss version: ${version}`); // Create and populate table db.exec(` CREATE VIRTUAL TABLE vss_items USING vss0(embedding(3)); INSERT INTO vss_items(rowid, embedding) VALUES (1, '[1.0, 2.0, 3.0]'), (2, '[2.0, 3.0, 4.0]'), (3, '[3.0, 4.0, 5.0]'); `); // Query with prepared statement const stmt = db.prepare(` SELECT rowid, distance FROM vss_items WHERE vss_search(embedding, ?) LIMIT 3 `); const results = stmt.all('[1.0, 2.0, 3.0]'); results.forEach(({ rowid, distance }) => { console.log(`rowid=${rowid}, distance=${distance}`); }); ``` -------------------------------- ### SQL: Create virtual table foo using vss0 Source: https://github.com/asg017/sqlite-vss/blob/main/docs.md Demonstrates the creation of a virtual table named 'foo' using the 'vss0' module with a specific dimension size (4). This is a prerequisite for performing vector similarity searches. ```sql create virtual table foo using vss0( bar(4) ); ``` -------------------------------- ### Load sqlite-vss Extension in SQLite CLI Source: https://github.com/asg017/sqlite-vss/blob/main/README.md Demonstrates how to load the 'vector0' and 'vss0' extensions into the SQLite command line interface and verify the 'vss_version'. Requires downloading the appropriate dynamic library files. ```sql .load ./vector0 .load ./vss0 select vss_version(); -- v0.0.1 ``` -------------------------------- ### Load sqlite-vss Extension and Use in Ruby Source: https://github.com/asg017/sqlite-vss/blob/main/site/using/ruby.md Demonstrates how to load the sqlite-vss extension into an in-memory SQLite database using the `sqlite3` and `sqlite_vss` Ruby gems. It shows enabling and disabling load extensions and calling the `vss_version()` SQL function to verify the extension is loaded. ```ruby require 'sqlite3' require 'sqlite_vss' db = SQLite3::Database.new(':memory:') db.enable_load_extension(true) SqliteVss.load(db) db.enable_load_extension(false) result = db.execute('SELECT vss_version()') puts result.first.first ```