### Install @sqliteai/sqlite-ai Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Install the package using npm. The correct native extension for your platform is automatically downloaded. ```bash npm install @sqliteai/sqlite-ai ``` -------------------------------- ### Flutter and Dart SQLite AI Setup Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Commands for adding the package and initializing the extension within a Dart/Flutter project. ```bash flutter pub add sqlite_ai # Flutter projects dart pub add sqlite_ai # Dart projects ``` ```dart import 'package:sqlite3/sqlite3.dart'; import 'package:sqlite_ai/sqlite_ai.dart'; sqlite3.loadSqliteAiExtension(); final db = sqlite3.openInMemory(); print(db.select('SELECT ai_version()')); ``` -------------------------------- ### Start SQLite CLI Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Initiates the SQLite command-line interface to interact with a database file. ```bash sqlite3 myapp.db ``` -------------------------------- ### Install SQLite-AI package Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/flutter/README.md Add the dependency to your Dart project. ```bash dart pub add sqlite_ai ``` -------------------------------- ### Python SQLite AI Installation Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Command to install the sqlite-ai package via pip. ```bash pip install sqlite-ai ``` -------------------------------- ### Python Integration Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Example of using the SQLite-AI extension in Python applications. ```APIDOC ## Python Integration Use the SQLite-AI extension in Python applications with the built-in sqlite3 module. ### Setup 1. Ensure the SQLite AI extension is installed and accessible. The exact method depends on your installation (e.g., system package, manual build). ### Code Example ```python import sqlite3 import os # Define the path to the SQLite AI extension library # This path will vary depending on your installation. # Example for a common location: # extension_path = '/usr/local/lib/sqlite-ai-extension.so' # Linux # extension_path = '/opt/homebrew/lib/sqlite-ai-extension.dylib' # macOS (Homebrew) # extension_path = 'C:\path\to\sqlite-ai-extension.dll' # Windows # Replace with the actual path to your extension library # You might need to find this path after installation. # For demonstration, we'll use a placeholder. extension_path = './sqlite_ai_extension.so' # Placeholder, adjust as needed # Check if the extension file exists if not os.path.exists(extension_path): print(f"Error: Extension file not found at {extension_path}") print("Please ensure the SQLite AI extension is installed and the path is correct.") exit() # Connect to an in-memory database conn = sqlite3.connect(':memory:') # Load the SQLite AI extension try: conn.enable_load_extension(True) conn.load_extension(extension_path) conn.enable_load_extension(False) print("SQLite AI extension loaded successfully.") except Exception as e: print(f"Error loading extension: {e}") conn.close() exit() # Example: Get AI extension version cursor = conn.cursor() cursor.execute('SELECT ai_version()') version = cursor.fetchone()[0] print(f'AI extension version: {version}') # Example: Load a model and generate text try: # Ensure the model file exists at the specified path model_path = './models/llama.gguf' # Placeholder, adjust as needed if not os.path.exists(model_path): print(f"Warning: Model file not found at {model_path}. Skipping text generation example.") else: cursor.execute("SELECT llm_model_load(?)", (model_path,)) cursor.execute("SELECT llm_context_create_textgen()") cursor.execute("SELECT llm_text_generate(?)", ('Hello, how are you?',)) response = cursor.fetchone()[0] print(f'Generated text: {response}') except Exception as e: print(f"Error during model loading or text generation: {e}") # Close the connection conn.close() ``` ### Notes - **Extension Path**: You MUST replace `./sqlite_ai_extension.so` with the actual path to your installed SQLite AI extension library file. The location varies significantly based on your operating system and installation method. - **Model Path**: Similarly, replace `./models/llama.gguf` with the correct path to your LLM model file. - **Error Handling**: The example includes basic error handling for extension loading and file existence checks. Adapt as needed for your application. - **Database**: This example uses an in-memory database (`:memory:`). For persistent storage, use a file path like `sqlite3.connect('my_database.db')`. ``` -------------------------------- ### Integrate SQLite-AI with Android Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Covers Gradle dependency setup and Java-based database configuration for Android. ```groovy // build.gradle implementation 'ai.sqlite:ai:0.7.55' ``` ```java // Java usage SQLiteCustomExtension aiExtension = new SQLiteCustomExtension( getApplicationInfo().nativeLibraryDir + "/ai", null ); SQLiteDatabaseConfiguration config = new SQLiteDatabaseConfiguration( getCacheDir().getPath() + "/ai_test.db", SQLiteDatabase.CREATE_IF_NECESSARY | SQLiteDatabase.OPEN_READWRITE, Collections.emptyList(), Collections.emptyList(), Collections.singletonList(aiExtension) ); SQLiteDatabase db = SQLiteDatabase.openDatabase(config, null, null); // Execute AI queries Cursor cursor = db.rawQuery("SELECT ai_version()", null); if (cursor.moveToFirst()) { String version = cursor.getString(0); Log.d("SQLiteAI", "Version: " + version); } cursor.close(); ``` -------------------------------- ### Node.js Integration Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Example of using the SQLite-AI extension in Node.js applications with better-sqlite3. ```APIDOC ## Node.js Integration Use the SQLite-AI extension in Node.js applications with better-sqlite3 or similar SQLite bindings. ### Setup 1. Install necessary packages: ```bash npm install better-sqlite3 @sqliteai/sqlite-ai ``` ### Code Example ```typescript import { getExtensionPath, getExtensionInfo } from '@sqliteai/sqlite-ai'; import Database from 'better-sqlite3'; // Load extension const db = new Database(':memory:'); db.loadExtension(getExtensionPath()); // Check version const version = db.prepare('SELECT ai_version()').pluck().get(); console.log('AI extension version:', version); // Get extension info const info = getExtensionInfo(); console.log(`Platform: ${info.platform}`); console.log(`Package: ${info.packageName}`); console.log(`Binary: ${info.binaryName}`); // Load model and generate text db.exec("SELECT llm_model_load('./models/llama.gguf', 'gpu_layers=99')"); db.exec("SELECT llm_context_create_textgen()"); const response = db.prepare("SELECT llm_text_generate(?)").pluck().get('Hello, how are you?'); console.log(response); // Close database connection db.close(); ``` ### Notes - Ensure the model path (`./models/llama.gguf`) is correct and accessible. - The `gpu_layers` option depends on your hardware and build configuration. ``` -------------------------------- ### Retrieve Extension Version Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Check the currently installed version of the SQLite-AI extension. ```sql SELECT ai_version(); -- Returns: '1.0.0' ``` -------------------------------- ### llm_chat_create Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Starts a new in-memory chat session and returns a unique UUID identifier. ```APIDOC ## SELECT llm_chat_create() ### Description Starts a new in-memory chat session. ### Request Example SELECT llm_chat_create(); ``` -------------------------------- ### Load SQLite AI Extension in Node.js Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Load the SQLite AI extension into a better-sqlite3 database instance using its path. This example also demonstrates querying the extension version. ```typescript import { getExtensionPath } from '@sqliteai/sqlite-ai'; import Database from 'better-sqlite3'; const db = new Database(':memory:'); db.loadExtension(getExtensionPath()); // Ready to use const version = db.prepare('SELECT ai_version()').pluck().get(); console.log('AI extension version:', version); ``` -------------------------------- ### GET ai_version Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Retrieves the current version of the SQLite-AI extension. ```APIDOC ## GET ai_version ### Description Returns the current version string of the SQLite-AI extension. ### Method SELECT ### Endpoint ai_version() ### Response #### Success Response (200) - **version** (string) - The version number of the extension. ``` -------------------------------- ### Get Context Size Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Returns the total token capacity of the current context. Raises an error if no context is active. ```sql SELECT llm_context_size(); -- 4096 ``` -------------------------------- ### Get SQLite AI Extension Path Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Retrieve the absolute path to the SQLite AI extension binary for the current platform. This is useful for manually loading the extension. ```typescript import { getExtensionPath } from '@sqliteai/sqlite-ai'; const path = getExtensionPath(); // => '/path/to/node_modules/@sqliteai/sqlite-ai-darwin-arm64/ai.dylib' ``` -------------------------------- ### Get Used Context Tokens Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Returns the number of tokens consumed in the current context. Useful for monitoring usage. ```sql SELECT llm_context_used(); -- 1024 ``` -------------------------------- ### Get SQLite AI Extension Information Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Obtain detailed information about the loaded SQLite AI extension, including its platform, package name, binary name, and full path. This can help in debugging or dynamic configuration. ```typescript import { getExtensionInfo } from '@sqliteai/sqlite-ai'; const info = getExtensionInfo(); console.log(`Running on ${info.platform}`); console.log(`Extension path: ${info.path}`); ``` -------------------------------- ### LLM Chat System Prompt API Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Gets or sets the system prompt for chat sessions. The system prompt is prepended as a system-role message. ```APIDOC ## `llm_chat_system_prompt(text TEXT)` ### Description Gets or sets the system prompt for chat sessions. When called without arguments, returns the current system prompt (or `NULL` if none is set). When called with a text argument, sets the system prompt and returns `NULL`. The system prompt is automatically prepended as a system-role message at the beginning of chat conversations. ### Parameters #### Path Parameters - **text** (TEXT) - Optional - The system prompt to set. ### Response #### Success Response (200) - **TEXT or NULL** - The current system prompt if no argument is provided, otherwise NULL. ### Request Example ```sql -- Set a system prompt SELECT llm_chat_system_prompt('You are a helpful assistant that speaks Italian.'); -- Get the current system prompt SELECT llm_chat_system_prompt(); ``` ``` -------------------------------- ### Manage Chat System Prompt in SQLite AI Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Gets or sets the system prompt for chat sessions. When called without arguments, it returns the current system prompt. When called with text, it sets the prompt and returns NULL. ```sql SELECT llm_chat_system_prompt('You are a helpful assistant that speaks Italian.'); ``` ```sql SELECT llm_chat_system_prompt(); ``` -------------------------------- ### Initialize Extended Temperature Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Applies advanced exponential scaling for dynamic temperature control. ```sql SELECT llm_sampler_init_temp_ext(0.8, 0.1, 2.0); ``` -------------------------------- ### Initialize Distribution Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Sets up a random distribution-based sampler using a specific seed. ```sql SELECT llm_sampler_init_dist(42); ``` -------------------------------- ### Initialize Typical Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Configures sampling to prefer tokens near the expected entropy level. ```sql SELECT llm_sampler_init_typical(0.95, 1); ``` -------------------------------- ### GET llm_context_used Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Returns the number of tokens consumed in the current context. ```APIDOC ## GET llm_context_used ### Description Returns the count of tokens currently consumed within the active inference context. ### Method SELECT ### Endpoint llm_context_used() ### Response #### Success Response (200) - **tokens** (integer) - Number of tokens used. ``` -------------------------------- ### Initialize Top-P Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Retains tokens with cumulative probability at least P, keeping at least min_keep tokens. ```sql SELECT llm_sampler_init_top_p(0.9, 1); ``` -------------------------------- ### Load and Use SQLite-AI in Python Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Demonstrates loading the extension, checking the version, loading a model, and performing chat or embedding operations. ```python # Load the sqlite-ai extension (choose cpu or gpu variant) ext_path = importlib.resources.files("sqliteai.binaries.cpu") / "ai" conn.enable_load_extension(True) conn.load_extension(str(ext_path)) conn.enable_load_extension(False) # Check version print(conn.execute("SELECT ai_version();").fetchone()) # Load model and create context conn.execute("SELECT llm_model_load('./models/llama.gguf', 'gpu_layers=99')") conn.execute("SELECT llm_context_create_chat()") # Chat interaction result = conn.execute("SELECT llm_chat_respond('What is machine learning?')").fetchone() print(result[0]) # Generate embeddings conn.execute("SELECT llm_context_create_embedding('embedding_type=FLOAT32')") embedding = conn.execute("SELECT llm_embed_generate('sample text', 'json_output=1')").fetchone() print(embedding[0]) ``` -------------------------------- ### Initialize XTC Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Initializes the XTC sampler with specified parameters. ```sql SELECT llm_sampler_init_xtc(0.9, 0.8, 1, 42); ``` -------------------------------- ### Initialize Mirostat Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Initializes Mirostat 1.0 sampling with entropy control. ```sql SELECT llm_sampler_init_mirostat(42, 5.0, 0.1, 100); ``` -------------------------------- ### Initialize Mirostat v2 Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Initializes Mirostat 2.0 entropy-based sampling. ```sql SELECT llm_sampler_init_mirostat_v2(42, 5.0, 0.1); ``` -------------------------------- ### Load SQLite-AI Extension in Python Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/python/README.md Demonstrates how to locate and load the sqlite-ai extension binary using importlib.resources and the sqlite3 module. ```python import importlib.resources import sqlite3 # Connect to your SQLite database conn = sqlite3.connect("example.db") # Load the sqlite-ai extension # pip will install the correct binary package for your platform and architecture # Choose between CPU or GPU variant ext_path = importlib.resources.files("sqliteai.binaries.cpu") / "ai" conn.enable_load_extension(True) conn.load_extension(str(ext_path)) conn.enable_load_extension(False) # Now you can use sqlite-ai features in your SQL queries print(conn.execute("SELECT ai_version();").fetchone()) ``` -------------------------------- ### Initialize Top-K Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Limits sampling to the top K most likely tokens. ```sql SELECT llm_sampler_init_top_k(40); ``` -------------------------------- ### Create Chat Context Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Initialize a context configured for conversational AI with a default 4096 token window. ```sql -- Create chat context SELECT llm_context_create_chat(); -- With larger context window SELECT llm_context_create_chat('context_size=8192'); ``` -------------------------------- ### Configure LLM Sampler Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Sets parameters for token generation, including decoding strategies, penalties, and grammar constraints. ```sql -- Create sampler (auto-created if not called) SELECT llm_sampler_create(); -- Greedy decoding (always pick most probable token) SELECT llm_sampler_init_greedy(); -- Top-K sampling (limit to top k tokens) SELECT llm_sampler_init_top_k(40); -- Top-P (nucleus) sampling SELECT llm_sampler_init_top_p(0.9, 1); -- Temperature control SELECT llm_sampler_init_temp(0.8); -- Minimum probability threshold SELECT llm_sampler_init_min_p(0.05, 1); -- Repetition penalties SELECT llm_sampler_init_penalties(64, 1.2, 0.5, 0.8); -- Parameters: n_last_tokens, repeat_penalty, frequency_penalty, presence_penalty -- Mirostat entropy-based sampling SELECT llm_sampler_init_mirostat_v2(42, 5.0, 0.1); -- Grammar-constrained output SELECT llm_sampler_init_grammar('root ::= "yes" | "no"', 'root'); -- Random distribution with seed SELECT llm_sampler_init_dist(42); -- Free sampler resources SELECT llm_sampler_free(); ``` -------------------------------- ### Get Current Platform Identifier Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Determine the current platform identifier that the @sqliteai/sqlite-ai package supports. This can be used for conditional logic or logging. ```typescript import { getCurrentPlatform } from '@sqliteai/sqlite-ai'; const platform = getCurrentPlatform(); // Example output: 'darwin-arm64' ``` -------------------------------- ### Initialize Top-N Sigma Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Limits sampling to tokens within n standard deviations. ```sql SELECT llm_sampler_init_top_n_sigma(1.5); ``` -------------------------------- ### Complete SQLite-AI Workflow Source: https://context7.com/sqliteai/sqlite-ai/llms.txt This SQL script demonstrates a full workflow including loading extensions, models for chat and embeddings, generating embeddings, and transcribing audio. Ensure models and audio files are in the correct paths. ```sql -- 1. Load the extension .load ./ai -- 2. Check version SELECT ai_version(); -- 3. Load a chat model with GPU acceleration SELECT llm_model_load('./models/Llama-3.2-3B-Instruct-Q4_K_M.gguf', 'gpu_layers=99'); -- 4. Create chat context and set system prompt SELECT llm_context_create_chat('context_size=4096'); SELECT llm_chat_system_prompt('You are a helpful AI assistant.'); -- 5. Have a conversation SELECT llm_chat_respond('What is the capital of France?'); SELECT llm_chat_respond('Tell me more about its history.'); -- 6. Save the chat for later SELECT llm_chat_save('Geography Discussion', '{"topic": "France"}'); -- 7. Clean up chat resources SELECT llm_chat_free(); SELECT llm_context_free(); -- 8. Switch to embeddings SELECT llm_model_load('./models/nomic-embed-text-v1.5-Q8_0.gguf', 'gpu_layers=99'); SELECT llm_context_create_embedding('embedding_type=FLOAT32'); -- 9. Generate embeddings for documents CREATE TABLE IF NOT EXISTS docs (id INTEGER PRIMARY KEY, content TEXT, embedding BLOB); INSERT INTO docs (content, embedding) VALUES ('Machine learning fundamentals', llm_embed_generate('Machine learning fundamentals')), ('Deep learning neural networks', llm_embed_generate('Deep learning neural networks')); -- 10. Clean up embedding resources SELECT llm_context_free(); SELECT llm_model_free(); -- 11. Transcribe audio SELECT audio_model_load('./models/ggml-base.bin'); SELECT audio_model_transcribe('./audio/recording.wav', 'language=en'); SELECT audio_model_free(); ``` -------------------------------- ### Initialize Min-P Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Filters tokens based on a minimum probability threshold P. ```sql SELECT llm_sampler_init_min_p(0.05, 1); ``` -------------------------------- ### Initialize Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Creates a new sampling strategy for text generation. Automatically created if not explicitly initialized. ```sql SELECT llm_sampler_create(); ``` -------------------------------- ### Integrate with Python Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Initializes a database connection for use with the SQLite-AI extension in Python. ```python import importlib.resources import sqlite3 # Connect to database conn = sqlite3.connect("example.db") ``` -------------------------------- ### Initialize Infill Mode Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Enables prefix-suffix infill mode for completions. ```sql SELECT llm_sampler_init_infill(); ``` -------------------------------- ### Swift SQLite AI Integration Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Demonstrates loading the AI extension into an in-memory SQLite database using Swift. ```swift import ai ... var db: OpaquePointer? sqlite3_open(":memory:", &db) sqlite3_enable_load_extension(db, 1) var errMsg: UnsafeMutablePointer? = nil sqlite3_load_extension(db, ai.path, nil, &errMsg) var stmt: OpaquePointer? sqlite3_prepare_v2(db, "SELECT ai_version()", -1, &stmt, nil) defer { sqlite3_finalize(stmt) } sqlite3_step(stmt) log("ai_version(): \(String(cString: sqlite3_column_text(stmt, 0)))") sqlite3_close(db) ``` -------------------------------- ### Initialize Temperature Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Adjusts the sampling temperature to control output randomness. ```sql SELECT llm_sampler_init_temp(0.8); ``` -------------------------------- ### Create Text Generation Context Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Initialize a context specifically optimized for text generation tasks. ```sql -- Create text generation context (equivalent to context_size=4096) SELECT llm_context_create_textgen(); -- With custom options SELECT llm_context_create_textgen('context_size=8192,n_threads=8'); ``` -------------------------------- ### Load SQLite-AI Extension Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Load the extension into the SQLite environment using either the CLI or SQL command. ```sql -- In SQLite CLI .load ./ai -- In SQL SELECT load_extension('./ai'); ``` -------------------------------- ### Integrate with Node.js Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Demonstrates loading the extension and running inference using better-sqlite3. ```typescript import { getExtensionPath, getExtensionInfo } from '@sqliteai/sqlite-ai'; import Database from 'better-sqlite3'; // Load extension const db = new Database(':memory:'); db.loadExtension(getExtensionPath()); // Check version const version = db.prepare('SELECT ai_version()').pluck().get(); console.log('AI extension version:', version); // Get extension info const info = getExtensionInfo(); console.log(`Platform: ${info.platform}`); console.log(`Package: ${info.packageName}`); console.log(`Binary: ${info.binaryName}`); // Load model and generate text db.exec("SELECT llm_model_load('./models/llama.gguf', 'gpu_layers=99')"); db.exec("SELECT llm_context_create_textgen()"); const response = db.prepare("SELECT llm_text_generate(?)").pluck().get('Hello, how are you?'); console.log(response); ``` -------------------------------- ### Context Settings Reference Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Reference for various configuration keys used in context creation, including RoPE/YaRN settings, KV cache types, and operational flags. ```APIDOC ## Context Settings Reference ### RoPE & YaRN (positional scaling) | Key | Type | Meaning | | ------------------- | ------------------------------ | ------------------------------------------------- | | `rope_scaling_type` | `unspecified, none, linear, yarn, longrope` | RoPE scaling strategy. | | `rope_freq_base` | `float number` | RoPE base frequency. `0` = from model. | | `rope_freq_scale` | `float number` | RoPE frequency scaling factor. `0` = from model. | | `yarn_ext_factor` | `float number` | YaRN extrapolation mix factor. `<0` = from model. | | `yarn_attn_factor` | `float number` | YaRN magnitude scaling. | | `yarn_beta_fast` | `float number` | YaRN low correction dimension. | | `yarn_beta_slow` | `float number` | YaRN high correction dimension. | | `yarn_orig_ctx` | `number` | YaRN original context size. | ### KV cache types (experimental) | Key | Type | Meaning | | -------- | ---------------- | ---------------------- | | `type_k` | [ggml_type](https://github.com/ggml-org/llama.cpp/blob/00681dfc16ba4cebb9c7fbd2cf2656e06a0692a4/ggml/include/ggml.h#L377) | Data type for K cache. | | `type_v` | [ggml_type](https://github.com/ggml-org/llama.cpp/blob/00681dfc16ba4cebb9c7fbd2cf2656e06a0692a4/ggml/include/ggml.h#L377) | Data type for V cache. | ### Flags > Place booleans at the end of your option string if you’re copy-by-value mirroring a struct; otherwise order doesn’t matter. | Key | Type | Meaning | | -------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `embeddings` | `1 or 0` | If `1`, extract embeddings (with logits). Used by the embedding preset. | | `offload_kqv` | `1 or 0` | Offload KQV ops (incl. KV cache) to GPU. | | `no_perf` | `1 or 0` | Disable performance timing. | | `op_offload` | `1 or 0` | Offload host tensor ops to device. | | `swa_full` | `1 or 0` | Use full-size SWA cache. When `false` and `n_seq_max > 1`, performance may degrade. | | `kv_unified` | `1 or 0` | Use a unified buffer across input sequences during attention. Try disabling when `n_seq_max > 1` and sequences do not share a long prefix. | | `defrag_thold` | `float number` | **Deprecated.** Defragment KV cache if `holes/size > thold`. `<= 0` disables. | --- ``` -------------------------------- ### Initialize Penalties Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Applies repetition, frequency, and presence penalties to sampling. ```sql SELECT llm_sampler_init_penalties(64, 1.2, 0.5, 0.8); ``` -------------------------------- ### Free Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Releases resources associated with the current sampler. ```sql SELECT llm_sampler_free(); ``` -------------------------------- ### Integrate with drift Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/flutter/README.md Configure the drift database to load the SQLite-AI extension upon initialization. ```dart import 'package:sqlite3/sqlite3.dart'; import 'package:sqlite_ai/sqlite_ai.dart'; import 'package:drift/native.dart'; Sqlite3 loadExtensions() { sqlite3.loadSqliteAiExtension(); return sqlite3; } // Use when creating the database: NativeDatabase.createInBackground( File(path), sqlite3: loadExtensions, ); ``` -------------------------------- ### Load LoRA Adapter Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Loads a LoRA adapter from a file path with a specified scale. ```sql SELECT llm_lora_load('./adapters/adapter.lora', 1.0); ``` -------------------------------- ### Initialize Greedy Sampler Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Configures the sampler to always select the most probable token. ```sql SELECT llm_sampler_init_greedy(); ``` -------------------------------- ### Sampler Configuration Functions Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Functions for configuring how the model selects tokens during text generation. ```APIDOC ## Sampler Configuration Configure how the model selects tokens during text generation for controlling randomness and quality. ### Method SQL ### Endpoint N/A (SQL Functions) ### Functions - **llm_sampler_create()**: Creates a sampler (auto-created if not called). - **llm_sampler_init_greedy()**: Initializes sampler for greedy decoding (always pick most probable token). - **llm_sampler_init_top_k(k: integer)**: Initializes sampler for Top-K sampling (limit to top k tokens). - **llm_sampler_init_top_p(p: float, n_recan: integer)**: Initializes sampler for Top-P (nucleus) sampling. - **llm_sampler_init_temp(temp: float)**: Initializes sampler with temperature control. - **llm_sampler_init_min_p(p: float, n_recan: integer)**: Initializes sampler with a minimum probability threshold. - **llm_sampler_init_penalties(n_last_tokens: integer, repeat_penalty: float, frequency_penalty: float, presence_penalty: float)**: Initializes sampler with repetition penalties. - **llm_sampler_init_mirostat_v2(tau: float, eta: float, m: integer)**: Initializes sampler with Mirostat V2 entropy-based sampling. - **llm_sampler_init_grammar(grammar: string, root: string)**: Initializes sampler for grammar-constrained output. - **llm_sampler_init_dist(seed: integer)**: Initializes sampler with random distribution and seed. - **llm_sampler_free()**: Frees sampler resources. ### Request Example ```sql -- Create sampler (auto-created if not called) SELECT llm_sampler_create(); -- Greedy decoding (always pick most probable token) SELECT llm_sampler_init_greedy(); -- Top-K sampling (limit to top k tokens) SELECT llm_sampler_init_top_k(40); -- Top-P (nucleus) sampling SELECT llm_sampler_init_top_p(0.9, 1); -- Temperature control SELECT llm_sampler_init_temp(0.8); -- Minimum probability threshold SELECT llm_sampler_init_min_p(0.05, 1); -- Repetition penalties SELECT llm_sampler_init_penalties(64, 1.2, 0.5, 0.8); -- Mirostat entropy-based sampling SELECT llm_sampler_init_mirostat_v2(42, 5.0, 0.1); -- Grammar-constrained output SELECT llm_sampler_init_grammar('root ::= "yes" | "no"', 'root'); -- Random distribution with seed SELECT llm_sampler_init_dist(42); -- Free sampler resources SELECT llm_sampler_free(); ``` ### Response - **status** (integer) - 0 on success, non-zero on failure. ``` -------------------------------- ### Create Chat Session Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Initiates a new in-memory chat session using `llm_chat_create` and returns a unique session identifier. ```sql SELECT llm_chat_create(); ``` -------------------------------- ### Load a GGUF model Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Loads a model from a file path with optional configuration settings. ```sql SELECT llm_model_load('./models/llama.gguf', 'gpu_layers=99'); ``` -------------------------------- ### Initialize Grammar Constraint Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Constrains output to match a specified grammar. ```sql SELECT llm_sampler_init_grammar('...BNF...', 'root'); ``` -------------------------------- ### Integrate with sqlite3 Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/flutter/README.md Load the extension and perform inference using standard sqlite3 database connections. ```dart import 'package:sqlite3/sqlite3.dart'; import 'package:sqlite_ai/sqlite_ai.dart'; void main() { // Load once at startup. sqlite3.loadSqliteAiExtension(); final db = sqlite3.openInMemory(); // Check version. final result = db.select('SELECT ai_version() AS version'); print(result.first['version']); // Load a GGUF model. db.execute("SELECT llm_model_load('./models/llama.gguf', 'gpu_layers=99')"); // Run inference. final response = db.select("SELECT llm_chat_respond('What is 2+2?') AS answer"); print(response.first['answer']); db.dispose(); } ``` -------------------------------- ### Load GGUF Models Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Load a model from a file path with optional configuration for GPU acceleration and memory management. ```sql -- Load a model with GPU acceleration SELECT llm_model_load('./models/Qwen2.5-3B-Q4_K_M.gguf', 'gpu_layers=99'); -- Load with multiple options SELECT llm_model_load('./models/llama.gguf', 'gpu_layers=99,use_mmap=1,use_mlock=0'); -- Available options: -- gpu_layers=N (layers to store in VRAM) -- main_gpu=K (GPU for entire model when split_mode=0) -- split_mode=N (0=none, 1=layer, 2=rows) -- vocab_only=1/0 (only load vocabulary) -- use_mmap=1/0 (use memory mapping) -- use_mlock=1/0 (keep model in RAM) -- check_tensors=1/0 (validate tensor data) ``` -------------------------------- ### Integrate SQLite-AI with Flutter and Dart Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Shows how to load the extension using the sqlite3 package and configure it with Drift ORM. ```dart import 'package:sqlite3/sqlite3.dart'; import 'package:sqlite_ai/sqlite_ai.dart'; void main() { // Load extension once at startup sqlite3.loadSqliteAiExtension(); final db = sqlite3.openInMemory(); // Check version final result = db.select('SELECT ai_version() AS version'); print(result.first['version']); // Load a GGUF model db.execute("SELECT llm_model_load('./models/llama.gguf', 'gpu_layers=99')"); db.execute("SELECT llm_context_create_chat()"); // Run inference final response = db.select("SELECT llm_chat_respond('What is 2+2?') AS answer"); print(response.first['answer']); db.dispose(); } // With Drift ORM import 'package:drift/native.dart'; Sqlite3 loadExtensions() { sqlite3.loadSqliteAiExtension(); return sqlite3; } // Use when creating database NativeDatabase.createInBackground( File(path), sqlite3: loadExtensions, ); ``` -------------------------------- ### Load SQLite-AI Extension Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Loads the SQLite-AI extension into the SQLite CLI session. Ensure the extension file is in the current directory or provide the correct path. ```sql -- Load the extension .load ./ai ``` -------------------------------- ### getExtensionInfo() Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Returns detailed information about the extension for the current platform. ```APIDOC ## getExtensionInfo() ### Description Returns detailed information about the extension for the current platform. ### Returns - **ExtensionInfo** (object) - Contains platform, packageName, binaryName, and path. ### Throws - **ExtensionNotFoundError** - If the extension binary cannot be found ### Example ```typescript import { getExtensionInfo } from '@sqliteai/sqlite-ai'; const info = getExtensionInfo(); ``` ``` -------------------------------- ### POST llm_context_create Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Creates an inference context for AI operations. ```APIDOC ## POST llm_context_create ### Description Initializes a new inference context with specific configuration settings for text generation, embeddings, or chat. ### Method SELECT ### Endpoint llm_context_create(options) ### Parameters #### Request Body - **options** (string) - Optional - Configuration settings like 'n_ctx', 'n_threads', or 'embedding_type'. ``` -------------------------------- ### llm_context_create(context_settings TEXT) Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Creates a new inference context with specified configuration. ```APIDOC ## SELECT llm_context_create(context_settings) ### Description Creates a new inference context with comma separated key=value configuration. Context must be explicitly created before performing any AI operation. ### Parameters #### Arguments - **context_settings** (TEXT) - Required - Comma-separated key=value pairs for configuration (e.g., 'n_ctx=2048,n_threads=4'). ``` -------------------------------- ### Integrate SQLite-AI with Swift Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Provides the C-API approach for loading the extension and executing queries in Swift. ```swift import ai var db: OpaquePointer? sqlite3_open(":memory:", &db) sqlite3_enable_load_extension(db, 1) var errMsg: UnsafeMutablePointer? = nil sqlite3_load_extension(db, ai.path, nil, &errMsg) // Check version var stmt: OpaquePointer? sqlite3_prepare_v2(db, "SELECT ai_version()", -1, &stmt, nil) defer { sqlite3_finalize(stmt) } sqlite3_step(stmt) print("ai_version(): \(String(cString: sqlite3_column_text(stmt, 0)))") sqlite3_close(db) ``` -------------------------------- ### Manage System Prompt Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Use `llm_chat_system_prompt` to set or retrieve the system prompt, controlling AI behavior and personality for chat sessions. ```sql SELECT llm_chat_system_prompt('You are a helpful assistant that speaks Italian.'); ``` ```sql SELECT llm_chat_system_prompt(); ``` ```sql SELECT llm_chat_system_prompt('You are an expert Python developer. Provide concise code examples.'); ``` -------------------------------- ### Load Vision Model for SQLite AI Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Loads a multimodal projector (mmproj) model for vision capabilities after a text model is loaded. Images can then be passed to llm_text_generate or llm_chat_respond. ```sql SELECT llm_model_load('./models/Gemma-3-4B-IT-Q4_K_M.gguf', 'gpu_layers=99'); SELECT llm_context_create_textgen(); SELECT llm_vision_load('./models/mmproj-Gemma-3-4B-IT-f16.gguf'); SELECT llm_text_generate('Describe this image', './photos/cat.jpg'); ``` -------------------------------- ### Transcribe Audio with Options Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Transcribes audio from a file path with specified options, such as language and translation. The 'language' parameter sets the source language, and 'translate' enables translation to English. ```sql -- Transcribe with options SELECT audio_model_transcribe('./audio/speech.mp3', 'language=it,translate=1'); ``` -------------------------------- ### llm_model_load(path TEXT, options TEXT) Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Loads a GGUF model from the specified file path with optional configuration. ```APIDOC ## SELECT llm_model_load(path, options) ### Description Loads a GGUF model from the specified file path with optional comma separated key=value configuration. ### Parameters #### Arguments - **path** (TEXT) - Required - File path to the GGUF model. - **options** (TEXT) - Optional - Comma separated key=value pairs (e.g., 'gpu_layers=99'). ``` -------------------------------- ### Monitor Context Usage Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Check how many tokens have been consumed in the current context and calculate usage percentage. ```sql SELECT llm_context_used(); -- Returns: 1024 -- Monitor context usage SELECT llm_context_used() AS used_tokens, llm_context_size() AS total_tokens, (llm_context_used() * 100.0 / llm_context_size()) AS usage_percent; ``` -------------------------------- ### llm_context_create Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Creates a general inference context with customizable settings. ```APIDOC ## `llm_context_create(context_settings TEXT)` **Parameters:** **`context_settings` (optional):** Comma-separated `key=value` pairs to override or extend default settings. **Returns:** `NULL` **Description:** Creates a new inference context. This is the base function for creating contexts and can be customized using various settings. **Example:** ```sql SELECT llm_context_create('n_ctx=2048,n_threads=6,n_batch=256'); ``` **Context must explicitly created before performing any AI operation!** ``` -------------------------------- ### Create Inference Context Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Initialize a new context for operations like text generation or embeddings with specific configuration parameters. ```sql -- Create context with custom settings SELECT llm_context_create('n_ctx=2048,n_threads=6,n_batch=256'); -- Key context settings: -- context_size=N (n_ctx and n_batch combined) -- n_ctx=N (text context length in tokens) -- n_batch=N (max batch size for llama_decode) -- n_threads=N (threads for generation) -- embedding_type=TYPE (FLOAT32, FLOAT16, BFLOAT16, UINT8, INT8) -- pooling_type=TYPE (none, mean, cls, last, rank) -- flash_attn_type=TYPE (auto, disabled, enabled) ``` -------------------------------- ### Create Embedding Context Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Initialize a context for generating embeddings with default normalization and mean pooling. ```sql -- Create embedding context SELECT llm_context_create_embedding(); -- Specify embedding data type SELECT llm_context_create_embedding('embedding_type=FLOAT32'); -- With custom options SELECT llm_context_create_embedding('embedding_type=FLOAT16,normalize_embedding=1'); ``` -------------------------------- ### Load LoRA Adapter Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Applies a LoRA adapter to the model with a specified scale factor. ```sql -- Load LoRA adapter with full scale SELECT llm_lora_load('./adapters/custom_style.lora', 1.0); -- Load with reduced influence SELECT llm_lora_load('./adapters/domain_specific.lora', 0.5); ``` -------------------------------- ### Create Text Generation Context Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Initializes a new inference context for text generation. Must be called before any AI operations. ```sql SELECT llm_context_create_textgen(); ``` -------------------------------- ### Generate Text Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Generates text based on a given prompt using the loaded text generation model and context. This is a direct query for text output. ```sql -- Generate text SELECT llm_text_generate('What is the most beautiful city in Italy?'); ``` -------------------------------- ### getExtensionPath() Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Retrieves the absolute path to the SQLite AI extension binary for the current platform. ```APIDOC ## getExtensionPath() ### Description Returns the absolute path to the SQLite AI extension binary for the current platform. ### Returns - **string** - Absolute path to the extension file (.so, .dylib, or .dll) ### Throws - **ExtensionNotFoundError** - If the extension binary cannot be found for the current platform ### Example ```typescript import { getExtensionPath } from '@sqliteai/sqlite-ai'; const path = getExtensionPath(); ``` ``` -------------------------------- ### Retrieve SQLite-AI version Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Returns the current version string of the SQLite-AI extension. ```sql SELECT ai_version(); -- e.g., '1.0.0' ``` -------------------------------- ### llm_vision_load Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Loads a multimodal projector model for vision capabilities. ```APIDOC ## SELECT llm_vision_load(model_path, [options]) ### Description Loads a multimodal projector model for vision capabilities. ### Parameters - **model_path** (string) - Required - Path to the GGUF model file. - **options** (string) - Optional - Configuration options. ### Request Example SELECT llm_vision_load('./models/mmproj.gguf', 'use_gpu=1'); ``` -------------------------------- ### Create Default Inference Context Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Creates a new inference context with specified parameters. Ensure context is created before any AI operations. ```sql SELECT llm_context_create('n_ctx=2048,n_threads=6,n_batch=256'); ``` -------------------------------- ### Android SQLite AI Configuration Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Provides Gradle dependency and Java configuration for initializing the SQLite AI extension in Android. ```gradle implementation 'ai.sqlite:ai:0.7.55' ``` ```java SQLiteCustomExtension aiExtension = new SQLiteCustomExtension(getApplicationInfo().nativeLibraryDir + "/ai", null); SQLiteDatabaseConfiguration config = new SQLiteDatabaseConfiguration( getCacheDir().getPath() + "/ai_test.db", SQLiteDatabase.CREATE_IF_NECESSARY | SQLiteDatabase.OPEN_READWRITE, Collections.emptyList(), Collections.emptyList(), Collections.singletonList(aiExtension) ); SQLiteDatabase db = SQLiteDatabase.openDatabase(config, null, null); ``` -------------------------------- ### Generate Text Completion Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Use `llm_text_generate` for basic text completion. Specify `n_predict` for token limits. Supports vision models for image descriptions. ```sql SELECT llm_text_generate('Once upon a time'); ``` ```sql SELECT llm_text_generate('Write a poem about the ocean', 'n_predict=256'); ``` ```sql SELECT llm_text_generate('Describe this image', './photos/cat.jpg'); ``` ```sql SELECT llm_text_generate('What is different between these images?', './img1.jpg', './img2.jpg'); ``` ```sql SELECT llm_text_generate('What do you see?', image_data) FROM photos WHERE id = 1; ``` -------------------------------- ### Load Multimodal Model and Vision Projector Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Loads a multimodal model and its associated vision projector, which are necessary for image analysis tasks. The 'gpu_layers' parameter offloads model layers to the GPU. ```sql -- Load a multimodal model and its vision projector SELECT llm_model_load('./models/Gemma-3-4B-IT-Q4_K_M.gguf', 'gpu_layers=99'); SELECT llm_vision_load('./models/mmproj-Gemma-3-4B-IT-f16.gguf'); ``` -------------------------------- ### Load Vision Model Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Loads a vision projector model using `llm_vision_load` after loading a compatible text model. Enables image understanding capabilities. ```sql SELECT llm_model_load('./models/Gemma-3-4B-IT-Q4_K_M.gguf', 'gpu_layers=99'); ``` ```sql SELECT llm_context_create_textgen(); ``` ```sql SELECT llm_vision_load('./models/mmproj-Gemma-3-4B-IT-f16.gguf'); ``` ```sql SELECT llm_vision_load('./models/mmproj.gguf', 'use_gpu=1,n_threads=4,warmup=1'); ``` ```sql SELECT llm_text_generate('Describe this image in detail', './photos/scene.jpg'); ``` -------------------------------- ### getCurrentPlatform() Source: https://github.com/sqliteai/sqlite-ai/blob/main/packages/node/README.md Returns the current platform identifier. ```APIDOC ## getCurrentPlatform() ### Description Returns the current platform identifier (e.g., 'darwin-arm64', 'linux-x86_64'). ### Returns - **Platform** (string) - The identifier for the current OS and architecture. ### Throws - **Error** - If the platform is unsupported ``` -------------------------------- ### LLM Vision Load API Source: https://github.com/sqliteai/sqlite-ai/blob/main/API.md Loads a multimodal projector (mmproj) model for vision capabilities. Requires a text model to be loaded first. ```APIDOC ## `llm_vision_load(path TEXT, options TEXT)` ### Description Loads a multimodal projector (mmproj) model for vision capabilities. This requires a text model to already be loaded via `llm_model_load()`. The mmproj file is a separate GGUF file that contains the vision encoder and projector weights. Once loaded, vision capabilities are available through `llm_text_generate()` and `llm_chat_respond()` by passing image arguments. ### Parameters #### Path Parameters - **path** (TEXT) - Required - Path to the mmproj model file. - **options** (TEXT) - Optional - Comma-separated key=value configuration options. - `use_gpu` (1 or 0) - Default: `1` - Use GPU for vision encoding. - `n_threads` (number) - Default: `4` - Number of threads for vision processing. - `warmup` (1 or 0) - Default: `1` - Run a warmup pass on load for faster first use. - `flash_attn_type` (auto, disabled, enabled) - Default: `auto` - Controls Flash Attention for the vision encoder. - `image_min_tokens` (number) - Default: `0` - Minimum image tokens for dynamic resolution models (0 = from model). - `image_max_tokens` (number) - Default: `0` - Maximum image tokens for dynamic resolution models (0 = from model). ### Response #### Success Response (200) - **NULL** - Indicates successful loading. ### Request Example ```sql -- Load text model first SELECT llm_model_load('./models/Gemma-3-4B-IT-Q4_K_M.gguf', 'gpu_layers=99'); SELECT llm_context_create_textgen(); -- Load vision projector SELECT llm_vision_load('./models/mmproj-Gemma-3-4B-IT-f16.gguf'); -- Now use vision with llm_text_generate or llm_chat_respond SELECT llm_text_generate('Describe this image', './photos/cat.jpg'); ``` ``` -------------------------------- ### Load Extension in SQL Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Loads the SQLite-AI extension directly within an SQL query. This is an alternative to using the `.load` command in the SQLite CLI. ```sql -- In SQL SELECT load_extension('./ai'); ``` -------------------------------- ### Transcribe Audio from File Source: https://github.com/sqliteai/sqlite-ai/blob/main/README.md Transcribes audio from a specified file path using the loaded Whisper model. Supports various audio formats. ```sql -- Transcribe from a file path SELECT audio_model_transcribe('./audio/speech.wav'); ``` -------------------------------- ### Free Audio Model Resources Source: https://context7.com/sqliteai/sqlite-ai/llms.txt Unloads the currently active Whisper model and releases memory. ```sql SELECT audio_model_free(); ```