### Snapshotting Docs for a New Release (Python Example) Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/README.md Command to snapshot the current 'main' branch docs for a specific binding to create a new versioned release. This example uses Python. ```bash cd docs npx docusaurus docs:version:python 1.5.0 ``` -------------------------------- ### Setup virtual environment with uv Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/python/README.md Initialize and sync the virtual environment using the uv package manager. ```bash uv venv uv sync ``` -------------------------------- ### Godot RAG Initialization and Setup Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/site/godot/rag.md Initializes the language model, sets up a reranker, and defines the system prompt for the tavern keeper persona. Connects to signals for response handling and starts the worker process. ```gdscript var ranked_docs = [] func _ready(): # Set up the chat for generating helpful responses self.model_node = chat_model reranker.connect("ranking_finished", func(result): ranked_docs = result) reranker.start_worker() self.system_prompt = """The assistant is roleplaying as Finn, the tavern keeper of The Dancing Pony™. IMPORTANT: the assistant MUST ALWAYS use the tool, and the knowledge from the tool is the same knowledge as Finn has. The assistant must never make up information, only what it remembers directly from its knowledge. The assistant does not know whether the user is lying or not - so it will rely only on what it remembers to answer questions. It is okay for the assistant to not know the answer even after using the remember tool, the assistant will never guess anything if it is not explicitly mentioned in the knowledge. The assistant must always speak like a tavern keeper. " # Add the tool to remember stuff self.add_tool(remember, "The assistant can use this tool to remember its limited knowledge about the ingame world.") self.connect("response_finished", func(response: String): print("Finn says: ", response)) start_worker() ``` -------------------------------- ### Run Android Test App Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/react-native/DEVELOPMENT.md Installs and launches the Android test app after starting the Metro bundler. Requires a connected device or emulator. ```bash # Terminal 1: start Metro bundler (must be running before the app launches) cd nobodywho/react-native/test-app npx react-native start --port 8081 # Terminal 2: install, set up port forwarding, and launch adb install -r nobodywho/react-native/test-app/android/app/build/outputs/apk/debug/app-debug.apk adb reverse tcp:8081 tcp:8081 adb shell am start -n com.nobodywhotest/.MainActivity ``` -------------------------------- ### Basic Chat Interaction Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/index.md Create a Chat instance using a model path (e.g., from Hugging Face) and interact with it by asking questions. This example shows how to stream tokens or get the complete response. ```kotlin import ai.nobodywho.Chat import kotlinx.coroutines.runBlocking fun main() = runBlocking { val chat = Chat.fromPath( modelPath = "hf://NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf" ) // stream tokens as they are generated chat.ask("Is water wet?").asFlow().collect { print(it) } // ...or get the entire response as a single string val response = chat.ask("Is water wet?").completed() println(response) } ``` -------------------------------- ### GPU Acceleration Setup Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/chat.md Instantiate the Model class with GPU acceleration enabled if available. This uses Vulkan. ```python Model('./model.gguf', use_gpu_if_available=True) ``` -------------------------------- ### Example Compact Format Output Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/site/godot/chat/structured-output.md An example of the compact, pipe-and-colon-separated data format. ```Text Gandalf:high,low,low:Staff,Robes|Aragorn:low,high,medium:Sword,Plate ``` -------------------------------- ### System Prompt Examples Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/site/godot/chat/simple-chat.md Define the LLM's behavior and persona using system prompts. These examples show character-based and task-specific configurations. ```markdown # Character-based behavior system_prompt = """You are a sarcastic but brilliant wizard. Your answers are always accurate, but delivered with a dry wit. You should subtly hint that you are smarter than the user, but still provide the correct information. """ ``` ```markdown # Task-specific behavior system_prompt = """You are a translation assistant. You will be given text in any language. Your job is to translate it into formal, academic French. Do not add any commentary or conversational text. Respond only with the translated text. """ ``` -------------------------------- ### Verify Python installation Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/python/README.md Activate the environment and verify the module import. ```bash > source .venv/bin/python > python >>> import nobodywho ... ``` -------------------------------- ### Basic CMake Project Setup Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/react-native/android/CMakeLists.txt Initializes CMake version and project name. Sets the C++ standard to C++17 and enables verbose build output. ```cmake cmake_minimum_required(VERSION 3.9.0) project(Nobodywho) set (CMAKE_VERBOSE_MAKEFILE ON) set (CMAKE_CXX_STANDARD 17) ``` -------------------------------- ### Load Vision Model with Chat.fromPath Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-swift/vision.md Load a multimodal model and its projection model directly using `Chat.fromPath`. This is a convenient way to get started with image and audio processing. ```swift import NobodyWho let chat = try await Chat.fromPath( modelPath: "/path/to/vision-model.gguf", projectionModelPath: "/path/to/mmproj.gguf", systemPrompt: "You are a helpful assistant, that can hear and see stuff!" ) ``` -------------------------------- ### Example: Qwen3/3.5 Reasoning with enable_thinking Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/chat.md Demonstrates using the 'enable_thinking' template variable with Qwen3 and Qwen3.5 models to show reasoning steps before the final answer. ```dart final chat = await nobodywho.Chat.fromPath( modelPath: "./model.gguf", templateVariables: {"enable_thinking": true} ); final response = chat.ask("Solve this logic puzzle: ..."); ``` -------------------------------- ### Install react-native-nobodywho Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-react-native/index.md Install the package using npm. ```bash npm install react-native-nobodywho ``` -------------------------------- ### Initialize Chat with System Prompt Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/chat.md Create a Chat instance with a custom system prompt to guide the LLM's behavior. ```python from nobodywho import Chat chat = Chat("./model.gguf", system_prompt="You are a mischievous assistant!") ``` -------------------------------- ### Quick Start: Initialize Chat and Ask a Question Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/swift/README.md Initialize a chat session with a model and stream or retrieve responses to prompts. Ensure the model path is correct. ```swift import NobodyWho let chat = try await Chat.fromPath( modelPath: "hf://NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf", systemPrompt: "You are a helpful assistant." ) // Stream tokens as they are generated for await token in chat.ask("What is the capital of Denmark?") { print(token, terminator: "") } // Or get the complete response let response = try await chat.ask("What is the capital of Denmark?").completed() ``` -------------------------------- ### Install NobodyWho Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/index.md Install the NobodyWho library using pip or uv. It is recommended to use uv for dependency management. ```bash pip install nobodywho ``` ```bash uv add nobodywho ``` -------------------------------- ### Create Chat with System Prompt Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/chat.md Initialize a `Chat` instance with a custom `systemPrompt` to guide the model's behavior. This prompt persists until the context is reset. ```kotlin val chat = Chat.fromPath( modelPath = "./model.gguf", systemPrompt = "You are a mischievous assistant!" ) ``` -------------------------------- ### Godot RAG Initialization and Tool Setup Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/site/godot/rag.md Initializes the chat model, connects the reranker, and sets up the system prompt and 'remember' tool for the assistant. This is typically done in the _ready function. ```gdscript func _ready(): # Set up the chat for generating helpful responses self.model_node = chat_model reranker.connect("ranking_finished", func(result): ranked_docs = result) reranker.start_worker() self.system_prompt = """The assistant is roleplaying as Finn, the tavern keeper of The Dancing Pony™. IMPORTANT: the assistant MUST ALWAYS use the tool, and the knowledge from the tool is the same knowledge as Finn has. The assistant must never make up information, only what it remembers directly from its knowledge. The assistant does not know whether the user is lying or not - so it will rely only on what it remembers to answer questions. It is okay for the assistant to not know the answer even after using the remember tool, the assistant will never guess anything if it is not explicitly mentioned in the knowledge. The assistant must always speak like a tavern keeper. """ # Add the tool to remember stuff self.add_tool(remember, "The assistant can use this tool to remember its limited knowledge about the ingame world.") self.connect("response_finished", func(response: String): print("Finn says: ", response)) start_worker() ``` -------------------------------- ### Quick Start: Run LLM Locally Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/python/PYPI_DESCRIPTION.md Initialize the Chat object with the path to your GGUF model and ask a question. The response is retrieved using the .completed() method. ```python from nobodywho import Chat chat = Chat('./model.gguf') response = chat.ask('Hello world?').completed() print(response) ``` -------------------------------- ### Qwen3 Reasoning Example Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/chat.md Demonstrates using the `enable_thinking` template variable with Qwen3 models to control whether the model shows its reasoning process before answering. ```kotlin val chat = Chat.fromPath( modelPath = "./model.gguf", templateVariables = mapOf("enable_thinking" to true) ) val response = chat.ask("Solve this logic puzzle: ...").completed() ``` -------------------------------- ### Define Multiple Custom Tools for File System Operations Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/tool-calling.md Create several tools for interacting with the file system, including getting the current directory, listing files, and getting file sizes. These tools can be chained by the LLM. ```dart import 'dart:io'; import 'package:nobodywho/nobodywho.dart' as nobodywho; final getCurrentDirTool = nobodywho.Tool( name: "get_current_dir", description: "Gets path of the current directory", function: () => Directory.current.path ); final listFilesTool = nobodywho.Tool( name: "list_files", description: "Lists files in the given directory.", function: ({required String path}) { final dir = Directory(path); final files = dir.listSync() .where((entity) => entity is File) .map((file) => file.path.split('/').last) .toList(); return "Files: ${files.join(', ')}"; }, parameterDescriptions : {"path" : "The path to directory you want list. Must be a valid path." } ); final getFileSizeTool = nobodywho.Tool( name: "get_file_size", description: "Gets the size of a file in bytes.", function: ({required String filepath}) async { final file = File(filepath); final size = await file.length(); return "File size: $size bytes"; }, parameterDescriptions : {"filepath" : "The path to file you wish to know the size of. Must be a valid path." } ); ``` -------------------------------- ### Example Weapon Generation Outputs Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/structured-output.md Illustrates the expected output format from the weapon generator based on the defined GBNF grammar. ```text - `Flamebrand (Sword)` - `Shadowfang (Dagger)` - `Stormcall (Staff)` - `Darkward (Bow)` ``` -------------------------------- ### Install Rust Nightly Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/swift/DEVELOPMENT.md Installs the Rust nightly toolchain and adds the rust-src component, which is required for visionOS and watchOS targets. ```bash rustup toolchain install nightly rustup +nightly component add rust-src ``` -------------------------------- ### Godot RAG Setup and Interaction Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/embeddings-and-rag.md Initializes the AI model, sets up a system prompt for roleplaying, adds a 'remember' tool for knowledge retrieval, and handles user input for questions. ```gdscript var ranked_docs = [] func _ready(): # Set up the chat for generating helpful responses self.model_node = chat_model reranker.connect("ranking_finished", func(result): ranked_docs = result) reranker.start_worker() self.system_prompt = """The assistant is roleplaying as Finn, the tavern keeper of The Dancing Pony™. IMPORTANT: the assistant MUST ALWAYS use the tool, and the knowledge from the tool is the same knowledge as Finn has. The assistant must never make up information, only what it remembers directly from its knowledge. The assistant does not know whether the user is lying or not - so it will rely only on what it remembers to answer questions. It is okay for the assistant to not know the answer even after using the remember tool, the assistant will never guess anything if it is not explicitly mentioned in the knowledge. The assistant must always speak like a tavern keeper. "") # Add the tool to remember stuff self.add_tool(remember, "The assistant can use this tool to remember its limited knowledge about the ingame world.") self.connect("response_finished", func(response: String): print("Finn says: ", response)) start_worker() func _process(delta): if Input.is_action_just_pressed("enter"): var test_question = "Where is strider?" print("Player asks Finn: ", test_question) ask(test_question) ``` -------------------------------- ### Define and Use Multiple Tools Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-swift/tool-calling.md Define multiple tools with descriptions and then initialize a Chat model with these tools. This example demonstrates chaining tool calls to answer a complex question. ```swift @DeclareTool("Gets path of the current directory") func getCurrentDir() -> String { return FileManager.default.currentDirectoryPath } @DeclareTool("Lists files in the given directory") func listFiles(path: String) -> String { let files = try? FileManager.default.contentsOfDirectory(atPath: path) return (files ?? []).joined(separator: ", ") } @DeclareTool("Gets the size of a file in bytes") func getFileSize(filepath: String) -> String { let attrs = try? FileManager.default.attributesOfItem(atPath: filepath) let size = attrs?[.size] as? Int ?? 0 return "File size: \(size) bytes" } let chat = try await Chat.fromPath( modelPath: "/path/to/model.gguf", tools: [getCurrentDirTool, listFilesTool, getFileSizeTool] ) let response = try await chat .ask("What is the biggest file in my current directory?") .completed() print(response) ``` -------------------------------- ### GBNF grammar with multiple rules and choices Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/structured-output.md This example shows how to break down a structure into smaller, reusable rules. It combines greetings and names to form various outputs. ```gbnf root ::= greeting " " name greeting ::= "Hello" | "Hi" | "Hey" name ::= "World" | "Friend" | "There" ``` -------------------------------- ### Project Configuration Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/flutter/nobodywho/linux/CMakeLists.txt Sets the project name and specifies the programming languages used. This is a standard CMake project setup. ```cmake set(PROJECT_NAME "nobodywho") project(${PROJECT_NAME} LANGUAGES CXX) ``` -------------------------------- ### Compact Format Example (Pipe-Separated) Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/structured-output.md Generates a compact, pipe-separated format for character data, including name, level, and class. This format is more token-efficient than JSON. ```GBNF root ::= [A-Z][a-z]+ "|" [1-9][0-9]? "|" class-type class-type ::= "Warrior" | "Mage" | "Rogue" | "Cleric" ``` -------------------------------- ### Initialize Chat with System Prompt Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/chat.md Specify a system prompt during Chat initialization to guide the LLM's behavior. This prompt is persisted until the context is reset. ```dart import 'package:nobodywho/nobodywho.dart' as nobodywho; final chat = await nobodywho.Chat.fromPath( modelPath: "./model.gguf", systemPrompt: "You are a mischievous assistant!" ); ``` -------------------------------- ### Alternative Prompt Composition Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/vision.md Experiment with different prompt compositions by reordering text and multimodal parts to potentially improve model performance. This example separates the image and audio descriptions. ```dart await chat.resetHistory(); final response2 = await chat.askWithPrompt(nobodywho.Prompt([ nobodywho.TextPart("Tell me what you see in the image."), nobodywho.ImagePart("./dog.png"), nobodywho.TextPart("Also tell me what you hear in the audio"), nobodywho.AudioPart("./sound.mp3"), ])).completed(); ``` -------------------------------- ### Install NobodyWho Python Package Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/README.md Command to install the NobodyWho package for Python using pip. ```sh pip install nobodywho ``` -------------------------------- ### Example Nested JSON Output Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/structured-output.md An example of the JSON output generated by the nested structure GBNF. ```JSON { "character":{"name":"Gandalf","stats":{"hp":"100","mp":"200"}} } ``` -------------------------------- ### Initialize Chat with a Tool Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/tool-calling.md Create a Chat instance and provide the custom `circle_area` tool to enable the LLM to use it. ```python chat = Chat('./model.gguf', tools=[circle_area]) ``` -------------------------------- ### Build Native Library for Host Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/kotlin/DEVELOPMENT.md Build the native UniFFI library for the host platform before running tests. This command should be executed from the `nobodywho/` workspace root. ```bash cargo build -p nobodywho-uniffi ``` -------------------------------- ### Run Full Suite with Sample Limit Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/evals/lm_eval_scripts/README.md Execute the entire benchmark suite with a uniform sample limit for all tasks. ```bash python main.py ./model.gguf -l 500 ``` -------------------------------- ### Define Multiple Tools for Chained Calls Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/tool-calling.md Define several tools including getting the current directory, listing files, and getting file size. These can be chained by the LLM for complex queries. ```python import os from pathlib import Path from nobodywho import Chat, tool @tool(description="Gets path of the current directory") def get_current_dir() -> str: return os.getcwd() @tool(description="Lists files in the given directory", params={"path": "a relative or absolute path to a directory"}) def list_files(path: str) -> str: files = [f.name for f in Path(path).iterdir() if f.is_file()] return f"Files: {', '.join(files)}" @tool(description="Gets the size of a file in bytes") def get_file_size(filepath: str) -> str: size = Path(filepath).stat().st_size return f"File size: {size} bytes" chat = Chat('./model.gguf', tools=[get_current_dir, list_files, get_file_size]) response = chat.ask('What is the biggest file in my current directory?').completed() print(response) # The largest file in your current directory is `model.gguf`. ``` -------------------------------- ### Get Completed Async Chatbot Response Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/streaming-and-async-api.md Use the `ChatAsync` class and `await` the `.completed()` method to get the full chatbot response asynchronously. This is suitable when you need the entire message before proceeding. ```python import asyncio from nobodywho import ChatAsync async def main(): chat = ChatAsync('./model.gguf') response = await chat.ask('How are you?').completed() print(response) asyncio.run(main()) ``` -------------------------------- ### Complete RAG Example with Two-Stage Retrieval Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/embeddings-and-rag.md A full example demonstrating a two-stage retrieval system using embeddings for fast filtering and a cross-encoder for precise reranking. This approach is efficient for large knowledge bases. ```dart import 'dart:typed_data'; import 'package:nobodywho/nobodywho.dart' as nobodywho; Future main() async { // Initialize models final encoder = await nobodywho.Encoder.fromPath(modelPath: './embedding-model.gguf'); final crossencoder = await nobodywho.CrossEncoder.fromPath(modelPath: './reranker-model.gguf'); // Large knowledge base final knowledgeBase = [ "Python 3.11 introduced performance improvements through faster CPython", "The Django framework is used for building web applications", "NumPy provides support for large multi-dimensional arrays", "Pandas is the standard library for data manipulation and analysis", // ... 100+ more documents ]; // Precompute embeddings for all documents final docEmbeddings = []; for (final doc in knowledgeBase) { docEmbeddings.add(await encoder.encode(text: doc)); } Future search({required String query}) async { // Stage 1: Fast filtering with embeddings final queryEmbedding = await encoder.encode(text: query); final similarities = <(String, double)>[]; for (int i = 0; i < knowledgeBase.length; i++) { final similarity = nobodywho.cosineSimilarity( a: queryEmbedding.toList(), b: docEmbeddings[i].toList() ); similarities.add((knowledgeBase[i], similarity)); } // Get top 20 candidates similarities.sort((a, b) => b.$2.compareTo(a.$2)); final candidateDocs = similarities.take(20).map((e) => e.$1).toList(); // Stage 2: Precise ranking with cross-encoder final ranked = await crossencoder.rankAndSort(query: query, documents: candidateDocs); // Return top 3 most relevant final topResults = ranked.take(3).map((e) => e.$1).toList(); return topResults.join("\n---\n"); } final searchTool = nobodywho.Tool( function: search, name: "search", description: "Search the knowledge base for information relevant to the query" ); // Create RAG-enabled chat final chat = await nobodywho.Chat.fromPath( modelPath: './model.gguf', systemPrompt: "You are a technical documentation assistant. Always use the search tool to find relevant information before answering programming questions.", templateVariables: {"enable_thinking": false}, tools: [searchTool] ); // The chat automatically searches and uses retrieved documents final response = await chat.ask("What Python libraries are best for data analysis?").completed(); print(response); } ``` -------------------------------- ### Setup Flutter Test Environment on macOS Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/flutter/README.md Prepare the Flutter test environment on macOS by building the Rust crate, creating a framework directory, and linking the dynamic library. This is a temporary solution due to flutter_rust_bridge's library path handling. ```bash cd flutter/rust cargo build cd ../nobodywho mkdir -p nobodywho_flutter.framework ln -sf ../../target/debug/libnobodywho_flutter.dylib nobodywho_flutter.framework/nobodywho_flutter flutter test test/nobodywho_test.dart ``` -------------------------------- ### Example of problematic prompt engineering for JSON Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/structured-output.md This example shows a common prompt that attempts to force JSON output but often fails due to model deviations. It highlights the need for more robust methods like GBNF. ```text """ Please respond in JSON format with name, level, and class fields Only use those fields. Only use valid json. All json attributes should have " around them. Please do not deviate from the instructions. You will lose 10 points if you use other fields than level, class and name. Do not write a message just json. If you do not respond in valid json I will lose my job and my kids will starve. """ ``` -------------------------------- ### Declare Multiple File System Tools Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-react-native/tool-calling.md Defines a set of tools for interacting with a file system, including getting the current directory, listing files, and getting file sizes. Each tool has a name, description, parameters, and a callback function. ```typescript import { Chat, Tool } from "react-native-nobodywho"; const getCurrentDir = (): string => '/home/user/documents'; // In a real app, you'd read the filesystem here const listFiles = (path: string): string => 'Files: report.pdf, notes.txt, model.gguf'; // In a real app, you'd check the actual file size const getFileSize = (filepath: string): string => 'File size: 1024 bytes'; const getCurrentDirTool = new Tool({ name: "get_current_dir", description: "Gets path of the current directory", parameters: [], call: getCurrentDir, }); const listFilesTool = new Tool({ name: "list_files", description: "Lists files in the given directory", parameters: [ { name: "path", type: "string", description: "The path to the directory to list" }, ], call: listFiles, }); const getFileSizeTool = new Tool({ name: "get_file_size", description: "Gets the size of a file in bytes", parameters: [ { name: "filepath", type: "string", description: "The path to the file" }, ], call: getFileSize, }); ``` -------------------------------- ### Initialize Chat with File System Tools and Ask a Question Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-react-native/tool-calling.md Initializes a Chat instance with multiple file system tools and then asks a question that may involve chaining these tools. The response from the LLM is logged to the console. ```typescript const chat = await Chat.fromPath({ modelPath: "/path/to/model.gguf", tools: [getCurrentDirTool, listFilesTool, getFileSizeTool], }); const response = await chat .ask("What is the biggest file in my current directory?") .completed(); console.log(response); ``` -------------------------------- ### Add NobodyWho Dependency Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/index.md Install the NobodyWho package using Flutter's package manager. ```bash flutter pub add nobodywho ``` -------------------------------- ### Initialize Model with Projection Model Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/vision.md Load a multimodal model and its corresponding projection model. Ensure the projection model path is correctly specified. ```python from nobodywho import Model, Chat model = Model("./vision-model.gguf", projection_model_path="./projection_model.gguf") chat = Chat( model, system_prompt="You are a helpful assistant, that can hear and see stuff!" ) ``` -------------------------------- ### Async API: Completed message Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/site/python/streaming-and-async-api.md Use ChatAsync to get the full completed message asynchronously. ```APIDOC ## Async API: Completed message ### Description Use the `ChatAsync` object to retrieve the entire completed message asynchronously. ### Method Asynchronous call to `.completed()` ### Endpoint N/A (SDK method) ### Parameters N/A ### Request Example ```python import asyncio from nobodywho import ChatAsync async def main(): chat = ChatAsync('./model.gguf') response = await chat.ask('How are you?').completed() print(response) asyncio.run(main()) ``` ### Response #### Success Response The complete response message. #### Response Example ``` I am doing well, thank you for asking! ``` ``` -------------------------------- ### Updating Latest Releases Configuration Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/README.md Example of updating the 'latestReleases' object in docusaurus.config.ts to reflect a new version. ```typescript python: '1.5.0', // was '1.4.0' ``` -------------------------------- ### Initialize Chat with Pre-packaged Python and Bash Tools Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/tool-calling.md Instantiate a Chat object and include the pre-packaged `python_tool` and `bash_tool` to provide the LLM with Python and Bash execution capabilities. ```python from nobodywho import python_tool, bash_tool chat = Chat('./model.gguf', tools=[python_tool(), bash_tool()]) ``` -------------------------------- ### Adjust Prompt Composition for Better Results Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/vision.md Experiment with the order of text and media elements in the prompt, and adjust descriptive text to potentially improve model performance. This example shows a different arrangement of text and media. ```kotlin chat.resetHistory() val response = chat.ask(Prompt( Prompt.Text("Tell me what you see in the image."), Prompt.Image("./dog.png"), Prompt.Text("Also tell me what you hear in the audio."), Prompt.Audio("./sound.mp3"), )).completed() ``` -------------------------------- ### Get Chat History Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/chat.md Retrieve the list of messages exchanged in the current chat session using `getChatHistory`. ```dart final msgs = await chat.getChatHistory(); print(msgs[0].content); // "Is water wet?" ``` -------------------------------- ### Run with Custom Output File Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/evals/lm_eval_scripts/README.md Execute benchmarks and save results to a custom CSV file. ```bash python main.py ./model.gguf -l 100 -o my_results.csv ``` -------------------------------- ### Get the Complete Response Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/chat.md Retrieve the entire response string from the `TokenStream` once the model has finished generating it by calling `completed()`. ```dart final fullResponse = await response.completed(); ``` -------------------------------- ### Pre-load LLM Worker for Performance Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/chat.md In the `_ready()` function, configure chat behavior, set the system prompt, and assign the model node. Crucially, call `start_worker()` early to pre-load the model into memory, ensuring faster responses during user interaction. ```gdscript # In your _ready() function, set up everything before the app starts. func _ready(): # 1. Configure the chat behavior self.system_prompt = "You are a helpful assistant." self.model_node = get_node("../SharedModel") # 2. Start the worker *before* the user can interact. # This pre-loads the model so the first interaction isn't slow. start_worker() # 3. Now other setup can happen print("Assistant chat is ready.") ``` -------------------------------- ### Get Completed Response Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-swift/chat.md Retrieves the entire generated response string from the TokenStream once the model has finished generating. This is an asynchronous operation. ```swift let fullResponse = try await response.completed() ``` -------------------------------- ### Build a RAG System with Chat and Tools Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/embeddings-and-rag.md Integrates a cross-encoder for knowledge search into a chat system to enable Retrieval-Augmented Generation. This example sets up a customer service assistant that uses a search tool to find relevant information before responding. Ensure the reranker and chat models are available at their respective paths. ```kotlin import ai.nobodywho.* suspend fun main() { val crossEncoder = CrossEncoder.fromPath(modelPath = "./reranker-model.gguf") val knowledge = listOf( "Our company offers a 30-day return policy for all products", "Free shipping is available on orders over $50", "Customer support is available via email and phone", "We accept credit cards, PayPal, and bank transfers", "Order tracking is available through your account dashboard" ) fun searchKnowledge(query: String): String { val ranked = runBlocking { crossEncoder.rankAndSort(query, knowledge) } return ranked.take(3).joinToString("\n") { it.first } } val searchTool = Tool( name = "search_knowledge", description = "Search the knowledge base for relevant information", function = ::searchKnowledge ) val chat = Chat.fromPath( modelPath = "./model.gguf", systemPrompt = "You are a customer service assistant. Use the search_knowledge tool to find relevant information before answering.", templateVariables = mapOf("enable_thinking" to false), tools = listOf(searchTool) ) val response = chat.ask("What is your return policy?").completed() println(response) } ``` -------------------------------- ### Get Completed Response Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/chat.md Retrieve the entire LLM response after generation is complete. This method blocks until the full response is available. ```python full_response: str = response.completed() ``` -------------------------------- ### Precompute Document Embeddings Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/embeddings-and-rag.md Precomputes embeddings for all documents in the knowledge base using an encoder. This is a one-time setup for efficient retrieval. ```python doc_embeddings = [encoder.encode(doc) for doc in knowledge_base] ``` -------------------------------- ### Custom Grammar Sampler (GBNF Syntax) Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-swift/sampling.md Use custom grammars for complex output constraints, accepting GBNF syntax which NobodyWho converts to Lark. This example defines file structure with records and fields. ```swift let sampler = SamplerPresets.constrainWithGrammar(""" file ::= record (newline record)* newline? record ::= field ("," field)* field ::= /[^,\"\n\r]/ newline ::= "\r\n" | "\n" """) ``` -------------------------------- ### Initialize Chat Model with Multiple Custom Tools and Template Variables Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/tool-calling.md Initialize the Chat model with a set of custom file system tools and configure template variables. This setup enables complex LLM interactions involving file operations. ```dart final chat = await nobodywho.Chat.fromPath( modelPath: './model.gguf', tools: [getCurrentDirTool, listFilesTool, getFileSizeTool], templateVariables: {"enable_thinking": false} ); final response = await chat.ask('What is the biggest file in my current directory?').completed(); print(response); // The largest file in your current directory is `model.gguf`. ``` -------------------------------- ### Enter Development Shell with Nix Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/CONTRIBUTING.md Use Nix to enter a development shell with the necessary tools like rustup and libclang pre-configured. Ensure flakes are enabled in your Nix configuration. ```bash nix develop ``` -------------------------------- ### Resolving uniffi-bindgen-react-native Path Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/react-native/android/CMakeLists.txt Determines the installation path of the 'uniffi-bindgen-react-native' package using Node.js. This is crucial for locating native header files. ```cmake execute_process( COMMAND node -p "require.resolve('uniffi-bindgen-react-native/package.json')" OUTPUT_VARIABLE UNIFFI_BINDGEN_PATH OUTPUT_STRIP_TRAILING_WHITESPACE ) # Get the directory; get_filename_component and cmake_path will normalize # paths with Windows path separators. get_filename_component(UNIFFI_BINDGEN_PATH "${UNIFFI_BINDGEN_PATH}" DIRECTORY) ``` -------------------------------- ### Get Chat History Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/chat.md Retrieve the current list of messages in the chat history. Each message is a dictionary containing role, content, and assets. ```python msgs: list[dict] = chat.get_chat_history() print(msgs[0]["content"]) # "Is water wet?" ``` -------------------------------- ### Modify and Get Template Variables Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/chat.md Update template variables on an existing chat using `setTemplateVariable()` and retrieve all current variables with `getTemplateVariables()`. ```kotlin chat.setTemplateVariable("enable_thinking", false) val variables = chat.getTemplateVariables() ``` -------------------------------- ### Build Custom Sampler with DSL Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/sampling.md Utilize the buildSampler DSL for fine-grained control over sampling parameters and terminal steps. This allows for complex sampler chaining and tuning. ```kotlin import ai.nobodywho.buildSampler val sampler = buildSampler { topK(40) temperature(0.8f) minP(0.05f) dist() } val chat = Chat.fromPath( modelPath = "./model.gguf", sampler = sampler ) ``` -------------------------------- ### Build Swift xcframework Locally Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/swift/DEVELOPMENT.md This command is used for local development to build the xcframework. The Package.swift manifest references this local xcframework. ```bash swift/scripts/build-swift-xcframework.sh ``` -------------------------------- ### Get Chat History Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/chat.md Retrieve the entire conversation history from the `Chat` object using `getChatHistory()`. Access individual messages by their index. ```kotlin val msgs = chat.getChatHistory() println(msgs[0].content) // "Is water wet?" ``` -------------------------------- ### Get Complete Response Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/chat.md Call `completed()` on the `TokenStream` returned by `chat.ask()` to retrieve the entire response as a single string after generation is finished. ```kotlin val fullResponse = chat.ask("Is water wet?").completed() ``` -------------------------------- ### Build a Customer Service Assistant with RAG Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-python/embeddings-and-rag.md Demonstrates building a RAG system for a customer service assistant. It initializes a cross-encoder, defines a knowledge base, creates a tool to search the knowledge base, and sets up a chat model that uses the tool to answer questions. ```python from nobodywho import Chat, CrossEncoder, tool # Initialize the cross-encoder for document ranking crossencoder = CrossEncoder('./reranker-model.gguf') # Your knowledge base knowledge = [ "Our company offers a 30-day return policy for all products", "Free shipping is available on orders over $50", "Customer support is available via email and phone", "We accept credit cards, PayPal, and bank transfers", "Order tracking is available through your account dashboard" ] # Create a tool that searches the knowledge base @tool(description="Search the knowledge base for relevant information") def search_knowledge(query: str) -> str: # Rank all documents by relevance to the query ranked = crossencoder.rank_and_sort(query, knowledge) # Return top 3 most relevant documents top_docs = [doc for doc, score in ranked[:3]] return "\n".join(top_docs) # Create a chat with access to the knowledge base chat = Chat( './model.gguf', system_prompt="You are a customer service assistant. Use the search_knowledge tool to find relevant information from our policies before answering customer questions.", tools=[search_knowledge] ) # The chat will automatically search the knowledge base when needed response = chat.ask("What is your return policy?").completed() print(response) ``` -------------------------------- ### Complete Godot RAG Script Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/site/godot/rag.md The complete script for a Godot node implementing RAG functionality, including Reranker and ChatModel setup. ```gdscript extends NobodyWhoChat @onready var reranker = $("../Rerank") @onready var chat_model = $("../ChatModel") ``` -------------------------------- ### Constraining Output with Regex Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-swift/sampling.md Enforce a specific output format using regular expressions. This example forces the model to answer only with 'yes' or 'no'. ```swift // Force the model to answer with exactly "yes" or "no" let chat = try await Chat.fromPath( modelPath: "/path/to/model.gguf", sampler: SamplerPresets.constrainWithRegex("yes|no") ) let answer = try await chat.ask("Is the sky blue?").completed() ``` -------------------------------- ### Initialize and Use CrossEncoder for Reranking Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/embeddings-and-rag.md Demonstrates how to load a CrossEncoder model and rank documents based on a query. Ensure the reranking model is downloaded. ```dart import 'package:nobodywho/nobodywho.dart' as nobodywho; // Download a reranking model like bge-reranker-v2-m3-Q8_0.gguf final crossencoder = await nobodywho.CrossEncoder.fromPath(modelPath: './reranker-model.gguf'); final query = "How do I install Python packages?"; final documents = [ "Someone previously asked about Python packages", "Use pip install package-name to install Python packages", "Python packages are not included in the standard library" ]; // Get relevance scores for each document final scores = await crossencoder.rank(query: query, documents: documents); print(scores); // [0.23, 0.89, 0.45] - second doc scores highest ``` -------------------------------- ### Get Chat History Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-swift/chat.md Retrieves the current list of messages (user prompts and model responses) stored in the Chat object. This is an asynchronous operation. ```swift let msgs = try await chat.getChatHistory() print(msgs[0]) // The first message ``` -------------------------------- ### Run Full Eval Suite Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/evals/lm_eval_scripts/README.md Execute the complete set of lm-eval-harness benchmarks using the specified model. ```bash python main.py /path/to/model.gguf ``` -------------------------------- ### Build a RAG Customer Service Assistant Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/embeddings-and-rag.md Complete example of building a RAG system for a customer service assistant. It initializes a CrossEncoder, defines a knowledge base, creates a search tool, and sets up a chat model that uses the tool to answer questions. ```dart import 'package:nobodywho/nobodywho.dart' as nobodywho; Future main() async { // Initialize the cross-encoder for document ranking final crossencoder = await nobodywho.CrossEncoder.fromPath(modelPath: './reranker-model.gguf'); // Your knowledge base final knowledge = [ "Our company offers a 30-day return policy for all products", "Free shipping is available on orders over $50", "Customer support is available via email and phone", "We accept credit cards, PayPal, and bank transfers", "Order tracking is available through your account dashboard" ]; // Create a tool that searches the knowledge base final searchKnowledgeTool = nobodywho.Tool( function: ({required String query}) async { // Rank all documents by relevance to the query final ranked = await crossencoder.rankAndSort(query: query, documents: knowledge); // Return top 3 most relevant documents final topDocs = ranked.take(3).map((e) => e.$1).toList(); return topDocs.join("\n"); }, name: "search_knowledge", description: "Search the knowledge base for relevant information" ); // Create a chat with access to the knowledge base final chat = await nobodywho.Chat.fromPath( modelPath: './model.gguf', systemPrompt: "You are a customer service assistant. Use the search_knowledge tool to find relevant information from our policies before answering customer questions.", templateVariables: {"enable_thinking": false}, tools: [searchKnowledgeTool] ); // The chat will automatically search the knowledge base when needed final response = await chat.ask("What is your return policy?").completed(); print(response); } ``` -------------------------------- ### Initialize Chat with Tools Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/kotlin_versioned_docs/version-1.1.0/tool-calling.md To enable the LLM to use a tool, pass it when creating the Chat instance. NobodyWho automatically configures the sampler based on tool parameter types. ```kotlin val chat = Chat.fromPath( modelPath = "./model.gguf", tools = listOf(weatherTool) ) ``` -------------------------------- ### Compose Prompt with Interspersed Text and Media Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-swift/vision.md Construct a prompt where text descriptions are interspersed with image and audio files. This can sometimes lead to better performance depending on the model. ```swift let prompt = Prompt([ Prompt.text("Tell me what you see in the image."), Prompt.image("/path/to/dog.png"), Prompt.text("Also tell me what you hear in the audio."), Prompt.audio("/path/to/sound.mp3"), ]) ``` -------------------------------- ### Compose Multimodal Prompt Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/vision.md Create a prompt that includes text, an image, and audio for the multimodal model. The `completed()` function is used to get the final response. ```kotlin import ai.nobodywho.Prompt val response = chat.ask(Prompt( Prompt.Text("Tell me what you see in the image and what you hear in the audio."), Prompt.Image("./dog.png"), Prompt.Audio("./sound.mp3"), )).completed() println(response) // It's a dog! ``` -------------------------------- ### Send Prompt and Get Token Stream Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-kotlin/chat.md Use `chat.ask()` to send a message to the LLM and receive a `TokenStream` of the response. This allows for processing the response as it generates. ```kotlin val chat = Chat.fromPath(modelPath = "./model.gguf") val stream = chat.ask("Is water wet?") ``` -------------------------------- ### Build a RAG Customer Service Assistant Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-react-native/embeddings-and-rag.md This snippet demonstrates building a RAG system for a customer service assistant. It initializes a cross-encoder for reranking, defines a knowledge base, creates a tool to search this knowledge base, and sets up a chat model with system prompts and tools. Use this to ground LLM responses in your specific knowledge base. ```typescript import { Chat, Tool, CrossEncoder } from "react-native-nobodywho"; // Initialize the cross-encoder for document ranking const crossencoder = await CrossEncoder.fromPath({ modelPath: "/path/to/reranker-model.gguf", }); // Your knowledge base const knowledge = [ "Our company offers a 30-day return policy for all products", "Free shipping is available on orders over $50", "Customer support is available via email and phone", "We accept credit cards, PayPal, and bank transfers", "Order tracking is available through your account dashboard", ]; // Create a tool that searches the knowledge base const searchKnowledgeTool = new Tool({ name: "search_knowledge", description: "Search the knowledge base for relevant information", parameters: [ { name: "query", type: "string", description: "The search query" }, ], call: async (query: string) => { const ranked = await crossencoder.rankAndSort(query, knowledge); const topDocs = ranked .slice(0, 3) .map(([doc]) => doc); return topDocs.join("\n"); }, }); // Create a chat with access to the knowledge base const chat = await Chat.fromPath({ modelPath: "/path/to/model.gguf", systemPrompt: "You are a customer service assistant. Use the search_knowledge tool to find relevant information from our policies before answering customer questions.", tools: [searchKnowledgeTool], }); // The chat will automatically search the knowledge base when needed const response = await chat.ask("What is your return policy?").completed(); console.log(response); ``` -------------------------------- ### Initialize Chat Model with Pre-packaged Python and Bash Tools Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-flutter/tool-calling.md Create a Chat model instance that includes the pre-packaged Monty Python interpreter and Bash interpreter tools. This provides the LLM with capabilities for code execution and command-line operations. ```dart import 'package:nobodywho/nobodywho.dart' as nobodywho; final chat = await nobodywho.Chat.fromPath( modelPath: './model.gguf', tools: [nobodywho.Tool.python(), nobodywho.Tool.bash()], ); ``` -------------------------------- ### Initialize Embedding Model and Worker Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/docs/docs-godot/embeddings-and-rag.md In the _ready function, create and configure the embedding model, specifying its path. Add the model to the scene tree, link it to the script, connect the encoding finished signal, and start the worker. Finally, precompute all embeddings. ```gdscript func _ready(): # Create and configure the embedding model var embedding_model = NobodyWhoModel.new() embedding_model.model_path = "res://models/bge-small-en-v1.5-q8_0.gguf" get_parent().add_child(embedding_model) # Link to the embedding model self.model_node = embedding_model self.encoding_finished.connect(_on_encoding_finished) self.start_worker() # Pre-generate embeddings for all statement types precompute_all_embeddings() ``` -------------------------------- ### Run Kotlin Tests with Native Library Source: https://github.com/nobodywho-ooo/nobodywho/blob/main/nobodywho/kotlin/DEVELOPMENT.md Execute Kotlin tests, including integration tests, from the project root. Set environment variables for the native library directory and the test model path. ```bash nix develop .#android --command bash -c \ 'cd nobodywho/kotlin && \ export NOBODYWHO_LIB_DIR="$(cd ../target/debug && pwd)" && \ export TEST_MODEL=/path/to/model.gguf && \ ./gradlew test' ```