### Project Setup Script (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md This Bash script sets up the development environment for org-db-v3. It checks for Python 3.10+, installs the 'uv' package manager if not present, and installs Python dependencies using 'uv sync'. It provides instructions on how to start the server. ```bash #!/bin/bash set -e echo "Setting up org-db v3 development environment..." # Check Python version python3 --version | grep -q "Python 3.1[0-9]" || { echo "Error: Python 3.10+ required" exit 1 } # Install uv if not available if ! command -v uv &> /dev/null; then echo "Installing uv..." curl -LsSf https://astral.sh/uv/install.sh | sh fi # Install Python dependencies cd python uv sync echo "Setup complete!" echo "To start the server: cd python && uv run uvicorn org_db_server.main:app --reload --port 8765" ``` -------------------------------- ### Start Server Command (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md This command starts the org-db server using 'uv run' and 'uvicorn'. It specifies the application entry point (`org_db_server.main:app`), enables auto-reloading (`--reload`), and sets the port to 8765. This is typically run after the setup script. ```bash cd python uv run uvicorn org_db_server.main:app --reload --port 8765 ``` -------------------------------- ### Python Server: Starting the Server Source: https://context7.com/jkitchin/org-db-v3/llms.txt Instructions for starting the FastAPI backend server for org-db-v3. It covers using `uv` with `uvicorn` for development, running the server in the background with logging, and starting it directly from Emacs. Configuration can also be managed via environment variables for host, port, and linked file settings. ```bash # Start server with uv cd python uv run uvicorn org_db_server.main:app --reload --port 8765 # Start server in background uv run uvicorn org_db_server.main:app --port 8765 > /tmp/org-db-server.log 2>&1 & # Or start from Emacs M-x org-db-v3-start-server # Configuration via environment variables export ORG_DB_HOST=127.0.0.1 export ORG_DB_PORT=8765 export ORG_DB_ENABLE_LINKED_FILES=true export ORG_DB_MAX_LINKED_FILE_SIZE_MB=20 export ORG_DB_MAX_LINKED_FILE_CHUNKS=50 ``` -------------------------------- ### Setup org-db v3 Project Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md Commands to clone the org-db v3 repository and run the setup script. Assumes git and a shell environment are available. ```bash git clone https://github.com/yourusername/org-db-v3 cd org-db-v3 ./scripts/setup.sh ``` -------------------------------- ### Start org-db v3 Server from Emacs Source: https://github.com/jkitchin/org-db-v3/blob/main/readme.org Starts the org-db v3 backend server directly from Emacs using the provided command. ```emacs-lisp M-x org-db-v3-start-server ``` -------------------------------- ### Start Python Server Source: https://github.com/jkitchin/org-db-v3/blob/main/CONTRIBUTING.md Command to start the org-db v3 Python server using uvicorn. ```bash cd python uv run uvicorn org_db_server.main:app --host 127.0.0.1 --port 8765 ``` -------------------------------- ### Start, Stop, and Check org-db Server (Elisp) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md Provides functions to manage the org-db server process. It includes starting the server with specified host and port, stopping it, and checking its running status. Dependencies include 'org-db-v3'. ```elisp ;;; org-db-v3-server.el --- Server management -*- lexical-binding: t; -*- ;;; Commentary: ;; Functions to start, stop, and check the org-db server. ;;; Code: (require 'org-db-v3) (defcustom org-db-v3-python-command "uv" "Command to run Python (uv, python3, etc)." :type 'string :group 'org-db-v3) (defcustom org-db-v3-server-directory (expand-file-name "python" (file-name-directory (or load-file-name buffer-file-name))) "Directory containing the Python server code." :type 'directory :group 'org-db-v3) (defun org-db-v3-start-server () "Start the org-db server." (interactive) (if (org-db-v3-server-running-p) (message "org-db server already running") (let* ((default-directory org-db-v3-server-directory) (process-name "org-db-server") (buffer-name "*org-db-server*") (cmd (list org-db-v3-python-command "run" "uvicorn" "org_db_server.main:app" "--reload" "--host" org-db-v3-server-host "--port" (number-to-string org-db-v3-server-port)))) (setq org-db-v3-server-process (make-process :name process-name :buffer buffer-name :command cmd :sentinel #'org-db-v3-server-sentinel)) ;; Wait a bit for server to start (sleep-for 2) (if (org-db-v3-server-running-p) (message "org-db server started on %s:%d" org-db-v3-server-host org-db-v3-server-port) (error "Failed to start org-db server. Check *org-db-server* buffer"))))) (defun org-db-v3-stop-server () "Stop the org-db server." (interactive) (when (and org-db-v3-server-process (process-live-p org-db-v3-server-process)) (kill-process org-db-v3-server-process) (setq org-db-v3-server-process nil) (message "org-db server stopped"))) (defun org-db-v3-server-sentinel (process event) "Sentinel for server PROCESS EVENT." (when (string-match-p "\(finished\|exited\)" event) (message "org-db server process %s" event) (setq org-db-v3-server-process nil))) (defun org-db-v3-ensure-server () "Ensure server is running, start if needed." (unless (org-db-v3-server-running-p) (when org-db-v3-auto-start-server (org-db-v3-start-server)))) (provide 'org-db-v3-server) ;;; org-db-v3-server.el ends here ``` -------------------------------- ### Install Python Backend Dependencies Source: https://github.com/jkitchin/org-db-v3/blob/main/readme.org Installs all Python dependencies for the org-db v3 backend using uv. This includes FastAPI, Uvicorn, sentence-transformers, CLIP, and SQLite. ```bash cd python uv sync ``` -------------------------------- ### Start Org-DB v3 Server with Uvicorn (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/performance/chunking-comparison.md Command to start the org-db-v3 server using uvicorn, a Python ASGI server. This command is typically run after setting environment variables or configuring the application. ```bash # Start server cd python uv run uvicorn org_db_server.main:app --port 8765 ``` -------------------------------- ### Start org-db v3 Python Server Source: https://github.com/jkitchin/org-db-v3/blob/main/readme.org Starts the org-db v3 FastAPI server in the background. It runs uvicorn with hot-reloading and redirects output to a log file. ```bash cd python uv run uvicorn org_db_server.main:app --reload --port 8765 > /tmp/org-db-server.log 2>&1 & ``` -------------------------------- ### Open Org DB v3 Web Interface Source: https://github.com/jkitchin/org-db-v3/blob/main/readme.org Opens the org-db-v3 web interface, providing access to statistics, API documentation, and a getting started guide. This can also be accessed directly via a browser. ```emacs-lisp M-x org-db-v3-open-web-interface ;; or press 'W' in the menu ;; or visit http://127.0.0.1:8765 in your browser ``` -------------------------------- ### Org Link Examples - Org Mode Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/architecture/supported-formats.md Examples of how to create links to office documents within Org mode files. These links specify the file path and an optional visible name for the link. ```org [[file:~/Documents/report.pdf][Q3 Report]] [[file:../presentations/slides.pptx][Conference Talk]] [[file:./data/analysis.xlsx][Sales Data]] ``` -------------------------------- ### User Migration Steps (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/architecture/multi-database.md Provides instructions for users migrating to the new multi-database architecture, recommending a clean start by backing up the old database and letting the server recreate the new structure. Assumes the server is then restarted. ```bash # Backup old database mv ~/org-db/org-db-v3.db ~/org-db/org-db-v3.db.old # Start server (creates new 3-DB structure) # Reindex from Emacs ``` -------------------------------- ### FastAPI Server Setup (Python) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md Sets up the main FastAPI application for the org-db server. It includes configuring CORS middleware to allow cross-origin requests, typically from Emacs. The server is initialized with a title and version. ```python from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from org_db_server.api import indexing app = FastAPI(title="org-db Server", version="0.1.0") # Allow Emacs to connect from localhost app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) ``` -------------------------------- ### Git Commit Command Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md This command stages all changes in the current directory using `git add .` and then commits them with the message 'feat: initial project structure and basic server'. This is used to save the initial project setup. ```bash git add . git commit -m "feat: initial project structure and basic server" ``` -------------------------------- ### Fortran Program Structure: Hello World Source: https://github.com/jkitchin/org-db-v3/blob/main/examples/programming.org Demonstrates the basic structure of a Fortran program, including the 'program' keyword, 'implicit none' for explicit typing, and the 'print' statement for output. This is a fundamental example for beginners. ```fortran program hello_world implicit none print *, "Hello, World!" end program hello_world ``` -------------------------------- ### Monitor Database Size (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/performance/linked-files-optimization.md A bash command to display the disk usage of the org-db-v3 database file, helping to track its size. ```bash # Database size du -h ~/Dropbox/emacs/cache/org-db-v3/org-db-v3.db ``` -------------------------------- ### Calculate Chunks Per File Ratio (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/performance/linked-files-optimization.md Fetches API stats and calculates the ratio of chunks to files, providing insight into the chunking strategy's density. ```bash # Chunks per file ratio curl -s http://127.0.0.1:8765/api/stats/ | jq '.chunks_count / .files_count' ``` -------------------------------- ### Monitor Database Size During Indexing (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/performance/linked-files-optimization.md This bash command uses `watch` to continuously monitor the size of the org-db-v3 database file in human-readable format at 5-second intervals. This is useful for observing the impact of indexing. ```bash watch -n 5 'du -h ~/Dropbox/emacs/cache/org-db-v3/org-db-v3.db' ``` -------------------------------- ### Compare Linked vs Org Files Count (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/performance/linked-files-optimization.md Fetches API stats and displays the counts of linked files versus org files, useful for understanding the scope of indexed content. ```bash # Linked files vs org files curl -s http://127.0.0.1:8765/api/stats/ | jq '{org: .org_files_count, linked: .linked_files_count}' ``` -------------------------------- ### Flask API Endpoint Source: https://github.com/jkitchin/org-db-v3/blob/main/examples/programming.org A simple Flask application example in Python that defines an API endpoint. It uses the `jsonify` function to return a JSON response for a GET request to '/api/hello/'. ```python from flask import Flask, jsonify app = Flask(__name__) @app.route('/api/hello/') def hello(name): return jsonify({'message': f'Hello, {name}!'}) if __name__ == '__main__': app.run(debug=True) ``` -------------------------------- ### Python Decorator for Timing Functions Source: https://github.com/jkitchin/org-db-v3/blob/main/examples/programming.org Provides an example of a Python decorator used to measure the execution time of a function. It includes a wrapper function that records start and end times and prints the duration. ```python def timing_decorator(func): """Measure function execution time.""" import time def wrapper(*args, **kwargs): start = time.time() result = func(*args, **kwargs) end = time.time() print(f"{func.__name__} took {end - start:.4f} seconds") return result return wrapper @timing_decorator def slow_function(): time.sleep(1) return "Done" ``` -------------------------------- ### Emacs Lisp: Buffer Manipulation Source: https://github.com/jkitchin/org-db-v3/blob/main/examples/programming.org Provides examples of common buffer operations in Emacs Lisp, such as getting the current buffer, switching to another buffer, creating temporary buffers, and using `save-excursion` to preserve point position. ```emacs-lisp ;; Get current buffer (current-buffer) ;; Switch buffers (with-current-buffer "*scratch*" (insert "Hello from scratch buffer")) ;; Create new buffer (with-temp-buffer (insert "Temporary content") (buffer-string)) ;; Save excursion (preserve point position) (save-excursion (goto-char (point-min)) (insert "At beginning")) ``` -------------------------------- ### Clone and Set Up Python Environment Source: https://github.com/jkitchin/org-db-v3/blob/main/CONTRIBUTING.md Commands to clone the repository and set up the Python development environment using uv. ```bash git clone https://github.com/yourusername/org-db-v3.git cd org-db-v3 cd python uv sync ``` -------------------------------- ### Conventional Commits Examples Source: https://github.com/jkitchin/org-db-v3/blob/main/CONTRIBUTING.md Examples of commit messages following the Conventional Commits specification. ```git feat: add property search functionality fix: resolve vector search performance issue docs: update installation instructions perf: optimize headline search for 100K+ headlines ``` -------------------------------- ### Find Org Files Linking to PDFs about Revenue (Elisp) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/architecture/linked-files.md This Elisp example demonstrates a search query that identifies org files linking to PDFs containing specific keywords like 'revenue trends'. The results are mixed, including direct org content and linked PDF content, with clear visual cues to distinguish between them. ```elisp (org-db-v3-semantic-search "revenue trends") → Returns mixed results: - Direct org content matching "revenue" - Content from linked PDFs matching "revenue" → User can distinguish by [📄] icon or chunk_type ``` -------------------------------- ### Database Class Initialization (Python) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/architecture/multi-database.md Compares the old monolithic Database class initialization with the new one that handles three separate database connections for main, semantic, and image databases. Requires the ' libsql' library. ```python class Database: def __init__(self, db_path): self.conn = libsql.connect(str(db_path)) ``` ```python class Database: def __init__(self, main_path, semantic_path, image_path): self.main_conn = libsql.connect(str(main_path)) self.semantic_conn = libsql.connect(str(semantic_path)) self.image_conn = libsql.connect(str(image_path)) ``` -------------------------------- ### Sample Org Fixture (Org Mode) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md This is a sample Org mode file used as a test fixture. It includes a title, author, date, headings with properties (ID, CUSTOM, EMAIL), paragraphs, links, and tags. This file serves as input data for testing Org database functionalities. ```org #+TITLE: Sample Org File #+AUTHOR: Test Author #+DATE: 2025-10-11 * TODO First Heading :PROPERTIES: :ID: test-id-001 :CUSTOM: value :END: This is a paragraph with some content about machine learning and neural networks. ** Second Level Heading Some more content here with a [[https://example.com][link]]. * DONE Completed Task CLOSED: [2025-10-11 Sat 10:00] :PROPERTIES: :EMAIL: test@example.com :END: Final content with #hashtag and @mention. ``` -------------------------------- ### Python Hello World Function Source: https://github.com/jkitchin/org-db-v3/blob/main/tests/fixtures/sample.org A simple Python function that prints 'Hello, world!' to the console. This function has no external dependencies and its output is printed directly. ```python def hello(): print("Hello, world!") ``` -------------------------------- ### GET /health Source: https://github.com/jkitchin/org-db-v3/blob/main/python/org_db_server/templates/homepage.html Checks the health status and uptime of the server. ```APIDOC ## GET /health ### Description Server health check - returns status and uptime. ### Method GET ### Endpoint /health ### Parameters #### Query Parameters None ### Request Example None ### Response #### Success Response (200) - **status** (string) - The health status of the server. - **uptime** (string) - The uptime of the server. #### Response Example ```json { "status": "ok", "uptime": "24h 30m 15s" } ``` ``` -------------------------------- ### Add Search UI Files to Git Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-11-org-db-v3-implementation.md This bash command stages all files within the `elisp/` directory for the next Git commit. This includes the newly created `org-db-v3-search.el` and `org-db-v3-ui.el` files, as well as any modifications to `org-db-v3.el`. ```bash git add elisp/ ``` -------------------------------- ### Emacs Lisp: Creating Interactive Commands Source: https://github.com/jkitchin/org-db-v3/blob/main/examples/programming.org Shows how to define interactive Emacs Lisp functions using the `interactive` form, allowing them to be called directly by the user. Examples include inserting the date and counting words in a region, with one example bound to a key sequence. ```emacs-lisp (defun my-insert-date () "Insert current date at point." (interactive) (insert (format-time-string "%Y-%m-%d"))) (defun my-count-words (start end) "Count words in region from START to END." (interactive "r") (let ((count (how-many "\\w+" start end))) (message "Word count: %d" count))) ;; Bind to key (global-set-key (kbd "C-c d") #'my-insert-date) ``` -------------------------------- ### Configure Emacs for org-db v3 Source: https://github.com/jkitchin/org-db-v3/blob/main/readme.org Adds the org-db v3 elisp directory to the load path, requires the package, and optionally binds the menu to a key and auto-starts the server. It also shows how to enable gptel integration. ```emacs-lisp (add-to-list 'load-path "/path/to/org-db-v3/elisp") (require 'org-db-v3) ;; Bind the menu to a convenient key (global-set-key (kbd "H-v") 'org-db-menu) ;; Auto-start server when Emacs starts (optional) (setq org-db-v3-auto-start-server t) ;; Optional: Enable gptel integration for AI-powered search (when (featurep 'gptel) (require 'org-db-v3-gptel-tools) (org-db-v3-gptel-register-tools)) ``` -------------------------------- ### Get Agenda Items Source: https://github.com/jkitchin/org-db-v3/blob/main/python/README.md Retrieves agenda items from the indexed org-mode files. ```APIDOC ## GET /api/agenda ### Description Retrieves agenda items (tasks, scheduled events) from the indexed org-mode files. ### Method GET ### Endpoint /api/agenda ### Parameters #### Query Parameters - **date** (string) - Optional - Filter agenda items for a specific date (YYYY-MM-DD). - **from_date** (string) - Optional - Filter agenda items from this date (YYYY-MM-DD). - **to_date** (string) - Optional - Filter agenda items up to this date (YYYY-MM-DD). ### Request Example ``` GET /api/agenda?date=2023-10-27 ``` ``` GET /api/agenda?from_date=2023-10-27&to_date=2023-10-31 ``` ### Response #### Success Response (200) - **agenda_items** (array) - A list of agenda items, each containing details like headline, file, date, and type. #### Response Example ```json { "agenda_items": [ { "file": "/path/to/tasks.org", "headline": "Prepare presentation slides", "date": "2023-10-27", "type": "SCHEDULED" }, { "file": "/path/to/calendar.org", "headline": "Team Meeting", "date": "2023-10-27", "type": "TIMED" } ] } ``` ``` -------------------------------- ### Example API Search Request (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/features/image-search.md Demonstrates a `curl` command for performing a POST request to the image search API. This snippet illustrates the input format for a search query and shows the performance difference before and after the optimization, highlighting the reduction in response time. ```bash $ time curl -X POST .../api/search/images -d '{"query": "cat", "limit": 10}' real 0m4.362s ``` ```bash $ time curl -X POST .../api/search/images -d '{"query": "cat", "limit": 10}' real 0m0.343s ``` -------------------------------- ### GET /api/stats/files Source: https://github.com/jkitchin/org-db-v3/blob/main/python/org_db_server/templates/homepage.html Retrieves a list of all indexed files along with their last indexed timestamps. ```APIDOC ## GET /api/stats/files ### Description Get list of all indexed files with timestamps. ### Method GET ### Endpoint /api/stats/files ### Parameters #### Query Parameters None ### Request Example None ### Response #### Success Response (200) - **files** (array) - A list of indexed files. - **filename** (string) - The name of the indexed file. - **timestamp** (string) - The last indexed timestamp for the file. #### Response Example ```json { "files": [ {"filename": "document1.org", "timestamp": "2023-10-27T09:00:00Z"}, {"filename": "notes.org", "timestamp": "2023-10-26T15:30:00Z"} ] } ``` ``` -------------------------------- ### GET /docs Source: https://github.com/jkitchin/org-db-v3/blob/main/python/org_db_server/templates/homepage.html Provides interactive API documentation using Swagger UI, allowing direct testing of endpoints in the browser. ```APIDOC ## GET /docs ### Description Interactive API documentation with Swagger UI - test endpoints directly in your browser. ### Method GET ### Endpoint /docs ### Parameters None ### Request Example None ### Response #### Success Response (200) - HTML content rendering the Swagger UI interface. #### Response Example (Browser renders the interactive documentation) ``` -------------------------------- ### GET /api/stats/ Source: https://github.com/jkitchin/org-db-v3/blob/main/python/org_db_server/templates/homepage.html Retrieves comprehensive statistics about the indexed databases, including file counts, embeddings, links, and database size. ```APIDOC ## GET /api/stats/ ### Description Get comprehensive database statistics including file counts, embeddings, links, and database size. ### Method GET ### Endpoint /api/stats/ ### Parameters #### Query Parameters None ### Request Example None ### Response #### Success Response (200) - **file_count** (integer) - The total number of indexed files. - **embedding_count** (integer) - The total number of embeddings. - **link_count** (integer) - The total number of links found. - **database_size** (string) - The total size of the databases. #### Response Example ```json { "file_count": 1500, "embedding_count": 50000, "link_count": 10000, "database_size": "1.2 GB" } ``` ``` -------------------------------- ### Find Content from Linked PDFs (Elisp) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/architecture/linked-files.md This Elisp example shows how to use `org-db-v3-semantic-search` to find content specifically from linked PDF files. The search query returns chunks from the PDF, indicating their original location in the org file and providing options to open either the PDF or the org file. ```elisp (org-db-v3-semantic-search "Q3 financial results") → Returns chunks from report.pdf → Each chunk shows: inbox.org:45 (where link exists) → Can open: report.pdf or inbox.org ``` -------------------------------- ### GET /api/files Source: https://context7.com/jkitchin/org-db-v3/llms.txt Retrieves a list of all files that have been indexed by the system, including the timestamp of their last indexing. ```APIDOC ## GET /api/files ### Description Returns a list of all files currently indexed by the org-db-v3 system. Each entry includes the filename and the timestamp when it was last indexed. ### Method GET ### Endpoint /api/files ### Parameters None ### Request Example ```bash curl http://127.0.0.1:8765/api/files ``` ### Response #### Success Response (200) - **files** (array) - An array of objects, where each object contains the `filename` and `indexed_at` timestamp for an indexed file. #### Response Example ```json { "files": [ {"filename": "/home/user/notes/work.org", "indexed_at": "2025-01-15T10:30:00"}, {"filename": "/home/user/notes/personal.org", "indexed_at": "2025-01-14T15:20:00"} ] } ``` ``` -------------------------------- ### Database File Structure (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/architecture/multi-database.md Illustrates the file organization of the new multi-database architecture, showing the separate .db files for main, semantic, and image data. Compares the total size to the previous monolithic database. ```bash # ~/org-db/ # ├── org-db-v3.db # Main: 200 MB (metadata + FTS5) # ├── org-db-v3-semantic.db # Semantic: 40 MB (chunks + embeddings) # └── org-db-v3-images.db # Images: 15 MB (images + CLIP) # ------------------------------------------------------------------- # Total: ~255 MB vs 29.5 GB previously! ``` -------------------------------- ### GET /api/linked-files/all Source: https://context7.com/jkitchin/org-db-v3/llms.txt Retrieves a list of all linked files that have been indexed by the system, including their file path, type, and size. ```APIDOC ## GET /api/linked-files/all ### Description Returns a list of all linked files (e.g., PDFs, DOCX) that have been indexed by the system. This endpoint provides metadata for each indexed linked file. ### Method GET ### Endpoint /api/linked-files/all ### Parameters None ### Request Example ```bash curl http://127.0.0.1:8765/api/linked-files/all ``` ### Response #### Success Response (200) - **linked_files** (array) - An array of objects, where each object contains details about an indexed linked file, including its ID, file path, file type, and file size. #### Response Example ```json { "linked_files": [ { "id": 1, "file_path": "/home/user/documents/paper.pdf", "file_type": "pdf", "file_size": 2500000 } ] } ``` ``` -------------------------------- ### Emacs Configuration for Org-db v3 Source: https://github.com/jkitchin/org-db-v3/blob/main/python/org_db_server/templates/homepage.html This snippet shows how to add org-db-v3 to your Emacs configuration. It includes loading the library, binding the org-db menu to a key, and optionally enabling auto-start of the server. ```emacs-lisp (add-to-list 'load-path "/path/to/org-db-v3/elisp") (require 'org-db-v3) ;; Bind the menu to a convenient key (global-set-key (kbd "H-v") 'org-db-menu) ;; Auto-start server when Emacs starts (optional) (setq org-db-v3-auto-start-server t) ``` -------------------------------- ### Test Emacs Lisp Scope Helper Functions Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/development/2025-10-12-search-scope.md Provides example Emacs Lisp code snippets for testing the `org-db-v3--scope-description` and `org-db-v3--scope-to-params` helper functions. These tests demonstrate how the functions behave with different scope settings, ensuring correct functionality before integration. ```emacs-lisp ;; Test scope description (let ((org-db-v3-search-scope '(directory . "/home/user/projects/"))) (org-db-v3--scope-description)) ;; Should return: "Directory: projects" ;; Test scope-to-params (let ((org-db-v3-search-scope '(project . "/home/user/work/"))) (org-db-v3--scope-to-params)) ;; Should return: (:filename_pattern "/home/user/work/%") (let ((org-db-v3-search-scope '(tag . "important"))) (org-db-v3--scope-to-params)) ;; Should return: (:keyword "important") ``` -------------------------------- ### Python Search Headlines Function Source: https://github.com/jkitchin/org-db-v3/blob/main/CONTRIBUTING.md Example Python function for searching headlines, including type hints and a Google-style docstring. ```python from typing import List, Optional def search_headlines( query: str, limit: Optional[int] = None ) -> List[dict]: """Search for headlines matching query. Args: query: Search query string limit: Maximum number of results (None for unlimited) Returns: List of headline dictionaries with title, filename, and position """ # Implementation pass ``` -------------------------------- ### Health Check API Endpoint Source: https://github.com/jkitchin/org-db-v3/blob/main/readme.org A simple GET request to the /health endpoint to check if the org-db-v3 server is running and responsive. ```http GET /health ``` -------------------------------- ### GET /api/stats/ Source: https://context7.com/jkitchin/org-db-v3/llms.txt Fetches comprehensive statistics about the main, semantic, and image databases, including counts of files, headlines, links, and database sizes. ```APIDOC ## GET /api/stats/ ### Description Retrieves detailed statistics for all databases managed by org-db-v3, including the main database, semantic database, and image database. This provides insights into the size and content of your indexed data. ### Method GET ### Endpoint /api/stats/ ### Parameters None ### Request Example ```bash curl http://127.0.0.1:8765/api/stats/ ``` ### Response #### Success Response (200) - **files_count** (integer) - Total number of indexed files. - **org_files_count** (integer) - Number of indexed org files. - **linked_files_count** (integer) - Number of indexed linked files (e.g., PDFs, DOCX). - **headlines_count** (integer) - Total number of headlines across all org files. - **links_count** (integer) - Total number of links found in org files. - **chunks_count** (integer) - Number of text chunks indexed for semantic search. - **embeddings_count** (integer) - Number of embeddings generated for text chunks. - **images_count** (integer) - Number of images indexed. - **image_embeddings_count** (integer) - Number of embeddings generated for images. - **main_db_path** (string) - File path to the main database. - **main_db_size_mb** (float) - Size of the main database in megabytes. - **semantic_db_path** (string) - File path to the semantic database. - **semantic_db_size_mb** (float) - Size of the semantic database in megabytes. - **image_db_path** (string) - File path to the image database. - **image_db_size_mb** (float) - Size of the image database in megabytes. - **total_db_size_mb** (float) - Total size of all databases in megabytes. - **recent_files** (array) - A list of recently indexed files with their indexed timestamps. #### Response Example ```json { "files_count": 150, "org_files_count": 150, "linked_files_count": 25, "headlines_count": 5420, "links_count": 1200, "chunks_count": 8500, "embeddings_count": 8500, "images_count": 340, "image_embeddings_count": 340, "main_db_path": "/home/user/org-db/org-db-v3.db", "main_db_size_mb": 12.5, "semantic_db_path": "/home/user/org-db/org-db-v3-semantic.db", "semantic_db_size_mb": 85.2, "image_db_path": "/home/user/org-db/org-db-v3-images.db", "image_db_size_mb": 45.0, "total_db_size_mb": 142.7, "recent_files": [ {"filename": "/home/user/notes/today.org", "indexed_at": "2025-01-15T10:30:00"} ] } ``` ``` -------------------------------- ### Load gptel Tools and Register (Emacs Lisp) Source: https://github.com/jkitchin/org-db-v3/blob/main/readme.org These Emacs Lisp commands load the `org-db-v3-gptel-tools.el` file and register the associated tools. This is crucial for resolving 'void-variable org-db-v3-server-url' errors when using gptel integration with org-db-v3. ```emacs-lisp (load-file "/path/to/org-db-v3/elisp/org-db-v3-gptel-tools.el") (org-db-v3-gptel-register-tools) ``` -------------------------------- ### Emacs Lisp: Defining Functions Source: https://github.com/jkitchin/org-db-v3/blob/main/examples/programming.org Shows how to define functions in Emacs Lisp using `defun`. Examples include basic functions, functions with optional arguments using `&optional`, and functions accepting a variable number of arguments using `&rest`. ```emacs-lisp ;; Basic function (defun greet (name) "Greet NAME with a friendly message." (message "Hello, %s!" name)) ;; Function with optional arguments (defun greet-with-title (&optional title name) "Greet NAME with TITLE." (let ((greeting (if title (format "%s %s" title name) name))) (message "Hello, %s!" greeting))) ;; Function with rest arguments (defun sum (&rest numbers) "Sum all NUMBERS." (apply #'+ numbers)) ``` -------------------------------- ### Emacs Lisp Search Headlines Function Source: https://github.com/jkitchin/org-db-v3/blob/main/CONTRIBUTING.md Example Emacs Lisp function for searching headlines, following project naming conventions and including a docstring. ```elisp (defun org-db-v3-search-headlines () "Search all headlines and jump to selection. You can filter candidates dynamically using completing-read." (interactive) ;; Implementation ) ``` -------------------------------- ### Test Semantic Search with Query (Emacs Lisp) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/performance/chunking-comparison.md Executes a semantic search query for 'machine learning' and provides instructions on how to evaluate the results, including relevance, context usefulness, and the number of results. ```elisp ;; Query you know should match M-x org-db-v3-semantic-search RET machine learning RET ;; Check: ;; 1. Does it find what you expect? ;; 2. Is the context shown useful? ;; 3. Are there too many/few results? ``` -------------------------------- ### Fetch API Stats (Bash) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/performance/linked-files-optimization.md Uses curl to fetch detailed statistics from the org-db-v3 server API and formats the output using jq for readability. ```bash # Detailed stats curl -s http://127.0.0.1:8765/api/stats/ | jq ``` -------------------------------- ### Search Across All Content (Org + Linked) (SQL) Source: https://github.com/jkitchin/org-db-v3/blob/main/docs/architecture/linked-files.md This SQL query demonstrates how to search across all content, including both regular org content and content from linked files. It joins `chunks`, `files`, and `linked_files` tables, and uses a subquery on the `embeddings` table for the semantic search condition. No changes are needed to the existing query structure. ```sql -- No change needed! Works automatically SELECT c.chunk_text, c.begin_line, f.filename, lf.file_path as linked_file FROM chunks c JOIN files f ON c.filename_id = f.id LEFT JOIN linked_files lf ON c.linked_file_id = lf.id WHERE c.id IN (SELECT chunk_id FROM embeddings WHERE ...); ```