### Configure Build Environment Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Examples of customizing the build process by setting environment variables for compilers and flags. ```bash export CC=clang export CFLAGS="-g -O2 -DCUSTOM_FLAG" make ``` -------------------------------- ### Verify Installation and Load Extension Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Steps to confirm the shared library exists and verify it loads correctly within the SQLite shell. ```bash ls -la build/libgraph.so sqlite3 :memory: ".load ./build/libgraph.so\nCREATE VIRTUAL TABLE graph USING graph();\nSELECT 'Extension loaded successfully' as status;" ``` -------------------------------- ### Platform-Specific Dependency Installation Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Commands to install necessary build tools and dependencies on various operating systems. ```bash # Ubuntu/Debian sudo apt-get install build-essential libsqlite3-dev # CentOS/RHEL sudo dnf install gcc make sqlite-devel # macOS xcode-select --install brew install sqlite3 ``` -------------------------------- ### Build Extension from Source Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Standard workflow for cloning the repository and compiling the extension using the provided Makefile. ```bash git clone https://github.com/agentflare-ai/sqlite-graph.git cd sqlite-graph make ``` -------------------------------- ### Troubleshoot sqlite-graph Installation Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Common commands to diagnose missing files, permission issues, and compilation errors. ```bash # Check file existence and permissions ls -la build/libgraph.so chmod +x build/libgraph.so # Check dependencies ldd build/libgraph.so # Check compiler and version gcc --version sqlite3 --version ``` -------------------------------- ### Install and Load sqlite-graph Extension Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Methods for loading the sqlite-graph extension into SQLite, including local path loading, system-wide installation, and Python package configuration. ```python import sqlite3 conn = sqlite3.connect(":memory:") conn.enable_load_extension(True) conn.load_extension("./build/libgraph.so") ``` ```bash # System installation sudo cp build/libgraph.so /usr/local/lib/ conn.load_extension("libgraph") ``` ```bash # Python package setup cat > setup.py << 'EOF' from setuptools import setup, Extension setup( name="sqlite-graph", version="1.0.0", ext_modules=[Extension("sqlite_graph", ["build/libgraph.so"], include_dirs=["include/"])] ) EOF pip install -e . ``` -------------------------------- ### Verify sqlite-graph Functionality Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Scripts to validate the installation through command-line interface or Python integration tests. ```bash sqlite3 :memory: ".load ./build/libgraph.so\nCREATE VIRTUAL TABLE graph USING graph();\nINSERT INTO graph (command) VALUES ('CREATE (p:Person {name: \"Alice\"})');\nSELECT * FROM graph WHERE command = 'MATCH (p:Person) RETURN p.name';" ``` ```python import sqlite3 conn = sqlite3.connect(":memory:") conn.enable_load_extension(True) conn.load_extension("./build/libgraph.so") conn.execute("CREATE VIRTUAL TABLE graph USING graph()") result = conn.execute("SELECT * FROM graph WHERE command = 'MATCH (p:Person) RETURN p.name'").fetchall() ``` -------------------------------- ### Optimize sqlite-graph Performance Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Techniques for improving performance through compiler flags and SQLite runtime pragmas. ```bash # Build with maximum optimization export CFLAGS="-O3 -DNDEBUG -march=native -flto" make clean && make ``` ```bash # Runtime SQLite optimization PRAGMA cache_size = 10000; PRAGMA synchronous = OFF; PRAGMA journal_mode = MEMORY; ``` -------------------------------- ### Download and Verify Pre-built Binaries Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/INSTALL.md Commands to download the latest SQLite Graph shared library and verify its integrity using SHA256 checksums. ```bash wget https://github.com/agentflare-ai/sqlite-graph/releases/latest/download/libgraph.so wget https://github.com/agentflare-ai/sqlite-graph/releases/latest/download/checksums.txt sha256sum -c checksums.txt ``` -------------------------------- ### Badge URL Examples Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TESTING_PLAN.md Provides example URLs for GitHub release and build status badges. These URLs can be used to monitor the project's release versions and CI/CD pipeline status. ```text https://img.shields.io/github/v/release/agentflare-ai/sqlite-graph?include_prereleases https://img.shields.io/github/actions/workflow/status/agentflare-ai/sqlite-graph/ci.yml?branch=main ``` -------------------------------- ### Registering Custom SQL Functions in C Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/ARCHITECTURE.md Shows a C code example for registering a new SQL function, `shortest_path`, within the SQLite database context. This function is intended to implement a graph algorithm. The example uses `sqlite3_create_function` to bind the C function `shortestPathFunc` to the SQL name. ```c void graphRegisterAlgorithms(sqlite3 *db) { sqlite3_create_function(db, "shortest_path", 2, SQLITE_UTF8 | SQLITE_DETERMINISTIC, NULL, shortestPathFunc, NULL, NULL); } ``` -------------------------------- ### Install sqlite-graph via CLI Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/README.md Instructions for obtaining the sqlite-graph extension, either by downloading a pre-built binary for Linux or by compiling the source code using make. ```bash wget https://github.com/agentflare-ai/sqlite-graph/releases/latest/download/libgraph.so ``` ```bash git clone https://github.com/agentflare-ai/sqlite-graph.git cd sqlite-graph make ``` -------------------------------- ### Unity Unit Testing for Graph Operations Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Demonstrates how to write unit tests for the graph extension using the Unity framework, including setup, teardown, and assertion logic. ```c #include "unity.h" #include "graph.h" void setUp(void) {} void tearDown(void) {} void test_graphAddNode_success(void) { GraphVtab *pVtab = createTestGraph(); int rc = graphAddNode(pVtab, 1, "{\"name\": \"Alice\"}"); TEST_ASSERT_EQUAL(SQLITE_OK, rc); GraphNode *pNode = graphFindNode(pVtab, 1); TEST_ASSERT_NOT_NULL(pNode); TEST_ASSERT_EQUAL(1, pNode->iNodeId); destroyTestGraph(pVtab); } int main(void) { UNITY_BEGIN(); RUN_TEST(test_graphAddNode_success); return UNITY_END(); } ``` -------------------------------- ### Parallel Query Patterns in SQL Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE_TUNING_GUIDE.md Shows examples of executing parallel graph algorithms and pattern matching queries. The `WITH parallel_iterations` clause and `graph_parallel_match` function enable parallel execution. ```sql -- Parallel graph algorithms SELECT * FROM graph_pagerank('my_graph') WITH parallel_iterations = 4; -- Parallel pattern matching SELECT * FROM graph_parallel_match('my_graph', 'MATCH (p:Person)-[:KNOWS]->(f) RETURN p, f' ); ``` -------------------------------- ### Social Network Analysis Example with SQLite Graph Source: https://context7.com/agentflare-ai/sqlite-graph/llms.txt A comprehensive Python example demonstrating the creation and analysis of a social network graph. It includes adding nodes (users) and edges (friendships) with properties, and then performing network analysis metrics like node count, edge count, connectivity, density, and degree centrality. ```python import sqlite3 import json conn = sqlite3.connect(":memory:") conn.enable_load_extension(True) conn.load_extension("./build/libgraph.so") conn.execute("CREATE VIRTUAL TABLE graph USING graph()") # Create users users = [ (1, {"name": "Alice", "age": 28, "location": "NYC"}), (2, {"name": "Bob", "age": 32, "location": "SF"}), (3, {"name": "Carol", "age": 26, "location": "LA"}), (4, {"name": "Dave", "age": 30, "location": "Chicago"}), (5, {"name": "Eve", "age": 24, "location": "Austin"}) ] for user_id, props in users: conn.execute("SELECT graph_node_add(?, ?)", (user_id, json.dumps(props))) # Create bidirectional friendships friendships = [ (1, 2, {"since": "2020", "closeness": 0.8}), (1, 3, {"since": "2019", "closeness": 0.6}), (2, 4, {"since": "2021", "closeness": 0.9}), (3, 4, {"since": "2020", "closeness": 0.7}), (4, 5, {"since": "2021", "closeness": 0.8}) ] for from_id, to_id, props in friendships: conn.execute("SELECT graph_edge_add(?, ?, 'FRIENDS', ?)", (from_id, to_id, json.dumps(props))) conn.execute("SELECT graph_edge_add(?, ?, 'FRIENDS', ?)", (to_id, from_id, json.dumps(props))) # Analyze the network cursor = conn.execute("SELECT graph_count_nodes(), graph_count_edges()") stats = cursor.fetchone() print(f"Network: {stats[0]} users, {stats[1]} friendships") cursor = conn.execute("SELECT graph_is_connected()") print(f"Network is connected: {bool(cursor.fetchone()[0])}") cursor = conn.execute("SELECT graph_density()") print(f"Network density: {cursor.fetchone()[0]:.3f}") # Find most connected users print("\nUser centrality scores:") for user_id, props in users: cursor = conn.execute("SELECT graph_degree_centrality(?)", (user_id,)) centrality = cursor.fetchone()[0] print(f" {props['name']}: {centrality:.3f}") ``` -------------------------------- ### Execute Parallel Pattern Matching Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE.md Demonstrates how to initialize a task scheduler and execute pattern matching in parallel to improve query performance. ```c // Parallel pattern matching example TaskScheduler *scheduler = graphCreateTaskScheduler(4); // 4 threads graphParallelPatternMatch(pGraph, pattern, &results, &nResults); ``` -------------------------------- ### Create Property Indexes on SQLite Graph Nodes and Edges Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE_TUNING_GUIDE.md Demonstrates how to create indexes on node and edge properties within the SQLite Graph Extension to speed up queries. It includes examples for indexing JSON properties, edge weights, and composite indexes. ```sql -- Index on node properties CREATE INDEX idx_person_name ON my_graph( json_extract(properties, '$.name') ) WHERE edge_id IS NULL; -- Index on edge properties CREATE INDEX idx_edge_weight ON my_graph(weight) WHERE edge_id IS NOT NULL; -- Composite index for complex queries CREATE INDEX idx_person_age_city ON my_graph( json_extract(properties, '$.age'), json_extract(properties, '$.city') ) WHERE edge_id IS NULL; ``` -------------------------------- ### Load and Verify SQLite Graph Extension Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TUTORIALS.md Demonstrates how to load the compiled SQLite Graph Extension into an SQLite database and verify its installation by checking the graph extension version. ```sql -- Start SQLite sqlite3 mydatabase.db -- Load the extension .load ./graph_extension -- Verify installation SELECT sqlite_version(); SELECT graph_version(); ``` -------------------------------- ### Run Full Test Suite Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Compiles and runs all tests in the project. This is a mandatory step before submitting changes to ensure code correctness. ```bash make test ``` -------------------------------- ### SQL Example: Social Network Analysis Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/API_REFERENCE.md Demonstrates creating and querying a social network graph using SQL. ```APIDOC ## SQL Example: Social Network Analysis ### Description This example shows how to create a virtual graph table, insert nodes and edges representing a social network, and perform queries to find relationships and analyze influence. ### Create Graph Table ```sql CREATE VIRTUAL TABLE social_network USING graph(); ``` ### Add Nodes (Users) ```sql INSERT INTO social_network (node_id, labels, properties) VALUES (1, '["User"]', '{"name": "Alice", "age": 28}'), (2, '["User"]', '{"name": "Bob", "age": 32}'), (3, '["User"]', '{"name": "Charlie", "age": 25}'); ``` ### Add Edges (Friendships) ```sql INSERT INTO social_network (edge_id, from_id, to_id, edge_type, weight) VALUES (101, 1, 2, 'FRIEND', 1.0), (102, 2, 3, 'FRIEND', 1.0), (103, 1, 3, 'FRIEND', 0.5); ``` ### Find Friends of Friends (using Recursive CTE) ```sql WITH RECURSIVE friends_of_friends AS ( SELECT 1 as user_id, 0 as depth UNION ALL SELECT CASE WHEN e.from_id = f.user_id THEN e.to_id ELSE e.from_id END as user_id, f.depth + 1 FROM friends_of_friends f JOIN social_network e ON ( e.edge_type = 'FRIEND' AND (e.from_id = f.user_id OR e.to_id = f.user_id) ) WHERE f.depth < 2 ) SELECT DISTINCT n.node_id, json_extract(n.properties, '$.name') as name FROM friends_of_friends f JOIN social_network n ON n.node_id = f.user_id WHERE f.depth = 2; ``` ### Find Most Influential Users (PageRank) ```sql SELECT n.node_id, json_extract(n.properties, '$.name') as name, p.rank FROM graph_pagerank('social_network') p JOIN social_network n ON n.node_id = p.node_id ORDER BY p.rank DESC LIMIT 10; ``` ``` -------------------------------- ### Force Index Usage in Cypher Queries with Index Hints Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE_TUNING_GUIDE.md Shows how to use index hints in Cypher queries to explicitly guide the query planner to use a specific index. This can be useful when the planner does not automatically choose the most efficient index. ```cypher -- Force index usage MATCH (p:Person) USING INDEX p:Person(name) WHERE p.name = 'John' RETURN p; ``` -------------------------------- ### Monitor and Benchmark Performance Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE.md Tools for analyzing query execution plans and running industry-standard benchmarks. ```sql SELECT * FROM cypher_execute_explain('MATCH (n:Person) RETURN n'); SELECT graph_benchmark(10); -- Scale factor 10 ``` -------------------------------- ### Development Workflow Commands Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Essential shell commands for cloning the repository, building the project, and running tests or memory checks. ```bash git clone https://github.com/agentflare-ai/sqlite-graph.git cd sqlite-graph make debug make test git checkout -b feature/your-feature-name make clean && make debug cd build/tests && ./test_cypher_basic valgrind --leak-check=full ./build/tests/test_cypher_basic ``` -------------------------------- ### Cypher Query Execution Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/MIGRATION_GUIDE.md Demonstrates how to execute Cypher queries within the C API to perform complex graph traversals. ```c CypherResult *pResult; int rc = cypher_execute(pGraph, "MATCH (n:Person)-[:KNOWS]->(friend) RETURN friend.name", &pResult); ``` -------------------------------- ### Building the SQLite Graph Extension Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/README.md Provides the necessary shell commands to install build dependencies, clone the repository, and compile the SQLite graph extension on a Linux-based system. ```bash # Install build tools sudo apt-get update sudo apt-get install build-essential sqlite3 libsqlite3-dev # Clone and build git clone https://github.com/agentflare-ai/sqlite-graph.git cd sqlite-graph make ``` -------------------------------- ### Debug with GDB Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Demonstrates how to use the GDB debugger to inspect the execution of a test binary. Includes commands to run the program and display the call stack. ```bash gdb ./build/tests/test_cypher_basic (gdb) run (gdb) bt ``` -------------------------------- ### Configure Performance Settings Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE.md PRAGMA settings to tune memory allocation, cache sizes, and parallel worker thread counts. ```sql PRAGMA graph_memory_pool_size = 10485760; -- 10MB PRAGMA graph_plan_cache_entries = 200; PRAGMA graph_plan_cache_memory = 20971520; -- 20MB PRAGMA graph_worker_threads = 0; ``` -------------------------------- ### Test Methodology Setup Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/SCALE_TEST_RESULTS.md This snippet outlines the testing environment and parameters used to evaluate the SQLite Graph Extension, including database configuration and query types. ```c // Test setup - Database: In-memory SQLite - Node distribution: Ages 20-69 (50 unique values) - Data: Person nodes with name and age properties - Queries: Full scan + WHERE clause with various selectivities // Measurements - Insert time: Bulk insert with BEGIN/COMMIT transaction - Query time: End-to-end execution including JSON serialization - Throughput: Calculated from total time and operation count ``` -------------------------------- ### Load SQLite Graph Extension Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/API_REFERENCE.md Demonstrates how to load the SQLite Graph Extension using the `.load` command and verify its installation by querying the available functions. ```sql -- Load the extension .load ./graph_extension -- Verify installation SELECT sqlite_version(); SELECT * FROM pragma_function_list WHERE name LIKE 'graph%'; ``` -------------------------------- ### Basic CREATE Cypher Statements Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/ROADMAP.md Shows basic CREATE statements for nodes in Cypher. These examples demonstrate the initial implementation of the Cypher query execution pipeline, allowing for node creation and storage. ```cypher CREATE (n) ``` ```cypher CREATE (p:Person) ``` -------------------------------- ### Manage Storage and Bulk Loading Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE.md Utilities for viewing compression statistics and performing high-performance bulk data imports from CSV files. ```sql -- View compression statistics SELECT graph_compression_stats(); -- Bulk load nodes from CSV SELECT graph_bulk_load('my_graph', '/path/to/nodes.csv', '{"batch_size":10000,"defer_indexing":true,"compress_properties":true}'); ``` -------------------------------- ### Verify Extension File Existence Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/examples/README.md Checks if the compiled shared object file for the graph extension exists at the expected path. This is a prerequisite for loading the extension into SQLite. ```python import os extension_path = "build/libgraph.so" if not os.path.exists(extension_path): print(f"Extension not found: {extension_path}") print("Please compile the extension first") ``` -------------------------------- ### Tutorial Summary and Next Steps (Python) Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/examples/graph_database_tutorial.ipynb Prints a summary of the tutorial's accomplishments, including data loading, graph operations, and analysis performed. Also suggests next steps for further exploration. ```python # Summary of what we accomplished print("šŸŽ‰ Tutorial Summary:") print("=" * 50) print("āœ… Loaded SQLite Graph Extension") print("āœ… Created nodes with JSON properties") print("āœ… Created edges with relationships and properties") print("āœ… Calculated graph statistics (density, connectivity)") print("āœ… Analyzed node centrality") print("āœ… Found shortest paths between nodes") print("āœ… Tested Cypher query parsing") print("āœ… Performed write operations") print("āœ… Built a larger social network graph") print("\nšŸ’” Next Steps:") print(" - Try building your own graph data") print(" - Experiment with different algorithms") print(" - Explore more complex Cypher queries") print(" - Test performance with larger datasets") print(" - Integrate with your applications") # Close connection conn.close() print("\nšŸ”š Database connection closed.") ``` -------------------------------- ### Run Graph Algorithms (Python) Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/examples/README.md Illustrates how to execute common graph algorithms like checking connectivity, finding the shortest path, and calculating degree centrality using the extension's SQL functions. Results are fetched and printed. ```python # Check if graph is connected cursor.execute("SELECT graph_is_connected() as connected") result = cursor.fetchone() print(f"Graph connected: {bool(result['connected'])}") # Find shortest path cursor.execute("SELECT graph_shortest_path(?, ?) as path", (1, 2)) result = cursor.fetchone() print(f"Shortest path: {result['path']}") # Calculate centrality cursor.execute("SELECT graph_degree_centrality(?) as centrality", (1,)) result = cursor.fetchone() print(f"Node centrality: {result['centrality']}") ``` -------------------------------- ### Populate Graph with Nodes and Edges Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/examples/graph_database_tutorial.ipynb Demonstrates batch insertion of nodes and relationships into the graph database using graph_node_add and graph_edge_add functions. ```python for person_id, properties in additional_people: cursor.execute("SELECT graph_node_add(?, ?) as result", (person_id, json.dumps(properties))) for from_id, to_id, rel_type, properties in new_connections: cursor.execute("SELECT graph_edge_add(?, ?, ?, ?) as result", (from_id, to_id, rel_type, json.dumps(properties))) ``` -------------------------------- ### Create Git Commit Message Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Example of a well-formatted Git commit message for submitting changes. It includes a concise summary and detailed bullet points about the changes. ```bash git commit -m "Add support for graph traversal optimization - Implement breadth-first search with pruning - Add performance tests for large graphs - Update API documentation" ``` -------------------------------- ### C Graph Node Implementation Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md An example of a C function for adding a node to the graph, demonstrating proper memory management and SQLite error handling conventions. ```c /* ** Add a node to the graph with proper error handling. ** Returns SQLITE_OK on success, error code on failure. */ int graphAddNode(GraphVtab *pVtab, sqlite3_int64 iNodeId, const char *zProperties) { GraphNode *pNode; int rc = SQLITE_OK; if( pVtab==NULL || iNodeId<0 ){ return SQLITE_MISUSE; } pNode = graphFindNode(pVtab, iNodeId); if( pNode!=NULL ){ return SQLITE_CONSTRAINT; } pNode = (GraphNode*)sqlite3_malloc(sizeof(GraphNode)); if( pNode==NULL ){ return SQLITE_NOMEM; } memset(pNode, 0, sizeof(GraphNode)); pNode->iNodeId = iNodeId; if( zProperties!=NULL ){ pNode->zProperties = sqlite3_mprintf("%s", zProperties); if( pNode->zProperties==NULL ){ sqlite3_free(pNode); return SQLITE_NOMEM; } } pNode->pNext = pVtab->pNodes; pVtab->pNodes = pNode; return SQLITE_OK; } ``` -------------------------------- ### Query Plan Analysis and Execution Statistics in SQL Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE_TUNING_GUIDE.md Shows how to analyze query plans and view execution statistics in SQL. `EXPLAIN QUERY PLAN` helps understand query execution paths, while `.stats on` provides detailed performance metrics. ```sql -- Analyze query plan EXPLAIN QUERY PLAN MATCH (p:Person)-[:KNOWS*1..3]->(friend:Person) WHERE p.name = 'John' RETURN friend; -- View execution statistics .stats on MATCH (p:Person) RETURN COUNT(p); ``` -------------------------------- ### Batch Updates in SQL Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE_TUNING_GUIDE.md Performs batch updates on graph properties using SQL. This example updates the 'processed' status for pending nodes in batches, improving efficiency over individual updates. ```sql -- Batch property updates UPDATE my_graph SET properties = json_set(properties, '$.processed', 'true') WHERE node_id IN ( SELECT node_id FROM my_graph WHERE json_extract(properties, '$.status') = 'pending' LIMIT 1000 ); ``` -------------------------------- ### Build and Install SQLite Graph Extension Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TUTORIALS.md Instructions for building the SQLite Graph Extension from source using make or CMake. It outlines the commands to compile the extension and indicates the output file locations for different operating systems. ```bash # Build the extension make # Or use CMake mkdir build && cd build cmake .. make # The extension file will be at: # ./graph_extension.so (Linux) # ./graph_extension.dylib (macOS) # ./graph_extension.dll (Windows) ``` -------------------------------- ### Performance Optimization: Create Indexes and Analyze (SQL) Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TUTORIALS.md Demonstrates how to create specific indexes on the 'social_network' table to improve query performance for common graph operations. It also includes commands to analyze the database tables for statistics. ```sql -- Create indexes for better performance CREATE INDEX idx_edge_type ON social_network(edge_type) WHERE edge_id IS NOT NULL; CREATE INDEX idx_node_labels ON social_network(labels) WHERE edge_id IS NULL; CREATE INDEX idx_from_to ON social_network(from_id, to_id) WHERE edge_id IS NOT NULL; -- Analyze tables ANALYZE social_network; ANALYZE knowledge_graph; ANALYZE transport_network; ``` -------------------------------- ### Create Knowledge Graph Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TUTORIALS.md Initializes a virtual table to store and manage knowledge graph data. ```sql CREATE VIRTUAL TABLE knowledge_graph USING graph(); ``` -------------------------------- ### Initialize and Query Graphs with Python Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/README.md Demonstrates how to load the sqlite-graph extension into a Python sqlite3 connection, create a virtual graph table, and execute Cypher queries to manage nodes and relationships. ```python import sqlite3 import json # Load the extension conn = sqlite3.connect(":memory:") conn.enable_load_extension(True) conn.load_extension("./build/libgraph.so") # Create a graph conn.execute("CREATE VIRTUAL TABLE graph USING graph()") # Use Cypher to create nodes and relationships conn.execute("SELECT cypher_execute('CREATE (p:Person {name: \"Alice\", age: 30})')") conn.execute("SELECT cypher_execute('CREATE (p:Person {name: \"Bob\", age: 25})')") conn.execute("SELECT cypher_execute(\"CREATE (a:Person {name: \\\"Alice\\\"})-[:KNOWS]->(b:Person {name: \\\"Bob\\\"})\")") # Query with pattern matching cursor = conn.execute("SELECT cypher_execute('MATCH (p:Person) WHERE p.age > 25 RETURN p')") results = json.loads(cursor.fetchone()[0]) print(results) ``` -------------------------------- ### Run Performance Tests Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Executes the performance test suite for the project. This helps in identifying performance bottlenecks. ```bash # Run performance tests cd build/tests && ./test_performance ``` -------------------------------- ### Manage Query Memory Pools Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE.md Allocates and destroys query-specific memory pools to reduce allocation overhead during execution. ```c QueryMemoryPool *pool = graphCreateMemoryPool(1024 * 1024); // 1MB pool void *data = graphPoolAlloc(pool, size); // No need to free individual allocations graphDestroyMemoryPool(pool); // Frees all at once ``` -------------------------------- ### Graph Algorithms (SQL) Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/examples/README.md Presents SQL functions for executing various graph algorithms, including finding the shortest path between two nodes, calculating degree centrality for a node, checking graph connectivity, density, and cycle detection. ```sql -- Path finding and analysis SELECT graph_shortest_path(from_id, to_id); SELECT graph_degree_centrality(node_id); SELECT graph_is_connected(); SELECT graph_density(); SELECT graph_has_cycle(); ``` -------------------------------- ### Linux: Checking SQLite Extension Support Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/README.md A command to verify if the installed SQLite library was compiled with support for loading extensions. This is a prerequisite for using the graph extension. ```bash sqlite3 -version # Should show extension support ``` -------------------------------- ### Use Prepared Statements in C Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE_TUNING_GUIDE.md Demonstrates how to use prepared statements in C with SQLite for efficient and secure query execution. Prepared statements allow for statement reuse, reducing parsing overhead and preventing SQL injection vulnerabilities. ```c sqlite3_stmt *stmt; sqlite3_prepare_v2(db, "SELECT * FROM graph_dijkstra(?, ?, ?)", -1, &stmt, NULL); // Reuse statement multiple times for (int i = 0; i < n; i++) { sqlite3_bind_int64(stmt, 1, start_id); sqlite3_bind_int64(stmt, 2, end_id); sqlite3_step(stmt); sqlite3_reset(stmt); } sqlite3_finalize(stmt); ``` -------------------------------- ### Index and Query Optimization Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE.md Best practices for creating indexes and writing efficient Cypher queries to optimize graph traversal. ```sql SELECT graph_create_label_index('my_graph', 'Person'); SELECT graph_create_property_index('my_graph', 'Person', 'name'); -- Efficient query pattern MATCH (p:Person {name: "Alice"})-[:KNOWS]->(friend) RETURN friend; ``` -------------------------------- ### Create Nodes and Relationships using Cypher Source: https://context7.com/agentflare-ai/sqlite-graph/llms.txt Demonstrates how to initialize the graph extension and execute Cypher CREATE statements to define nodes with labels, properties, and directed relationships. ```python import sqlite3 import json conn = sqlite3.connect(":memory:") conn.enable_load_extension(True) conn.load_extension("./build/libgraph.so") conn.execute("CREATE VIRTUAL TABLE graph USING graph()") # Create node with properties conn.execute("SELECT cypher_execute('CREATE (p:Person {name: \"Alice\", age: 30})')") # Create nodes with relationship conn.execute("""SELECT cypher_execute(' CREATE (alice:Person {name: \"Alice\"})-[:KNOWS {since: 2020}]->(bob:Person {name: \"Bob\"}) ')""") ``` -------------------------------- ### Create Release Command Line Script Bash Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TESTING_PLAN.md Executes the command-line script to create a new release. This is expected to create a Git tag, push it to the repository, and provide a URL for monitoring the release process. ```bash # Create the release ./scripts/create-release.sh # Expected output: # - Creates v0.1.0-alpha.0 tag # - Pushes to GitHub # - Provides monitoring URL ``` -------------------------------- ### Social Network Analysis Example Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/API_REFERENCE.md Demonstrates creating a virtual graph table, inserting data, and performing recursive queries and PageRank analysis. ```sql -- Create social network graph CREATE VIRTUAL TABLE social_network USING graph(); -- Add users INSERT INTO social_network (node_id, labels, properties) VALUES (1, '["User"]', '{"name": "Alice", "age": 28}'), (2, '["User"]', '{"name": "Bob", "age": 32}'), (3, '["User"]', '{"name": "Charlie", "age": 25}'); -- Add friendships INSERT INTO social_network (edge_id, from_id, to_id, edge_type, weight) VALUES (101, 1, 2, 'FRIEND', 1.0), (102, 2, 3, 'FRIEND', 1.0), (103, 1, 3, 'FRIEND', 0.5); -- Find friends of friends WITH RECURSIVE friends_of_friends AS ( SELECT 1 as user_id, 0 as depth UNION ALL SELECT CASE WHEN e.from_id = f.user_id THEN e.to_id ELSE e.from_id END as user_id, f.depth + 1 FROM friends_of_friends f JOIN social_network e ON ( e.edge_type = 'FRIEND' AND (e.from_id = f.user_id OR e.to_id = f.user_id) ) WHERE f.depth < 2 ) SELECT DISTINCT n.node_id, json_extract(n.properties, '$.name') as name FROM friends_of_friends f JOIN social_network n ON n.node_id = f.user_id WHERE f.depth = 2; -- Find most influential users (PageRank) SELECT n.node_id, json_extract(n.properties, '$.name') as name, p.rank FROM graph_pagerank('social_network') p JOIN social_network n ON n.node_id = p.node_id ORDER BY p.rank DESC LIMIT 10; ``` -------------------------------- ### Verify and Build sqlite-graph from Source Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/README.md Instructions for verifying the integrity of the release using checksums and compiling the library from source code using make. ```bash wget https://github.com/agentflare-ai/sqlite-graph/releases/latest/download/checksums.txt sha256sum -c checksums.txt git clone https://github.com/agentflare-ai/sqlite-graph.git cd sqlite-graph make make test ``` -------------------------------- ### C API Graph Operations Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/MIGRATION_GUIDE.md Updates function signatures for graph creation, node management, and pathfinding to support the new production architecture. ```c int graph_create(sqlite3 *db, const char *zName, GraphOptions *pOptions); int graph_add_node(Graph *pGraph, sqlite3_int64 iNodeId, const char **azLabels, int nLabels, const char *zProperties); int graph_shortest_path(Graph *pGraph, sqlite3_int64 iStartId, sqlite3_int64 iEndId, GraphPath **ppPath); ``` -------------------------------- ### Build with AddressSanitizer Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Compiles the project with the AddressSanitizer enabled. This helps detect memory errors like buffer overflows and use-after-free at runtime. ```bash export CFLAGS="-g -O1 -fsanitize=address" make clean && make ``` -------------------------------- ### Quick Database Rollback (Bash) Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/MIGRATION_GUIDE.md Provides a quick method to restore a SQLite database from a backup file in case of issues. It also demonstrates how to load an older version of the graph extension to maintain compatibility during a rollback. ```bash # If issues occur, quickly restore from backup cp your_database_backup.db your_database.db # Use old extension temporarily sqlite3 your_database.db << EOF .load ./old_graph_extension -- Your existing application should work .quit EOF ``` -------------------------------- ### Create a Virtual Graph Table in SQLite Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TUTORIALS.md Shows how to create a virtual table using the graph() function in SQLite. This is the initial step for setting up a graph database within SQLite. ```sql -- Create a virtual table for our social network CREATE VIRTUAL TABLE social_network USING graph(); -- Verify creation .tables ``` -------------------------------- ### Create Movie Recommendation Graph in SQL Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TUTORIALS.md This SQL script sets up a virtual graph table named 'movie_recommendations' and populates it with users, movies, and their ratings. It defines nodes for users and movies, and edges representing user ratings with associated properties. ```sql CREATE VIRTUAL TABLE movie_recommendations USING graph(); -- Add users INSERT INTO movie_recommendations (node_id, labels, properties) VALUES (1, '["User"]', '{"name": "John", "age": 25, "preferences": ["action", "sci-fi"]}'), (2, '["User"]', '{"name": "Sarah", "age": 30, "preferences": ["drama", "romance"]'), (3, '["User"]', '{"name": "Mike", "age": 28, "preferences": ["action", "comedy"]'), (4, '["User"]', '{"name": "Emma", "age": 35, "preferences": ["drama", "thriller"]'), (5, '["User"]', '{"name": "David", "age": 22, "preferences": ["sci-fi", "fantasy"]}'); -- Add movies INSERT INTO movie_recommendations (node_id, labels, properties) VALUES (101, '["Movie"]', '{"title": "The Matrix", "year": 1999, "genres": ["sci-fi", "action"], "rating": 8.7}'), (102, '["Movie"]', '{"title": "Inception", "year": 2010, "genres": ["sci-fi", "thriller"], "rating": 8.8}'), (103, '["Movie"]', '{"title": "The Godfather", "year": 1972, "genres": ["drama", "crime"], "rating": 9.2}'), (104, '["Movie"]', '{"title": "Titanic", "year": 1997, "genres": ["drama", "romance"], "rating": 7.9}'), (105, '["Movie"]', '{"title": "The Dark Knight", "year": 2008, "genres": ["action", "thriller"], "rating": 9.0}'), (106, '["Movie"]', '{"title": "Forrest Gump", "year": 1994, "genres": ["drama", "romance"], "rating": 8.8}'), (107, '["Movie"]', '{"title": "Star Wars", "year": 1977, "genres": ["sci-fi", "adventure"], "rating": 8.6}'), (108, '["Movie"]', '{"title": "The Avengers", "year": 2012, "genres": ["action", "adventure"], "rating": 8.0}'); -- Add ratings (edges) INSERT INTO movie_recommendations (edge_id, from_id, to_id, edge_type, weight, properties) VALUES -- John's ratings (201, 1, 101, 'RATED', 5.0, '{"date": "2023-01-15"}'), (202, 1, 105, 'RATED', 5.0, '{"date": "2023-02-20"}'), (203, 1, 107, 'RATED', 4.5, '{"date": "2023-03-10"}'), (204, 1, 108, 'RATED', 4.0, '{"date": "2023-04-05"}'), -- Sarah's ratings (205, 2, 103, 'RATED', 5.0, '{"date": "2023-01-20"}'), (206, 2, 104, 'RATED', 4.5, '{"date": "2023-02-15"}'), (207, 2, 106, 'RATED', 5.0, '{"date": "2023-03-25"}'), -- Mike's ratings (208, 3, 101, 'RATED', 4.5, '{"date": "2023-01-10"}'), (209, 3, 105, 'RATED', 5.0, '{"date": "2023-02-28"}'), (210, 3, 108, 'RATED', 4.5, '{"date": "2023-04-15"}'), -- Emma's ratings (211, 4, 102, 'RATED', 5.0, '{"date": "2023-01-25"}'), (212, 4, 103, 'RATED', 4.5, '{"date": "2023-02-10"}'), (213, 4, 105, 'RATED', 4.0, '{"date": "2023-03-30"}'), -- David's ratings (214, 5, 101, 'RATED', 5.0, '{"date": "2023-01-05"}'), (215, 5, 102, 'RATED', 5.0, '{"date": "2023-02-25"}'), (216, 5, 107, 'RATED', 5.0, '{"date": "2023-03-15"}'); ``` -------------------------------- ### Create Financial Transaction Network using SQL Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TUTORIALS.md This SQL script initializes a virtual graph table named 'transaction_network' and populates it with account nodes and transaction edges. It demonstrates how to insert nodes with labels and properties, and edges with types, weights, and properties, setting up a network for fraud detection analysis. ```sql CREATE VIRTUAL TABLE transaction_network USING graph(); -- Add accounts INSERT INTO transaction_network (node_id, labels, properties) VALUES -- Regular accounts (1, '["Account", "Personal"]', '{"owner": "Alice Smith", "created": "2020-01-15", "status": "active", "risk": "low"}'), (2, '["Account", "Personal"]', '{"owner": "Bob Johnson", "created": "2019-06-20", "status": "active", "risk": "low"}'), (3, '["Account", "Business"]', '{"owner": "Tech Corp", "created": "2018-03-10", "status": "active", "risk": "low"}'), (4, '["Account", "Personal"]', '{"owner": "Charlie Brown", "created": "2021-11-05", "status": "active", "risk": "medium"}'), -- Suspicious accounts (5, '["Account", "Personal"]', '{"owner": "David Wilson", "created": "2023-01-01", "status": "active", "risk": "high"}'), (6, '["Account", "Shell"]', '{"owner": "XYZ Holdings", "created": "2023-01-05", "status": "active", "risk": "high"}'), (7, '["Account", "Shell"]', '{"owner": "ABC Trading", "created": "2023-01-10", "status": "active", "risk": "high"}'), -- Merchant accounts (8, '["Account", "Merchant"]', '{"owner": "Online Store", "created": "2017-05-20", "status": "active", "risk": "low"}'), (9, '["Account", "Merchant"]', '{"owner": "Coffee Shop", "created": "2016-08-15", "status": "active", "risk": "low"}'); -- Add transactions INSERT INTO transaction_network (edge_id, from_id, to_id, edge_type, weight, properties) VALUES -- Normal transactions (101, 1, 8, 'TRANSFER', 150.00, '{"date": "2023-06-01", "time": "10:30:00", "method": "online", "status": "completed"}'), (102, 1, 9, 'TRANSFER', 5.50, '{"date": "2023-06-01", "time": "08:15:00", "method": "card", "status": "completed"}'), (103, 2, 3, 'TRANSFER', 2500.00, '{"date": "2023-06-02", "time": "14:20:00", "method": "wire", "status": "completed"}'), (104, 3, 2, 'TRANSFER', 5000.00, '{"date": "2023-06-03", "time": "09:45:00", "method": "wire", "status": "completed"}'), -- Suspicious pattern: rapid transfers through shell companies (105, 4, 5, 'TRANSFER', 9500.00, '{"date": "2023-06-04", "time": "23:45:00", "method": "online", "status": "completed"}'), (106, 5, 6, 'TRANSFER', 9400.00, '{"date": "2023-06-05", "time": "00:15:00", "method": "online", "status": "completed"}'), (107, 6, 7, 'TRANSFER', 9300.00, '{"date": "2023-06-05", "time": "00:45:00", "method": "online", "status": "completed"}'), (108, 7, 5, 'TRANSFER', 9200.00, '{"date": "2023-06-05", "time": "01:15:00", "method": "online", "status": "completed"}'), -- More suspicious transactions (109, 5, 4, 'TRANSFER', 4500.00, '{"date": "2023-06-05", "time": "02:00:00", "method": "online", "status": "completed"}'), (110, 6, 4, 'TRANSFER', 4500.00, '{"date": "2023-06-05", "time": "02:30:00", "method": "online", "status": "completed"}'); ``` -------------------------------- ### Graph Maintenance and Performance Baselines Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE_TUNING_GUIDE.md Routine SQL maintenance tasks including statistics updates, index management, and performance baseline tracking. ```sql -- Regular Statistics Updates ANALYZE my_graph; -- Index Maintenance REINDEX my_graph; VACUUM my_graph; -- Create performance baseline CREATE TABLE performance_baseline AS SELECT * FROM graph_performance_stats(); -- Compare current performance SELECT current.metric, baseline.value as baseline_value, current.value as current_value, ROUND((current.value - baseline.value) * 100.0 / baseline.value, 2) as pct_change FROM graph_performance_stats() current JOIN performance_baseline baseline USING (metric); ``` -------------------------------- ### Performance Optimization and Indexing Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/MIGRATION_GUIDE.md Configures memory pools and creates specific indexes on labels, edge types, and JSON properties to optimize query performance. ```sql CREATE INDEX idx_node_labels ON my_graph(labels) WHERE edge_id IS NULL; CREATE INDEX idx_name ON my_graph(json_extract(properties, '$.name')) WHERE edge_id IS NULL; SELECT * FROM my_graph WHERE edge_id IS NULL AND json_extract(properties, '$.name') = 'John'; ``` -------------------------------- ### Check Documentation and Executable Status Bash Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/TESTING_PLAN.md Verifies the existence of key documentation files and checks if the release script is executable. It also confirms that the CONTRIBUTING.md file references the new release process documentation. ```bash # Check files exist ls -la docs/RELEASE_PROCESS.md ls -la docs/FIXES_SUMMARY.md # Check release script is executable test -x scripts/create-release.sh && echo "Executable" || echo "Not executable" # Check CONTRIBUTING.md references new docs grep -q "RELEASE_PROCESS.md" CONTRIBUTING.md && echo "Found" || echo "Not found" ``` -------------------------------- ### Update SQL Queries for Graph Data Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/MIGRATION_GUIDE.md Demonstrates the transition from simple node insertion to structured JSON-based property storage in the production graph schema. ```sql -- Old prototype queries INSERT INTO graph_table (id, data) VALUES (1, 'node_data'); -- New production queries INSERT INTO graph_table (node_id, labels, properties) VALUES (1, '["Person"]', '{"name": "John", "age": 30}'); ``` -------------------------------- ### Build with Debug Symbols Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Compiles the project with debug symbols enabled using 'make debug'. This facilitates debugging with tools like GDB. ```bash # Build with debug symbols make debug ``` -------------------------------- ### Push Feature Branch to Fork Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/CONTRIBUTING.md Command to push the local feature branch to the remote fork repository. This is a step in the pull request process. ```bash git push origin feature/your-feature-name ``` -------------------------------- ### Convert Graph to CSR Format Source: https://github.com/agentflare-ai/sqlite-graph/blob/main/docs/PERFORMANCE.md Converts the graph structure into a Compressed Sparse Row (CSR) format to enable O(1) neighbor access. ```c CSRGraph *csr = graphConvertToCSR(pGraph); // Provides O(1) neighbor access ```