### Example Workflow: Local Graphiti Setup

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Step-by-step commands to set up and run Local Graphiti, including starting Neo4j, llama.cpp server, and the main application.

```bash
# Terminal 1: Start Neo4j
neo4j start

# Terminal 2: Start llama.cpp server
cd ~/apps/llama.cpp
./llama-server -m ~/ai_models/magnum-v4-12b.gguf -c 4096 --port 8080

# Terminal 3: Run local_graphiti
cd ~/local_graphiti
source kg_venv/bin/activate
export NEO4J_DB=neo4j
export NEO4J_DB_PASS=your_password
python app.py

# View results at http://localhost:7474 (Neo4j browser)
```

--------------------------------

### Install and Start Neo4j on macOS

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Installs Neo4j using Homebrew and starts the service. Ensure you have Homebrew installed.

```bash
brew install neo4j
neo4j start
```

--------------------------------

### Start Neo4j Server (Manual Download)

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Starts the Neo4j server after manual download and setting the NEO4J_HOME environment variable. Ensure the path to your Neo4j installation is correct.

```bash
export NEO4J_HOME=$HOME/apps/neo4j-community-5.24.1
$NEO4J_HOME/bin/neo4j start
```

--------------------------------

### Complete Example Application Configuration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

This comprehensive example demonstrates how to configure LLM clients, embedders, and Neo4j credentials, then initializes and uses the Graphiti instance. It includes loading environment variables, setting up clients, and performing basic operations like building indices and searching.

```python
import asyncio
import os
from dotenv import load_dotenv
from graphiti_core import Graphiti
from graphiti_core.nodes import EpisodeType
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient
from HuggingFaceEmbedder import HuggingFaceEmbedder, HuggingFaceEmbedderConfig
from datetime import datetime

# Load environment variables
load_dotenv()

async def main():
    # Configure LLM Client
    llm_config = LLMConfig(
        base_url="http://127.0.0.1:8080/completion",
        temperature=0.7,
        max_tokens=2048
    )
    llm_client = LocalAiClient(
        config=llm_config,
        grammar_file="./json.gbnf"
    )
    
    # Configure Embedder
    embedder_config = HuggingFaceEmbedderConfig(
        embedding_model='sentence-transformers/all-MiniLM-L6-v2'
    )
    embedder = HuggingFaceEmbedder(config=embedder_config)
    
    # Configure Neo4j credentials
    neo4j_url = "bolt://localhost:7687"
    neo4j_user = os.environ.get('NEO4J_DB', 'neo4j')
    neo4j_password = os.environ.get('NEO4J_DB_PASS')
    
    # Create Graphiti instance
    graphiti = Graphiti(
        neo4j_url=neo4j_url,
        neo4j_user=neo4j_user,
        neo4j_password=neo4j_password,
        embedder=embedder,
        llm_client=llm_client
    )
    
    # Use the configured Graphiti instance
    await graphiti.build_indices_and_constraints()
    
    # Add episodes
    await graphiti.add_episode(
        name="Example",
        episode_body="Alice works at Acme Corp",
        source=EpisodeType.text,
        reference_time=datetime.now()
    )
    
    # Search
    results = await graphiti.search('Who works at Acme?')
    print(results)
    
    await graphiti.close()

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Clone local_graphiti and Setup Virtual Environment

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Clones the local_graphiti repository and creates/activates a Python virtual environment. Ensure Python 3.11+ is installed.

```bash
git clone https://github.com/tonymantoan/local_graphiti.git
cd local_graphiti
python3.11 -m venv kg_venv
source kg_venv/bin/activate  # On Windows: kg_venv\Scripts\activate
```

--------------------------------

### Start Neo4j Database

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Starts the Neo4j database server. Access the database via http://localhost:7474. Ensure the NEO4J_HOME environment variable is set correctly.

```bash
$NEO4J_HOME/bin/neo4j start
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Installs the project's Python dependencies from the requirements.txt file. Ensure the virtual environment is activated.

```bash
pip install -r requirements.txt
```

--------------------------------

### Start llama.cpp server with reduced context window

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Command to start the llama.cpp server with a smaller context window size. A smaller context window consumes less memory during inference.

```bash
# Smaller context uses less memory
./llama-server -m model.gguf -c 2048 --port 8080  # Instead of 4096
```

--------------------------------

### Start llama.cpp Server

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Starts the llama.cpp inference server with a specified model and context size. Keep this terminal running while using the application.

```bash
cd ~/apps/llama.cpp
./llama-server \
  -m ~/ai_models/magnum-v4-12b.gguf \
  -c 4096 \
  --port 8080
```

--------------------------------

### Start llama.cpp server with smaller quantization

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Command to start the llama.cpp server using a quantized model (e.g., Q2 or Q4). Lower quantization levels reduce memory footprint.

```bash
# Use Q2 or Q4 instead of Q8
./llama-server -m model_q2.gguf -c 2048 --port 8080
```

--------------------------------

### LocalAiClient Constructor Examples

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/LocalAiClient.md

Demonstrates initializing the LocalAiClient with default settings, custom configuration, or with a grammar file for output constraints.

```python
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient

# Basic initialization with default config
client = LocalAiClient()

# With custom configuration
config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7
)
client = LocalAiClient(config=config)

# With grammar file for JSON constraint
config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.5
)
client = LocalAiClient(config=config, grammar_file="./json.gbnf")
```

--------------------------------

### Run the Sample Application

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Executes the main application script. This will connect to Neo4j, load models, and process example episodes.

```bash
python app.py
```

--------------------------------

### Start llama.cpp Server with Metal Acceleration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Start the llama.cpp inference server with Metal acceleration for macOS. Specify the number of layers to offload to the GPU.

```bash
# With Metal acceleration (macOS)
./llama-server \
  -m /path/to/model.gguf \
  -c 4096 \
  --port 8080 \
  -ngl 40  # Metal layers
```

--------------------------------

### Start llama.cpp Server with GPU Acceleration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Start the llama.cpp inference server with GPU acceleration using CUDA. Specify the number of layers to offload to the GPU.

```bash
# With GPU acceleration (CUDA)
./llama-server \
  -m /path/to/model.gguf \
  -c 4096 \
  --port 8080 \
  -ngl 40  # Number of layers to offload to GPU
```

--------------------------------

### Start Neo4j service

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Command to start the Neo4j database service if it is not running. Ensure Neo4j is accessible on the configured port.

```bash
neo4j start
```

--------------------------------

### Graphiti Initialization Sequence

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/MODULE_INDEX.md

A step-by-step guide to initializing the Graphiti instance, including creating LLM and embedder configurations and clients.

```python
# Step 1: Import modules
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient
from HuggingFaceEmbedder import HuggingFaceEmbedderConfig, HuggingFaceEmbedder
from graphiti_core import Graphiti

# Step 2: Create LLM configuration
llm_config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7
)

# Step 3: Create LLM client
llm_client = LocalAiClient(config=llm_config, grammar_file="./json.gbnf")

# Step 4: Create embedder configuration
embedder_config = HuggingFaceEmbedderConfig()

# Step 5: Create embedder
embedder = HuggingFaceEmbedder(config=embedder_config)

# Step 6: Initialize Graphiti
graphiti = Graphiti(
    "bolt://localhost:7687",
    "neo4j",
    "password",
    llm_client=llm_client,
    embedder=embedder
)

# Step 7: Use graphiti instance
# graphiti.add_episode(...)
# graphiti.search(...)
```

--------------------------------

### Install Python Packages

Source: https://github.com/tonymantoan/local_graphiti/blob/main/README.md

Command to install Python packages within an activated virtual environment.

```bash
pip3 install
```

--------------------------------

### Run the Local Graphiti Application

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Navigates to the project directory and executes the main Python application file. This starts the Graphiti process.

```bash
cd /workspace/home/local_graphiti
python app.py
```

--------------------------------

### Start llama.cpp Server with Fast Quantization

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Launch the llama.cpp server using a fast quantization model (e.g., Q4). This is a solution for slow graph building due to model size or quantization level.

```bash
# Start with fast quantization
./llama-server -m model_q4.gguf -c 2048 --port 8080
```

--------------------------------

### DeepSeek Prompt Format

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Example of the DeepSeek prompt template used by LocalAiClient.

```text
{system content}<｜end▁of▁sentence｜><|user|>{user content}<｜Assistant｜>
```

--------------------------------

### ChatML Prompt Format

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Example of the ChatML prompt template used by LocalAiClient.

```text
<|im_start|>system
{system content}
<|im_end|>
<|im_start|>user
{user content}
<|im_end|>
```

--------------------------------

### Starting llama.cpp Server with GPU Acceleration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Launch the llama.cpp server with GPU acceleration enabled. Specify the model path, context size, and the number of layers to offload to the GPU.

```bash
# Start llama.cpp with GPU:
./llama-server \
  -m ~/ai_models/model.gguf \
  -c 4096 \
  -ngl 40 \
  # Number of layers to GPU
  --port 8080
```

--------------------------------

### Run Llama Server with GPU Acceleration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Start the llama.cpp server, specifying the model and enabling GPU layers for faster inference.

```bash
./llama-server -m model.gguf -ngl 40 --port 8080
```

--------------------------------

### Read Grammar File Example

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/LocalAiClient.md

Shows how to load a GBNF grammar file into the client's grammar_content attribute for output validation.

```python
client = LocalAiClient()
client.read_grammar_file("./json.gbnf")
# client.grammar_content now contains the grammar
```

--------------------------------

### Run Async Entry Point

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Executes the main asynchronous function `doSearch()` using the asyncio.run() method. This is the standard way to start an asyncio application.

```python
asyncio.run( doSearch() )
```

--------------------------------

### Mistral Prompt Format

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Example of the Mistral prompt template used by LocalAiClient.

```text
<s>
[INST]
{system content}
[/INST]
[INST]
{user content}
[/INST]
```

--------------------------------

### llama.cpp Request Payload Example

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Example JSON payload sent by LocalAiClient to the llama.cpp completion endpoint. Includes prompt, prediction count, temperature, and grammar.

```json
{
  "prompt": "formatted prompt string",
  "n_predict": 2048,
  "temperature": 0.7,
  "grammar": "GBNF grammar content or empty string"
}
```

--------------------------------

### Initialize HuggingFaceEmbedder with Default Configuration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/types.md

Instantiates a HuggingFaceEmbedder using the default configuration settings. No specific setup is required beyond importing the necessary classes.

```python
from HuggingFaceEmbedder import HuggingFaceEmbedderConfig, HuggingFaceEmbedder

# Use default configuration
config = HuggingFaceEmbedderConfig()
embedder = HuggingFaceEmbedder(config)
```

--------------------------------

### Basic Usage Pattern for Local Graphiti

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/INDEX.md

This snippet demonstrates the basic setup and usage pattern for integrating local_graphiti into your Python code. It shows how to configure the LLM client, embedder, and the main Graphiti class, followed by common operations like adding data and searching.

```python
from LocalAiClient import LocalAiClient
from HuggingFaceEmbedder import HuggingFaceEmbedder
from graphiti_core import Graphiti
from graphiti_core.llm_client.config import LLMConfig

# Configure
config = LLMConfig(base_url="http://127.0.0.1:8080/completion")
client = LocalAiClient(config=config, grammar_file="./json.gbnf")
embedder = HuggingFaceEmbedder()

# Use
graphiti = Graphiti(..., llm_client=client, embedder=embedder)
await graphiti.add_episode(...)
results = await graphiti.search(...)
```

--------------------------------

### Activate Virtual Environment

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Activate the Python virtual environment before installing dependencies or running the application.

```bash
source kg_venv/bin/activate  # Linux/macOS
```

```bash
kg_venv\Scripts\activate     # Windows
```

--------------------------------

### Dockerfile for Containerization

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

A Dockerfile for containerizing the Python application, including dependency installation and command execution.

```dockerfile
FROM python:3.11
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY *.py ./
COPY json.gbnf ./
CMD ["python", "app.py"]
```

--------------------------------

### Reinstall Requirements

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Force reinstall all project dependencies to resolve potential conflicts or corrupted installations.

```bash
pip install --force-reinstall -r requirements.txt
```

--------------------------------

### Neo4j Configuration via .env File

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Example of a .env file for storing Neo4j connection details. This file can be used with libraries like python-dotenv to load environment variables.

```ini
NEO4J_DB=neo4j
NEO4J_DB_PASS=password123
NEO4J_BOLT_URL=bolt://localhost:7687
```

--------------------------------

### Download LLM Model using Hugging Face CLI

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Downloads a specified GGUF model file using the huggingface-cli. Ensure the CLI is installed and the model name/path are correct.

```bash
mkdir -p ~/ai_models
cd ~/ai_models
# Use huggingface-hub CLI or download from website
huggingface-cli download anthracite-org/magnum-v4-12b-gguf \
  magnum-v4-12b.gguf --local-dir . --local-dir-use-symlinks False
```

--------------------------------

### Check Neo4j credentials

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Bash commands to display Neo4j database and password environment variables. Verify these match your Neo4j setup.

```bash
echo $NEO4J_DB
echo $NEO4J_DB_PASS
```

--------------------------------

### Enable GPU Acceleration with llama.cpp (NVIDIA CUDA)

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Start the llama.cpp server with GPU acceleration enabled for NVIDIA CUDA. The `-ngl` flag specifies the number of layers to offload to the GPU, speeding up inference.

```bash
# NVIDIA CUDA
./llama-server -m model.gguf -c 4096 -ngl 40 --port 8080
```

--------------------------------

### Custom Mistral Prompt Formatting

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Example of a Python function to format prompts specifically for Mistral models. This is useful for prompt customization.

```python
def format_prompt_for_mistral(self, messages):
    # Custom Mistral prompt formatting
    pass
```

--------------------------------

### Example Constrained JSON Output

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/grammar-files.md

This is an example of a JSON response constrained by a grammar, ensuring specific fields and structure for extracted nodes.

```json
{
  "extracted_nodes": [
    {
      "name": "Kamala Harris",
      "labels": ["Person", "Politician"],
      "summary": "Attorney General of California"
    },
    {
      "name": "California",
      "labels": ["Location", "State"],
      "summary": "US state"
    }
  ]
}
```

--------------------------------

### LLM Response Example

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

JSON structure of the response received from the LLM inference endpoint.

```json
{
  "content": "...",
  "tokens_predicted": 42,
  "tokens_evaluated": 100,
  "timings": {...}
}
```

--------------------------------

### Sample Application Output

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Illustrates the expected output when running the Local Graphiti application, including logs for embedding, LLM responses, and search results.

```text
embbed episodes
Generate Local llm response for prompts:
Full prompt is: <|im_start|>system
You are an AI assistant that extracts entity nodes...<|im_end|>
<|im_start|>user
Follow the instructions...<|im_end|>
response json string:
{"extracted_nodes": [{"name": "Kamala Harris", "labels": ["Person"], "summary": "Attorney General of California"}...]}
Final llm respons is {"extracted_nodes": [...]}
...
Results: 
[Node(name='Kamala Harris', summary='Attorney General of California and former district attorney for San Francisco')]
```

--------------------------------

### LLMConfig Initialization for LocalAiClient

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/types.md

Demonstrates how to initialize LLMConfig for a LocalAiClient, specifying the local inference endpoint URL and generation parameters. The base_url must point to a valid local inference server.

```python
LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7,
    max_tokens=2048
)
```

--------------------------------

### Clone and Build llama.cpp

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Clones the llama.cpp repository and builds the server binary. This is required to run local LLM models.

```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
```

--------------------------------

### Graphiti Relationship Extraction Prompt

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Example JSON payload for relationship extraction prompts sent to the LLM.

```json
{
  "edges": [
    {
      "relation_type": "RELATION_TYPE",
      "source_node_uuid": "uuid",
      "target_node_uuid": "uuid",
      "fact": "relationship description",
      "valid_at": "YYYY-MM-DDTHH:MM:SSZ",
      "invalid_at": null
    }
  ]
}
```

--------------------------------

### Graphiti Entity Extraction Prompt

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Example JSON payload for entity extraction prompts sent to the LLM.

```json
{
  "extracted_nodes": [
    {
      "name": "entity name",
      "labels": ["label1", "label2"],
      "summary": "brief description"
    }
  ]
}
```

--------------------------------

### Minimal LocalAiClient Configuration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Sets up a basic LocalAiClient instance with only the required base URL for the LLM endpoint.

```python
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient

config = LLMConfig(base_url="http://127.0.0.1:8080/completion")
client = LocalAiClient(config=config)
```

--------------------------------

### LLM Request Payload Example

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

JSON structure for the request payload sent to the LLM inference endpoint.

```json
{
  "prompt": "formatted prompt string",
  "n_predict": 2048,
  "temperature": 0.7,
  "grammar": "GBNF grammar or empty string"
}
```

--------------------------------

### Create and Activate Python Virtual Environment

Source: https://github.com/tonymantoan/local_graphiti/blob/main/README.md

Steps to create a Python virtual environment using Python 3.11 and activate it.

```bash
python3.11 -m venv kg_venv
source kg_venv/bin/activate
```

--------------------------------

### HuggingFace Embedder Configuration

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Configure the HuggingFace embedder with a specific model. This example uses a faster model for quicker embeddings.

```python
embedder_config = HuggingFaceEmbedderConfig(
    embedding_model='sentence-transformers/all-MiniLM-L6-v2'  # Faster
)
```

--------------------------------

### LLM Configuration for Local Inference

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Sets up configuration for a local llama.cpp inference server, specifying the base URL and a temperature for balanced output.

```python
llm_config = LLMConfig( 
    base_url="http://127.0.0.1:8080/completion", 
    temperature=0.7 
)
```

--------------------------------

### LocalAiClient Initialization

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Initializes the LocalAiClient with the specified LLM configuration and a JSON grammar file to constrain output format.

```python
local_llm_client = LocalAiClient.LocalAiClient( 
    config=llm_config, 
    grammar_file="./json.gbnf"
)
```

--------------------------------

### Search Pipeline Flow

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Details the process for querying the knowledge graph, starting from user input to retrieving relevant information from Neo4j.

```text
User Query
     ↓
┌──────────────────────────────────────┐
│ Encode Query with HuggingFaceEmbedder│
└──────────────────────────────────────┘
     ↓
┌──────────────────────────────────────┐
│ Search Neo4j for Similar Embeddings  │
│ (Semantic search)                    │
└──────────────────────────────────────┘
     ↓
┌──────────────────────────────────────┐
│ Return Relevant Nodes/Relationships  │
└──────────────────────────────────────┘
```

--------------------------------

### Integrate LocalAiClient with Graphiti

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/LocalAiClient.md

Demonstrates how to initialize and configure the Graphiti knowledge graph builder with the LocalAiClient and HuggingFaceEmbedder. Ensure your Neo4j instance is running and accessible.

```python
from graphiti_core import Graphiti
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient
from HuggingFaceEmbedder import HuggingFaceEmbedder

config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7
)
client = LocalAiClient(config=config, grammar_file="./json.gbnf")
embedder = HuggingFaceEmbedder()

graphiti = Graphiti(
    neo4j_url="bolt://localhost:7687",
    neo4j_user="neo4j",
    neo4j_password="password",
    llm_client=client,
    embedder=embedder
)
```

--------------------------------

### Graphiti Initialization with LocalAI and HuggingFace Embedder

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/MODULE_INDEX.md

Initialize the Graphiti instance, integrating LocalAiClient for LLM operations and HuggingFaceEmbedder for text encoding.

```python
graphiti = Graphiti(
    neo4j_url="bolt://localhost:7687",
    neo4j_user="neo4j",
    neo4j_password="password",
    llm_client=LocalAiClient(...),        # LLM integration
    embedder=HuggingFaceEmbedder(...)     # Embedder integration
)
```

--------------------------------

### Configuration Flow Diagram

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Visual representation of the configuration process, from environment variables to the final Graphiti instance.

```text
Environment Variables
(NEO4J_DB, NEO4J_DB_PASS)
        ↓
┌──────────────────────────┐
│ Create LLMConfig         │
├──────────────────────────┤
│ base_url                 │
│ temperature              │
│ max_tokens               │
└──────────────────────────┘
        ↓
┌──────────────────────────────────────┐
│ Create LocalAiClient                 │
├──────────────────────────────────────┤
│ - Loads grammar from file            │
│ - Inherits config settings           │
└──────────────────────────────────────┘
        ↓
┌──────────────────────────────────────┐
│ Create HuggingFaceEmbedderConfig     │
├──────────────────────────────────────┤
│ embedding_model                      │
│ api_key (unused)                     │
│ base_url (unused)                    │
└──────────────────────────────────────┘
        ↓
┌──────────────────────────────────────┐
│ Create HuggingFaceEmbedder           │
├──────────────────────────────────────┤
│ - Loads model from HuggingFace Hub   │
└──────────────────────────────────────┘
        ↓
┌──────────────────────────────────────┐
│ Create Graphiti Instance             │
├──────────────────────────────────────┤
│ neo4j_url                            │
│ neo4j_user                           │
│ neo4j_password                       │
│ llm_client (LocalAiClient)           │
│ embedder (HuggingFaceEmbedder)       │
└──────────────────────────────────────┘
```

--------------------------------

### Customize Episodes in app.py

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Modifies the 'episodes' list within the app.py file to use custom text data. Replace the example strings with your own content.

```python
episodes = [
    "Your first text here",
    "Your second text here",
]
```

--------------------------------

### Perform Multiple Search Queries

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Demonstrates how to perform multiple search queries against the knowledge graph sequentially. Each query can ask different questions to retrieve varied information.

```python
results1 = await graphiti.search('Who was the California Attorney General?')
results2 = await graphiti.search('When was Harris in office?')
results3 = await graphiti.search('What cities are mentioned?')
```

--------------------------------

### Reduce Context Window in llama.cpp Server

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Launch the llama.cpp server with a smaller context window size using the `-c` flag. This can help improve performance and reduce processing time for graph building.

```bash
./llama-server -m model.gguf -c 1024 --port 8080  # Smaller context
```

--------------------------------

### Print Current Working Directory

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Print the current working directory to verify it matches the expected project root.

```python
import os
print(os.getcwd())  # Should be /path/to/local_graphiti/
```

--------------------------------

### Login using HuggingFace CLI

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Log in to HuggingFace Hub using the CLI to set your authentication token. The CLI will prompt you to enter your token interactively.

```bash
huggingface-cli login
# Enter your token when prompted
```

--------------------------------

### LocalAiClient Constructor

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/LocalAiClient.md

Initializes the LocalAiClient with configuration and an optional grammar file. It sets up the client to communicate with local LLM inference endpoints and can optionally load grammar rules for output constraints.

```APIDOC
## LocalAiClient `__init__`

### Description
Initializes the LocalAiClient with configuration and optional grammar file. It sets up the client to communicate with local LLM inference endpoints and can optionally load grammar rules for output constraints.

### Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| config | `LLMConfig | None` | No | `None` | Configuration object containing API endpoint, temperature, max tokens. If `None`, a default `LLMConfig()` is created. |
| grammar_file | `str | None` | No | `None` | Path to a GBNF grammar file. If provided, the file will be read and its content used to constrain LLM output via the inference endpoint. |

### Attributes Set

| Attribute | Type | Description |
|-----------|------|-------------|
| base_url | `str` | The URL of the local inference endpoint (from `config.base_url`). |
| grammar_content | `str` | The GBNF grammar content, loaded from file if `grammar_file` provided, otherwise empty string. |
| temperature | `float` | Temperature setting from config. |
| max_tokens | `int` | Maximum token limit from config. |

### Raises

- `FileNotFoundError` — if `grammar_file` path does not exist (raised by `read_grammar_file`)

### Example

```python
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient

# Basic initialization with default config
client = LocalAiClient()

# With custom configuration
config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7
)
client = LocalAiClient(config=config)

# With grammar file for JSON constraint
config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.5
)
client = LocalAiClient(config=config, grammar_file="./json.gbnf")
```
```

--------------------------------

### Custom LLM Client Implementation

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Python snippet demonstrating how to extend the base LLMClient for custom LLM integrations.

```python
class CustomLLMClient(LLMClient):
    async def _generate_response(self, messages: list[Message]) -> dict:
        # Custom implementation
        pass
```

--------------------------------

### Entity Node Extraction Prompt (Mistral Format)

Source: https://github.com/tonymantoan/local_graphiti/blob/main/README.md

This prompt guides the AI to extract entity nodes from conversational text, identifying speakers and significant entities. It specifies the desired JSON output format for the extracted nodes.

```text
<s>
        [INST]

        You are an AI assistant that extracts entity nodes from conversational text. Your primary task is to identify and extract the speaker and other significant entities mentioned in the conversation.
        [/INST]
        [INST]
        Follow the instructions. Only produce the required JSON response, do not include any explanation or other text.

        Given the following conversation, extract entity nodes from the CURRENT MESSAGE that are explicitly or implicitly mentioned:

        Conversation:
        []
        <CURRENT MESSAGE>
        Kamala Harris is the Attorney General of California. She was previously the district attorney for San Francisco.

        Guidelines:
        2. Extract significant entities, concepts, or actors mentioned in the conversation.
        3. Provide concise but informative summaries for each extracted node.
        4. Avoid creating nodes for relationships or actions.
        5. Avoid creating nodes for temporal information like dates, times or years (these will be added to edges later).
        6. Be as explicit as possible in your node names, using full names and avoiding abbreviations.

        Respond with a JSON object in the following format:
        {
            "extracted_nodes": [
                {
                    "name": "Unique identifier for the node (use the speaker's name for speaker nodes)",
                    "labels": ["Entity", "OptionalAdditionalLabel"],
                    "summary": "Brief summary of the node's role or significance"
                }
            ]
        }


        [/INST]
```

--------------------------------

### Graphiti Initialization with Neo4j and Local AI

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Creates a Graphiti instance, connecting to a Neo4j database and integrating a HuggingFace embedder and a LocalAiClient for LLM inference.

```python
graphiti = Graphiti(
    "bolt://localhost:7687",           # Neo4j URL
    neo4j_db_name,                      # Username
    neo4j_db_pass,                      # Password
    embedder=HuggingFaceEmbedder.HuggingFaceEmbedder(),  # Embedder
    llm_client=local_llm_client         # LLM Client
)
```

--------------------------------

### Initialize HuggingFaceEmbedder

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/HuggingFaceEmbedder.md

Demonstrates initializing the HuggingFaceEmbedder with default, custom, or larger sentence-transformer models. Ensure the 'HuggingFaceEmbedder' and 'HuggingFaceEmbedderConfig' classes are imported.

```python
from HuggingFaceEmbedder import HuggingFaceEmbedder, HuggingFaceEmbedderConfig

# Initialize with default model
embedder = HuggingFaceEmbedder()

# Initialize with custom model
config = HuggingFaceEmbedderConfig(
    embedding_model='sentence-transformers/all-MiniLM-L6-v2'
)
embedder = HuggingFaceEmbedder(config=config)

# Initialize with larger model
config = HuggingFaceEmbedderConfig(
    embedding_model='sentence-transformers/all-mpnet-base-v2'
)
embedder = HuggingFaceEmbedder(config=config)
```

--------------------------------

### LLM and Embedder Configuration for Speed

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Configure LLM and embedder settings for performance on limited hardware. This uses a deterministic temperature and a lightweight embedding model.

```python
# Use small, fast model and settings
llm_config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.3,      # Deterministic
    max_tokens=512        # Short responses
)

embedder_config = HuggingFaceEmbedderConfig(
    embedding_model='sentence-transformers/all-MiniLM-L6-v2'  # Lightweight
)
```

--------------------------------

### Monitor System Resources

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Use system monitoring tools to observe CPU, memory, and application logs during operation.

```bash
# Terminal 1: Watch CPU/Memory
top

# Terminal 2: Monitor llama.cpp
tail -f llama.log

# Terminal 3: Run application
python app.py
```

--------------------------------

### Initialize HuggingFaceEmbedderConfig

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Sets up the HuggingFaceEmbedderConfig with a specified embedding model and initializes the HuggingFaceEmbedder.

```python
from HuggingFaceEmbedder import HuggingFaceEmbedderConfig, HuggingFaceEmbedder

config = HuggingFaceEmbedderConfig(
    embedding_model='dunzhang/stella_en_1.5B_v5',
    api_key=None,
    base_url=None
)
embedder = HuggingFaceEmbedder(config=config)
```

--------------------------------

### LocalAiClient Configuration with Grammar Enforcement

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Configures LocalAiClient with a specific temperature and a grammar file to constrain output format.

```python
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient

config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.5
)
client = LocalAiClient(config=config, grammar_file="./json.gbnf")
```

--------------------------------

### Format Prompt for Mistral with LocalAiClient

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/LocalAiClient.md

Demonstrates formatting a list of messages into the Mistral chat template format. This is useful when interacting with LLMs that expect prompts in this specific structure.

```python
from graphiti_core.prompts import Message
from LocalAiClient import LocalAiClient
from graphiti_core.llm_client.config import LLMConfig

config = LLMConfig(base_url="http://127.0.0.1:8080/completion")
client = LocalAiClient(config=config)

messages = [
    Message(role="system", content="Extract entities"),
    Message(role="user", content="Alice works at Acme")
]

formatted = client.format_prompt_for_mistral(messages)
print(formatted)
# Output: <s>[INST]
Extract entities
[/INST]
[INST]
Follow the instructions...
```

--------------------------------

### Configure Default llama.cpp Endpoint

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Configure the LLMConfig to connect to a default local llama.cpp server. Ensure the base URL and port match the server's configuration.

```python
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient

# Default llama.cpp server
config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7,
    max_tokens=2048
)
client = LocalAiClient(config=config)
```

--------------------------------

### Troubleshooting: Restart llama.cpp Server

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Command to restart the llama.cpp server if connection issues arise. Ensure the model path and port are correct.

```bash
# In another terminal:
./llama-server -m ~/ai_models/model.gguf -c 4096 --port 8080
```

--------------------------------

### Access Neo4j CLI

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Command to open the Neo4j command-line interface shell. Use this to test direct connection and authentication.

```bash
neo4j-cli shell
```

--------------------------------

### Full Graphiti Application Workflow

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Orchestrates the entire knowledge graph workflow, including LLM client initialization, database connection, adding episodes, and performing a search query.

```python
async def doSearch():
    llm_config = LLMConfig( base_url="http://127.0.0.1:8080/completion", temperature=0.7 )
    local_llm_client = LocalAiClient.LocalAiClient( config=llm_config, grammar_file="./json.gbnf" )
    neo4j_db_name = os.environ['NEO4J_DB']
    neo4j_db_pass = os.environ['NEO4J_DB_PASS']
    graphiti = Graphiti("bolt://localhost:7687", neo4j_db_name, neo4j_db_pass, embedder=HuggingFaceEmbedder.HuggingFaceEmbedder(), llm_client=local_llm_client )
    
    await graphiti.build_indices_and_constraints()
    
    episodes = [
        "Kamala Harris is the Attorney General of California. She was previously "
        "the district attorney for San Francisco.",
        "As AG, Harris was in office from January 3, 2011 – January 3, 2017",
    ]
    print('embbed episodes')
    for i, episode in enumerate(episodes):
        await graphiti.add_episode(
            name=f"Freakonomics Radio {i}",
            episode_body=episode,
            source=EpisodeType.text,
            source_description="podcast",
            reference_time=datetime.now()
        )

    results = await graphiti.search('Who was the California Attorney General?')
    await graphiti.close()
    
    print( 'Results: ' )
    print( results )
```

--------------------------------

### Configure LLM connection URL

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Python code to initialize LLMConfig with a base URL. Verify this URL matches the running llama.cpp server address.

```python
config = LLMConfig(base_url="http://127.0.0.1:8080/completion")
```

--------------------------------

### Verify Working Directory

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Ensure the current working directory is the root of the local_graphiti project.

```bash
pwd
# Should be /path/to/local_graphiti/
```

--------------------------------

### Initialize LLMConfig for LocalAiClient

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Initializes the LLMConfig object with essential parameters for connecting to a local LLM inference endpoint.

```python
from graphiti_core.llm_client.config import LLMConfig

config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7,
    max_tokens=2048
)
```

--------------------------------

### LocalAiClient Structure

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/ARCHITECTURE.md

Overview of the LocalAiClient class, its methods, and key overrides for interacting with local LLM inference engines.

```text
graphiti_core.llm_client.client.LLMClient
    ↓
    └── LocalAiClient (custom extension)
            ├── __init__(config, grammar_file)
            ├── read_grammar_file()
            ├── _generate_response()          [async]
            ├── execute_llm_query()
            ├── format_prompt_for_mistral()
            ├── format_prompt_for_chatml()
            └── format_prompt_for_deepseek()
```

--------------------------------

### Generate Response with LocalAiClient

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/LocalAiClient.md

Demonstrates how to initialize LocalAiClient, define messages, and generate a response from the LLM. This method handles the entire process from sending messages to parsing the JSON response.

```python
import asyncio
from graphiti_core.prompts import Message
from LocalAiClient import LocalAiClient
from graphiti_core.llm_client.config import LLMConfig

async def test_generate():
    config = LLMConfig(base_url="http://127.0.0.1:8080/completion")
    client = LocalAiClient(config=config)
    
    messages = [
        Message(role="system", content="Extract entities from text"),
        Message(role="user", content="Alice works at Acme Corp")
    ]
    
    response = await client._generate_response(messages)
    print(response)  # {"extracted_nodes": [...]} 

asyncio.run(test_generate())
```

--------------------------------

### Verify Neo4j connection URL in Python

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/TROUBLESHOOTING.md

Python code snippet showing how to initialize the Graphiti client with a Neo4j connection URL. Ensure the bolt URL and port are correct.

```python
graphiti = Graphiti("bolt://localhost:7687", ...)
```

--------------------------------

### LLM and Embedder Configuration for Quality

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Configure LLM and embedder settings for higher quality results when more hardware is available. This uses a more creative temperature and a higher quality embedding model.

```python
# Use larger model and settings
llm_config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7,      # More creative
    max_tokens=4096       # Longer responses
)

embedder_config = HuggingFaceEmbedderConfig(
    embedding_model='sentence-transformers/all-mpnet-base-v2'  # Higher quality
)
```

--------------------------------

### Troubleshooting: Activate Virtual Environment

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/QUICKSTART.md

Command to activate the Python virtual environment if encountering 'Module not found' errors. Ensure the venv name is correct.

```bash
source kg_venv/bin/activate
```

--------------------------------

### Integrate HuggingFaceEmbedder with Graphiti

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/HuggingFaceEmbedder.md

Demonstrates how to integrate HuggingFaceEmbedder and LocalAiClient with the Graphiti knowledge graph builder by configuring and instantiating them.

```python
from graphiti_core import Graphiti
from graphiti_core.llm_client.config import LLMConfig
from LocalAiClient import LocalAiClient
from HuggingFaceEmbedder import HuggingFaceEmbedder, HuggingFaceEmbedderConfig

# Configure embedder
embedder_config = HuggingFaceEmbedderConfig(
    embedding_model='sentence-transformers/all-MiniLM-L6-v2'
)
embedder = HuggingFaceEmbedder(config=embedder_config)

# Configure LLM client
llm_config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.7
)
llm_client = LocalAiClient(config=llm_config)

# Create Graphiti instance with both
graphiti = Graphiti(
    neo4j_url="bolt://localhost:7687",
    neo4j_user="neo4j",
    neo4j_password="password",
    embedder=embedder,
    llm_client=llm_client
)
```

--------------------------------

### Set Custom HuggingFace Cache Directory

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/HuggingFaceEmbedder.md

Shows how to set the HF_HOME environment variable to customize the cache location for downloaded HuggingFace models.

```bash
export HF_HOME=~/custom_hf_cache
python app.py  # Will use custom cache directory
```

--------------------------------

### LocalAiClient Configuration for Deterministic Responses

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Optimizes LocalAiClient for faster, more predictable responses by setting a low temperature and limiting max tokens.

```python
config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=0.1,  # Very deterministic
    max_tokens=1024   # Shorter responses, faster inference
)
```

--------------------------------

### Initialize LocalAiClient Without Grammar

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/api-reference/sample-app.md

Initializes the LocalAiClient without specifying a grammar file. This is useful for testing or when JSON grammar enforcement is not required for the LLM's output.

```python
local_llm_client = LocalAiClient.LocalAiClient(config=llm_config)
# No grammar_file parameter
```

--------------------------------

### LocalAiClient Configuration for Creative Responses

Source: https://github.com/tonymantoan/local_graphiti/blob/main/_autodocs/configuration.md

Configures LocalAiClient for more varied and creative output by increasing temperature and allowing longer responses.

```python
config = LLMConfig(
    base_url="http://127.0.0.1:8080/completion",
    temperature=1.5,  # More randomness
    max_tokens=4096   # Longer responses allowed
)
```