### Install Frontend Dependencies and Start Development Server

Source: https://github.com/hkuds/videorag/blob/main/Vimo-desktop/README.md

In the project root directory, install frontend dependencies using pnpm and then start the development server.

```bash
# Install dependencies
pnpm install

# Start development server
pnpm dev
```

--------------------------------

### LLM Configuration Examples

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/llm-config.md

Examples demonstrating how to initialize VideoRAG with different LLM configurations, including default OpenAI, cost-optimized, and custom settings.

```APIDOC
## Using Default OpenAI Configuration

```python
from videorag import VideoRAG
from videorag._llm import openai_config

rag = VideoRAG(llm=openai_config)
# Automatically uses GPT-4o for best_model and GPT-4o mini for cheap_model
```

## Using Cost-Optimized Configuration

```python
from videorag import VideoRAG
from videorag._llm import openai_4o_mini_config

rag = VideoRAG(llm=openai_4o_mini_config)
# Uses GPT-4o mini exclusively for cost savings
```

## Custom LLM Configuration

```python
from videorag import VideoRAG
from videorag._llm import LLMConfig, openai_embedding, gpt_4o_complete

custom_config = LLMConfig(
    embedding_func_raw=openai_embedding,
    embedding_model_name="text-embedding-3-large",
    embedding_dim=3072,
    embedding_max_token_size=8192,
    embedding_batch_num=16,
    embedding_func_max_async=8,
    query_better_than_threshold=0.15,
    
    best_model_func_raw=gpt_4o_complete,
    best_model_name="gpt-4o",
    best_model_max_token_size=32768,
    best_model_max_async=8,
    
    cheap_model_func_raw=gpt_4o_complete,
    cheap_model_name="gpt-4o",
    cheap_model_max_token_size=32768,
    cheap_model_max_async=8
)

rag = VideoRAG(llm=custom_config)
```
```

--------------------------------

### Start Vimo Backend API Server

Source: https://github.com/hkuds/videorag/blob/main/Vimo-desktop/README.md

Navigate to the backend directory and execute the Python script to start the VideoRAG API server.

```bash
# Navigate to backend directory and start API server
cd python_backend
python videorag_api.py
```

--------------------------------

### Usage Example for NetworkXStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Demonstrates how to initialize NetworkXStorage, add entities and relationships, and query for node edges.

```python
graph = NetworkXStorage(
    namespace="entity_relation",
    global_config=config
)

# Add entity
await graph.upsert_node("ALBERT_EINSTEIN", {
    "entity_type": "PERSON",
    "description": "Physicist known for relativity theory",
    "source_id": "chunk-1;chunk-3",
})

# Add relationship
await graph.upsert_edge("ALBERT_EINSTEIN", "PHYSICS", {
    "weight": 0.9,
    "description": "Contributed to",
    "source_id": "chunk-1",
})

# Query
edges = await graph.get_node_edges("ALBERT_EINSTEIN")
# Returns [("ALBERT_EINSTEIN", "PHYSICS")]
```

--------------------------------

### Install VideoRAG Dependencies

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/README.md

Navigate to the VideoRAG-algorithm directory and install the required Python packages using pip.

```bash
cd VideoRAG-algorithm
pip install -r requirements.txt
```

--------------------------------

### Python Example: Using split_video

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/video-processing.md

Demonstrates how to call the split_video function with specified parameters and how to access the returned segment names and timing information.

```python
segment_names, segment_info = split_video(
    video_path="/videos/lecture.mp4",
    working_dir="./videorag_cache",
    segment_length=30,
    num_frames_per_segment=5,
    audio_output_format='mp3'
)

# Access segment information
for idx, name in segment_names.items():
    frame_times = segment_info[idx]["frame_times"]
    start, end = segment_info[idx]["timestamp"]
    print(f"Segment {idx}: {start}s-{end}s, frames at {frame_times}")

```

--------------------------------

### Usage Example for Configuration File

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

Import and use the configuration function from your config file to initialize VideoRAG and perform operations like inserting videos and querying.

```python
from config import get_videorag_config

rag = get_videorag_config()
rag.insert_video(["video.mp4"])
response = rag.query("What is this about?")
```

--------------------------------

### Python Example: Using saving_video_segments with Multiprocessing

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/video-processing.md

Shows how to use saving_video_segments within a multiprocessing.Process. It includes starting the process, joining it, and checking an error queue for any issues during segment saving.

```python
from multiprocessing import Process, Queue

segment_names, segment_info = split_video(...)

error_queue = Queue()
process = Process(
    target=saving_video_segments,
    args=(
        "lecture1",
        "/videos/lecture.mp4",
        "./videorag_cache",
        segment_names,
        segment_info,
        error_queue,
        "mp4"
    )
)
process.start()
process.join()

# Check for errors
if not error_queue.empty():
    error = error_queue.get()
    raise RuntimeError(f"Video saving failed: {error}")

```

--------------------------------

### Download Model Checkpoints

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Instructions to download pre-trained model checkpoints for MiniCPM-V, Whisper, and ImageBind using git lfs and wget. Ensure git-lfs is installed first.

```bash
git lfs install
```

```bash
git lfs clone https://huggingface.co/openbmb/MiniCPM-V-2_6-int4
```

```bash
git lfs clone https://huggingface.co/Systran/faster-distil-whisper-large-v3
```

```bash
mkdir .checkpoints
cd .checkpoints
wget https://dl.fbaipublicfiles.com/imagebind/imagebind_huge.pth
cd ../
```

--------------------------------

### Set Up Python Environment for Vimo Backend

Source: https://github.com/hkuds/videorag/blob/main/Vimo-desktop/README.md

Create and activate a Conda environment for Vimo, then install core dependencies including numerical, deep learning, video processing, multi-modal, audio, language model, and server libraries.

```bash
conda create --name vimo python=3.11
conda activate vimo

# Core numerical and deep learning libraries
pip install numpy==1.26.4 torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2

# Video processing utilities
pip install moviepy==1.0.3
pip install git+https://github.com/Re-bin/pytorchvideo.git@58f50da4e4b7bf0b17b1211dc6b283ba42e522df
pip install --no-deps git+https://github.com/facebookresearch/ImageBind.git@3fcf5c9039de97f6ff5528ee4a9dce903c5979b3

# Multi-modal and vision libraries
pip install timm ftfy regex einops fvcore eva-decord==0.6.1 iopath matplotlib types-regex cartopy

# Audio processing and vector databases
pip install neo4j hnswlib xxhash nano-vectordb

# Language models and utilities
pip install tiktoken openai tenacity dashscope

# Server
pip install flask psutil flask_cors setproctitle
```

--------------------------------

### Example Usage of Entity Extraction

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/entity-extraction.md

Demonstrates how to initialize storage instances and call the `extract_entities` function with sample data. It includes basic error handling and prints the number of extracted entities and relationships.

```python
from videorag._op import extract_entities
from videorag._storage import NetworkXStorage, NanoVectorDBStorage

graph = NetworkXStorage(namespace="kg", global_config=config)
entity_vdb = NanoVectorDBStorage(
    namespace="entities",
    global_config=config,
    embedding_func=embedding_func
)

chunks = {
    "chunk-1": {
        "tokens": 950,
        "content": "Albert Einstein published the theory of relativity...",
        "video_segment_id": ["seg_0"],
        "chunk_order_index": 0
    }
}

result = await extract_entities(
    chunks=chunks,
    knowledge_graph_inst=graph,
    entity_vdb=entity_vdb,
    global_config=config
)

if result is not None:
    graph, entities, relationships = result
    print(f"Extracted {len(entities)} entities and {len(relationships)} relationships")
```

--------------------------------

### Initialize VideoRAG with Custom LLM Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/llm-config.md

Configure VideoRAG with a custom LLM setup, allowing fine-grained control over embedding and completion models, their parameters, and caching behavior.

```python
from videorag import VideoRAG
from videorag._llm import LLMConfig, openai_embedding, gpt_4o_complete

custom_config = LLMConfig(
    embedding_func_raw=openai_embedding,
    embedding_model_name="text-embedding-3-large",
    embedding_dim=3072,
    embedding_max_token_size=8192,
    embedding_batch_num=16,
    embedding_func_max_async=8,
    query_better_than_threshold=0.15,
    
    best_model_func_raw=gpt_4o_complete,
    best_model_name="gpt-4o",
    best_model_max_token_size=32768,
    best_model_max_async=8,
    
    cheap_model_func_raw=gpt_4o_complete,
    cheap_model_name="gpt-4o",
    cheap_model_max_token_size=32768,
    cheap_model_max_async=8
)

rag = VideoRAG(llm=custom_config)
```

--------------------------------

### Install Core Dependencies for VideoRAG

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Installs essential Python packages for VideoRAG, including numerical libraries, deep learning frameworks, video processing tools, multi-modal libraries, audio processing tools, and language model utilities.

```bash
pip install numpy==1.26.4
```

```bash
pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2
```

```bash
pip install accelerate==0.30.1
```

```bash
pip install bitsandbytes==0.43.1
```

```bash
pip install moviepy==1.0.3
```

```bash
pip install git+https://github.com/facebookresearch/pytorchvideo.git@28fe037d212663c6a24f373b94cc5d478c8c1a1d
```

```bash
pip install --no-deps git+https://github.com/facebookresearch/ImageBind.git@3fcf5c9039de97f6ff5528ee4a9dce903c5979b3
```

```bash
pip install timm ftfy regex einops fvcore eva-decord==0.6.1 iopath matplotlib types-regex cartopy
```

```bash
pip install ctranslate2==4.4.0 faster_whisper==1.0.3 neo4j hnswlib xxhash nano-vectordb
```

```bash
pip install transformers==4.37.1
```

```bash
pip install tiktoken openai tenacity
```

```bash
pip install ollama==0.5.3
```

--------------------------------

### Local Development VideoRAG Setup

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

This configuration is ideal for local testing and development, utilizing local models via Ollama and minimal processing settings for speed and reduced resource usage.

```python
from videorag import VideoRAG
from videorag._llm import ollama_config

rag = VideoRAG(
    working_dir="./videorag_test",
    
    # Use local models
    llm=ollama_config,
    
    # Minimal processing for speed
    video_segment_length=60,
    rough_num_frames_per_segment=2,
    fine_num_frames_per_segment=3,
    
    # Lower resource usage
    video_embedding_batch_num=1,
    threads_for_split=2,
)
```

--------------------------------

### Set multiprocessing start method

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/notesbooks/videorag.ipynb

Configures the start method for multiprocessing to 'spawn'. This is often necessary for compatibility and stability in certain environments, especially when using libraries that manage subprocesses.

```python
multiprocessing.set_start_method('spawn')
```

--------------------------------

### Example Usage of Speech to Text

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/video-processing.md

Demonstrates how to call the speech_to_text function with specified parameters and iterate through the returned transcripts to print each segment's transcription.

```python
transcripts = speech_to_text(
    video_name="lecture1",
    working_dir="./videorag_cache",
    segment_index2name=segment_names,
    audio_output_format='mp3'
)

for idx, transcript in transcripts.items():
    print(f"Segment {idx}:")
    print(transcript)
```

--------------------------------

### Create VideoRAG Configuration File

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

Define a Python configuration function to instantiate VideoRAG with specific settings for reproducible setups. This includes video processing, retrieval, model, and storage configurations.

```python
# config.py
from videorag import VideoRAG
from videorag._llm import openai_config

def get_videorag_config():
    return VideoRAG(
        working_dir="/data/videorag",
        
        # Video processing
        video_segment_length=30,
        fine_num_frames_per_segment=15,
        
        # Retrieval
        query_better_than_threshold=0.2,
        
        # Models
        llm=openai_config,
        
        # Storage
        enable_llm_cache=True,
    )
```

--------------------------------

### Azure OpenAI Configuration Setup

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/llm-config.md

Configuration for Azure OpenAI, which requires specific environment variables for authentication and endpoint. It supports deployment-based model routing.

```python
from videorag._llm import azure_openai_config

# Requires environment variables:
# AZURE_OPENAI_KEY
# AZURE_OPENAI_ENDPOINT
# AZURE_OPENAI_API_VERSION
```

--------------------------------

### Ollama Configuration Setup

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/llm-config.md

Configuration for Ollama, which requires a running Ollama server. It defaults to http://127.0.0.1:11434 but can be customized via the OLLAMA_HOST environment variable. Supports local model serving.

```python
from videorag._llm import ollama_config

# Requires running Ollama server
# Defaults to http://127.0.0.1:11434
# Can be customized via OLLAMA_HOST environment variable
```

--------------------------------

### Import necessary libraries and configure environment

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/notesbooks/videorag.ipynb

Imports essential libraries for the VideoRAG algorithm and configures the environment by applying nest_asyncio, suppressing warnings, setting logging levels, and specifying the CUDA device. This setup is crucial before initializing any VideoRAG components.

```python
import os
import logging
import warnings
import multiprocessing
import nest_asyncio
    
nest_asyncio.apply()

warnings.filterwarnings("ignore")
logging.getLogger("httpx").setLevel(logging.WARNING)
os.environ["CUDA_VISIBLE_DEVICES"] = '0'

from videorag._llm import  openai_config, openai_4o_mini_config, azure_openai_config, ollama_config
from videorag import VideoRAG, QueryParam
```

--------------------------------

### Cost-Optimized VideoRAG Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

Use this setup for production environments where cost is a primary concern. It prioritizes cheaper LLMs and faster video processing with longer segments and fewer frames.

```python
from videorag import VideoRAG
from videorag._llm import openai_4o_mini_config

rag = VideoRAG(
    working_dir="/data/videorag",
    
    # Video processing - faster
    video_segment_length=60,  # Longer segments
    rough_num_frames_per_segment=3,  # Fewer frames
    fine_num_frames_per_segment=5,
    
    # Retrieval - selective
    query_better_than_threshold=0.3,  # Higher threshold
    segment_retrieval_top_k=2,
    
    # LLM - cheaper
    llm=openai_4o_mini_config,
    
    # Storage - local
    enable_llm_cache=True,  # Cache responses
)
```

--------------------------------

### Example EmbeddingFunc Usage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/types.md

Demonstrates how to wrap an OpenAI embedding function using `wrap_embedding_func_with_attrs` and then call the resulting `EmbeddingFunc` instance.

```python
from videorag._utils import wrap_embedding_func_with_attrs, EmbeddingFunc
from videorag._llm import openai_embedding

embedding_func = wrap_embedding_func_with_attrs(
    embedding_dim=1536,
    max_token_size=8192,
    model_name="text-embedding-3-small"
)(openai_embedding)

# Call it
embeddings = await embedding_func(["text1", "text2"])
# Returns: np.ndarray of shape (2, 1536)
```

--------------------------------

### Speech to Text Return Value Example

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/video-processing.md

Illustrates the expected return format of the speech_to_text function, showing a dictionary where keys are segment indices and values are timestamped transcriptions.

```python
{
    "0": "[0.00s -> 5.34s] Hello world\n[5.34s -> 10.20s] This is a test...",
    "1": "[30.00s -> 35.50s] Continuing the lecture...",
}
```

--------------------------------

### Load caption model and set query parameters

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/notesbooks/videorag.ipynb

Loads the caption model required for video analysis and sets up the query parameters, including the mode. Ensure the caption model is compatible with your setup.

```python
# To query
videorag.load_caption_model(debug=False)
param = QueryParam(mode="videorag")
```

--------------------------------

### Reproduce Quantitative Comparison - Step 2: Download Results

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Downloads the results from OpenAI for the quantitative comparison. Input the batch ID and output file ID as prompted. This may require running the script twice.

```shell
# Second Step: Download the results. Please enter the batch ID and then the output file ID in the file. Generally, you need to run this twice: first to obtain the output file ID, and then to download it.
python batch_quant_eval_download.py
```

--------------------------------

### Initialize and Use VideoRAG

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/videorag.md

Demonstrates how to initialize VideoRAG, load a caption model, process videos, and perform a synchronous query. This is useful for standard video analysis workflows.

```python
from videorag import VideoRAG, QueryParam

# Initialize with default OpenAI config
rag = VideoRAG(working_dir="./videorag_output")

# Load caption model for video understanding
rag.load_caption_model()

# Process videos
rag.insert_video([
    "./videos/lecture1.mp4",
    "./videos/lecture2.mp4",
])

# Query the system
response = rag.query(
    "What are the main topics covered?",
    QueryParam(mode="videorag", top_k=20)
)
print(response)
```

--------------------------------

### LLM Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/README.md

The LLMConfig class manages the setup and configuration of Large Language Models (LLMs) and embedding models used by the framework.

```APIDOC
## LLMConfig Class

### Description
Handles the configuration for various Large Language Models (LLMs) and embedding models, supporting different providers like OpenAI, Azure, and Ollama.

### Supported Providers
- OpenAI
- Azure OpenAI
- Ollama

### Configuration Options
- Specifies which LLM and embedding models to use.
- Configures asynchronous concurrency limits for model calls.
- Sets up API keys and endpoint details for chosen providers.
```

--------------------------------

### Reproduce Quantitative Comparison - Step 1: Upload Batch Request

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Uploads the batch request to OpenAI for quantitative evaluation. Ensure your API key is correctly set in the script.

```shell
cd reproduce/quantitative_comparison

# First Step: Upload the batch request to OpenAI (remember to enter your key in the file, same for the following steps).
python batch_quant_eval_upload.py
```

--------------------------------

### Initialize VideoRAG with Cost-Optimized OpenAI Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/llm-config.md

Initialize VideoRAG using a cost-optimized configuration that exclusively uses GPT-4o mini for all operations, reducing expenses.

```python
from videorag import VideoRAG
from videorag._llm import openai_4o_mini_config

rag = VideoRAG(llm=openai_4o_mini_config)
# Uses GPT-4o mini exclusively for cost savings
```

--------------------------------

### Initialize Neo4jStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Initializes a Neo4jStorage instance for enterprise-grade knowledge graph management. Requires connection details like URL, username, and password.

```python
from videorag._storage import Neo4jStorage

graph_db = Neo4jStorage(
    namespace="chunk_entity_relation",
    global_config=config,
    url="neo4j://localhost:7687",
    username="neo4j",
    password="password"
)
```

--------------------------------

### Initialize NetworkXStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Initializes an in-memory NetworkX graph database for storing knowledge graph information. Requires a namespace and global configuration.

```python
from videorag._storage import NetworkXStorage

graph_db = NetworkXStorage(
    namespace="chunk_entity_relation",
    global_config=config
)
```

--------------------------------

### Initialize VideoRAG with Default OpenAI Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/llm-config.md

Use this to initialize VideoRAG with the default OpenAI configuration, which automatically selects GPT-4o for best performance and GPT-4o mini for cost-effectiveness.

```python
from videorag import VideoRAG
from videorag._llm import openai_config

rag = VideoRAG(llm=openai_config)
# Automatically uses GPT-4o for best_model and GPT-4o mini for cheap_model
```

--------------------------------

### Reproduce Quantitative Comparison - Step 3: Parse Results

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Parses the downloaded results for the quantitative evaluation. The output file ID must be provided within the script.

```shell
# Third Step: Parsing the results. Please the output file ID in the file.
python batch_quant_eval_parse.py
```

--------------------------------

### Large-Scale VideoRAG Deployment Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

Designed for processing hundreds of videos efficiently. This setup emphasizes scalable vector storage and optimized video processing for high throughput.

```python
from videorag import VideoRAG
from videorag._storage import HNSWVectorStorage

rag = VideoRAG(
    working_dir="/mnt/videorag_data",
    
    # Vector storage - scalable
    vector_db_storage_cls=HNSWVectorStorage,
    vector_db_storage_cls_kwargs={
        "max_elements": 1000000,
        "ef": 200,
        "M": 16,
    },
    
    # Video processing - efficient
    video_embedding_batch_num=8,
    threads_for_split=32,
    
    # Async limits - high throughput
    llm=custom_llm_config_with_higher_async,
)
```

--------------------------------

### Import Ollama Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

Import the configuration object for Ollama. Requires an Ollama server to be running and a model downloaded. OLLAMA_HOST can be set if not running locally.

```python
from videorag._llm import ollama_config
```

--------------------------------

### Configure Custom Storage for VideoRAG

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/README.md

Initialize VideoRAG with custom graph and vector database storage backends. Configure parameters for the vector database like max_elements and ef.

```python
from videorag import VideoRAG
from videorag._storage import Neo4jStorage, HNSWVectorStorage

rag = VideoRAG(
    graph_storage_cls=Neo4jStorage,
    vector_db_storage_cls=HNSWVectorStorage,
    vector_db_storage_cls_kwargs={
        "max_elements": 1000000,
        "ef": 200,
    }
)
```

--------------------------------

### NetworkXStorage Core Methods

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Illustrates the core methods for interacting with the NetworkXStorage, such as upserting nodes and edges, and retrieving node edges.

```python
async def upsert_node(self, node_id: str, node_data: dict[str, str]):
    """Insert or update a node in the knowledge graph."""

async def upsert_edge(self, src_id: str, tgt_id: str, 
                      edge_data: dict[str, str]):
    """Insert or update an edge between nodes."""

async def get_node_edges(self, src_id: str) -> list[tuple[str, str]]:
    """Get all outgoing edges from a node."""

async def community_schema(self) -> dict[str, SingleCommunitySchema]:
    """Get detected communities and their structure."""
```

--------------------------------

### Reproduce Win-Rate Comparison - Step 1: Upload Batch Request

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Uploads the batch request to OpenAI for win-rate evaluation. Remember to enter your API key in the script.

```shell
cd reproduce/winrate_comparison

# First Step: Upload the batch request to OpenAI (remember to enter your key in the file, same for the following steps).
python batch_winrate_eval_upload.py
```

--------------------------------

### Manage Token Budget with Tiktoken

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/utility-functions.md

Use encode_string_by_tiktoken to get token counts and truncate_list_by_token_size to ensure content fits within a specified token limit. This is crucial for managing API request sizes.

```python
from videorag._utils import encode_string_by_tiktoken, truncate_list_by_token_size

# Check if text fits in budget
text = "Long document..."
tokens = encode_string_by_tiktoken(text)
if len(tokens) > max_tokens:
    # Truncate to fit
    chunks = truncate_list_by_token_size(chunks, lambda x: x["content"], max_tokens)
```

--------------------------------

### Async Query Multiple Videos

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/query-param.md

Demonstrates how to asynchronously query multiple videos using the VideoRAG library. Ensure VideoRAG is initialized and videos are inserted before querying.

```python
import asyncio

async def query_multiple_videos():
    rag = VideoRAG()
    rag.insert_video(["video1.mp4", "video2.mp4"])
    
    params = QueryParam(
        mode="global",
        top_k=25,
        level=2
    )
    
    results = await rag.aquery("Compare topics across videos", params)
    return results

asyncio.run(query_multiple_videos())
```

--------------------------------

### Get or Create Asyncio Event Loop Safely

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/utility-functions.md

Safely retrieves an asyncio event loop, creating a new one in threads if necessary. This is essential for using asyncio within multiprocessing contexts.

```python
from videorag._utils import always_get_an_event_loop

# In main thread
loop = always_get_an_event_loop()
result = loop.run_until_complete(async_function())

# In subprocess
def worker():
    loop = always_get_an_event_loop()  # Creates new loop
    # Use loop...

```

--------------------------------

### Prepare LongerVideos Dataset

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Commands to prepare the LongerVideos dataset by creating collection folders and downloading videos. Navigate to the 'longervideos' directory before running these commands.

```shell
cd longervideos
python prepare_data.py # create collection folders
sh download.sh # obtain videos
```

--------------------------------

### Python Data Structure: segment_times_info

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/video-processing.md

Details the segment_times_info dictionary, which contains frame times and timestamp tuples for each segment. Frame times are provided as a list of floats, and timestamps as a tuple of start and end seconds.

```python
segment_times_info = {
    "0": {
        "frame_times": [0.0, 6.0, 12.0, 18.0, 24.0],  # Numpy array
        "timestamp": (0, 30)  # (start_sec, end_sec)
    },
    "1": {
        "frame_times": [30.0, 36.0, 42.0, 48.0, 54.0],
        "timestamp": (30, 60)
    }
}

```

--------------------------------

### Reproduce Win-Rate Comparison - Step 3: Parse Results

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Parses the downloaded results from the win-rate evaluation. You need to provide the output file ID in the script.

```shell
# Third Step: Parsing the results. Please the output file ID in the file.
python batch_winrate_eval_parse.py
```

--------------------------------

### Initialize and Use Module Logger

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/utility-functions.md

Demonstrates how to import and use the module-level logger instance for different log levels. The log prefix is 'nano-graphrag'.

```python
from videorag._utils import logger

logger.info("Processing video...")
logger.warning("Audio track not found")
logger.error("Failed to process segment")
logger.debug("Token count: 1234")
```

--------------------------------

### Reproduce Quantitative Comparison - Step 4: Calculate Results

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Calculates the final quantitative comparison scores. Specify the parsed result file name in the script to complete the process.

```shell
# Fourth Step: Calculate the results. Please enter the parsed result file name in the file.
python batch_quant_eval_calculate.py
```

--------------------------------

### Initialize NanoVectorDBStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Initializes a vector database for text embeddings using Nano-VectorDB. Supports efficient similarity search and metadata filtering.

```python
from videorag._storage import NanoVectorDBStorage

vector_db = NanoVectorDBStorage(
    namespace="chunks",
    global_config=config,
    embedding_func=embedding_function,
    meta_fields={"field_name"}
)
```

--------------------------------

### Initialize VideoRAG with LLM and working directory

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/notesbooks/videorag.ipynb

Initializes the VideoRAG object, specifying the language model configuration (e.g., ollama_config) and the working directory for storing intermediate files and results. Choose an appropriate LLM and working directory for your use case.

```python
videorag = VideoRAG(llm=ollama_config, working_dir=f"./videorag-workdir/lexington")
```

--------------------------------

### Reproduce Win-Rate Comparison - Step 2: Download Results

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Downloads the results from OpenAI after the batch request has been processed. You will need to enter the batch ID and output file ID. This step may need to be run twice.

```shell
# Second Step: Download the results. Please enter the batch ID and then the output file ID in the file. Generally, you need to run this twice: first to obtain the output file ID, and then to download it.
python batch_winrate_eval_download.py
```

--------------------------------

### Combine Multiple Queries

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/query-param.md

Illustrates a two-step query process: first retrieving context with `only_need_context=True`, then using that context to synthesize a detailed answer. This approach is useful for complex information gathering.

```python
rag = VideoRAG()
rag.insert_video(["video.mp4"])

# First, get context
context_param = QueryParam(only_need_context=True, mode="naive")
context = rag.query("What topics are covered?", context_param)

# Then, get synthesized answer
answer_param = QueryParam(mode="global", top_k=20)
answer = rag.query("Explain these topics in detail", answer_param)
```

--------------------------------

### Reproduce Win-Rate Comparison - Step 4: Calculate Results

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Calculates the final win-rate comparison results. You need to specify the parsed result file name in the script.

```shell
# Fourth Step: Calculate the results. Please enter the parsed result file name in the file.
python batch_winrate_eval_calculate.py
```

--------------------------------

### Initialize JsonKVStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Initializes a file-based key-value storage using JSON format. Useful for simple persistence of human-readable data.

```python
from videorag._storage import JsonKVStorage

storage = JsonKVStorage(
    namespace="my_namespace",
    global_config={...}
)
```

--------------------------------

### Basic Video Query

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/query-param.md

Demonstrates a simple query to the VideoRAG system using default parameters.

```python
from videorag import VideoRAG, QueryParam

rag = VideoRAG()
rag.insert_video(["video.mp4"])

# Use default parameters
response = rag.query("What is discussed in the video?")
```

--------------------------------

### Query VideoRAG with Different Patterns

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/README.md

Demonstrates various query modes and parameter configurations for factual, comprehensive, quick searches, and multiple-choice questions. Adjust top_k and level for retrieval scope.

```python
# Factual question
param = QueryParam(mode="local", top_k=15)
answer = rag.query("Who is mentioned in the video?", param)
```

```python
# Comprehensive question
param = QueryParam(mode="global", top_k=30, level=2)
answer = rag.query("Explain the main concepts", param)
```

```python
# Quick search (no LLM synthesis)
param = QueryParam(only_need_context=True, mode="naive")
context = rag.query("Find relevant segments", param)
```

```python
# Multiple choice
param = QueryParam(mode="local", response_type="Bullet Points")
answer = rag.query("What are the options?", param)
```

--------------------------------

### Custom Storage Implementation in VideoRAG

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Implement custom storage solutions by inheriting from `BaseKVStorage` and implementing all abstract methods. This allows for flexible storage configurations within VideoRAG.

```python
from videorag.base import BaseKVStorage

@dataclass
class MyCustomStorage(BaseKVStorage):
    async def all_keys(self) -> list[str]:
        # Implementation
        pass
    
    async def get_by_id(self, id: str):
        # Implementation
        pass
    
    # Implement all abstract methods...

# Use in VideoRAG
rag = VideoRAG(
    key_string_value_json_storage_cls=MyCustomStorage
)
```

--------------------------------

### Initialize HNSWVectorStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Initializes an HNSWVectorStorage for scalable similarity search. Requires a namespace, global configuration, an embedding function, and index parameters.

```python
from videorag._storage import HNSWVectorStorage

hnsw_db = HNSWVectorStorage(
    namespace="embeddings",
    global_config=config,
    embedding_func=embedding_function,
    max_elements=100000,
    ef=200,
    M=16
)
```

--------------------------------

### Asynchronous Query with VideoRAG

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/videorag.md

Shows how to perform an asynchronous query using VideoRAG. This is beneficial for applications requiring non-blocking operations.

```python
import asyncio
result = asyncio.run(rag.aquery(
    "Who are the key speakers?",
    QueryParam(mode="videorag")
))
```

--------------------------------

### Initialize NanoVectorDBVideoSegmentStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Initializes a specialized vector storage for video segment embeddings using ImageBind. Supports multimodal embeddings and CUDA acceleration.

```python
from videorag._storage import NanoVectorDBVideoSegmentStorage

video_vdb = NanoVectorDBVideoSegmentStorage(
    namespace="video_segment_feature",
    global_config=config,
    embedding_func=None  # Handled internally
)
```

--------------------------------

### Usage of JsonKVStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Demonstrates storing and retrieving video paths using JsonKVStorage. Operations are synchronous in-memory with an async interface.

```python
# Store video paths
video_db = JsonKVStorage(namespace="video_path", global_config=config)
await video_db.upsert({"video1": "/path/to/video1.mp4"})
path = await video_db.get_by_id("video1")
```

--------------------------------

### Configure OpenAI LLM

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

Set up the LLMConfig for OpenAI, specifying embedding and completion models, dimensions, token limits, and batch sizes. Ensure OPENAI_API_KEY is set.

```python
from videorag._llm import openai_config

openai_config = LLMConfig(
    # Embedding
    embedding_func_raw=openai_embedding,
    embedding_model_name="text-embedding-3-small",
    embedding_dim=1536,
    embedding_max_token_size=8192,
    embedding_batch_num=32,
    embedding_func_max_async=16,
    query_better_than_threshold=0.2,
    
    # Best model (GPT-4o)
    best_model_func_raw=gpt_4o_complete,
    best_model_name="gpt-4o",
    best_model_max_token_size=32768,
    best_model_max_async=16,
    
    # Cheap model (GPT-4o mini)
    cheap_model_func_raw=gpt_4o_mini_complete,
    cheap_model_name="gpt-4o-mini",
    cheap_model_max_token_size=32768,
    cheap_model_max_async=16
)
```

--------------------------------

### Usage of NanoVectorDBStorage

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/storage-classes.md

Demonstrates initializing, upserting, and querying text embeddings with NanoVectorDBStorage. Requires an embedding function and configuration.

```python
# Initialize
embedding_func = wrap_embedding_func_with_attrs(
    embedding_dim=1536,
    max_token_size=8192,
    model_name="text-embedding-3-small"
)(openai_embedding)

vdb = NanoVectorDBStorage(
    namespace="chunks",
    global_config=config,
    embedding_func=embedding_func
)

# Insert chunks
chunks = {
    "chunk-1": {"content": "The quick brown fox..."},
    "chunk-2": {"content": "Another text snippet..."},
}
await vdb.upsert(chunks)

# Query
results = await vdb.query("fox", top_k=2)
# Returns top 2 similar chunks
```

--------------------------------

### Download Evaluation Answers

Source: https://github.com/hkuds/videorag/blob/main/VideoRAG-algorithm/README.md

Commands to download pre-computed answers for all methods used in the evaluation. These are needed for reproducing win-rate and quantitative comparisons.

```shell
cd reproduce
wget https://archive.org/download/videorag/all_answers.zip
unzip all_answers
```

--------------------------------

### Storage Implementations

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/README.md

Provides interfaces and concrete implementations for different storage backends, including key-value, vector, and graph databases.

```APIDOC
## Storage Interfaces and Implementations

### Description
Defines abstract interfaces for key-value, vector, and graph storage, along with concrete implementations for various database systems.

### Interfaces
- `BaseKVStorage`: Interface for key-value storage.
- `BaseVectorStorage`: Interface for vector database storage.
- `BaseGraphStorage`: Interface for knowledge graph storage.

### Concrete Implementations
- JSON storage
- NanoVectorDB for vector search
- NetworkX and Neo4j for graph storage
- HNSW (Hierarchical Navigable Small Worlds) for efficient vector indexing.
```

--------------------------------

### VideoRAG Class Initialization

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/videorag.md

Instantiate the VideoRAG class with various configuration parameters to customize the video processing and retrieval pipeline.

```APIDOC
## VideoRAG Class Definition

```python
from videorag import VideoRAG, QueryParam

@dataclass
class VideoRAG:
    working_dir: str
    threads_for_split: int
    video_segment_length: int
    rough_num_frames_per_segment: int
    fine_num_frames_per_segment: int
    video_output_format: str
    audio_output_format: str
    video_embedding_batch_num: int
    segment_retrieval_top_k: int
    video_embedding_dim: int
    retrieval_topk_chunks: int
    query_better_than_threshold: float
    enable_local: bool
    enable_naive_rag: bool
    chunk_func: Callable
    chunk_token_size: int
    tiktoken_model_name: str
    entity_extract_max_gleaning: int
    entity_summary_to_max_tokens: int
    llm: LLMConfig
    entity_extraction_func: callable
    key_string_value_json_storage_cls: Type[BaseKVStorage]
    vector_db_storage_cls: Type[BaseVectorStorage]
    vs_vector_db_storage_cls: Type[BaseVectorStorage]
    vector_db_storage_cls_kwargs: dict
    graph_storage_cls: Type[BaseGraphStorage]
    enable_llm_cache: bool
    always_create_working_dir: bool
    addon_params: dict
    convert_response_to_json_func: callable
```

## Configuration Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `working_dir` | str | `videorag_cache_{timestamp}` | Root directory for storing cache, embeddings, and processed data |
| `threads_for_split` | int | 10 | Number of threads for video splitting operations |
| `video_segment_length` | int | 30 | Length of each video segment in seconds |
| `rough_num_frames_per_segment` | int | 5 | Number of frames extracted per segment for initial processing |
| `fine_num_frames_per_segment` | int | 15 | Number of frames extracted per segment for detailed analysis |
| `video_output_format` | str | `"mp4"` | Output format for processed video segments |
| `audio_output_format` | str | `"mp3"` | Output format for extracted audio tracks |
| `video_embedding_batch_num` | int | 2 | Batch size for video embedding operations |
| `segment_retrieval_top_k` | int | 4 | Number of top-k video segments to retrieve for queries |
| `video_embedding_dim` | int | 1024 | Dimension of video embedding vectors |
| `retrieval_topk_chunks` | int | 2 | Number of top-k text chunks to retrieve for queries |
| `query_better_than_threshold` | float | 0.2 | Minimum similarity threshold for retrieved results |
| `enable_local` | bool | True | Enable local graph-based entity retrieval |
| `enable_naive_rag` | bool | True | Enable naive RAG using text chunks only |
| `chunk_func` | Callable | `chunking_by_video_segments` | Function for chunking text tokens into coherent units |
| `chunk_token_size` | int | 1200 | Maximum token size per text chunk |
| `tiktoken_model_name` | str | `"gpt-4o"` | Tokenization model for encoding text |
| `entity_extract_max_gleaning` | int | 1 | Number of refinement iterations for entity extraction |
| `entity_summary_to_max_tokens` | int | 500 | Maximum tokens for entity summaries |
| `llm` | LLMConfig | `openai_config` | LLM configuration including models and embedding functions |
| `entity_extraction_func` | callable | `extract_entities` | Function for extracting entities and relationships from chunks |
| `key_string_value_json_storage_cls` | Type | `JsonKVStorage` | Storage backend for key-value data |
| `vector_db_storage_cls` | Type | `NanoVectorDBStorage` | Vector database for text embeddings |
| `vs_vector_db_storage_cls` | Type | `NanoVectorDBVideoSegmentStorage` | Vector database for video segment embeddings |
| `vector_db_storage_cls_kwargs` | dict | `{}` | Additional kwargs for vector storage initialization |
| `graph_storage_cls` | Type | `NetworkXStorage` | Graph storage backend for knowledge graphs |
| `enable_llm_cache` | bool | True | Enable caching of LLM responses to reduce API calls |
| `always_create_working_dir` | bool | True | Create working directory if it doesn't exist |
| `addon_params` | dict | `{}` | Additional extension parameters |
| `convert_response_to_json_func` | callable | `convert_response_to_json` | Function for parsing LLM responses to JSON |
```

--------------------------------

### Import Azure OpenAI Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/configuration.md

Import the configuration object for Azure OpenAI. Requires AZURE_OPENAI_KEY, AZURE_OPENAI_ENDPOINT, and AZURE_OPENAI_API_VERSION environment variables.

```python
from videorag._llm import azure_openai_config
```

--------------------------------

### OpenAI 4o Mini Configuration

Source: https://github.com/hkuds/videorag/blob/main/_autodocs/api-reference/llm-config.md

A cost-optimized OpenAI configuration using GPT-4o mini for both best and cheap models. It utilizes the same embedding model as the standard configuration and is suitable for lower-cost deployments.

```python
from videorag._llm import openai_4o_mini_config

openai_4o_mini_config = LLMConfig(
    embedding_func_raw=openai_embedding,
    embedding_model_name="text-embedding-3-small",
    embedding_dim=1536,
    embedding_max_token_size=8192,
    embedding_batch_num=32,
    embedding_func_max_async=16,
    query_better_than_threshold=0.2,
    
    best_model_func_raw=gpt_4o_mini_complete,
    best_model_name="gpt-4o-mini",
    best_model_max_token_size=32768,
    best_model_max_async=16,
    
    cheap_model_func_raw=gpt_4o_mini_complete,
    cheap_model_name="gpt-4o-mini",
    cheap_model_max_token_size=32768,
    cheap_model_max_async=16
)
```