### Start vLLM Server using 'serve' command

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

Start a vLLM server compatible with the OpenAI API using the 'vllm serve' command. Specify the model path and desired configurations.

```bash
# Method 1: Using vllm serve command
vllm serve /path/to/your/model \
  --tensor-parallel-size 1 \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.95
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

Use this command to install all necessary Python packages listed in the requirements.txt file. Ensure you have Python 3.10 or above installed.

```bash
pip install -r requirements.txt
```

--------------------------------

### Example QA File Format

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

This is an example of a single line in the qas.jsonl file. Each line represents a question with fields like id, question, and golden_answers.

```json
{"id": "1", "question": "...", "golden_answers": ["..."]}
```

--------------------------------

### Start vLLM Server using Python module

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

Start a vLLM server compatible with the OpenAI API by running the vLLM module directly. This method allows for more detailed configuration options.

```bash
# Method 2: Using python -m vllm.entrypoints.openai.api_server
python -m vllm.entrypoints.openai.api_server \
  --model /path/to/your/model \
  --served-model-name your-model-name \
  --tensor-parallel-size 1 \
  --max-model-len 32768 \
  --dtype auto
```

--------------------------------

### Complete ComoRAG Pipeline Example

Source: https://context7.com/eternityjune25/comorag/llms.txt

Demonstrates the full ComoRAG pipeline, including loading a dataset, initializing the ComoRAG model, indexing documents, answering queries, and saving results. Requires a dataset with 'corpus.jsonl' and 'qas.jsonl'. Ensure the OPENAI_API_KEY environment variable is set.

```python
import os
import json
import copy
from src.comorag.ComoRAG import ComoRAG
from src.comorag.utils.config_utils import BaseConfig
from src.comorag.utils.misc_utils import get_gold_answers

def process_dataset(dataset_path, config):
    """Process a dataset through the complete ComoRAG pipeline."""

    # Load corpus
    corpus_path = os.path.join(dataset_path, "corpus.jsonl")
    with open(corpus_path, 'r', encoding='utf-8') as f:
        corpus = [json.loads(line) for line in f if line.strip()]
    docs = [doc['contents'] for doc in corpus]

    # Load questions
    qas_path = os.path.join(dataset_path, "qas.jsonl")
    with open(qas_path, 'r', encoding='utf-8') as f:
        samples = [json.loads(line) for line in f if line.strip()]
    queries = [s['question'] for s in samples]

    # Update config with corpus info
    config.corpus_len = len(corpus)

    # Initialize and index
    print(f"Initializing ComoRAG for {len(docs)} documents...")
    comorag = ComoRAG(global_config=config)

    print("Indexing documents...")
    comorag.index(docs)

    # Answer questions
    print(f"Processing {len(queries)} queries...")
    solutions = comorag.try_answer(queries)

    # Attach gold answers for evaluation
    gold_answers = get_gold_answers(samples)
    for idx, solution in enumerate(solutions):
        solution.gold_answers = list(gold_answers[idx])

    # Save results
    results = []
    for idx, (query, solution) in enumerate(zip(queries, solutions)):
        results.append({
            "idx": idx,
            "question": query,
            "golden_answers": solution.gold_answers,
            "output": solution.answer
        })

    os.makedirs(config.output_dir, exist_ok=True)
    output_path = os.path.join(config.output_dir, "results.json")
    with open(output_path, "w", encoding="utf-8") as f:
        json.dump(results, f, ensure_ascii=False, indent=2)

    print(f"Results saved to {output_path}")
    return results

# Main execution
if __name__ == "__main__":
    # Set API key
    os.environ["OPENAI_API_KEY"] = "your-api-key-here"

    # Configure pipeline
    config = BaseConfig(
        llm_base_url='https://api.openai.com/v1',
        llm_name='gpt-4o-mini',
        llm_api_key=os.environ["OPENAI_API_KEY"],
        embedding_model_name='nvidia/NV-Embed-v2',
        embedding_batch_size=32,
        need_cluster=True,
        output_dir='results/cinderella',
        save_dir='outputs/cinderella',
        max_meta_loop_max_iterations=5,
        is_mc=False,
        max_tokens_ver=2000,
        max_tokens_sem=2000,
        max_tokens_epi=2000
    )

    # Process dataset
    dataset_path = './dataset/cinderella/cinderella_1'
    results = process_dataset(dataset_path, config)

    # Print sample results
    for r in results[:3]:
        print(f"\nQ: {r['question']}")
        print(f"Gold: {r['golden_answers']}")
        print(f"Pred: {r['output'][:200]}...")
```

--------------------------------

### Example Corpus File Format

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

This is an example of a single line in the corpus.jsonl file. Each line represents a document with fields like id, doc_id, title, and contents.

```json
{"id": 0, "doc_id": 1, "title": "...", "contents": "..."}
```

--------------------------------

### Initialize ComoRAG System with BaseConfig

Source: https://context7.com/eternityjune25/comorag/llms.txt

Configure and initialize the ComoRAG system using the BaseConfig dataclass for LLM, embedding, and memory settings. Ensure API keys are set as environment variables.

```python
import os
from src.comorag.ComoRAG import ComoRAG
from src.comorag.utils.config_utils import BaseConfig

# Configure the RAG system
config = BaseConfig(
    # LLM Configuration
    llm_name='gpt-4o-mini',
    llm_base_url='https://api.openai.com/v1',
    llm_api_key=os.environ.get("OPENAI_API_KEY"),

    # Embedding Model Configuration
    embedding_model_name='nvidia/NV-Embed-v2',
    embedding_base_url='https://api.openai.com/v1',
    embedding_api_key=os.environ.get("OPENAI_API_KEY"),
    embedding_batch_size=32,

    # Memory System Configuration
    need_cluster=True,  # Enable semantic clustering
    use_ver=True,       # Use veridical (detail) memory
    use_sem=True,       # Use semantic (summary) memory
    use_epi=True,       # Use episodic (timeline) memory

    # Token Limits for Context
    max_tokens_ver=2000,  # Max tokens for detail chunks
    max_tokens_sem=2000,  # Max tokens for summaries
    max_tokens_epi=2000,  # Max tokens for timelines

    # Retrieval Configuration
    qa_ver_top_k=50,      # Top-k veridical chunks
    qa_sem_top_k=50,      # Top-k semantic summaries
    qa_epi_top_k=50,      # Top-k episodic entries
    linking_top_k=5,      # Top-k facts for graph linking

    # Reasoning Configuration
    max_meta_loop_max_iterations=5,  # Max self-probe iterations
    is_mc=False,                      # Multiple choice mode

    # Storage Paths
    save_dir='outputs/my_project',
    output_dir='results/my_project'
)

# Initialize ComoRAG
comorag = ComoRAG(global_config=config)
```

--------------------------------

### Configure Project with vLLM Local Server

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

Configure the project to use a local vLLM server. Set 'llm_base_url' to your vLLM server address and provide a placeholder for 'llm_api_key'.

```python
# vLLM server configuration
vllm_base_url = 'http://localhost:8000/v1'  # vLLM server address
served_model_name = '/path/to/your/model'    # Model path

config = BaseConfig(
    llm_base_url=vllm_base_url,
    llm_name=served_model_name,
    llm_api_key="your-api-key-here",  # Any value, local server doesn't need real API key
    dataset='cinderella',
    embedding_model_name='/path/to/your/embedding/model',
    embedding_batch_size=4,
    need_cluster=True,
    output_dir='result/cinderella_vllm',
    save_dir='outputs/cinderella_vllm',
    max_meta_loop_max_iterations=5,
    is_mc=False,
    max_tokens_ver=2000,
    max_tokens_sem=2000,
    max_tokens_epi=2000
)
```

--------------------------------

### Initialize BaseConfig for ComoRAG

Source: https://context7.com/eternityjune25/comorag/llms.txt

Configures LLM, embedding, graph, retrieval, and storage parameters. Use asdict to inspect the final configuration state.

```python
from src.comorag.utils.config_utils import BaseConfig
from dataclasses import asdict

# Full configuration with all options
config = BaseConfig(
    # === LLM Configuration ===
    llm_name="gpt-4o-mini",                    # Model name/path
    llm_base_url="https://api.openai.com/v1", # API endpoint
    llm_api_key="sk-...",                      # API key
    max_new_tokens=2048,                       # Max generation tokens
    temperature=0,                             # Sampling temperature
    seed=42,                                   # Random seed

    # === Embedding Configuration ===
    embedding_model_name="nvidia/NV-Embed-v2", # Embedding model
    embedding_base_url="https://api.openai.com/v1",
    embedding_api_key="sk-...",
    embedding_batch_size=32,                   # Batch size for encoding
    embedding_max_seq_len=2048,                # Max sequence length
    embedding_return_as_normalized=True,       # L2 normalize embeddings

    # === Graph Construction ===
    is_directed_graph=False,                   # Undirected graph
    synonymy_edge_topk=2047,                   # k for synonym KNN
    synonymy_edge_sim_threshold=0.8,           # Similarity threshold

    # === Information Extraction ===
    openie_mode="online",                      # "online" or "offline"
    save_openie=True,                          # Cache OpenIE results

    # === Clustering ===
    need_cluster=True,                         # Enable semantic clustering

    # === Retrieval ===
    linking_top_k=5,                           # Facts for graph linking
    retrieval_top_k=200,                       # DPR retrieval depth
    damping=0.5,                               # PPR damping factor
    passage_node_weight=0.05,                  # Passage weight in PPR

    # === Question Answering ===
    qa_ver_top_k=50,                           # Veridical top-k
    qa_sem_top_k=50,                           # Semantic top-k
    qa_epi_top_k=50,                           # Episodic top-k
    max_tokens_ver=3000,                       # Max veridical tokens
    max_tokens_sem=1000,                       # Max semantic tokens
    max_tokens_epi=1000,                       # Max episodic tokens
    max_meta_loop_max_iterations=5,            # Self-probe iterations
    is_mc=False,                               # Multiple choice mode
    use_ver=True,                              # Use veridical memory
    use_sem=True,                              # Use semantic memory
    use_epi=True,                              # Use episodic memory

    # === Storage ===
    save_dir="outputs/experiment1",            # Working directory
    output_dir="results/experiment1"           # Results directory
)

# Print all configuration values
for key, value in asdict(config).items():
    print(f"{key}: {value}")
```

--------------------------------

### Initialize ComoRAG System with Individual Parameters

Source: https://context7.com/eternityjune25/comorag/llms.txt

Alternative initialization of ComoRAG using individual parameters, useful for overriding specific configurations.

```python
comorag = ComoRAG(
    save_dir='outputs/custom',
    llm_model_name='gpt-4o',
    llm_base_url='https://api.openai.com/v1',
    llm_api_key='your-api-key',
    embedding_model_name='nvidia/NV-Embed-v2'
)
```

--------------------------------

### Run ComoRAG with vLLM

Source: https://github.com/eternityjune25/comorag/blob/main/README_zh.md

Execute the main application script using the vLLM configuration.

```bash
python main_vllm.py
```

--------------------------------

### Initialize and Use ProbeAgent for Sub-Questions

Source: https://context7.com/eternityjune25/comorag/llms.txt

Initializes the ProbeAgent to generate follow-up probes (sub-questions) when initial context is insufficient. Requires 'gpt-4o-mini' model and valid API key. The agent analyzes query, context, and previous probes to identify missing information.

```python
from src.comorag.utils.agents import ProbeAgent

# Initialize probe agent
probe_agent = ProbeAgent(
    model="gpt-4o-mini",
    llm_base_url="https://api.openai.com/v1",
    llm_api_key="your-api-key"
)

# Current query and context
query = "How did Cinderella escape from her stepmother's control?"
context = """
### Detail Chunks
Cinderella lived with her stepmother and two stepsisters after her father died.
She was forced to do all the housework and sleep in the attic.

### Semantic Summary
The story follows Cinderella's difficult life under her stepmother's rule.

### Timeline Summary
Early period: Father's death and stepmother's takeover.
"""

# Previous probes (if any)
previous_probes = "What was Cinderella's daily life like?"

# Generate new probes to find missing information
probes = probe_agent.find_probes(
    query=query,
    context=context,
    previous_probes=previous_probes,
    max_completion_tokens=500
)

# Returns list of targeted sub-questions
print(f"Generated probes: {probes}")
# Example output: ["Who helped Cinderella attend the ball?",
#                  "What happened when Cinderella met the prince?"]

# Use probes for additional retrieval
for probe in probes:
    additional_docs, _ = comorag.tri_retrieve(
        query=probe,
        memory_pool=memory_pool
    )
    memory_pool = comorag.mem_encode(
        query=query + " " + probe,
        docs=additional_docs,
        memory_pool=memory_pool,
        probe=probe  # Tag with specific probe
    )
```

--------------------------------

### Create and Add Fusion Content

Source: https://context7.com/eternityjune25/comorag/llms.txt

Demonstrates creating fusion content from historical context and adding it to a memory pool. Ensure 'memory_pool' and 'similar_nodes' are initialized prior to use.

```python
historical_info = memory_pool.create_fusion_content(
    probe="What was the outcome of the ball?",
    top_k_percent=0.3  # Use top 30% similar nodes
)
print(f"Fused historical context: {historical_info[:500]}...")

# Add fusion node to track reasoning history
memory_pool.add_fused_node(
    probe="What was the outcome of the ball?",
    fused_content=historical_info,
    source_nodes=similar_nodes
)
```

--------------------------------

### Run Main Program with OpenAI API

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

Execute the main Python script to run the project using the OpenAI API with the configured settings.

```bash
python main_openai.py
```

--------------------------------

### Configure vLLM Server Settings

Source: https://github.com/eternityjune25/comorag/blob/main/README_zh.md

Define the base URL, model path, and configuration parameters for the BaseConfig object to interface with a local vLLM instance.

```python
vllm_base_url = 'http://localhost:8000/v1'  # vLLM 服务器地址
served_model_name = '/path/to/your/model'    # 模型路径

config = BaseConfig(
    llm_base_url=vllm_base_url,
    llm_name=served_model_name,
    llm_api_key="your-api-key-here",  # 任意值，本地服务器不需要真实 API key
    dataset='cinderella',
    embedding_model_name='/path/to/your/embedding/model',
    embedding_batch_size=4,
    need_cluster=True,
    output_dir='result/cinderella_vllm',
    save_dir='outputs/cinderella_vllm',
    max_meta_loop_max_iterations=5,
    is_mc=False,
    max_tokens_ver=2000,
    max_tokens_sem=2000,
    max_tokens_epi=2000
)
```

--------------------------------

### Configure Project with OpenAI API

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

Configure dataset path and model parameters for the OpenAI API method. Ensure 'llm_base_url' points to the OpenAI API endpoint.

```python
config = BaseConfig(
    llm_base_url='https://api.example.com/v1',  # OpenAI API
    llm_name='gpt-4o-mini',
    dataset='cinderella',
    embedding_model_name='/path/to/your/embedding/model',
    embedding_batch_size=32,
    need_cluster=True,  # Enable Semantic/Episodic enhancement
    output_dir='result/cinderella',
    save_dir='outputs/cinderella',
    max_meta_loop_max_iterations=5,
    is_mc=False,  # Multiple-choice?
    max_tokens_ver=2000,  # Veridical layer tokens
    max_tokens_sem=2000,  # Semantic layer tokens
    max_tokens_epi=2000   # Episodic layer tokens
)
```

--------------------------------

### Initialize and Use OpenIE for Information Extraction

Source: https://context7.com/eternityjune25/comorag/llms.txt

Initializes the OpenIE class for Named Entity Recognition (NER) and relation triple extraction from text using LLM-based methods. Requires an initialized LLM model. This is used for constructing knowledge graphs.

```python
from src.comorag.information_extraction.openie_openai import OpenIE
from src.comorag.llm.openai_gpt import CacheOpenAI

# Initialize LLM for OpenIE
llm_model = CacheOpenAI(global_config=config)

# Create OpenIE extractor
openie = OpenIE(llm_model=llm_model)

# Extract from single passage
passage = """
Cinderella's fairy godmother appeared and transformed a pumpkin into
a golden carriage. She also turned mice into horses and Cinderella's
rags into a beautiful ball gown with glass slippers.
"""

chunk_key = "chunk-001"

```

--------------------------------

### Manage Memory with MemoryPool

Source: https://context7.com/eternityjune25/comorag/llms.txt

Handles memory node creation, staging, and similarity-based retrieval. Requires an initialized PoolAgent and embedding model.

```python
from src.comorag.utils.memory_utils import MemoryPool, MemoryNode, NodeType
from src.comorag.utils import agents
import numpy as np

# Initialize agents
pool_agent = agents.PoolAgent(
    model="gpt-4o-mini",
    llm_base_url="https://api.openai.com/v1",
    llm_api_key="your-api-key"
)

# Create memory pool with embedding model for similarity search
memory_pool = MemoryPool(
    agent=pool_agent,
    embedding_model=comorag.embedding_model
)

# Create and add memory nodes manually
ver_node = MemoryNode(
    probe="What happened at the ball?",
    node_type=NodeType.VER,
    original_content=["Cinderella danced with the prince...", "The clock struck midnight..."],
    cue="Cinderella attended the royal ball and danced with the prince until midnight."
)
ver_node.update_hashes()  # Generate content hashes

# Add to temporary pool (staging area)
memory_pool.add_to_temp_pool(ver_node)

# After reasoning iteration, merge to main pool
memory_pool.merge_temp_to_main()
# Output: "Merged 1 temporary memories"

# Get all nodes of a specific type
all_ver_nodes = memory_pool.get_nodes_by_type(NodeType.VER)
print(f"Total veridical nodes: {len(all_ver_nodes)}")

# Get all unique probes used
probes = memory_pool.get_all_probes()
print(f"Probes used: {probes}")

# Retrieve similar historical nodes for fusion
similar_nodes = memory_pool.retrieve_similar_nodes(
    current_probe="Who did Cinderella meet at the ball?",
    top_percent=0.5  # Return top 50% most similar
)
```

--------------------------------

### Initialize and Use EmbeddingStore for Vector Storage

Source: https://context7.com/eternityjune25/comorag/llms.txt

Initializes the EmbeddingStore for persistent storage of embeddings using Parquet files with automatic deduplication. It supports batch encoding and retrieval. Requires an initialized embedding model and a configuration object.

```python
from src.comorag.embedding_store import EmbeddingStore
from src.comorag.embedding_model import _get_embedding_model_class
import numpy as np

# Initialize embedding model
embedding_model = _get_embedding_model_class("nvidia/NV-Embed-v2")(
    global_config=config,
    embedding_model_name="nvidia/NV-Embed-v2"
)

# Create embedding store
store = EmbeddingStore(
    embedding_model=embedding_model,
    db_filename="./embeddings/chunks",  # Storage directory
    batch_size=32,                        # Encoding batch size
    namespace="chunk"                     # Prefix for hash IDs
)

# Insert texts (automatically embeds and deduplicates)
texts = [
    "Cinderella lived with her stepmother.",
    "The fairy godmother appeared.",
    "The prince found the glass slipper."
]
store.insert_strings(texts)
# Output: "Inserting 3 new records, 0 records already exist."

# Re-inserting same texts is idempotent
store.insert_strings(texts)
# Output: "Inserting 0 new records, 3 records already exist."

# Get all stored hash IDs
all_ids = store.get_all_ids()
print(f"Stored items: {len(all_ids)}")

# Retrieve text by hash ID
row = store.get_row(all_ids[0])
print(f"Content: {row['content']}")
print(f"Hash ID: {row['hash_id']}")

# Get embeddings for multiple items
embeddings = store.get_embeddings(all_ids[:2], dtype=np.float32)
print(f"Embedding shape: {embeddings.shape}")  # (2, embedding_dim)

# Access text-to-hash mapping
text_to_hash = store.text_to_hash_id
hash_id = text_to_hash["The fairy godmother appeared."]
print(f"Hash for 'fairy godmother': {hash_id}")

# Get insertion order mapping (for chronological sorting)
hash_to_order = store.get_hash_id_to_order()
for h_id, order in list(hash_to_order.items())[:3]:
    print(f"Hash {h_id}: order {order}")
```

--------------------------------

### Preprocess Documents with chunk_doc_corpus.py

Source: https://context7.com/eternityjune25/comorag/llms.txt

Use the CLI script to chunk documents into a JSONL corpus using various strategies. Includes input and output format specifications.

```bash
# Token-based chunking (default)
python script/chunk_doc_corpus.py \
    --input_path ./dataset/raw/documents.jsonl \
    --output_path ./dataset/processed/corpus.jsonl \
    --chunk_by token \
    --chunk_size 512 \
    --tokenizer_name_or_path nvidia/NV-Embed-v2

# Sentence-based chunking
python script/chunk_doc_corpus.py \
    --input_path ./dataset/raw/documents.jsonl \
    --output_path ./dataset/processed/corpus.jsonl \
    --chunk_by sentence \
    --chunk_size 512

# Recursive chunking (semantic boundaries)
python script/chunk_doc_corpus.py \
    --input_path ./dataset/raw/documents.jsonl \
    --output_path ./dataset/processed/corpus.jsonl \
    --chunk_by recursive \
    --chunk_size 512
```

```python
# Input JSONL format (documents.jsonl):
# {"id": "doc_0", "contents": "Title, Full document text here..."}
# {"id": "doc_1", "contents": "Title, Another document..."}

# Output JSONL format (corpus.jsonl):
# {"id": 0, "doc_id": "doc_0", "title": "First 30 chars...", "contents": "Chunk text..."}
```

--------------------------------

### Programmatic Document Chunking

Source: https://context7.com/eternityjune25/comorag/llms.txt

Initializes a TokenChunker and processes documents from a JSONL file, saving the chunked output to another JSONL file. Ensure 'documents.jsonl' exists and is correctly formatted.

```python
import json
from chonkie import TokenChunker

# Load documents
with open('documents.jsonl', 'r') as f:
    documents = [json.loads(line) for line in f]

# Initialize chunker
chunker = TokenChunker(
    tokenizer='nvidia/NV-Embed-v2',
    chunk_size=512
)

# Process documents
chunked_docs = []
chunk_id = 0
for doc in documents:
    title, text = doc['contents'].split(',', 1)
    chunks = chunker.chunk(text)
    for chunk in chunks:
        chunked_docs.append({
            'id': chunk_id,
            'doc_id': doc['id'],
            'title': chunk.text[:30],
            'contents': chunk.text
        })
        chunk_id += 1

# Save chunked corpus
with open('corpus.jsonl', 'w') as f:
    for doc in chunked_docs:
        f.write(json.dumps(doc) + '\n')

print(f"Processed {len(documents)} documents into {len(chunked_docs)} chunks")
```

--------------------------------

### Evaluate QA Results

Source: https://github.com/eternityjune25/comorag/blob/main/README_zh.md

Run the evaluation script on the generated results directory to calculate metrics like EM and F1.

```bash
python script/eval_qa.py /path/to/result/<dataset>/<subset>
```

--------------------------------

### Perform Semantic Clustering with ChunkSoftClustering

Source: https://context7.com/eternityjune25/comorag/llms.txt

Initialize and execute soft clustering on document chunks to build a semantic memory index. Requires a summarization model for cluster descriptions.

```python
from src.comorag.utils.cluster_utils import ChunkSoftClustering
from src.comorag.utils.summarization_utils import GPT4SummarizationModel

# Initialize summarization model
summarization_model = GPT4SummarizationModel(
    model="gpt-4o-mini",
    llm_base_url="https://api.openai.com/v1",
    llm_api_key="your-api-key"
)

# Create clustering manager
clustering = ChunkSoftClustering(
    embedding_store=comorag.ver_embedding_store,
    reduction_dimension=10,        # UMAP target dimension
    threshold=0.01,                # Membership threshold
    max_clusters=50,               # Max cluster count
    verbose=True,
    db_filename="./outputs/clusters",
    namespace="rag_chunks",
    summarization_model=summarization_model
)

# Perform soft clustering
clusters = clustering.perform_clustering()
print(f"Created {len(clusters)} clusters")

# Get clustering statistics
stats = clustering.get_cluster_stats()
print(f"Total chunks: {stats['total_chunks']}")
print(f"Total clusters: {stats['total_clusters']}")
print(f"Avg memberships per chunk: {stats['avg_membership_per_chunk']:.2f}")

# Get members of a specific cluster
cluster_id = 0
members = clustering.get_cluster_members(cluster_id, threshold=0.1)
print(f"Cluster {cluster_id} has {len(members)} members")

for hash_id, score in members[:3]:
    print(f"  {hash_id}: membership={score:.3f}")

# Get cluster texts with membership scores
texts_with_scores = clustering.get_cluster_texts(cluster_id, top_k=5)
for text, score in texts_with_scores:
    print(f"Score {score:.3f}: {text[:100]}...")

# Find related chunks through shared clusters
chunk_hash = members[0][0]
related = clustering.get_related_chunks(chunk_hash, threshold=0.05)
print(f"Related chunks: {len(related)}")

# Generate cluster summary
summary = clustering.create_cluster_summary(cluster_id)
print(f"Cluster {cluster_id} summary: {summary}")

# Find clusters for membership lookup
memberships = clustering.get_cluster_membership(chunk_hash)
print(f"Chunk belongs to clusters: {list(memberships.keys())}")
```

--------------------------------

### Check vLLM Server Status

Source: https://github.com/eternityjune25/comorag/blob/main/README.md

Use these commands to verify if the vLLM server is running on the expected port and to test its API connection.

```bash
# Check if port is occupied
netstat -tlnp | grep 8000

# Test API connection
curl http://localhost:8000/v1/models
```

--------------------------------

### Verify vLLM Server Status

Source: https://github.com/eternityjune25/comorag/blob/main/README_zh.md

Check if the required port is occupied and verify the API connectivity.

```bash
# 检查端口是否被占用
netstat -tlnp | grep 8000

# 测试 API 连接
curl http://localhost:8000/v1/models
```

--------------------------------

### Index Documents with ComoRAG

Source: https://context7.com/eternityjune25/comorag/llms.txt

Load documents from a JSONL file and index them using the ComoRAG system. This process generates embeddings, extracts entities and triples, builds a knowledge graph, and creates semantic and episodic summaries.

```python
import json

# Load corpus from JSONL file
corpus_path = './dataset/my_corpus/corpus.jsonl'
with open(corpus_path, 'r', encoding='utf-8') as f:
    corpus = [json.loads(line) for line in f if line.strip()]

# Extract document contents
docs = [doc['contents'] for doc in corpus]

# Example document format in corpus.jsonl:
# {"id": "doc_0", "contents": "Once upon a time, there was a young girl named Cinderella..."}
# {"id": "doc_1", "contents": "Cinderella lived with her stepmother and two stepsisters..."}

# Index all documents - this performs:
# 1. Embedding generation for all chunks
# 2. Named Entity Recognition (NER)
# 3. Triple extraction (subject, predicate, object)
# 4. Knowledge graph construction with synonym edges
# 5. Soft clustering and semantic summarization
# 6. Timeline/episodic summarization
comorag.index(docs)

# After indexing, the following stores are populated:
# - ver_embedding_store: Veridical chunk embeddings
# - entity_embedding_store: Entity node embeddings
# - fact_embedding_store: Fact triple embeddings
# - sem_embedding_store: Semantic summary embeddings (if need_cluster=True)
# - epi_embedding_store: Episodic timeline embeddings (if need_cluster=True)

# Graph is saved automatically to: {working_dir}/graph.graphml
print(comorag.get_graph_info())
```

--------------------------------

### ComoRAG.mem_encode

Source: https://context7.com/eternityjune25/comorag/llms.txt

Creates memory nodes from retrieved documents and generates cue summaries.

```APIDOC
## ComoRAG.mem_encode

### Description
Creates memory nodes from retrieved documents, generating cue summaries for each memory type using the PoolAgent's fusion capability.

### Parameters
#### Request Body
- **query** (string) - Required - The original query.
- **docs** (dict) - Required - The retrieved documents from tri_retrieve.
- **memory_pool** (MemoryPool) - Required - The memory pool to update.
- **probe** (object) - Optional - A sub-probe for targeted retrieval.

### Response
#### Success Response (200)
- **memory_pool** (MemoryPool) - The updated memory pool containing encoded memory nodes.
```

--------------------------------

### Process Questions with try_answer

Source: https://context7.com/eternityjune25/comorag/llms.txt

Use `try_answer` to process a list of questions through the meta-control loop. It handles retrieval, encoding, and reasoning to generate answers. Results are saved to a JSON file.

```python
import json

# Load questions from JSONL file
qas_path = './dataset/my_corpus/qas.jsonl'
with open(qas_path, 'r', encoding='utf-8') as f:
    samples = [json.loads(line) for line in f if line.strip()]

# Example QA format in qas.jsonl:
# {"question": "Who helped Cinderella attend the ball?", "answers": ["fairy godmother"]}
# {"question": "What did Cinderella leave behind at the palace?", "answers": ["glass slipper"]}

queries = [s['question'] for s in samples]

# Process all queries with parallel execution
solutions = comorag.try_answer(queries)

# Each solution contains:
# - question: The original query
# - answer: The generated answer with reasoning
# - docs: Retrieved veridical chunks
# - summary: Retrieved semantic summaries
# - timeline: Retrieved episodic content

# Process results
for idx, solution in enumerate(solutions):
    print(f"Q{idx}: {solution.question}")
    print(f"A{idx}: {solution.answer}")
    print("---")

# Save results to JSON
results = []
for idx, (query, solution) in enumerate(zip(queries, solutions)):
    results.append({
        "idx": idx,
        "question": query,
        "output": solution.answer,
        "retrieved_chunks": solution.docs[:3]  # Top 3 chunks
    })

with open('results.json', 'w', encoding='utf-8') as f:
    json.dump(results, f, ensure_ascii=False, indent=2)
```

--------------------------------

### Extract Entities and Triples with OpenIE

Source: https://context7.com/eternityjune25/comorag/llms.txt

Perform named entity recognition and triple extraction on text chunks. Supports individual chunk processing and batch operations.

```python
ner_result = openie.ner(chunk_key=chunk_key, passage=passage)
print(f"Entities: {ner_result.unique_entities}")
# Output: ['Cinderella', 'fairy godmother', 'pumpkin', 'golden carriage',
#          'mice', 'horses', 'ball gown', 'glass slippers']

# Step 2: Triple Extraction using extracted entities
triple_result = openie.triple_extraction(
    chunk_key=chunk_key,
    passage=passage,
    named_entities=ner_result.unique_entities
)
print(f"Triples: {triple_result.triples}")
# Output: [['fairy godmother', 'transformed', 'pumpkin'],
#          ['pumpkin', 'became', 'golden carriage'],
#          ['fairy godmother', 'turned', 'mice'],
#          ['mice', 'became', 'horses'],
#          ['Cinderella', 'received', 'glass slippers']]

# Combined extraction (NER + Triples)
result = openie.openie(chunk_key=chunk_key, passage=passage)
print(f"NER: {result['ner'].unique_entities}")
print(f"Triples: {result['triplets'].triples}")

# Batch processing for multiple chunks
chunks = {
    "chunk-001": {"content": "The prince searched the kingdom..."},
    "chunk-002": {"content": "Cinderella tried on the slipper..."},
    "chunk-003": {"content": "The prince and Cinderella married..."}
}

ner_dict, triple_dict = openie.batch_openie(chunks)

for chunk_id in chunks:
    print(f"\n{chunk_id}:")
    print(f"  Entities: {ner_dict[chunk_id].unique_entities}")
    print(f"  Triples: {triple_dict[chunk_id].triples}")
```

--------------------------------

### Perform Tri-Memory Retrieval with tri_retrieve

Source: https://context7.com/eternityjune25/comorag/llms.txt

Use `tri_retrieve` to fetch information from veridical, semantic, and episodic memory indices concurrently. Customize retrieval with `*_top_k` parameters. Retrieved documents are organized by memory type.

```python
from src.comorag.utils.memory_utils import MemoryPool, NodeType
from src.comorag.utils import agents

# Initialize agents for memory operations
pool_agent = agents.PoolAgent(
    model=config.llm_name,
    llm_base_url=config.llm_base_url,
    llm_api_key=config.llm_api_key
)

# Create memory pool to track retrieved content
memory_pool = MemoryPool(
    agent=pool_agent,
    embedding_model=comorag.embedding_model
)

# Perform tri-memory retrieval
query = "What magical transformation happened before the ball?"
docs, nodes = comorag.tri_retrieve(
    query=query,
    memory_pool=memory_pool,
    ver_top_k=10,  # Override default top-k for veridical
    sem_top_k=5,   # Override default top-k for semantic
    epi_top_k=5    # Override default top-k for episodic
)

# Retrieved documents organized by memory type
veridical_chunks = docs["veridical"]   # Detail-level text chunks
semantic_summaries = docs["semantic"]   # Cluster-based summaries
episodic_entries = docs["episodic"]     # Timeline-based events

print(f"Retrieved {len(veridical_chunks)} veridical chunks")
print(f"Retrieved {len(semantic_summaries)} semantic summaries")
print(f"Retrieved {len(episodic_entries)} episodic entries")

# Access retrieved content
for i, chunk in enumerate(veridical_chunks[:3]):
    print(f"Chunk {i}: {chunk[:200]}...")
```

--------------------------------

### ComoRAG.try_answer

Source: https://context7.com/eternityjune25/comorag/llms.txt

Processes queries through a meta-control loop, performing retrieval, encoding, and reasoning to generate answers.

```APIDOC
## ComoRAG.try_answer

### Description
Processes queries through the meta-control loop, performing tri-memory retrieval, memory encoding, self-probing, and iterative reasoning until a final answer is reached or max iterations are exceeded.

### Parameters
#### Request Body
- **queries** (list) - Required - A list of question strings to be processed.

### Response
#### Success Response (200)
- **solutions** (list) - A list of objects containing the original question, generated answer, retrieved veridical chunks, semantic summaries, and episodic timeline content.
```

--------------------------------

### Encode Memories with mem_encode

Source: https://context7.com/eternityjune25/comorag/llms.txt

Use `mem_encode` to convert retrieved documents into structured memory nodes using the PoolAgent. This method generates condensed cues for each memory type. An optional `probe` can be provided for targeted encoding.

```python
# After tri_retrieve, encode retrieved docs into memory nodes
memory_pool = comorag.mem_encode(
    query=query,
    docs=docs,
    memory_pool=memory_pool,
    probe=None  # Optional: specify a sub-probe for targeted retrieval
)

# Access encoded memory nodes from temporary pool
ver_nodes = memory_pool.get_temp_nodes_by_type(NodeType.VER)
sem_nodes = memory_pool.get_temp_nodes_by_type(NodeType.SEM)
epi_nodes = memory_pool.get_temp_nodes_by_type(NodeType.EPI)

# Each node contains:
# - probe: The query/probe that triggered retrieval
# - node_type: VER, SEM, EPI, or FUSION
# - original_content: List of original texts
# - cue: Condensed summary/cue for the content
# - content_hash: Hash IDs for deduplication

for node in ver_nodes:
    print(f"Probe: {node.probe}")
    print(f"Cue: {node.cue}")
    print(f"Original chunks: {len(node.original_content)}")

# Build context from encoded memories
ver_context = "\n".join([node.cue for node in ver_nodes])
sem_context = "\n".join([node.cue for node in sem_nodes])
epi_context = "\n".join([node.cue for node in epi_nodes])

print(f"Veridical context length: {len(ver_context)} chars")
print(f"Semantic context length: {len(sem_context)} chars")
print(f"Episodic context length: {len(epi_context)} chars")
```

--------------------------------

### ComoRAG.tri_retrieve

Source: https://context7.com/eternityjune25/comorag/llms.txt

Performs parallel retrieval from veridical, semantic, and episodic memory indices.

```APIDOC
## ComoRAG.tri_retrieve

### Description
Performs parallel retrieval from all three memory indices: veridical (detail chunks), semantic (cluster summaries), and episodic (timeline entries).

### Parameters
#### Request Body
- **query** (string) - Required - The search query.
- **memory_pool** (MemoryPool) - Required - The memory pool instance to track retrieved content.
- **ver_top_k** (int) - Optional - Number of veridical chunks to retrieve.
- **sem_top_k** (int) - Optional - Number of semantic summaries to retrieve.
- **epi_top_k** (int) - Optional - Number of episodic entries to retrieve.

### Response
#### Success Response (200)
- **docs** (dict) - Dictionary containing retrieved content organized by 'veridical', 'semantic', and 'episodic' keys.
- **nodes** (list) - List of retrieved memory nodes.
```

--------------------------------

### Chunk Document Corpus

Source: https://github.com/eternityjune25/comorag/blob/main/README_zh.md

Process raw documents into smaller chunks for retrieval using the specified tokenizer and chunking strategy.

```bash
python script/chunk_doc_corpus.py \
  --input_path dataset/<name>/<subset>/corpus.jsonl \
  --output_path dataset/<name>/<subset>/corpus_chunked.jsonl \
  --chunk_by token \
  --chunk_size 512 \
  --tokenizer_name_or_path /path/to/your/tokenizer
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.