### Benchmark Evaluation Setup and Execution with Bash

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

This snippet details the process of setting up the environment and running benchmark evaluations using shell scripts. It includes commands for creating and activating a Conda environment, installing dependencies, setting API keys in a `.env` file, and executing specific evaluation scripts for long context and RAG agents. It also shows how to run single evaluations with custom configurations and apply ablation settings.

```bash
# Setup environment
conda create --name MABench python=3.10.16
conda activate MABench
pip install torch
pip install -r requirements.txt
pip install "numpy<2"

# Set API keys in .env file
echo "OPENAI_API_KEY=your_key_here" >> .env
echo "Anthropic_API_KEY=your_key_here" >> .env
echo "Google_API_KEY=your_key_here" >> .env

# Run long context agent evaluation
bash bash_files/sh/run_memagent_longcontext.sh

# Run RAG agents evaluation
bash bash_files/sh/run_memagent_rag_agents.sh

# Run single evaluation with specific configs
python main.py \
    --agent_config configs/agent_conf/Long_Context_Agents/Long_context_agent_gpt-4o-mini.yaml \
    --dataset_config configs/data_conf/Accurate_Retrieval/Ruler/QA/Ruler_qa1_197k.yaml

# Run with ablation settings
python main.py \
    --agent_config configs/agent_conf/RAG_Agents/gpt-4o-mini/Embedding_rag_gpt-4o-mini-text_embedding_3_small.yaml \
    --dataset_config configs/data_conf/Accurate_Retrieval/Ruler/QA/Ruler_qa1_197k.yaml \
    --chunk_size_ablation 2048 \
    --max_test_queries_ablation 50 \
    --force
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Installs necessary Python packages for the project, including PyTorch, dependencies from requirements.txt, and a specific version of NumPy. It also provides troubleshooting steps for potential conflicts with 'cognee' and 'letta' packages.

```bash
pip install torch
pip install -r requirements.txt
pip install "numpy<2"

pip install letta
pip uninstall letta   
pip install cognee
pip uninstall cognee
```

--------------------------------

### Initialize Text Retriever and Build Vector Store (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Initializes a TextRetriever with OpenAI embeddings or local models, builds a vector store from document chunks, and retrieves relevant documents for a given query. It demonstrates the basic setup for a retrieval-augmented generation system.

```Python
from memoryagentbench.retriever import TextRetriever

# Initialize text retriever with OpenAI embeddings
retriever = TextRetriever(
    embedding_model_name="text-embedding-3-small"
)

# Alternatively, use local embedding models
# retriever = TextRetriever(embedding_model_name="facebook/contriever")
# retriever = TextRetriever(embedding_model_name="Qwen/Qwen3-Embedding-4B")
# retriever = TextRetriever(embedding_model_name="nvidia/NV-Embed-v2")

# Build vector store from document chunks
documents = [
    "The Python programming language was created by Guido van Rossum.",
    "JavaScript is the most popular language for web development.",
    "Rust provides memory safety without garbage collection.",
    "Go was developed at Google for concurrent programming."
]
retriever.build_vectorstore(documents)

# Retrieve relevant documents for a query
query = "Who created Python?"
relevant_docs = retriever.retrieve(query, top_k=2)
print(f"Retrieved documents: {relevant_docs}")
# Output: ['The Python programming language was created by Guido van Rossum.', ...]
```

--------------------------------

### Manage Students and School Data in JavaScript

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/cognee/tests/test_data/code.txt

Demonstrates initializing school data, enrolling students, celebrating birthdays using forEach, promoting students using map, and updating the school name. It covers array manipulation and data flow.

```javascript
let schoolName = "Greenwood High School";
let students = [];

students.push(enrollStudent("Alice", 14, 9));
students.push(enrollStudent("Bob", 15, 10));

students.forEach(student => {
  console.log(student.celebrateBirthday());
});

students = students.map(promoteStudent);

console.log("Final Students List:");
students.forEach(student => console.log(student.describe()));

schoolName = "Greenwood International School";
console.log(`School Name Updated to: ${schoolName}`);
```

--------------------------------

### Custom Agent Configuration with YAML

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

This snippet provides an example of a custom agent configuration file in YAML format. It defines parameters for a long context agent, including the model name, temperature, input length limit, buffer length, and output directory. This allows for flexible customization of agent behavior and settings.

```yaml
# configs/agent_conf/custom_agent.yaml

# Long context agent configuration
agent_name: Long_context_agent_custom
model: gpt-4o-mini  # or gpt-4o, claude-3-7-sonnet-20250219, gemini-2.0-flash
temperature: 0.7
input_length_limit: 128000
buffer_length: 4000
output_dir: ./outputs/custom_agent
```

--------------------------------

### Enroll New Student Function in JavaScript

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/cognee/tests/test_data/code.txt

A function to create a new 'Student' instance, log a greeting and description, and return the created student object. It demonstrates object instantiation and method invocation.

```javascript
function enrollStudent(name, age, grade) {
  const student = new Student(name, age, grade);
  console.log(student.greet());
  console.log(student.describe());
  return student;
}
```

--------------------------------

### Create Conda Environment

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Creates a dedicated conda environment named 'MABench' with Python version 3.10.16 for ensuring reproducibility of the project setup.

```bash
conda create --name MABench python=3.10.16
```

--------------------------------

### Run RAG Agents and Memory Methods Evaluation

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Runs an example evaluation command for RAG agents and agentic memory methods using a bash script. This command is used for evaluating different memory strategies.

```bash
bash bash_files/eniac/run_memagent_rag_agents.sh
```

--------------------------------

### Run Long Context Agent Evaluation

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Executes an example evaluation command for long context agents using a bash script. Requires specifying paths to agent and dataset configuration files via --agent_config and --dataset_config arguments.

```bash
bash bash_files/eniac/run_memagent_longcontext.sh --agent_config <path_to_agent_config> --dataset_config <path_to_dataset_config>
```

--------------------------------

### Promote Student Function in JavaScript

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/cognee/tests/test_data/code.txt

A function that takes a 'Student' object, increments their grade, logs a promotion message, and returns the updated student object. It modifies object properties.

```javascript
function promoteStudent(student) {
  student.grade += 1;
  console.log(`${student.name} has been promoted to grade ${student.grade}.`);
  return student;
}
```

--------------------------------

### Define Student Class with Inheritance in JavaScript

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/cognee/tests/test_data/code.txt

Defines a 'Student' class that extends the 'Person' class, adding a 'grade' property and a 'describe' method. It utilizes super() to call the parent constructor.

```javascript
class Student extends Person {
  constructor(name, age, grade) {
    super(name, age);
    this.grade = grade;
  }

  describe() {
    return `${this.name} is a ${this.grade} grade student and is ${this.age} years old.`;
  }
}
```

--------------------------------

### Generate Cypher Query for Entities Connected to 'John'

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/cognee/infrastructure/llm/prompts/natural_language_retriever_system.txt

This example demonstrates how to retrieve all nodes directly connected to an 'Entity' node named 'John'. It uses a basic pattern match to find neighbors, returning both the 'John' node and its connected neighbors.

```cypher
MATCH (n:Entity {name: 'John'})--(neighbor)
RETURN n, neighbor
```

--------------------------------

### Initialize and Use AgentWrapper for LLM Agent Interaction (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Demonstrates how to initialize the AgentWrapper with specific configurations for an LLM agent and a dataset. It shows the process of memorizing context chunks and then querying the agent for information, along with how to access response details like token counts and query time.

```python
from agent import AgentWrapper

# Define agent configuration
agent_config = {
    'agent_name': 'Long_context_agent_gpt-4o-mini',
    'model': 'gpt-4o-mini',
    'temperature': 0.7,
    'input_length_limit': 128000,
    'buffer_length': 4000,
    'output_dir': './outputs/gpt-4o-mini'
}

# Define dataset configuration
dataset_config = {
    'dataset': 'Accurate_Retrieval',
    'sub_dataset': 'ruler_qa1_197K',
    'context_max_length': 220000,
    'generation_max_length': 50,
    'chunk_size': 4096,
    'max_test_samples': 100,
    'seed': 42,
    'debug': False
}

# Initialize the agent wrapper
agent = AgentWrapper(
    agent_config=agent_config,
    dataset_config=dataset_config,
    load_agent_from='./agent_states/context_0'
)

# Memorize context chunks
context_chunks = [
    "The capital of France is Paris. It is known for the Eiffel Tower.",
    "London is the capital of United Kingdom. Big Ben is located there."
]

for chunk in context_chunks:
    result = agent.send_message(chunk, memorizing=True, context_id=0)
    print(f"Memorization result: {result}")  # Output: "Memorized"

# Query the agent
query = "What is the capital of France?"
response = agent.send_message(
    query,
    memorizing=False,
    query_id=1,
    context_id=0
)

print(f"Answer: {response['output']}")  # Output: "Paris"
print(f"Input tokens: {response['input_len']}")  # Token count
print(f"Output tokens: {response['output_len']}")  # Token count
print(f"Query time: {response['query_time_len']:.2f}s")  # Response time

```

--------------------------------

### Use RAGSystem for Question Answering (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Sets up and uses a RAGSystem for complete question-answering by combining a retriever with a language model. It allows specifying the model, temperature, max tokens, and system message for the assistant. The output includes the answer, context used, and query time.

```Python
from memoryagentbench.retriever import TextRetriever
from memoryagentbench.rag_system import RAGSystem

# Assuming retriever is already initialized and built as shown previously
# retriever = TextRetriever(embedding_model_name="text-embedding-3-small")
# documents = [...] # Your documents
# retriever.build_vectorstore(documents)

# Use RAGSystem for complete question-answering
rag_system = RAGSystem(
    retriever=retriever,
    model="gpt-4o-mini",
    temperature=0.7,
    max_tokens=500
)

result = rag_system.answer_query(
    query="Who created Python and what makes Rust special?",
    top_k=3,
    system_message="You are a helpful assistant. Answer based on the provided context."
)

print(f"Answer: {result['answer']}")
print(f"Context used: {result['context_used'][:200]}...")
print(f"Query time: {result['query_time_len']:.2f}s")
```

--------------------------------

### Define Person Class in JavaScript

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/cognee/tests/test_data/code.txt

Defines a 'Person' class with a constructor for name and age, and methods for greeting and celebrating birthdays. This serves as a base class for other entities.

```javascript
class Person {
  constructor(name, age) {
    this.name = name;
    this.age = age;
  }

  greet() {
    return `Hello, my name is ${this.name} and I'm ${this.age} years old.`;
  }

  celebrateBirthday() {
    this.age += 1;
    return `Happy Birthday, ${this.name}! You are now ${this.age} years old.`;
  }
}
```

--------------------------------

### Process Datasets and Create Q&A Pairs with ConversationCreator (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Illustrates how to use the ConversationCreator to load datasets from HuggingFace and process them into context chunks and query-answer pairs. It shows how to configure the creator with agent and dataset settings and then retrieve formatted queries, answers, and their IDs for evaluation.

```python
from conversation_creator import ConversationCreator

# Configuration for agent and dataset
agent_config = {
    'agent_name': 'Embedding_rag_text_embedding_3_small',
    'model': 'gpt-4o-mini',
    'temperature': 0.7,
    'input_length_limit': 300000,
    'buffer_length': 15000,
    'output_dir': './outputs/rag_small',
    'retrieve_num': 10
}

dataset_config = {
    'dataset': 'Accurate_Retrieval',
    'sub_dataset': 'ruler_qa1_197K',
    'context_max_length': 220000,
    'generation_max_length': 50,
    'chunk_size': 4096,
    'max_test_samples': 100,
    'seed': 42,
    'debug': False
}

# Create conversation data structures
conversation = ConversationCreator(agent_config, dataset_config)

# Get text chunks for each context (suitable for memory agents)
all_context_chunks = conversation.get_chunks()
print(f"Number of contexts: {len(all_context_chunks)}")
print(f"Chunks in first context: {len(all_context_chunks[0])}")

# Get query-answer pairs for evaluation
query_answer_pairs = conversation.get_query_and_answers()
print(f"Q&A pairs for first context: {len(query_answer_pairs[0])}")

# Each Q&A pair contains: (formatted_query, answer, qa_pair_id)
query, answer, qa_pair_id = query_answer_pairs[0][0]
print(f"Query: {query[:100]}...")
print(f"Expected answer: {answer}")
print(f"QA Pair ID: {qa_pair_id}")

```

--------------------------------

### HippoRAG: Knowledge Graph Retrieval with Python

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

This snippet demonstrates how to use the HippoRAG library for knowledge graph-based retrieval. It covers initializing the HippoRAG object, indexing documents to build the knowledge graph, retrieving relevant information based on queries, and generating answers using the retrieved context. Dependencies include the `HippoRAG` class and specified LLM/embedding models.

```python
from methods.hipporag import HippoRAG

# Initialize HippoRAG with configuration
hipporag = HippoRAG(
    save_dir="./outputs/hipporag_cache",
    llm_model_name="gpt-4o-mini",
    embedding_model_name="nvidia/NV-Embed-v2"  # or "text-embedding-ada-002"
)

# Index documents (builds knowledge graph)
documents = [
    "Albert Einstein developed the theory of relativity.",
    "The theory of relativity revolutionized our understanding of space and time.",
    "Einstein was awarded the Nobel Prize in Physics in 1921.",
    "Quantum mechanics was developed in the early 20th century."
]
hipporag.index(docs=documents)

# Retrieve relevant documents with knowledge graph reasoning
queries = ["What did Einstein contribute to physics?"]
retrieval_results, top_k_docs = hipporag.retrieve(
    queries=queries,
    num_to_retrieve=3
)

print(f"Retrieved documents: {top_k_docs}")

# Generate answers using retrieved context
qa_results = hipporag.rag_qa(retrieval_results)
answer = qa_results[0][0].answer
print(f"Answer: {answer}")
```

--------------------------------

### Configure OpenAI API Key

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Sets up the OpenAI API key by creating a .env file at the project root and adding the OPENAI_API_KEY variable with the user's actual API key.

```dotenv
OPENAI_API_KEY= ###your_openai_api_key
```

--------------------------------

### Mem0: Memory Management for Agents with Python

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

This snippet illustrates the usage of the Mem0 library for intelligent memory management in conversational agents. It shows how to initialize the memory system, add conversational messages with user context, and search for relevant memories based on a query. Dependencies include the `Memory` class from `mem0.memory.main`.

```python
from mem0.memory.main import Memory

# Initialize memory system
memory = Memory()

# Add conversational memories
messages = [
    {"role": "system", "content": "You are a helpful assistant."}, 
    {"role": "user", "content": "My name is John and I work at Anthropic."}, 
    {"role": "assistant", "content": "Nice to meet you, John! Working at Anthropic sounds exciting."}
]

# Store memories with user context
memory.add(messages, user_id="user_123")

# Add more context
memory.add([
    {"role": "user", "content": "I'm working on AI safety research."}, 
    {"role": "assistant", "content": "That's fascinating work!"}
], user_id="user_123")

# Search relevant memories
results = memory.search(
    query="What does John do for work?",
    user_id="user_123",
    limit=5
)

# Access retrieved memories
for entry in results["results"]:
    print(f"Memory: {entry['memory']}")
    print(f"Relevance: {entry.get('score', 'N/A')}")
```

--------------------------------

### Configure Other API Keys

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Sets up other API keys required for the project, such as Anthropic_API_KEY and Google_API_KEY, by adding them to the .env file.

```dotenv
Anthropic_API_KEY= ###your_anthropic_api
Google_API_KEY=    ###your_google_api
```

--------------------------------

### Template System for Task-Specific Prompts (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Implements a template system for generating task-specific prompts, including system messages, memorization prompts, and query prompts. It allows users to retrieve templates based on dataset, template name, and agent type, facilitating consistent prompt engineering for various agent configurations.

```Python
from utils.templates import get_template, SYSTEM_MESSAGE

# Get templates for different datasets and agent types
# Available datasets: ruler_qa, longmemeval, eventqa, in_context_learning,
#                     recsys_redial, infbench_sum, detective_qa, factconsolidation

# System message template
system = get_template(
    sub_dataset='ruler_qa1_197K',
    template_name='system',
    agent_name='Long_context_agent_gpt-4o-mini'
)
print(f"System: {system}")
# Output: "You are a helpful assistant that can read the context and memorize it..."

# Memorization template (for injecting context)
memorize = get_template(
    sub_dataset='ruler_qa1_197K',
    template_name='memorize',
    agent_name='Long_context_agent_gpt-4o-mini'
)
# Use with format: memorize.format(context="...", time_stamp="2024-01-01 10:00:00")

# Query template (for asking questions)
query = get_template(
    sub_dataset='ruler_qa1_197K',
    template_name='query',
    agent_name='Long_context_agent_gpt-4o-mini'  # or 'rag_agent', 'agentic_memory_agent'
)
formatted_query = query.format(question="What is the answer to life?")
print(f"Formatted query: {formatted_query}")
```

--------------------------------

### Load Evaluation Datasets (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Provides utilities to load evaluation datasets from HuggingFace Hub or local files. It supports automatic filtering and preprocessing of data, allowing users to specify dataset configurations like name, sub-dataset, and sample limits. The loaded data can be accessed and formatted for further use.

```Python
from utils.eval_data_utils import load_eval_data, load_data_huggingface, format_chat

# Load dataset using configuration dictionary
dataset_config = {
    'dataset': 'Accurate_Retrieval',
    'sub_dataset': 'ruler_qa1_197K',
    'max_test_samples': 50,
    'seed': 42
}

loaded_data = load_eval_data(dataset_config)
dataset = loaded_data['data']

# Access individual samples
sample = dataset[0]
print(f"Context length: {len(sample['context'])} characters")
print(f"Questions: {sample['questions']}")
print(f"Answers: {sample['answers']}")
print(f"Source: {sample['source']}")

# Load directly from HuggingFace
# Available datasets: Accurate_Retrieval, Test_Time_Learning,
#                     Long_Range_Understanding, Conflict_Resolution
hf_data = load_data_huggingface(
    dataset_name="Accurate_Retrieval",
    sub_dataset_source="ruler_qa1_197K",
    max_test_samples=100,
    seed=42
)

# Format messages for chat completion API
message = "What is the capital of France?"
system_message = "You are a helpful assistant that answers questions accurately."

formatted = format_chat(
    message=message,
    system_message=system_message,
    include_system=True
)
# Output: [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}]
```

--------------------------------

### Run Summarization Evaluation Python Script

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Executes the Python script for InfBench Summarization, which conducts LLM-based metric evaluation for summarization tasks within the InfBench benchmark. Configuration settings are necessary.

```python
python llm_based_eval/summarization_evaluate.py
```

--------------------------------

### Initialize TextRetriever for Embedding-Based Document Retrieval (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Shows the initialization of the TextRetriever class, which is part of the embedding-based document retrieval system. This class supports various embedding models for retrieving relevant text segments.

```python
from methods.embedding_retriever import TextRetriever, RAGSystem

```

--------------------------------

### Run Ablation Study for Chunk Size

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Executes an evaluation command for an ablation study focused on chunk size, specifically for RAG agents. This helps in understanding the impact of chunk size on agent performance.

```bash
bash bash_files/eniac/run_memagent_rag_agents_chunksize.sh
```

--------------------------------

### AgentWrapper - Main Agent Interface

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

The AgentWrapper class provides a unified interface for interacting with different memory agent implementations. It handles agent initialization, message sending for memorization and querying, and agent state persistence.

```APIDOC
## AgentWrapper - Main Agent Interface

### Description
The `AgentWrapper` class provides a unified interface for interacting with different memory agent implementations. It handles agent initialization, message sending for memorization and querying, and agent state persistence.

### Method
`AgentWrapper(agent_config, dataset_config, load_agent_from)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **agent_config** (dict) - Configuration for the agent.
- **dataset_config** (dict) - Configuration for the dataset.
- **load_agent_from** (str, optional) - Path to load agent state from.

### Request Example
```python
from agent import AgentWrapper

agent_config = {
    'agent_name': 'Long_context_agent_gpt-4o-mini',
    'model': 'gpt-4o-mini',
    'temperature': 0.7,
    'input_length_limit': 128000,
    'buffer_length': 4000,
    'output_dir': './outputs/gpt-4o-mini'
}

dataset_config = {
    'dataset': 'Accurate_Retrieval',
    'sub_dataset': 'ruler_qa1_197K',
    'context_max_length': 220000,
    'generation_max_length': 50,
    'chunk_size': 4096,
    'max_test_samples': 100,
    'seed': 42,
    'debug': False
}

agent = AgentWrapper(
    agent_config=agent_config,
    dataset_config=dataset_config,
    load_agent_from='./agent_states/context_0'
)

context_chunks = [
    "The capital of France is Paris. It is known for the Eiffel Tower.",
    "London is the capital of United Kingdom. Big Ben is located there."
]

for chunk in context_chunks:
    result = agent.send_message(chunk, memorizing=True, context_id=0)
    print(f"Memorization result: {result}")

query = "What is the capital of France?"
response = agent.send_message(
    query,
    memorizing=False,
    query_id=1,
    context_id=0
)

print(f"Answer: {response['output']}")
print(f"Input tokens: {response['input_len']}")
print(f"Output tokens: {response['output_len']}")
print(f"Query time: {response['query_time_len']:.2f}s")
```

### Response
#### Success Response (200)
- **output** (str) - The agent's response to the query.
- **input_len** (int) - The number of input tokens.
- **output_len** (int) - The number of output tokens.
- **query_time_len** (float) - The time taken to process the query in seconds.

#### Response Example
```json
{
  "output": "Paris",
  "input_len": 150,
  "output_len": 5,
  "query_time_len": 2.34
}
```
```

--------------------------------

### Evaluation Metrics Utilities (Python)

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

Imports various utility functions for calculating evaluation metrics. This includes general metrics like F1 score and ROUGE scores, as well as task-specific metrics such as exact match scores (DRQA and substring). It also includes helper functions for normalizing answers and post-processing results.

```Python
from utils.eval_other_utils import (
    calculate_metrics,
    f1_score,
    drqa_exact_match_score,
    substring_exact_match_score,
    normalize_answer,
    post_process,
    chunk_text_into_sentences
)
```

--------------------------------

### Calculate Evaluation Metrics with Python

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

This snippet demonstrates how to calculate various evaluation metrics such as exact match, F1 score, substring match, and ROUGE scores for comparing predictions against ground truth. It also shows how to normalize answers and compute F1 scores individually. Dependencies include the `calculate_metrics`, `normalize_answer`, `f1_score`, and `chunk_text_into_sentences` functions.

```python
prediction = "The capital of France is Paris."
ground_truth = ["Paris", "paris", "PARIS"]

metrics = calculate_metrics(prediction, ground_truth)
print(f"Exact Match: {metrics['exact_match']}")  # 0 or 1
print(f"F1 Score: {metrics['f1']:.4f}")  # 0.0 - 1.0
print(f"Substring Match: {metrics['substring_exact_match']}")  # True/False
print(f"ROUGE-L F1: {metrics['rougeL_f1']:.4f}")
print(f"ROUGE-L Recall: {metrics['rougeL_recall']:.4f}")

# Individual metric functions
normalized = normalize_answer("The Answer is: PARIS!")
print(f"Normalized: '{normalized}'")  # "answer is paris"

f1, precision, recall = f1_score(
    "Paris is the capital of France",
    "The capital is Paris"
)
print(f"F1: {f1:.4f}, Precision: {precision:.4f}, Recall: {recall:.4f}")

# Text chunking for long documents
long_text = "This is a very long document... " * 1000
chunks = chunk_text_into_sentences(
    text=long_text,
    model_name="gpt-4o-mini",
    chunk_size=4096  # tokens per chunk
)
print(f"Number of chunks: {len(chunks)}")
```

--------------------------------

### Configure Cognee Settings

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Configures settings for the Cognee model, including the LLM_MODEL (e.g., gpt-4o-mini) and the LLM_API_KEY, by adding these variables to the .env file.

```dotenv
LLM_MODEL=gpt-4o-mini
LLM_API_KEY=  ###your_api_key
```

--------------------------------

### Configure Custom Dataset

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

This YAML snippet defines configuration for a custom dataset named 'Accurate_Retrieval' with a sub-dataset 'ruler_qa1_197K'. It specifies chunk size, random seed, and various task-specific generation parameters like context length, generation length, and number of test samples.

```yaml
# configs/data_conf/custom_dataset.yaml

# Dataset configuration
dataset: Accurate_Retrieval  # Main category
sub_dataset: ruler_qa1_197K  # Specific task identifier
chunk_size: 4096  # Tokens per chunk
debug: false
seed: 42

# Task-specific settings
context_max_length: 220000
generation_max_length: 50
max_test_samples: 100
shots: 0
use_chat_template: true
```

--------------------------------

### Run LongmemEval Python Script

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/README.md

Executes the Python script for LongmemEval, which performs LLM-based metric evaluation for the LongMem benchmark. Requires appropriate configuration settings to be in place.

```python
python llm_based_eval/longmem_qa_evaluate.py
```

--------------------------------

### Configure RAG and Agentic Memory Settings

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

This snippet shows how to configure settings for RAG agents and agentic memory. For RAG, it specifies retrieval settings like `retrieve_num`. For agentic memory, it includes parameters such as `agent_chunk_size`, `letta_mode`, `text_embedding`, and `system_path`.

```yaml
# For RAG agents, add retrieval settings:
# retrieve_num: 10
# For agentic memory, add:
# agent_chunk_size: 2048
# letta_mode: chat  # or 'insert', 'api'
# text_embedding: text-embedding-3-small
# system_path: configs/agent_conf/letta_system.txt
```

--------------------------------

### Manage Theme with JavaScript

Source: https://github.com/hust-ai-hyz/memoryagentbench/blob/main/letta/server/static_files/index.html

This JavaScript code checks the 'theme' value in localStorage and applies the corresponding 'dark' class to the document's root element. It also handles system preference detection and updates localStorage accordingly. No external dependencies are required.

```javascript
if (localStorage.theme === 'dark') { if (document && document.documentElement) { document.documentElement.classList.add('dark'); } } else if (localStorage.theme === 'light') { if (document && document.documentElement) { document.documentElement.classList.remove('dark'); localStorage.setItem('theme', 'light'); } } else if (window.matchMedia('(prefers-color-scheme: dark)').matches) { localStorage.setItem('theme', 'system'); if (document && document.documentElement) { document.documentElement.classList.add('dark'); } } else { if (document && document.documentElement) { document.documentElement.classList.remove('dark'); } }
```

--------------------------------

### TextRetriever and RAGSystem - Embedding-Based Retrieval

Source: https://context7.com/hust-ai-hyz/memoryagentbench/llms.txt

The TextRetriever and RAGSystem classes provide embedding-based document retrieval with support for multiple embedding models including OpenAI, Contriever, Qwen3, and NV-Embed.

```APIDOC
## TextRetriever and RAGSystem - Embedding-Based Retrieval

### Description
The `TextRetriever` and `RAGSystem` classes provide embedding-based document retrieval with support for multiple embedding models including OpenAI, Contriever, Qwen3, and NV-Embed.

### Method
`TextRetriever(...)`
`RAGSystem(...)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
(Details for TextRetriever and RAGSystem initialization parameters would go here, including embedding model configurations, index paths, etc.)

### Request Example
```python
from methods.embedding_retriever import TextRetriever, RAGSystem

# Example initialization (specific parameters depend on usage)
# retriever = TextRetriever(embedding_model='openai', ...) 
# rag_system = RAGSystem(retriever=retriever, ...) 
```

### Response
(Details on the return types and data structures for TextRetriever and RAGSystem methods would go here.)

#### Success Response (200)
(Specific success responses depend on the methods called within TextRetriever and RAGSystem.)

#### Response Example
(Example responses would depend on the specific retrieval or RAG operations performed.)
```