### Install GainRAG Dependencies

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Installs the main dependencies for GainRAG using conda and pip. Requires Python 3.9.18, torch 2.5.1, vllm 0.7.3, FlagEmbedding 1.3.3, DeepSpeed, trl, peft, and faiss/faiss-gpu.

```bash
conda create -n GainRAG python=3.9.18
conda activate GainRAG
pip install -r requirements.txt
```

--------------------------------

### Construct RAG Prompts

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Utilities for building instruction-based prompts for LLMs. Includes logic for both retrieval-augmented generation (with context passages) and standard instruction-only prompts.

```python
def build_input_with_retrieval(data, k=5, task='HotpotQA'):
    for item in data:
        passages = item['ctxs'][:k]
        evidences = [
            f'Passage #{i+1}{p["title"]}\nPassage #{i+1}{p["text"]}'
            for i, p in enumerate(passages)
        ]
        evidence = '\n\n'.join(evidences)
        item['prompt'] = INSTRUCTION_PROMPT['Instruction_With_Retrieval'].format_map({
            'passage': evidence,
            'instruction': TASK_INST[task],
            'input': item['question']
        })
    return data
```

--------------------------------

### Execute RAG Generation and Evaluation Pipeline

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Orchestrates batch generation using vLLM and evaluates responses against ground truth using Exact Match and F1 metrics.

```python
for start in range(0, len(retrieved_inputs), batch_size):
    batch = retrieved_inputs[start:start + batch_size]
    prompts = [item['prompt'] for item in batch]
    tokens = llm_prompts("Llama-3-8B-Instruct", prompts, tokenizer, tokenize=False)
    responses = llm_response_vllm(model, tokens, max_tokens=256)
    for i, item in enumerate(batch):
        em = em_max_over_ground_truths(responses[i], item['answers'], regex=True)
        f1 = f1_max_over_ground_truths(responses[i], item['answers'])
        total_em += em
        total_f1 += f1
```

--------------------------------

### Signal Construction Pipeline (Python)

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Orchestrates the gain signal synthesis process, generating internal knowledge passages, computing retrieval-augmented predictions, and calculating contrastive PPL scores. It takes retrieved data as input and outputs formatted gain signals.

```python
# llm_supervision/construct_hf.py - Full signal construction
# Command line usage:
# python -m llm_supervision.construct_hf \
#     --data_path /path/to/retrieved_data.jsonl \
#     --output_path /path/to/train_data.json \
#     --task HotpotQA \
#     --lm_type Llama-3-8B-Instruct \
#     --alpha 0.5

from llm_supervision.construct_hf import signal_construction

# Input data format (JSONL):
# {
#   "question": "What is the capital of France?",
#   "answers": ["Paris"],
#   "ctxs": [
#     {"title": "France", "text": "...", "hasanswer": true, "score": "0.85"},
#     {"title": "Europe", "text": "...", "hasanswer": false, "score": "0.72"},
#     ...
#   ]
# }

# Run signal construction
signal_construction(
    data_path="/path/to/retrieved_data.jsonl",
    output_file="/path/to/gain_signals.json",
    lm_type="Llama-3-8B-Instruct",
    task="HotpotQA",
    alpha=0.5
)

# Output format:
# [
#   {
#     "question": "...",
#     "answers": ["..."],
#     "prompt_standard": "...",
#     "passages": [
#       {
#         "title": "...",
#         "text": "...",
#         "has_answer": true,
#         "retrieval_score": "0.85",
#         "prompt_retrieved": "...",
#         "EM_with_retrieval": 1,
#         "F1_with_retrieval": 1.0,
#         "PPL_CD": 2.34  # Contrastive perplexity (gain signal)
#       },
#       ...
#     ]
#   }
# ]
```

--------------------------------

### Synthesize Gain Signals for Training

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Constructs training data for the selector model by synthesizing gain signals. This script is used for data preparation and requires specifying data paths, task, and an alpha parameter.

```bash
cd ./gainRAG
python -m llm_supervision.construct_hf \
    --data_path  TODOpath/data.jsonl \
    --output_path  TODOpath/data_train.json \
    --task HotpotQA \
    --alpha 0.5
```

--------------------------------

### Initialize Contriever Retrieval Engine and Index Passages

Source: https://context7.com/liunian-jay/gainrag/llms.txt

This Python code initializes the Contriever model and tokenizer for dense passage retrieval. It sets up a FAISS indexer, loads pre-computed passage embeddings, and indexes them for efficient k-nearest neighbor search. The code also demonstrates embedding queries and retrieving top-k passages.

```python
from retrieval_engine.index import Indexer, load_passages
from retrieval_engine.modeling import Contriever
from transformers import AutoTokenizer
import numpy as np
import pickle
import torch

# Initialize the Contriever model
model = Contriever.from_pretrained("facebook/contriever-msmarco")
tokenizer = AutoTokenizer.from_pretrained("facebook/contriever-msmarco")
model.eval()
model = model.cuda()

# Create FAISS indexer (768-dim vectors, GPU-accelerated)
indexer = Indexer(
    vector_sz=768,
    n_subquantizers=0,  # Set > 0 for product quantization
    n_bits=8,
    use_gpu=True
)

# Load and index passage embeddings
# Expects pre-computed embeddings in pickle format: (ids, embeddings)
embedding_files = ["wikipedia_embeddings/embeddings_0.pkl"]
for file_path in embedding_files:
    with open(file_path, "rb") as f:
        ids, embeddings = pickle.load(f)
    indexer.index_data(ids, embeddings.astype('float32'))

# Save/load index for reuse
indexer.serialize("./index_dir")
indexer.deserialize_from("./index_dir")

# Embed queries and retrieve top-k passages
queries = ["What is the capital of France?", "Who wrote Romeo and Juliet?"]
with torch.no_grad():
    encoded = tokenizer.batch_encode_plus(
        queries, return_tensors="pt", max_length=512,
        padding=True, truncation=True
    )
    encoded = {k: v.cuda() for k, v in encoded.items()}
    query_embeddings = model(**encoded).cpu().numpy()

# Search for top 100 passages per query
top_ids_and_scores = indexer.search_knn(query_embeddings, top_docs=100)
# Returns: [(doc_ids, scores), ...] for each query
```

--------------------------------

### Data Conversion for Selector Training (Python)

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Converts gain signal data into the training format required by the selector model. It uses log-transformed PPL scores as soft labels for knowledge distillation and formats the data into query-passage pairs.

```python
# data/data2selector.py - Convert gain signals to selector training format
import json
import math

input_file = '/path/to/gain_signals.json'
output_file = '/path/to/selector_train.jsonl'

with open(input_file, "r", encoding="utf-8") as f:
    data = json.load(f)

new_data = []
for item in data:
    question = item['question']
    passages = item['passages']

    # Sort passages by PPL (lower is better)
    sorted_passages = sorted(passages, key=lambda x: x["PPL_CD"])

    # Extract texts and compute scores
    texts = [p['title'] + '\n' + p['text'] for p in sorted_passages]
    scores = [-math.log(p['PPL_CD'] + 1) for p in sorted_passages]

    # Format for selector training
    new_item = {
        'query': question,
        'pos': texts[0:1],           # Best passage (lowest PPL)
        'neg': texts[1:],            # Rest as negatives
        'pos_scores': scores[0:1],   # Soft labels for KD
        'neg_scores': scores[1:]
        'prompt': "Given a query A and a passage B, determine whether the passage directly or indirectly contains an answer to the query by providing a prediction of either 'Yes' or 'No'."
    }
    new_data.append(new_item)

with open(output_file, 'w', encoding='utf-8') as f:
    for record in new_data:
        f.write(json.dumps(record, ensure_ascii=False) + '\n')
```

--------------------------------

### Compute Gain Signals using Contrastive Decoding

Source: https://context7.com/liunian-jay/gainrag/llms.txt

This Python code defines the ContrastiveTool for calculating gain signals by measuring contrastive perplexity. It loads a causal language model and tokenizer, then uses the ContrastiveTool to compare LLM output distributions with and without retrieved context to quantify passage utility. It supports computing perplexity for single or multiple passages.

```python
from llm_supervision.decoding import ContrastiveTool
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load LLM for gain signal computation
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
model.eval()
model = model.cuda()

contrastive_tool = ContrastiveTool()

# Define prompts
query_prompt = """### Instruction:
Answer the question below concisely in a few words.

### Input:
What is the capital of France?
"""

query_prompt_with_context = """Passage #1 title: France
Passage #1 text: France is a country in Western Europe. Its capital is Paris.

### Instruction:
Answer the question below concisely in a few words.

### Input:
What is the capital of France?
"""

gold_answer = "Paris"
alpha = 0.5  # Contrastive weight coefficient

# Compute contrastive PPL for single passage
ppl = contrastive_tool.contrastive_PPL(
    model, tokenizer,
    query_prompt,              # Prompt without context
    query_prompt_with_context, # Prompt with retrieved passage
    gold_answer,               # Ground truth answer
    alpha=alpha
)
# Lower PPL indicates the passage helps more

# Compute contrastive PPL for multiple passages simultaneously
other_context_prompt = """Passage #2 text: Paris is the capital of France."""
multiple_contexts = [query_prompt_with_context, other_context_prompt]
ppl_list = contrastive_tool.contrastive_PPL_multi_pro(
    model, tokenizer,
    query_prompt,
    multiple_contexts,  # List of prompts with different passages
    gold_answer,
    alpha=alpha
)
# Returns: [ppl_for_passage_1, ppl_for_passage_2, ...] 
```

--------------------------------

### Train Selector Model

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Trains the selector model using PyTorch distributed training (torchrun) and DeepSpeed for optimization. It supports knowledge distillation and various training parameters like learning rate, batch size, and epochs.

```bash
cd ./gainRAG
torchrun --nproc_per_node 1 \
	-m selector_finetune \
	--model_name_or_path  path/bge-rerank-base \
    --train_data TODOpath/data.jsonl \
	--deepspeed TODOpath/deepspeed/ds_stage0.json \
	--output_dir TODOpath/model_outputs/\
	--overwrite_output_dir \
    --train_group_size 16 \
	--knowledge_distillation True \
    --query_max_len 512 \
    --passage_max_len 512 \
    --pad_to_multiple_of 8 \
    --learning_rate 6e-5 \
    --fp16 \
    --num_train_epochs 2 \
    --per_device_train_batch_size 8 \
    --gradient_accumulation_steps 1 \
    --dataloader_drop_last True \
    --warmup_ratio 0.1 \
    --gradient_checkpointing \
    --weight_decay 0.01 \
    --logging_steps 1 \
    --save_steps 1000
```

--------------------------------

### Run RAG Generation and Evaluation

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Executes the RAG workflow for generation and evaluation. This script takes the selector output, task type, and language model type as input to produce results.

```bash
cd ./gainRAG
python -m rag_workflow.rag_generation \
    --data_path "selector_output_path" \
    --task "HotpotQA" \
    --lm_type "Llama-3-8B-Instruct" \
    --K_docs 1
```

--------------------------------

### Perform vLLM Model Inference

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Functions to load models using vLLM and generate batched responses. Designed for high-performance inference with support for Llama and Mistral architectures.

```python
def llm_response_vllm(llm, prompts, max_tokens=256):
    sampling_params = SamplingParams(
        temperature=0.0,
        top_p=1.0,
        max_tokens=max_tokens,
        logprobs=1
    )
    outputs = llm.generate(prompts, sampling_params)
    return [output.outputs[0].text for output in outputs]
```

--------------------------------

### Apply Llama 3 Chat Templates

Source: https://context7.com/liunian-jay/gainrag/llms.txt

This function formats input queries using the Llama 3 chat template structure. It handles both single strings and lists of queries, optionally returning tokenized input tensors suitable for model inference.

```python
def llm_prompts(lm_type, query, tokenizer, tokenize=True):
    def build_llama3_prompt(input_text):
        messages = [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": input_text}
        ]
        return tokenizer.apply_chat_template(
            messages, tokenize=False, add_generation_prompt=True
        )

    queries = query if isinstance(query, list) else [query]
    prompts = [build_llama3_prompt(q) for q in queries]

    if tokenize:
        tokenizer.pad_token = tokenizer.eos_token
        return tokenizer(prompts, return_tensors="pt", padding=True)
    return prompts
```

--------------------------------

### Selector Training Script (Bash)

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Shell script for distributed training of a cross-encoder reranker model using DeepSpeed. It configures training parameters, data paths, and model settings for knowledge distillation.

```bash
# scripts/selector_finetune.sh - Distributed training with DeepSpeed
cd ./gainRAG
torchrun --nproc_per_node 1 \
    -m selector_finetune \
    --model_name_or_path BAAI/bge-reranker-base \
    --train_data /path/to/selector_train.jsonl \
    --deepspeed /path/to/ds_stage0.json \
    --output_dir /path/to/selector_model/ \
    --overwrite_output_dir \
    --train_group_size 16 \
    --knowledge_distillation True \
    --query_max_len 512 \
    --passage_max_len 512 \
    --pad_to_multiple_of 8 \
    --learning_rate 6e-5 \
    --fp16 \
    --num_train_epochs 2 \
    --per_device_train_batch_size 8 \
    --gradient_accumulation_steps 1 \
    --dataloader_drop_last True \
    --warmup_ratio 0.1 \
    --gradient_checkpointing \
    --weight_decay 0.01 \
    --logging_steps 1 \
    --save_steps 1000
```

--------------------------------

### Convert Data for Selector

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Converts data into the format required by the selector model. This is a data preparation step that may require custom parameter configuration.

```bash
cd ./data
python data2selector.py # Remember to configure your parameters
```

--------------------------------

### RAG Generation and Evaluation

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Pipeline for generating answers using vLLM-accelerated inference and evaluating performance with EM and F1 metrics.

```APIDOC
## RAG Generation and Evaluation

### Description
This script orchestrates the Retrieval-Augmented Generation (RAG) pipeline. It loads selected passages, builds prompts for a large language model (LLM), generates answers using vLLM for efficient inference, and evaluates the generated answers using Exact Match (EM) and F1 score metrics.

### Command Line Usage
```bash
python -m rag_workflow.rag_generation \
    --data_path /path/to/selected_data.json \
    --task HotpotQA \
    --lm_type Llama-3-8B-Instruct \
    --K_docs 1 \
    --batch_size 256
```

### Parameters
#### Command Line Arguments
- **--data_path** (str) - Required - Path to the data file containing selected passages (JSON format).
- **--task** (str) - Required - The specific task or dataset (e.g., "HotpotQA").
- **--lm_type** (str) - Required - The type or name of the language model to use (e.g., "Llama-3-8B-Instruct").
- **--K_docs** (int) - Optional - The number of retrieved documents to consider for generation (default is 1).
- **--batch_size** (int) - Optional - The batch size for LLM inference (default is 256).

### Core Functionality
- **`load_model_vllm`**: Loads a language model using vLLM for accelerated inference.
- **`llm_response_vllm`**: Generates responses from the loaded vLLM model.
- **`llm_prompts`**: Formats prompts according to the specified LLM's chat template.
- **`em_max_over_ground_truths`**: Calculates the Exact Match score.
- **`f1_max_over_ground_truths`**: Calculates the F1 score.
- **`load_data`**: Loads the input data.
- **`build_input_with_retrieval`**: Constructs the input prompts with retrieved context.

### Request Example (Input Data Format)
`/path/to/selected_data.json`:
```json
[
    {
        "question": "What is the capital of France?",
        "ctxs": [
            {"title": "Paris", "text": "Paris is the capital and most populous city of France..."}
        ],
        "answers": ["Paris"]
    }
]
```

### Response
#### Success Response (Console Output)
- **EM**: Exact Match score percentage.
- **F1**: F1 score percentage.

#### Response Example (Console Output)
```
EM: 95.20%
F1: 96.50%
```
```

--------------------------------

### CrossEncoderModel for Knowledge Distillation

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Implements a CrossEncoderModel for knowledge distillation from soft labels, calculating KL divergence loss.

```APIDOC
## CrossEncoderModel

### Description
This class implements a cross-encoder model for knowledge distillation, utilizing soft labels from a teacher model to train a student model. It computes the Kullback-Leibler (KL) divergence loss between the teacher's and student's probability distributions.

### Method
`__init__(self, base_model, tokenizer=None, train_batch_size=4)`: Initializes the CrossEncoderModel.
`encode(self, features)`: Performs a forward pass through the cross-encoder model to obtain logits.
`forward(self, pair, teacher_scores=None)`: Computes the KL divergence loss for knowledge distillation during training. Returns RerankerOutput containing loss and scores.

### Parameters
#### Request Body (for `forward` method)
- **pair** (dict) - Required - Input features for the model.
- **teacher_scores** (list of float) - Optional - Soft labels from the teacher model for knowledge distillation.

### Request Example (Conceptual)
```json
{
  "pair": {
    "input_ids": [ ... ],
    "attention_mask": [ ... ]
  },
  "teacher_scores": [0.1, 0.9, ...]
}
```

### Response (for `forward` method)
#### Success Response (200)
- **loss** (torch.Tensor | None) - The KL divergence loss if in training mode, otherwise None.
- **scores** (torch.Tensor) - The raw logits output by the model.

#### Response Example
```json
{
  "loss": 0.05,
  "scores": [ ... ]
}
```
```

--------------------------------

### Evaluate QA Model Performance

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Provides functions to calculate Exact Match (EM) and F1 scores for QA models. It supports text normalization and handles multiple ground truth answers by selecting the maximum score.

```python
from evaluation.evaluation import get_evaluation

em_score, f1_score = get_evaluation(
    task="HotpotQA",
    res_answer="Paris",
    gold_answers=["Paris", "Paris, France"],
    regex=True
)
```

--------------------------------

### Retrieve Top-K Passages

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Executes the retrieval process to find the top-k relevant passages. This script is part of the retrieval engine and requires parameter configuration.

```bash
cd ./gainRAG/retrieval_engine
python retrieval.py # Remember to configure your parameters
```

--------------------------------

### Passage Selection with Trained Selector

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Inference script for selecting top-k relevant passages using a trained selector model, optimized for RAG.

```APIDOC
## Passage Selection with Trained Selector

### Description
This script utilizes a trained selector model (e.g., a reranker) to identify and select the most relevant passages from a set of retrieved documents for a given query. It reranks the passages based on a computed score and retains the top-K passages for downstream RAG generation.

### Command Line Usage
```bash
python -m selector_engine.selector_gainRag \
    --model_name_or_path /path/to/selector_model/ \
    --data_path /path/to/retrieved_data.jsonl \
    --output_path /path/to/selected_data.json \
    --K_docs 1
```

### Parameters
#### Command Line Arguments
- **--model_name_or_path** (str) - Required - Path to the pre-trained selector model.
- **--data_path** (str) - Required - Path to the input data file containing retrieved passages (JSON Lines format).
- **--output_path** (str) - Required - Path to save the selected passages (JSON format).
- **--K_docs** (int) - Optional - The number of top documents to select (default is 1).

### Core Functionality
- **`get_top_k_passages(query, passages, model, k=5)`**: Reranks a list of passages for a given query using the provided model and returns the indices and scores of the top-k passages.

### Request Example (Input Data Format)
`/path/to/retrieved_data.jsonl`:
```json
{"question": "What is the capital of France?", "ctxs": [{"title": "Paris", "text": "Paris is the capital and most populous city of France..."}, {"title": "France", "text": "France, officially the French Republic, is a country..."}]}
```

### Response Example (Output Data Format)
`/path/to/selected_data.json`:
```json
[
    {
        "question": "What is the capital of France?",
        "ctxs": [
            {"title": "Paris", "text": "Paris is the capital and most populous city of France..."}
        ]
    }
]
```
```

--------------------------------

### Implement Cross-Encoder Knowledge Distillation

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Defines a CrossEncoderModel class that computes KL divergence loss between student logits and teacher scores. This is used to distill knowledge from a gain signal into a reranker model.

```python
class CrossEncoderModel(AbsRerankerModel):
    def __init__(self, base_model, tokenizer=None, train_batch_size=4):
        super().__init__(base_model, tokenizer, train_batch_size)

    def encode(self, features):
        return self.model(**features, return_dict=True).logits

    def forward(self, pair, teacher_scores=None):
        ranker_logits = self.encode(pair)
        if self.training:
            grouped_logits = ranker_logits.view(self.train_batch_size, -1)
            teacher_targets = torch.Tensor(teacher_scores).view(self.train_batch_size, -1).to(grouped_logits.device)
            teacher_targets = torch.softmax(teacher_targets.detach(), dim=-1)
            student_inputs = torch.log_softmax(grouped_logits, dim=-1)
            loss = F.kl_div(student_inputs, teacher_targets, reduction="batchmean")
        else:
            loss = None
        return RerankerOutput(loss=loss, scores=ranker_logits)
```

--------------------------------

### Select Top-1 Passages for Evaluation

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Selects the top-k passages using the trained selector model for evaluation. This script is part of the selector engine and requires specifying model path, data path, and output path.

```bash
cd ./gainRAG
python -m selector_engine.selector_gainRag \
    --model_name_or_path "model_path/" \
    --data_path "path/GainRAG/data/eval_data/HotpotQA.jsonl" \
    --output_path "path/GainRAG/data/test.json" \
    --K_docs 1
```

--------------------------------

### CrossEncoder Model Architecture (Python)

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Defines the CrossEncoder model architecture used for selector training, inheriting from AbsRerankerModel and utilizing KL divergence loss. It leverages transformers for sequence classification.

```python
# selector_finetune/modeling.py - CrossEncoder model architecture
from selector_finetune.modeling import CrossEncoderModel
from selector_finetune.abc import AbsRerankerModel, RerankerOutput
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Model inherits from AbsRerankerModel and uses KL divergence loss

```

--------------------------------

### Evaluation Metrics

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Provides functions for calculating Exact Match (EM) and F1 scores for evaluating generated text against ground truth answers.

```APIDOC
## Evaluation Metrics

### Description
This module provides utility functions for evaluating the quality of generated text, specifically focusing on Exact Match (EM) and F1 scores. These metrics are crucial for assessing the accuracy and completeness of answers generated by RAG systems, especially when compared against multiple ground truth answers.

### Functions
- **`em_max_over_ground_truths(prediction, ground_truths, regex=False)`**:
  Calculates the Exact Match score between a single prediction and a list of ground truth answers. If `regex` is True, the prediction is matched against ground truths using regular expressions.

- **`f1_max_over_ground_truths(prediction, ground_truths)`**:
  Calculates the F1 score between a single prediction and a list of ground truth answers. This involves tokenizing both the prediction and ground truths and computing precision, recall, and F1.

### Parameters
#### `em_max_over_ground_truths`
- **prediction** (str) - The generated answer string.
- **ground_truths** (list of str) - A list of possible correct answer strings.
- **regex** (bool) - Optional - Whether to use regex matching (default is False).

#### `f1_max_over_ground_truths`
- **prediction** (str) - The generated answer string.
- **ground_truths** (list of str) - A list of possible correct answer strings.

### Usage Example
```python
from evaluation.evaluation import em_max_over_ground_truths, f1_max_over_ground_truths

prediction = "Paris is the capital of France."
ground_truths = ["Paris", "The capital of France is Paris"]

em_score = em_max_over_ground_truths(prediction, ground_truths)
f1_score = f1_max_over_ground_truths(prediction, ground_truths)

print(f"EM Score: {em_score}")
print(f"F1 Score: {f1_score}")
```

### Response Example (from Usage Example)
```
EM Score: 0.0
F1 Score: 0.75
```
```

--------------------------------

### Download Passage Embeddings

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Downloads pre-computed passage embeddings for retrieval. Supports embeddings generated by Contriever and Contriever-msmarco, which are used to represent passages for efficient searching.

```bash
wget https://dl.fbaipublicfiles.com/contriever/embeddings/contriever/wikipedia_embeddings.tar
wget https://dl.fbaipublicfiles.com/contriever/embeddings/contriever-msmarco/wikipedia_embeddings.tar
```

--------------------------------

### Download DPR Wikipedia Passages

Source: https://github.com/liunian-jay/gainrag/blob/master/README.md

Downloads the Wikipedia passages used in DPR for retrieval. These passages are essential for the retrieval component of the GainRAG framework.

```bash
wget https://dl.fbaipublicfiles.com/dpr/wikipedia_split/psgs_w100.tsv.gz
```

--------------------------------

### Perform Passage Selection with Trained Selector

Source: https://context7.com/liunian-jay/gainrag/llms.txt

Uses a trained reranker model to score retrieved passages against a query and select the top-k documents. It processes JSONL data and outputs the filtered results.

```python
def get_top_k_passages(query, passages, model, k=5):
    pairs = [(query, passage) for passage in passages]
    scores = model.compute_score(pairs)
    scores_np = np.array(scores)
    top_k_indices = np.argsort(scores_np)[-k:][::-1]
    top_k_scores = scores_np[top_k_indices]
    return top_k_indices, top_k_scores
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.