### Install Dependencies with Pip

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Installs the necessary Python libraries for the project using a requirements file. Ensure you have pip installed.

```bash
pip install -r requirements.txt
```

--------------------------------

### Setup ALCE Environment for ASQA Evaluation

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Clones the ALCE repository and sets up the necessary environment for evaluating ASQA. This involves cloning the git repository, initializing the environment, and downloading task-specific data.

```bash
git clone https://github.com/princeton-nlp/ALCE.git
python3 -m alce_env
cd ALCE
bash download_data.sh
```

--------------------------------

### Create Conda Environment

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Sets up a dedicated Conda environment for the project, isolating dependencies. Requires Conda to be installed.

```bash
conda env create -f environment.yml
```

--------------------------------

### Short-Form QA Evaluation: Input File Format and Key Parameters

Source: https://context7.com/akariasai/self-rag/llms.txt

Describes the expected JSONL format for the input file used in short-form QA evaluation, including example fields like question, answers, and contexts. It also lists and explains key parameters for the evaluation script, such as retrieval mode, threshold, and flags for enabling specific scoring metrics.

```yaml
# Input file format (JSONL):
# {"question": "Who is the current president?", "answers": ["Joe Biden"], "ctxs": [...]}

# Key parameters:
# - mode: 'adaptive_retrieval' (threshold-based), 'no_retrieval', or 'always_retrieve'
# - threshold: 0.2 (probability threshold for retrieval decision)
# - use_groundness: Enable [Fully supported] token scoring
# - use_utility: Enable [Utility:1-5] token scoring
# - use_seqscore: Include sequence log probability in final score
```

--------------------------------

### Load and Run Self-RAG Model with vLLM for Inference

Source: https://context7.com/akariasai/self-rag/llms.txt

Loads a Self-RAG model using vLLM for inference, allowing for queries with or without provided evidence. It demonstrates how to format prompts and generate text, showing example outputs with and without retrieval tokens. Dependencies include `vllm`.

```python
from vllm import LLM, SamplingParams

# Load Self-RAG model
model = LLM("selfrag/selfrag_llama2_7b", download_dir="/model_cache", dtype="half")
sampling_params = SamplingParams(temperature=0.0, top_p=1.0, max_tokens=100, skip_special_tokens=False)

def format_prompt(input, paragraph=None):
    prompt = "### Instruction:\n{0}\n\n### Response:\n".format(input)
    if paragraph is not None:
        prompt += "[Retrieval]<paragraph>{0}</paragraph>".format(paragraph)
    return prompt

# Query without retrieval (model decides)
query = "Leave odd one out: twitter, instagram, whatsapp."
preds = model.generate([format_prompt(query)], sampling_params)
print(preds[0].outputs[0].text)
# Output: Twitter, Instagram, and WhatsApp are all social media platforms. [No Retrieval]WhatsApp is the odd one out...

# Query with provided evidence
query_with_evidence = "Can you tell me the difference between llamas and alpacas?"
evidence = "The alpaca (Lama pacos) is a species of South American camelid mammal. It is similar to, and often confused with, the llama. Alpacas are considerably smaller than llamas, and unlike llamas, they were not bred to be working animals, but were bred specifically for their fiber."
prompt_with_evidence = format_prompt(query_with_evidence, evidence)
preds = model.generate([prompt_with_evidence], sampling_params)
print(preds[0].outputs[0].text)
# Output: [Relevant]Alpacas are considerably smaller than llamas...[Fully supported][Utility:5]</s>
```

--------------------------------

### Format Prompts for Different Tasks (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This section shows how to format prompts for various Natural Language Processing tasks using predefined templates. It includes examples for standard question answering (QA) and retrieval-augmented generation (RAG) prompts, as well as a long-form QA prompt with task instructions. These formatted prompts are crucial for guiding the language model's responses.

```python
from utils import TASK_INST

# QA prompt
qa_prompt = PROMPT_DICT["prompt_no_input"].format(instruction="What is the capital of France?")

# RAG prompt with evidence
rag_prompt = PROMPT_DICT["prompt_no_input_retrieval"].format(
    instruction="What is the capital of France?",
    paragraph="Paris is the capital and most populous city of France..."
)

# Long-form QA with task instruction
asqa_instruction = TASK_INST["asqa"] + "## Input:\n\nWhat are the causes of climate change?"
```

--------------------------------

### Run ASQA Evaluation using ALCE

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Performs ASQA evaluations using the ALCE toolkit after setup. This command requires the output file from the Self-RAG model and enables various evaluation metrics like citations, QA, and MAUVE.

```python
python eval.py --f YOUR_OUTPUT_FILE --citations --qa --mauve
```

--------------------------------

### Self-RAG Generator Training Data Format (JSONL)

Source: https://context7.com/akariasai/self-rag/llms.txt

This snippet illustrates the JSONL format for training data used by the Self-RAG generator model. It shows examples of instructions and their corresponding outputs, which include special tokens indicating retrieval status, relevance, factual support, and utility. The format demonstrates how the model learns to generate responses with integrated Self-RAG capabilities.

```json
{
  "instruction": "What is the capital of France?",
  "output": "[No Retrieval]The capital of France is Paris.[Utility:5]"
}

{
  "instruction": "What causes climate change?",
  "output": "[Retrieval]<paragraph>Climate change is caused by greenhouse gas emissions...</paragraph>[Relevant]Climate change is primarily caused by greenhouse gas emissions from human activities.[Fully supported][Utility:5]"
}
```

--------------------------------

### Setting up and Using the Retrieval System Demo

Source: https://github.com/akariasai/self-rag/blob/main/README.md

This snippet shows how to initialize and use the Retriever for a demo. It sets up the retriever with a specific model and corpus, then uses it to search for relevant documents based on a query. Finally, it formats prompts using the retrieved documents and generates predictions.

```python
from passage_retrieval import Retriever
retriever = Retriever({})
retriever.setup_retriever_demo("facebook/contriever-msmarco", "enwiki_2020_intro_only/enwiki_2020_dec_intro_only.jsonl", "enwiki_2020_intro_only/enwiki_dec_2020_contriever_intro/*",  n_docs=5, save_or_load_index=False)
retrieved_documents = retriever.search_document_demo(query_3, 5)
prompts = [format_prompt(query_3, doc["title"] +"\n"+ doc["text"]) for doc in retrieved_documents] preds = model.generate(prompts, sampling_params)
top_doc = retriever.search_document_demo(query_3, 1)[0]
print("Reference: {0}\nModel prediction: {1}".format(top_doc["title"] + "\n" + top_doc["text"], preds[0].outputs[0].text))
```

--------------------------------

### Cloning Repository and Downloading Demo Corpus

Source: https://github.com/akariasai/self-rag/blob/main/README.md

These shell commands demonstrate how to clone the self-rag repository and download the demo corpus, which includes introductory paragraphs from Wikipedia articles. This data is used for running retrieval demonstrations.

```shell
git clone git@github.com:AkariAsai/self-rag.git
cd retrieval_lm
bash download_demo_corpus.sh
```

--------------------------------

### Run Vanilla LM Baseline with OpenAI API

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Executes a vanilla language model baseline using an OpenAI API model. Requires the model name, input and output file paths, and an API key file. The organization key also needs to be set in the script. Supports various tasks like 'qa'.

```shell
python run_baseline_lm.py \
--model_name gpt-3.5-turbo-0301 \
--input_file INPUT_FILE_SAME_AS_SELF_RAG \
--max_new_tokens 100 --metric match \
--result_fp RESULT_FILE_PATH \
 --task qa \
--api_key YOUR_OPEN_AI_API_KEY_FILE \
--prompt_name "prompt_no_input"
```

--------------------------------

### Critic Model Training Configuration (Bash)

Source: https://context7.com/akariasai/self-rag/llms.txt

This bash command initiates the training of a critic model for special tokens. It specifies the number of GPUs, master port, model name, data path, output directory, and various training parameters such as number of epochs, batch size, gradient accumulation steps, learning rate, and scheduler type. It also includes FSDP configuration for distributed training.

```bash
cd data_creation
torchrun --nproc_per_node=2 --master_port=2568 train_special_tokens.py \
  --model_name_or_path meta-llama/Llama-2-7b-hf \
  --data_path critic_training_data.json \
  --bf16 True \
  --output_dir models/critic_7b \
  --num_train_epochs 3 \
  --per_device_train_batch_size 1 \
  --per_device_eval_batch_size 1 \
  --gradient_accumulation_steps 8 \
  --evaluation_strategy "no" \
  --save_strategy "steps" \
  --save_steps 300 \
  --save_total_limit 1 \
  --learning_rate 2e-5 \
  --weight_decay 0. \
  --warmup_ratio 0.01 \
  --lr_scheduler_type "cosine" \
  --logging_steps 10 \
  --fsdp "full_shard auto_wrap"
```

--------------------------------

### Run Vanilla LM Baseline with Huggingface Model

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Executes a vanilla language model baseline using a Huggingface model. Requires specifying the model name, input file, and output file path. Supports various tasks like 'qa' and 'fever', with specific instructions for setting task names for PubHealth and ARC.

```shell
python run_baseline_lm.py \
--model_name meta-llama/Llama-2-7b-hf \
--input_file INPUT_FILE_SAME_AS_SELF_RAG \
 --max_new_tokens 100 --metric match \
--result_fp RESULT_FILE_PATH --task qa --prompt_name "prompt_no_input"
```

```shell
python run_baseline_lm.py \
--model_name meta-llama/Llama-2-7b-hf \
--input_file eval_data/health_claims_processed.jsonl \
--max_new_tokens 20 \
--metric accuracy \
--result_fp llama2_7b_pubhealth_results.json \
--task fever
```

--------------------------------

### Run Retrieval-Augmented Baseline with Huggingface Model

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Executes a retrieval-augmented language model baseline using a Huggingface model. This involves specifying the model, input/output files, task, and setting the mode to 'retrieval'. A specific prompt name for retrieval is also required.

```shell
python run_baseline_refactor.py \
--model_name meta-llama/Llama-2-7b-hf \
--input_file INPUT_FILE_SAME_AS_SELF_RAG \
 --max_new_tokens 100 --metric match \
--result_fp RESULT_FILE_PATH --task qa \
--mode retrieval \
--prompt_name "prompt_no_input_retrieval"
```

--------------------------------

### Run Baseline Models for Comparison (Bash)

Source: https://context7.com/akariasai/self-rag/llms.txt

These bash commands execute baseline language models and RAG models for comparison. They specify the model name, input file, output file, and task type. Different prompts and modes (e.g., 'retrieval' for RAG) can be configured.

```bash
# Vanilla Llama2 baseline
python run_baseline_lm.py \
  --model_name meta-llama/Llama-2-7b-hf \
  --input_file eval_data/popqa_longtail_w_gs.jsonl \
  --max_new_tokens 100 \
  --metric match \
  --result_fp results/llama2_baseline.json \
  --task qa \
  --prompt_name "prompt_no_input"

# RAG baseline (always retrieves)
python run_baseline_lm.py \
  --model_name meta-llama/Llama-2-7b-hf \
  --input_file eval_data/popqa_longtail_w_gs.jsonl \
  --max_new_tokens 100 \
  --metric match \
  --result_fp results/llama2_rag_baseline.json \
  --task qa \
  --mode retrieval \
  --prompt_name "prompt_no_input_retrieval"

# GPT-3.5 baseline with OpenAI API
python run_baseline_lm.py \
  --model_name gpt-3.5-turbo-0301 \
  --input_file eval_data/popqa_longtail_w_gs.jsonl \
  --max_new_tokens 100 \
  --metric match \
  --result_fp results/gpt35_baseline.json \
  --task qa \
  --api_key openai_key.txt \
  --prompt_name "prompt_no_input"
```

--------------------------------

### Run Retrieval-Augmented Baseline with OpenAI API

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Executes a retrieval-augmented language model baseline using an OpenAI API model. Configuration includes model name, input/output files, task, API key, and setting the mode to 'retrieval'. A specific prompt name for retrieval is also necessary.

```shell
python run_baseline_lm.py \
--model_name gpt-3.5-turbo-0301 \
--input_file INPUT_FILE_SAME_AS_SELF_RAG \
--max_new_tokens 100 --metric match \
--result_fp RESULT_FILE_PATH \
 --task qa \
--api_key YOUR_OPEN_AI_API_KEY_FILE \
--mode retrieval \
--prompt_name "prompt_no_input_retrieval"
```

--------------------------------

### Self-RAG Inference with vLLM

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Demonstrates how to perform inference using the Self-RAG model with the vLLM library. It loads a pre-trained model and generates responses to queries, showing dynamic retrieval behavior.

```python
from vllm import LLM, SamplingParams

model = LLM("selfrag/selfrag_llama2_7b", download_dir="/gscratch/h2lab/akari/model_cache", dtype="half")
sampling_params = SamplingParams(temperature=0.0, top_p=1.0, max_tokens=100, skip_special_tokens=False)

def format_prompt(input, paragraph=None):
  prompt = "### Instruction:\n{0}\n\n### Response:\n".format(input)
  if paragraph is not None:
    prompt += "[Retrieval]<paragraph>{0}</paragraph>".format(paragraph)
  return prompt

query_1 = "Leave odd one out: twitter, instagram, whatsapp."
query_2 = "Can you tell me the difference between llamas and alpacas?"
queries = [query_1, query_2]

# for a query that doesn't require retrieval
preds = model.generate([format_prompt(query) for query in queries], sampling_params)
for pred in preds:
  print("Model prediction: {0}".format(pred.outputs[0].text))
```

--------------------------------

### Generator Model Training Script Configuration (Bash)

Source: https://context7.com/akariasai/self-rag/llms.txt

This bash script prepares for training the Self-RAG generator model. It involves changing the directory to 'retrieval_lm' and then executing 'script_finetune_7b.sh'. It indicates that environment variables such as MODEL_SIZE, NUM_GPUS, BATCH_SIZE_PER_GPU, GRADIENT_ACC_STEPS, MODEL_PATH, DATA_PATH, and OUTPUT_PATH need to be edited within the script before execution.

```bash
cd retrieval_lm

# Edit script_finetune_7b.sh with paths:
# MODEL_SIZE=7B
# NUM_GPUS=8
# BATCH_SIZE_PER_GPU=1
# GRADIENT_ACC_STEPS=8
# MODEL_PATH=meta-llama/Llama-2-7b-hf
# DATA_PATH=selfrag_train_data.json
# OUTPUT_PATH=models/selfrag_7b

bash script_finetune_7b.sh
```

--------------------------------

### Evaluate FactScore using FActScore Repository

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Evaluates generated outputs using the official FActScore repository. This command requires the output file from Self-RAG, specifies the model type for evaluation, and requires cache directory and OpenAI API key configurations.

```python
python -m factscore.factscorer --data_path YOUR_OUTPUT_FILE  --model_name retrieval+ChatGPT --cache_dir YOUR_CACHE_DIR --openai_key YOUR_OPEN_AI_KEY --verbose
```

--------------------------------

### Run Short-Form QA Evaluation with Adaptive Retrieval

Source: https://context7.com/akariasai/self-rag/llms.txt

Executes an evaluation script for short-answer question-answering datasets using the Self-RAG model with adaptive retrieval. It specifies various command-line arguments to control the model, input/output files, retrieval mode, and evaluation metrics. This is a bash command.

```bash
python run_short_form.py \
  --model_name selfrag/selfrag_llama2_7b \
  --input_file eval_data/popqa_longtail_w_gs.jsonl \
  --mode adaptive_retrieval \
  --max_new_tokens 100 \
  --threshold 0.2 \
  --output_file results/popqa_output.json \
  --metric match \
  --ndocs 10 \
  --use_groundness \
  --use_utility \
  --use_seqscore \
  --dtype half
```

--------------------------------

### Load Special Tokens and Tokenizer (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This Python script demonstrates how to load a tokenizer and its associated special tokens for the Self-RAG model. It uses the `transformers` library and a custom `utils` module to load token IDs for reflection, relevance, grounding, and utility tokens.

```python
from utils import load_special_tokens, PROMPT_DICT, control_tokens, postprocess
from transformers import AutoTokenizer

# Load tokenizer with special tokens
tokenizer = AutoTokenizer.from_pretrained("selfrag/selfrag_llama2_7b")

# Get token IDs for reflection tokens
ret_tokens, rel_tokens, grd_tokens, ut_tokens = load_special_tokens(
    tokenizer,
    use_grounding=True,
    use_utility=True
)

print(ret_tokens)
# {
#   "[No Retrieval]": 32000,
#   "[Retrieval]": 32001,
#   "[Continue to Use Evidence]": 32002
# }

print(rel_tokens)
# {"[Irrelevant]": 32003, "[Relevant]": 32004}

print(grd_tokens)
# {
#   "[Fully supported]": 32011,
#   "[Partially supported]": 32012,
#   "[No support / Contradictory]": 32013
# }

print(ut_tokens)
# {
#   "[Utility:1]": 32006, "[Utility:2]": 32007,
#   "[Utility:3]": 32008, "[Utility:4]": 32009, "[Utility:5]": 32010
# }
```

--------------------------------

### Downloading Preprocessed Passage Data for DPR

Source: https://github.com/akariasai/self-rag/blob/main/README.md

This command downloads the preprocessed passage data used in DPR (Dense Passage Retrieval). The data is in a compressed TSV format and is a prerequisite for running the retriever.

```shell
cd retrieval_lm
wget https://dl.fbaipublicfiles.com/dpr/wikipedia_split/psgs_w100.tsv.gz
```

--------------------------------

### Preprocess Data for Initial Retrieval

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

A Python script to preprocess input files for the initial retrieval step. It takes input files and an output file path as arguments.

```bash
python create_retrieval_data.py \
    --input_files INPUT_FILE \
    --output_file INITIAL_RETRIEVAL_INPUT
```

--------------------------------

### Initialize and Use Contriever for Document Retrieval

Source: https://context7.com/akariasai/self-rag/llms.txt

Sets up a Contriever-based retrieval system for fetching relevant passages from a corpus, demonstrated with Wikipedia data. It includes initializing the retriever, setting up the index, searching for documents based on a query, and using the retrieved documents to prompt the Self-RAG model. Dependencies include `passage_retrieval` and `glob`.

```python
from passage_retrieval import Retriever
import glob

# Initialize retriever
retriever = Retriever({})

# Setup with Wikipedia embeddings
retriever.setup_retriever_demo(
    model_name_or_path="facebook/contriever-msmarco",
    passages="enwiki_2020_intro_only/enwiki_2020_dec_intro_only.jsonl",
    passages_embeddings="enwiki_2020_intro_only/enwiki_dec_2020_contriever_intro/*",
    n_docs=5,
    save_or_load_index=False
)

# Search for documents
query = "What is overfitting in machine learning?"
retrieved_docs = retriever.search_document_demo(query, n_docs=5)

# Use retrieved documents with Self-RAG
prompts = [format_prompt(query, doc["title"] + "\n" + doc["text"]) for doc in retrieved_docs]
preds = model.generate(prompts, sampling_params)

# Display top result
top_doc = retrieved_docs[0]
print(f"Reference: {top_doc['title']}\n{top_doc['text']}")
print(f"Model prediction: {preds[0].outputs[0].text}")
# Model generates grounded response using retrieved evidence
```

--------------------------------

### Critic Model Training Data Format (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This Python dictionary represents the training data format for the critic model. It includes an 'instruction' field describing the task, an 'input' field containing the question and answer, and an 'output' field with the expected reflection token (e.g., '[No Retrieval]'). This structure helps the critic model learn to predict appropriate reflection tokens based on given context.

```python
{
  "instruction": "Given the question and answer, evaluate if retrieval is needed.",
  "input": "Question: What is the capital of France?\nAnswer: Paris",
  "output": "[No Retrieval]"
}
```

--------------------------------

### Run Critic for Retrieval Tokens (Multi-Sentence)

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Executes the critic model to predict retrieval necessity considering input data and preceding sentences. This requires an initial retrieval file. Parameters include input file, critic model path, task, modes, metric, and output file path.

```bash
python run_reward_vllm.py \
    --input_file YOUR_INPUT_FILENAME \
    --model_name YOUR_CRITIC_MODEL_PATH \
    --task 'multi_retrieval' \
    --inst_mode retrieval_multi_instruction \
    --input_mode retrieval_multi_input \
    --metric match \
    --result_fp MULTI_RETRIEVAL_TOKEN_OUTPUT \
    --split test 
```

--------------------------------

### Run Critic for Retrieval Tokens (Initial)

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Executes the critic model to predict retrieval necessity based solely on the input data. Requires specifying input file, critic model path, task, instruction/input modes, metric, and output file path.

```bash
python run_reward_vllm.py \
    --input_file YOUR_INPUT_FILENAME \
    --model_name YOUR_CRITIC_MODEL_PATH \
    --task 'retrieval' \
    --inst_mode retrieval_instruction 
    --input_mode retrieval_input \
    --metric match \
    --result_fp INITIAL_RETRIEVAL_TOKEN_OUTPUT \
    --split test 
```

--------------------------------

### Train Critic Model (Llama2-7B)

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Fine-tunes a Llama2-7B model to act as a Critic, using special reflection tokens. This requires pre-created or downloaded training data. Adjust the data path and output directory accordingly. The training uses bf16 precision and a cosine learning rate scheduler.

```bash
cd data_creation
torchrun --nproc_per_node=2 \
  --master_port=2568 train_special_tokens.py \
  --model_name_or_path meta-llama/Llama-2-7b-hf \
  --data_path PATH_TO_TRAIN_DATA_FILE \
  --bf16  True \
  --output_dir PATH_TO_CRITIC_MODEL \
  --num_train_epochs 3  \
  --per_device_train_batch_size 1 --per_device_eval_batch_size 1 \
  --gradient_accumulation_steps 8 \
  --evaluation_strategy "no" \
  --save_strategy "steps" \
  --save_steps 300 \
  --save_total_limit 1 \
  --learning_rate 2e-5 \
  --weight_decay 0. \
  --warmup_ratio 0.01 \
  --lr_scheduler_type "cosine" \
  --logging_steps 10 \
  --fsdp "full_shard auto_wrap"
```

--------------------------------

### Create Prompt Data for IsRel and IsSUp

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Generates input data for 'IsRel' and 'IsSUp' tokens based on retrieval results. Supports multi-job processing for efficiency and requires specifying input file, output directory, number of jobs, top N passages, and multi-retrieval prediction file.

```bash
python create_prompt_data.py \
    --input_file MULTI_RETRIEVAL_OUTPUT \
    --output_dir YOUR_PROMPT_INPUT_DIR \
    --num_jobs NUM_JOBS \
    --top_n 10 \
    --multi_need_retrieval_pred_file MULTI_RETRIEVAL_TOKEN_OUTPUT
```

--------------------------------

### Running Passage Retrieval with Contriever

Source: https://github.com/akariasai/self-rag/blob/main/README.md

This command executes the passage retrieval script. It requires specifying the Contriever model name, the path to the passage data, passage embeddings, input data file, output directory, and the number of documents to retrieve. The input file should contain queries in JSON or JSONL format.

```shell
cd retrieval_lm
python passage_retrieval.py \
    --model_name_or_path facebook/contriever-msmarco --passages psgs_w100.tsv \
    --passages_embeddings "wikipedia_embeddings/*" \
    --data YOUR_INPUT_FILE  \
    --output_dir YOUR_OUTPUT_FILE \
    --n_docs 20
```

--------------------------------

### Run Contriever for Initial Retrieval

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Executes the Contriever model for initial passage retrieval. Requires navigating to the 'retrieval_lm' directory, specifying the Contriever model, corpus paths, preprocessed input data, and output directory.

```bash
cd retrieval_lm
python passage_retrieval.py \
    --model_name_or_path facebook/contriever-msmarco \
    --passages PATH_TO_CORPUS --passages_embeddings PATH_TO_EMBEDDINGS \
    --data INPUT_FILE \
    --output_dir INITIAL_RETRIEVAL_OUTPUT  --n_docs 10
```

--------------------------------

### ASQA Output Format (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This Python dictionary structure defines the output format for the ASQA (Answer Sentence Quality Assessment) task within the Self-RAG project. It includes fields for the original question, the generated output with citations, the supporting documents, and intermediate retrieval information. This format facilitates the evaluation of answer quality and citation accuracy.

```python
# Output format for ASQA:
# {
#   "data": [
#     {
#       "question": "What are the health benefits of coffee?",
#       "output": "Coffee contains antioxidants [0]. It may reduce risk of type 2 diabetes [1]. Studies show improved cognitive function [2].",
#       "docs": [
#         {"title": "Coffee", "text": "Coffee contains antioxidants..."},
#         {"title": "Diabetes Prevention", "text": "Coffee consumption linked..."},
#         {"title": "Cognitive Effects", "text": "Caffeine improves alertness..."}
#       ],
#       "intermediate": ["[Retrieval]<paragraph>..." ]
#     }
#   ]
# }
```

--------------------------------

### Run Generator Training Script

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Executes the main script for training the Generator model. This script utilizes DeepSpeed for efficient training. Ensure the training data path is correctly set before running. Different scripts exist for 7B and 13B model variants.

```bash
cd retrieval_lm
bash script_finetune_7b.sh
```

--------------------------------

### Run Self-RAG for FactScore Long-Form QA

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Runs Self-RAG for long-form QA on the FactScore dataset using pre-retrieved passages. Similar to ASQA, this command configures the model and retrieval parameters, specifying the FactScore task and input file.

```python
python run_long_form_static.py \
  --model_name selfrag/selfrag_llama2_7b \
  --ndocs 5 --max_new_tokens 300 --threshold 0.2 \
  --use_grounding --use_utility --use_seqscore \
  --task factscore --input_file eval_data/factscore_unlabeled_alpaca_13b_retrieval.jsonl \
  --output_file YOUR_OUTPUT_FILE_NAME --max_depth 7 \

```

--------------------------------

### Create Input for Continuous Retrieval

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Generates the input file for continuous retrieval (when t>1) using the initial retrieval token output and initial retrieval results. It requires input files, output file, and paths to the previous outputs.

```bash
python create_retrieval_data.py \
    --input_files INPUT_FILE \
    --output_file MULTI_RETRIEVAL_INPUT
    --need_retrieval_files INITIAL_RETRIEVAL_TOKEN_OUTPUT \
    --multiple_sent --initial_retrieval_file INITIAL_RETRIEVAL_OUTPUT
```

--------------------------------

### Long-Form Generation with Beam Search (Bash)

Source: https://context7.com/akariasai/self-rag/llms.txt

This bash script executes the long-form generation task using a pre-trained model. It specifies input and output files, task type, retrieval parameters, and various flags to control grounding, utility, sequence scoring, and beam search depth and width. The weights for relevance, support, and utility scores are also configurable.

```bash
python run_long_form_static.py \
  --model_name selfrag/selfrag_llama2_7b \
  --input_file eval_data/asqa_eval_gtr_top100.json \
  --output_file results/asqa_output.json \
  --task asqa \
  --ndocs 5 \
  --max_new_tokens 300 \
  --threshold 0.2 \
  --mode always_retrieve \
  --use_grounding \
  --use_utility \
  --use_seqscore \
  --max_depth 7 \
  --beam_width 2 \
  --w_rel 1.0 \
  --w_sup 1.0 \
  --w_use 0.5
```

--------------------------------

### Run Critic Evaluation with vLLM

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

This script runs critic evaluation using vLLM. It takes input files, model paths, task details, and output file specifications as arguments. It is designed for 'groundness' tasks with multi-instruction and multi-input modes.

```python
run_reward_vllm.py \
    --input_file YOUR_PROMPT_INPUT_DIR/prompt_data_batch_{BATCH_NUM}.jsonl  \
    --model_name YOUR_CRITIC_MODEL_PATH \
    --task 'groudness' \
    --inst_mode ground_multi_instruction \
     --input_mode ground_multi_input 
    --metric match \
    --result_fp SUP_OUTPUT_FILE_{BATCH_NUM} \
    --split test
```

--------------------------------

### Run Critic for 'isUse' Prediction

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Runs the critic model to predict the utility of retrieved information. Requires specifying the input file, critic model path, task, relevant instruction/input modes, and the output file path.

```bash
python run_reward_vllm.py \
    --input_file YOUR_INPUT_FILENAME \
    --model_name YOUR_CRITIC_MODEL_PATH \
    --task utility \
    --inst_mode utility_instruction \
    --input_mode utility_input \
    --result_fp UTILITY_OUTPUT \
    --split test 
```

--------------------------------

### Short-Form Generation Evaluation (Question Answering)

Source: https://github.com/akariasai/self-rag/blob/main/README.md

This Python script evaluates short-form generation tasks, specifically for Question Answering, using pre-retrieved documents. It supports different inference modes like 'adaptive_retrieval', 'no_retrieval', and 'always_retrieve'. Replace placeholders for input file, mode, and output file. 'dtype half' is used for memory efficiency.

```python
python run_short_form.py \
--model_name selfrag/selfrag_llama2_7b \
--input_file eval_data/popqa_longtail_w_gs.jsonl \
--mode MODE --max_new_tokens 100 \
--threshold 0.2 \
--output_file YOUR_OUTPUT_FILE \
--metric match --ndocs 10 --use_groundness --use_utility --use_seqscore \
--dtype half
```

--------------------------------

### Run Critic for 'isRel' Prediction

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Executes the critic model to evaluate the relevance ('isRel') of retrieved passages. Requires specifying the input prompt data file (batch-wise), critic model path, task, instruction/input modes, metric, and output file path for each batch.

```bash
python run_reward_vllm.py \
    --input_file YOUR_PROMPT_INPUT_DIR/prompt_data_batch_{BATCH_NUM}.jsonl  \
    --model_name YOUR_CRITIC_MODEL_PATH \
    --task 'relevance' --inst_mode relevance_instruction \
    --input_mode relevance_input \
    --metric match \
    --result_fp REL_OUTPUT_FILE_{BATCH_NUM} \
    --split test
```

--------------------------------

### Run Self-RAG for ARC Challenge Evaluation

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Executes the Self-RAG model for the ARC Challenge dataset. It specifies the model name, input file, output file, and various evaluation metrics and flags. Key parameters include max new tokens, retrieval threshold, and task type.

```python
python run_short_form.py \
  --model_name selfrag/selfrag_llama2_7b \
  --input_file eval_data/arc_challenge_processed.jsonl \
  --max_new_tokens 50 --threshold 0.2 \
  --output_file OUTPUT_FILE_NAME \
  --metric match --ndocs 5 --use_groundness --use_utility --use_seqscore \
  --task arc_c
```

--------------------------------

### Run Self-RAG for ASQA Long-Form QA

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Executes Self-RAG for long-form question answering on the ASQA dataset using pre-retrieved passages. It sets the model, retrieval parameters, and task-specific configurations for ASQA. Note the `mode always_retrieve` parameter.

```python
python run_long_form_static.py \
  --model_name selfrag/selfrag_llama2_7b \
  --ndocs 5 --max_new_tokens 300 --threshold 0.2 \
  --use_grounding --use_utility --use_seqscore \
  --task asqa --input_file eval_data/asqa_eval_gtr_top100.json \
  --output_file YOUR_OUTPUT_FILE_NAME --max_depth 7 --mode always_retrieve \

```

--------------------------------

### Factual Grounding with Self-RAG

Source: https://github.com/akariasai/self-rag/blob/main/README.md

This snippet demonstrates how to use Self-RAG to generate an answer that is factually grounded in provided evidence. It formats a prompt with a question and supporting text, then uses a model to generate a prediction. The output shows the model identifying relevant and supported information.

```python
prompt = format_prompt("Can you tell me the difference between llamas and alpacas?", "The alpaca (Lama pacos) is a species of South American camelid mammal. It is similar to, and often confused with, the llama. Alpacas are considerably smaller than llamas, and unlike llamas, they were not bred to be working animals, but were bred specifically for their fiber.")
preds = model.generate([prompt], sampling_params)
print([pred.outputs[0].text for pred in preds])
```

--------------------------------

### Downloading Contriever Embeddings

Source: https://github.com/akariasai/self-rag/blob/main/README.md

This command downloads the pre-computed embeddings for Wikipedia passages using the Contriever-MSMARCO model. These embeddings are essential for efficient passage retrieval.

```shell
wget https://dl.fbaipublicfiles.com/contriever/embeddings/contriever-msmarco/wikipedia_embeddings.tar
```

--------------------------------

### Run Contriever for Continuous Retrieval

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Executes the Contriever model for continuous passage retrieval, focusing on queries where initial retrieval necessity was true. Requires specifying the Contriever model, corpus paths, preprocessed multi-retrieval input, and output directory.

```bash
cd retrieval_lm
python passage_retrieval.py \
    --model_name_or_path facebook/contriever-msmarco \
    --passages PATH_TO_CORPUS --passages_embeddings PATH_TO_EMBEDDINGS \
    --data MULTI_RETRIEVAL_INPUT \
    --output_dir MULTI_RETRIEVAL_OUTPUT  --n_docs 10
```

--------------------------------

### Generate Embeddings for Own Data

Source: https://github.com/akariasai/self-rag/blob/main/README.md

This command generates embeddings for custom data using a pre-trained Contriever model. It's designed to run in parallel across multiple GPUs for large datasets. Ensure you are in the 'retrieval_lm' directory and replace placeholders for output directory and passage data.

```bash
cd retrieval_lm
for i in {0..3}; do
  export CUDA_VISIBLE_DEVICES=${i}
  python generate_passage_embeddings.py  --model_name_or_path facebook/contriever-msmarco \
  --output_dir YOUR_OUTPUT_DIR \
  --passages YOUR_PASSAGE_DATA --shard_id ${i}  --num_shards 4 > ./log/nohup.my_embeddings.${i} 2>&1 &
done
```

--------------------------------

### Self-RAG Beam Search Parameters (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This Python comment lists and explains the beam search parameters used in the Self-RAG generation process. It clarifies the meaning of 'max_depth' (maximum retrieval iterations), 'beam_width' (number of candidates kept), and the weights 'w_rel', 'w_sup', and 'w_use' for relevance, support, and utility scores, respectively. These parameters control the trade-offs between exploration and exploitation during generation.

```python
# Beam search parameters:
# - max_depth: Maximum retrieval iterations (corresponds to T in paper)
# - beam_width: Number of top candidates kept at each depth
# - w_rel: Weight for relevance score ([Relevant] token probability)
# - w_sup: Weight for support score ([Fully supported] token probability)
# - w_use: Weight for utility score ([Utility:1-5] token probability)
```

--------------------------------

### Combine Processed Data

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

This script combines various processed data files into a final output. It takes arguments for utility predictions, retrieval tokens (initial and multi), groundness predictions, relevance predictions, original input data, and formatted retrieval data.

```python
postprocess_data.py \
    --utility_pred YOUR_INPUT_FILENAME \
    --retrieval_i_only INITIAL_RETRIEVAL_TOKEN_OUTPUT \
    --retrieval_multi  MULTI_RETRIEVAL_TOKEN_OUTPUT \
    --groudness_pred ALL_SUP_OUTPUT_FILE \
    --relevance_pred ALL_REL_OUTPUT_FILE\
    --orig_input_data YOUR_INPUT_FILENAME \
    --retrieval_data MULTI_RETRIEVAL_OUTPUT \
    --splitted_input_data MULTI_RETRIEVAL_INPUT_splitted \
    --output_fn FINAL_OUTPUT_PATH
```

--------------------------------

### Post-process Model Output to Remove Special Tokens (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This snippet demonstrates how to clean raw model output by removing special tokens like '[Relevant]', '[Fully supported]', '[Utility:5]', and '</s>'. It takes a raw string as input and returns a cleaned string, suitable for presenting model responses to users.

```python
raw_output = "[Relevant]Paris is the capital of France.[Fully supported][Utility:5]</s>"
clean_output = postprocess(raw_output)
print(clean_output)
```

--------------------------------

### Generate Passage Embeddings for Custom Corpus (Bash)

Source: https://context7.com/akariasai/self-rag/llms.txt

This script generates embeddings for a custom document corpus using multiple GPUs. It requires the 'generate_passage_embeddings.py' script and specifies the model, output directory, input passages, and shard configuration. The output logs capture the embedding generation process.

```bash
cd retrieval_lm

# Multi-GPU embedding generation (4 GPUs)
for i in {0..3}; do
  export CUDA_VISIBLE_DEVICES=${i}
  python generate_passage_embeddings.py \
    --model_name_or_path facebook/contriever-msmarco \
    --output_dir embeddings/my_corpus \
    --passages my_corpus.tsv \
    --shard_id ${i} \
    --num_shards 4 \
    --per_gpu_batch_size 512 > logs/embed_${i}.log 2>&1 &
done

# Wait for completion, then run retrieval
python passage_retrieval.py \
  --model_name_or_path facebook/contriever-msmarco \
  --passages my_corpus.tsv \
  --passages_embeddings "embeddings/my_corpus/*" \
  --data input_queries.jsonl \
  --output_dir retrieved_results.jsonl \
  --n_docs 20
```

--------------------------------

### Run Self-RAG for PubHealth Evaluation

Source: https://github.com/akariasai/self-rag/blob/main/README.md

Runs the Self-RAG model for the PubHealth (Health Claims) dataset. This command configures the model, input/output files, and evaluation settings similar to the ARC Challenge, with a specific task type for health-related data.

```python
python run_short_form.py \
  --model_name selfrag/selfrag_llama2_7b \
  --input_file eval_data/health_claims_processed.jsonl \
  --max_new_tokens 50 \
  --threshold 0.2 --output_file OUTPUT_FILE_NAME \
  --metric match --ndocs 5 \
  --use_groundness --use_utility --use_seqscore \
  --task fever
```

--------------------------------

### Process Input Data Schema

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/generator/README.md

Defines the expected JSON schema for input data used in the training data creation pipeline. This schema includes fields for instruction, output, input, topic, id, and dataset_name.

```json
{
"instruction": instruction, 
"output": output, 
"input": "",
"topic": "", 
"id": q_id, 
"dataset_name": dataset_name
}
```

--------------------------------

### Special Tokens for Self-RAG (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This Python list defines special tokens used by the Self-RAG model for various purposes, including indicating retrieval status, relevance, factual support, and utility. These tokens are added to the model's vocabulary and are crucial for training and inference to control the generation process and evaluate output quality.

```python
SPECIAL_TOKENS = [
    "[No Retrieval]", "[Retrieval]", "[Continue to Use Evidence]",
    "[Irrelevant]", "[Relevant]",
    "<paragraph>", "</paragraph>",
    "[Utility:1]", "[Utility:2]", "[Utility:3]", "[Utility:4]", "[Utility:5]",
    "[Fully supported]", "[Partially supported]", "[No support / Contradictory]"
]
```

--------------------------------

### Shell Script for Collecting Reward Data using OpenAI API

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/critic/gpt4_reward/README.md

A shell script to execute Python utilities for collecting reward data. It requires specifying input and output file paths, OpenAI API key, organization name, and the model name to be used for generation.

```sh
python chatgpt_utility.py \
    --input_file path_to_input_file \
    --output_file_name path_to_output_file \
    --api_key path_to_open_ai_api_txt_file \
    -org_name your_organization_name \
    --model_name open_ai_model_name \

```

--------------------------------

### Self-RAG Scoring Formula (Python)

Source: https://context7.com/akariasai/self-rag/llms.txt

This Python comment outlines the scoring formula used when 'use_seqscore' is enabled during Self-RAG generation. It combines the exponential of the sequence score with weighted relevance, ground truth (support), and utility scores. This formula is used to evaluate the quality and factual accuracy of the generated text.

```python
# Scoring formula (when use_seqscore=True):
# final_score = exp(seq_score) + w_rel * relevance_score + w_sup * ground_score + w_use * utility_score
```

--------------------------------

### Python Data Structure for Reward Collection

Source: https://github.com/akariasai/self-rag/blob/main/data_creation/critic/gpt4_reward/README.md

Defines the expected JSON structure for input files used in collecting machine-generated rewards. Each entry includes fields for instruction, target output, evidence, preceding sentences, full output, and unique identifiers.

```python
{
    "instruction": str, # input instruction 
    "target_output": str, # segment-level output
    "evidence": str, # retrieved Wikipedia paragraph
    "preceding_sentences": str, # previously generated sentences
    "output": str # full output (only used for utility),
    "q_id": str # unique instance id,
    "sent_id": int # sentence index
    "p_id": int # paragraph index
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.