### Training Pipeline Setup (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Sets up the training pipeline using argparse for command-line arguments and imports the main train function. It defines various hyperparameters for training such as learning rate, batch size, and model configuration. Dependencies include argparse and the main module's train function.

```python
import argparse
from main import train

parser = argparse.ArgumentParser()
parser.add_argument('--mode', type=str, default='train')
parser.add_argument('--para_limit', type=int, default=2250)
parser.add_argument('--batch_size', type=int, default=24)
parser.add_argument('--init_lr', type=float, default=0.1)
parser.add_argument('--keep_prob', type=float, default=1.0)
parser.add_argument('--sp_lambda', type=float, default=1.0)
parser.add_argument('--hidden', type=int, default=80)
parser.add_argument('--char_hidden', type=int, default=100)
parser.add_argument('--patience', type=int, default=1)
parser.add_argument('--checkpoint', type=int, default=1000)
parser.add_argument('--period', type=int, default=100)
parser.add_argument('--seed', type=int, default=13)
parser.add_argument('--save', type=str, default='HOTPOT')
config = parser.parse_args()

# Train model on single GPU
# CUDA_VISIBLE_DEVICES=0 python main.py --mode train --para_limit 2250 --batch_size 24 --init_lr 0.1 --keep_prob 1.0 --sp_lambda 1.0

# Expected console output:

```

--------------------------------

### Install PyTorch 0.3.0 with CUDA 8

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Installs PyTorch version 0.3.0 using conda, specifically for CUDA 8. This is a prerequisite for running the HotpotQA experiments.

```bash
conda install pytorch=0.3.0 cuda80 -c pytorch
```

--------------------------------

### Install spaCy

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Installs the spaCy library using conda. spaCy is a dependency for natural language processing tasks within the HotpotQA project.

```bash
conda install spacy
```

--------------------------------

### Process HotpotQA JSON Data with `process_file`

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Processes a single HotpotQA JSON file to extract examples, tokenize contexts and questions, and determine answer spans. It also builds vocabulary counters for words and characters. This function relies on the `prepro` module and `ujson` for efficient JSON parsing.

```python
import ujson as json
from prepro import process_file
from collections import Counter

config = type('Config', (), {
    'para_limit': 2250,
    'ques_limit': 80,
    'char_limit': 16
})()

word_counter = Counter()
char_counter = Counter()

# Process training file
examples, eval_examples = process_file(
    'hotpot_train_v1.1.json',
    config,
    word_counter,
    char_counter
)

print(f"Processed {len(examples)} training examples")
print(f"Vocabulary size: {len(word_counter)}")
print(f"Character set size: {len(char_counter)}")

# Example structure:
# examples[0] = {
#     'context_tokens': ['<t>', 'Title', '</t>', 'The', 'text', ...],
#     'context_chars': [['<', 't', '>'], ['T', 'i', 't', 'l', 'e'], ...],
#     'ques_tokens': ['What', 'is', 'the', 'answer', '?'],
#     'ques_chars': [['W', 'h', 'a', 't'], ['i', 's'], ...],
#     'y1s': [15],  # Answer start index
#     'y2s': [17],  # Answer end index
#     'id': '5a8b57f25542995d1e6f1371',
#     'start_end_facts': [(0, 5, False), (5, 20, True), ...]
# }
```

--------------------------------

### Configure Data Preprocessing Arguments

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Sets up command-line arguments for the data preprocessing script, defining parameters for data files, limits, embeddings, and splitting. This script uses `argparse` to handle configuration, allowing flexible control over the preprocessing pipeline.

```python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--mode', type=str, default='prepro')
parser.add_argument('--data_file', type=str, default='hotpot_train_v1.1.json')
parser.add_argument('--para_limit', type=int, default=2250)
parser.add_argument('--ques_limit', type=int, default=80)
parser.add_argument('--char_limit', type=int, default=16)
parser.add_argument('--data_split', type=str, default='train')
parser.add_argument('--glove_word_file', type=str, default='glove.840B.300d.txt')
parser.add_argument('--glove_word_size', type=int, default=int(2.2e6))
parser.add_argument('--glove_dim', type=int, default=300)
parser.add_argument('--char_dim', type=int, default=8)
config = parser.parse_args()

# Preprocess training data
# python main.py --mode prepro --data_file hotpot_train_v1.1.json --para_limit 2250 --data_split train

# Preprocess dev data (run after training preprocessing)
# python main.py --mode prepro --data_file hotpot_dev_distractor_v1.json --para_limit 2250 --data_split dev

# Expected output:
# - word_emb.json: Word embedding matrix
# - char_emb.json: Character embedding matrix
# - word2idx.json: Word to index mapping
# - char2idx.json: Character to index mapping
# - train_record.pkl: Preprocessed training features
# - dev_record.pkl: Preprocessed dev features
# - train_eval.json: Training evaluation metadata
# - dev_eval.json: Dev evaluation metadata
```

--------------------------------

### Run Local Evaluation for FullWiki Setting

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Generates predictions for the fullwiki setting by running the main script with the `--fullwiki` flag. Similar to the distractor setting, this step is followed by evaluation using the corresponding ground truth file.

```shell
CUDA_VISIBLE_DEVICES=0 python main.py --mode test --data_split dev --para_limit 2250 --batch_size 24 --init_lr 0.1 \
--keep_prob 1.0 --sp_lambda 1.0 --save HOTPOT-20180924-160521 --prediction_file dev_fullwiki_pred.json --fullwiki
```

--------------------------------

### Evaluate Predictions for Distractor Setting

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Runs the evaluation script using the generated prediction file and a ground truth file for the distractor setting. This script compares the model's predictions against the expected answers and supporting facts.

```shell
python hotpot_evaluate_v1.py dev_distractor_pred.json hotpot_dev_distractor_v1.json
```

--------------------------------

### Run Local Evaluation for Distractor Setting

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Executes the main script to generate predictions in the distractor setting and saves them to a specified file. This is the first step in local evaluation, followed by running the evaluation script.

```shell
CUDA_VISIBLE_DEVICES=0 python main.py --mode test --data_split dev --para_limit 2250 --batch_size 24 --init_lr 0.1 \
--keep_prob 1.0 --sp_lambda 1.0 --save HOTPOT-20180924-160521 --prediction_file dev_distractor_pred.json
```

--------------------------------

### Evaluate Predictions for FullWiki Setting

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Evaluates the model's predictions in the fullwiki setting using the generated prediction file and the corresponding ground truth file. This script assesses the performance of the model in a more comprehensive context.

```shell
python hotpot_evaluate_v1.py dev_fullwiki_pred.json hotpot_dev_fullwiki_v1.json
```

--------------------------------

### Preprocess Development Data (FullWiki Setting)

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Preprocesses the HotpotQA development dataset in the fullwiki setting. This command includes the `--fullwiki` flag and specifies the data split as 'dev'.

```python
python main.py --mode prepro --data_file hotpot_dev_fullwiki_v1.json --data_split dev --fullwiki --para_limit 2250
```

--------------------------------

### Train Model on Single GPU

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Trains the HotpotQA model on a specified GPU (CUDA_VISIBLE_DEVICES=0). It includes parameters for paragraph limit, batch size, initial learning rate, keep probability, and span lambda.

```bash
CUDA_VISIBLE_DEVICES=0 python main.py --mode train --para_limit 2250 --batch_size 24 --init_lr 0.1 --keep_prob 1.0 \ 
--sp_lambda 1.0
```

--------------------------------

### Download HotpotQA Data and GloVe Embeddings

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Executes a shell script to download the HotpotQA dataset files (training, development, and test sets) and GloVe word embeddings. It also downloads necessary spaCy packages.

```bash
./download.sh
```

--------------------------------

### Initialize Baseline QA Model with PyTorch

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Initializes the baseline neural question-answering model using PyTorch. It loads pre-trained word and character embeddings and configures the model's architecture, including GRU encoders and attention mechanisms. The model is then moved to the CUDA-enabled GPU for accelerated computation.

```python
import torch
from model import Model
import numpy as np

# Load embeddings
word_mat = np.random.randn(10000, 300).astype(np.float32)
char_mat = np.random.randn(100, 8).astype(np.float32)

config = type('Config', (), {
    'glove_dim': 300,
    'char_dim': 8,
    'char_hidden': 100,
    'hidden': 80,
    'keep_prob': 0.8
})()

model = Model(config, word_mat, char_mat)
model = model.cuda()

# Prepare input tensors
batch_size = 4
context_idxs = torch.randint(0, 10000, (batch_size, 500)).cuda()
ques_idxs = torch.randint(0, 10000, (batch_size, 25)).cuda()
context_char_idxs = torch.randint(0, 100, (batch_size, 500, 16)).cuda()
ques_char_idxs = torch.randint(0, 100, (batch_size, 25, 16)).cuda()
context_lens = torch.tensor([500, 450, 480, 500]).cuda()
start_mapping = torch.zeros(batch_size, 500, 50).cuda()
end_mapping = torch.zeros(batch_size, 500, 50).cuda()
all_mapping = torch.zeros(batch_size, 500, 50).cuda()
```

--------------------------------

### SPModel Forward Pass and Initialization (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Demonstrates the forward pass of the SPModel for answer and supporting fact prediction, along with its initialization. It takes context and question embeddings as input and outputs various logits and predictions. Dependencies include torch and the SPModel class.

```python
import torch
from sp_model import SPModel
import numpy as np

word_mat = np.random.randn(10000, 300).astype(np.float32)
char_mat = np.random.randn(100, 8).astype(np.float32)

config = type('Config', (), {
    'glove_dim': 300,
    'char_dim': 8,
    'char_hidden': 100,
    'hidden': 80,
    'keep_prob': 1.0,
    'sp_lambda': 1.0  # Supporting fact loss weight
})()

sp_model = SPModel(config, word_mat, char_mat)
sp_model = sp_model.cuda()

batch_size = 4
context_idxs = torch.randint(0, 10000, (batch_size, 500)).cuda()
ques_idxs = torch.randint(0, 10000, (batch_size, 25)).cuda()
context_char_idxs = torch.randint(0, 100, (batch_size, 500, 16)).cuda()
ques_char_idxs = torch.randint(0, 100, (batch_size, 25, 16)).cuda()
context_lens = torch.tensor([500, 450, 480, 500]).cuda()
start_mapping = torch.zeros(batch_size, 500, 50).cuda()
end_mapping = torch.zeros(batch_size, 500, 50).cuda()
all_mapping = torch.zeros(batch_size, 500, 50).cuda()

# Get predictions with supporting fact integration
logit1, logit2, predict_type, predict_support, yp1, yp2 = sp_model(
    context_idxs, ques_idxs, context_char_idxs, ques_char_idxs,
    context_lens, start_mapping, end_mapping, all_mapping,
    return_yp=True
)

# yp1: (batch_size,) - Predicted answer start positions
# yp2: (batch_size,) - Predicted answer end positions

```

--------------------------------

### Train Model on Multiple GPUs

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Trains the HotpotQA model using all available GPUs. This is achieved by omitting the CUDA_VISIBLE_DEVICES environment variable. It uses the same training parameters as the single-GPU command.

```bash
python main.py --mode train --para_limit 2250 --batch_size 24 --init_lr 0.1 --keep_prob 1.0 --sp_lambda 1.0
```

--------------------------------

### Preprocess Development Data (Distractor Setting)

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Preprocesses the HotpotQA development dataset in the distractor setting. It uses the specified data file and paragraph limit, marking the split as 'dev'.

```python
python main.py --mode prepro --data_file hotpot_dev_distractor_v1.json --para_limit 2250 --data_split dev
```

--------------------------------

### DataIterator Class for Batch Processing (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

The DataIterator class efficiently handles batch creation for training and inference. It supports features like bucketing, padding, dynamic batching, and sentence-level mapping. It takes preprocessed data and configuration parameters to yield batches containing context, question, character indices, and answer labels.

```python
from util import DataIterator, get_buckets

# Load preprocessed data
train_buckets = get_buckets('train_record.pkl')

# Create iterator
batch_size = 24
para_limit = 2250
ques_limit = 80
char_limit = 16
sent_limit = 100
shuffle = True

iterator = DataIterator(
    train_buckets, batch_size, para_limit, ques_limit,
    char_limit, shuffle, sent_limit
)

# Iterate through batches
for batch in iterator:
    context_idxs = batch['context_idxs']  # (cur_bsz, max_c_len)
    ques_idxs = batch['ques_idxs']        # (cur_bsz, max_q_len)
    context_char_idxs = batch['context_char_idxs']  # (cur_bsz, max_c_len, char_limit)
    ques_char_idxs = batch['ques_char_idxs']        # (cur_bsz, max_q_len, char_limit)
    y1 = batch['y1']  # Answer start indices
    y2 = batch['y2']  # Answer end indices
    q_type = batch['q_type']  # 0=span, 1=yes, 2=no
    is_support = batch['is_support']  # Supporting fact labels

    print(f"Batch size: {context_idxs.size(0)}")
    print(f"Max context length: {context_idxs.size(1)}")
    print(f"Max question length: {ques_idxs.size(1)}")
    break
```

--------------------------------

### Preprocess Training Data

Source: https://github.com/hotpotqa/hotpot/blob/master/README.md

Preprocesses the HotpotQA training dataset. This involves setting parameters like paragraph limit and specifying the data split as 'train'. This step must be completed before preprocessing development sets.

```python
python main.py --mode prepro --data_file hotpot_train_v1.1.json --para_limit 2250 --data_split train
```

--------------------------------

### BiAttention Mechanism Implementation (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Implements a bi-directional attention mechanism using scaled dot-product attention. It takes context and question representations as input and outputs an attention-weighted representation. Dependencies include torch and the BiAttention class.

```python
from model import BiAttention
import torch

input_size = 160  # hidden * 2 for bidirectional
dropout = 0.2
bi_att = BiAttention(input_size, dropout).cuda()

batch_size = 4
context_len = 500
question_len = 25

# Context and question representations from encoder
context_repr = torch.randn(batch_size, context_len, input_size).cuda()
question_repr = torch.randn(batch_size, question_len, input_size).cuda()
question_mask = torch.ones(batch_size, question_len).cuda()

# Apply bi-directional attention
attention_output = bi_att(context_repr, question_repr, question_mask)

# Output shape: (batch_size, context_len, input_size * 4)
# Contains: [context, context2question, element_wise_mul, question2context]
print(f"Attention output shape: {attention_output.shape}")

```

--------------------------------

### Official HotpotQA Evaluation Script (Bash)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

This bash command executes the official HotpotQA evaluation script. It takes the prediction file and a ground truth file as input and outputs comprehensive evaluation metrics, including answer accuracy, supporting fact accuracy, and joint metrics.

```bash
# Run official evaluation
python hotpot_evaluate_v1.py dev_distractor_pred.json hotpot_dev_distractor_v1.json

# Output:
# {
#   'em': 0.4523,           # Answer exact match
#   'f1': 0.5876,           # Answer F1
#   'prec': 0.6123,         # Answer precision
#   'recall': 0.5654,       # Answer recall
#   'sp_em': 0.3821,        # Supporting fact exact match
#   'sp_f1': 0.6234,        # Supporting fact F1
#   'sp_prec': 0.6543,      # Supporting fact precision
#   'sp_recall': 0.5956,    # Supporting fact recall
#   'joint_em': 0.2145,     # Joint answer + SP exact match
#   'joint_f1': 0.4234,     # Joint answer + SP F1
#   'joint_prec': 0.4567,   # Joint precision
#   'joint_recall': 0.3956  # Joint recall
# }
```

--------------------------------

### Tokenize Text with spaCy (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Tokenizes input text into a list of words and punctuation using spaCy's blank English tokenizer. Ensures consistent preprocessing for both questions and contexts. Handles contractions and punctuation effectively.

```python
from prepro import word_tokenize

text = "What is the capital of France?"
tokens = word_tokenize(text)

print(tokens)
# Output: ['What', 'is', 'the', 'capital', 'of', 'France', '?']

# Handles punctuation and special characters
text2 = "It's a beautiful day, isn't it?"
tokens2 = word_tokenize(text2)
print(tokens2)
# Output: ['It', "'s", 'a', 'beautiful', 'day', ',', 'is', "n't", 'it', '?']
```

--------------------------------

### Generate Word Embeddings with GloVe (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Generates an embedding matrix from a GloVe file, creating mappings between tokens and their indices. Filters vocabulary based on a counter and a specified limit, handling out-of-vocabulary (OOV) tokens. Requires a vocabulary counter and the path to the GloVe file.

```python
from prepro import get_embedding
from collections import Counter

# Build vocabulary counter
word_counter = Counter({
    'the': 15234,
    'of': 8923,
    'and': 7654,
    'Barack': 123,
    'Obama': 98,
    'random_word': 2
})

# Generate embeddings from GloVe
emb_mat, token2idx, idx2token = get_embedding(
    counter=word_counter,
    data_type='word',
    limit=5,  # Filter words with count <= 5
    emb_file='glove.840B.300d.txt',
    size=2200000,
    vec_size=300
)

print(f"Embedding matrix shape: {len(emb_mat)} x {len(emb_mat[0])}")
print(f"Token 'the' index: {token2idx['the']}")
print(f"Token at index 2: {idx2token[2]}")
print(f"NULL token index: {token2idx['--NULL--']}")
print(f"OOV token index: {token2idx['--OOV--']}")

# Output:
# Embedding matrix shape: 5432 x 300
# Token 'the' index: 2
# Token at index 2: the
# NULL token index: 0
# OOV token index: 1
```

--------------------------------

### Model Testing and Prediction Generation (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

The test function loads a trained model checkpoint and generates predictions for a specified dataset split. It utilizes argparse for configuration and outputs predictions in a JSON format suitable for evaluation or submission, including answer spans and supporting facts.

```python
import argparse
from main import test

parser = argparse.ArgumentParser()
parser.add_argument('--mode', type=str, default='test')
parser.add_argument('--data_split', type=str, default='dev')
parser.add_argument('--para_limit', type=int, default=2250)
parser.add_argument('--batch_size', type=int, default=24)
parser.add_argument('--init_lr', type=float, default=0.1)
parser.add_argument('--keep_prob', type=float, default=1.0)
parser.add_argument('--sp_lambda', type=float, default=1.0)
parser.add_argument('--save', type=str, default='HOTPOT-20180924-160521')
parser.add_argument('--prediction_file', type=str, default='dev_distractor_pred.json')
parser.add_argument('--sp_threshold', type=float, default=0.3)
config = parser.parse_args()

# Generate predictions
# CUDA_VISIBLE_DEVICES=0 python main.py --mode test --data_split dev --para_limit 2250 --batch_size 24 --init_lr 0.1 --keep_prob 1.0 --sp_lambda 1.0 --save HOTPOT-20180924-160521 --prediction_file dev_distractor_pred.json

# Output file format (dev_distractor_pred.json):
# {
#   "answer": {
#     "5a8b57f25542995d1e6f1371": "Barack Obama",
#     "5a8c1f0b5542995153361231": "yes",
#     ...
#   },
#   "sp": {
#     "5a8b57f25542995d1e6f1371": [
#       ["Barack Obama", 0],
#       ["Barack Obama", 3],
#       ["United States", 1]
#     ],
#     ...
#   }
# }
```

--------------------------------

### Normalize Answer Strings (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Normalizes answer strings by converting them to lowercase, removing leading/trailing whitespace, articles ('a', 'an', 'the'), and punctuation. This ensures consistent comparison for evaluation. It takes a single string as input and returns the normalized string.

```python
from util import normalize_answer

answers = [
    "The Great Wall of China",
    "  Barack Obama  ",
    "It's a beautiful day!",
    "A cat and a dog",
    "yes"
]

for ans in answers:
    normalized = normalize_answer(ans)
    print(f"{ans:30s} -> {normalized}")

# Output:
# The Great Wall of China       -> great wall china
#   Barack Obama                 -> barack obama
# It's a beautiful day!          -> it s beautiful day
# A cat and a dog                -> cat dog
# yes                            -> yes
```

--------------------------------

### Convert Predicted Tokens to Answer Strings (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Converts predicted token indices back into human-readable answer strings using span offsets from the original context. Handles both span extraction and yes/no answer types. Requires a dictionary containing context and span information, prediction indices, and prediction types.

```python
from util import convert_tokens

# Load eval file with context spans
eval_file = {
    'abc123': {
        'context': 'Barack Obama was the 44th president of the United States.',
        'spans': [[0, 6], [7, 12], [13, 16], [17, 20], [21, 25], [26, 35], ...],
        'answer': ['Barack Obama']
    }
}

qa_ids = ['abc123']
pp1 = [0]  # Predicted start position
pp2 = [1]  # Predicted end position
p_type = [0]  # Type 0 = span extraction

# Convert predictions to text
answer_dict = convert_tokens(eval_file, qa_ids, pp1, pp2, p_type)

print(answer_dict)
# Output: {'abc123': 'Barack Obama'}

# Handle yes/no answers
pp1_yn = [0]
pp2_yn = [0]
p_type_yn = [1]  # Type 1 = yes
answer_dict_yn = convert_tokens(eval_file, qa_ids, pp1_yn, pp2_yn, p_type_yn)
# Output: {'abc123': 'yes'}
```

--------------------------------

### Evaluation Function for Metrics Calculation (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

The evaluate function computes exact match and F1 scores by comparing predicted answers against ground truth. It requires the ground truth evaluation file and a dictionary of model predictions. The function performs necessary normalization before calculating the metrics.

```python
import ujson as json
from util import evaluate

# Load evaluation data
with open('dev_eval.json', 'r') as f:
    eval_file = json.load(f)

# Mock answer dictionary (from model predictions)
answer_dict = {
    '5a8b57f25542995d1e6f1371': 'Barack Obama',
    '5a8c1f0b5542995153361231': 'yes',
    '5ab2f4c955429933744ab2b6': 'The Great Wall of China'
}

# Compute metrics
metrics = evaluate(eval_file, answer_dict)

print(f"Exact Match: {metrics['exact_match']:.2f}%")
print(f"F1 Score: {metrics['f1']:.2f}%")

# Example output:
# Exact Match: 45.23%
# F1 Score: 58.76%
```

--------------------------------

### EncoderRNN for Sequence Encoding (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

A multi-layer bidirectional GRU encoder designed for variable-length sequences using packed sequences. It accepts input tensors and lengths, outputting encoded representations. Dependencies include torch and the EncoderRNN class.

```python
from model import EncoderRNN
import torch

input_size = 308  # word_dim (300) + char_hidden (100)
num_units = 80
nlayers = 1
concat = True
bidir = True
dropout = 0.2
return_last = False

encoder = EncoderRNN(
    input_size, num_units, nlayers, concat, bidir, dropout, return_last
).cuda()

batch_size = 4
seq_len = 500
input_tensor = torch.randn(batch_size, seq_len, input_size).cuda()
input_lengths = torch.tensor([500, 450, 480, 500]).cuda()

# Encode sequences
encoded = encoder(input_tensor, input_lengths)

# Output shape: (batch_size, seq_len, num_units * 2) for bidirectional
# The encoder handles variable lengths efficiently with packed sequences
print(f"Encoded shape: {encoded.shape}")

```

--------------------------------

### Compute Token-Level F1 Score (Python)

Source: https://context7.com/hotpotqa/hotpot/llms.txt

Calculates the token-level F1 score, precision, and recall between a predicted answer string and a ground truth string. It includes special handling for 'yes', 'no', and 'noanswer' types. This function is crucial for evaluating the accuracy of predictions.

```python
from util import f1_score

# Span extraction example
prediction = "Barack Hussein Obama"
ground_truth = "Barack Obama"
f1, precision, recall = f1_score(prediction, ground_truth)
print(f"F1: {f1:.4f}, Precision: {precision:.4f}, Recall: {recall:.4f}")
# Output: F1: 0.8000, Precision: 0.6667, Recall: 1.0000

# Yes/No example
pred_yes = "yes"
gt_no = "no"
f1_yn, prec_yn, rec_yn = f1_score(pred_yes, gt_no)
print(f"F1: {f1_yn:.4f}, Precision: {prec_yn:.4f}, Recall: {rec_yn:.4f}")
# Output: F1: 0.0000, Precision: 0.0000, Recall: 0.0000

# Matching yes/yes
f1_match, prec_match, rec_match = f1_score("yes", "yes")
print(f"F1: {f1_match:.4f}")
# Output: F1: 1.0000
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.