### Download Mr. TyDi v1.1 Swahili Data

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Use this command to download the compressed Swahili dataset for Mr. TyDi v1.1. Ensure you have wget installed. The data will be saved in the 'mrtydi' directory.

```bash
wget https://git.uwaterloo.ca/jimmylin/mr.tydi/-/raw/master/data/mrtydi-v1.1-swahili.tar.gz -P mrtydi
tar -xf mrtydi/mrtydi-v1.1-swahili.tar.gz -C mrtydi
gzip -d mrtydi/mrtydi-v1.1-swahili/collection/docs.jsonl.gz
```

--------------------------------

### Preprocess MKQA Data

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Run the data_scripts/preprocess_xmkqa.py script to preprocess the downloaded MKQA dataset, saving it as xmkqa.

```bash
python data_scripts/preprocess_xmkqa.py mkqa.jsonl xmkqa
```

--------------------------------

### mContriever Training Configuration

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Use this configuration for training the multilingual mContriever model on 32 GPUs. It includes a comprehensive list of language datasets and specific training hyperparameters for multilingual support.

```bash
TDIR=encoded-data/bert-base-multilingual-cased/
TRAINDATASETS="${TDIR}fr_XX ${TDIR}en_XX ${TDIR}ar_AR ${TDIR}bn_IN ${TDIR}fi_FI ${TDIR}id_ID ${TDIR}ja_XX ${TDIR}ko_KR ${TDIR}ru_RU ${TDIR}sw_KE ${TDIR}hu_HU ${TDIR}he_IL ${TDIR}it_IT ${TDIR}km_KM ${TDIR}ms_MY ${TDIR}nl_XX ${TDIR}no_XX ${TDIR}pl_PL ${TDIR}pt_XX ${TDIR}sv_SE ${TDIR}te_IN ${TDIR}th_TH ${TDIR}tr_TR ${TDIR}vi_VN ${TDIR}zh_CN ${TDIR}zh_TW ${TDIR}es_XX ${TDIR}de_DE ${TDIR}da_DK"

python train.py \
        --retriever_model_id bert-base-multilingual-cased --pooling average \
        --train_data ${TRAINDATASETS} --loading_mode split \
        --ratio_min 0.1 --ratio_max 0.5 --chunk_length 256 \
        --momentum 0.999 --moco_queue 32768 --temperature 0.05 \
        --warmup_steps 20000 --total_steps 500000 --lr 0.00005 \
        --scheduler linear --optim adamw --per_gpu_batch_size 64 \
        --output_dir /checkpoint/gizacard/contriever/xling/mcontriever \

```

--------------------------------

### Download MKQA Dataset

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Use wget to download the MKQA dataset in JSONL.GZ format.

```bash
wget https://raw.githubusercontent.com/apple/ml-mkqa/master/dataset/mkqa.jsonl.gz
```

--------------------------------

### Download Pre-computed Passage Embeddings

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Alternatively, download pre-computed passage embeddings for mContriever or mContriever-msmarco models using wget.

```bash
wget https://dl.fbaipublicfiles.com/contriever/embeddings/mcontriever/wikipedia_embeddings.tar
```

```bash
wget https://dl.fbaipublicfiles.com/contriever/embeddings/mcontriever-msmarco/wikipedia_embeddings.tar
```

--------------------------------

### Contriever Training Configuration

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Use this configuration for training the English monolingual Contriever model on 32 GPUs. It specifies data paths, augmentation strategies, and training hyperparameters.

```bash
python train.py \
        --retriever_model_id bert-base-uncased --pooling average \
        --augmentation delete --prob_augmentation 0.1 \
        --train_data "data/wiki/ data/cc-net/" --loading_mode split \
        --ratio_min 0.1 --ratio_max 0.5 --chunk_length 256 \
        --momentum 0.9995 --moco_queue 131072 --temperature 0.05 \
        --warmup_steps 20000 --total_steps 500000 --lr 0.00005 \
        --scheduler linear --optim adamw --per_gpu_batch_size 64 \
        --output_dir /checkpoint/gizacard/contriever/xling/contriever \

```

--------------------------------

### Evaluate mContriever on Mr. TyDi Dataset

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Execute this command to perform the evaluation using the mContriever model on the prepared Mr. TyDi dataset. The `--normalize_text` flag is used for text normalization during evaluation.

```bash
python beireval.py --model_name_or_path facebook/mcontriever --dataset mrtydi/mrtydi-v1.1-swahili --normalize_text
```

--------------------------------

### Load Contriever Model and Tokenizer

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Load the pre-trained Contriever model and its associated tokenizer from HuggingFace. This is the initial step for using Contriever.

```python
from src.contriever import Contriever
from transformers import AutoTokenizer

contriever = Contriever.from_pretrained("facebook/contriever") 
tokenizer = AutoTokenizer.from_pretrained("facebook/contriever") #Load the associated tokenizer:
```

--------------------------------

### Download Pre-computed Passage Embeddings

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Download pre-computed passage embeddings for Contriever and Contriever-msmarco models. This saves computation time for evaluation.

```bash
wget https://dl.fbaipublicfiles.com/contriever/embeddings/contriever/wikipedia_embeddings.tar
wget https://dl.fbaipublicfiles.com/contriever/embeddings/contriever-msmarco/wikipedia_embeddings.tar
```

--------------------------------

### Run BEIR Evaluation Script

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Execute the beireval.py script to reproduce scores on a specified dataset. Ensure the model name or path and dataset are correctly provided.

```bash
python beireval.py --model_name_or_path contriever-msmarco --dataset scifact
```

--------------------------------

### Convert Mr. TyDi Data to BEIR Format

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

This command converts the downloaded Mr. TyDi dataset into the BEIR format required for evaluation. It uses a Python script provided in the data_scripts directory.

```bash
python data_scripts/convertmrtydi2beir.py mrtydi/mrtydi-v1.1-swahili mrtydi/mrtydi-v1.1-swahili
```

--------------------------------

### Load Various Pre-trained Contriever Models

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Load different versions of the Contriever model, including fine-tuned and multilingual variants, for specific retrieval tasks.

```python
from src.contriever import Contriever

contriever = Contriever.from_pretrained("facebook/contriever") 
contriever_msmarco = Contriever.from_pretrained("facebook/contriever-msmarco")
mcontriever = Contriever.from_pretrained("facebook/mcontriever")
mcontriever_msmarco = Contriever.from_pretrained("facebook/mcontriever-msmarco")
```

--------------------------------

### Generate Sentence Embeddings

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Tokenize a list of sentences and generate their embeddings using the Contriever model. Ensure sentences are preprocessed for the model.

```python
sentences = [
    "Where was Marie Curie born?",
    "Maria Sklodowska, later known as Marie Curie, was born on November 7, 1867.",
    "Born in Paris on 15 May 1859, Pierre Curie was the son of Eugène Curie, a doctor of French Catholic origin from Alsace."
]

inputs = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")
embeddings = model(**inputs)
```

--------------------------------

### Download Wikipedia Passages

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Download the Wikipedia passages dataset used for evaluation. This is a prerequisite for generating passage embeddings.

```bash
wget https://dl.fbaipublicfiles.com/dpr/wikipedia_split/psgs_w100.tsv.gz
```

--------------------------------

### Generate Passage Embeddings with mContriever

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Generate passage embeddings using the facebook/mcontriever model. This process requires specifying the model, output directory, passage file, and shard information, along with options for lowercasing and text normalization.

```bash
python generate_passage_embeddings.py \
    --model_name_or_path facebook/mcontriever \
    --output_dir mcontriever_embeddings  \
    --passages psgs_w100.tsv \
    --shard_id 0 --num_shards 1 \
    --lowercase --normalize_text \

```

--------------------------------

### Compute Retrieval Accuracy

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Run the passage_retrieval.py script to compute retrieval accuracy. This involves specifying the model, passages, pre-computed embeddings, data, output directory, and normalization options.

```bash
python passage_retrieval.py \
    --model_name_or_path facebook/mcontriever \
    --passages psgs_w100.tsv \
    --passages_embeddings "mcontriever_embeddings/*" \
    --data "xmkqa/*.jsonl" \
    --output_dir mcontriever_xmkqa \
    --lowercase --normalize_text \

```

--------------------------------

### Calculate Sentence Similarity

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Compute similarity scores between sentence embeddings using a dot product. This helps in understanding semantic relatedness.

```python
score01 = embeddings[0] @ embeddings[1] #1.0473
score02 = embeddings[0] @ embeddings[2] #1.0095
```

--------------------------------

### Perform Passage Retrieval

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Execute the passage retrieval script using Contriever. This script takes questions and passage embeddings to find relevant passages.

```python
python passage_retrieval.py \
    --model_name_or_path facebook/contriever \
    --passages psgs_w100.tsv \
    --passages_embeddings "contriever_embeddings/*" \
    --data nq_dir/test.json \
    --output_dir contriever_nq \
```

--------------------------------

### Generate Passage Embeddings

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

Generate embeddings for Wikipedia passages using a specified Contriever model. This script is used for creating retrieval indexes.

```bash
python generate_passage_embeddings.py \
    --model_name_or_path facebook/contriever \
    --output_dir contriever_embeddings  \
    --passages psgs_w100.tsv \
    --shard_id 0 --num_shards 1 \
```

--------------------------------

### Contriever Citation

Source: https://github.com/facebookresearch/contriever/blob/main/README.md

BibTeX entry for citing the Contriever paper. Use this in academic publications when referencing the Contriever model.

```bibtex
@misc{izacard2021contriever,
      title={Unsupervised Dense Information Retrieval with Contrastive Learning},
      author={Gautier Izacard and Mathilde Caron and Lucas Hosseini and Sebastian Riedel and Piotr Bojanowski and Armand Joulin and Edouard Grave},
      year={2021},
      url = {https://arxiv.org/abs/2112.09118},
      doi = {10.48550/ARXIV.2112.09118},
}

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.