### Example Dataset Provider Entry Point in setup.py

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/extending/datasets.rst

Register a DatasetProvider with PyTerrier using an entry point in your package's setup.py file. This allows PyTerrier to discover your datasets when your package is installed.

```python
from setuptools import setup

setup(
    ... # <-- the rest of your configuration
    entry_points={
        "pyterrier.dataset_provider": [ # <-- PyTerrier looks for this entry point
            "my_prefix = my_package.MyDatasetProvider" # <-- when a dataset looks like 'my_prefix:{name}', it will load MyDatasetProvider
        ]
    },
)
```

--------------------------------

### Install PyTerrier from GitHub

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/installation.rst

Install the latest version of PyTerrier directly from its GitHub repository. Use this to get the most recent updates.

```bash
pip install --upgrade git+https://github.com/terrier-org/pyterrier.git
```

--------------------------------

### Install Libraries

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/sentence_transformers.ipynb

Installs the python-terrier and sentence-transformers libraries. This is a prerequisite for running the notebook.

```python
%pip install -q python-terrier sentence-transformers
```

--------------------------------

### Install and Configure FastRank for Coordinate Ascent

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/ltr.rst

Demonstrates how to install FastRank and configure it for coordinate ascent learning. The snippet shows setting up training parameters and applying the learned model to a pipeline.

```python
!pip install fastrank
import fastrank
train_request = fastrank.TrainRequest.coordinate_ascent()
params = train_request.params
params.init_random = True
params.normalize = True
params.seed = 1234567

ca_pipe = pipeline >> pt.ltr.apply_learned_model(train_request, form="fastrank")
ca_pipe.fit(train_topics, train_qrels)
```

--------------------------------

### Install PyTerrier with all dependencies

Source: https://github.com/terrier-org/pyterrier/blob/master/README.md

Use this command to install PyTerrier along with all its optional dependencies. Ensure you have Python's pip package manager installed.

```bash
pip install 'pyterrier[all]'
```

--------------------------------

### Basic Non-English Retrieval Setup

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Illustrates the initial setup for non-English retrieval, including index creation and basic retrieval operations. Ensure the correct language models and tokenizers are used for your target language.

```python
import pyterrier as pt
from pyterrier.measures import MRR

if not pt.started():
    pt.init()

# Example: Indexing a non-English dataset (e.g., Spanish)
# Replace 'path/to/spanish_docs' with the actual path to your documents
# and 'spanish_index' with your desired index name.
# Ensure your documents are in a format PyTerrier can read (e.g., CSV, JSON lines).
# For non-English, you might need to specify language-specific configurations
# during index creation if not automatically detected.

# Example of index creation (conceptual - actual parameters may vary based on dataset format):
# index_ref = pt.index(
#     index_path='spanish_index',
#     fields={'docno': 20, 'text': 4096},
#     content_field='text',
#     tokeniser='es_tokeniser' # Example: Spanish tokenizer
# )

# For demonstration, we'll use a pre-existing index or a simplified setup.
# Assume 'es_dataset' is a PyTerrier dataset object for Spanish documents.
# es_dataset = pt.get_dataset('some_spanish_dataset')

# Placeholder for index creation if needed:
# pt.index(es_dataset.get_corpus(), index_path='es_index', ...) 

# Load an existing index (replace 'es_index' with your actual index path)
# index = pt.IndexFactory.of('es_index')

# Placeholder for a retrieval system (e.g., BM25)
# ret = pt.BatchRetrieve(index, wmodel="BM25")

# Example query (Spanish)
# query = "inteligencia artificial"

# Perform retrieval
# results = ret.search(query)
# print(results)

# Example: Evaluating retrieval results (using MRR)
# Assume 'qrels' is a PyTerrier dataset object for Spanish relevance judgments.
# qrels = pt.get_dataset('some_spanish_qrels')
# mrr = MRR()
# score = mrr.evaluate(ret, qrels, metrics=['mrr'])
# print(f"MRR Score: {score}")

print("Non-English retrieval setup complete. Replace placeholders with actual dataset and index paths.")
```

--------------------------------

### Install PyTerrier

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/experiments/Robust04.ipynb

Installs the latest version of PyTerrier from the GitHub repository.

```python
!pip install python-terrier

```

--------------------------------

### Install Dependencies

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/experiments/wow2025-msmarco-v1.ipynb

Installs necessary Python packages for PyTerrier, caching, and T5 models. Restart the kernel after installation if required.

```python
%pip install -q python-terrier pyterrier_caching pyterrier_t5
```

--------------------------------

### Install libpcre3-dev on Linux

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/troubleshooting/installation.rst

Install the pcre development library on Linux systems using apt-get.

```bash
apt-get update -y
apt-get install libpcre3-dev -y
```

--------------------------------

### Example DatasetProvider Implementation

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/extending/datasets.rst

Implement a custom DatasetProvider class by inheriting from pt.datasets.DatasetProvider. This example shows how to define get_dataset and list_dataset_names methods.

```python
import pyterrier as pt

class MyDatasetProvider(pt.datasets.DatasetProvider):
    def get_dataset(self, name):
        if name == "my_dataset":
            return MyDataset()
        else:
            raise ValueError(f"Dataset {name} not found")

    def list_dataset_names(self):
        return ["my_dataset"]
```

--------------------------------

### Install to Local Maven Repo

Source: https://github.com/terrier-org/pyterrier/blob/master/terrier-python-helper/README.md

Installs the package to your local Maven repository. This is a prerequisite for using the package in your projects.

```bash
mvn install
```

--------------------------------

### Install minimal PyTerrier package

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/installation.rst

Install only the core PyTerrier package without any optional dependencies. Use this for a lightweight installation.

```bash
pip install pyterrier
```

--------------------------------

### Initialize PyTerrier and Set Up Indexing

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Initializes the PyTerrier library and sets up the necessary components for indexing documents. Ensure PyTerrier is installed and configured.

```python
import pyterrier as pt
import os 

if not pt.started():
    pt.init()

index_path = "index_fr"
```

--------------------------------

### Example Dataset Implementation

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/extending/datasets.rst

Implement a custom Dataset class by inheriting from pyterrier.datasets.Dataset. This example shows how to define get_topics, get_qrels, and get_corpus_iter methods.

```python
from typing import Iterable, Dict, Any
import pyterrier as pt
import pandas as pd

class MyDataset(pt.datasets.Dataset):
    def get_topics(self, variant=None) -> pd.DataFrame:
        # Logic to load and return topics
        return pd.DataFrame(read_topics('my_topics'))

    def get_qrels(self, variant=None) -> pd.DataFrame:
        # Logic to load and return qrels
        return pd.DataFrame(read_qrels('my_qrels'))

    def get_corpus_iter(self, verbose=True) -> Iterable[Dict[str, Any]]:
        # Logic to load and return corpus iterator
        for line in open('my_file'):
            docno, text = parse_line(line)
            yield {'docno': docno, 'text': text}
```

--------------------------------

### Install Python-Terrier

Source: https://github.com/terrier-org/pyterrier/wiki/Installation-of-Wrong-Package

Use this command to install the correct Python-Terrier package. Avoid installing the similarly named incorrect package.

```bash
pip install python-terrier
```

--------------------------------

### Initialize PyTerrier

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/experiments/Robust04.ipynb

Starts PyTerrier using the latest snapshot version from its GitHub repository. Requires specific boot packages for PRF functionality.

```python
import pyterrier as pt
if not pt.started():
    pt.init(mem=8000, version='snapshot', tqdm='notebook',
            boot_packages=["com.github.terrierteam:terrier-prf:-SNAPSHOT"]
           )

```

--------------------------------

### Install pyterrier_t5

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/experiments/msmarco_BM25_MonoT5.ipynb

Installs the pyterrier_t5 library, which is required for using the MonoT5 reranker.

```python
%pip install -q pyterrier_t5
```

--------------------------------

### Experiment Setup and Execution

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/experiments/msmarco_BM25_MonoT5.ipynb

Sets up an experiment to compare BM25 and MonoT5 retrieval pipelines on the MSMARCO dataset. It retrieves topics and qrels, defines the evaluation metric (NDCG@10), and runs the comparison.

```python
from pyterrier.measures import *
dataset = pt.get_dataset('irds:msmarco-passage/trec-dl-2019/judged')
pt.Experiment(
    [bm25, monot5_pipe],
    dataset.get_topics(),
    dataset.get_qrels(),
    [NDCG@10],
    names=["BM25", "BM25 >> monoT5"]
)
```

--------------------------------

### Install PyTerrier RAG and T5

Source: https://github.com/terrier-org/pyterrier/blob/master/tests/schematics/rag-schematics.ipynb

Installs the necessary PyTerrier RAG and T5 libraries. Use this at the beginning of your notebook or script.

```python
#% pip install -q pyterrier_rag pyterrier_t5
```

--------------------------------

### Initialize PyTerrier and Set Up Index

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Initializes the PyTerrier library and sets up the index for non-English retrieval. Ensure you have the necessary language models and tokenizers installed.

```python
import pyterrier as pt
import pandas as pd

if not pt.started():
    pt.init()

index_path = "/home/user/data/index/wikipedia-en-index"

# For non-English, you might need to specify a different language model and tokenizer
# Example for German:
# index_conf = pt.IndexFactory.of(
#     "/home/user/data/index/wikipedia-de-index",
#     language="de",
#     stemmer="de",
#     stopwords="de"
# )

# For this example, we assume an English index is already available or will be created.
# If creating, you'd use pt.IndexFactory.of(index_path, language="en", ...) 
# and then pt.ingest(index_conf, dataset)

# For demonstration, we'll load an existing index.
# If the index does not exist, this will raise an error.
# In a real scenario, you would create it first.
# index_ref = pt.get_index(index_path)

# Placeholder for index loading/creation logic
print("PyTerrier initialized. Index path set to:", index_path)
print("Note: Actual index creation/loading logic needs to be implemented based on your data.")
```

--------------------------------

### Pipeline Optimization Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/transformer.rst

Demonstrates semantic equivalence and potential optimization of PyTerrier pipelines. The second pipeline might be more efficient due to compilation.

```python
pipe1 = pt.terrier.Retrieve(index, "BM25") % 10
pipe2 = pipe1.compile()
```

--------------------------------

### Basic Experiment Setup

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/experiments.rst

Compares two retrievers (TF_IDF and BM25) on a dataset, evaluating 'map' and 'recip_rank' metrics. Requires dataset, topics, and qrels.

```python
dataset = pt.get_dataset("vaswani")
# vaswani dataset provides an index, topics and qrels

# lets generate two BRs to compare
tfidf = pt.terrier.Retriever(dataset.get_index(), wmodel="TF_IDF")
bm25 = pt.terrier.Retriever(dataset.get_index(), wmodel="BM25")

pt.Experiment(
    [tfidf, bm25],
    dataset.get_topics(),
    dataset.get_qrels(),
    eval_metrics=["map", "recip_rank"]
)
```

--------------------------------

### Configure Retrieval for German

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

This example demonstrates setting up a retrieval pipeline for German documents. It uses a pre-trained German language model for analysis.

```python
from pyterrier.pipelines import Tfidf, Searcher

# Assuming 'german_index' is already created
retrieval_pipeline = pt.pipelines.Searcher(index_path='german_index', language='de')
```

--------------------------------

### Bo1QueryExpansion Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/terrier/rewrite.rst

Shows how to use Bo1QueryExpansion for query expansion with a Retriever. The rewritten query is visible outside the Retriever.

```python
bo1 = pt.rewrite.Bo1QueryExpansion(index)
dph = pt.terrier.Retriever(index, wmodel="DPH")
pipelineQE = dph >> bo1 >> dph
```

--------------------------------

### Example of a Different Language (Spanish)

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Demonstrates setting up retrieval for another language, Spanish ('es'), by specifying the appropriate language parameters for indexing and retrieval.

```python
# Example for Spanish retrieval
# Assume you have a Spanish dataset and index path
index_path_es = "/home/user/data/index/wikipedia-es-index"

# If creating index for Spanish:
# if not pt.IndexFactory.exists(index_path_es):
#     index_conf_es = pt.IndexFactory.of(index_path_es, language="es", stemmer="es", stopwords="es")
#     print(f"Created Spanish index at {index_path_es}")
#     # pt.ingest(index_conf_es, spanish_docs_df)
#     # print("Ingested Spanish documents.")
# else:
#     print(f"Spanish index already exists at {index_path_es}")

# Load the Spanish index (assuming it exists)
# index_es = pt.get_index(index_path_es)

# Define a retrieval pipeline for Spanish
# retrieval_pipeline_es = index_es.bm25(text_field="text")

# Define a Spanish query
# query_es = "idioma español"

# Perform retrieval
# results_es = retrieval_pipeline_es(query_es)

# print(f"Retrieval results for query: '{query_es}'")
# print(results_es.head())

print("Spanish retrieval setup outlined. Actual execution requires Spanish data and index.")
```

--------------------------------

### Precomputing Common Pipeline Prefixes

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/experiments.rst

This example shows how to use the `precompute_prefix=True` argument in `pt.Experiment`. This optimizes performance by precomputing results for common initial pipeline stages, avoiding redundant computations when multiple pipelines share a prefix.

```python
from pyterrier_t5 import MonoT5ReRanker
bm25 = pt.terrier.Retriever.from_dataset('vaswani', 'terrier_stemmed_text', wmodel='BM25', num_results=100)
monoT5 = MonoT5ReRanker()

monoT5 = bm25 >> monoT5
pt.Experiment(
    [bm25, monoT5], 
    pt.get_dataset('vaswani').get_topics(), 
    pt.get_dataset('vaswani').get_qrels(), 
    eval_metrics=['map'], 
    precompute_prefix=True
)
```

--------------------------------

### Example Pipeline with Query Reset

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/terrier/rewrite.rst

Demonstrates a pipeline that includes query rewriting (RM3) and then reverts to the original query formulation using reset, suitable for multi-stage ranking like with MonoT5.

```python
from pyterrier_t5 import MonoT5ReRanker
index = pt.terrier.TerrierIndex.example()
dph = index.dph()
monoT5 = MonoT5ReRanker()
# FOLD
pipeline = index.dph() >> index.rm3() >> index.dph() >> pt.rewrite.reset() >> pt.get_dataset('irds:vaswani').text_loader() >> monoT5
```

--------------------------------

### Sequential Dependence Model Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/terrier/rewrite.rst

Demonstrates the pipeline for the Sequential Dependence Model (SDM) using PyTerrier. It checks if the index has the necessary positional information for SDM.

```python
pipeline = index.sdm() >> index.dph()
```

--------------------------------

### Install PyTerrier and related libraries

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/schematics-demo.ipynb

Installs the necessary Python packages for PyTerrier, including extensions for T5, alpha features, and Doc2Query.

```python
!pip install python-terrier pyterrier-t5 pyterrier-alpha pyterrier-doc2query
```

--------------------------------

### Using a Multilingual Index

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

This example shows how to query a multilingual index. The system automatically detects the language of the query and applies appropriate analysis.

```python
multilingual_pipeline = pt.pipelines.Searcher(index_path='multilingual_index')
query_es = "¿Qué es PyTerrier?"
results_es = multilingual_pipeline.search(query_es)

query_fr = "Qu'est-ce que PyTerrier?"
results_fr = multilingual_pipeline.search(query_fr)
```

--------------------------------

### Indexing pipeline with Doc2Query

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/terrier/indexing.rst

An example of an indexing pipeline that first expands documents with Doc2Query and then indexes them using the Terrier indexer.

```python
import pyterrier as pt
from pyterrier_doc2query import Doc2Query
index = pt.terrier.TerrierIndex("my_index.terrier")
pipeline = Doc2Query(append=True) >> index.indexer()
```

--------------------------------

### Learning to Rank (LTR) Pipeline Setup

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/pipeline_examples.md

Constructs a Learning to Rank pipeline by combining candidate retrieval mechanisms and applying a learned model. Uses XGBoost for ranking.

```python
bm25_cands = pt.terrier.Retriever(indexref, wmodel="BM25")
dph_cands = pt.terrier.Retriever(indexref, wmodel="DPH")
all_cands = bm25_cands | dph_cands

all_features = all_cands >> (  
    pt.terrier.Retriever(indexref, wmodel="BM25F") **
    pt.rewrite.SDM() >> pt.terrier.Retriever(indexref, wmodel="BM25")
    )

import xgboost as xgb
params = {'objective': 'rank:ndcg', 
          'learning_rate': 0.1, 
          'gamma': 1.0, 'min_child_weight': 0.1,
          'max_depth': 6,
          'verbose': 2,
          'random_state': 42 
         }
lambdamart = pt.ltr.apply_learned_model(xgb.sklearn.XGBRanker(**params), form='ltr')
final_pipe = all_features >> lambdamart
final_pipe.fit(tr_topics, tr_qrels, va_topics, va_qrels)
```

--------------------------------

### Evaluating Retrieval Performance

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

This example demonstrates how to evaluate the performance of a retrieval system using standard metrics. It requires a dataset with queries and relevance judgments.

```python
from pyterrier.measures import MAP, MRR

# Assuming 'qrels' and 'queries' are loaded
trecc_eval = pt.Transformers.from_properties(
    name='trec-eval',
    properties={
        'trec_eval.metric': 'map',
        'trec_eval.qrels': 'path/to/qrels.txt',
        'trec_eval.queries': 'path/to/queries.txt'
    }
)

performance = trecc_eval.transform(results)
print(performance)
```

--------------------------------

### Building a Retrieval Pipeline with Terrier

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/terrier/retrieval.rst

Chain multiple retrieval and re-ranking steps to create a complex search pipeline. This example first retrieves top 100 results with BM25, then re-ranks them with PL2.

```python
index = pt.terrier.TerrierIndex.example()
index.bm25() % 100 >> index.pl2()
```

--------------------------------

### Create and use an indexer

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/terrier/indexing.rst

Demonstrates creating a Terrier index and an indexer with custom tokenisation and position storage. It then indexes a list of documents.

```python
import pyterrier as pt
my_index = pt.terrier.TerrierIndex("my_index.terrier")
indexer = index.indexer(tokeniser='twitter', store_positions=True) # :footnote: See :meth:`TerrierIndex.indexer <pyterrier.terrier.TerrierIndex.indexer>` for full list of parameters.
indexer.index([
    {"docno" : "tweet1", "text" : "This is a tweet! #pyterrier"},
    {"docno" : "tweet2", "text" : "Another tweet, with a link: https://example.com"}
])
```

--------------------------------

### Configure and Run BM25+BM25 Retrieval

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Configures and runs a retrieval experiment combining two BM25 models. This demonstrates a more complex retrieval setup.

```python
retrieval_model_bm25_bm25 = pt.BatchRetrieve(index_ref, wmodel="BM25", controls={'k1': 0.5, 'b': 0.2})

# Example query
query = "inteligencia artificial"

# Perform retrieval
results_bm25_bm25 = retrieval_model_bm25_bm25.search(query)
print(f"BM25+BM25 results for query: '{query}'")
print(results_bm25_bm25)
```

--------------------------------

### Visualize Complex Pyterrier Pipeline

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/schematic.rst

Example of creating a schematic for a complex pipeline involving multiple retrieval methods and query rewrites, followed by text loading.

```python
import pyterrier_alpha as pta
index = pt.terrier.TerrierIndex.example()
dataset = pt.get_dataset('irds:vaswani')
pta.fusion.RRFusion(
    index.bm25(),
    pt.rewrite.SDM() >> index.bm25(),
    index.bm25() >> pt.rewrite.RM3(index.index_ref()) >> index.bm25(),
) >> dataset.text_loader()
```

--------------------------------

### Multilingual Retrieval Example (English and Spanish)

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Demonstrates retrieving documents from an index that contains both English and Spanish documents. This requires a multilingual index or separate indexes for each language.

```python
# Assuming you have an index that supports multiple languages or separate indexes
# For simplicity, this example assumes a single index that can handle mixed languages or a pre-configured multilingual index.
# In a real scenario, you might need to configure the indexer with appropriate language detection or multiple tokenizers.

# Example: Using a hypothetical multilingual index
# multilingual_index_path = "/tmp/terrier-data/multilingual-index"
# multilingual_indexer = pt.indexers.DFRIndexer(multilingual_index_path, language='multilingual') # Hypothetical setting
# multilingual_indexer.index(mixed_language_dataset.get_corpus_iter(), mixed_language_dataset.get_topics_iter())

# For this example, we'll simulate retrieval from an existing index (e.g., msmarco-es)
# and show how to query in different languages.

# Load the Spanish index
spanish_index_ref = pt.IndexRef.of(index_path)
spanish_retrieval_index = pt.IndexFactory.of(spanish_index_ref)

# Query in Spanish
spanish_query = "inteligencia artificial"
spanish_results = spanish_retrieval_index.search(spanish_query)
print("--- Spanish Query Results ---")
print(spanish_results)

# If you had an English index, you would load and query it similarly:
# english_index_path = "/tmp/terrier-data/msmarco-en"
# english_index_ref = pt.IndexRef.of(english_index_path)
# english_retrieval_index = pt.IndexFactory.of(english_index_ref)
# english_query = "artificial intelligence"
# english_results = english_retrieval_index.search(english_query)
# print("\n--- English Query Results ---")
# print(english_results)

# Note: True multilingual retrieval often involves more complex setup, like a single index with a multilingual tokenizer
# or using translation services before querying.
```

--------------------------------

### Uninstall Incorrect Package and Install Correct One

Source: https://github.com/terrier-org/pyterrier/wiki/Installation-of-Wrong-Package

If you have installed the wrong package, use these commands to uninstall it and then install the correct Python-Terrier package. Remember to restart your kernel if using a notebook.

```bash
pip uninstall pyterrier
```

```bash
pip install python-terrier
```

--------------------------------

### Initialize PyTerrier and Set Up Index

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Initializes the PyTerrier library and sets up a document index. This is a prerequisite for most retrieval tasks.

```python
import pyterrier as pt
import os

if not pt.started():
    pt.init()

index_path = "/tmp/pt_index_non_en"

```

--------------------------------

### Count PyPI Downloads by Version (Last 90 Days)

Source: https://github.com/terrier-org/pyterrier/wiki/Release-Statistics

This query counts PyPI downloads for the 'pyterrier' project, filtered by version starting with '1' and within the last 90 days.

```SQL
#standardSQL 

SELECT file.version, COUNT(*) AS num_downloads FROM `bigquery-public-data.pypi.file_downloads` 
WHERE 
  file.project = 'pyterrier' 
  AND starts_with(file.version, '1')
  AND DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY) AND CURRENT_DATE() 
  GROUP BY file.version
  ORDER BY file.version

```

--------------------------------

### Initialize PyTerrier and Set Up Index

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Initializes PyTerrier and sets up a retrieval index. This is a prerequisite for most retrieval tasks.

```python
import pyterrier as pt
import os

if not pt.started():
    pt.init()

index_path = "./index_non_en"

if not os.path.exists(index_path):
    os.makedirs(index_path)

indexer = pt.TRECCollectionIndexer(index_path, overwrite=True)
index_ref = indexer.index(['./non_en_docs'], docs='text',зацию='text')
print(f"Index created at: {index_path}")
print(index_ref)
```

--------------------------------

### Initialize PyTerrier and Set Up Index

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Initializes the PyTerrier library and sets up a document index. This is a prerequisite for performing any retrieval tasks.

```python
import pyterrier as pt
import os 

if not pt.started():
    pt.init()

index_path = "/tmp/pt_index_non_english"

if not os.path.exists(index_path):
    os.makedirs(index_path)

indexer = pt.DFIndexer(index_path, overwrite=True)

# Example: Add documents to the index (replace with your actual document loading)
docs = [
    {'docno': 'doc1', 'text': 'This is the first document in English.'},
    {'docno': 'doc2', 'text': 'Dies ist das zweite Dokument auf Deutsch.'},
    {'docno': 'doc3', 'text': 'Ceci est le troisième document en français.'}
]

indexer.index(docs)
print(f"Index created at {index_path}")

```

--------------------------------

### Install Python-Terrier

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/experiment.ipynb

Installs the python-terrier package and its dependencies. This is a prerequisite for using Pyterrier.

```python
%pip install -q python-terrier
```

--------------------------------

### Run Basic Experiment

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/experiment.ipynb

Execute an experiment with a list of retrieval objects, topics, qrels, and specified metrics. Results are returned as a DataFrame by default.

```python
pt.Experiment(
    [TF_IDF,BM25,PL2],
    vaswani.get_topics(),
    vaswani.get_qrels(),
    ['map','ndcg'])
```

--------------------------------

### Using a Different Language Code

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

This example shows how to specify a different language code (e.g., 'de' for German) when creating or loading an index. This is crucial for correct text processing.

```python
import pyterrier as pt

pt.init()

# Example for German documents
indexer = pt.index.Foreach do
    pt.index.IndexFactory.of(
        pt.index.PrecomputedIndex.from_files(
            "/path/to/your/german_docs",
            "/path/to/your/german_index",
            "de"  # Language code for German
        )
    )
    
indexer.index()

```

--------------------------------

### Install PyTerrier DR

Source: https://github.com/terrier-org/pyterrier/blob/master/tests/schematics/dr-schematics.ipynb

Installs the pyterrier_dr package quietly. This is a prerequisite for using the DR functionalities.

```python
#% pip install -q pyterrier_dr
```

--------------------------------

### Initialize PyTerrier and Set Up Index

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Initializes the PyTerrier library and sets up a retrieval index. This is a prerequisite for performing any retrieval tasks.

```python
import pyterrier as pt
import os

if not pt.started():
    pt.init()

index_path = "/tmp/pt_index_fr"

if not os.path.exists(index_path):
    os.makedirs(index_path)

index_ref = pt.IndexRef.of(index_path)
print(f"Index created at: {index_path}")
```

--------------------------------

### Install pcre on macOS

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/troubleshooting/installation.rst

Use Homebrew to install the pcre library on macOS if pyautocorpus fails to run.

```bash
brew install pcre
```

--------------------------------

### Running an Experiment with Multiple Metrics

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/experiments.rst

This snippet demonstrates how to set up and run an experiment using `pt.Experiment`. It specifies the retrieval models, dataset topics and qrels, and a list of evaluation metrics including graded and binary relevance measures.

```python
from pyterrier.measures import *
dataset = pt.get_dataset("trec-deep-learning-passages")
pt.Experiment(
    [tfidf, bm25],
    dataset.get_topics("test-2019"),
    dataset.get_qrels("test-2019"),
    eval_metrics=[RR(rel=2), nDCG@10, nDCG@100, AP(rel=2)],
)
```

--------------------------------

### Retrieval Transformer Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/transformer.rst

Example of a retrieval transformer. This snippet shows how to use pt.terrier.Retriever() for retrieval operations.

```python
pt.terrier.Retriever()
```

--------------------------------

### Feature Retrieval Transformer Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/transformer.rst

Example of a feature retrieval transformer. This snippet shows the use of pt.terrier.FeaturesRetriever() for feature scoring.

```python
pt.terrier.FeaturesRetriever()
```

--------------------------------

### Document Scoring Transformer Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/transformer.rst

Example of a document scoring transformer. This snippet demonstrates the use of pt.apply.doc_score() for re-ranking documents.

```python
pt.apply.doc_score()
```

--------------------------------

### Query Expansion Transformer Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/transformer.rst

Example of a query expansion transformer. This snippet illustrates the use of pt.rewrite.RM3() for query expansion.

```python
pt.rewrite.RM3()
```

--------------------------------

### Query Rewriting Transformer Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/transformer.rst

Example of a query rewriting transformer. This snippet demonstrates the use of pt.rewrite.SDM() for query rewriting.

```python
pt.rewrite.SDM()
```

--------------------------------

### Check Java Installation for PyTerrier

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/installation.rst

Verify that Java is installed and properly configured for PyTerrier functionality. This function can be called to check the Java environment.

```python
pt.java.init()
```

--------------------------------

### Install Cookiecutter and Create PyTerrier Extension

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/extending/packages.rst

Use cookiecutter to generate a new PyTerrier extension package. This command installs the cookiecutter tool and then uses a specific template to scaffold your package.

```bash
pip install -U cookiecutter
cookiecutter https://github.com/seanmacavaney/cookiecutter-pyterrier.git
```

--------------------------------

### Count PyPI Downloads by Operating System (Last 30 Days)

Source: https://github.com/terrier-org/pyterrier/wiki/Release-Statistics

This query counts PyPI downloads for the 'python-terrier' project, aggregated by operating system name for the last 30 days.

```SQL
SELECT details.system.name v, COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'python-terrier'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND CURRENT_DATE()
  GROUP BY v
  ORDER BY v

```

--------------------------------

### Get Text Generator from Backend

Source: https://github.com/terrier-org/pyterrier/blob/master/tests/schematics/rag-schematics.ipynb

Retrieves the text generator object from the configured backend.

```python
backend.text_generator()
```

--------------------------------

### First Passage Aggregation Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/text.rst

Demonstrates the output of the first_passage transformer on a sample retrieval dataframe.

```text
The output of the `first_passage()` transformer would be:

+-------+---------+--------+--------+
+  qid  +  docno  +  rank  + score  +
+=======+=========+========+========+
|  q1   | d2      +  0     + 4.0    +
+-------+---------+--------+--------+
|  q1   | d1      +  1     + 1.0    +
+-------+---------+--------+--------+
```

--------------------------------

### Mean Passage Aggregation Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/text.rst

Demonstrates the output of the mean_passage transformer on a sample retrieval dataframe.

```text
The output of the `mean_passage()` transformer would be:

+-------+---------+--------+--------+
+  qid  +  docno  +  rank  + score  +
+=======+=========+========+========+
|  q1   | d1      +  0     + 4.5    +
+-------+---------+--------+--------+
|  q1   | d2      +  1     + 4.0    +
+-------+---------+--------+--------+
```

--------------------------------

### Count PyPI Downloads by Date and Version (Last 30 Days)

Source: https://github.com/terrier-org/pyterrier/wiki/Release-Statistics

This query counts PyPI downloads for the 'python-terrier' project, aggregated by both date and version for the last 30 days.

```SQL
SELECT DATE(timestamp) d, file.version v, COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'python-terrier'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND CURRENT_DATE()
  GROUP BY d, v
  ORDER BY d DESC, v

```

--------------------------------

### Initialize MonoT5 ReRanker and QueryScorer

Source: https://github.com/terrier-org/pyterrier/blob/master/tests/schematics/indexer-schematics.ipynb

Sets up the MonoT5 re-ranker and a QueryScorer using it. This is used for scoring and potentially re-ranking queries.

```python
monot5 = pyterrier_t5.MonoT5ReRanker()
qs = pyterrier_doc2query.QueryScorer(monot5)
```

--------------------------------

### Max Passage Aggregation Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/text.rst

Demonstrates the output of the max_passage transformer on a sample retrieval dataframe.

```text
+-------+---------+--------+--------+
+  qid  +  docno  +  rank  + score  +
+=======+=========+========+========+
|  q1   | d1%p5   +  0     + 5.0    +
+-------+---------+--------+--------+
|  q1   | d2%p4   +  1     + 4.0    +
+-------+---------+--------+--------+
|  q1   | d1%p3   +  2     + 3.0    +
+-------+---------+--------+--------+
|  q1   | d1%p1   +  3     + 1.0    +
+-------+---------+--------+--------+

The output of the `max_passage()` transformer would be:

+-------+---------+--------+--------+
+  qid  +  docno  +  rank  + score  +
+=======+=========+========+========+
|  q1   | d1      +  0     + 5.0    +
+-------+---------+--------+--------+
|  q1   | d2      +  1     + 4.0    +
+-------+---------+--------+--------+
```

--------------------------------

### Schematic Example of Feature Union

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/operators.rst

A schematic representation of initializing an index and applying feature union to its retrievers.

```python
index = pt.terrier.TerrierIndex.example()
index.dph() >> ( index.retriever("BM25F") ** index.retriever("PL2F") )
```

--------------------------------

### Initialize a Terrier indexer

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/schematics-demo.ipynb

Creates an instance of a Terrier indexer, which is a pipeline for building a search index.

```python
# Indexers are pipelines, too.
indexer = pt.terrier.TerrierIndex("./my-index").indexer()
indexer
```

--------------------------------

### K-Max Average Passage Aggregation Example

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/text.rst

Demonstrates the output of the kmaxavg_passage(2) transformer on a sample retrieval dataframe.

```text
Finally, the output of the `kmaxavg_passage(2)` transformer would be:

+-------+---------+--------+--------+
+  qid  +  docno  +  rank  + score  +
+=======+=========+========+========+
|  q1   | d2      +  1     + 4.0    +
+-------+---------+--------+--------+
|  q1   | d1      +  0     + 1.0    +
+-------+---------+--------+--------+
```

--------------------------------

### Count PyPI Downloads by Date (Last 30 Days)

Source: https://github.com/terrier-org/pyterrier/wiki/Release-Statistics

This query counts PyPI downloads for the 'python-terrier' project, aggregated by date for the last 30 days.

```SQL
#standardSQL
SELECT DATE(timestamp) d, COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'python-terrier'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND CURRENT_DATE()
  GROUP BY  d
  ORDER BY d

```

--------------------------------

### Indexing with Doc2Query in PyTerrier

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/extending/indexer_retrieval.rst

Use this snippet to index a corpus using the doc2query model with PyTerrier. Ensure pyterrier_doc2query is installed.

```python
import pyterrier_doc2query
doc2query = pyterrier_doc2query.Doc2Query(append=True)
pipeline = doc2query >> MyIndexer()
pipeline.index(corpus)
```

--------------------------------

### Deploy Artifacts with Maven

Source: https://github.com/terrier-org/pyterrier/wiki/Making-a-release

Clean, package, and deploy artifacts using Maven. Ensure GPG_TTY is set for signing on macOS.

```shell
#on Mac
export GPG_TTY=$(tty)
mvn clean package deploy
```

--------------------------------

### Initialize PyTerrier and Set Up German Index

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

Initializes PyTerrier and sets up an index for German documents. This is the first step in performing non-English retrieval for German.

```python
import pyterrier as pt
import os

if not pt.started():
    pt.init()

index_path_de = "/tmp/pt_index_de"

if not os.path.exists(index_path_de):
    os.makedirs(index_path_de)

index_ref_de = pt.IndexRef.of(index_path_de)
print(f"Index created at: {index_path_de}")
```

--------------------------------

### Deploy to Maven Central

Source: https://github.com/terrier-org/pyterrier/blob/master/terrier-python-helper/README.md

Deploys the package to Maven Central for public distribution. Requires GPG setup for signing artifacts.

```bash
GPG_TTY=$(tty) mvn -P release deploy
```

--------------------------------

### Get Text from Dataset

Source: https://github.com/terrier-org/pyterrier/blob/master/tests/schematics/schematics.ipynb

Retrieves the text content from a specified PyTerrier dataset. This is useful for accessing document bodies for further processing.

```python
text = pt.text.get_text(pt.get_dataset("irds:vaswani"), "text")
text
```

--------------------------------

### Create a Pandas DataFrame

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/indexing.ipynb

Creates a sample Pandas DataFrame with 'docno', 'url', and 'text' columns to be indexed.

```python
df = pd.DataFrame({
'docno':
['1', '2', '3'],
'url': 
['url1', 'url2', 'url3'],
'text': 
['He ran out of money, so he had to stop playing',
'The waves were crashing on the shore; it was a',
'The body may perhaps compensates for the loss'
})
df
```

--------------------------------

### Get Vaswani Topics

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/retrieval_and_evaluation.ipynb

Retrieves pre-parsed topics directly from the Vaswani dataset object. Displays the first 5 topics.

```python
topics = vaswani_dataset.get_topics()
topics.head(5)
```

--------------------------------

### Build an Artifact Package

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/artifacts/how-to.rst

Create a distributable artifact package (e.g., a tar.lz4 archive) that can be uploaded to external services. This is the first step before sharing artifacts via services like Dropbox or Google Drive.

```python
import pyterrier as pt

artifact = ...  # e.g., pt.Artifact.load('path/to/index')
package_path = artifact.build_package('artifact.tar.lz4')
```

--------------------------------

### Get Text from IRDS Dataset

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/text.rst

Demonstrates using pt.text.get_text with an IRDS dataset, specifying the metadata attribute 'text' to retrieve.

```python
pt.text.get_text(pt.get_dataset('irds:vaswani'))
```

--------------------------------

### Create and Index with FlexIndex

Source: https://github.com/terrier-org/pyterrier/blob/master/tests/schematics/dr-schematics.ipynb

Creates a FlexIndex instance for a local index directory and defines an indexing pipeline. The indexer mode defaults to 'create'.

```python
flex = pyterrier_dr.FlexIndex("./dr_index")
index_pipe = model >> flex.indexer()
```

--------------------------------

### Index Non-English Documents

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

This snippet shows how to create an index for non-English documents. Ensure you have the necessary language models and tokenizers installed.

```python
import pyterrier as pt
from pyterrier.index import IndexFactory

# Assuming 'docs' is a pandas DataFrame with 'docno' and 'text' columns
# and 'index_path' is the desired directory for the index

# Example: Indexing French documents
indexer = IndexFactory.of("TF_IDF", {
```

--------------------------------

### Index Non-English Documents

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

This snippet shows how to index documents in a non-English language. Ensure you have the appropriate language models and tokenizers installed.

```python
import pyterrier as pt
from pyterrier.measures import ROUGE_1, ROUGE_2, ROUGE_L

if not pt.started():
    pt.init()

# Example for indexing French documents
# Assumes documents are in a pandas DataFrame with 'docno' and 'text' columns
# and a 'language' column set to 'fr'
# pt.text.get_text_from_file('path/to/french_docs.jsonl') # Example of loading

# For demonstration, we'll use a placeholder DataFrame
import pandas as pd
data = {'docno': [1, 2, 3], 'text': ['Ceci est le premier document.', 'Voici le deuxième document.', 'Et voici le troisième.'], 'language': ['fr', 'fr', 'fr']}
df = pd.DataFrame(data)

# Define the index path
index_path = "/tmp/terrier-index-fr-docs"

# Create an English-language index
# For non-English, you might need to specify language-specific settings if available
# For simplicity, we use a generic indexer here.
# In a real scenario, consider language-specific stemmers/tokenizers if PyTerrier supports them for the language.

# If you have a specific non-English indexer or configuration, use it here.
# Example: pt.index(df, index_path, text_field='text', docno_field='docno', language_field='language')

# Using a basic indexer for demonstration
indexer = pt.index.IterDictIndex(index_path, meta={'docno': 20, 'text': 2000, 'language': 20})
index_ref = indexer.index(df["text"], df["docno"], df["language"])
```

--------------------------------

### PyTerrier Indexer Input Format

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/extending/indexer_retrieval.rst

Example of document data format for PyTerrier indexers. Documents should include 'docno' and 'text' keys.

```python
[
  { 'docno' : 'd1', 'text' : 'Hello there'},
  { 'docno' : 'd2', 'text' : 'Nice to meet you'}
]
```

--------------------------------

### Count PyPI Downloads by Python Version (Last 30 Days)

Source: https://github.com/terrier-org/pyterrier/wiki/Release-Statistics

This query counts PyPI downloads for the 'python-terrier' project, aggregated by Python version (e.g., '3.9.') for the last 30 days.

```SQL
SELECT REGEXP_EXTRACT(details.python, r'\d.\d.') v, COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'python-terrier'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND CURRENT_DATE()
  GROUP BY v
  ORDER BY v

```

--------------------------------

### Get Vaswani Qrels

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/retrieval_and_evaluation.ipynb

Retrieves pre-parsed qrels directly from the Vaswani dataset object. This simplifies the process of obtaining relevance judgments.

```python
qrels = vaswani_dataset.get_qrels()
```

--------------------------------

### Get Document Frequency of a Term

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/index_api.ipynb

Retrieve the document frequency of a stemmed term from the index lexicon. Ensure the term is stemmed before querying.

```python
index.getLexicon()["chemic"].getDocumentFrequency()
```

--------------------------------

### Count PyPI Downloads by Country (Last 30 Days)

Source: https://github.com/terrier-org/pyterrier/wiki/Release-Statistics

This query counts PyPI downloads for the 'python-terrier' project, aggregated by country code for the last 30 days, ordered by download count.

```SQL
SELECT country_code v, COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'python-terrier'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND CURRENT_DATE()
  GROUP BY  v
  ORDER BY  num_downloads DESC
  

```

--------------------------------

### Index Chinese Documents

Source: https://github.com/terrier-org/pyterrier/blob/master/examples/notebooks/non_en_retrieval.ipynb

This snippet shows how to index Chinese documents using the Chinese Analyzer. Ensure you have the necessary language models installed.

```python
import pyterrier as pt
from pyterrier.index import IndexFactory

if not pt.started():
    pt.init()

index_path = "chinese_index"

indexer = IndexFactory.of(
    "TrecCollection",
    index_path,
    "chinese",
    pt.get_transformer("chinese_analyzer"),
    overwrite=True
)

indexer.index_dir("path/to/your/chinese/documents")
print(f"Index created at {index_path}")
```

--------------------------------

### Create a Terrier Index

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/terrier/quick-start.rst

Creates a Terrier index at a specified file path and indexes the ANTIQUE dataset. The '.terrier' extension is conventional but not required.

```python
import pyterrier as pt
my_index = pt.terrier.TerrierIndex('my_index.terrier')
dataset = pt.get_dataset('irds:antique')
my_index.index(dataset.get_corpus_iter())
```

--------------------------------

### Receive a P2P Artifact

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/artifacts/how-to.rst

Use this snippet to securely transfer and load an artifact using a P2P connection. Ensure you have the 'magic-wormhole' package installed.

```python
import pyterrier as pt

artifact = pt.Artifact.from_p2p('xx-xxx-xxx', 'my_index')
```

--------------------------------

### Load PyTerrier Artifact from HuggingFace

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/artifacts/how-to.rst

Use this method to download and load a PyTerrier artifact from the HuggingFace Hub. Ensure you have the 'huggingface-hub' package installed.

```python
import pyterrier as pt

index = pt.Artifact.from_hf('username/myindex')
```

--------------------------------

### Experiment Configuration Options

Source: https://github.com/terrier-org/pyterrier/blob/master/docs/experiments.rst

Illustrates different combinations of filter_by_topics and filter_by_qrels parameters in PyTerrier's Experiment class and their impact on result sets.

```text
+----------------------+----------------------+------------------+--------------------------------------------------------------------+
| ``filter_by_topics`` | ``filter_by_qrels``  | Results consider | Notes                                                              |
+======================+======================+==================+====================================================================+
| ``True`` (default)   | ``False`` (default)  | ``A,B``          | ``C`` is removed because it does not appear in the topics.         |
+----------------------+----------------------+------------------+--------------------------------------------------------------------+
| ``True`` (default)   | ``True``             | ``B``            | Acts as an intersection of the qids found in the qrels and topics. |
+----------------------+----------------------+------------------+--------------------------------------------------------------------+
| ``False``            | ``False`` (default)  | ``A,B,C``        | Acts as a union of the qids found in qrels and topics.             |
+----------------------+----------------------+------------------+--------------------------------------------------------------------+
| ``False``            | ``True``             | ``B,C``          | ``A`` is removed because it does not appear in the qrels.          |
+----------------------+----------------------+------------------+--------------------------------------------------------------------+
```

--------------------------------

### Initialize IRCOT with Different Pipeline

Source: https://github.com/terrier-org/pyterrier/blob/master/tests/schematics/rag-schematics.ipynb

Initializes the IRCOT framework with a different retrieval pipeline, omitting the text loading step and directly using BM25 and MonoT5.

```python
ircotbad = pyterrier_rag.frameworks.IRCOT(bm25 >> monoT5, flant5)
ircotbad
```