### Import Libraries and Setup

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_customization_guide.ipynb

Imports necessary libraries and sets up environment variables for deterministic behavior. This is a common setup for machine learning projects.

```python
import os
import typing as tp
import typing_extensions as tpe
import warnings
from pathlib import Path

import torch.nn as nn
import pandas as pd
import torch
import numpy as np
from lightning_fabric import seed_everything
from pytorch_lightning import Trainer

from rectools import Columns
from rectools.dataset import Dataset
from rectools.models import BERT4RecModel, SASRecModel, PopularModel
from rectools.dataset.dataset import DatasetSchema
from rectools.model_selection import TimeRangeSplitter, cross_validate
from rectools.metrics import (
    MAP,
    CoveredUsers,
    AvgRecPopularity,
    Intersection,
    HitRate,
    Serendipity,
)
from rectools.models.nn.item_net import (
    ItemNetBase,
    SumOfEmbeddingsConstructor,
)
from rectools.models.nn.transformers.net_blocks import (
    PreLNTransformerLayer,
    TransformerLayersBase,
)
from rectools.models.nn.transformers.constants import MASKING_VALUE
from rectools.models.nn.transformers.bert4rec import BERT4RecDataPreparator
from rectools.models.nn.transformers.lightning import TransformerLightningModule
from rectools.visuals import MetricsApp

# Enable deterministic behaviour with CUDA >= 10.2
os.environ["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8"
warnings.simplefilter("ignore")
```

--------------------------------

### Construct ItemToItemVisualApp

Source: https://github.com/mtswebservices/rectools/blob/main/examples/7_visualization.ipynb

Example of constructing an `ItemToItemVisualApp` for visualizing item-to-item recommendations, supporting multiple models, random targets, and custom formatting.

```python
app = ItemToItemVisualApp.construct(
    reco=reco, 
    item_data=item_data,
    n_random_items=2,
    formatters=formatters,
)
```

--------------------------------

### Install RecTools with All Extensions

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Command to install RecTools with all available extensions. This is useful for accessing the full range of features.

```bash
pip install rectools[all]
```

--------------------------------

### Configure EASEModel using a dictionary

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Initialize an EASEModel by providing a configuration dictionary. This example sets the regularization strength and verbosity level for the model.

```python
config = {
    "regularization": 100,
    "verbose": 1,
}

model = EASEModel.from_config(config)
model.get_params(simple_types=True)
```

--------------------------------

### Install RecTools with Optional Dependencies

Source: https://github.com/mtswebservices/rectools/blob/main/docs/source/index.rst

Install RecTools with specific optional dependencies like 'lightfm' and 'torch' for enhanced functionality.

```bash
$ pip install rectools[lightfm,torch]
```

--------------------------------

### Load Model from Config

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Initialize a recommender model from a configuration dictionary. This allows for reproducible model setups.

```python
model = model_from_config(config)
```

--------------------------------

### Setup and Import Dependencies

Source: https://github.com/mtswebservices/rectools/blob/main/examples/6_benchmark_lightfm_inference.ipynb

Sets up the environment by importing essential libraries for data manipulation, plotting, and machine learning models. It also configures environment variables for parallel processing and sets a plotting theme.

```python
import os
import warnings
import time

from pathlib import Path
from pprint import pprint

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

from rectools import Columns
from rectools.dataset import Dataset
from rectools.models import LightFMWrapperModel

from lightfm import LightFM

os.environ["OPENBLAS_NUM_THREADS"] = "1"
sns.set_theme(style="whitegrid")
```

--------------------------------

### Initialize CandidateRankingModel with LightGBM

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Initializes a two-stage LightGBM ranker model. This setup includes candidate generators, a splitter, a reranker, a sampler, and a feature collector.

```python
two_stage_lgbm_ranker = CandidateRankingModel(
        candidate_generators=first_stage_lgbm,
        splitter=splitter,
        reranker=LGBMReranker(LGBMRanker(random_state=RANDOM_STATE), fit_kwargs=fit_params),
        sampler=PerUserNegativeSampler(n_negatives=3, random_state=RANDOM_STATE),  # pass sampler to fix random_state
        feature_collector=CustomFeatureCollector(user_features_path=user_features_path, user_cat_cols=user_cat_cols)
        )
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Imports necessary libraries for data manipulation, deep learning, and RecTools functionalities. Sets up environment variables for deterministic CUDA behavior and suppresses warnings.

```python
import os
import itertools
import typing as tp
import warnings
from collections import Counter
from pathlib import Path

import pandas as pd
import numpy as np
import torch
from lightning_fabric import seed_everything
from pytorch_lightning import Trainer, LightningModule
from pytorch_lightning.loggers import CSVLogger
from pytorch_lightning.callbacks import EarlyStopping, ModelCheckpoint, Callback

from rectools import Columns, ExternalIds
from rectools.dataset import Dataset
from rectools.metrics import NDCG, Recall, Serendipity, calc_metrics
from rectools.models import BERT4RecModel, SASRecModel, load_model
from rectools.models.nn.item_net import IdEmbeddingsItemNet
from rectools.models.nn.transformers.base import TransformerModelBase

# Enable deterministic behaviour with CUDA >= 10.2
os.environ["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8"
warnings.simplefilter("ignore", UserWarning)
warnings.simplefilter("ignore", FutureWarning)
```

--------------------------------

### Install RecTools with Pip

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Basic installation command for RecTools using pip. For additional functionality, consider installing with specific extensions.

```bash
pip install rectools
```

--------------------------------

### Configure PureSVDModel using a dictionary

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Create a PureSVDModel instance using a configuration dictionary. This example specifies the number of factors for the SVD decomposition.

```python
config = {
    "factors": 32,
}

model = PureSVDModel.from_config(config)
model.get_params(simple_types=True)
```

--------------------------------

### Install RecTools Requirements

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Run this command to install all necessary requirements for RecTools. Ensure Python 3 and Poetry are installed and no virtual environments are active.

```bash
make install
```

--------------------------------

### Configure SASRecModel with basic parameters

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Use a dictionary to define configuration parameters for SASRecModel. This example shows basic settings like epochs, blocks, heads, and factors.

```python
config = {
    "epochs": 2,
    "n_blocks": 1,
    "n_heads": 1,
    "n_factors": 64, 
}

model = SASRecModel.from_config(config)
model.get_params(simple_types=True)
```

--------------------------------

### Install rectools visuals extension

Source: https://github.com/mtswebservices/rectools/blob/main/examples/8_debiased_metrics.ipynb

Install the 'visuals' extension for rectools to enable visualizations. Kaleido is also required for rendering widgets to PNG.

```python
pip install rectools[visuals]
pip install kaleido
```

--------------------------------

### Download and Unzip Dataset

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Downloads the KION dataset from a GitHub repository and unzips it. Cleans up the zip file afterwards. This is a prerequisite for running the training examples.

```bash
# %%time
!wget -q https://github.com/irsafilo/KION_DATASET/raw/f69775be31fa5779907cf0a92ddedb70037fb5ae/data_en.zip -O data_en.zip
!unzip -o data_en.zip
!rm data_en.zip
```

--------------------------------

### Install RecTools via Pip

Source: https://github.com/mtswebservices/rectools/blob/main/docs/source/index.rst

Install the RecTools library from PyPI using pip. Ensure you are using Python 3.9+.

```bash
$ pip install rectools
```

--------------------------------

### Make Recommendations with RecTools

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Example of how to generate recommendations using the RecTools model. Ensure the 'dataset' and 'ratings' variables are properly defined and the model is initialized.

```python
recos = model.recommend(
    users=ratings[Columns.User].unique(),
    dataset=dataset,
    k=10,
    filter_viewed=True,
)
```

--------------------------------

### Install RecTools with Specific Extension

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Command to install RecTools with a specific extension, such as 'lightfm' or 'torch'. Replace '[extension-name]' with the desired extension.

```bash
pip install rectools[extension-name]
```

--------------------------------

### Initialize and Train LightFM Model

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/baselines_extended_tutorial.ipynb

Instantiate and train a LightFM model with BPR loss. Ensure the 'rectools[lightfm]' extension is installed. This model is suitable for recommendation tasks and can handle cold-start users.

```python
model = LightFMWrapperModel(LightFM(no_components=10, loss="bpr", random_state=RANDOM_STATE))
model.fit(dataset);
```

--------------------------------

### Configure Trainer with Custom Callbacks

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Instantiate the PyTorch Lightning Trainer, configuring it with GPU acceleration, epoch limits, batch limits for testing, a logger, and the custom callbacks for metrics and checkpointing. This example limits training to 2 batches per epoch for a quick test run.

```python
trainer = Trainer(
    accelerator="gpu",
    devices=1,
    min_epochs=1,
    max_epochs=6,
    deterministic=True,
    limit_train_batches=2,  # use only 2 batches for each epoch for a test run
    logger = CSVLogger("test_logs"),
    callbacks=[val_metrics_callback, best_ndcg_ckpt],  # pass our callbacks
    enable_progress_bar=False,
    enable_model_summary=False,
)

# Replace trainer with our custom one
model._trainer = trainer

```

--------------------------------

### Initialize Two-Stage Candidate Ranking Model with GradientBoostingClassifier

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Construct a two-stage candidate ranking model using GradientBoostingClassifier as the reranker. This setup includes candidate generators, a splitter, the reranker, and a negative sampler.

```python
two_stage_gbc = CandidateRankingModel(
    candidate_generators=first_stage_gbc,
    splitter=splitter,
    reranker=Reranker(GradientBoostingClassifier(random_state=RANDOM_STATE)),
    sampler=PerUserNegativeSampler(n_negatives=3, random_state=RANDOM_STATE) # pass sampler to fix random_state
)
```

--------------------------------

### Instantiate and Train BERT4Rec Model

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_customization_guide.ipynb

Demonstrates how to instantiate a BERT4RecModel with custom data preparator and lightning module types, and then train it on a dataset. Includes example output from the training process.

```python
next_action_model = BERT4RecModel(
    data_preparator_type=NextActionDataPreparator,
    lightning_module_type=NextActionLightningModule,
    get_trainer_func = get_debug_trainer,
)

next_action_model.fit(dataset)
```

--------------------------------

### Initialize Candidate Ranking Model with CatBoostClassifier

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Construct a candidate ranking model using CatBoostClassifier as the reranker. This setup utilizes a CustomFeatureCollector for processing categorical features and specifies pool_kwargs for CatBoost.

```python
# To transfer CatBoostClassifier we use CatBoostReranker (for faster work with large amounts of data)
# You can also pass parameters in fit_kwargs and pool_kwargs in CatBoostReranker

two_stage_catboost_classifier = CandidateRankingModel(
    candidate_generators=first_stage_catboost,
    splitter=splitter,
    reranker=CatBoostReranker(CatBoostClassifier(verbose=False, random_state=RANDOM_STATE), pool_kwargs=pool_kwargs),
    sampler=PerUserNegativeSampler(n_negatives=3, random_state=RANDOM_STATE), # pass sampler to fix random_state
    feature_collector=CustomFeatureCollector(user_features_path=user_features_path, user_cat_cols=user_cat_cols),
)
```

--------------------------------

### Setup and Device Configuration for UniSRec

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/unisrec_tutorial.ipynb

Imports necessary libraries and configures the device (GPU or CPU) for PyTorch operations. Sets float32 matmul precision to 'high' if a CUDA-enabled GPU is available.

```python
import io
import time
import warnings
import zipfile
from pathlib import Path

import numpy as np
import pandas as pd
import requests
import torch
from tqdm import tqdm

from rectools.fast_transformers import UniSRecModel, compute_metrics
from rectools.fast_transformers.preprocessing import build_sequences

warnings.simplefilter("ignore")

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Device: {DEVICE}")
if DEVICE == "cuda":
    print(f"GPU: {torch.cuda.get_device_name(0)}")
    torch.set_float32_matmul_precision("high")
```

--------------------------------

### Get test fold borders

Source: https://github.com/mtswebservices/rectools/blob/main/examples/2_cross_validation.ipynb

Retrieves the start and end timestamps for each test fold generated by the `TimeRangeSplitter`. This helps in understanding the time intervals used for validation.

```python
splitter.get_test_fold_borders(dataset.interactions)
```

--------------------------------

### Construct a VisualApp for user recommendations

Source: https://github.com/mtswebservices/rectools/blob/main/examples/7_visualization.ipynb

Initializes a VisualApp instance with recommendation data, user interactions, item metadata, selected users for visualization, and custom formatters for displaying images.

```python
app = VisualApp.construct(
    reco=reco,
    interactions=interactions,
    item_data=item_data,
    selected_users={"detective user": 10, "action user": 20},  # users that we want to visualise
    formatters={"img_url": lambda x: f"<img src={x} width=100>"}  # process "img_url" links to html code
)
```

--------------------------------

### Import Libraries and Set Up Environment

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_tutorial.ipynb

Imports necessary libraries for data manipulation, modeling, and visualization. Sets up environment for deterministic behavior and random seeding.

```python
import numpy as np
import os
import pandas as pd
import torch
import typing as tp
import warnings
import threadpoolctl
import re
import plotly.express as px

from lightning_fabric import seed_everything
from pathlib import Path

from rectools import Columns
from rectools.dataset import Dataset
from rectools.metrics import (
    MAP,
    CoveredUsers,
    AvgRecPopularity,
    Intersection,
    HitRate,
    Serendipity,
)
from rectools.models import PopularModel, EASEModel, SASRecModel, BERT4RecModel
from rectools.model_selection import TimeRangeSplitter, cross_validate
from rectools.models.nn.item_net import CatFeaturesItemNet, IdEmbeddingsItemNet
from rectools.visuals import MetricsApp

warnings.simplefilter("ignore")

# Enable deterministic behaviour with CUDA >= 10.2
os.environ["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8"

# Random seed
RANDOM_STATE=60
torch.use_deterministic_algorithms(True)
seed_everything(RANDOM_STATE, workers=True)
```

--------------------------------

### Initialize PopularModel using from_config

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Demonstrates initializing a PopularModel from a dictionary of hyperparameters using the `from_config` class method.

```python
config = {
    "popularity": "n_interactions",
    "period": timedelta(weeks=2),
}
model = PopularModel.from_config(config)
```

--------------------------------

### Display Sample Recommendations

Source: https://github.com/mtswebservices/rectools/blob/main/examples/1_simple_example.ipynb

Shows the first few rows of the generated recommendations, sorted by relevance score for each user.

```python
# Sample of recommendations - it's sorted by relevance (= rank) for each user
recos.head()
```

--------------------------------

### Initialize CandidateRankingModel with CatBoostReranker

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Set up a two-stage ranking model using CatBoostReranker. This involves defining candidate generators, a splitter, the reranker itself, a sampler for negative instances, and a feature collector.

```python
two_stage_catboost_ranker = CandidateRankingModel(
    candidate_generators=first_stage_catboost,
    splitter=splitter,
    reranker=CatBoostReranker(CatBoostRanker(verbose=False, random_state=RANDOM_STATE), pool_kwargs=pool_kwargs),
    sampler=PerUserNegativeSampler(n_negatives=3, random_state=RANDOM_STATE), # pass sampler to fix random_state
    feature_collector=CustomFeatureCollector(user_features_path=user_features_path, user_cat_cols=user_cat_cols),
               
)
```

--------------------------------

### Get Recommendations

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Obtain recommendations for users, with options to filter out already viewed items or specify the number of items to recommend.

```python
model.recommend(dataset, filter_viewed=True, items_to_recommend=10)
```

--------------------------------

### Initialize Metrics with Debiasing Configuration

Source: https://github.com/mtswebservices/rectools/blob/main/examples/8_debiased_metrics.ipynb

Demonstrates how to initialize metrics with a debiasing configuration using `DebiasConfig`. This allows for calculating metrics on debiased test interactions without manual data manipulation.

```python
metrics = {
    'Serendipity': Serendipity(k=10),
    'MAP': MAP(k=10),
    'MAP_debiased': MAP(k=10, debias_config=DebiasConfig(random_state=32, iqr_coef=10)),  # debiased version of MAP
}
```

--------------------------------

### Get JSON-compatible model configuration

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Obtains a model configuration dictionary in a JSON-compatible format by setting `simple_types=True` in the `get_config` method.

```python
model.get_config(simple_types=True)
```

--------------------------------

### Get and Display Model Logs

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Retrieves training and validation logs for losses and metrics, then displays the head and tail of the metrics DataFrame.

```python
loss_df, metrics_df = get_logs(model)
pd.concat([metrics_df.head(5), metrics_df.tail(5)])
```

--------------------------------

### Create sample recommendations DataFrame

Source: https://github.com/mtswebservices/rectools/blob/main/examples/7_visualization.ipynb

Generates a pandas DataFrame containing recommendations for users, including item IDs, ranks, and model names.

```python
reco = pd.DataFrame({
    "user_id": [10, 10, 20, 20],
    "item_id": [4, 0, 1, 2],
    "rank": [1, 2, 1, 2],
    "model": ["Random model"] * 4
})
reco
```

--------------------------------

### Configure Trainer with Callbacks

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_HSTU_tutorial.ipynb

Sets up a PyTorch Lightning Trainer with common callbacks for sequential model training. Includes recall calculation, checkpointing based on best recall, early stopping, and loading the best model. Uses GPU acceleration and deterministic training.

```python
# Prepare trainer function for models

# We use callbacks for calculating recall on validation fold, making model checkpoint based on best recall, early stopping and best model load
# We train for maximum 100 epochs
# This is the most common academic training setup for sequential models

RECALL_K = 10
PATIENCE = 5
DIVERGENCE_TRESHOLD = 0.01
EPOCHS = 100
recall_callback = RecallCallback(k=RECALL_K, progress_bar=True)
# Checkpoints based on best recall
max_recall_ckpt = ModelCheckpoint(
    monitor=f"recall@{RECALL_K}",   # or just pass "val_loss" here,
    mode="max",
    filename="best_recall",
)
early_stopping_recall = EarlyStopping(
    monitor=f"recall@{RECALL_K}",
    mode="max",
    patience=PATIENCE,
    divergence_threshold=DIVERGENCE_TRESHOLD,
)
best_model_load = BestModelLoadCallback("best_recall")
callbacks = [recall_callback, max_recall_ckpt, best_model_load]

# Function to get custom trainer
def get_trainer() -> Trainer:
    return Trainer(
        accelerator="gpu",
        devices=1,
        min_epochs=10,
        max_epochs=EPOCHS,
        deterministic=True,
        enable_model_summary=False,
        enable_progress_bar=True,
        callbacks=callbacks,
        logger = CSVLogger("test_logs"),  # We use CSV logging for this guide but there are many other options
    )
```

--------------------------------

### Import necessary libraries for RecTools models

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Imports common libraries and various model classes from the rectools library for use in examples.

```python
from datetime import timedelta
import pandas as pd

from rectools.models import (
    SASRecModel,
    BERT4RecModel,
    ImplicitItemKNNWrapperModel, 
    ImplicitALSWrapperModel, 
    ImplicitBPRWrapperModel, 
    EASEModel,
    PopularInCategoryModel,
    PopularModel,
    RandomModel,
    LightFMWrapperModel,
    PureSVDModel,
    model_from_config,
    load_model,
    model_from_params
)
```

--------------------------------

### Import necessary libraries

Source: https://github.com/mtswebservices/rectools/blob/main/examples/2_cross_validation.ipynb

Imports all required modules from implicit, rectools, and other common data science libraries. Ensure these libraries are installed before running.

```python
from pprint import pprint

import numpy as np
import pandas as pd

from tqdm.auto import tqdm

from implicit.nearest_neighbours import TFIDFRecommender, BM25Recommender
from implicit.als import AlternatingLeastSquares

from rectools import Columns
from rectools.dataset import Dataset
from rectools.metrics import Precision, Recall, MeanInvUserFreq, Serendipity, calc_metrics
from rectools.models import ImplicitItemKNNWrapperModel, RandomModel, PopularModel
from rectools.model_selection import TimeRangeSplitter, cross_validate
from rectools.visuals import MetricsApp
```

--------------------------------

### Re-initialize model from its configuration

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Demonstrates creating a new model instance with identical hyperparameters by passing the output of `get_config` to `from_config`.

```python
source_config = model.get_config()
new_model = PopularModel.from_config(source_config)
```

--------------------------------

### Initialize Trainer for Cross-Validation

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_customization_guide.ipynb

Defines a trainer function with specific configurations for GPU usage, epochs, and deterministic behavior, suitable for cross-validation setups.

```python
def get_trainer() -> Trainer:
    return Trainer(
        accelerator="gpu",
        devices=[1],
        min_epochs=3,
        max_epochs=3,
        deterministic=True,
        enable_model_summary=False,
        enable_progress_bar=False,
    )
```

--------------------------------

### Configure PopularInCategoryModel with flat parameters

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Instantiate PopularInCategoryModel using a flat dictionary of parameters, providing an alternative to the nested configuration.

```python
params = {  # flat form
    "popularity": "n_interactions",
    "period.days": 1,
    "category_feature": "genres",
    "mixing_strategy": "group"
}

model = PopularInCategoryModel.from_params(params)
model.get_params(simple_types=True)
```

--------------------------------

### Get model hyperparameters as a flat dictionary

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Retrieves model hyperparameters as a flat dictionary, suitable for experiment trackers, using `get_params` with `simple_types=True`.

```python
model.get_params(simple_types=True)
```

--------------------------------

### Configure LightFMWrapperModel with flat parameters

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Instantiate LightFMWrapperModel using a flat dictionary of parameters. This is an alternative to the nested configuration dictionary.

```python
params = {  # flat form
    "model.no_components": 16,  
    "model.learning_rate": 0.03,
    "model.random_state": 32,
    "model.loss": "warp",
    "epochs": 2,
}

model = LightFMWrapperModel.from_params(params)
model.get_params(simple_types=True)
```

--------------------------------

### Get full model configuration with get_config

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Retrieves a dictionary containing all model hyperparameters, including default values, using the `get_config` method.

```python
model.get_config()
```

--------------------------------

### Load Model from Parameters

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Instantiate a recommender model using its parameters. This is an alternative to loading from a full configuration.

```python
model = model_from_params(params)
```

--------------------------------

### Construct MetricsApp for Interactive Analysis

Source: https://github.com/mtswebservices/rectools/blob/main/examples/2_cross_validation.ipynb

Constructs an interactive MetricsApp using cross-validation results and optional model metadata. Requires the 'visuals' extension to be installed.

```python
metadata_example = {
    Columns.Model: ["random", "popular", "most_rated", "tfidf_k=5", "tfidf_k=10", "bm25_k=10_k1=0.05_b=0.1"],
    "k": [None, None, None, 5, 10, 10]
}

app = MetricsApp.construct(
    models_metrics=pd.DataFrame(cv_results["metrics"]),
    models_metadata=pd.DataFrame(metadata_example),  # optional
)
```

--------------------------------

### Instantiate Recommendation Metrics

Source: https://github.com/mtswebservices/rectools/blob/main/examples/3_metrics.ipynb

Create metric objects by specifying parameters like 'k' for the number of recommendations. Some metrics accept additional parameters such as 'r_precision' or 'log_base'.

```python
serendipity = Serendipity(k=10)
precision = Precision(k=10, r_precision=True)  # r_precision means division by min(k, n_user_test_items)
ndcg = NDCG(k=10, log_base=3)
```

--------------------------------

### Initialize Dataset and Random Seed

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_HSTU_tutorial.ipynb

Sets up the random seed for reproducibility and loads the MovieLens dataset for model training and evaluation.

```python
seed_everything(RANDOM_STATE, workers=True)
dataset_name = "ml-1m"
pivot_name = f"pivot_results_ablation_{dataset_name}.json"
ml_df = get_movielens_df(dataset_name)
dataset = Dataset.construct(ml_df)
```

--------------------------------

### Cross-validation loop setup

Source: https://github.com/mtswebservices/rectools/blob/main/examples/2_cross_validation.ipynb

This code block initiates the cross-validation process. It is intended to loop through each fold generated by the splitter, preparing training and testing datasets for each iteration.

```python
%%time

# For each fold generate train and test part of dataset

```

--------------------------------

### Prepare Evaluation Metrics

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_HSTU_tutorial.ipynb

Initializes dictionaries for various recommendation system metrics like Recall, NDCG, Catalog Coverage, and Serendipity. These metrics are used to evaluate model performance.

```python
metrics_add = {}
metrics_recall ={}
metrics_ndcg = {}
k_base =  10
K = [10, 50,100,200]
K_RECS= max(K)
for k in K:
    metrics_recall.update({
            f"recall@{k}": Recall(k=k),
        })
    metrics_ndcg.update({
            f"ndcg@{k}": NDCG(k=k, divide_by_achievable=True),
        })
metrics_add = {
    f"arp@{k_base}": AvgRecPopularity(k=k_base, normalize=True),
    f"coverage@{k_base}": CatalogCoverage(k=k_base, normalize=True),
    f"covered_users@{k_base}": CoveredUsers(k=k_base),
    f"sufficient_reco@{k_base}": SufficientReco(k=k_base),
    f"serendipity@{k_base}": Serendipity(k=k_base),
}
metrics  = metrics_recall | metrics_ndcg | metrics_add
metrics_to_show = ['recall@10', 'ndcg@10', 'recall@50', 'ndcg@50', 'recall@200', 'ndcg@200', 'coverage@10',
                       'serendipity@10']
```

--------------------------------

### Download and Unzip Kion Dataset

Source: https://github.com/mtswebservices/rectools/blob/main/examples/5_benchmark_iALS_with_features.ipynb

Downloads the Kion dataset from a GitHub repository and unzips it. This is a preliminary step for data loading and preprocessing.

```bash
%%time
!wget -q https://github.com/irsafilo/KION_DATASET/raw/f69775be31fa5779907cf0a92ddedb70037fb5ae/data_original.zip -O data_original.zip
!unzip -o data_original.zip
!rm data_original.zip
```

--------------------------------

### Configure PopularInCategoryModel with a dictionary

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Set up PopularInCategoryModel with a configuration dictionary, including parameters for popularity, period, category features, and mixing strategy.

```python
config = {
    "popularity": "n_interactions",
    "period": timedelta(days=1),
    "category_feature": "genres",
    "mixing_strategy": "group"
}

model = PopularInCategoryModel.from_config(config)
model.get_params(simple_types=True)
```

--------------------------------

### Initialize Two-Stage LGBM Classifier

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Construct a two-stage candidate ranking model using LGBMClassifier as the reranker. This setup includes candidate generators, a splitter, a reranker, a sampler, and a feature collector.

```python
if LGBM_AVAILABLE:
    two_stage_lgbm_classifier = CandidateRankingModel(
        candidate_generators=first_stage_lgbm,
        splitter=splitter,
        reranker=Reranker(LGBMClassifier(random_state=RANDOM_STATE), fit_params),
        sampler=PerUserNegativeSampler(n_negatives=3, random_state=RANDOM_STATE),  # pass sampler to fix random_state
        feature_collector=CustomFeatureCollector(user_features_path=user_features_path, user_cat_cols=user_cat_cols)
    )
```

--------------------------------

### Initialize CandidateRankingModel

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Initializes the CandidateRankingModel with a first-stage model, a splitter, and a reranker. A default negative sampler is used if none is provided.

```python
# Initialize CandidateRankingModel
# We can also pass negative sampler but here we are just using the default one

two_stage = CandidateRankingModel(first_stage, splitter, reranker)
```

--------------------------------

### Initialize Callbacks for Metrics and Checkpointing

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Set up the metrics dictionary, determine the maximum k value, and initialize the custom validation metrics callback and a model checkpoint callback that monitors NDCG@10.

```python
# Initialize callbacks for metrics calculation and checkpoint based on NDCG value

metrics = {
    "NDCG@10": NDCG(k=10),
    "Recall@10": Recall(k=10),
    "Serendipity@10": Serendipity(k=10),
}
top_k = max([metric.k for metric in metrics.values()])

# Callback for calculating RecSys metrics
val_metrics_callback = ValidationMetrics(top_k=top_k, val_metrics=metrics, verbose=0)

# Callback for checkpoint based on maximization of NDCG@10
best_ndcg_ckpt = ModelCheckpoint(
    monitor="NDCG@10",
    mode="max",
    filename="{epoch}-{NDCG@10:.2f}",
)

```

--------------------------------

### Import Libraries and Set Up Environment

Source: https://github.com/mtswebservices/rectools/blob/main/examples/5_benchmark_iALS_with_features.ipynb

Imports necessary libraries for data manipulation, modeling, and visualization. Sets up environment variables and thread pool limits for the implicit ALS library.

```python
import os
import threadpoolctl
import warnings
from pathlib import Path

import pandas as pd
import numpy as np
import seaborn as sns
from matplotlib import pyplot as plt

from rectools.metrics import MAP, calc_metrics, MeanInvUserFreq, Serendipity
from rectools.models import ImplicitALSWrapperModel
from rectools import Columns
from rectools.dataset import Dataset
from implicit.als import AlternatingLeastSquares

warnings.filterwarnings('ignore')
sns.set_theme(style="whitegrid")

# For implicit ALS
os.environ["OPENBLAS_NUM_THREADS"] = "1"
threadpoolctl.threadpool_limits(1, "blas")
```

--------------------------------

### Prepare Features for Diversity Metric

Source: https://github.com/mtswebservices/rectools/blob/main/examples/3_metrics.ipynb

To calculate diversity metrics like IntraListDiversity, you need to define a distance calculator. This example prepares movie genres as features using pandas for dummy encoding.

```python
movies["genre"] = movies["genres"].str.split("|")
genre_exploded = movies[["item_id", "genre"]].set_index("item_id").explode("genre")
genre_dummies = pd.get_dummies(genre_exploded, prefix="", prefix_sep="").groupby("item_id").sum()
genre_dummies.head()
```

--------------------------------

### Load Transformer Model and Recommend

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Loads a previously saved transformer model from a file and uses it to generate recommendations. Ensure custom functions/classes used during initialization are available.

```python
loaded = load_model("my_model.pkl")
print(type(loaded))
loaded.recommend(users=VAL_USERS[:1], dataset=dataset, filter_viewed=True, k=5)
```

--------------------------------

### Download and Unzip KION Dataset

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/baselines_extended_tutorial.ipynb

Downloads the KION dataset and extracts it. This data is used for the recommender systems tutorial.

```python
%%time
!wget -q https://github.com/irsafilo/KION_DATASET/raw/f69775be31fa5779907cf0a92ddedb70037fb5ae/data_en.zip -O data_en.zip
!unzip -o data_en.zip
!rm data_en.zip
```

--------------------------------

### Using CatBoostRanker

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Demonstrates the flexibility of the CandidateRankingModel by showing that CatBoostRanker can be used interchangeably with CatBoostClassifier without requiring additional modifications.

```python
# Using CatBoostRanker
# Instead of `CatBoostClassifier` you can also easily use `CatBoostRanker` without any additional modifications
```

--------------------------------

### Download and Unzip Movielens 20m Dataset

Source: https://github.com/mtswebservices/rectools/blob/main/examples/6_benchmark_lightfm_inference.ipynb

Downloads the Movielens 20m dataset and unzips it. This is a time-consuming step and is marked with %%time to show its execution duration.

```bash
!wget -q https://files.grouplens.org/datasets/movielens/ml-20m.zip -O ml-20m.zip
!unzip -o ml-20m.zip
!rm ml-20m.zip
```

--------------------------------

### Train and Infer Recommendations with RecTools

Source: https://github.com/mtswebservices/rectools/blob/main/docs/source/index.rst

This Python script demonstrates how to read data, construct a dataset, fit an ImplicitItemKNNWrapperModel, and generate recommendations using RecTools.

```python
import pandas as pd
from implicit.nearest_neighbours import TFIDFRecommender

from rectools import Columns
from rectools.dataset import Dataset
from rectools.models import ImplicitItemKNNWrapperModel

# Read the data
ratings = pd.read_csv(
    "ml-1m/ratings.dat",
    sep=":::",
    engine="python",  # Because of 2-chars separators
    header=None,
    names=[Columns.User, Columns.Item, Columns.Weight, Columns.Datetime],
)

# Create dataset
dataset = Dataset.construct(ratings)

# Fit model
model = ImplicitItemKNNWrapperModel(TFIDFRecommender(K=10))
model.fit(dataset)

# Make recommendations
recos = model.recommend(
    users=ratings[Columns.User].unique(),
    dataset=dataset,
    k=10,
    filter_viewed=True,
)
```

--------------------------------

### Initialize first-stage candidate generators

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Prepares a list of `CandidateGenerator` models for the first stage of the ranking pipeline. These models generate initial candidates using 'PopularModel' and 'ImplicitItemKNNWrapperModel' with CosineRecommender, keeping ranks and scores.

```python
# Prepare first stage models. They will be used to generate candidates for reranking
first_stage = [
    CandidateGenerator(PopularModel(), num_candidates=30, keep_ranks=True, keep_scores=True), 
    CandidateGenerator(
        ImplicitItemKNNWrapperModel(CosineRecommender()), 
        num_candidates=30, 
        keep_ranks=True, 
        keep_scores=True
    )
]
```

--------------------------------

### Configure ImplicitItemKNNWrapperModel with flat parameters

Source: https://github.com/mtswebservices/rectools/blob/main/examples/9_model_configs_and_saving.ipynb

Instantiate ImplicitItemKNNWrapperModel using a flat dictionary of parameters. This format is useful for providing configuration values directly.

```python
params = {  # flat form
    "model.cls": "TFIDFRecommender",  
    "model.K": 50,
    "model.num_threads": 1,
}
model = ImplicitItemKNNWrapperModel.from_params(params)
model.get_params(simple_types=True)
```

--------------------------------

### Initialize Metrics Trade-off Analysis App

Source: https://github.com/mtswebservices/rectools/blob/main/examples/8_debiased_metrics.ipynb

Constructs and displays an interactive widget for analyzing trade-offs between different metrics. This app helps in understanding the performance of models across various evaluation criteria.

```python
# When you run this notebook, the output of this cell will have an interactive widget for metrics trade-off analysis

app = MetricsApp.construct(models_metrics, models_meta)
```

--------------------------------

### Initialize CatBoostReranker

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Initializes a CatBoostReranker with a specified number of estimators and random state. Set verbose to False to suppress training output.

```python
reranker = CatBoostReranker(model=CatBoostClassifier(n_estimators=100, verbose=False, random_state=RANDOM_STATE))
```

--------------------------------

### Load and Prepare MovieLens 20M Dataset

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_HSTU_tutorial.ipynb

Sets a random seed, loads the MovieLens 20M dataset, and constructs a Dataset object for model training and evaluation.

```python
seed_everything(RANDOM_STATE, workers=True)
dataset_name = "ml-20m"
pivot_name = f"pivot_results_{dataset_name}.json"
ml_df = get_movielens_df(dataset_name)
dataset = Dataset.construct(ml_df)
```

--------------------------------

### Prepare Test User with One Interaction

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_tutorial.ipynb

Selects a user with a single interaction to test recommendation handling for users with minimal history. Prints the shape and details of the user's interactions.

```python
# Prepare test user with 1 interaction
test_user_one = 324373
print(interactions[interactions["user_id"] == test_user_one].shape)
interactions[interactions["user_id"] == test_user_one]
```

--------------------------------

### Train Recommender Models

Source: https://github.com/mtswebservices/rectools/blob/main/examples/8_debiased_metrics.ipynb

Constructs a Dataset object and then fits each model in the 'models' dictionary to the dataset. This step is crucial for training the recommender systems.

```python
# %%time
dataset = Dataset.construct(
    interactions_df=train,
)
for model in models.values():
    model.fit(dataset)
```

--------------------------------

### Prepare Cold Test Users

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/baselines_extended_tutorial.ipynb

Select users with no features or interactions to test model behavior with entirely new users.

```python
# Prepare cold test users
test_cold_users = [99999999]  # don't have features or interactions
interactions[interactions["user_id"] == test_cold_users[0]].shape
```

```python
user_features[user_features["id"] == test_cold_users[0]].shape
```

--------------------------------

### Initialize and Fit PureSVD Model

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/baselines_extended_tutorial.ipynb

Initializes a PureSVD model with a specified number of factors and fits it to the dataset. Model parameters like factors are set during initialization.

```python
%%time
model = PureSVDModel(factors=10)
model.fit(dataset);
```

--------------------------------

### Download and Unzip Data

Source: https://github.com/mtswebservices/rectools/blob/main/docs/source/index.rst

Use wget to download the MovieLens 1M dataset and unzip it for use with RecTools.

```bash
$ wget https://files.grouplens.org/datasets/movielens/ml-1m.zip
$ unzip ml-1m.zip
```

--------------------------------

### Construct Dataset

Source: https://github.com/mtswebservices/rectools/blob/main/README.md

Prepare data for recommender models by constructing a Dataset object from a pandas DataFrame. No manual creation of sparse matrices or ID mapping is required.

```python
dataset = Dataset.construct(interactions_df)
```

--------------------------------

### Create Trainer with Checkpoint Callbacks

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Instantiate a PyTorch Lightning Trainer, passing a list containing both last epoch and best validation loss checkpoint callbacks.

```python
trainer = Trainer(
    accelerator="gpu",
    devices=1,
    min_epochs=1,
    max_epochs=6,
    deterministic=True,
    limit_train_batches=2,  # use only 2 batches for each epoch for a test run
    logger = CSVLogger("test_logs"),
    callbacks=[last_epoch_ckpt, least_val_loss_ckpt],  # pass our callbacks for checkpoints
    enable_progress_bar=False,
    enable_model_summary=False,
)

# Replace trainer with our custom one
model._trainer = trainer
```

--------------------------------

### Prepare Context for Recommendations

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_HSTU_tutorial.ipynb

Creates a pandas DataFrame with user IDs and query timestamps to be used as context for context-aware recommendation models. The 'Columns' object is assumed to provide predefined column names.

```python
from rectools.dataset.context import  get_context

users = [1,2,3] # users we are recommending for
query_time = max(ml_df[Columns.Datetime]) # for example
context_df = pd.DataFrame(
    {
        Columns.User: [1, 2, 3],
        Columns.Datetime: [query_time]*3,
    }
)
context = get_context(context_df) # context preprocessing. You can also just pass full test interactions df here if you have it
```

--------------------------------

### Fit Model and Generate Recommendations

Source: https://github.com/mtswebservices/rectools/blob/main/examples/1_simple_example.ipynb

Initializes and fits an ImplicitItemKNNWrapperModel with TFIDFRecommender. It then generates recommendations for all unique users in the dataset.

```python
%%time
# Fit model and generate recommendations for all users
model = ImplicitItemKNNWrapperModel(TFIDFRecommender(K=10))
model.fit(dataset)
recos = model.recommend(
    users=ratings[Columns.User].unique(),
    dataset=dataset,
    k=10,
    filter_viewed=True,
)
```

--------------------------------

### Instantiate IntraListDiversity Metric

Source: https://github.com/mtswebservices/rectools/blob/main/examples/3_metrics.ipynb

Initialize the IntraListDiversity metric by providing the number of recommendations 'k' and a pre-configured distance calculator, such as one based on Hamming distance of genre dummies.

```python
distance_calculator = PairwiseHammingDistanceCalculator(genre_dummies)
ild = IntraListDiversity(k=10, distance_calculator=distance_calculator)
```

--------------------------------

### Initialize First Stage Models for GradientBoostingClassifier

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/candidate_ranking_model_tutorial.ipynb

Prepare candidate generators for the first stage of the ranking model. When using GradientBoostingClassifier, it's necessary to fill in empty scores and ranks with specified values.

```python
first_stage_gbc = [
    CandidateGenerator(
        model=PopularModel(),
        num_candidates=30,
        keep_ranks=True,
        keep_scores=True,
        scores_fillna_value=1.01, # when working with the GradientBoostingClassifier, you need to fill in the empty scores (e.g. max score)
        ranks_fillna_value=31  # when working with the GradientBoostingClassifier, you need to fill in the empty ranks (e.g. min rank)
    ),
    CandidateGenerator(
        model=ImplicitItemKNNWrapperModel(CosineRecommender()),
        num_candidates=30,
        keep_ranks=True,
        keep_scores=True,
        scores_fillna_value=1.01, # when working with the GradientBoostingClassifier, you need to fill in the empty scores (e.g. max score)
        ranks_fillna_value=31  # when working with the GradientBoostingClassifier, you need to fill in the empty ranks (e.g. min rank)
    )
]
```

--------------------------------

### Create Dataset with Features

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/baselines_extended_tutorial.ipynb

Construct a dataset incorporating both user and item features for use in recommendation models.

```python
# Create dataset with both user and item features
dataset = Dataset.construct(
    interactions_df=interactions,
    user_features_df=user_features,
    cat_user_features=["sex", "age"],
    item_features_df=item_features,
    cat_item_features=["genre"],
)

RANDOM_STATE=60
```

--------------------------------

### Fit EASE Model

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/baselines_extended_tutorial.ipynb

Initializes and fits the EASE model. Requires a dataset and regularization parameter. Be aware of potential out-of-memory issues with large datasets.

```python
%%time
model = EASEModel(regularization=500)
model.fit(dataset);
```

--------------------------------

### Load Model and Weights from Checkpoint

Source: https://github.com/mtswebservices/rectools/blob/main/examples/tutorials/transformers_advanced_training_guide.ipynb

Load both the model architecture and its weights from a checkpoint file using a class method. Note that the loaded model will not have a 'fit_trainer' and cannot be saved again, but it is ready for recommendations.

```python
ckpt_path = os.path.join(model.fit_trainer.log_dir, "checkpoints", "last_epoch.ckpt")
loaded = SASRecModel.load_from_checkpoint(ckpt_path)
loaded.recommend(users=VAL_USERS[:1], dataset=dataset, filter_viewed=True, k=5)
```

--------------------------------

### Prepare Popular Model and Combine Models

Source: https://github.com/mtswebservices/rectools/blob/main/examples/8_debiased_metrics.ipynb

Initializes a PopularModel and combines it with previously defined ALS and BM25 models into a single dictionary. This prepares all models for training and evaluation.

```python
# Prepare popular model

from datetime import timedelta
models = {"popular": PopularModel(period=timedelta(days=14))}
models.update(als_models)
models.update(bm25_models)
models
```