### Install Recommenders with Examples

Source: https://github.com/recommenders-team/recommenders/blob/main/recommenders/README.md

Installs the recommenders package along with dependencies needed for running example notebooks.

```bash
pip install recommenders[examples]
```

--------------------------------

### Install Recommenders with Examples and GPU Support

Source: https://github.com/recommenders-team/recommenders/blob/main/recommenders/README.md

Installs the recommenders package with dependencies for both example notebooks and GPU functionality.

```bash
pip install recommenders[examples,gpu]
```

--------------------------------

### BiVAE Model Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

This snippet shows the basic setup and initialization of the BiVAE model. It requires the cornac library to be installed.

```python
from cornac.models import BiVAE

# Initialize the BiVAE model
model = BiVAE(k=10, max_iter=50, batch_size=100, learning_rate=0.001, 
            num_neg=1, early_stop=5, lambda_elbo=0.001, lambda_kl=0.001, 
            lambda_reg=0.01, verbose=True)
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/rlrmc_movielens.ipynb

Imports necessary libraries and sets up the environment for the RLRMC example. Includes version checks for Pandas and Python.

```python
import sys
import time
import pandas as pd

from recommenders.datasets.python_splitters import python_random_split
from recommenders.datasets.python_splitters import python_stratified_split
from recommenders.datasets import movielens
from recommenders.models.rlrmc.RLRMCdataset import RLRMCdataset 
from recommenders.models.rlrmc.RLRMCalgorithm import RLRMCalgorithm 
from recommenders.evaluation.python_evaluation import rmse, mae
from recommenders.utils.notebook_utils import store_metadata

print(f"Pandas version: {pd.__version__}")
print(f"System version: {sys.version}")

%load_ext autoreload
%autoreload 2
```

--------------------------------

### Install All Packages

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Use this command to install the staging branch with all available packages.

```bash
git checkout staging
pip install -e .[all]
```

--------------------------------

### Basic Recommenders Installation

Source: https://github.com/recommenders-team/recommenders/blob/main/recommenders/README.md

Installs the core utilities, CPU-based algorithms, and their dependencies for the recommenders package.

```bash
pip install --upgrade pip setuptools
pip install recommenders
```

--------------------------------

### BiVAE Model Setup and Training

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Demonstrates how to initialize and train the BiVAE model using Cornac. Ensure you have Cornac installed and the necessary datasets prepared.

```python
from cornac.models import BiVAE
from cornac.experiment import Experiment
from cornac.metrics import Precision, Recall, NDCG, MAP

# Initialize the BiVAE model
model = BiVAE(k=100, max_iter=50, batch_size=100, learning_rate=0.001, 
              num_threads=1, seed=123, verbose=True, 
              # BiVAE specific parameters
              num_factors=100, layer_sizes=[100, 100], 
              # Training parameters
              early_stop=True, patience=3, tolerance=0.001, 
              # Evaluation parameters
              eval_batch_size=1000, 
              # Device parameters
              gpu=False)

# Define metrics for evaluation
metrics = [
    Precision(k=10),
    Recall(k=10),
    NDCG(k=10),
    MAP(k=10),
]

# Setup experiment
exp = Experiment(model=model, dataset=None, metrics=metrics)

# Train the model and evaluate
exp.run()

# Print results
print(exp.result)
```

--------------------------------

### Install Recommenders with GPU Support

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Use this command to install the recommenders package with GPU capabilities. Ensure CUDA is installed beforehand.

```bash
pip install recommenders[gpu]
```

--------------------------------

### Install Recommenders from Local Copy

Source: https://github.com/recommenders-team/recommenders/blob/main/recommenders/README.md

Install the recommenders package from a local clone of the source code. This requires an environment to be set up beforehand.

```bash
pip install -e .
```

--------------------------------

### BiVAE Recommendation Example

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Illustrates how to get top N recommendations for a given user using the trained BiVAE model.

```python
# Get top 10 recommendations for user 0
recommendations = model.recommend(user_id=0, n=10)
print(recommendations)

```

--------------------------------

### Install Recommenders with Spark Support

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Use this command to install the recommenders package with Spark capabilities. Ensure JDK is installed beforehand.

```bash
pip install recommenders[spark]
```

--------------------------------

### Install dependencies from requirements file

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/07_tutorials/KDD2020-tutorial/README.md

Installs all project dependencies listed in 'requirements_kdd.txt' using uv within the activated virtual environment.

```bash
uv pip install -r requirements_kdd.txt
```

--------------------------------

### Install Recommenders on Databricks

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Install the recommenders package with example dependencies on a Databricks cluster. Additional packages like numpy and scipy may also be required.

```bash
Install "recommenders[examples]" as a PyPI package.
Repeat for:
  a. numpy<2.0.0
  b. scipy<=1.13.1
```

--------------------------------

### Install Recommenders from GitHub

Source: https://github.com/recommenders-team/recommenders/blob/main/recommenders/README.md

Install the recommenders package directly from a GitHub repository. This can be from the main branch or a specific branch.

```bash
pip install -e git+https://github.com/microsoft/recommenders/#egg=pkg
```

```bash
pip install -e git+https://github.com/microsoft/recommenders/@staging#egg=pkg
```

--------------------------------

### BiVAE Training and Evaluation Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Sets up the BiVAE model, optimizer, and data loaders for training and evaluation.

```python
import cornac
from cornac.models import BaseModel
from cornac.data import Reader
from cornac.evaluator import Evaluator
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

# Load data
reader = Reader()
train_data = reader.read('/Users/soeren/.cornac/data/ml-100k/ml-100k.train.rating')
test_data = reader.read('/Users/soeren/.cornac/data/ml-100k/ml-100k.test.rating')

# Initialize model parameters
n_items = 1682
n_users = 943
latent_dim = 50
hidden_dims = [256, 128]
dropout_rate = 0.5
learning_rate = 0.001
batch_size = 100
epochs = 100

# Create model and optimizer
model = BiVAE(BiVAEAutoEncoder(n_items, n_users, latent_dim, hidden_dims, dropout_rate), 
              n_items, n_users, item_ids=None, user_ids=None)
optimizer = optim.Adam(model.parameters(), lr=learning_rate)

# Prepare data for PyTorch
def prepare_data(data, n_items, n_users):
    user_map = {user_id: i for i, user_id in enumerate(sorted(list(set(r[0] for r in data))))}
    item_map = {item_id: i for i, item_id in enumerate(sorted(list(set(r[1] for r in data))))}
    
    X = torch.zeros(n_users, n_items)
    for user_id, item_id, rating in data:
        X[user_map[user_id], item_map[item_id]] = 1
    return X

train_X = prepare_data(train_data, n_items, n_users)
test_X = prepare_data(test_data, n_items, n_users)

train_dataset = TensorDataset(train_X)
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

test_dataset = TensorDataset(test_X)
test_loader = DataLoader(test_dataset, batch_size=batch_size)

# Train the model
model.fit(train_loader, epochs=epochs, lr=learning_rate, batch_size=batch_size)

# Evaluate the model
results = model.score(test_loader)
print(results)

```

--------------------------------

### Install Recommenders package

Source: https://github.com/recommenders-team/recommenders/blob/main/README.md

Installs the core Recommenders package, enabling the use of CPU-based notebooks.

```bash
# 4. Install the core recommenders package. It can run all the CPU notebooks.
uv pip install recommenders
```

--------------------------------

### Install GCC on Ubuntu

Source: https://github.com/recommenders-team/recommenders/blob/main/README.md

Installs the GCC compiler, which may be a prerequisite for some installations.

```bash
# 1. Install gcc if it is not installed already. On Ubuntu, this could done by using the command
# sudo apt install gcc
```

--------------------------------

### Install Development and GPU Packages

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Use this command to install the staging branch with development and GPU support packages.

```bash
git checkout staging
pip install -e .[dev,gpu]
```

--------------------------------

### BiVAE Model Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Sets up the BiVAE model with specified parameters for training.

```python
from recommenders.models.newsrec.BiVAE import BiVAE

# Model parameters
args = {
    "batch_size": 128,
    "epochs": 10,
    "latent_dim": 128,
    "learning_rate": 0.001,
    "beta": 0.1,
    "dropout_rate": 0.5,
    "data_path": "/path/to/your/data",
    "model_path": "/path/to/save/model"
}

# Initialize BiVAE model
model = BiVAE(args)
model.fit(train_data, valid_data)
```

--------------------------------

### Install Recommenders with Optional Dependencies

Source: https://github.com/recommenders-team/recommenders/blob/main/CONTRIBUTING.md

Install the Recommenders package locally with specific optional dependencies for testing, such as GPU support, and the development option.

```bash
pip install -e .[gpu,dev]
```

--------------------------------

### Configure and Start Evolution Tuner

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/04_model_select_and_optimize/nni_surprise_svd.ipynb

Sets the tuner to 'Evolution', writes the configuration to a YAML file, and starts the NNI experiment.

```python
config['tuner']['builtinTunerName'] = 'Evolution'
with open(config_path, 'w') as fp:
    fp.write(yaml.dump(config, default_flow_style=False))
```

```python
stop_nni()
with Timer() as time_evolution:
    start_nni(config_path, wait=WAITING_TIME, max_retries=MAX_RETRIES)
```

--------------------------------

### Configure and Check SMAC Tuner Installation

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/04_model_select_and_optimize/nni_surprise_svd.ipynb

Sets the tuner to 'SMAC', writes the configuration, and checks if SMAC is installed, installing it if necessary.

```python
# SMAC
config['tuner']['builtinTunerName'] = 'SMAC'
with open(config_path, 'w') as fp:
    fp.write(yaml.dump(config, default_flow_style=False))
```

```python
# Check if installed
proc = subprocess.run([sys.prefix + '/bin/nnictl', 'package', 'show'], stdout=subprocess.PIPE)
if proc.returncode != 0:
    raise RuntimeError("'nnictl package show' failed with code %d" % proc.returncode)
if 'SMAC' not in proc.stdout.decode().strip().split():
    proc = subprocess.run([sys.prefix + '/bin/nnictl', 'package', 'install', '--name=SMAC'])
    if proc.returncode != 0:
        raise RuntimeError("'nnictl package install' failed with code %d" % proc.returncode)
```

--------------------------------

### Start Spark Session

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/05_operationalize/als_movie_o16n.ipynb

Starts a Spark session, downloading the Cosmos DB connector if not running on Databricks.

```python
if not is_databricks():
    cosmos_connector = (
        "https://search.maven.org/remotecontent?filepath=com/microsoft/azure/"
        "azure-cosmosdb-spark_2.3.0_2.11/1.3.3/azure-cosmosdb-spark_2.3.0_2.11-1.3.3-uber.jar"
    )
    jar_filepath = maybe_download(url=cosmos_connector, filename="cosmos.jar")
    spark = start_or_get_spark("ALS", memory="10g", jars=[jar_filepath])
    sc = spark.sparkContext
print(sc)
```

--------------------------------

### Temporary Directory Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_content_based_filtering/dkn_deep_dive.ipynb

Creates a temporary directory for storing downloaded and processed data.

```python
# Temp dir
tmpdir = TemporaryDirectory()
```

--------------------------------

### Install ipykernel and create Jupyter kernel

Source: https://github.com/recommenders-team/recommenders/blob/main/README.md

Installs ipykernel and registers a Jupyter kernel named 'recommenders' for the current environment.

```bash
# 5. Create a Jupyter kernel
uv pip install ipykernel
python -m ipykernel install --user --name recommenders --display-name "Python (recommenders)"
```

--------------------------------

### Install dependencies manually

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/07_tutorials/KDD2020-tutorial/README.md

Installs common dependencies manually using uv if 'requirements_kdd.txt' is not available. Includes libraries like numpy, pandas, tensorflow, etc.

```bash
uv pip install numpy pandas jupyter ipykernel scikit-learn matplotlib scipy pytest numba tensorflow
```

--------------------------------

### BiVAE Training Example

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Demonstrates how to train a BiVAE model using a dataset. This involves initializing the model and calling the fit method.

```python
# Assuming 'train_data' and 'val_data' are loaded and preprocessed
# Assuming 'autoencoder_model' is a pre-defined autoencoder architecture

# Initialize BiVAE model
bivae = BiVAE(autoencoder=autoencoder_model, beta=0.5, learning_rate=0.001, n_epochs=50)

# Train the model
bivae.fit(train_data, val_data=val_data)
```

--------------------------------

### Install Recommenders Library

Source: https://github.com/recommenders-team/recommenders/blob/main/docs/intro.md

Install the Recommenders library using pip. This is the primary method for adding the library to your Python environment.

```shell
pip install recommenders
```

--------------------------------

### BiVAE Training Example

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Demonstrates how to train a BiVAE model using a dataset. This involves instantiating the model and calling the fit method.

```python
# Assuming 'train_data' and 'test_data' are already loaded Cornac datasets
# and 'BiVAE' and 'BiVAEAutoencoder' classes are defined as above.

# Instantiate the BiVAEAutoencoder
# Adjust latent_dim, hidden_dims, etc. as needed
bi_vae_autoencoder = BiVAEAutoencoder(n_users=train_data.num_users, n_items=train_data.num_items, latent_dim=50, hidden_dims=[128, 64])

# Instantiate the BiVAE model
bi_vae = BiVAE(auto_encoder=bi_vae_autoencoder, 
               n_epochs=500, 
               lr=0.001, 
               batch_size=256, 
               validation_split=0.1, 
               early_stop_patience=50, 
               verbose=True, 
               seed=42)

# Train the model
bi_vae.fit(train_set=train_data, val_set=None) # Use validation_split if val_set is None

# The model is now trained and can be used for prediction
print("BiVAE model training complete.")
```

--------------------------------

### BiVAE Example Usage

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Demonstrates how to use the BiVAE model. This snippet shows a typical workflow including model instantiation and training.

```python
from cornac.models import BiVAE

# Assuming you have your data loaded and preprocessed
# For example, train_data is a numpy array of shape (n_users, n_items)
# and test_data is similarly shaped.

# Initialize the BiVAE model
model = BiVAE(batch_size=128, num_epochs=100, lr=0.001, num_factors=64, 
              num_hidden_factors=[128, 64], dropout_rate=0.5, 
              optimizer="adam", activation="relu", 
              early_stop=True, patience=10, random_state=42)

# Train the model
model.fit(train_data=train_data, test_data=test_data)

# Make predictions (e.g., predict scores for all users and items)
# predictions = model.predict(user_ids=all_users, item_ids=all_items)

```

--------------------------------

### Import Libraries and Print Version

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/tfidf_covid.ipynb

Imports necessary libraries for the project and prints the system version. Ensure you have the recommenders library installed.

```python
import sys

from recommenders.datasets import covid_utils
from recommenders.models.tfidf.tfidf_utils import TfidfRecommender

# Print version
print(f"System version: {sys.version}")
```

--------------------------------

### Navigate to tutorial directory

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/07_tutorials/KDD2020-tutorial/README.md

Changes the current directory to the KDD 2020 tutorial folder within the cloned repository.

```bash
cd recommenders/examples/07_tutorials/KDD2020-tutorial
```

--------------------------------

### Prepare Data Paths and Download Resources

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/xdeepfm_criteo.ipynb

Sets up temporary directory for data and defines file paths for configuration and datasets. It downloads necessary resources if they don't exist.

```python
tmpdir = TemporaryDirectory()
data_path = tmpdir.name
yaml_file = os.path.join(data_path, r'xDeepFM.yaml')
output_file = os.path.join(data_path, r'output.txt')
train_file = os.path.join(data_path, r'cretio_tiny_train')
valid_file = os.path.join(data_path, r'cretio_tiny_valid')
test_file = os.path.join(data_path, r'cretio_tiny_test')

if not os.path.exists(yaml_file):
    download_deeprec_resources(r'https://raw.githubusercontent.com/recommenders-team/resources/main/deeprec/', data_path, 'xdeepfmresources.zip')
```

--------------------------------

### BiVAE Reconstruction Example

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Demonstrates how to reconstruct input data using a trained BiVAE model. It encodes the input, reparameterizes, and decodes to get the reconstructed output.

```python
def reconstruct(model, x):
    mean, log_var = model.encode(x)
    z = model.reparameterize(mean, log_var)
    reconstructed_x = model.decode(z)
    return reconstructed_x
```

--------------------------------

### BiVAE Reconstruction Example

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Shows how to use the trained BiVAE model to reconstruct input data. It encodes the input, reparameterizes to get a latent representation, and then decodes it.

```python
for test_x in test_dataset.take(1):
    x_logit, mean, logvar = bivae(test_x)
    reconstructed_prob = tf.sigmoid(x_logit)
    plt.figure(figsize=(10, 4))
    for i in range(5):
        plt.subplot(2, 5, i + 1)
        plt.imshow(tf.reshape(test_x[i], [28, 28]))
        plt.title('Original')
        plt.axis('off')
        plt.subplot(2, 5, i + 1 + 5)
        plt.imshow(tf.reshape(reconstructed_prob[i], [28, 28]))
        plt.title('Reconstructed')
        plt.axis('off')
    plt.tight_layout()
    plt.show()

```

--------------------------------

### NRMS Baseline for MIND

Source: https://github.com/recommenders-team/recommenders/blob/main/scenarios/news/README.md

Provides a complete notebook for the NRMS model, developed and tested on MIND sample datasets. Useful for getting started with news recommendation baselines.

```python
import os

from recommenders.datasets.mind import MIND
from recommenders.models.newsrec.nrms import NRMS
from recommenders.models.newsrec.utils import calculate_hit_rate, calculate_mrr
from recommenders.models.newsrec.newsrec_utils import prepare_hparams

# Parameters
EPOCHS = 5
BATCH_SIZE = 128

# Load the dataset
news_title_index = {
    "title_entity_linked": "title_entity_linked",
    "title_word_segment": "title_word_segment",
    "title_entity_unlinked": "title_entity_unlinked",
    "title_word_unlinked": "title_word_unlinked",
}
news_title_entity_linked_column = news_title_index["title_entity_linked"]
news_title_word_segment_column = news_title_index["title_word_segment"]

news_data = MIND(path="../data", version="MINDsmall",
                news_title_col=news_title_word_segment_column,
                news_title_entity_linked_col=news_title_entity_linked_column)

news_data.load_train_news()  # Load news data for training
news_data.load_test_news()   # Load news data for testing
news_data.load_train_clickout() # Load clickout data for training
news_data.load_test_clickout()  # Load clickout data for testing

# Prepare hyperparameters
hparams = prepare_hparams(
    [
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/embdotbias_movielens.ipynb

Imports necessary libraries for data manipulation, model building, and evaluation. Sets up logging and prints system and library versions.

```python
# Suppress all warnings
import warnings
warnings.filterwarnings("ignore")

import os
import sys
import logging
import numpy as np
import pandas as pd
import torch
from tempfile import TemporaryDirectory

from recommenders.utils.constants import (
    DEFAULT_USER_COL as USER, 
    DEFAULT_ITEM_COL as ITEM, 
    DEFAULT_RATING_COL as RATING, 
    DEFAULT_TIMESTAMP_COL as TIMESTAMP, 
    DEFAULT_PREDICTION_COL as PREDICTION
)

from recommenders.datasets import movielens
from recommenders.datasets.python_splitters import python_stratified_split
from recommenders.evaluation.python_evaluation import (
    exp_var,
    mae,
    map_at_k,
    ndcg_at_k,
    precision_at_k,
    recall_at_k,
    rmse,
    rsquared
)
from recommenders.models.embdotbias.data_loader import RecoDataLoader
from recommenders.models.embdotbias.model import EmbeddingDotBias
from recommenders.models.embdotbias.training_utils import (
    Trainer,
    predict_rating
)
from recommenders.models.embdotbias.utils import cartesian_product, score
from recommenders.utils.notebook_utils import store_metadata
from recommenders.utils.timer import Timer

logging.basicConfig(level=logging.INFO, format="%(levelname)s - %(message)s")

print(f"System version: {sys.version}")
print(f"Pandas version: {pd.__version__}")
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA Available: {torch.cuda.is_available()}")
print(f"CuDNN Enabled: {torch.backends.cudnn.enabled}")
```

--------------------------------

### NPA Baseline for MIND

Source: https://github.com/recommenders-team/recommenders/blob/main/scenarios/news/README.md

Provides a complete notebook for the NPA model, developed and tested on MIND sample datasets. Useful for getting started with news recommendation baselines.

```python
import os

from recommenders.datasets.mind import MIND
from recommenders.models.newsrec.npa import NPA
from recommenders.models.newsrec.utils import calculate_hit_rate, calculate_mrr
from recommenders.models.newsrec.newsrec_utils import prepare_hparams

# Parameters
EPOCHS = 5
BATCH_SIZE = 128

# Load the dataset
news_title_index = {
    "title_entity_linked": "title_entity_linked",
    "title_word_segment": "title_word_segment",
    "title_entity_unlinked": "title_entity_unlinked",
    "title_word_unlinked": "title_word_unlinked",
}
news_title_entity_linked_column = news_title_index["title_entity_linked"]
news_title_word_segment_column = news_title_index["title_word_segment"]

news_data = MIND(path="../data", version="MINDsmall",
                news_title_col=news_title_word_segment_column,
                news_title_entity_linked_col=news_title_entity_linked_column)

news_data.load_train_news()  # Load news data for training
news_data.load_test_news()   # Load news data for testing
news_data.load_train_clickout() # Load clickout data for training
news_data.load_test_clickout()  # Load clickout data for testing

# Prepare hyperparameters
hparams = prepare_hparams(
    [
```

--------------------------------

### NAML Baseline for MIND

Source: https://github.com/recommenders-team/recommenders/blob/main/scenarios/news/README.md

Provides a complete notebook for the NAML model, developed and tested on MIND sample datasets. Useful for getting started with news recommendation baselines.

```python
import os

from recommenders.datasets.mind import MIND
from recommenders.models.newsrec.naml import NAML
from recommenders.models.newsrec.utils import calculate_hit_rate, calculate_mrr
from recommenders.models.newsrec.newsrec_utils import prepare_hparams

# Parameters
EPOCHS = 5
BATCH_SIZE = 128

# Load the dataset
news_title_index = {
    "title_entity_linked": "title_entity_linked",
    "title_word_segment": "title_word_segment",
    "title_entity_unlinked": "title_entity_unlinked",
    "title_word_unlinked": "title_word_unlinked",
}
news_title_entity_linked_column = news_title_index["title_entity_linked"]
news_title_word_segment_column = news_title_index["title_word_segment"]

news_data = MIND(path="../data", version="MINDsmall",
                news_title_col=news_title_word_segment_column,
                news_title_entity_linked_col=news_title_entity_linked_column)

news_data.load_train_news()  # Load news data for training
news_data.load_test_news()   # Load news data for testing
news_data.load_train_clickout() # Load clickout data for training
news_data.load_test_clickout()  # Load clickout data for testing

# Prepare hyperparameters
hparams = prepare_hparams(
    [
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/04_model_select_and_optimize/azureml_hyperdrive_surprise_svd.ipynb

Imports necessary libraries for Surprise, pandas, Azure ML, and dataset handling. Initializes a temporary directory for caching files.

```python
import sys
import os
import surprise
import pandas as pd
import shutil
from tempfile import TemporaryDirectory

import azureml as aml
import azureml.widgets
import azureml.train.hyperdrive as hd

from recommenders.datasets import movielens
from recommenders.datasets.python_splitters import python_random_split
from recommenders.evaluation.python_evaluation import rmse, precision_at_k, ndcg_at_k
from recommenders.models.surprise.surprise_utils import predict, compute_ranking_predictions

print("System version: {}".format(sys.version))
print("Surprise version: {}".format(surprise.__version__))
print("Azure ML SDK Version:", aml.core.VERSION)

# Temp dir to cache temporal files while running this notebook
tmp_dir = TemporaryDirectory()
```

--------------------------------

### LSTUR Baseline for MIND

Source: https://github.com/recommenders-team/recommenders/blob/main/scenarios/news/README.md

Provides a complete notebook for the LSTUR model, developed and tested on MIND sample datasets. Useful for getting started with news recommendation baselines.

```python
import os

from recommenders.datasets.mind import MIND
from recommenders.models.newsrec.lstur import LSTUR
from recommenders.models.newsrec.utils import calculate_hit_rate, calculate_mrr
from recommenders.models.newsrec.newsrec_utils import prepare_hparams

# Parameters
EPOCHS = 5
BATCH_SIZE = 128

# Load the dataset
news_title_index = {
    "title_entity_linked": "title_entity_linked",
    "title_word_segment": "title_word_segment",
    "title_entity_unlinked": "title_entity_unlinked",
    "title_word_unlinked": "title_word_unlinked",
}
news_title_entity_linked_column = news_title_index["title_entity_linked"]
news_title_word_segment_column = news_title_index["title_word_segment"]

news_data = MIND(path="../data", version="MINDsmall",
                news_title_col=news_title_word_segment_column,
                news_title_entity_linked_col=news_title_entity_linked_column)

news_data.load_train_news()  # Load news data for training
news_data.load_test_news()   # Load news data for testing
news_data.load_train_clickout() # Load clickout data for training
news_data.load_test_clickout()  # Load clickout data for testing

# Prepare hyperparameters
hparams = prepare_hparams(
    [
```

--------------------------------

### DKN Baseline for MIND

Source: https://github.com/recommenders-team/recommenders/blob/main/scenarios/news/README.md

Provides a complete notebook for the DKN model, developed and tested on MIND sample datasets. Useful for getting started with news recommendation baselines.

```python
import os

from recommenders.datasets.mind import MIND
from recommenders.models.newsrec.dkn import DKN
from recommenders.models.newsrec.utils import calculate_hit_rate, calculate_mrr
from recommenders.models.newsrec.newsrec_utils import prepare_hparams

# Parameters
EPOCHS = 5
BATCH_SIZE = 128

# Load the dataset
news_title_index = {
    "title_entity_linked": "title_entity_linked",
    "title_word_segment": "title_word_segment",
    "title_entity_unlinked": "title_entity_unlinked",
    "title_word_unlinked": "title_word_unlinked",
}
news_title_entity_linked_column = news_title_index["title_entity_linked"]
news_title_word_segment_column = news_title_index["title_word_segment"]

news_data = MIND(path="../data", version="MINDsmall",
                news_title_col=news_title_word_segment_column,
                news_title_entity_linked_col=news_title_entity_linked_column)

news_data.load_train_news()  # Load news data for training
news_data.load_test_news()   # Load news data for testing
news_data.load_train_clickout() # Load clickout data for training
news_data.load_test_clickout()  # Load clickout data for testing

# Prepare hyperparameters
hparams = prepare_hparams(
    [
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Imports necessary libraries for data handling, model building, and evaluation. Sets up the environment for the BiVAE model implementation.

```python
import os
import sys
import torch
import cornac

from recommenders.datasets import movielens
from recommenders.datasets.python_splitters import python_random_split
from recommenders.models.cornac.cornac_utils import predict_ranking
from recommenders.utils.timer import Timer
from recommenders.utils.constants import SEED
from recommenders.evaluation.python_evaluation import (
    map_at_k,
    ndcg_at_k,
    precision_at_k,
    recall_at_k,
)
from recommenders.utils.notebook_utils import store_metadata

print(f"System version: {sys.version}")
print(f"PyTorch version: {torch.__version__}")
print(f"Cornac version: {cornac.__version__}")


```

--------------------------------

### Run CPU Docker Container

Source: https://github.com/recommenders-team/recommenders/blob/main/tools/docker/README.md

Runs a Docker container for the CPU environment, mounting the local examples directory and exposing the Jupyter notebook port. This is the basic setup for accessing notebooks.

```bash
docker run -v ../../examples:/root/examples -p 8888:8888 -d recommenders:cpu

```

--------------------------------

### Quick Install for Databricks Operationalization

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Use this command to quickly set up Azure Databricks for operationalization tasks. The `--overwrite` option can be used if the script has been run previously.

```shell
python tools/databricks_install.py --overwrite --prepare-o16n <CLUSTER_ID>
```

--------------------------------

### Install Build Essentials on Linux

Source: https://github.com/recommenders-team/recommenders/blob/main/recommenders/README.md

Installs build-essential and Python development libraries required for compiling dependencies during pip installation on Linux.

```bash
sudo apt-get install -y build-essential libpython<version>
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/surprise_svd_deep_dive.ipynb

Imports necessary libraries for Surprise SVD model, data handling, and evaluation. Prints system and Surprise versions.

```python
import sys
import surprise

from recommenders.utils.timer import Timer
from recommenders.datasets import movielens
from recommenders.datasets.python_splitters import python_random_split
from recommenders.evaluation.python_evaluation import (
    rmse,
    mae,
    rsquared,
    exp_var,
    map_at_k,
    ndcg_at_k,
    precision_at_k,
    recall_at_k,
    get_top_k_items,
)
from recommenders.models.surprise.surprise_utils import (
    predict,
    compute_ranking_predictions,
)
from recommenders.utils.notebook_utils import store_metadata


print(f"System version: {sys.version}")
print(f"Surprise version: {surprise.__version__}")
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/04_model_select_and_optimize/nni_ncf.ipynb

Imports necessary libraries for data manipulation, model building, evaluation, and NNI integration. It also sets up the environment by loading extensions and creating temporary directories.

```python
import sys
import json
import os
import surprise
import pandas as pd
import shutil
import subprocess
import yaml
import pkg_resources
from tempfile import TemporaryDirectory
import torch

import recommenders
from recommenders.utils.timer import Timer
from recommenders.datasets import movielens
from recommenders.datasets.python_splitters import python_chrono_split
from recommenders.evaluation.python_evaluation import rmse, precision_at_k, ndcg_at_k
from recommenders.tuning.nni.nni_utils import (
    check_experiment_status,
    check_stopped,
    check_metrics_written,
    get_trials,
    stop_nni, start_nni
)
from recommenders.models.ncf.dataset import Dataset as NCFDataset
from recommenders.models.ncf.ncf_singlenode import NCF
from recommenders.tuning.nni.ncf_utils import compute_test_results, combine_metrics_dicts

print("System version: {}".format(sys.version))
print("PyTorch version: {}".format(torch.__version__))
print("NNI version: {}".format(pkg_resources.get_distribution("nni").version))

tmp_dir = TemporaryDirectory()

%load_ext autoreload
%autoreload 2
```

--------------------------------

### Initialize and Fit Dataset with Features

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/lightfm_deep_dive.ipynb

Initializes a Dataset instance and fits it with user IDs, item IDs, and extracted item/user features. This prepares the data for LightFM.

```python
dataset2 = Dataset()
dataset2.fit(data['userID'], 
            data['itemID'], 
            item_features=all_movie_genre,
            user_features=all_occupations)
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/04_model_select_and_optimize/azureml_hyperdrive_wide_and_deep.ipynb

Imports necessary libraries for Azure ML, TensorFlow, data manipulation, and recommender utilities. Sets up a temporary directory for caching files.

```python
import os
import itertools
import shutil
from tempfile import TemporaryDirectory

from IPython.display import clear_output
import numpy as np
import pandas as pd
import sklearn.preprocessing
import tensorflow as tf

import azureml as aml
import azureml.widgets as widgets
import azureml.train.hyperdrive as hd

from recommenders.utils.timer import Timer
from recommenders.utils.constants import SEED
from recommenders.utils.tf_utils import pandas_input_fn_for_saved_model
from recommenders.datasets import movielens
from recommenders.datasets.pandas_df_utils import user_item_pairs
from recommenders.datasets.python_splitters import python_random_split
import recommenders.evaluation.python_evaluation as evaluator

print("Azure ML SDK Version:", aml.core.VERSION)
print("Tensorflow Version:", tf.__version__)

# Temp dir to cache temporal files while running this notebook
tmp_dir = TemporaryDirectory()
```

--------------------------------

### Install SMAC Tuner

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/04_model_select_and_optimize/nni_surprise_svd.ipynb

Provides the command to install the SMAC tuner package using nnictl.

```bash
nnictl package install --name=SMAC
```

--------------------------------

### Prepare Training Script Directory

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/04_model_select_and_optimize/azureml_hyperdrive_surprise_svd.ipynb

Sets up a temporary directory to copy the recommender scripts. This is a preparatory step for using a custom training script with Azure ML.

```python
SCRIPT_DIR = os.path.join(tmp_dir.name, 'aml_script')

# Clean-up scripts if already exists
shutil.rmtree(SCRIPT_DIR, ignore_errors=True)

# Copy scripts to SCRIPT_DIR temporarly
shutil.copytree(os.path.join('..', '..', 'recommenders'), os.path.join(SCRIPT_DIR, 'recommenders'))
```

--------------------------------

### Install g++ compiler

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/07_tutorials/KDD2020-tutorial/README.md

Installs the g++ compiler on a Linux system, which is required for certain model pre-training tools.

```bash
sudo apt-get install g++
```

--------------------------------

### Create and activate virtual environment with uv

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/07_tutorials/KDD2020-tutorial/README.md

Creates a Python 3.11 virtual environment named 'kdd_tutorial_2020' using uv and activates it. This isolates project dependencies.

```bash
uv venv ~/.venvs/kdd_tutorial_2020 --python 3.11
source ~/.venvs/kdd_tutorial_2020/bin/activate
```

--------------------------------

### Install uv package manager

Source: https://github.com/recommenders-team/recommenders/blob/main/README.md

Installs the uv package manager, a faster alternative to conda/pip for environment management.

```bash
# 2. Install uv (if not already installed)
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Install CMake on Linux

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Installs the CMake build system on Linux using apt-get, which is a dependency for the xlearn package.

```shell
sudo apt-get install -y build-essential cmake
```

--------------------------------

### Install azureml.contrib.notebook Package

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/run_notebook_on_azureml.ipynb

Install the azureml.contrib.notebook package, which is required for running notebooks on AzureML. This package contains experimental components.

```python
#!pip install "azureml.contrib.notebook>=1.0.21.1"
```

--------------------------------

### BiVAE Example Usage

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Demonstrates how to instantiate and use the BiVAE model with a sample dataset. This snippet shows the typical workflow for training and evaluating the model.

```python
from cornac.models import BiVAE
from cornac.losses import BiVAELoss
from cornac.data import Dataset
from cornac.eval_methods import BaseMethod

# Assume 'train_data', 'valid_data', 'test_data' are loaded datasets
# Example: train_data = Dataset(train_set=...) 

# Placeholder for a compatible autoencoder model
# In a real scenario, this would be a pre-defined autoencoder architecture
class DummyAutoencoder:
    def __init__(self):
        self.encoder = lambda x: x # Dummy encoder
        self.decoder = lambda x: x # Dummy decoder
        self.mean = 0
        self.log_var = 0

    def build(self, *args, **kwargs):
        pass
    def fit(self, *args, **kwargs):
        pass

# Instantiate the BiVAE model
autoencoder = DummyAutoencoder()
vae_loss = BiVAELoss(alpha=0.8, beta=0.2)
bi_vae = BiVAE(autoencoder=autoencoder, vae_loss=vae_loss, latent_dim=50)

# Build and fit the model (using dummy data for illustration)
# train_set = Dataset.load_from_uir_dict({'0': {'0': 1.0}})
# bi_vae.build(train_set)
# bi_vae.fit(train_set)

# Example of recommending items (requires a proper implementation of recommend method)
# recommendations = bi_vae.recommend(user_id=0, n=10)
# print(recommendations)

```

--------------------------------

### Activate Environment and Install Azure CLI

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/sar_movieratings_with_azureml_designer.ipynb

Activate the 'reco_base' conda environment and install the Azure CLI using pip.

```python
conda activate reco_base
pip install azure-cli
```

--------------------------------

### Create and activate a virtual environment with uv

Source: https://github.com/recommenders-team/recommenders/blob/main/README.md

Creates a new virtual environment named 'recommenders' using Python 3.11 and activates it.

```bash
# 3. Create and activate a new virtual environment
uv venv ~/.venvs/recommenders --python 3.11
source ~/.venvs/recommenders/bin/activate
```

--------------------------------

### Import Libraries and Set Up Parameters

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/lightgbm_movielens.ipynb

Imports necessary libraries and defines data and model parameters for the LightGBM LambdaRank pipeline. Includes settings for dataset size, column names, feature engineering, and model hyperparameters.

```python
import os
import sys
import warnings
from tempfile import TemporaryDirectory

import lightgbm as lgb
import numpy as np
import pandas as pd
from lightgbm import LGBMRanker
from recommenders.datasets import movielens
from recommenders.datasets.pandas_df_utils import negative_feedback_sampler
from recommenders.datasets.python_splitters import python_chrono_split
from recommenders.evaluation.python_evaluation import map_at_k, ndcg_at_k
from recommenders.models.lightgbm.lightgbm_utils import NumEncoder
from recommenders.utils.notebook_utils import store_metadata

warnings.filterwarnings("ignore")
print(f"System version: {sys.version}")
print(f"LightGBM version: {lgb.__version__}")
```

```python
# Data settings
MOVIELENS_DATA_SIZE = "1m"
USER_COL = "userID"
ITEM_COL = "itemID"
RATING_COL = "rating"
LABEL_COL = "label"  # 1 if rating > user's personal mean, else 0
SEED = 42

# Engineered feature columns
NUME_COLS = [
    "user_mean_rating",
    "user_rating_count",
    "user_rating_std",
    "item_mean_rating",
    "item_rating_count",
    "item_rating_std",
    "user_genre_mean_rating",
    "user_genre_count",
    "release_year",
]
CATE_COLS = ["main_genre"]

# lgb.LGBMRanker parameters (lambdarank with binary relevance labels)
PARAMS = {
    "objective": "lambdarank",
    "metric": "ndcg",
    "num_leaves": 64,
    "learning_rate": 0.01,
    "colsample_bytree": 0.8,
    "n_jobs": 4,
}
NUM_BOOST_ROUND = 10000
EARLY_STOPPING_ROUNDS = 100
N_NEG_TRAIN = 100
N_NEG_VALID = 100
N_NEG_EVAL = 100
```

--------------------------------

### Define Constants and Parameters

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/embdotbias_movielens.ipynb

Sets up constants for recommendation parameters like TOP_K, MovieLens data size, and model training parameters such as number of factors, epochs, and random seed.

```python
# top k items to recommend
TOP_K = 10

# Select MovieLens data size: 100k, 1m, 10m, or 20m
MOVIELENS_DATA_SIZE = "100k"

# Model parameters
N_FACTORS = 40
EPOCHS = 7
SEED = 101
```

--------------------------------

### BiVAE Training Example

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/cornac_bivae_deep_dive.ipynb

Demonstrates how to train a BiVAE model using a dataset. This involves loading data, instantiating the model, and calling the fit method.

```python
from cornac.models import BiVAE
from cornac.datasets import movielens

# Load the dataset
train_set, val_set, test_set = movielens.load_maybes(variant="100k")

# Instantiate the BiVAE model
model = BiVAE(embedding_size=100, epochs=10, learning_rate=0.001, batch_size=128)

# Train the model
model.fit(train_set, val_set=val_set)

```

--------------------------------

### Install pymanopt for RLRMC and GeoIMC

Source: https://github.com/recommenders-team/recommenders/blob/main/recommenders/README.md

Installs a specific version of pymanopt compatible with TensorFlow 2, required for RLRMC and GeoIMC algorithms.

```bash
pip install "pymanopt@https://github.com/pymanopt/pymanopt/archive/fb36a272cdeecb21992cfd9271eb82baafeb316d.zip"
```

--------------------------------

### Global Settings and Imports

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/02_model_collaborative_filtering/rbm_deep_dive.ipynb

Sets up the environment by importing necessary libraries like pandas, matplotlib, numpy, and TensorFlow, along with specific modules from the recommenders library. It also configures TensorFlow's logging level and prints version information for key packages.

```python
import sys
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline 

import logging
import numpy as np
import tensorflow as tf
tf.get_logger().setLevel(logging.ERROR)

from recommenders.models.rbm.rbm import RBM
from recommenders.datasets.python_splitters import numpy_stratified_split
from recommenders.datasets.sparse import AffinityMatrix
from recommenders.utils.timer import Timer
from recommenders.utils.plot import line_graph
from recommenders.datasets import movielens 
from recommenders.evaluation.python_evaluation import (
    map_at_k,
    ndcg_at_k,
    precision_at_k,
    recall_at_k,
)

#For interactive mode only
%load_ext autoreload
%autoreload 2

print("System version: {}".format(sys.version))
print("Pandas version: {}".format(pd.__version__))
print("Tensorflow version: {}".format(tf.__version__))
```

--------------------------------

### Install uv on macOS

Source: https://github.com/recommenders-team/recommenders/blob/main/SETUP.md

Installs the uv package manager on macOS using a curl script. This is recommended for managing Python packages on macOS.

```shell
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Create Cosmos DB and Collection

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/05_operationalize/als_movie_o16n.ipynb

Initializes a Cosmos DB client, creates a database and collection for storing recommendation results if they don't exist. Requires endpoint and master key.

```python
# explicitly pass subscription_id in case user has multiple subscriptions
client = get_client_from_cli_profile(
    azure.mgmt.cosmosdb.CosmosDB,
    subscription_id=subscription_id
)

async_cosmosdb_create = client.database_accounts.create_or_update(
    resource_group,
    account_name,
    {
        'location': location,
        'locations': [{
            'location_name': location
        }]
    }
)
account = async_cosmosdb_create.result()

my_keys = client.database_accounts.list_keys(resource_group, account_name)
master_key = my_keys.primary_master_key
endpoint = "https://" + account_name + ".documents.azure.com:443/"

# DB client
client = document_client.DocumentClient(endpoint, {'masterKey': master_key})

if not find_database(client, cosmos_database):
    db = client.CreateDatabase({'id': cosmos_database })
    print("Database created")
else:
    db = read_database(client, cosmos_database)
    print("Database found")

# Create collection options
options = dict(offerThroughput=11000)

# Create a collection
collection_definition = {
    'id': cosmos_collection,
    'partitionKey': {'paths': ['/id'],'kind': 'Hash'}
}
if not find_collection(client, cosmos_database, cosmos_collection):
    collection = client.CreateCollection(
        db['_self'], 
        collection_definition,
        options
    )
    print("Collection created")
else:
    collection = read_collection(client, cosmos_database, cosmos_collection)
    print("Collection found")
    
dbsecrets = dict(
    Endpoint=endpoint, 
    Masterkey=master_key, 
    Database=cosmos_database, 
    Collection=cosmos_collection, 
    Upsert=True
)
```

--------------------------------

### Display Full Text Example

Source: https://github.com/recommenders-team/recommenders/blob/main/examples/00_quick_start/tfidf_covid.ipynb

Shows the 'full_text' content from the first article in the DataFrame. This is an example of the raw text that needs cleaning before TF-IDF processing.

```python
all_text['full_text'].head(1)
```