### Run Movielens example with Docker

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Launch the Movielens example notebook using Docker. This allows interactive exploration of LightFM's capabilities.

```bash
docker-compose run --service-ports lightfm jupyter notebook lightfm/examples/movielens/example.ipynb --allow-root --ip="0.0.0.0" --port=8888 --no-browser
```

--------------------------------

### Install LightFM for Development

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Install the project in editable mode after cloning. This command should be run from the project's root directory.

```bash
cd lightfm && pip install -e .
```

--------------------------------

### Install LightFM with Pip

Source: https://github.com/lyst/lightfm/blob/master/README.md

Install the LightFM library using pip. This is the standard method for installing Python packages.

```bash
pip install lightfm
```

--------------------------------

### Development Installation

Source: https://github.com/lyst/lightfm/blob/master/README.md

Install LightFM for development purposes. This involves cloning the repository, setting up a virtual environment, and installing the package in editable mode along with test requirements.

```bash
git clone git@github.com:lyst/lightfm.git
cd lightfm && python3 -m venv venv && source ./venv/bin/activate
pip install -e . && pip install -r test-requirements.txt
```

--------------------------------

### Install Pre-commit Hooks

Source: https://github.com/lyst/lightfm/blob/master/README.md

Install pre-commit to locally enforce code formatting and linting. This is an optional step for development.

```bash
pip install pre-commit
pre-commit install
```

--------------------------------

### Install LightFM with Conda

Source: https://github.com/lyst/lightfm/blob/master/README.md

Install the LightFM library using Conda from the conda-forge channel. This is an alternative installation method.

```bash
conda install -c conda-forge lightfm
```

--------------------------------

### Quickstart: Fit Implicit Feedback Model

Source: https://github.com/lyst/lightfm/blob/master/README.md

Load the MovieLens 100k dataset and fit an implicit feedback model using the WARP loss. Only five-star ratings are considered positive. Ensure data is loaded and the model is instantiated before fitting.

```python
from lightfm import LightFM
from lightfm.datasets import fetch_movielens
from lightfm.evaluation import precision_at_k

# Load the MovieLens 100k dataset. Only five
# star ratings are treated as positive.
data = fetch_movielens(min_rating=5.0)

# Instantiate and train the model
model = LightFM(loss='warp')
model.fit(data['train'], epochs=30, num_threads=2)

# Evaluate the trained model
test_precision = precision_at_k(model, data['test'], k=5).mean()
```

--------------------------------

### Run LightFM tests with Docker

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Execute LightFM tests within the Docker container. Ensure Docker is installed and the repository is cloned.

```bash
docker-compose run lightfm py.test -x lightfm/tests/
```

--------------------------------

### Display Dataset Shape

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/dataset.md

Example output showing the number of users and items after fitting the dataset.

```text
Num users: 105283, num_items 340553.
```

--------------------------------

### Clone LightFM Repository

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Use this command to get a local copy of the LightFM source code.

```bash
git clone git@github.com:lyst/lightfm.git
```

--------------------------------

### get_params

Source: https://github.com/lyst/lightfm/blob/master/doc/lightfm.md

Get the parameters of the LightFM estimator.

```APIDOC
## get_params(deep=True)

### Description
Get parameters for this estimator.

### Parameters
* **deep** (*boolean* *,* *optional*) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

### Returns
**params** – Parameter names mapped to their values.

### Return type
mapping of string to any
```

--------------------------------

### Cythonize Extension Files

Source: https://github.com/lyst/lightfm/blob/master/README.md

When modifying `.pyx` extension files, run this command to generate the corresponding `.c` files before installing the package. This is a necessary step for development involving C extensions.

```bash
python setup.py cythonize
```

--------------------------------

### Run LightFM Tests

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Execute the test suite for LightFM using the setup.py script.

```bash
python setup.py test
```

--------------------------------

### Download Sample Data

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Downloads the Goodbooks-10k dataset from a provided URL. Ensures the 'data' directory exists before saving the zip file.

```python
import os
import zipfile
import csv

import requests


def _download(url: str, dest_path: str):

    req = requests.get(url, stream=True)
    req.raise_for_status()

    with open(dest_path, "wb") as fd:
        for chunk in req.iter_content(chunk_size=2 ** 20):
            fd.write(chunk)


def get_data():

    ratings_url = ("http://www2.informatik.uni-freiburg.de/" ~cziegler/BX/BX-CSV-Dump.zip")

    if not os.path.exists("data"):
        os.makedirs("data")

            _download(ratings_url, "data/data.zip")

    with zipfile.ZipFile("data/data.zip") as archive:
        return (
            csv.DictReader(
                (x.decode("utf-8", "ignore") for x in archive.open("BX-Book-Ratings.csv")),
                delimiter=";",
            ),
            csv.DictReader(
                (x.decode("utf-8", "ignore") for x in archive.open("BX-Books.csv")),";",
            ),
        )


def get_ratings():

    return get_data()[0]


def get_book_features():

    return get_data()[1]
```

--------------------------------

### auc_score

Source: https://github.com/lyst/lightfm/blob/master/doc/lightfm.evaluation.md

Measures the ROC AUC metric for a model, representing the probability that a randomly chosen positive example has a higher score than a randomly chosen negative example. A perfect score is 1.0.

```APIDOC
## auc_score(model, test_interactions, train_interactions=None, user_features=None, item_features=None, preserve_rows=False, num_threads=1, check_intersections=True)

### Description
Measure the ROC AUC metric for a model: the probability that a randomly chosen positive example has a higher score than a randomly chosen negative example. A perfect score is 1.0.

### Parameters
* **model** (*LightFM instance*) – the fitted model to be evaluated
* **test_interactions** (*np.float32 csr_matrix* *of* *shape* *[**n_users* *,* *n_items* *]*) – Non-zero entries representing known positives in the evaluation set.
* **train_interactions** (*np.float32 csr_matrix* *of* *shape* *[**n_users* *,* *n_items* *]* *,* *optional*) – Non-zero entries representing known positives in the train set. These will be omitted from the score calculations to avoid re-recommending known positives.
* **user_features** (*np.float32 csr_matrix* *of* *shape* *[**n_users* *,* *n_user_features* *]* *,* *optional*) – Each row contains that user’s weights over features.
* **item_features** (*np.float32 csr_matrix* *of* *shape* *[**n_items* *,* *n_item_features* *]* *,* *optional*) – Each row contains that item’s weights over features.
* **preserve_rows** (*boolean* *,* *optional*) – When False (default), the number of rows in the output will be equal to the number of users with interactions in the evaluation set. When True, the number of rows in the output will be equal to the number of users.
* **num_threads** (*int* *,* *optional*) – Number of parallel computation threads to use. Should not be higher than the number of physical cores.
* **check_intersections** (*bool* *,* *optional* *,* *True by default* *,*) – Only relevant when train_interactions are supplied. A flag that signals whether the test and train matrices should be checked for intersections to prevent optimistic ranks / wrong evaluation / bad data split.

### Returns
Numpy array containing AUC scores for each user. If there are no interactions for a given user the returned AUC will be 0.5.
* **Return type:**
np.array of shape [n_users with interactions or n_users,]
```

--------------------------------

### Initialize LightFM Model

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Create an instance of the LightFM model with a specified number of latent components.

```python
from lightfm import LightFM

model = LightFM(no_components=30)
```

--------------------------------

### Initialize and Train LightFM Model

Source: https://github.com/lyst/lightfm/blob/master/examples/stackexchange/hybrid_crossvalidated.ipynb

Imports the LightFM model and sets hyperparameters like number of components, epochs, and item alpha. It then fits a WARP model to the training data.

```python
# Import the model
from lightfm import LightFM

# Set the number of threads; you can increase this
# ify you have more physical cores available.
NUM_THREADS = 2
NUM_COMPONENTS = 30
NUM_EPOCHS = 3
ITEM_ALPHA = 1e-6

# Let's fit a WARP model: these generally have the best performance.
model = LightFM(loss='warp',
                item_alpha=ITEM_ALPHA,
               no_components=NUM_COMPONENTS)

# Run 3 epochs and time it.
%time model = model.fit(train, epochs=NUM_EPOCHS, num_threads=NUM_THREADS)
```

--------------------------------

### Get User Embeddings

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Retrieves user embeddings from the model. This is a prerequisite for generating user-specific recommendations.

```python
# Define our user vectors

_, user_embeddings = model.get_user_representations()
```

--------------------------------

### Import Libraries and Load Data

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/learning_schedules.md

Imports necessary libraries and fetches the Movielens dataset for training and testing.

```python
import numpy as np
import data

%matplotlib inline

import matplotlib
import numpy as np
import matplotlib.pyplot as plt

from lightfm import LightFM
from lightfm.datasets import fetch_movielens
from lightfm.evaluation import auc_score

movielens = fetch_movielens()

train, test = movielens['train'], movielens['test']
```

--------------------------------

### Get Item Embeddings

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Retrieves the item embeddings from the trained LightFM model using the item features.

```python
_, item_embeddings = model.get_item_representations(movielens['item_features'])
```

--------------------------------

### Get Predictions from LightFM Model

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Generate predictions for given user and item IDs using the trained LightFM model.

```python
predictions = model.predict(test_user_ids, test_item_ids)
```

--------------------------------

### Display Sample Book Features

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Prints the first book feature entry in JSON format. Illustrates the available book details like title, author, and publication year.

```python
for line in islice(book_features, 1):
    print(json.dumps(line, indent=4))
```

--------------------------------

### Get Dataset Shape

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Retrieves and prints the number of users and items known to the dataset. This confirms the dimensions for the LightFM model.

```python
num_users, num_items = dataset.interactions_shape()
print('Num users: {}, num_items {}.'.format(num_users, num_items))
```

--------------------------------

### Import Libraries and Fetch Data

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Imports necessary libraries and fetches the MovieLens dataset for model training.

```python
%matplotlib inline
import numpy as np
import pandas as pd
from sklearn.metrics import pairwise_distances
import time
from lightfm.datasets import fetch_movielens

movielens = fetch_movielens()
```

--------------------------------

### reciprocal_rank

Source: https://github.com/lyst/lightfm/blob/master/doc/lightfm.evaluation.md

Measures the reciprocal rank metric for a model. It calculates 1 divided by the rank of the highest-ranked positive example. A perfect score is 1.0.

```APIDOC
## reciprocal_rank

### Description
Measure the reciprocal rank metric for a model: 1 / the rank of the highest ranked positive example. A perfect score is 1.0.

### Parameters
* **model** (*LightFM instance*) – the fitted model to be evaluated
* **test_interactions** (*np.float32 csr_matrix* *of* *shape* *[**n_users* *,* *n_items* *]*) – Non-zero entries representing known positives in the evaluation set.
* **train_interactions** (*np.float32 csr_matrix* *of* *shape* *[**n_users* *,* *n_items* *]* *,* *optional*) – Non-zero entries representing known positives in the train set. These will be omitted from the score calculations to avoid re-recommending known positives.
* **user_features** (*np.float32 csr_matrix* *of* *shape* *[**n_users* *,* *n_user_features* *]* *,* *optional*) – Each row contains that user’s weights over features.
* **item_features** (*np.float32 csr_matrix* *of* *shape* *[**n_items* *,* *n_item_features* *]* *,* *optional*) – Each row contains that item’s weights over features.
* **preserve_rows** (*boolean* *,* *optional*) – When False (default), the number of rows in the output will be equal to the number of users with interactions in the evaluation set. When True, the number of rows in the output will be equal to the number of users.
* **num_threads** (*int* *,* *optional*) – Number of parallel computation threads to use. Should not be higher than the number of physical cores.
* **check_intersections** (*bool* *,* *optional* *,* *True by default* *,*) – Only relevant when train_interactions are supplied. A flag that signals whether the test and train matrices should be checked for intersections to prevent optimistic ranks / wrong evaluation / bad data split.

### Returns
Numpy array containing reciprocal rank scores for each user. If there are no interactions for a given user the returned value will be 0.0.

### Return type
np.array of shape [n_users with interactions or n_users,]
```

--------------------------------

### Download Sample Data

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/dataset.md

Downloads the Goodbooks-10k dataset from a provided URL and extracts ratings and book details. Ensures the data directory exists before downloading.

```python
import os
import zipfile
import csv

import requests


def _download(url: str, dest_path: str):

    req = requests.get(url, stream=True)
    req.raise_for_status()

    with open(dest_path, "wb") as fd:
        for chunk in req.iter_content(chunk_size=2 ** 20):
            fd.write(chunk)


def get_data():

    ratings_url = ("http://www2.informatik.uni-freiburg.de/" "~cziegler/BX/BX-CSV-Dump.zip")

    if not os.path.exists("data"):
        os.makedirs("data")

        _download(ratings_url, "data/data.zip")

    with zipfile.ZipFile("data/data.zip") as archive:
        return (
            csv.DictReader(
                (x.decode("utf-8", "ignore") for x in archive.open("BX-Book-Ratings.csv")),
                delimiter=";",
            ),
            csv.DictReader(
                (x.decode("utf-8", "ignore") for x in archive.open("BX-Books.csv")),
                delimiter=";",
            ),
        )


def get_ratings():

    return get_data()[0]


def get_book_features():

    return get_data()[1]
```

--------------------------------

### Get Predictions Using Multiple Cores

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Generate predictions using multiple CPU cores for faster inference by specifying the 'num_threads' parameter.

```python
predictions = model.predict(test_user_ids, test_item_ids, num_threads=4)
```

--------------------------------

### Initialize and Fit LightFM Model

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/dataset.md

Initializes a LightFM model with a specified loss function and fits it to the prepared interaction and item feature matrices.

```python
from lightfm import LightFM

# Assuming 'interactions' and 'item_features' are already built sparse matrices
# Example placeholder matrices:
from scipy.sparse import csr_matrix
interactions = csr_matrix((10, 10))
item_features = csr_matrix((10, 10))

model = LightFM(loss='bpr')
model.fit(interactions, item_features=item_features)
```

```default
<lightfm.lightfm.LightFM at 0x7f5f0e8f7c88>
```

--------------------------------

### Generate Sample User Recommendations

Source: https://github.com/lyst/lightfm/blob/master/examples/quickstart/quickstart.ipynb

Defines and calls a function to generate and print top recommendations for a list of specified users, showing known positive interactions and recommended items.

```python
def sample_recommendation(model, data, user_ids):
    

    n_users, n_items = data['train'].shape

    for user_id in user_ids:
        known_positives = data['item_labels'][data['train'].tocsr()[user_id].indices]
        
        scores = model.predict(user_id, np.arange(n_items))
        top_items = data['item_labels'][np.argsort(-scores)]
        
        print("User %s" % user_id)
        print("     Known positives:")
        
        for x in known_positives[:3]:
            print("        %s" % x)

        print("     Recommended:")
        
        for x in top_items[:3]:
            print("        %s" % x)
        
sample_recommendation(model, data, [3, 25, 450])
```

--------------------------------

### Get Predictions with User and Item Features

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Generate predictions when user and item features were used during training. Assumes 'user_features' and 'item_features' are sparse matrices.

```python
predictions = model.predict(test_user_ids,
                            test_item_ids,
                            user_features=user_features,
                            item_features=item_features)
```

--------------------------------

### Run Tests

Source: https://github.com/lyst/lightfm/blob/master/README.md

Execute the test suite for LightFM within the development environment. Ensure the virtual environment is activated before running this command.

```bash
./venv/bin/py.test tests
```

--------------------------------

### Get Similar Tags Function

Source: https://github.com/lyst/lightfm/blob/master/examples/stackexchange/hybrid_crossvalidated.ipynb

Calculates the most similar tags to a given tag ID based on cosine similarity of their latent vectors. Requires a trained model and tag labels.

```python
def get_similar_tags(model, tag_id):
    # Define similarity as the cosine of the angle
    # between the tag latent vectors
    
    # Normalize the vectors to unit length
    tag_embeddings = (model.item_embeddings.T
                      / np.linalg.norm(model.item_embeddings, axis=1)).T
    
    query_embedding = tag_embeddings[tag_id]
    similarity = np.dot(tag_embeddings, query_embedding)
    most_similar = np.argsort(-similarity)[1:4]
    
    return most_similar
```

--------------------------------

### Initialize and Train LightFM Model

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Initializes a LightFM model with a specified loss function and fits it to the prepared interaction and item feature data.

```python
from lightfm import LightFM

    model = LightFM(loss='bpr')
    model.fit(interactions, item_features=item_features)
```

--------------------------------

### Predict Scores for User-Item Pairs with LightFM

Source: https://github.com/lyst/lightfm/blob/master/doc/lightfm.md

Use the `predict` method to get scores for specific user-item pairs. Ensure input arrays are correctly formatted, with repeated user IDs for multiple items.

```python
lfm.predict([0, 1], [8, 9])
```

```python
lfm.predict([0, 0, 0, 1, 1, 1], [7, 8, 9, 7, 8, 9])
```

--------------------------------

### Import LightFM Model

Source: https://github.com/lyst/lightfm/blob/master/doc/quickstart.md

Imports the LightFM class, which is necessary for initializing and training the recommendation model.

```python
from lightfm import LightFM
```

--------------------------------

### Get Similar Tags using Hybrid Model Embeddings

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/hybrid_crossvalidated.md

Calculates and returns the most similar tags to a given tag ID based on cosine similarity of their latent vectors. Requires a trained model and tag labels.

```python
def get_similar_tags(model, tag_id):
    # Define similarity as the cosine of the angle
    # between the tag latent vectors

    # Normalize the vectors to unit length
    tag_embeddings = (model.item_embeddings.T
                      / np.linalg.norm(model.item_embeddings, axis=1)).T

    query_embedding = tag_embeddings[tag_id]
    similarity = np.dot(tag_embeddings, query_embedding)
    most_similar = np.argsort(-similarity)[1:4]

    return most_similar


for tag in (u'bayesian', u'regression', u'survival'):
    tag_id = tag_labels.tolist().index(tag)
    print('Most similar tags for %s: %s' % (tag_labels[tag_id],
                                            tag_labels[get_similar_tags(model, tag_id)]))
```

```default
Most similar tags for bayesian: [u'posterior' u'mcmc' u'bayes']
Most similar tags for regression: [u'multicollinearity' u'stepwise-regression' u'multiple-regression']
Most similar tags for survival: [u'cox-model' u'kaplan-meier' u'odds-ratio']
```

--------------------------------

### Build LightFM Docker container

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Build the Docker container for LightFM. This is useful for users on OSX and Windows who want to leverage multi-threading capabilities.

```bash
docker-compose build lightfm
```

--------------------------------

### Build Annoy Index

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Initializes an Annoy index, adds item embeddings to it, builds the index with 10 trees, and saves it to a file.

```python
from annoy import AnnoyIndex

factors = item_embeddings.shape[1] # Length of item vector that will be indexed
annoy_idx = AnnoyIndex(factors)
for i in range(item_embeddings.shape[0]):
    v = item_embeddings[i]
    annoy_idx.add_item(i, v)

annoy_idx.build(10) # 10 trees
annoy_idx.save('movielens_item_Annoy_idx.ann')
```

--------------------------------

### Generate Recommendations for Sample Users

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Calls the `sample_recommendation` function to generate and print recommendations for specific user IDs (3, 25, and 450).

```python
sample_recommendation([3,25,450], model, movielens, print_output=True)
```

--------------------------------

### Initialize and Train LightFM WARP Model

Source: https://github.com/lyst/lightfm/blob/master/examples/quickstart/quickstart.ipynb

Initializes a LightFM model with the WARP loss function and trains it on the training data for 30 epochs using 2 threads. Measures training time.

```python
from lightfm import LightFM

model = LightFM(loss='warp')
%time model.fit(data['train'], epochs=30, num_threads=2)
```

--------------------------------

### Generate Sample Recommendations

Source: https://github.com/lyst/lightfm/blob/master/doc/quickstart.md

Defines and uses a function to generate top item recommendations for specified users, displaying known positive interactions and recommended items.

```python
def sample_recommendation(model, data, user_ids):

    n_users, n_items = data['train'].shape

    for user_id in user_ids:
        known_positives = data['item_labels'][data['train'].tocsr()[user_id].indices]

        scores = model.predict(user_id, np.arange(n_items))
        top_items = data['item_labels'][np.argsort(-scores)]

        print("User %s" % user_id)
        print("     Known positives:")

        for x in known_positives[:3]:
            print("        %s" % x)

        print("     Recommended:")

        for x in top_items[:3]:
            print("        %s" % x)

sample_recommendation(model, data, [3, 25, 450])
```

--------------------------------

### Initialize and Build NMSLIB Index

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Initializes an NMSLIB index using HNSW on Cosine Similarity, adds item embeddings, and creates the index.

```python
import nmslib

# initialize a new nmslib index, using a HNSW index on Cosine Similarity
nms_idx = nmslib.init(method='hnsw', space='cosinesimil')
nms_idx.addDataPointBatch(item_embeddings)
nms_idx.createIndex(print_progress=True)
```

--------------------------------

### Display Sample Ratings

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Prints the first two rating entries in JSON format for inspection. Shows the structure of user ID, ISBN, and book rating.

```python
for line in islice(ratings, 2):
    print(json.dumps(line, indent=4))
```

--------------------------------

### Fit Dataset with User and Item IDs

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Initializes a Dataset object and fits it with user and item IDs from the ratings data. This creates internal mappings for model usage.

```python
from lightfm.data import Dataset

dataset = Dataset()
dataset.fit((x['User-ID'] for x in get_ratings()),
            (x['ISBN'] for x in get_ratings()))
```

--------------------------------

### Train Models with Adagrad and Adadelta (k-OS Loss)

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/learning_schedules.md

Initializes and trains two LightFM models with the k-OS loss, comparing 'adagrad' and 'adadelta' learning schedules. It iteratively fits the models and records the AUC score on the test set after each epoch.

```python
alpha = 1e-3
epochs = 70

adagrad_model = LightFM(no_components=30,
                        loss='warp-kos',
                        learning_schedule='adagrad',
                        user_alpha=alpha, item_alpha=alpha)
adadelta_model = LightFM(no_components=30,
                        loss='warp-kos',
                        learning_schedule='adadelta',
                        user_alpha=alpha, item_alpha=alpha)

adagrad_auc = []

for epoch in range(epochs):
    adagrad_model.fit_partial(train, epochs=1)
    adagrad_auc.append(auc_score(adagrad_model, test).mean())


adadelta_auc = []

for epoch in range(epochs):
    adadelta_model.fit_partial(train, epochs=1)
    adadelta_auc.append(auc_score(adadelta_model, test).mean())
```

--------------------------------

### Fetch and Preprocess Movielens Dataset

Source: https://github.com/lyst/lightfm/blob/master/doc/quickstart.md

Downloads and preprocesses the Movielens dataset into sparse matrices for training and testing. Ensures minimum rating is 5.0.

```python
import numpy as np

from lightfm.datasets import fetch_movielens

data = fetch_movielens(min_rating=5.0)
```

--------------------------------

### Train LightFM Model Using Multiple Cores

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Utilize multiple CPU cores for faster training by specifying the 'num_threads' parameter.

```python
model.fit(train, epochs=20, num_threads=4)
```

--------------------------------

### Train LightFM Model with User and Item Features

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Incorporate user and item features into the training process. Assumes 'train', 'user_features', and 'item_features' are sparse matrices.

```python
model.fit(train,
          user_features=user_features,
          item_features=item_features,
          epochs=20)
```

--------------------------------

### Load Data into Memory

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Loads the ratings and book features from the downloaded data. This is a preliminary step before processing.

```python
import json
from itertools import islice

ratings, book_features = get_data()
```

--------------------------------

### Build NMSLIB Index with Normalized Data

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Initializes and builds an NMSLIB index using the 'hnsw' method and 'cosinesimil' space with the normalized item embeddings. Progress is printed during index creation.

```python
nms_member_idx = nmslib.init(method='hnsw', space='cosinesimil')
nms_member_idx.addDataPointBatch(norm_data)
nms_member_idx.createIndex(print_progress=True)
```

--------------------------------

### Fit a LightFM WARP Model

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/hybrid_crossvalidated.md

This snippet shows how to initialize and fit a LightFM WARP model with specified parameters. It's useful for training a collaborative filtering model on interaction data.

```python
# Import the model
from lightfm import LightFM

# Set the number of threads; you can increase this
# if you have more physical cores available.
NUM_THREADS = 2
NUM_COMPONENTS = 30
NUM_EPOCHS = 3
ITEM_ALPHA = 1e-6

# Let's fit a WARP model: these generally have the best performance.
model = LightFM(loss='warp',
                item_alpha=ITEM_ALPHA,
               no_components=NUM_COMPONENTS)

# Run 3 epochs and time it.
%time model = model.fit(train, epochs=NUM_EPOCHS, num_threads=NUM_THREADS)
```

--------------------------------

### Build Interactions Matrix

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/dataset.md

Constructs the user-item interaction matrix from rating data. This is the primary input for a LightFM model.

```python
from lightfm.datasets import Dataset

# Assuming get_ratings() is a function that yields rating dictionaries
# Example placeholder for get_ratings():
def get_ratings():
    return [
        {'User-ID': 1, 'ISBN': 'A', 'Rating': 5},
        {'User-ID': 2, 'ISBN': 'B', 'Rating': 3},
        {'User-ID': 1, 'ISBN': 'C', 'Rating': 4}
    ]

dataset = Dataset()
dataset.fit_update(users=set([x['User-ID'] for x in get_ratings()]), 
                   items=set([x['ISBN'] for x in get_ratings()]))

(interactions, weights) = dataset.build_interactions(((x['User-ID'], x['ISBN'])
                                                      for x in get_ratings()))

print(repr(interactions))
```

```default
<105283x341762 sparse matrix of type '<class 'numpy.int32'>'
        with 1149780 stored elements in COOrdinate format>
```

--------------------------------

### Fit LightFM with WARP Loss and Limited Sampling

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/warp_loss.md

Initializes and fits a LightFM model using the WARP loss with `max_sampled` set to 3. It then iteratively fits the model for one epoch at a time, recording the duration and AUC score for each epoch. This demonstrates how limiting negative sampling attempts affects fitting time and model performance.

```python
warp_model = LightFM(no_components=num_components,
                     max_sampled=3,
                    loss='warp',
                    learning_schedule='adagrad',
                    user_alpha=alpha,
                    item_alpha=alpha)

warp_duration = []
warp_auc = []

for epoch in range(epochs):
    start = time.time()
    warp_model.fit_partial(train, epochs=1)
    warp_duration.append(time.time() - start)
    warp_auc.append(auc_score(warp_model, test, train_interactions=train).mean())
```

--------------------------------

### Train and Evaluate Adagrad and Adadelta Models (WARP-KOS Loss)

Source: https://github.com/lyst/lightfm/blob/master/examples/movielens/learning_schedules.ipynb

Initializes and trains two LightFM models using the 'warp-kos' loss function, comparing 'adagrad' and 'adadelta' learning schedules. Similar to the 'warp' loss experiment, it tracks ROC AUC on the test set per epoch.

```python
alpha = 1e-3
epochs = 70

adagrad_model = LightFM(no_components=30,
                        loss='warp-kos',
                        learning_schedule='adagrad',
                        user_alpha=alpha, item_alpha=alpha)
adadelta_model = LightFM(no_components=30,
                        loss='warp-kos',
                        learning_schedule='adadelta',
                        user_alpha=alpha, item_alpha=alpha)

adagrad_auc = []

for epoch in range(epochs):
    adagrad_model.fit_partial(train, epochs=1)
    adagrad_auc.append(auc_score(adagrad_model, test).mean())
    
    
adadelta_auc = []

for epoch in range(epochs):
    adadelta_model.fit_partial(train, epochs=1)
    adadelta_auc.append(auc_score(adadelta_model, test).mean())
```

--------------------------------

### Fetch MovieLens Dataset

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/warp_loss.md

Loads the MovieLens 100K dataset for training and testing. Ensure necessary libraries like numpy and matplotlib are imported.

```python
import time

import numpy as np

%matplotlib inline

import matplotlib
import numpy as np
import matplotlib.pyplot as plt

from lightfm import LightFM
from lightfm.datasets import fetch_movielens
from lightfm.evaluation import auc_score

movielens = fetch_movielens()

train, test = movielens['train'], movielens['test']
```

--------------------------------

### Train LightFM Model with WARP Loss for Implicit Feedback

Source: https://github.com/lyst/lightfm/blob/master/doc/home.md

Train a LightFM model using the WARP loss function, suitable for implicit feedback scenarios. Assumes 'train' is a sparse matrix with positive interactions.

```python
model = LightFM(no_components=30, loss='warp')
model.fit(train, epochs=20)
```

--------------------------------

### Train Models with Adagrad and Adadelta (WARP Loss)

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/learning_schedules.md

Initializes and trains two LightFM models, one with 'adagrad' and another with 'adadelta' learning schedules, using the WARP loss. It iteratively fits the models and records the AUC score on the test set after each epoch.

```python
alpha = 1e-3
epochs = 70

adagrad_model = LightFM(no_components=30,
                        loss='warp',
                        learning_schedule='adagrad',
                        user_alpha=alpha,
                        item_alpha=alpha)
adadelta_model = LightFM(no_components=30,
                        loss='warp',
                        learning_schedule='adadelta',
                        user_alpha=alpha,
                        item_alpha=alpha)

adagrad_auc = []

for epoch in range(epochs):
    adagrad_model.fit_partial(train, epochs=1)
    adagrad_auc.append(auc_score(adagrad_model, test).mean())


adadelta_auc = []

for epoch in range(epochs):
    adadelta_model.fit_partial(train, epochs=1)
    adadelta_auc.append(auc_score(adadelta_model, test).mean())
```

--------------------------------

### Sample Recommendation Function

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Defines a function to generate movie recommendations for given user IDs. It retrieves known positive items and recommended items using the Annoy index and the normalized user vector.

```python
def sample_recommendation(user_ids, model, data, n_items=10, print_output=True):
    n_users, n_items = data['train'].shape

    for user_id in user_ids:
        known_positives = data['item_labels'][data['train'].tocsr()[user_id].indices]
        top_items = [data['item_labels'][i] for i in annoy_member_idx.get_nns_by_vector(np.append(user_embeddings[user_id], 0), 50)]
        if print_output == True:
            print("User %s" % user_id)
            print("     Known positives:")

            for x in known_positives[:3]:
                print("        %s" % x)

            print("     Recommended:")

            for x in top_items[:3]:
                print("        %s" % x)
```

--------------------------------

### Train BPR Model and Evaluate

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/movielens_implicit.md

Trains a BPR (Bayesian Personalized Ranking) model on the training data and evaluates its performance using precision@k and AUC on both training and testing sets. Requires LightFM and evaluation metrics to be imported.

```python
from lightfm import LightFM
from lightfm.evaluation import precision_at_k
from lightfm.evaluation import auc_score

model = LightFM(learning_rate=0.05, loss='bpr')
model.fit(train, epochs=10)

train_precision = precision_at_k(model, train, k=10).mean()
test_precision = precision_at_k(model, test, k=10).mean()

train_auc = auc_score(model, train).mean()
test_auc = auc_score(model, test).mean()

print('Precision: train %.2f, test %.2f.' % (train_precision, test_precision))
print('AUC: train %.2f, test %.2f.' % (train_auc, test_auc))
```

--------------------------------

### Prepare Item Embeddings for Inner Product Search

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Prepares item embeddings for approximate maximum inner product search by adding a normalizing factor. This involves calculating norms, an extra dimension, and appending it to the embeddings.

```python
norms = np.linalg.norm(item_embeddings, axis=1)
max_norm = norms.max()
extra_dimension = np.sqrt(max_norm ** 2 - norms ** 2)
norm_data = np.append(item_embeddings, extra_dimension.reshape(norms.shape[0], 1), axis=1)
```

--------------------------------

### Calculate Precision at K for LightFM

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Calculates the mean 'precision at K' for the LightFM model using the test dataset. This serves as a baseline for comparison.

```python
lightfm_pak = precision_at_k(model, test, k=10).mean()
```

--------------------------------

### Build Annoy Index with Normalized Data

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Initializes and builds an Annoy index using the prepared normalized item embeddings. The index is configured with the length of the item vectors.

```python
user_factors = norm_data.shape[1]
annoy_member_idx = AnnoyIndex(user_factors)  # Length of item vector that will be indexed

for i in range(norm_data.shape[0]):
    v = norm_data[i]
    annoy_member_idx.add_item(i, v)
    
annoy_member_idx.build(10)
```

--------------------------------

### Inspect Movielens Dataset Structure

Source: https://github.com/lyst/lightfm/blob/master/doc/examples/movielens_implicit.md

Prints the keys, types, and shapes of the loaded Movielens dataset. This helps in understanding the structure of the data available for training and evaluation.

```python
for key, value in movielens.items():
    print(key, type(value), value.shape)
```

--------------------------------

### lightfm.datasets.stackexchange.fetch_stackexchange

Source: https://github.com/lyst/lightfm/blob/master/doc/datasets.md

Fetches a dataset from the StackExchange network, representing users answering questions. It allows specifying the dataset (CrossValidated or StackOverflow) and controlling the test set fraction and minimum training interactions.

```APIDOC
## lightfm.datasets.stackexchange.fetch_stackexchange

### Description
Fetch a dataset from the StackExchange network. The datasets contain users answering questions: an interaction is defined as a user answering a given question.

### Method
`fetch_stackexchange`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **dataset** (*string*, *one of* (*'crossvalidated'*, *'stackoverflow'*)*) – The part of the StackExchange network for which to fetch the dataset.
* **test_set_fraction** (*float*, *optional*) – The fraction of the dataset used for testing. Splitting into the train and test set is done in a time-based fashion: all interactions before a certain time are in the train set and all interactions after that time are in the test set.
* **min_training_interactions** (*int*, *optional*) – Only include users with this amount of interactions in the training set.
* **data_home** (*path*, *optional*) – Path to the directory in which the downloaded data should be placed. Defaults to `~/lightfm_data/`.
* **indicator_features** (*bool*, *optional*) – Use an [n_users, n_users] identity matrix for item features. When True with genre_features, indicator and genre features are concatenated into a single feature matrix of shape [n_users, n_users + n_genres].
* **download_if_missing** (*bool*, *optional*) – Download the data if not present. Raises an IOError if False and data is missing.

### Returns
* **train** (*sp.coo_matrix of shape [n_users, n_items]*) – Contains training set interactions.
* **test** (*sp.coo_matrix of shape [n_users, n_items]*) – Contains testing set interactions.
* **item_features** (*sp.csr_matrix of shape [n_items, n_item_features]*) – Contains item features.
* **item_feature_labels** (*np.array of strings of shape [n_item_features,]*) – Labels of item features.
```

--------------------------------

### Train and Evaluate Adagrad and Adadelta Models (WARP Loss)

Source: https://github.com/lyst/lightfm/blob/master/examples/movielens/learning_schedules.ipynb

Initializes and trains two LightFM models, one with 'adagrad' and another with 'adadelta' learning schedules, using the 'warp' loss. It iteratively fits the models and records the ROC AUC score on the test set after each epoch.

```python
alpha = 1e-3
epochs = 70

adagrad_model = LightFM(no_components=30,
                        loss='warp',
                        learning_schedule='adagrad',
                        user_alpha=alpha,
                        item_alpha=alpha)
adadelta_model = LightFM(no_components=30,
                        loss='warp',
                        learning_schedule='adadelta',
                        user_alpha=alpha,
                        item_alpha=alpha)

adagrad_auc = []

for epoch in range(epochs):
    adagrad_model.fit_partial(train, epochs=1)
    adagrad_auc.append(auc_score(adagrad_model, test).mean())
    
    
adadelta_auc = []

for epoch in range(epochs):
    adadelta_model.fit_partial(train, epochs=1)
    adadelta_auc.append(auc_score(adadelta_model, test).mean())
```

--------------------------------

### Visualize Precision at K Comparison

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Generates a bar plot to visually compare the 'precision at K' values obtained from LightFM, Annoy, and NMSLib.

```python
plt.bar(['LightFm', 'Annoy', 'NMSLib'], [lightfm_pak, annoy_pak, nms_pak])
plt.title('Precision at K=10, ANN vs. exact')
plt.ylabel('Precision at k=10')
plt.show()
```

--------------------------------

### Time Original Recommendation

Source: https://github.com/lyst/lightfm/blob/master/examples/ann/annoy_nsmlib_example.ipynb

Measures the execution time of the original recommendation function using %%timeit. This is used for performance comparison.

```python
%%timeit
sample_recommendation_original(model, movielens, [3, 25, 450], print_output=False)
```

--------------------------------

### Build User-Item Interactions for LightFM

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Constructs the interactions matrix, which is the primary input for a LightFM model. Ensure your data is formatted as (user_id, item_id) pairs.

```python
(interactions, weights) = dataset.build_interactions(((x['User-ID'], x['ISBN'])
                                                          for x in get_ratings()))

    print(repr(interactions))
```

--------------------------------

### Train WARP Model

Source: https://github.com/lyst/lightfm/blob/master/doc/quickstart.md

Initializes and trains a LightFM model using the WARP loss function. The training is performed using stochastic gradient descent for 30 epochs with 2 threads.

```python
model = LightFM(loss='warp')
%time model.fit(data['train'], epochs=30, num_threads=2)
```

--------------------------------

### Build Item Features Matrix for LightFM

Source: https://github.com/lyst/lightfm/blob/master/examples/dataset/readme.rst

Creates the item features matrix, which can be used alongside interactions for more advanced modeling. The format is (item_id, [feature_list]).

```python
item_features = dataset.build_item_features(((x['ISBN'], [x['Book-Author']])
                                                  for x in get_book_features()))
    print(repr(item_features))
```

--------------------------------

### Set Parameters for LightFM Estimator

Source: https://github.com/lyst/lightfm/blob/master/doc/lightfm.md

Use the `set_params` method to adjust the parameters of a LightFM estimator after initialization.

```python
lfm.set_params(**params)
```