### Install required libraries (prophet, mlforecast, utilsforecast)

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/electricity_load_forecasting.html

Installation commands for the libraries used in the tutorial. The cell is commented out and uses %%capture to suppress output. Run only if the libraries are not already installed.

```python
# %%capture
# !pip install prophet
# !pip install -U mlforecast
# !pip install -U utilsforecast
```

--------------------------------

### Install required libraries with pip

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/prediction_intervals.html

Install the necessary packages for the tutorial using pip. The command installs mlforecast and utilsforecast.

```bash
pip install mlforecast utilsforecast
```

--------------------------------

### Install development version from GitHub

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/install.html

Install the development version directly from GitHub, or clone and install from the current main branch.

```bash
pip install git+https://github.com/Nixtla/mlforecast
```

```bash
git clone https://github.com/Nixtla/mlforecast mlforecast-dev && pip install mlforecast-dev/
```

--------------------------------

### Setup imports for direct forecasting

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/one_model_per_horizon.html

Imports required libraries and modules for the direct forecasting example, including mlforecast, lightgbm, pandas, and utility functions.

```python
import random
import lightgbm as lgb
import pandas as pd
from datasetsforecast.m4 import M4, M4Info
from utilsforecast.evaluation import evaluate
from utilsforecast.losses import smape

from mlforecast import MLForecast
from mlforecast.lag_transforms import ExponentiallyWeightedMean, RollingMean
from mlforecast.target_transforms import Differences
```

--------------------------------

### Install mlforecast via PyPI

Source: https://nixtlaverse.nixtla.io/mlforecast

Install mlforecast using pip from PyPI.

```bash
pip install mlforecast
```

--------------------------------

### Import required libraries for M4 and LightGBMCV example

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Imports necessary modules including datasetsforecast, fastcore, mlforecast, and nbdev for the example.

```python
import random

from datasetsforecast.m4 import M4, M4Info
from fastcore.test import test_eq, test_fail
from mlforecast.target_transforms import Differences
from nbdev import show_doc

from mlforecast.lag_transforms import SeasonalRollingMean
```

--------------------------------

### Install mlforecast and required libraries

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/incremental_forecasting.html

Installs mlforecast, datasetsforecast, s3fs, and xgboost quietly. Recommended to restart the kernel after installation in Colab or fresh environments.

```bash
!pip install mlforecast datasetsforecast s3fs xgboost -q
```

--------------------------------

### Import dependencies for optimization example

Source: https://nixtlaverse.nixtla.io/mlforecast/optimization.html

Imports required libraries: lightgbm, datasetsforecast, utilsforecast, and mlforecast modules for the optimization example.

```python
import lightgbm as lgb
from datasetsforecast.m4 import M4, M4Evaluation, M4Info
from utilsforecast.losses import smape

from mlforecast.lag_transforms import ExpandingMean, RollingMean
from mlforecast.target_transforms import Differences, LocalBoxCox, LocalStandardScaler
```

--------------------------------

### Install MLForecast and dependencies

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/transfer_learning.html

Installation command for MLForecast and related libraries. The command is commented out and wrapped in %%capture to suppress output.

```bash
%%capture
# !pip install mlforecast datasetsforecast utilsforecast s3fs
```

--------------------------------

### Install extras for distributed training

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/install.html

Install mlforecast with extras for distributed training using dask, ray, or spark.

```bash
pip install "mlforecast[dask]"
```

```bash
pip install "mlforecast[ray]"
```

```bash
pip install "mlforecast[spark]"
```

--------------------------------

### Set up data for MLflow example

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/mlflow.html

Generates a series, splits into train and validation, and adds Fourier features.

```python
freq = 'h'
h = 10
series = generate_series(5, freq=freq)
valid = series.groupby('unique_id', observed=True).tail(h)
train = series.drop(valid.index)
train, X_df = fourier(train, freq=freq, season_length=24, k=2, h=h)
```

--------------------------------

### Import required libraries for MLForecast example

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/predict_subset.html

Imports LGBMRegressor, test_fail, MLForecast, and generate_daily_series. These are needed for the subsequent example.

```python
from lightgbm import LGBMRegressor
from fastcore.test import test_fail

from mlforecast import MLForecast
from mlforecast.utils import generate_daily_series
```

--------------------------------

### Import MLForecast and models

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/training_with_numpy.html

Import numpy, LightGBM, LinearRegression, and MLForecast for the forecasting setup.

```python
import numpy as np
from lightgbm import LGBMRegressor
from sklearn.linear_model import LinearRegression

from mlforecast import MLForecast
```

--------------------------------

### Import libraries for sample weights example

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/sample_weights.html

Imports numpy and the generate_daily_series utility from mlforecast.utils.

```python
import numpy as np
from mlforecast.utils import generate_daily_series
```

--------------------------------

### Setup LightGBMCV for partial fitting

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Sets up a LightGBMCV model with lags and two windows for partial fitting, showing the model configuration.

```python
cv4 = LightGBMCV(
    freq=1,
    lags=[24 * (i+1) for i in range(7)],
)
cv4.setup(
    train,
    n_windows=2,
    h=horizon,
    params={'verbose': -1},
)
```

```text
LightGBMCV(freq=1, lag_features=['lag24', 'lag48', 'lag72', 'lag96', 'lag120', 'lag144', 'lag168'], date_features=[], num_threads=1, bst_threads=8)
```

--------------------------------

### Install specific version from PyPI

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/install.html

Use version filters to install a specific version or any version prior to a given version.

```bash
pip install "mlforecast==0.3.0"
```

```bash
pip install "mlforecast<0.4.0"
```

--------------------------------

### Setup: generate sample series and prices

Source: https://nixtlaverse.nixtla.io/mlforecast/feature_engineering.html

Generates a daily series with 100 series, creates a prices DataFrame with random prices and a price2 column, and displays the first rows.

```python
rng = np.random.RandomState(0)
series = generate_daily_series(100, equal_ends=True)
starts_ends = series.groupby(
    'unique_id', observed=True, as_index=False
)['ds'].agg(['min', 'max'])
prices = []
for r in starts_ends.itertuples():
    dates = pd.date_range(r.min, r.max + 14 * pd.offsets.Day())
    df = pd.DataFrame({'ds': dates, 'price': rng.rand(dates.size)})
    df['unique_id'] = r.Index
    prices.append(df)
prices = pd.concat(prices)
prices['price2'] = prices['price'] * rng.rand(prices.shape[0])
prices.head()
```

--------------------------------

### Install mlforecast via conda-forge

Source: https://nixtlaverse.nixtla.io/mlforecast

Install mlforecast using conda from the conda-forge channel.

```bash
conda install -c conda-forge mlforecast
```

--------------------------------

### Import test utilities and show_doc

Source: https://nixtlaverse.nixtla.io/mlforecast/utils.html

Imports testing helpers from fastcore and nbdev for use in the documentation examples.

```python
from fastcore.test import test_eq, test_fail
from nbdev import show_doc
```

--------------------------------

### Import libraries for MLForecast

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/custom_training.html

Import lightgbm, numpy, LinearRegression, and MLForecast for the custom training example.

```python
import lightgbm as lgb
import numpy as np
from sklearn.linear_model import LinearRegression

from mlforecast import MLForecast
```

--------------------------------

### Import Ray and Cluster for Session Setup

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/quick_start_distributed.html

Imports Ray and the Cluster utility needed to create a local Ray cluster for distributed training.

```python
# ray
import ray
from ray.cluster_utils import Cluster
```

--------------------------------

### Import MLForecast and sklearn models

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/prediction_intervals.html

Imports the required classes from mlforecast and sklearn for the example.

```python
from mlforecast import MLForecast
from mlforecast.target_transforms import Differences
from mlforecast.utils import PredictionIntervals
from sklearn.linear_model import Lasso, LinearRegression, Ridge
from sklearn.neighbors import KNeighborsRegressor
from sklearn.neural_network import MLPRegressor
```

--------------------------------

### Import MLForecast and generate_daily_series

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/custom_date_features.html

Imports the MLForecast class and the generate_daily_series utility function, which are required for the examples on this page.

```python
from mlforecast import MLForecast
from mlforecast.utils import generate_daily_series
```

--------------------------------

### GlobalSklearnTransformer usage example

Source: https://nixtlaverse.nixtla.io/mlforecast/target_transforms.html

Shows how to use GlobalSklearnTransformer with a PowerTransformer and Differences transform in an MLForecast pipeline. Includes the required import for isinstance to work, preprocessing, and prediction steps.

```python
# need this import in order for isinstance to work
from mlforecast.target_transforms import Differences as ExportedDifferences
```

```python
sk_boxcox = PowerTransformer(method='box-cox', standardize=False)
boxcox_global = GlobalSklearnTransformer(sk_boxcox)
single_difference = ExportedDifferences([1])
series = generate_daily_series(10)
fcst = MLForecast(
    models=[LinearRegression(), HistGradientBoostingRegressor()],
    freq='D',
    lags=[1, 2],
    target_transforms=[boxcox_global, single_difference]
)
prep = fcst.preprocess(series, dropna=False)
expected = (
    pd.Series(
        sk_boxcox.fit_transform(series[['y']])[:, 0], index=series['unique_id']
    ).groupby('unique_id', observed=True)
    .diff()
    .dropna()
    .values
)
np.testing.assert_allclose(prep['y'].values, expected)
preds = fcst.fit(series).predict(5)
```

--------------------------------

### Install extras for polars and remote storage

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/install.html

Install mlforecast with extras for polars dataframes, or for saving artifacts to remote storages like S3, GCS, or Azure.

```bash
pip install "mlforecast[polars]"
```

```bash
pip install "mlforecast[aws]"
```

```bash
pip install "mlforecast[gcp]"
```

```bash
pip install "mlforecast[azure]"
```

--------------------------------

### Generate daily series for examples

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/predict_callbacks.html

Creates a daily series with one unique id using generate_daily_series. This series is used in all subsequent examples.

```python
series = generate_daily_series(1)
```

--------------------------------

### Install and import required libraries for cross-validation

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/cross_validation.html

Install MLForecast with pip and import pandas, utilsforecast.plotting, and MLForecast. Required to instantiate MLForecast object and use cross-validation method.

```python
import pandas as pd

from utilsforecast.plotting import plot_series

from mlforecast import MLForecast # required to instantiate MLForecast object and use cross-validation method
```

--------------------------------

### Import libraries for hyperparameter optimization

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/hyperparameter_optimization.html

Imports required for the guide, including MLForecast, Optuna, LightGBM, and utilities. The output shows a TqdmWarning about IProgress not found, which is non-fatal.

```python
import os
import tempfile
import time

import lightgbm as lgb
import optuna
import pandas as pd
from datasetsforecast.m4 import M4, M4Evaluation, M4Info
from sklearn.linear_model import Ridge
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder
from utilsforecast.plotting import plot_series

from mlforecast import MLForecast
from mlforecast.auto import (
    AutoLightGBM,
    AutoMLForecast,
    AutoModel,
    AutoRidge,
    ridge_space,
)
from mlforecast.lag_transforms import ExponentiallyWeightedMean, RollingMean
```

```text
/Users/janrathfelder/miniconda3/envs/mlforecast-dev/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

--------------------------------

### Imports for AutoMLForecast example

Source: https://nixtlaverse.nixtla.io/mlforecast/auto.html

Imports required for the AutoMLForecast example, including pandas, datasetsforecast M4 utilities, sklearn models, and preprocessing tools.

```python
import time

import pandas as pd
from datasetsforecast.m4 import M4, M4Evaluation, M4Info
from sklearn.linear_model import Ridge
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder
```

--------------------------------

### Setup: Import libraries and MLForecast components

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/transfer_conformal_prediction.html

Imports required libraries including lightgbm, numpy, pandas, matplotlib, sklearn's FunctionTransformer, and MLForecast modules for lag transforms, target transforms, and prediction intervals.

```python
import warnings
warnings.filterwarnings('ignore')

import lightgbm as lgb
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.preprocessing import FunctionTransformer

from mlforecast import MLForecast
from mlforecast.lag_transforms import (
    ExpandingMean,
    ExponentiallyWeightedMean,
    RollingMean,
    RollingStd,
    SeasonalRollingMean,
)
from mlforecast.target_transforms import Differences, GlobalSklearnTransformer
from mlforecast.utils import PredictionIntervals
```

--------------------------------

### Import generate_series and generate_prices_for_series

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/transforming_exog.html

Imports the utility functions used to generate sample series and price data for the example.

```python
from mlforecast.utils import generate_series, generate_prices_for_series
```

--------------------------------

### Import libraries for lag transformations

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/lag_transforms_guide.html

Imports numpy, MLForecast, and generate_daily_series for the examples.

```python
import numpy as np

from mlforecast import MLForecast
from mlforecast.utils import generate_daily_series
```

--------------------------------

### Import required libraries for MLForecast

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/electricity_load_forecasting.html

Imports LightGBM, scikit-learn models, and MLForecast transforms needed for the forecasting examples.

```python
import lightgbm as lgb
from sklearn.base import BaseEstimator
from sklearn.linear_model import Lasso, LinearRegression, Ridge
from sklearn.neighbors import KNeighborsRegressor
from sklearn.neural_network import MLPRegressor
from sklearn.ensemble import RandomForestRegressor

from mlforecast.lag_transforms import ExpandingMean, RollingMean
from mlforecast.target_transforms import Differences
```

--------------------------------

### Generate daily series data

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/lag_transforms_guide.html

Creates a sample dataset with 10 daily series for the examples.

```python
data = generate_daily_series(10)
```

--------------------------------

### Local partition with partition_by alone

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/pooled_lag_transforms.html

Creates an MLForecast with a RollingMean transform using partition_by=['promo'] and no global_ or groupby. Each (unique_id, promo) combination gets its own bucket. Preprocesses the series and shows the first 8 rows.

```python
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={1: [RollingMean(window_size=7, partition_by=["promo"])]},
)
fcst.preprocess(series).head(8)
```

--------------------------------

### Import required libraries for distributed MLForecast

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/quick_start_distributed.html

Imports the necessary libraries for the distributed quick start, including MLForecast distributed modules, feature engineering utilities, and data generation helpers.

```python
import platform
import sys
import tempfile

import matplotlib.pyplot as plt
import git
import numpy as np
import pandas as pd
import s3fs
from sklearn.base import BaseEstimator
from utilsforecast.feature_engineering import fourier

from mlforecast.distributed import DistributedMLForecast
from mlforecast.lag_transforms import ExpandingMean, ExponentiallyWeightedMean, RollingMean
from mlforecast.target_transforms import Differences
from mlforecast.utils import generate_daily_series, generate_prices_for_series
```

--------------------------------

### Import libraries for M4 data setup

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/end_to_end_walkthrough.html

Imports random, tempfile, Path, pandas, M4 from datasetsforecast, and plot_series from utilsforecast. These are used for loading and plotting the M4 hourly dataset.

```python
import random
import tempfile
from pathlib import Path

import pandas as pd
from datasetsforecast.m4 import M4
from utilsforecast.plotting import plot_series
```

--------------------------------

### Import required libraries for predict callbacks

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/predict_callbacks.html

Imports copy, lightgbm, numpy, IPython display, MLForecast, and generate_daily_series. These are used in the examples that follow.

```python
import copy

import lightgbm as lgb
import numpy as np
from IPython.display import display

from mlforecast import MLForecast
from mlforecast.utils import generate_daily_series
```

--------------------------------

### Preprocess with return_X_y and as_numpy

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/training_with_numpy.html

Get features and target as numpy arrays using preprocess with return_X_y=True and as_numpy=True.

```python
X, y = fcst.preprocess(series_pl, return_X_y=True, as_numpy=True)
X[:2]
```

--------------------------------

### LightGBMCV.setup

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Initializes internal data structures to iteratively train the boosters. Call this before using partial_fit.

```APIDOC
## LightGBMCV.setup

### Description
Initializes internal data structures to iteratively train the boosters. Use this before calling partial_fit.

### Method
setup

### Signature
setup(df, n_windows, h, id_col='unique_id', time_col='ds', target_col='y', step_size=None, params=None, static_features=None, dropna=True, keep_last_n=None, weights=None, metric='mape', input_size=None, weight_col=None)

### Parameters
- **df** (pandas DataFrame) - Required - Series data in long format.
- **n_windows** (int) - Required - Number of windows to evaluate.
- **h** (int) - Required - Forecast horizon.
- **id_col** (str) - Optional - Column that identifies each serie. Defaults to 'unique_id'.
- **time_col** (str) - Optional - Column that identifies each timestep, its values can be timestamps or integers. Defaults to 'ds'.
- **target_col** (str) - Optional - Column that contains the target. Defaults to 'y'.
- **step_size** (int) - Optional - Step size between each cross validation window. If None it will be equal to h. Defaults to None.
- **params** (dict) - Optional - Parameters to be passed to the LightGBM Boosters. Defaults to None.
- **static_features** (list of str) - Optional - Names of the features that are static and will be repeated when forecasting. Defaults to None.
- **dropna** (bool) - Optional - Drop rows with missing values produced by the transformations. Defaults to True.
- **keep_last_n** (int) - Optional - Keep only these many records from each serie for the forecasting step. Can save time and memory if your features allow it. Pooled lag transforms (global_/groupby/partition_by) with a window wider than this keep that wider window instead, since their shared aggregates have no per-series buffer to trim below it. Defaults to None.
- **weights** (sequence of float) - Optional - Weights to multiply the metric of each window. If None, all windows have the same weight. Defaults to None.
- **metric** (str or callable) - Optional - Metric used to assess the performance of the models and perform early stopping. Defaults to 'mape'.
- **input_size** (int) - Optional - Maximum training samples per serie in each window. If None, will use an expanding window. Defaults to None.
- **weight_col** (str) - Optional - Column containing sample weights. Higher weights increase the influence of those samples during fitting and evaluation.

### Returns
- **LightGBMCV** - CV object with internal data structures for partial_fit.
```

--------------------------------

### Get best configuration for AutoLightGBM

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/hyperparameter_optimization.html

Accesses the best trial's user attributes to retrieve the configuration dictionary for the AutoLightGBM model.

```python
auto_mlf.results_['AutoLightGBM'].best_trial.user_attrs['config']
```

--------------------------------

### End-to-end MLForecast with pooled lag transforms

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/pooled_lag_transforms.html

This example shows how to configure an MLForecast model with pooled lag transforms (RollingMean, ExpandingMean, ExponentiallyWeightedMean) using global_ and groupby parameters. It fits the model on the series without the 'promo' column and then predicts 7 steps ahead, displaying the first 6 rows of the forecast.

```python
from sklearn.linear_model import LinearRegression

fcst = MLForecast(
    models=[LinearRegression()],
    freq="D",
    lags=[1, 7],
    lag_transforms={
        1: [
            RollingMean(window_size=7, global_=True),
            RollingMean(window_size=7, groupby=["brand"]),
            ExpandingMean(groupby=["brand"]),
            ExponentiallyWeightedMean(alpha=0.3, global_=True),
        ],
    },
)
# `promo` was only needed for the partition_by examples above; this end-to-end
# model groups by `brand` and doesn't use it, so drop it to keep predict X_df-free
fcst.fit(series.drop(columns="promo"), static_features=["brand"])
fcst.predict(h=7).head(6)
```

--------------------------------

### Preprocess data with MLForecast

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/sklearn_pipelines.html

Create an MLForecast instance with lags and date features, then preprocess the series to get the feature matrix.

```python
fcst = MLForecast(
    models=[],
    freq='D',
    lags=[1],
    date_features=['dayofweek']
)
X, y = fcst.preprocess(series, return_X_y=True)
X.head()
```

--------------------------------

### Import libraries for pooled lag transforms

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/pooled_lag_transforms.html

Imports warnings, numpy, pandas, MLForecast, and the lag transforms (ExpandingMean, ExponentiallyWeightedMean, RollingMean) needed for the examples.

```python
import warnings

import numpy as np
import pandas as pd

from mlforecast import MLForecast
from mlforecast.lag_transforms import (
    ExpandingMean,
    ExponentiallyWeightedMean,
    RollingMean,
)
from mlforecast.utils import generate_daily_series
```

--------------------------------

### Download and sample M4 hourly data

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Downloads the M4 hourly dataset, loads it, and selects a random sample of 4 series for the example.

```python
group = 'Hourly'
await M4.async_download('data', group=group)
df, *_ = M4.load(directory='data', group=group)
df['ds'] = df['ds'].astype('int')
ids = df['unique_id'].unique()
random.seed(0)
sample_ids = random.choices(ids, k=4)
sample_df = df[df['unique_id'].isin(sample_ids)]
sample_df
```

--------------------------------

### Differences usage and testing example

Source: https://nixtlaverse.nixtla.io/mlforecast/target_transforms.html

Demonstrates fitting, transforming, updating, and stacking the Differences transform on generated series, including assertions for correctness and handling of short series.

```python
series = generate_daily_series(10, min_length=50, max_length=100)


diffs = Differences([1, 2, 5])
id_counts = counts_by_id(series, 'unique_id')
indptr = np.append(0, id_counts['counts'].cumsum())
ga = GroupedArray(series['y'].values, indptr)

# differences are applied correctly
transformed = diffs.fit_transform(ga)
assert diffs.fitted_ == []
expected = series.copy()
for d in diffs.differences:
    expected['y'] -= expected.groupby('unique_id', observed=True)['y'].shift(d)
np.testing.assert_allclose(transformed.data, expected['y'].values)

# fitted differences are restored correctly
diffs.store_fitted = True
transformed = diffs.fit_transform(ga)
keep_mask = ~np.isnan(transformed.data)
restored = diffs.inverse_transform_fitted(transformed)
np.testing.assert_allclose(ga.data[keep_mask], restored.data[keep_mask])

# test transform
new_ga = GroupedArray(np.random.rand(10), np.arange(11))
prev_orig = [diffs.scalers_[i].tails_[::d].copy() for i, d in enumerate(diffs.differences)]
expected = new_ga.data - np.add.reduce(prev_orig)
updates = diffs.update(new_ga)
np.testing.assert_allclose(expected, updates.data)
np.testing.assert_allclose(diffs.scalers_[0].tails_, new_ga.data)
np.testing.assert_allclose(diffs.scalers_[1].tails_[1::2], new_ga.data - prev_orig[0])
np.testing.assert_allclose(diffs.scalers_[2].tails_[4::5], new_ga.data - np.add.reduce(prev_orig[:2]))
# variable sizes
diff1 = Differences([1])
ga = GroupedArray(np.arange(10), np.array([0, 3, 10]))
diff1.fit_transform(ga)
new_ga = GroupedArray(np.arange(4), np.array([0, 1, 4]))
updates = diff1.update(new_ga)
np.testing.assert_allclose(updates.data, np.array([0 - 2, 1 - 9, 2 - 1, 3 - 2]))
np.testing.assert_allclose(diff1.scalers_[0].tails_, np.array([0, 3]))

# short series
ga = GroupedArray(np.arange(20), np.array([0, 2, 20]))
test_fail(lambda: diffs.fit_transform(ga), contains="[0]")

# stack
diffs = Differences([1, 2, 5])
ga = GroupedArray(series['y'].values, indptr)
diffs.fit_transform(ga)
stacked = Differences.stack([diffs, diffs])
for i in range(len(diffs.differences)):
    np.testing.assert_allclose(
        stacked.scalers_[i].tails_,
        np.tile(diffs.scalers_[i].tails_, 2)
    )
```

--------------------------------

### Get predictions from all trained models

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Calls predict to get forecasts from every booster trained during cross-validation.

```python
preds = cv.predict(horizon)
preds
```

--------------------------------

### Train MLForecast model with lag and date features

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/electricity_load_forecasting.html

Configures an MLForecast model with LightGBM and Lasso regressors, hourly frequency, lag features, date features, and a single thread. This is the initial model setup for the forecasting workflow.

```python
MLForecast(models=[lgbm, lasso], freq=H, lag_features=['lag1', 'lag12', 'lag24', 'expanding_mean_lag1', 'rolling_mean_lag24_window_size48'], date_features=['month', 'hour', 'dayofweek'], num_threads=1)
```

--------------------------------

### Install specific version from conda-forge

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/install.html

Use version filters to install a specific version or any version prior to a given version from conda-forge.

```bash
conda install -c conda-forge "mlforecast==0.3.0"
```

```bash
conda install -c conda-forge "mlforecast<0.4.0"
```

--------------------------------

### MLForecast.get_missing_future

Source: https://nixtlaverse.nixtla.io/mlforecast/forecast.html

Gets the missing id and time combinations in X_df.

```APIDOC
## MLForecast.get_missing_future

### Description
Gets the missing id and time combinations in X_df.

### Method
get_missing_future

### Signature
get_missing_future(h, X_df)

### Parameters
- **h** (int) - Required - Number of periods to predict.
- **X_df** (pandas or polars DataFrame) - Required - Dataframe with the future exogenous features. Should have the id column and the time column.

### Returns
- **DFType** (pandas or polars DataFrame) - DataFrame with expected ids and future times missing in X_df.

```

--------------------------------

### Import LightGBMCV

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/end_to_end_walkthrough.html

Import the LightGBMCV class from mlforecast.lgb_cv.

```python
from mlforecast.lgb_cv import LightGBMCV
```

--------------------------------

### Create MLForecast for preprocess

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/training_with_numpy.html

Create an MLForecast instance with no models, daily frequency, lags, and weekday date feature for preprocessing.

```python
fcst = MLForecast(
    models=[],
    freq='1d',
    lags=[7, 14],
    date_features=['weekday'],
)
```

--------------------------------

### Get feature names after one-hot encoding

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/sklearn_pipelines.html

Display the feature names generated by the OneHotEncoder.

```python
ohe.get_feature_names_out()
```

--------------------------------

### Get MLflow artifact URI

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/mlflow.html

Retrieves the artifact URI for the current MLflow run.

```python
mlflow.get_artifact_uri()
```

--------------------------------

### Generate training set with preprocess

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/custom_training.html

Use MLForecast.preprocess to generate the training data and display the first rows.

```python
prep = fcst.preprocess(series)
prep.head()
```

--------------------------------

### Configure MLForecast with LightGBM, lags, and transforms

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/m4.html

Sets up an MLForecast model with a LightGBM regressor, hourly frequency, lags at 24-hour intervals, an exponentially weighted mean lag transform on lag 48, and a differencing target transform. Uses a single thread.

```python
fcst = MLForecast(
    models=lgb.LGBMRegressor(random_state=0, verbosity=-1),
    freq=1,
    lags=[24 * (i+1) for i in range(7)],
    lag_transforms={
        48: [ExponentiallyWeightedMean(alpha=0.3)],
    },
    num_threads=1,
    target_transforms=[Differences([24])],
)
```

--------------------------------

### MLflow artifact URI output

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/mlflow.html

Example output of mlflow.get_artifact_uri() showing the artifact path.

```text
'file:///Users/deven367/projects/public/mlforecast/nbs/docs/how-to-guides/mlruns/135880152043890861/2a94793bbbe244f09dd68961d5883f5b/artifacts'
```

--------------------------------

### LightGBMCV training output with early stopping

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Shows the training log with MAPE per iteration and early stopping behavior.

```text
[LightGBM] [Info] Start training from score 51.745632
[10] mape: 0.590690
[20] mape: 0.251093
[30] mape: 0.143643
[40] mape: 0.109723
[50] mape: 0.102099
[60] mape: 0.099448
[70] mape: 0.098349
[80] mape: 0.098006
[90] mape: 0.098718
Early stopping at round 90
Using best iteration: 80
```

--------------------------------

### Get last values of each series

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/end_to_end_walkthrough.html

Extracts the last row for each unique_id from the dataframe to compare with the predictions.

```python
last_vals = df.groupby('unique_id').tail(1)
last_vals
```

--------------------------------

### Usage with MLForecast

Source: https://nixtlaverse.nixtla.io/mlforecast/lag_transforms.html

Example showing how to pass lag transformations to MLForecast using the lag_transforms argument.

```APIDOC
## Usage with MLForecast

### Description
Demonstrates how to configure MLForecast with lag transformations.

### Code Example
```python
from mlforecast import MLForecast
from mlforecast.lag_transforms import ExpandingStd, RollingMean

fcst = MLForecast(
    models=[...],
    freq='D',
    lag_transforms={
        1: [ExpandingStd()],
        7: [RollingMean(window_size=7), RollingMean(window_size=28)],
    },
)
```
```

--------------------------------

### Prediction results table

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/mlflow.html

Example output of results.head(2) showing predictions for unique_id 3.

```text
| | unique_id | ds | lgb | lr |
| --- | --- | --- | --- | --- |
| 0 | 3 | 2000-01-10 16:00:00 | 22.187576 | 22.168456 |
| 1 | 3 | 2000-01-10 17:00:00 | 23.204075 | 23.413620 |
```

--------------------------------

### Import MLForecast and scikit-learn components

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/sklearn_pipelines.html

Import the necessary classes for building the pipeline.

```python
from mlforecast import MLForecast
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder
```

--------------------------------

### Create and fit MLForecast with LightGBM

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/electricity_load_forecasting.html

Configures an MLForecast model with LightGBM, hourly frequency, target transforms, lags, lag transforms, and date features. Fits the model with prediction intervals and generates 24-step forecasts with 90% levels.

```python
models_comparison ={
    'lgbm': lgb.LGBMRegressor(verbosity=-1)
}

mlf_comparison = MLForecast(
    models = models_comparison,
    freq='H',  # our series have hourly frequency
    target_transforms=[Differences([24, 24*7])],
    lags=[1,12,24],
    lag_transforms={
        1: [ExpandingMean()],
        24: [RollingMean(window_size=48)],
    },
    date_features=['month', 'hour', 'dayofweek']
)

init = time()
mlf_comparison.fit(
    df = df_train,
    prediction_intervals=PredictionIntervals(n_windows=4, h=24)
)

levels = [90]
forecasts_comparison = mlf_comparison.predict(24, level=levels)
end = time()
forecasts_comparison.head()
```

--------------------------------

### Configure and fit LightGBMCV

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/end_to_end_walkthrough.html

Create a LightGBMCV instance with specified lags, lag transforms, date features, and target transforms, then fit it with cross-validation parameters. The output shows per-iteration MAPE and early stopping at round 30, using best iteration 5.

```python
cv = LightGBMCV(
    freq=1,
    target_transforms=[Differences([24])],
    lags=[1, 24],
    lag_transforms={
        1: [ExpandingMean()],
        24: [RollingMean(window_size=48)],
    },
    date_features=[hour_index],
    num_threads=2,
)
cv_hist = cv.fit(
    df,
    n_windows=4,
    h=48,
    params=lgb_params,
    eval_every=5,
    early_stopping_evals=5,    
    compute_cv_preds=True,
)
```

```text
[5] mape: 0.158639
[10] mape: 0.163739
[15] mape: 0.161535
[20] mape: 0.169491
[25] mape: 0.163690
[30] mape: 0.164198
Early stopping at round 30
Using best iteration: 5
```

--------------------------------

### Extract feature matrix X

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/analyzing_models.html

Drops the info columns (unique_id, ds, y) to get the feature matrix.

```python
X = prep.drop(columns=['unique_id', 'ds', 'y'])
X.head()
```

--------------------------------

### Initialize Ray Cluster with Mock Node

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/getting-started/quick_start_distributed.html

Creates a Ray cluster with a head node and adds a mock node to simulate a cluster. The cluster address is used to initialize Ray.

```python
# ray
ray_cluster = Cluster(
    initialize_head=True,
    head_node_args={"num_cpus": 2}
)
ray.init(address=ray_cluster.address, ignore_reinit_error=True)
# add mock node to simulate a cluster
mock_node = ray_cluster.add_node(num_cpus=2)
```

--------------------------------

### Create and fit LightGBMCV

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/m4.html

Creates a LightGBMCV object with specified lags, lag transforms, and target transforms, then fits it on the training data with two windows and a horizon. The output shows the MAPE at each boosting round and early stopping at round 50, using the best iteration at round 40.

```python
cv = LightGBMCV(
    freq=1,
    lags=[24 * (i+1) for i in range(7)],
    lag_transforms={
        48: [ExponentiallyWeightedMean(alpha=0.3)],
    },
    num_threads=1,
    target_transforms=[Differences([24])]
)
hist = cv.fit(
    train,
    n_windows=2,
    h=horizon,
    params={'verbosity': -1},
)
```

```text
[10] mape: 0.118569
[20] mape: 0.111506
[30] mape: 0.107314
[40] mape: 0.106089
[50] mape: 0.106630
Early stopping at round 50
Using best iteration: 40
```

--------------------------------

### Configure and instantiate LightGBMCV with lags

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Sets up the cross-validation configuration and creates a LightGBMCV model with a week of hourly lags.

```python
static_fit_config = dict(
    n_windows=2,
    h=horizon,
    params={'verbose': -1},
    compute_cv_preds=True,
)
cv = LightGBMCV(
    freq=1,
    lags=[24 * (i+1) for i in range(7)],  # one week of lags
)
```

--------------------------------

### Get number of optimization results

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/hyperparameter_optimization.html

Returns the number of optimization results stored in the results_ attribute of the AutoMLForecast object.

```python
len(auto_mlf.results_)
```

--------------------------------

### Create and fit Prophet model, generate forecasts

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/electricity_load_forecasting.html

Creates a Prophet model with 90% interval width, fits it on the training data, generates hourly forecasts for the last 24 hours, and formats the output columns.

```python
# create prophet model
prophet = Prophet(interval_width=0.9)
init = time()
prophet.fit(df_train)
# produce forecasts
future = prophet.make_future_dataframe(periods=len(df_last_24_hours), freq='H', include_history=False)
forecast_prophet = prophet.predict(future)
end = time()
# data wrangling
forecast_prophet = forecast_prophet[['ds', 'yhat', 'yhat_lower', 'yhat_upper']]
forecast_prophet.columns = ['ds', 'Prophet', 'Prophet-lo-90', 'Prophet-hi-90']
forecast_prophet.insert(0, 'unique_id', 'PJM_Load_hourly')
forecast_prophet.head()
```

--------------------------------

### Create MLForecast object with models

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/sample_weights.html

Instantiates MLForecast with linear regression and LightGBM models, daily frequency, one lag, and day-of-week date features.

```python
fcst = MLForecast(
    models={
        'lr': LinearRegression(),
        'lgbm': lgb.LGBMRegressor(verbosity=-1),
    },
    freq='D',
    lags=[1],
    date_features=['dayofweek'],
)
```

--------------------------------

### Get fitted values from MLForecast model

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/prediction_intervals_in_forecasting_models.html

Retrieves the fitted values from the MLForecast model and sets the index to 'unique_id' for display.

```python
result = mlf.forecast_fitted_values()
result = result.set_index("unique_id")
result
```

--------------------------------

### Download and load M4 Hourly data

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/target_transforms_guide.html

Downloads the M4 Hourly dataset and loads a single series (H196) for the examples.

```python
data_path = 'data'
await M4.async_download(data_path, group='Hourly')
df, *_ = M4.load(data_path, 'Hourly')
df['ds'] = df['ds'].astype('int32')
serie = df[df['unique_id'].eq('H196')]
```

--------------------------------

### Default optimization with AutoMLForecast

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/hyperparameter_optimization.html

Sets up AutoMLForecast with default models (AutoLightGBM and AutoRidge) and fits it on hourly data with season_length=24. The verbosity of optuna is set to ERROR to suppress logging. The fit uses n_windows=2, h=horizon, and num_samples=2 for the number of trials.

```python
optuna.logging.set_verbosity(optuna.logging.ERROR)
auto_mlf = AutoMLForecast(
    models={'lgb': AutoLightGBM(), 'ridge': AutoRidge()},
    freq=1,
    season_length=24,
)
auto_mlf.fit(
    train,
    n_windows=2,
    h=horizon,
    num_samples=2,  # number of trials to run
)
```

```python
AutoMLForecast(models={'lgb': AutoModel(model=LGBMRegressor), 'ridge': AutoModel(model=Ridge)})
```

```python
preds = auto_mlf.predict(horizon)
preds.head()
```

```python
def evaluate(df, group):
    results = []
    for model in df.columns.drop(['unique_id', 'ds']):
        model_res = M4Evaluation.evaluate(
            'data', group, df[model].to_numpy().reshape(-1, horizon)
        )
        model_res.index = [model]
        results.append(model_res)
    return pd.concat(results).T.round(2)

evaluate(preds, group)
```

--------------------------------

### Get Fitted Values with 95% Confidence Level

Source: https://nixtlaverse.nixtla.io/mlforecast/auto.html

Retrieve fitted values with 95% prediction intervals from the fitted AutoMLForecast model.

```python
auto_mlf.forecast_fitted_values(level=[95])
```

--------------------------------

### Partial fit for 20 more iterations

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Trains the model for 20 more iterations and asserts the score matches the third metric from the first example.

```python
score2 = cv4.partial_fit(20)
```

```python
assert hist[2][1] == score2
```

--------------------------------

### Import MLForecast and LightGBM

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/electricity_peak_forecasting.html

Imports the MLForecast class, the Differences target transform, and the LightGBM regressor.

```python
import lightgbm as lgb

from mlforecast import MLForecast
from mlforecast.target_transforms import Differences
```

--------------------------------

### Log MLForecast model to MLflow with metrics

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/mlflow.html

Sets up an MLflow experiment, logs training and validation datasets, logs parameters (with model class names and params), fits the model, evaluates predictions, logs metrics, and logs the model using mlforecast.flavor.log_model.

```python
mlflow.set_experiment("mlforecast")
with mlflow.start_run() as run:
    train_ds = mlflow.data.from_pandas(train)
    valid_ds = mlflow.data.from_pandas(valid)
    mlflow.log_input(train_ds, context="training")
    mlflow.log_input(valid_ds, context="validation")
    logged_params = copy.deepcopy(params)
    logged_params['init']['models'] = {
        k: (v.__class__.__name__, v.get_params())
        for k, v in params['init']['models'].items()
    }
    mlflow.log_params(logged_params)
    mlf = MLForecast(**params['init'])
    mlf.fit(train, **params['fit'])
    preds = mlf.predict(h, X_df=X_df)
    eval_result = evaluate(
        valid.merge(preds, on=['unique_id', 'ds']),
        metrics=[rmse, smape],
        agg_fn='mean',
    )
    models = mlf.models_.keys()
    logged_metrics = {}
    for _, row in eval_result.iterrows():
        metric = row['metric']
        for model in models:
            logged_metrics[f'{metric}_{model}'] = row[model]
    mlflow.log_metrics(logged_metrics)
    mlforecast.flavor.log_model(model=mlf, artifact_path="model", registered_model_name=None)
    model_uri = mlflow.get_artifact_uri("model")
    run_id = run.info.run_id
```

--------------------------------

### Partial fit for 10 iterations

Source: https://nixtlaverse.nixtla.io/mlforecast/lgb_cv.html

Trains the model for 10 iterations and returns the score, which matches the first evaluation from the first example.

```python
score = cv4.partial_fit(10)
score
```

```text
[LightGBM] [Info] Start training from score 51.745632
```

```text
0.5906900462828166
```

```python
assert hist[0][1] == score
```

--------------------------------

### Create MLForecast forecaster

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/m4.html

Creates an MLForecast object with LightGBM model, lags, lag transforms, and target transforms for the M4 example.

```python
fcst = MLForecast(
    models=lgb.LGBMRegressor(random_state=0, verbosity=-1),
    freq=1,
    lags=[24 * (i+1) for i in range(7)],
    lag_transforms={
        48: [ExponentiallyWeightedMean(alpha=0.3)],
    },
    num_threads=1,
    target_transforms=[Differences([24])],
)
```

```python
fcst
```

```text
MLForecast(models=[LGBMRegressor], freq=1, lag_features=['lag24', 'lag48', 'lag72', 'lag96', 'lag120', 'lag144', 'lag168', 'exponentially_weighted_mean_lag48_alpha0.3'], date_features=[], num_threads=1)
```

--------------------------------

### catboost_space(trial)

Source: https://nixtlaverse.nixtla.io/mlforecast/auto.html

Function that takes an Optuna trial and produces a configuration for a CatBoost model.

```APIDOC
## catboost_space(trial)

### Description
Function that takes an Optuna trial and produces a configuration for a CatBoost model.

### Method
Function call

### Endpoint
catboost_space(trial)

### Parameters
- **trial** (optuna.trial.Trial) - Required - The Optuna trial object used to suggest hyperparameters.

### Returns
Configuration dictionary for the CatBoost model.
```

--------------------------------

### Instantiate MLForecast with parameters

Source: https://nixtlaverse.nixtla.io/mlforecast/docs/tutorials/electricity_peak_forecasting.html

Creates an MLForecast object with the models, hourly frequency, a Differences target transform with lag 24, and lags from 1 to 24.

```python
# Instantiate MLForecast class as mlf
mlf = MLForecast(
    models=models,
    freq='H', 
    target_transforms=[Differences([24])],
    lags=range(1, 25)
)
```

--------------------------------

### Select a single time series

Source: https://nixtlaverse.nixtla.io/mlforecast/core.html

Filters the generated series to keep only the first unique_id, creating a single-series dataframe for simpler examples.

```python
uids = series['unique_id'].unique()
serie = series[series['unique_id'].eq(uids[0])]
serie
```