### Install Dependencies and Initialize Project

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-HuggingFace.ipynb

Installs necessary Python packages for the project and initializes Git and DVC for experiment tracking. Ensures the environment is ready for development and tracking.

```python
from datasets import load_dataset
from transformers import AutoTokenizer

dataset = load_dataset("imdb")
```

```python
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-cased")

def tokenize_function(examples):
    return tokenizer(examples["text"], padding="max_length", truncation=True)

small_train_dataset = (
    dataset["train"]
    .shuffle(seed=42)
    .select(range(2000))
    .map(tokenize_function, batched=True)
)
small_eval_dataset = (
    dataset["test"]
    .shuffle(seed=42)
    .select(range(200))
    .map(tokenize_function, batched=True)
)
```

```python
import evaluate
import numpy as np

metric = evaluate.load("f1")

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return metric.compute(predictions=predictions, references=labels)
```

--------------------------------

### Install DVCLive for Lightning

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-PyTorch-Lightning.ipynb

Installs the necessary DVCLive package with Lightning support using pip.

```bash
%pip install "dvclive[lightning]"
```

--------------------------------

### Initialize and Log Metrics with DVCLive

Source: https://github.com/treeverse/dvclive/wiki/General

Demonstrates the standard workflow for initializing the logger, recording metrics during a training loop, and finalizing steps. Requires the dvclive library to be installed and configured.

```python
from dvclive import Live

# Initialize the logger
live = Live(directory="logs")

# Training loop example
for epoch in range(10):
    # Log metrics
    live.log("accuracy", 0.95)
    live.log("loss", 0.05)
    
    # Signal the end of a step
    live.next_step()
```

--------------------------------

### Install DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Quickstart.ipynb

Installs the dvclive library using pip. This is the first step to using DVCLive in your Python environment.

```python
%pip install dvclive
```

--------------------------------

### Install DVCLive and Scikit-learn

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-scikit-learn.ipynb

Installs the necessary Python packages, dvclive and scikit-learn, using pip.

```python
!pip install dvclive scikit-learn
```

--------------------------------

### Install DVCLive and Ultralytics

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-YOLO.ipynb

Installs the necessary Python packages for DVCLive and Ultralytics YOLOv8. It also checks the Ultralytics installation.

```python
%pip install dvclive ultralytics
import ultralytics

ultralytics.checks()
```

--------------------------------

### Initialize DVC Repository

Source: https://github.com/treeverse/dvclive/blob/main/README.md

These commands initialize a new DVC repository in your project directory and commit the initial DVC setup to Git.

```console
git init
dvc init
git commit -m "DVC init"
```

--------------------------------

### Install DVCLive and Dependencies

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Installs or uninstalls specific versions of DVC, DVCLive, Evidently, and pandas. The %%capture magic command suppresses output during installation.

```python
!pip uninstall -q -y sqlalchemy pyarrow ipython-sql pandas-gbq
```

```python
%%capture
!pip install -q dvc==3.25.0 dvclive==3.0.1 evidently==0.4.5 pandas==1.5.3
```

--------------------------------

### Install DVCLive using pip

Source: https://github.com/treeverse/dvclive/blob/main/README.md

This command installs the DVCLive Python library using pip, making it available for use in your projects.

```console
pip install dvclive
```

--------------------------------

### Train YOLOv8 Models with DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-YOLO.ipynb

Trains YOLOv8 models with specified configurations (model size, image size, epochs). DVCLive automatically logs experiment metrics when installed.

```bash
!yolo train model=yolov8n.pt data=coco8.yaml epochs=5 imgsz=512
!yolo train model=yolov8n.pt data=coco8.yaml epochs=5 imgsz=640
!yolo train model=yolov8n.pt data=coco8.yaml epochs=10 imgsz=640
!yolo train model=yolov8s.pt data=coco8.yaml epochs=10 imgsz=640
!yolo train model=yolov8m.pt data=coco8.yaml epochs=10 imgsz=640
```

--------------------------------

### Train Model and Log Experiments with DVCLiveCallback

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-HuggingFace.ipynb

Fine-tunes a pre-trained model for sequence classification and integrates DVCLiveCallback to automatically log experiments, including model artifacts, during training. This setup facilitates experiment tracking and comparison.

```python
%env HF_DVCLIVE_LOG_MODEL=true
```

```python
from transformers import AutoModelForSequenceClassification, Trainer, TrainingArguments
from transformers.integrations import DVCLiveCallback

model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-cased", num_labels=2
)
for param in model.base_model.parameters():
    param.requires_grad = False

lr = 3e-4

training_args = TrainingArguments(
    eval_strategy="epoch",
    learning_rate=lr,
    logging_strategy="epoch",
    num_train_epochs=5,
    output_dir="output",
    overwrite_output_dir=True,
    load_best_model_at_end=True,
    save_strategy="epoch",
    weight_decay=0.01,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=small_train_dataset,
    eval_dataset=small_eval_dataset,
    compute_metrics=compute_metrics,
)
trainer.train()
```

```python
from dvclive import Live

lr = 1e-4

training_args = TrainingArguments(
    eval_strategy="epoch",
    learning_rate=lr,
    logging_strategy="epoch",
    num_train_epochs=5,
    output_dir="output",
    overwrite_output_dir=True,
    load_best_model_at_end=True,
    save_strategy="epoch",
    weight_decay=0.01,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=small_train_dataset,
    eval_dataset=small_eval_dataset,
    compute_metrics=compute_metrics,
    callbacks=[DVCLiveCallback(live=Live(report="notebook"), log_model=True)],
)
trainer.train()
```

--------------------------------

### Initialize Git and DVC Repository

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Sets up a new directory as a Git repository and initializes DVC for version control. This is the foundational step for tracking experiments with DVC.

```bash
!rm -rf experiments && mkdir experiments
%cd experiments
!git init
!git add .gitignore
!git commit -m "Init repo"
!dvc init
!git commit -m "Init DVC"
```

--------------------------------

### Initialize Git and DVC Repository

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Sets up a new directory as a Git repository and initializes DVC for version control. This is the foundational step for tracking experiments.

```bash
!rm -rf experiments && mkdir experiments
cd experiments
git init
git add .gitignore
git commit -m "Init repo"
dvc init
git commit -m "Init DVC"
```

--------------------------------

### Configure Hyperparameters for Training

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Fabric.ipynb

Demonstrates the initialization of training hyperparameters using SimpleNamespace to pass configuration settings into the training loop.

```python
hparams = SimpleNamespace(
    batch_size=64,
    epochs=5,
    lr=1.0,
    gamma=0.7,
    dry_run=False,
    seed=1,
    log_interval=10,
    save_model=True,
)
run(hparams)
```

--------------------------------

### Initialize DVCLive Logger

Source: https://context7.com/treeverse/dvclive/llms.txt

Demonstrates how to initialize the Live class as a context manager for tracking experiments. It covers basic usage and advanced configuration options like output directories, report generation, and system monitoring.

```python
from dvclive import Live

# Basic usage with context manager (recommended)
with Live() as live:
    live.log_param("learning_rate", 0.001)
    live.log_param("epochs", 100)

    for epoch in range(100):
        train_loss = train_one_epoch()
        val_loss = validate()

        live.log_metric("train/loss", train_loss)
        live.log_metric("val/loss", val_loss)
        live.next_step()

# Advanced configuration
with Live(
    dir="experiments/run_001",
    resume=True,
    report="html",
    save_dvc_exp=True,
    dvcyaml="dvc.yaml",
    cache_images=True,
    exp_name="my_experiment",
    exp_message="Initial training",
    monitor_system=True
) as live:
    pass
```

--------------------------------

### Initialize DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Imports the DVCLive library, preparing it for logging metrics and parameters during experiments.

```python
from dvclive import Live
```

--------------------------------

### Log Training Metrics and Artifacts with DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Quickstart.ipynb

Demonstrates how to use the Live context manager to log hyperparameters, metrics, confusion matrices, and images during a training loop. It also shows how to save the best model state and finalize the experiment step.

```python
params = {"epochs": 5, "lr": 0.003, "weight_decay": 0}
best_test_acc = 0
with Live(report="notebook") as live:
    live.log_params(params)
    for _ in range(params["epochs"]):
        train_one_epoch(model, criterion, x_train, y_train, params["lr"], params["weight_decay"])
        metrics_train, acual_train, predicted_train = evaluate(model, x_train, y_train)
        for k, v in metrics_train.items():
            live.log_metric(f"train/{k}", v)
        live.log_sklearn_plot("confusion_matrix", acual_train, predicted_train, name="train/confusion_matrix")
        metrics_test, actual, predicted = evaluate(model, x_test, y_test)
        for k, v in metrics_test.items():
            live.log_metric(f"test/{k}", v)
        live.log_sklearn_plot("confusion_matrix", actual, predicted, name="test/confusion_matrix")
        live.log_image("misclassified.jpg", get_missclassified_image(actual, predicted, mnist_test))
        if metrics_test["acc"] > best_test_acc:
            torch.save(model.state_dict(), "model.pt")
        live.next_step()
    live.log_artifact("model.pt")
```

--------------------------------

### Download and Prepare Raw Data

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Creates a 'raw_data' directory, downloads the bike sharing dataset using wget, and unzips the downloaded file to extract CSVs and a readme.

```bash
!mkdir raw_data && \
 cd raw_data && \
 wget https://archive.ics.uci.edu/static/public/275/bike+sharing+dataset.zip && \
 unzip bike+sharing+dataset.zip
```

--------------------------------

### Initialize and Log Metrics Manually with DVCLive

Source: https://github.com/treeverse/dvclive/wiki/Home

Demonstrates how to initialize the DVCLive logger and manually log custom metrics within a training loop. This approach is useful for tracking arbitrary scalar values over time steps.

```python
import dvclive
import time

dvclive.init("logs", summarize=True)

def metric(i):
    return 1 - 1/(1+i)

def loss(i):
    return 1-(metric(i))

for i in range(100):
    dvclive.log("metric", metric(i))
    dvclive.log("loss", loss(i))

    dvclive.next_step()
    time.sleep(3)
```

--------------------------------

### DVCLive Basic API Usage (Python)

Source: https://github.com/treeverse/dvclive/blob/main/README.md

This Python script demonstrates the basic usage of the DVCLive library. It logs parameters, simulates a training loop, and logs metrics like accuracy and loss for each step. The `Live` context manager handles the logging process.

```python
import time
import random

from dvclive import Live

params = {"learning_rate": 0.002, "optimizer": "Adam", "epochs": 20}

with Live() as live:

    # log a parameters
    for param in params:
        live.log_param(param, params[param])

    # simulate training
    offset = random.uniform(0.2, 0.1)
    for epoch in range(1, params["epochs"]):
        fuzz = random.uniform(0.01, 0.1)
        accuracy = 1 - (2 ** - epoch) - fuzz - offset
        loss = (2 ** - epoch) + fuzz + offset

        # log metrics to studio
        live.log_metric("accuracy", accuracy)
        live.log_metric("loss", loss)
        live.next_step()
        time.sleep(0.2)
```

--------------------------------

### Log Hyperparameters with DVCLive

Source: https://context7.com/treeverse/dvclive/llms.txt

Explains how to log configuration settings and hyperparameters to a params.yaml file. Supports individual key-value pairs, dictionaries, and nested structures.

```python
from dvclive import Live

with Live() as live:
    live.log_param("learning_rate", 0.001)
    live.log_param("optimizer", "Adam")
    live.log_param("batch_size", 32)

    live.log_params({
        "model": "ResNet50",
        "epochs": 100,
        "dropout": 0.5,
        "augmentation": True,
        "layers": [64, 128, 256],
        "config": {
            "weight_decay": 0.0001,
            "momentum": 0.9
        }
    })
```

--------------------------------

### Initialize DVC Environment

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-PyTorch-Lightning.ipynb

Initializes a Git repository and DVC project to track experiment metadata.

```bash
!git init -q
!git config --local user.email "you@example.com"
!git config --local user.name "Your Name"
!dvc init -q
!git commit -m "DVC init"
```

--------------------------------

### Log Metrics and Parameters with DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Demonstrates using the DVCLive library to log parameters and metrics within a training loop. It iterates through batches and records drift metrics for each step.

```python
from dvclive import Live

for step, date in enumerate(experiment_batches):
    with Live() as live:
        live.log_param("step", step)
        live.log_param("begin", date[0])
        live.log_param("end", date[1])

        metrics = eval_drift(
            df.loc[df.dteday.between(reference_dates[0], reference_dates[1])],
            df.loc[df.dteday.between(date[0], date[1])],
            column_mapping=data_columns,
        )

        for feature in metrics:
            live.log_metric(feature[0], round(feature[1], 3))
```

--------------------------------

### Log Images with DVCLive

Source: https://context7.com/treeverse/dvclive/llms.txt

Demonstrates how to log various image formats including NumPy arrays, PIL images, matplotlib figures, and local file paths. It also shows how to create image sliders by logging sequential images across training steps.

```python
from dvclive import Live
import numpy as np
from PIL import Image as PILImage
import matplotlib.pyplot as plt

with Live(cache_images=True) as live:
    image_array = np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8)
    live.log_image("random_image.png", image_array)

    pil_image = PILImage.open("sample.jpg")
    live.log_image("sample_processed.jpg", pil_image)

    fig, ax = plt.subplots()
    ax.plot([1, 2, 3, 4], [1, 4, 2, 3])
    ax.set_title("Training Progress")
    live.log_image("training_plot.png", fig)
    plt.close(fig)

    live.log_image("input_sample", "data/images/sample.png")

    for step in range(10):
        generated_image = generate_sample()
        live.log_image(f"generated/{step}.png", generated_image)
        live.next_step()
```

--------------------------------

### DVCLive Logger Integration and Training Loop (PyTorch Lightning Fabric)

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Fabric.ipynb

Sets up DVCLiveLogger, initializes PyTorch Lightning Fabric, and prepares data loaders and the model for distributed training. It includes the main training and testing loops, logging hyperparameters and metrics (loss, accuracy) using DVCLiveLogger, and handling learning rate scheduling. The code is adapted for distributed environments using fabric.setup and fabric.backward.

```python
def run(hparams):
    # Create the DVCLive Logger
    logger = DVCLiveLogger(report="notebook")

    # Log dict of hyperparameters
    logger.log_hyperparams(hparams.__dict__)

    # Create the Lightning Fabric object. The parameters like accelerator, strategy,
    # devices etc. will be proided by the command line. See all options: `lightning
    # run model --help`
    fabric = Fabric()

    seed_everything(hparams.seed)  # instead of torch.manual_seed(...)

    transform = T.Compose([T.ToTensor(), T.Normalize((0.1307,), (0.3081,))])

    # Let rank 0 download the data first, then everyone will load MNIST
    with fabric.rank_zero_first(
        local=False
    ):  # set `local=True` if your filesystem is not shared between machines
        train_dataset = MNIST(
            DATASETS_PATH,
            download=fabric.is_global_zero,
            train=True,
            transform=transform,
        )
        test_dataset = MNIST(
            DATASETS_PATH,
            download=fabric.is_global_zero,
            train=False,
            transform=transform,
        )

    train_loader = torch.utils.data.DataLoader(
        train_dataset,
        batch_size=hparams.batch_size,
    )
    test_loader = torch.utils.data.DataLoader(
        test_dataset, batch_size=hparams.batch_size
    )

    # don't forget to call `setup_dataloaders` to prepare for dataloaders for
    # distributed training.
    train_loader, test_loader = fabric.setup_dataloaders(train_loader, test_loader)

    model = Net()  # remove call to .to(device)
    optimizer = optim.Adadelta(model.parameters(), lr=hparams.lr)

    # don't forget to call `setup` to prepare for model / optimizer for
    # distributed training. The model is moved automatically to the right device.
    model, optimizer = fabric.setup(model, optimizer)

    scheduler = StepLR(optimizer, step_size=1, gamma=hparams.gamma)

    # use torchmetrics instead of manually computing the accuracy
    test_acc = Accuracy(task="multiclass", num_classes=10).to(fabric.device)

    # EPOCH LOOP
    for epoch in range(1, hparams.epochs + 1):
        # TRAINING LOOP
        model.train()
        for batch_idx, (data, target) in enumerate(train_loader):
            # NOTE: no need to call `.to(device)` on the data, target
            optimizer.zero_grad()
            output = model(data)
            loss = F.nll_loss(output, target)
            fabric.backward(loss)  # instead of loss.backward()

            optimizer.step()
            if (batch_idx == 0) or ((batch_idx + 1) % hparams.log_interval == 0):
                done = (batch_idx * len(data)) / len(train_loader.dataset)
                pct = 100.0 * batch_idx / len(train_loader)
                print(  # noqa: T201
                    f"-> Epoch: {epoch} [{done} ({pct:.0f}%)]\tLoss: {loss.item():.6f}"
                )

                # Log dict of metrics
                logger.log_metrics({"loss": loss.item()})

                if hparams.dry_run:
                    break

        scheduler.step()

        # TESTING LOOP
        model.eval()
        test_loss = 0
        with torch.no_grad():
            for data, target in test_loader:
                # NOTE: no need to call `.to(device)` on the data, target
                output = model(data)
                test_loss += F.nll_loss(output, target, reduction="sum").item()

                # WITHOUT TorchMetrics
                # pred = output.argmax(dim=1, keepdim=True)  # get the index of the max
                # log-probability correct += pred.eq(target.view_as(pred)).sum().item()

                # WITH TorchMetrics
                test_acc(output, target)

                if hparams.dry_run:
                    break

        # all_gather is used to aggregated the value across processes
        test_loss = fabric.all_gather(test_loss).sum() / len(test_loader.dataset)
        acc = 100 * test_acc.compute()

        print(  # noqa: T201
            f"\nTest set: Average loss: {test_loss:.4f}, Accuracy: ({acc:.0f}%)\n"
        )

        # log additional metrics

```

--------------------------------

### DVC command line workflow

Source: https://github.com/treeverse/dvclive/wiki/DVC-magic

Shell commands to initialize a DVC project, run the training script with parameter tracking, and compare results using DVC plots.

```bash
echo "power: 0.2" > params.yaml
git init --quiet && dvc init --quiet
git add -A && git commit -m "initial"
dvc run -n train --params power -d train.py --live training_logs python train.py
git add -A && git commit -am "baseline"
echo "power: 0.4" > params.yaml && dvc repro train
dvc plots diff
```

--------------------------------

### Optuna Integration with DVCLive

Source: https://context7.com/treeverse/dvclive/llms.txt

Logs hyperparameter optimization trials as separate experiments using DVCLiveCallback for Optuna. Each trial's parameters and metrics are logged, creating distinct experiments in the specified directory.

```python
import optuna
from dvclive.optuna import DVCLiveCallback

def objective(trial):
    # Suggest hyperparameters
    lr = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
    n_layers = trial.suggest_int("n_layers", 1, 5)
    dropout = trial.suggest_float("dropout", 0.1, 0.5)

    # Train model with suggested params
    model = create_model(n_layers=n_layers, dropout=dropout)
    accuracy = train_and_evaluate(model, lr=lr)

    return accuracy

# Create study
study = optuna.create_study(
    direction="maximize",
    study_name="hyperparameter_search"
)

# Create DVCLive callback
# Each trial creates a separate experiment in dvclive-optuna/
dvclive_callback = DVCLiveCallback(
    metric_name="accuracy",  # or list for multi-objective: ["accuracy", "f1"]
    dir="dvclive-optuna"
)

# Optimize with callback
study.optimize(
    objective,
    n_trials=100,
    callbacks=[dvclive_callback]
)

# Each trial logged as separate experiment with:
# - params.yaml: learning_rate, n_layers, dropout
# - metrics.json: accuracy
```

--------------------------------

### DVC Experiment Commands

Source: https://github.com/treeverse/dvclive/wiki/Experiments-example

This section outlines the DVC commands used to manage and compare experiments. It includes initializing DVC, running baseline experiments, modifying parameters for new experiments, and visualizing results through plots and logs.

```bash
git init && dvc init && git add -A && git commit -am "initial commit"
echo "logs.html" >> .gitignore
dvc run -n train -d train.py --params batch_size,optimizer --plots-no-cache logs/ --metrics-no-cache logs.json python train.py
git add -A && git commit -am "run baseline"
dvc exp run train --params optimizer=adam
dvc exp run train --params optimizer=adamax
dvc exp show --sha
dvc plots diff master {sha1} {sha2}
dvc live logs --rev master {sha1} {sha2}
```

--------------------------------

### Log Artifacts with DVCLive

Source: https://context7.com/treeverse/dvclive/llms.txt

Demonstrates how to track files, directories, and models using log_artifact. It covers basic caching, metadata annotation for the Model Registry, and handling external file copies.

```python
from dvclive import Live
import joblib

with Live() as live:
    model = train_model()
    joblib.dump(model, "model.pkl")

    live.log_artifact("model.pkl")

    live.log_artifact(
        path="model.pkl",
        type="model",
        name="fraud_detector",
        desc="XGBoost fraud detection model trained on Q4 data",
        labels=["production", "fraud", "xgboost"],
        meta={
            "framework": "xgboost",
            "accuracy": 0.95,
            "dataset_version": "v2.1"
        }
    )

    live.log_artifact(
        path="data/processed/",
        type="dataset",
        name="training_data",
        desc="Preprocessed training dataset"
    )

    live.log_artifact(
        path="/tmp/checkpoints/best_model.pt",
        type="model",
        name="best_checkpoint",
        copy=True
    )

    live.log_artifact(
        path="outputs/predictions.csv",
        type="dataset",
        cache=False
    )
```

--------------------------------

### Log Training Metrics with DVCLive

Source: https://context7.com/treeverse/dvclive/llms.txt

Shows how to log scalar metrics during training loops. It demonstrates logging single metrics, nested paths, time-series data, and controlling whether a metric is plotted in the history files.

```python
from dvclive import Live

with Live() as live:
    for epoch in range(10):
        live.log_metric("accuracy", 0.85 + epoch * 0.01)
        live.log_metric("loss", 1.0 - epoch * 0.08)
        live.log_metric("train/accuracy", 0.82 + epoch * 0.015)
        live.log_metric("train/loss", 1.2 - epoch * 0.1)
        live.log_metric("val/accuracy", 0.80 + epoch * 0.012)
        live.log_metric("val/loss", 1.3 - epoch * 0.09)
        live.log_metric("throughput", 1000 + epoch * 50, timestamp=True)
        live.log_metric("final_score", 0.92, plot=False)
        live.next_step()
```

--------------------------------

### Import DVCLive and Lightning Fabric Modules

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Fabric.ipynb

Imports essential modules from PyTorch, torchvision, and Lightning Fabric, along with DVCLive's Fabric logger. These imports are necessary for building and training models with experiment tracking.

```python
from types import SimpleNamespace

import torch
import torch.nn.functional as F  # noqa: N812
import torchvision.transforms as T  # noqa: N812
from lightning.fabric import Fabric, seed_everything
from lightning.fabric.utilities.rank_zero import rank_zero_only
from torch import nn, optim
from torch.optim.lr_scheduler import StepLR
from torchmetrics.classification import Accuracy
from torchvision.datasets import MNIST

from dvclive.fabric import DVCLiveLogger

DATASETS_PATH = "Datasets"
```

--------------------------------

### Initialize PyTorch Model and Dataset

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Quickstart.ipynb

Initializes a sequential PyTorch model for image classification, defines the loss criterion, and downloads and transforms the MNIST training and testing datasets. The data is moved to the appropriate device (CPU or CUDA).

```python
# @title Initialize model and dataset. { display-mode: "form" }

model = torch.nn.Sequential(
    torch.nn.Flatten(),
    torch.nn.Linear(28 * 28, 128),
    torch.nn.ReLU(),
    torch.nn.Dropout(0.1),
    torch.nn.Linear(128, 64),
    torch.nn.ReLU(),
    torch.nn.Dropout(0.1),
    torch.nn.Linear(64, 10),
).to(device)

criterion = torch.nn.CrossEntropyLoss()

mnist_train = torchvision.datasets.MNIST("data", download=True)
x_train, y_train = transform(mnist_train)
mist_test = torchvision.datasets.MNIST("data", download=True, train=False)
x_test, y_test = transform(mnist_test)
```

--------------------------------

### Advance Training Steps and Metric Logging

Source: https://context7.com/treeverse/dvclive/llms.txt

Shows how to log training metrics and advance the experiment step counter. This process triggers automated updates to metrics.json, dvc.yaml, and reports.

```python
from dvclive import Live

with Live(report="html") as live:
    live.log_params({"epochs": 50, "lr": 0.001})

    for epoch in range(50):
        train_loss, train_acc = train_epoch(model, train_loader)
        val_loss, val_acc = validate(model, val_loader)

        live.log_metric("train/loss", train_loss)
        live.log_metric("train/accuracy", train_acc)
        live.log_metric("val/loss", val_loss)
        live.log_metric("val/accuracy", val_acc)

        live.next_step()

        print(f"Epoch {live.step}: val_acc={val_acc:.4f}")
```

--------------------------------

### Monitor System Resources

Source: https://context7.com/treeverse/dvclive/llms.txt

Configures automated monitoring of CPU, GPU, RAM, and disk usage. This helps identify hardware bottlenecks during training.

```python
from dvclive import Live

with Live(monitor_system=True) as live:
    for epoch in range(100):
        train_epoch()
        live.next_step()

with Live() as live:
    live.monitor_system(
        interval=0.05,
        num_samples=20,
        directories_to_monitor={
            "data": "/data",
            "models": "/home/user/models"
        }
    )

    for epoch in range(100):
        train_epoch()
        live.next_step()
```

--------------------------------

### monitor_system

Source: https://context7.com/treeverse/dvclive/llms.txt

Monitors and logs system resources like CPU, GPU, RAM, and disk usage during training.

```APIDOC
## monitor_system

### Description
Monitors and logs CPU, GPU, RAM, and disk usage throughout training. Useful for identifying resource bottlenecks.

### Method
Python Method

### Parameters
#### Request Body
- **interval** (float) - Optional - Sampling interval in seconds.
- **num_samples** (int) - Optional - Number of samples to aggregate before logging.
- **directories_to_monitor** (dict) - Optional - Mapping of labels to directory paths for disk monitoring.
```

--------------------------------

### Log Experiment Metrics with DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Uses the DVCLive library to log parameters and metrics during an experiment loop. It iterates through batches, evaluates drift, and logs results per step.

```python
with Live(report="notebook") as live:
    for date in experiment_batches:
        live.log_param("begin", date[0])
        live.log_param("end", date[1])
        metrics = eval_drift(
            df.loc[df.dteday.between(reference_dates[0], reference_dates[1])],
            df.loc[df.dteday.between(date[0], date[1])],
            column_mapping=data_columns,
        )
        for feature in metrics:
            live.log_metric(feature[0], round(feature[1], 3))
        live.next_step()
```

--------------------------------

### Integrate DVCLive with LightGBM

Source: https://context7.com/treeverse/dvclive/llms.txt

Uses the DVCLiveCallback to monitor LightGBM training sessions. It logs evaluation metrics defined in the training parameters.

```python
import lightgbm as lgb
from dvclive.lgbm import DVCLiveCallback

train_data = lgb.Dataset(X_train, label=y_train)
val_data = lgb.Dataset(X_val, label=y_val, reference=train_data)

params = {
    "objective": "binary",
    "metric": ["binary_logloss", "auc"],
    "num_leaves": 31,
    "learning_rate": 0.05
}

dvclive_callback = DVCLiveCallback(dir="dvclive", report="html")

model = lgb.train(
    params, train_data, num_boost_round=200, 
    valid_sets=[train_data, val_data], 
    valid_names=["train", "val"], 
    callbacks=[dvclive_callback]
)
```

--------------------------------

### PyTorch Training Utilities

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Quickstart.ipynb

Provides utility functions for training a PyTorch model, including data transformation, single epoch training, prediction, metric calculation, and evaluation. It also includes logic for visualizing misclassified images.

```python
# @title Training helpers. { display-mode: "form" }

import numpy as np
import torch
import torchvision

from dvclive import Live

device = "cuda:0" if torch.cuda.is_available() else "cpu"


def transform(dataset):
    """Get inputs and targets from dataset."""
    x = dataset.data.reshape(len(dataset.data), 1, 28, 28) / 255
    y = dataset.targets
    return x.to(device), y.to(device)


def train_one_epoch(model, criterion, x, y, lr, weight_decay):
    model.train()
    optimizer = torch.optim.Adam(model.parameters(), lr=lr, weight_decay=weight_decay)
    y_pred = model(x)
    loss = criterion(y_pred, y)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()


def predict(model, x):
    """Get model prediction scores."""
    model.eval()
    with torch.no_grad():
        return model(x)


def get_metrics(y, y_pred, y_pred_label):
    """Get loss and accuracy metrics."""
    metrics = {}
    criterion = torch.nn.CrossEntropyLoss()
    metrics["loss"] = criterion(y_pred, y).item()
    metrics["acc"] = (y_pred_label == y).sum().item() / len(y)
    return metrics


def evaluate(model, x, y):
    """Evaluate model and save metrics."""
    scores = predict(model, x)
    _, labels = torch.max(scores, 1)
    actual = [int(v) for v in y]
    predicted = [int(v) for v in labels]

    metrics = get_metrics(y, scores, labels)

    return metrics, actual, predicted


def get_missclassified_image(actual, predicted, dataset):
    confusion = {}
    for n, (a, p) in enumerate(zip(actual, predicted)):
        image = np.array(dataset[n][0]) / 255
        confusion[(a, p)] = image

    max_i, max_j = 0, 0
    for i, j in confusion:
        max_i = max(i, max_i)
        max_j = max(j, max_j)

    frame_size = 30
    image_shape = (28, 28)
    incorrect_color = np.array((255, 100, 100), dtype="uint8")
    label_color = np.array((100, 100, 240), dtype="uint8")

    out_matrix = (
        np.ones(
            shape=((max_i + 2) * frame_size, (max_j + 2) * frame_size, 3), dtype="uint8"
        )
        * 240
    )

    for i in range(max_i + 1):
        if (i, i) in confusion:
            image = confusion[(i, i)]
            xs = (i + 1) * frame_size + 1
            xe = (i + 2) * frame_size - 1
            ys = 1
            ye = frame_size - 1
            for c in range(3):
                out_matrix[xs:xe, ys:ye, c] = (1 - image) * label_color[c]
                out_matrix[ys:ye, xs:xe, c] = (1 - image) * label_color[c]

    for i, j in confusion:  # noqa: PLC0206
        image = confusion[(i, j)]
        assert image.shape == image_shape  # noqa: S101
        xs = (i + 1) * frame_size + 1
        xe = (i + 2) * frame_size - 1
        ys = (j + 1) * frame_size + 1
        ye = (j + 2) * frame_size - 1
        assert (xe - xs, ye - ys) == image_shape  # noqa: S101
        if i != j:
            for c in range(3):
                out_matrix[xs:xe, ys:ye, c] = (1 - image) * incorrect_color[c]

    return out_matrix
```

--------------------------------

### Compare Experiments with DVC CLI

Source: https://github.com/treeverse/dvclive/blob/main/README.md

These DVC CLI commands are used to display and visualize the results of multiple experiments. `dvc exp show` lists experiment details, while `dvc plots diff` generates plots comparing metrics across experiments.

```console
dvc exp show
```

```console
dvc plots diff $(dvc exp list --names-only) --open
```

--------------------------------

### Prepare MNIST Data Loaders

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-PyTorch-Lightning.ipynb

Downloads and prepares the MNIST dataset for training and validation using PyTorch DataLoader.

```python
from torchvision import transforms
from torchvision.datasets import MNIST

transform = transforms.ToTensor()
train_set = MNIST(root="MNIST", download=True, train=True, transform=transform)
validation_set = MNIST(root="MNIST", download=True, train=False, transform=transform)
train_loader = torch.utils.data.DataLoader(train_set)
validation_loader = torch.utils.data.DataLoader(validation_set)
```

--------------------------------

### Model Training Parameters (YAML)

Source: https://github.com/treeverse/dvclive/wiki/Experiments-example

This YAML file defines the parameters used for training the machine learning model. It specifies the optimizer and batch size, which can be modified to run different experiments.

```yaml
optimizer: sgd
batch_size: 64
```

--------------------------------

### Manual Sync Operations

Source: https://context7.com/treeverse/dvclive/llms.txt

Provides control over when metrics, reports, and configuration files are generated. Useful for custom training loops that do not use the standard next_step() flow.

```python
from dvclive import Live

live = Live(report="html")

try:
    for epoch in range(100):
        metrics = train_and_evaluate()

        for name, value in metrics.items():
            live.log_metric(name, value)

        if epoch % 10 == 0:
            live.make_summary()
            live.make_dvcyaml()
            live.make_report()

        live.step += 1
finally:
    live.end()

live = Live()
for epoch in range(100):
    live.log_metric("loss", compute_loss())
    live.sync()
    live.step += 1
live.end()
```

--------------------------------

### Log Custom Plots with DVCLive

Source: https://context7.com/treeverse/dvclive/llms.txt

Shows how to create custom visualizations by logging data points from pandas DataFrames, NumPy arrays, or lists of dictionaries. Supports various templates like linear, bar_horizontal, and scatter.

```python
from dvclive import Live
import pandas as pd
import numpy as np

with Live() as live:
    live.log_plot(
        "learning_curve",
        datapoints=[{"epoch": 1, "train_loss": 2.5, "val_loss": 2.8}, {"epoch": 2, "train_loss": 1.8, "val_loss": 2.1}],
        x="epoch",
        y=["train_loss", "val_loss"],
        template="linear",
        title="Learning Curves"
    )

    df = pd.DataFrame({"feature_importance": [0.35, 0.25], "feature_name": ["age", "income"]})
    live.log_plot("feature_importance", datapoints=df, x="feature_name", y="feature_importance", template="bar_horizontal")
```

--------------------------------

### Integrate DVCLive with Keras Training

Source: https://github.com/treeverse/dvclive/wiki/Home

Shows how to use the DvcLiveCallback to automatically track training metrics during a Keras model fit process. This requires the dvclive.keras module and initializes logging to a specified directory.

```python
import numpy as np
from tensorflow import keras
from tensorflow.keras import layers

import dvclive
from dvclive.keras import DvcLiveCallback

num_classes = 10
input_shape = (28, 28, 1)

def get_data():
    (x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
    x_train = x_train.astype("float32") / 255
    x_test = x_test.astype("float32") / 255
    x_train = np.expand_dims(x_train, -1)
    x_test = np.expand_dims(x_test, -1)
    y_train = keras.utils.to_categorical(y_train, num_classes)
    y_test = keras.utils.to_categorical(y_test, num_classes)
    return (x_train, y_train), (x_test, y_test)

def get_model():
    model= keras.Sequential([
        keras.Input(shape=input_shape),
        layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Flatten(),
        layers.Dropout(0.5),
        layers.Dense(num_classes, activation="softmax"),
    ])
    model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
    return model

if __name__ == "__main__":
    model = get_model()
    (x_train, y_train), (_, _) = get_data()
    dvclive.init("logs")
    model.fit(x_train, y_train, batch_size=128, epochs=10, validation_split=0.1, callbacks=DvcLiveCallback())
```

--------------------------------

### Load and Inspect Data with Pandas

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Imports the pandas library and loads the 'day.csv' file into a DataFrame. It specifies the header, separator, and date parsing for the 'dteday' column, then displays the first few rows.

```python
import pandas as pd
```

```python
df = pd.read_csv("raw_data/day.csv", header=0, sep=",", parse_dates=["dteday"])
df.head()
```

--------------------------------

### Prepare Scikit-learn Data

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-scikit-learn.ipynb

Generates a synthetic dataset using make_circles from scikit-learn and splits it into training and testing sets.

```python
from sklearn.datasets import make_circles
from sklearn.model_selection import train_test_split

X, y = make_circles(noise=0.3, factor=0.5, random_state=42)

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    random_state=42)
```

--------------------------------

### Integrate DVCLive with FastAI

Source: https://context7.com/treeverse/dvclive/llms.txt

Integrates the DVCLiveCallback into the FastAI training workflow to track metrics and training parameters. It supports standard fit and fine-tuning methods.

```python
from fastai.vision.all import *
from dvclive.fastai import DVCLiveCallback

dls = ImageDataLoaders.from_folder(path, train='train', valid='valid', item_tfms=Resize(224), batch_tfms=aug_transforms())
learn = vision_learner(dls, resnet34, metrics=[accuracy, error_rate])

dvclive_callback = DVCLiveCallback(with_opt=False, dir="dvclive", save_dvc_exp=True)

learn.fit_one_cycle(10, lr_max=1e-3, cbs=[dvclive_callback])
learn.fine_tune(5, freeze_epochs=2, cbs=[dvclive_callback])
```

--------------------------------

### Configure Evidently Column Mapping

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Initializes a ColumnMapping object from Evidently and defines numerical and categorical features for data analysis.

```python
from evidently.pipeline.column_mapping import ColumnMapping
```

```python
data_columns = ColumnMapping()
data_columns.numerical_features = ["weathersit", "temp", "atemp", "hum", "windspeed"]
data_columns.categorical_features = ["holiday", "workingday"]
```

--------------------------------

### Configure Git for DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Sets the global Git user email and name, which is often a prerequisite for DVCLive to properly track experiments within a Git repository.

```bash
!git config --global user.email "you@example.com"
!git config --global user.name "Your Name"
```

--------------------------------

### Retrieve and Display Experiments

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Evidently.ipynb

Shows how to view experiment results programmatically using the DVC API or via the command line interface.

```python
import dvc.api
import pandas as pd

# Retrieve experiments as a DataFrame
pd.DataFrame(dvc.api.exp_show())

# Alternatively, use the CLI
!dvc exp show
```

--------------------------------

### Track Scikit-learn Experiments with DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-scikit-learn.ipynb

Trains a RandomForestClassifier with varying n_estimators and logs parameters, metrics (F1-score), and confusion matrix plots for each experiment using DVCLive.

```python
from dvclive import Live

from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import f1_score

for n_estimators in (10, 50, 100):

  with Live() as live:

    live.log_param("n_estimators", n_estimators)

    clf = RandomForestClassifier(n_estimators=n_estimators)
    clf.fit(X_train, y_train)

    y_train_pred = clf.predict(X_train)

    live.log_metric("train/f1", f1_score(y_train, y_train_pred, average="weighted"), plot=False)
    live.log_sklearn_plot(
      "confusion_matrix", y_train, y_train_pred, name="train/confusion_matrix",
      title="Train Confusion Matrix")

    y_test_pred = clf.predict(X_test)

    live.log_metric("test/f1", f1_score(y_test, y_test_pred, average="weighted"), plot=False)
    live.log_sklearn_plot(
      "confusion_matrix", y_test, y_test_pred, name="test/confusion_matrix",
      title="Test Confusion Matrix")
```

--------------------------------

### Log Metrics and Save Checkpoints with DVCLive

Source: https://github.com/treeverse/dvclive/blob/main/examples/DVCLive-Fabric.ipynb

This snippet shows how to log test metrics, save a model checkpoint using fabric.save to handle distributed environments, and log the resulting artifact to DVCLive. It also includes the finalization step to mark the experiment as successful.

```python
logger.log_metrics({"test_loss": test_loss, "test_acc": 100 * test_acc.compute()})
test_acc.reset()
if hparams.dry_run:
    break
if hparams.save_model:
    fabric.save("mnist_cnn.pt", model.state_dict())
    if rank_zero_only.rank == 0:
        logger.experiment.log_artifact("mnist_cnn.pt")
logger.finalize("success")
```

--------------------------------

### Log metrics with DVCLive

Source: https://github.com/treeverse/dvclive/wiki/DVC-magic

A Python script that logs metric and loss values using DVCLive. It reads parameters from a YAML file and iterates through a loop to record data points without requiring explicit logger initialization.

```python
import dvclive
import time

def get_params():
    from ruamel.yaml import YAML
    yaml = YAML(typ="safe")
    params_path = "params.yaml"
    with open(params_path, "r") as fd:
        return yaml.load(fd)

def metric(i):
    p = float(get_params()["power"])
    return 1 - 1/(1+i**p)

def loss(i):
    return 1-(metric(i))

if __name__ == "__main__":
    for i in range(100):
        dvclive.log("metric", metric(i))
        dvclive.log("loss", loss(i))
```