### Install act Tool Locally

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

This shell command downloads and installs the act tool for running GitHub Actions locally using curl and bash. Requires sudo access for installation to /bin. Outputs the act binary; add ~/bin to PATH for use. Supports alternative installation methods via package managers.

```shell
curl -s https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash -s -- -d -b ~/bin
```

```shell
PATH=~/bin:$PATH
```

--------------------------------

### Parallel Data Valuation with Ray Backend

Source: https://github.com/aai-institute/pydvl/blob/develop/docs/getting-started/advanced-usage.md

Demonstrates how to configure and execute data valuation in parallel using Ray backend with pyDVL. Shows setup of Dataset, ModelUtility, ShapleyValuation with parallel configuration using joblib's parallel_config context manager. Requires Ray installation and cluster setup for distributed execution across multiple workers.

```python
import sklearn as sk
from joblib import parallel_config, register_parallel_backend
from pydvl.valuation import *
from ray.util.joblib import register_ray

register_ray()
    
train, test = Dataset.from_arrays(...)
model = sk.svm.SVC()
scorer = SupervisedScorer("accuracy", test, default=0.0, range=(0, 1))
utility = ModelUtility(model, scorer)
sampler = PermutationSampler(truncation=NoTruncation())
stopping = MinUpdates(7000) | MaxTime(3600)
shapley = ShapleyValuation(utility, sampler, stopping, progress=True)

with parallel_config(backend="ray", n_jobs=128):
    shapley.fit(train)

results = shapley.result
```

--------------------------------

### Setup and imports for data valuation with pyDVL

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/least_core_basic.ipynb

Initial setup including library imports, configuration of plotting defaults, and environment checks for CI. This includes setting random seeds and adjusting parameters based on the environment.

```python
%matplotlib inline

import os
import random
import warnings

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from tqdm.auto import tqdm, trange

warnings.simplefilter("ignore")

plt.ioff()  # Prevent jupyter from automatically plotting
plt.rcParams["figure.figsize"] = (20, 8)
plt.rcParams["font.size"] = 12
plt.rcParams["xtick.labelsize"] = 12
plt.rcParams["ytick.labelsize"] = 10
plt.rcParams["axes.facecolor"] = (1, 1, 1, 0)
plt.rcParams["figure.facecolor"] = (1, 1, 1, 0)

mean_colors = ["dodgerblue", "indianred", "limegreen", "darkorange", "darkorchid"]
shade_colors = ["lightskyblue", "firebrick", "seagreen", "gold", "plum"]

seed = 16
random.seed(seed)
np.random.seed(seed)

is_CI = os.environ.get("CI")

dataset_size = 200
n_iterations = 5000
train_mini_size = 12
n_jobs = 12
n_runs = 10

if is_CI:
    dataset_size = 20
    n_iterations = 500
    train_mini_size = 4
    n_jobs = 1
    n_runs = 2
```

--------------------------------

### Manual Release Version Setup

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

Sets up environment variables and creates a release branch for manual version control in pyDVL. This establishes the foundation for manual release processes.

```shell
export RELEASE_VERSION="vX.Y.Z"
git checkout develop
git branch release/${RELEASE_VERSION} && git checkout release/${RELEASE_VERSION}
```

--------------------------------

### Setup environment and imports

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_skorch.ipynb

Configures the environment and imports necessary libraries for the notebook. Sets up plotting parameters, random seeds, and device configuration.

```python
from __future__ import annotations

%matplotlib inline

import os
import random
import warnings

import matplotlib.pyplot as plt
import torch
from support.common import filecache
from support.datasets import load_digits_dataset

from pydvl.reporting.plots import plot_result_errors

warnings.filterwarnings("ignore")

plt.ioff()  # Prevent jupyter from automatically plotting
plt.rcParams["figure.figsize"] = (20, 6)
plt.rcParams["font.size"] = 12
plt.rcParams["xtick.labelsize"] = 10
plt.rcParams["ytick.labelsize"] = 10
plt.rcParams["axes.facecolor"] = (1, 1, 1, 0)
plt.rcParams["figure.facecolor"] = (1, 1, 1, 0)

is_CI = os.environ.get("CI")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
random_state = 36

n_jobs = 6
n_epochs = 24
batch_size = 128
random.seed(random_state)
torch.manual_seed(random_state);
```

--------------------------------

### Creating pyDVL Package Locally with Setup.py

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

This command builds source distribution and wheel for local package creation and testing. It relies on setup.py configuration and pyproject.toml. Outputs are in dist directory; suitable for installation tests before CI.

```shell
python setup.py sdist bdist_wheel
```

--------------------------------

### Display example images

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_skorch.ipynb

Displays a subset of example images from the MNIST dataset along with their labels.

```python
fig, axes = plt.subplots(1, 4, figsize=(12, 3))
for i in range(4):
    ax = axes[i]
    ax.imshow(train.data().cpu().x[i].reshape((8, 8)), cmap="gray")
    ax.set_xlabel(f"Label: {train.data().y[i]}")
plt.suptitle("Example images from the dataset")
plt.tight_layout()
plt.show();
```

--------------------------------

### Setup and Configuration in Python

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_utility_learning.ipynb

Imports necessary libraries and configures plotting and runtime parameters for the pydvl project. It includes conditional adjustments for CI environments and sets random seeds for reproducibility. Dependencies: matplotlib, numpy, pandas, scikit-learn, tqdm, pydvl libraries.

```python
%matplotlib inline

import os
import random
import warnings
from collections import defaultdict
from itertools import product

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.datasets import load_iris
from tqdm.auto import tqdm

from pydvl.reporting.plots import shaded_mean_std

plt.ioff()  # Prevent jupyter from automatically plotting
plt.rcParams["figure.figsize"] = (16, 10)
plt.rcParams["font.size"] = 15
plt.rcParams["xtick.labelsize"] = 15
plt.rcParams["ytick.labelsize"] = 15
plt.rcParams["axes.facecolor"] = (1, 1, 1, 0)
plt.rcParams["figure.facecolor"] = (1, 1, 1, 0)

random_state = 42

train_size = 15
batch_size = 32
training_budgets = np.logspace(2, np.log10(4000), num=8, base=10).astype(int)
n_runs = 10
n_jobs = 16

is_CI = os.environ.get("CI")

if is_CI:
    train_size = 4
    batch_size = 1
    training_budgets = [2, 4, 6]
    n_runs = 1
    n_jobs = 1

random.seed(random_state)
np.random.seed(random_state)

# raised by joblib after cancelling running tasks
warnings.simplefilter("ignore", UserWarning)

computation_times = defaultdict(list)
```

--------------------------------

### Setup and Initialization for Neural Network Analysis

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_imagenet.ipynb

Imports necessary libraries and sets up configurations for neural network analysis, including plotting parameters, device selection (GPU/CPU), and random seeding. It prepares the environment for subsequent influence function computations.

```python
%matplotlib inline
import logging
import os
from typing import Tuple

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import torch
from support.common import (
    compute_mean_corrupted_influences,
    plot_corrupted_influences_distribution,
    plot_losses,
    plot_lowest_highest_influence_images,
    plot_sample_images,
)
from support.datasets import corrupt_imagenet, load_preprocess_imagenet
from support.influence import MODEL_PATH, Losses, TrainingManager, new_resnet_model
from torch import nn
from torch.utils.data import DataLoader, TensorDataset

logging.basicConfig(level=logging.INFO)
default_figsize = (7, 7)
plt.rcParams["figure.figsize"] = default_figsize
plt.rcParams["font.size"] = 12
plt.rcParams["xtick.labelsize"] = 12
plt.rcParams["ytick.labelsize"] = 10
plt.rcParams["axes.facecolor"] = (1, 1, 1, 0)
plt.rcParams["figure.facecolor"] = (1, 1, 1, 0)
hessian_reg = 1e4 if os.environ.get("CI") else 1e-3
random_state = 42
np.random.seed(random_state)
DEVICE = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

```

--------------------------------

### Prepare small IMDB train/test datasets and DataLoaders (Python)

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

Selects a limited number of examples from the IMDB dataset for quick experimentation, wraps them in a custom dataset class, and creates PyTorch DataLoaders with specified batch sizes for training and testing.

```python
NUM_TRAIN_EXAMPLES = 100 if not is_CI else 7
NUM_TEST_EXAMPLES = 100 if not is_CI else 5

small_train_dataset = (
    imdb["train"]
    .shuffle(seed=seed)
    .select([i for i in list(range(NUM_TRAIN_EXAMPLES))])
)
small_test_dataset = (
    imdb["test"].shuffle(seed=seed).select([i for i in list(range(NUM_TEST_EXAMPLES))])
)

train_dataset = ImdbDataset(small_train_dataset, tokenizer=tokenizer)
test_dataset = ImdbDataset(small_test_dataset, tokenizer=tokenizer)

train_dataloader = torch.utils.data.DataLoader(
    train_dataset, batch_size=7, shuffle=True
)
test_dataloader = torch.utils.data.DataLoader(test_dataset, batch_size=5, shuffle=True)
```

--------------------------------

### Analyze High-Influence Training Example

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

This code examines a specific training example by printing sentiment predictions and displaying the text. It uses a wrapped model for predictions and selects by index. Outputs probabilities and HTML-rendered sentence, assuming IPython.display availability.

```python
train_sentence_idx = 3

print(f"Training example with idx {train_sentence_idx}: \n")

print_sentiment_preds(
    wrapped_model,
    train_input[train_sentence_idx],
    train_labels[train_sentence_idx].item(),
)

print("Sentence:")
display(HTML(train_text[train_sentence_idx]))
```

--------------------------------

### Setup Environment and Libraries for pyDVL

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_basic_spotify.ipynb

Imports necessary libraries for plotting, data manipulation, machine learning, and pyDVL functionalities. It also sets environment variables and random seeds for reproducibility. This code configures plotting defaults and handles differences between CI and local environments for parallel processing.

```python
import os
import random

import matplotlib.pyplot as plt
import numpy as np
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.metrics._scorer import neg_mean_absolute_error_scorer
from support.datasets import load_spotify_dataset

plt.ioff()  # Prevent jupyter from automatically plotting
plt.rcParams["figure.figsize"] = (20, 6)
plt.rcParams["font.size"] = 12
plt.rcParams["xtick.labelsize"] = 12
plt.rcParams["ytick.labelsize"] = 10
plt.rcParams["axes.facecolor"] = (1, 1, 1, 0)
plt.rcParams["figure.facecolor"] = (1, 1, 1, 0)

is_CI = os.environ.get("CI")
random_state = 24
random.seed(random_state)
n_jobs = 4
if is_CI:
    n_jobs = 1
```

--------------------------------

### Use Ray for Distributed Computing with Joblib

Source: https://context7.com/aai-institute/pydvl/llms.txt

Integrates Ray with Joblib for distributed computation by setting the backend to 'ray' within `parallel_config`. This example initializes a Ray cluster, performs distributed fitting, and then shuts down the Ray instance. Requires `ray` and `joblib`.

```python
import ray
from joblib import parallel_config

# Assuming valuation and train_data are defined

# Initialize Ray cluster
ray.init(num_cpus=16)

# Use Ray backend for distributed computation
with parallel_config(n_jobs=16, backend="ray"):
    valuation.fit(train_data)

ray.shutdown()
```

--------------------------------

### Setup and Imports for PyDVL Data Valuation

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/msr_banzhaf_digits.ipynb

Imports necessary libraries for data valuation, plotting, and PyTorch model definition. Sets up global configurations like device, random state, and plotting parameters. This code is essential for initializing the environment for subsequent data valuation tasks.

```python
from __future__ import annotations

%matplotlib inline

import os
import random
import warnings
from collections import OrderedDict, defaultdict
from typing import Type

import matplotlib.pyplot as plt
import numpy as np
import torch
from support.common import filecache
from support.datasets import load_digits_dataset

from pydvl.reporting.plots import plot_result_errors

warnings.filterwarnings("ignore")

plt.ioff()  # Prevent jupyter from automatically plotting
plt.rcParams["figure.figsize"] = (20, 6)
plt.rcParams["font.size"] = 12
plt.rcParams["xtick.labelsize"] = 10
plt.rcParams["ytick.labelsize"] = 10
plt.rcParams["axes.facecolor"] = (1, 1, 1, 0)
plt.rcParams["figure.facecolor"] = (1, 1, 1, 0)

is_CI = os.environ.get("CI")
device = "cuda" if torch.cuda.is_available() else "cpu"
random_state = 42

n_jobs = 16
n_epochs = 24
batch_size = 64
random.seed(random_state)
torch.manual_seed(random_state);
```

--------------------------------

### Apply time‑based stopping criteria (Python)

Source: https://context7.com/aai-institute/pydvl/llms.txt

Demonstrates MaxTime to limit valuation runtime and combines it with other quality criteria. The example creates a one‑hour wall‑clock limit and merges it with a history‑based criterion and a minimum update requirement.

```python
from pydvl.valuation import MaxTime

# Stop after wall clock time (in seconds)
max_time = MaxTime(3600)  # 1 hour

# Combine with quality criterion
stopping = (HistoryDeviation(n_steps=50, rtol=0.02) | max_time) & MinUpdates(50)
```

--------------------------------

### Compute Classwise Shapley Values with pydvl

Source: https://github.com/aai-institute/pydvl/blob/develop/docs/value/classwise-shapley.md

Demonstrates the instantiation and configuration of ClasswiseShapleyValuation in pydvl. It involves setting up the model, dataset, scorer, utility, sampler, and stopping criteria, mirroring the example from the CWS paper.

```python
from pydvl.valuation import *

seed = 42
model = ...
train, test = Dataset.from_arrays(X, y, train_size=0.6, random_state=seed)
n_labels = len(get_unique_labels(train.data().y))
scorer = ClasswiseSupervisedScorer("accuracy", test)
utility = ClasswiseModelUtility(model, scorer)
sampler = ClasswiseSampler(
    in_class=PermutationSampler(
        truncation=RelativeTruncation(rtol=0.01, burn_in_fraction=0.3), seed=seed
    ),
    out_of_class=UniformSampler(index_iteration=NoIndexIteration),
    max_in_class_samples=1,
)
# 500 permutations per label as in the paper
stopping = MaxSamples(sampler, 500*n_labels)
# Save the history in valuation.stopping.criteria[1]
stopping |= History(n_steps=5000),
valuation = ClasswiseShapleyValuation(
    utility=utility, sampler=sampler, is_done=stopping, normalize_values=True
)
```

--------------------------------

### Running Tests with Tox in pyDVL

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

Tox executes the test suite, builds documentation, and tests package installation. It supports optional pytest arguments like patterns or markers, and requires Ray and Memcached. Use flags like --memcached-service to specify Memcached address, -n for parallel workers, --slow-tests for slow tests, and --with-cuda for GPU support; limitations include potential segmentation faults with high parallelism.

```shell
tox -e tests -- <optional arguments>
```

```shell
tox -e tests -- -m "torch"
```

```shell
tox -e notebook-tests
```

--------------------------------

### Run act Commands for GitHub Workflows

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

These shell commands use act to run, list, or simulate GitHub Actions workflows locally. Requires act installed and .github/workflows present. Accepts flags like -W for workflow, -j for job, and event types; outputs action execution logs. Useful for testing without pushing to repo.

```shell
act -W .github/workflows/run-tests-workflow.yaml \
    -j run-tests \
    --input tests_to_run=base\
    --input python_version=3.9
```

```shell
act -l
```

```shell
act workflow_dispatch -l
```

```shell
act -j lint -l
```

```shell
act
```

```shell
act pull_request
```

```shell
act -j lint
```

```shell
act --artifact-server-path /tmp/artifacts
```

```shell
act -j lint -W .github/workflows/publish.yml
```

```shell
act -n
```

```shell
act -v
```

```shell
act release -j publish --eventpath events.json
```

```shell
act workflow_dispatch -j publish --eventpath events.json
```

--------------------------------

### Setup Data Removal Experiment Factories in Python

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/data_oob.ipynb

Defines factory functions to generate data splits, model evaluation utilities, and Out-of-Bag (OOB) valuation methods for a data removal experiment. It includes data resampling using RandomOverSampler and model training with RandomForestClassifier. Dependencies include pydvl, sklearn, and matplotlib.

```python
import warnings

from pydvl.reporting.point_removal import run_removal_experiment
from pydvl.valuation import KNNClassifierUtility, ModelUtility, SupervisedScorer
from pydvl.valuation.methods.data_oob import point_wise_accuracy
from pydvl.valuation.methods.random import RandomValuation


def make_data(random_state: int) -> tuple[Dataset, Dataset]:
    train, test = load_adult_data(
        train_size=train_size,
        subsample=subsample,
        random_state=random_state,
        _silent=True,
    )
    # suppress sklearn >= 1.6 warnings originating in imblearn
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        sampler = RandomOverSampler(random_state=random_state)
        resampled_x, resampled_y = sampler.fit_resample(*train.data())
    train = Dataset(resampled_x, resampled_y, train.feature_names, train.target_names)
    return train, test


def make_utility(test: Dataset, random_state: int) -> ModelUtility:
    model = RandomForestClassifier(
        n_estimators=n_est,
        max_samples=max_samples,
        class_weight="balanced",
        random_state=random_state + 1,
    )

    return ModelUtility(model, SupervisedScorer("accuracy", test, 0.0))


def make_oob(train: Dataset, random_state: int) -> DataOOBValuation:
    model = RandomForestClassifier(
        n_estimators=n_est,
        max_samples=max_samples,
        class_weight="balanced",
        random_state=random_state,
    )

    model.fit(*train.data())
    return DataOOBValuation(model, point_wise_accuracy)


def make_random(train: Dataset, random_state: int) -> RandomValuation:
    return RandomValuation(random_state=random_state)

removal_percentages = np.arange(0, 0.51, 0.02)

```

--------------------------------

### Configure Joblib Parallel Backends for Computation

Source: https://context7.com/aai-institute/pydvl/llms.txt

Demonstrates how to configure Joblib's parallel execution using `parallel_config`. It shows examples for using the default 'loky' backend (multiprocessing), 'threading' backend for I/O bound tasks, and sequential execution for debugging. Requires the `joblib` library.

```python
from joblib import parallel_config

# Assuming valuation and train_data are defined

# Joblib backend (default) - uses multiprocessing
with parallel_config(n_jobs=8, backend="loky"):
    valuation.fit(train_data)

# Joblib with threading (for I/O bound tasks)
with parallel_config(n_jobs=4, backend="threading"):
    valuation.fit(train_data)

# Sequential execution (debugging)
with parallel_config(n_jobs=1):
    valuation.fit(train_data)
```

--------------------------------

### Compute Beta Shapley Values with pyDVL (Python)

Source: https://github.com/aai-institute/pydvl/blob/develop/docs/value/beta-shapley.md

This code snippet demonstrates how to compute Beta Shapley values using the pyDVL library. It requires installing pyDVL and joblib; inputs include a model, training/test datasets, and parameters alpha and beta; outputs are fitted valuation scores. Dependencies: pyDVL for valuation classes, joblib for parallelism. Limitations: Assumes access to model and datasets, may be computationally intensive.

```python
from joblib import parallel_config
from pydvl.valuation import *

model = ...
train, test = Dataset.from_arrays(...)
scorer = SupervisedScorer(model, test, default=0.0)
utility = ModelUtility(model, scorer)
sampler = PermutationSampler()
stopping = RankCorrelation(rtol=1e-5, burn_in=100) | MaxUpdates(2000)
valuation = BetaShapleyValuation(
    utility, sampler, stopping, alpha=1, beta=16
)
with parallel_config(n_jobs=16):
    valuation.fit(train)
```

--------------------------------

### Write LaTeX math in Python docstrings with raw strings

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

Provides Python examples illustrating the correct way to include LaTeX math in docstrings using raw string literals. The first function works with a raw triple‑quoted string, while the second demonstrates the error that occurs without the raw prefix. No external dependencies are needed.

```python
# This will work

def f(x: float) -> float:
    r""" Computes 
    $${ f(x) = \frac{1}{x^2} }$$
    """
    return 1/(x*x)
```

```python
# This throws an obscure error

def f(x: float) -> float:
    """ Computes 
    $$\frac{1}{x^2}$$
    """
    return 1/(x*x)
```

--------------------------------

### Show Release Script Help

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

Displays the help information for the release-version script, explaining all available flags, parameters, and usage patterns.

```shell
build_scripts/release-version.sh --help
```

--------------------------------

### Building Documentation with mkdocs

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

Provides the command-line instructions for building the project's documentation using mkdocs. This command is used in CI environments to generate the static site.

```bash
mkdocs build
```

--------------------------------

### Analyze High-Influence Test Example

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

This code analyzes a specific test example by printing sentiment predictions and displaying the text. Similar to the training example analysis, it selects by index and outputs probabilities and HTML sentence. Requires a wrapped model and IPython.display.

```python
test_sentence_idx = 4

print(f"Test example with idx {test_sentence_idx}: \n")

print_sentiment_preds(
    wrapped_model, test_input[test_sentence_idx], test_labels[test_sentence_idx].item()
)

print("Sentence:")
display(HTML(test_text[test_sentence_idx]))
```

--------------------------------

### Plot per-layer influence trends for selected train examples

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

For a specific test example, extracts influence values across all layers for multiple train examples (excluding index 3) and plots them. Includes legend, custom x‑ticks with layer names, and axis labels.

```python
test_idx = 0\n\ntrain_idx_to_plot = list(range(len(ekfac_train_influences[0])))\ntrain_idx_to_plot.pop(3)\nfor train_idx in train_idx_to_plot:\n    infl_across_layers = []\n    idx = (test_idx, train_idx)\n    for layer_id, value in influences_by_layer.items():\n        infl_across_layers.append(value[idx].item())\n    plt.plot(infl_across_layers, label=f\"Train example {train_idx}\")\nplt.legend()\nplt.xticks(\n    range(len(influences_by_layer.keys())),\n    strip_layer_names(influences_by_layer.keys()),\n    rotation=70,\n)\nplt.xlabel(\"Layer id\")\nplt.ylabel(\"Influence value\")\nplt.title(f\"Influence of test example {test_idx} on test examples\")\nplt.show()
```

--------------------------------

### Serving Local mkdocs Documentation

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

Details the command to serve the mkdocs documentation locally, enabling live rebuilding of the documentation site as changes are made to markdown files, notebooks, and python files within the 'docs' and 'src' directories.

```bash
mkdocs serve
```

--------------------------------

### Display Example Images from MNIST Dataset

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/msr_banzhaf_digits.ipynb

Visualizes four example grayscale images from the loaded MNIST training dataset. Each image is displayed with its corresponding digit label. This helps in understanding the data distribution and quality.

```python
fig, axes = plt.subplots(1, 4, figsize=(12, 3))
for i in range(4):
    ax = axes[i]
    ax.imshow(train.data().x[i].reshape((8, 8)).cpu(), cmap="gray")
    ax.set_xlabel(f"Label: {train.data().y[i]}")
plt.suptitle("Example images from the dataset")
plt.tight_layout()
plt.show();
```

--------------------------------

### Compute Influence Functions using EK-FAC

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

This code calculates the influence of training examples on test examples via the EK-FAC approximation method. It takes test and training inputs and labels as inputs and outputs a tensor of influence values. Computation is expensive and limited to small batches for performance.

```python
ekfac_train_influences = ekfac_influence_model.influences(
    test_input,
    test_labels,
    train_input,
    train_labels,
)
```

--------------------------------

### Initialize Utility and Scorer in Python

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_utility_learning.ipynb

Sets up a SupervisedScorer for accuracy calculation on the test data and a ModelUtility object using the trained model and the scorer. Dependencies: pydvl.valuation.ModelUtility, pydvl.valuation.SupervisedScorer.

```python
from pydvl.valuation import ModelUtility, SupervisedScorer

scorer = SupervisedScorer("accuracy", test_data=test, default=0, range=(0, 1))
utility = ModelUtility(model=model, scorer=scorer, show_warnings=False)
```

--------------------------------

### Organize Influence Data into Pandas DataFrame

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

Processes the output from `explore_hessian_regularization`, organizing it into a pandas DataFrame. It calculates the mean influence for each layer across training examples and associates it with the corresponding regularization value.

```python
cols = ["reg_value", "layer_id", "mean_infl"]
infl_df = pd.DataFrame(influences_by_reg_value, columns=cols)
for reg_value in influences_by_reg_value:
    for layer_id, layer_influences in influences_by_reg_value[reg_value].items():
        mean_infl = torch.mean(layer_influences, dim=0).detach().numpy()
        infl_df = pd.concat(
            [infl_df, pd.DataFrame([[reg_value, layer_id, mean_infl]], columns=cols)]
        )
```

--------------------------------

### Plot heatmap of corrupted influences

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

Visualizes the corrupted influence tensor as a heatmap using matplotlib. Displays a color bar and labels for training and test example indices. Requires matplotlib.pyplot imported as plt.

```python
plt.imshow(corrupted_ekfac_train_influences.numpy().astype(int), vmin=-1000, vmax=1000)\nplt.colorbar(label=\"Influence value \")\nplt.title(\"Influence of corrupted training examples\")\nplt.xlabel(\"Training examples idx\")\nplt.ylabel(\"Test examples idx\")\nplt.show()
```

--------------------------------

### Perturbation Influence Calculation in PyDVL

Source: https://github.com/aai-institute/pydvl/blob/develop/docs/influence/index.md

This snippet shows how to compute perturbation-based influences using the `influences` method of a DirectInfluence model. It specifies the `InfluenceMode.Perturbation` to get feature-level influences for each training point on each test point.

```python
from pydvl.influence import InfluenceMode

influences = infl_model.influences(x_test, y_test, x, y,
                                   mode=InfluenceMode.Perturbation)
```

--------------------------------

### Tokenize and Get Model Output with Pydvl

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

This snippet shows how to tokenize a phrase and obtain model output using the Pydvl library. It requires a tokenizer and a model to be pre-defined. The input is a string, and the output is a model output object.

```python
example_phrase = (
    "Pydvl is the best data valuation library, and it is fully open-source!"
)

tokenized_example = tokenizer(
    [example_phrase],
    return_tensors="pt",
    truncation=True,
)

model_output = model(
    input_ids=tokenized_example.input_ids,
)
```

--------------------------------

### Visualize Influence Values as Heatmap

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

This snippet generates a heatmap visualization of influence scores between training and test examples using matplotlib. It converts the tensor to a numpy array for plotting with color scaling. Requires matplotlib and numpy libraries.

```python
plt.imshow(ekfac_train_influences.numpy().astype(int), vmin=-1000, vmax=1000)
plt.colorbar(label="Influence value ")
plt.title("Influence of training examples on test examples")
plt.xlabel("Training examples idx")
plt.ylabel("Test examples idx")
plt.show()
```

--------------------------------

### Load and Split Adult Dataset

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/data_oob.ipynb

Loads the Adult dataset using the `load_adult_data` helper function. The data is split into training and testing sets with specified `train_size`, `subsample`, and `random_state` for reproducibility. This is a crucial step for preparing data for model training.

```python
train, test = load_adult_data(
    train_size=train_size, subsample=subsample, random_state=random_state
)
```

--------------------------------

### Create ModelUtility with SupervisedScorer in pyDVL

Source: https://github.com/aai-institute/pydvl/blob/develop/docs/value/index.md

Demonstrates creating a ModelUtility object for data valuation in pyDVL. It sets up a dataset from sklearn, initializes an SVM classifier, and creates a SupervisedScorer using the model's default score() method. The utility object tracks model training and evaluation for subsequent valuation methods, with dependencies on sklearn and pydvl.valuation modules.

```python
import sklearn as sk
from pydvl.valuation import Dataset, ModelUtility, SupervisedScorer


train, test = Dataset.from_sklearn(sk.datasets.load_iris(), train_size=0.6)
model = sk.svm.SVC()
# Uses the model.score() method by default
scorer = SupervisedScorer(model, test)
utility = ModelUtility(model, scorer)
```

--------------------------------

### Import pyDVL functions for data valuation

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/least_core_basic.ipynb

Imports necessary functions and classes from pyDVL for data valuation, including exact and Monte Carlo Least Core valuation methods.

```python
from pydvl.reporting.plots import shaded_mean_std
from pydvl.reporting.scores import compute_removal_score
from pydvl.valuation import (
    Dataset,
    ExactLeastCoreValuation,
    ModelUtility,
    MonteCarloLeastCoreValuation,
    SupervisedScorer,
)
```

--------------------------------

### Computing exact Least-Core values with Python

Source: https://github.com/aai-institute/pydvl/blob/develop/docs/value/the-core.md

Demonstrates how to compute exact Least-Core values using pyDVL's ExactLeastCoreValuation class. Requires a model, dataset, and scorer. Uses joblib for parallel computation. The valuation ensures stability by satisfying efficiency and coalitional rationality properties.

```python
from joblib import parallel_config
from pydvl.valuation import (
    Dataset, ExactLeastCoreValuation, ModelUtility, SupervisedScorer
)

train, test = Dataset.from_arrays(...)
model = ...
scorer = SupervisedScorer(model, test, default=..)
utility = ModelUtility(model, scorer)
valuation = ExactLeastCoreValuation(utility, subsidy)

with parallel_config(n_jobs=12):
    valuation.fit(data)
```

--------------------------------

### Evaluate Model Performance on Test Subset

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_sentiment_analysis.ipynb

This snippet demonstrates how to prepare a subset of the test data, tokenize it, and get model predictions. It then calculates the F1 score to evaluate the model's performance. This requires a tokenizer, model, dataset, and F1 score function.

```python
sample_test_set = imdb["test"].shuffle(seed=seed).select(range(50 if not is_CI else 5))
sample_test_set = sample_test_set.map(
    lambda example: tokenizer(example["text"], truncation=True, padding="max_length"),
    batched=True,
)
sample_test_set.set_format("torch", columns=["input_ids", "attention_mask", "label"])
model.eval()
with torch.no_grad():
    logits = model(
        input_ids=sample_test_set["input_ids"],
        attention_mask=sample_test_set["attention_mask"],
    ).logits
    predictions = torch.argmax(logits, dim=1)
```

--------------------------------

### Create Memory-Mapped Dataset for Efficient Parallel Processing

Source: https://context7.com/aai-institute/pydvl/llms.txt

Shows how to create a memory-mapped `pydvl.valuation.Dataset` using NumPy arrays. Memory mapping avoids copying large datasets to subprocesses, enhancing efficiency during parallel processing with Joblib. Requires `numpy` and `pydvl`.

```python
import numpy as np
from pydvl.valuation import Dataset

# Assuming valuation is defined

# Create memory-mapped dataset
large_X = np.random.rand(1000000, 100)
large_y = np.random.randint(0, 10, 1000000)

# Memory mapping prevents copying data to subprocesses
dataset = Dataset(x=large_X, y=large_y, mmap=True)

# Efficient parallel processing
with parallel_config(n_jobs=16):
    valuation.fit(dataset)
```

--------------------------------

### Plot highest and lowest influence training images

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/influence_imagenet.ipynb

Filters training images with the same label as the test image, then visualizes the most and least influential training examples. Reveals patterns in what makes certain training points more influential than others.

```python
images_with_same_label = train_ds["labels"] == test_ds["labels"][test_image_idx]
influence_values_with_same_label = influences[test_image_idx][
    images_with_same_label
].cpu()
images_same_label = train_ds["images"][images_with_same_label].values
plot_lowest_highest_influence_images(
    influence_values_with_same_label, subset_images=images_same_label, num_to_plot=3
)
```

--------------------------------

### Store Test Durations with Tox or Pytest

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

This shell command stores test durations for pytest-split to balance parallel execution. It supports tox environments or direct pytest. No dependencies beyond pytest-split installed. Outputs a .test_durations file; assume average times for new tests initially.

```shell
tox -e tests -- --store-durations --slow-tests
```

```shell
pytest --store-durations --slow-tests
```

--------------------------------

### Run Utility Learning Experiment in Python

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_utility_learning.ipynb

Encapsulates a single experiment run for Data Utility Learning (DUL). This function initializes the IndicatorUtilityModel with an MLPRegressor, wraps it with DataUtilityLearning, sets up a PermutationSampler with RelativeTruncation, and defines a stopping criterion using MaxUpdates. It then fits the DUL utility model to the training data and measures the execution time.

```python
from joblib import Parallel, delayed
from sklearn.neural_network import MLPRegressor

from pydvl.utils.functional import suppress_warnings, timed
from pydvl.valuation.result import ValuationResult
from pydvl.valuation.samplers import PermutationSampler
from pydvl.valuation.samplers.truncation import RelativeTruncation
from pydvl.valuation.stopping import MaxUpdates
from pydvl.valuation.utility import DataUtilityLearning
from pydvl.valuation.utility.learning import IndicatorUtilityModel


@suppress_warnings(categories=(RuntimeWarning,))
def run_once(run: int, budget: int) -> tuple[int, int, ValuationResult, float]:
    # DUL will kick in after `budget` calls to utility
    utility_model = IndicatorUtilityModel(MLPRegressor(**mlp_params), n_data=len(train))
    dul_utility = DataUtilityLearning(
        utility=utility,
        training_budget=budget,
        model=utility_model,
        show_warnings=False,
    )

    truncation = RelativeTruncation(rtol=0.001)
    sampler = PermutationSampler(truncation=truncation)
    stopping = MaxUpdates(300)
    valuation = ShapleyValuation(dul_utility, sampler, is_done=stopping, progress=False)

    # Note that DUL does not support parallel fitting (yet?)
    timed_fit = timed(valuation.fit)
    timed_fit(train)

    return run, budget, valuation.result, timed_fit.execution_time
```

--------------------------------

### Skorch Scorer and Model Utility Setup

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_skorch.ipynb

This code sets up a SkorchSupervisedScorer to evaluate model performance on test data and groups the model and scorer into a ModelUtility object. This utility is essential for subsequent valuation methods like Shapley value computation.

```python
from pydvl.valuation.scorers import SkorchSupervisedScorer
from pydvl.valuation.utility import ModelUtility

accuracy_over_test_set = SkorchSupervisedScorer(
    model, test_data=test, default=0.0, range=(0, 1)
)

utility = ModelUtility(model=model, scorer=accuracy_over_test_set)
```

--------------------------------

### Configure Sampler and Stopping Criterion in Python

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_utility_learning.ipynb

Initializes a DeterministicUniformSampler for generating data subsets and a NoStopping criterion to ensure all subsets are processed. These are essential for exact Shapley value computation. Dependencies: pydvl.valuation.DeterministicUniformSampler, pydvl.valuation.NoStopping.

```python
from pydvl.valuation import (
    DeterministicUniformSampler,
    NoStopping,
    ShapleyValuation,
)

sampler = DeterministicUniformSampler(batch_size=32)
stopping = NoStopping(sampler)

valuation = ShapleyValuation(
    utility=utility, sampler=sampler, is_done=stopping, progress=True
)
```

--------------------------------

### Import pyDVL Valuation Components

Source: https://github.com/aai-institute/pydvl/blob/develop/notebooks/shapley_basic_spotify.ipynb

Imports core classes and functions from the pyDVL library for implementing Shapley-based data valuation. This includes classes for datasets, samplers, scorers, stopping criteria, and model utilities.

```python
from pydvl.reporting.plots import plot_result_errors
from pydvl.valuation.dataset import Dataset, GroupedDataset
from pydvl.valuation.methods.shapley import ShapleyValuation
from pydvl.valuation.samplers import PermutationSampler, RelativeTruncation
from pydvl.valuation.scorers import SupervisedScorer
from pydvl.valuation.stopping import HistoryDeviation, MaxUpdates
from pydvl.valuation.utility import ModelUtility
```

--------------------------------

### Starting Memcached with Docker for pyDVL Testing

Source: https://github.com/aai-institute/pydvl/blob/develop/CONTRIBUTING.md

Memcached is required for caching in tests and speeding up methods like Permutation Shapley. This command runs Memcached in a Docker container in the background on the default port. It exposes port 11211; use --memcached-service flag in tox to connect if needed.

```shell
docker run --name pydvl-memcache -p 11211:11211 -d memcached
```