UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

https://umap-learn.readthedocs.io/en/latest/
Admin
UMAP is a dimension reduction technique that can be used for visualisation similarly to t-SNE, but...
Tokens:122,760
Snippets:1,243
Trust Score:8.3
Update:2 months ago
Show doc for...
Tokens:
Raw
### Importing Required Libraries

Source: https://umap-learn.readthedocs.io/en/latest/basic_usage.html

Initial setup for data manipulation, visualization, and scikit-learn utilities.

```python
import numpy as np
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
%matplotlib inline
```

--------------------------------

### Setup MNIST Data and Parameters

Source: https://umap-learn.readthedocs.io/en/latest/precomputed_k-nn.html

Initializes the MNIST dataset and defines the grid of n_neighbors and min_dist parameters for testing.

```python
from sklearn.datasets import fetch_openml
import numpy as np
import umap
import umap.plot
from umap.umap_ import nearest_neighbors

data, labels = fetch_openml('mnist_784', version=1, return_X_y=True)
labels = np.asarray(labels, dtype=np.int32)

n_neighbors = [5, 50, 100, 250]
min_dists = [0, 0.2, 0.5, 0.9]
normal_embeddings = np.zeros((4, 4, 70000, 2))
precomputed_knn_embeddings = np.zeros((4, 4, 70000, 2))
```

--------------------------------

### Importing required libraries

Source: https://umap-learn.readthedocs.io/en/latest/parameters.html

Initial setup for data manipulation, visualization, and UMAP dimension reduction.

```python
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import seaborn as sns
import umap
%matplotlib inline
```

--------------------------------

### Importing required libraries

Source: https://umap-learn.readthedocs.io/en/latest/aligned_umap_politics_demo.html

Initial setup including UMAP, data manipulation, and string matching libraries.

```python
import umap
import umap.utils as utils
import umap.aligned_umap
import sklearn.decomposition

import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
import seaborn as sns

from fuzzywuzzy import fuzz, process
import re
```

--------------------------------

### Benchmark execution output

Source: https://umap-learn.readthedocs.io/en/latest/performance.html

Example console output showing completion timestamps for each algorithm.

```text
[Sat Feb 22 09:50:24 2020] Completed PCA
[Sat Feb 22 09:51:23 2020] Completed UMAP
[Sat Feb 22 09:53:24 2020] Completed MulticoreTSNE
[Sat Feb 22 10:00:50 2020] Completed OpenTSNE
[Sat Feb 22 10:02:22 2020] Completed TSNE
[Sat Feb 22 10:02:44 2020] Completed LocallyLinearEmbedding
[Sat Feb 22 10:03:06 2020] Completed SpectralEmbedding
[Sat Feb 22 10:03:31 2020] Completed Isomap
[Sat Feb 22 10:11:45 2020] Completed MDS
```

--------------------------------

### Install UMAP via PyPI

Source: https://umap-learn.readthedocs.io/en/latest/index.html

Use this command to install the package via pip. Ensure dependencies like numba, sklearn, numpy, and scipy are installed first.

```bash
pip install umap-learn
```

--------------------------------

### Install UMAP with Pip

Source: https://umap-learn.readthedocs.io/en/latest/_sources/index.rst.txt

Install the UMAP library using pip. Ensure that numba and sklearn, along with their dependencies (numpy and scipy), are already installed.

```bash
pip install umap-learn
```

--------------------------------

### Fast algorithm benchmark output

Source: https://umap-learn.readthedocs.io/en/latest/performance.html

Example console output for the larger dataset benchmark.

```text
[Sat Feb 22 10:12:15 2020] Completed PCA
[Sat Feb 22 10:14:51 2020] Completed UMAP
[Sat Feb 22 11:16:05 2020] Completed MulticoreTSNE
[Sat Feb 22 11:50:17 2020] Completed OpenTSNE
[Sat Feb 22 13:06:38 2020] Completed TSNE
[Sat Feb 22 14:14:36 2020] Completed LocallyLinearEmbedding
```

--------------------------------

### Execution timing output

Source: https://umap-learn.readthedocs.io/en/latest/supervised.html

Example output showing the wall time for the UMAP fitting process.

```text
CPU times: user 3min 8s, sys: 7.85 s, total: 3min 16s
Wall time: 2min 40s
```

--------------------------------

### Install UMAP via Conda

Source: https://umap-learn.readthedocs.io/en/latest/index.html

Use this command to install the umap-learn package using the conda-forge channel.

```bash
conda install -c conda-forge umap-learn
```

--------------------------------

### Runtime warning output

Source: https://umap-learn.readthedocs.io/en/latest/parameters.html

Example of a runtime warning encountered during execution.

```text
/opt/anaconda3/envs/umap_dev/lib/python3.6/site-packages/sklearn/metrics/pairwise.py:257: RuntimeWarning: invalid value encountered in sqrt
  return distances if squared else np.sqrt(distances, out=distances)
```

--------------------------------

### Install UMAP with Conda

Source: https://umap-learn.readthedocs.io/en/latest/_sources/index.rst.txt

Install the UMAP library using the conda package manager from the conda-forge channel. This is recommended for Linux, OS X, and Windows 64-bit systems.

```bash
conda install -c conda-forge umap-learn
```

--------------------------------

### Import umap.plot package

Source: https://umap-learn.readthedocs.io/en/latest/_sources/plotting.rst.txt

Import the umap.plot package for visualization utilities. Ensure matplotlib, pandas, datashader, bokeh, and holoviews are installed.

```python3
import umap.plot
```

--------------------------------

### Load sample datasets

Source: https://umap-learn.readthedocs.io/en/latest/_sources/plotting.rst.txt

Load the pendigits, mnist, and fmnist datasets for demonstration.

```python3
pendigits = sklearn.datasets.load_digits()
mnist = sklearn.datasets.fetch_openml('mnist_784')
fmnist = sklearn.datasets.fetch_openml('Fashion-MNIST')
```

--------------------------------

### Loading datasets

Source: https://umap-learn.readthedocs.io/en/latest/plotting.html

Loading sample datasets for demonstration purposes.

```python
pendigits = sklearn.datasets.load_digits()
mnist = sklearn.datasets.fetch_openml('mnist_784')
fmnist = sklearn.datasets.fetch_openml('Fashion-MNIST')
```

--------------------------------

### Initialize Algorithms for Performance Testing

Source: https://umap-learn.readthedocs.io/en/latest/_sources/performance.rst.txt

Creates a list of various dimension reduction algorithm instances to be used in the performance comparison.

```python
all_algorithms = [
    PCA(),
    UMAP(),
    MulticoreTSNE(),
    OpenTSNE(),
    TSNE(),
    LocallyLinearEmbedding(),
    SpectralEmbedding(), 
    Isomap(), 
    MDS(),
]
performance_data = {}
for algorithm in all_algorithms:
    if 'openTSNE' in str(algorithm.__class__):
        alg_name = "OpenTSNE"
    elif 'MulticoreTSNE' in str(algorithm.__class__):
        alg_name = "MulticoreTSNE"
    else:
        alg_name = str(algorithm).split('(')[0]
        
```

--------------------------------

### Importing base libraries

Source: https://umap-learn.readthedocs.io/en/latest/supervised.html

Initializes the environment with necessary libraries for data manipulation, loading, and visualization.

```python
import numpy as np
from mnist.loader import MNIST
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
sns.set(style='white', context='poster')
```

--------------------------------

### Configure Plotting Tools

Source: https://umap-learn.readthedocs.io/en/latest/benchmarking.html

Initialize matplotlib and seaborn for visualization.

```python
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
```

--------------------------------

### Initialize tqdm progress bar settings

Source: https://umap-learn.readthedocs.io/en/latest/_modules/umap/umap_.html

Sets default parameters for the tqdm progress bar if not provided by the user.

```python
if self.tqdm_kwds is None:
    self.tqdm_kwds = {}
else:
    if isinstance(self.tqdm_kwds, dict) is False:
        raise ValueError(
            "tqdm_kwds must be a dictionary. Please provide valid tqdm "
            "parameters as key value pairs. Valid tqdm parameters can be "
            "found here: https://github.com/tqdm/tqdm#parameters"
        )
if "desc" not in self.tqdm_kwds:
    self.tqdm_kwds["desc"] = "Epochs completed"
if "bar_format" not in self.tqdm_kwds:
    bar_f = "{desc}: {percentage:3.0f}%| {bar} {n_fmt}/{total_fmt} [{elapsed}]"
    self.tqdm_kwds["bar_format"] = bar_f
```

--------------------------------

### Define Simple Custom Metric

Source: https://umap-learn.readthedocs.io/en/latest/parameters.html

Example of a Numba-compiled function calculating distance based on the red channel.

```python
@numba.njit()
def red_channel_dist(a,b):
    return np.abs(a[0] - b[0])
```

--------------------------------

### Importing required libraries

Source: https://umap-learn.readthedocs.io/en/latest/embedding_space.html

Initializes the environment with necessary libraries for data manipulation, visualization, and UMAP processing.

```python
import numpy as np
import numba
import sklearn.datasets
import matplotlib.pyplot as plt
import seaborn as sns
from mpl_toolkits.mplot3d import Axes3D
import umap
%matplotlib inline
```

--------------------------------

### Import required libraries for sparse UMAP

Source: https://umap-learn.readthedocs.io/en/latest/sparse.html

Initializes the environment with necessary libraries including numpy, scipy.sparse, sklearn, and umap.

```python
import numpy as np
import scipy.sparse
import sympy
import sklearn.datasets
import sklearn.feature_extraction.text
import umap
import umap.plot
import matplotlib.pyplot as plt
%matplotlib inline
```

--------------------------------

### Import Benchmarking Dependencies

Source: https://umap-learn.readthedocs.io/en/latest/benchmarking.html

Load essential libraries for data manipulation, resampling, and timing.

```python
import numpy as np
import pandas as pd
from sklearn.datasets import fetch_openml
from sklearn.utils import resample
import time
```

--------------------------------

### Perform Supervised UMAP Embedding

Source: https://umap-learn.readthedocs.io/en/latest/supervised.html

Pass the target labels to the fit_transform method to guide the dimension reduction process.

```python
%%time
embedding = umap.UMAP().fit_transform(data, y=target)
```

--------------------------------

### Importing Required Libraries

Source: https://umap-learn.readthedocs.io/en/latest/transform_landmarked_pumap.html

Initial imports for data handling, model training, and visualization.

```python
import keras
from sklearn.model_selection import train_test_split

from umap import UMAP, ParametricUMAP

import matplotlib.pyplot as plt

import numpy as np
import pandas as pd
```

--------------------------------

### Basic Parametric UMAP Usage

Source: https://umap-learn.readthedocs.io/en/latest/_sources/parametric_umap.rst.txt

Replace standard UMAP with ParametricUMAP for basic embedding. Ensure `parametric_umap` is installed and imported.

```python3
from umap.parametric_umap import ParametricUMAP
embedder = ParametricUMAP()
embedding = embedder.fit_transform(my_data)
```

--------------------------------

### Import Libraries for UMAP

Source: https://umap-learn.readthedocs.io/en/latest/_sources/composing_models.rst.txt

Imports necessary libraries for data manipulation, scaling, and UMAP modeling. Ensure these are installed before running.

```python
import sklearn.datasets
from sklearn.preprocessing import RobustScaler
import seaborn as sns
import pandas as pd
import numpy as np
import umap
import umap.plot
```

--------------------------------

### Display Sample Documents

Source: https://umap-learn.readthedocs.io/en/latest/_sources/document_embedding.rst.txt

Iterates through the first three documents of the dataset, printing their category and the first 500 characters of their content.

```python
for idx, document in enumerate(dataset.data[:3]):
    category = dataset.target_names[dataset.target[idx]]
    
    print(f'Category: {category}')
    print('---------------------------')
    # Print the first 500 characters of the post
    print(document[:500])
    print('---------------------------')
```

--------------------------------

### Import necessary libraries for image embedding

Source: https://umap-learn.readthedocs.io/en/latest/_sources/basic_usage.rst.txt

Imports required for encoding images into a base64 format for embedding in plots. Ensure you have Pillow and base64 installed.

```python3
from io import BytesIO
from PIL import Image
import base64
```

--------------------------------

### Initialize Bokeh Environment

Source: https://umap-learn.readthedocs.io/en/latest/basic_usage.html

Configures the notebook environment for Bokeh plotting.

```python
from bokeh.plotting import figure, show, output_notebook
from bokeh.models import HoverTool, ColumnDataSource, CategoricalColorMapper
from bokeh.palettes import Spectral10

output_notebook()
```

--------------------------------

### Fit UMAP with Hyperboloid Metric

Source: https://umap-learn.readthedocs.io/en/latest/_sources/embedding_space.rst.txt

Fits a UMAP model to the data using the 'hyperboloid' metric for hyperbolic space embedding. Ensure the 'umap-learn' library is installed.

```python3
hyperbolic_mapper = umap.UMAP(output_metric='hyperboloid', 
                                  random_state=42).fit(digits.data)
```

--------------------------------

### Run Benchmarks for All Algorithms

Source: https://umap-learn.readthedocs.io/en/latest/_sources/benchmarking.rst.txt

Initializes a list of dimension reduction algorithms and iterates through them, applying the data_size_scaling function to collect performance data for each. The results are stored in a dictionary keyed by algorithm name.

```ipython3
all_algorithms = [
    PCA(),
    UMAP(),
    MulticoreTSNE(),
    LocallyLinearEmbedding(),
    SpectralEmbedding(), 
    Isomap(), 
    TSNE(),
    MDS(),
]
performance_data = {}
for algorithm in all_algorithms:
    alg_name = str(algorithm)
    if 'MulticoreTSNE' in alg_name:
        alg_name = 'MulticoreTSNE'
    else:
        alg_name = alg_name.split('(')[0]
        
    performance_data[alg_name] = data_size_scaling(algorithm, mnist.data, n_runs=3)
```

--------------------------------

### Fit Base AlignedUMAP Model

Source: https://umap-learn.readthedocs.io/en/latest/_sources/aligned_umap_basic_usage.rst.txt

Initializes and fits an AlignedUMAP model using the first two data slices and one relation dictionary. This is the starting point for online updating.

```python3
%%time
updating_mapper = umap.AlignedUMAP().fit(slices[:2], relations=relation_dicts[:1])
```

--------------------------------

### Importing required libraries

Source: https://umap-learn.readthedocs.io/en/latest/plotting.html

Initial imports for data handling and UMAP model creation.

```python
import sklearn.datasets
import pandas as pd
import numpy as np
import umap
```

--------------------------------

### Defining a Custom CNN Encoder Network

Source: https://umap-learn.readthedocs.io/en/latest/_sources/parametric_umap.rst.txt

Define a custom neural network architecture, such as a Convolutional Neural Network (CNN), for Parametric UMAP using TensorFlow/Keras. This example shows a network suitable for MNIST images.

```python3
import tensorflow as tf
dims = (28, 28, 1)
n_components = 2
encoder = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=dims),
    tf.keras.layers.Conv2D(
        filters=32, kernel_size=3, strides=(2, 2), activation="relu", padding="same"
    ),
    tf.keras.layers.Conv2D(
        filters=64, kernel_size=3, strides=(2, 2), activation="relu", padding="same"
    ),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(units=256, activation="relu"),
    tf.keras.layers.Dense(units=256, activation="relu"),
    tf.keras.layers.Dense(units=n_components),
])
encoder.summary()
```

--------------------------------

### Initializing UMAP

Source: https://umap-learn.readthedocs.io/en/latest/basic_usage.html

Imports the umap library and instantiates the UMAP reducer object.

```python
import umap
```

```python
reducer = umap.UMAP()
```

--------------------------------

### Configure Validation Data and Train

Source: https://umap-learn.readthedocs.io/en/latest/parametric_umap.html

Prepare validation data and initialize ParametricUMAP with encoder and decoder networks.

```python
validation_images = test_images.reshape((test_images.shape[0], -1))/255.
```

```python
embedder = ParametricUMAP(
    encoder=encoder,
    decoder=decoder,
    dims=dims,
    parametric_reconstruction= True,
    reconstruction_validation=validation_images,
    verbose=True,
)
embedding = embedder.fit_transform(train_images)
```

--------------------------------

### Importing Required Modules

Source: https://umap-learn.readthedocs.io/en/latest/transform.html

Load necessary libraries for data handling, machine learning, and visualization.

```python
import numpy as np
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC

import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
```

--------------------------------

### Import Basic Tools

Source: https://umap-learn.readthedocs.io/en/latest/_sources/benchmarking.rst.txt

Imports necessary libraries for data manipulation, fetching data, resampling, and timing operations.

```ipython3
import numpy as np
import pandas as pd
from sklearn.datasets import fetch_openml
from sklearn.utils import resample
import time
```

--------------------------------

### Initialize Prime Mapping

Source: https://umap-learn.readthedocs.io/en/latest/sparse.html

Generate a list of primes and a dictionary mapping each prime to a unique column index.

```python
primes = list(sympy.primerange(2, 110000))
prime_to_column = {p:i for i, p in enumerate(primes)}
```

--------------------------------

### Prepare Training and Test Data

Source: https://umap-learn.readthedocs.io/en/latest/supervised.html

Convert training and test datasets into numpy arrays.

```python
train_data = np.array(train)
test_data = np.array(test)
```

--------------------------------

### Construct Sparse Matrix Data (LIL Format)

Source: https://umap-learn.readthedocs.io/en/latest/_sources/sparse.rst.txt

Iterates through integers to build lists of non-zero entries (prime factor column indices) and their corresponding values (all ones) for constructing a sparse matrix in LIL format.

```python3
%%time
lil_matrix_rows = []
lil_matrix_data = []
for n in range(100000):
    prime_factors = sympy.primefactors(n)
    lil_matrix_rows.append([prime_to_column[p] for p in prime_factors])
    lil_matrix_data.append([1] * len(prime_factors))
```

--------------------------------

### Import Plotting Tools and Fetch Data

Source: https://umap-learn.readthedocs.io/en/latest/_sources/benchmarking.rst.txt

Imports plotting libraries and fetches the Fashion-MNIST dataset for benchmarking.

```ipython3
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
```

```ipython3
sns.set(context='notebook', 
        rc={'figure.figsize':(12,10)},
        palette=sns.color_palette('tab10', 10))
```

```ipython3
mnist = fetch_openml('Fashion-MNIST', version=1)
```

--------------------------------

### Downloading the 20 newsgroups dataset

Source: https://umap-learn.readthedocs.io/en/latest/document_embedding.html

Fetch the 20 newsgroups dataset for analysis.

```python
%%time
dataset = fetch_20newsgroups(subset='all',
                             shuffle=True, random_state=42)
```

--------------------------------

### Plot Points with Contrast Mapper

Source: https://umap-learn.readthedocs.io/en/latest/_sources/composing_models.rst.txt

Visualizes data points using a contrast mapper and price values, with a 'viridis' colormap. This helps in observing price gradients within the composed structure.

```python3
umap.plot.points(contrast_mapper, values=diamonds["price"], cmap="viridis")
```

--------------------------------

### Run UMAP with Default Settings

Source: https://umap-learn.readthedocs.io/en/latest/_sources/reproducibility.rst.txt

Fits UMAP to the loaded data using default settings, which leverage multi-threading for performance. This run demonstrates the default stochastic behavior without a fixed random seed.

```python
%%time
mapper1 = umap.UMAP().fit(data)
```

--------------------------------

### Import Plotting Tools

Source: https://umap-learn.readthedocs.io/en/latest/_sources/performance.rst.txt

Imports matplotlib and seaborn for plotting and sets up the plotting environment.

```python
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
```

--------------------------------

### Initialize and Fit AlignedUMAP with Parameter Lists

Source: https://umap-learn.readthedocs.io/en/latest/aligned_umap_basic_usage.html

Configures AlignedUMAP with lists of parameters and fits the model using repeated datasets.

```python
neighbors_mapper = umap.AlignedUMAP(
    n_neighbors=[3,4,5,7,11,16,22,29,37,45,54],
    min_dist=[0.01,0.05,0.1,0.15,0.2,0.25,0.3,0.35,0.4,0.45],
    alignment_window_size=2,
    alignment_regularisation=1e-3,
).fit(
    [digits.data for i in range(10)], relations=constant_relations
)
```

--------------------------------

### Importing UMAP

Source: https://umap-learn.readthedocs.io/en/latest/transform.html

Load the umap module.

```python
import umap
```

--------------------------------

### Create Sparse Matrix

Source: https://umap-learn.readthedocs.io/en/latest/sparse.html

Initialize a scipy.sparse LIL matrix and populate it with the generated row and data structures.

```python
factor_matrix = scipy.sparse.lil_matrix((len(lil_matrix_rows), len(primes)), dtype=np.float32)
factor_matrix.rows = np.array(lil_matrix_rows)
factor_matrix.data = np.array(lil_matrix_data)
factor_matrix
```

--------------------------------

### Initialize and Fit Parametric UMAP

Source: https://umap-learn.readthedocs.io/en/latest/transform_landmarked_pumap.html

Initializes a ParametricUMAP model and fits it to the input data.

```python
p_embedder = ParametricUMAP()

p_emb_x1 = p_embedder.fit_transform(x1)
```

--------------------------------

### Fetch Fashion-MNIST Dataset

Source: https://umap-learn.readthedocs.io/en/latest/benchmarking.html

Download the Fashion-MNIST dataset using scikit-learn's fetch_openml.

```python
mnist = fetch_openml('Fashion-MNIST', version=1)
```

--------------------------------

### Visualize contrast mapper

Source: https://umap-learn.readthedocs.io/en/latest/composing_models.html

Plots the contrast model colored by price.

```python
umap.plot.points(contrast_mapper, values=diamonds["price"], cmap="viridis")
```

--------------------------------

### UMAP Model Initialization and Fitting

Source: https://umap-learn.readthedocs.io/en/latest/_sources/basic_usage.rst.txt

Initializes a UMAP reducer with a specified random state and fits it to the digits dataset. This trains the model without immediately returning the transformed data.

```python3
reducer = umap.UMAP(random_state=42)
reducer.fit(digits.data)
```

--------------------------------

### Visualize fast algorithm performance

Source: https://umap-learn.readthedocs.io/en/latest/benchmarking.html

Plots the runtime scaling for the selected fast algorithms.

```python
for alg_name, perf_data in fast_performance_data.items():
    sns.regplot('dataset size', 'runtime (s)', perf_data, order=2, label=alg_name)
plt.legend();
```

--------------------------------

### Import Necessary Libraries

Source: https://umap-learn.readthedocs.io/en/latest/_sources/transform.rst.txt

Loads numpy, scikit-learn modules for data handling and modeling, and matplotlib/seaborn for plotting.

```python3
import numpy as np
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC

import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
```

--------------------------------

### Import Animation and Display Tools

Source: https://umap-learn.readthedocs.io/en/latest/_sources/aligned_umap_basic_usage.rst.txt

Import necessary modules from IPython.display and matplotlib for creating and displaying animations. This is required for rendering the embedding evolution as a GIF.

```python3
from IPython.display import display, Image, HTML
from matplotlib import animation
```

--------------------------------

### Download 20 Newsgroups Dataset

Source: https://umap-learn.readthedocs.io/en/latest/_sources/document_embedding.rst.txt

Downloads the 20 newsgroups dataset for text analysis. The %%time magic command measures the execution time.

```python
%%time
dataset = fetch_20newsgroups(subset='all',
                                 shuffle=True, random_state=42)
```

--------------------------------

### Prepare Data for Parametric UMAP Retraining with Landmarks

Source: https://umap-learn.readthedocs.io/en/latest/_sources/transform_landmarked_pumap.rst.txt

Selects landmark indices from the initial dataset (x1), concatenates them with the new dataset (x2) and their labels (y2), and constructs a landmark positions array. This prepares the data for fine-tuning the ParametricUMAP model.

```python3
landmark_idx = list(np.random.choice(range(x1.shape[0]), int(x1.shape[0]/100), replace=False))

x2_lmk = np.concatenate((x2, x1[landmark_idx]))
y2_lmk = np.concatenate((y2, y1[landmark_idx]))

landmarks = np.stack(
    [np.array([np.nan, np.nan])]*x2.shape[0] + list(
        p_embedder.transform(
            x1[landmark_idx]
        )
    )
)

p_embedder.landmark_loss_weight = 0.01
p_embedder.fit(x2_lmk, landmark_positions=landmarks)
p_emb2_x2 = p_embedder.transform(x2)

p_emb2_x1 = p_embedder.transform(x1)
```

--------------------------------

### Import UMAP and dependencies

Source: https://umap-learn.readthedocs.io/en/latest/reproducibility.html

Load the necessary libraries for data handling and UMAP embedding.

```python
import numpy as np
import sklearn.datasets
import umap
import umap.plot
```

--------------------------------

### Load and Prepare Fashion-MNIST Data

Source: https://umap-learn.readthedocs.io/en/latest/_sources/supervised.rst.txt

Loads the Fashion-MNIST dataset, combines training and testing sets, normalizes pixel values to the range [0,1], and defines class labels. Assumes the dataset is located in 'fashion-mnist/data/fashion'.

```python3
mndata = MNIST('fashion-mnist/data/fashion')
train, train_labels = mndata.load_training()
test, test_labels = mndata.load_testing()
data = np.array(np.vstack([train, test]), dtype=np.float64) / 255.0
target = np.hstack([train_labels, test_labels])
classes = [
    'T-shirt/top',
    'Trouser',
    'Pullover',
    'Dress',
    'Coat',
    'Sandal',
    'Shirt',
    'Sneaker',
    'Bag',
    'Ankle boot']
```

--------------------------------

### Display DensMAP execution time for Fashion-MNIST

Source: https://umap-learn.readthedocs.io/en/latest/densmap_demo.html

Output of the timing command for DensMAP on Fashion-MNIST.

```text
CPU times: user 3min 48s, sys: 8.07 s, total: 3min 56s
Wall time: 2min 21s
```

--------------------------------

### Epoch and Coordinate Utilities

Source: https://umap-learn.readthedocs.io/en/latest/_modules/umap/umap_.html

Utilities for calculating training epochs per sample and applying noise to coordinate scaling.

```python
def make_epochs_per_sample(weights, n_epochs):
    """Given a set of weights and number of epochs generate the number of
    epochs per sample for each weight.

    Parameters
    ----------
    weights: array of shape (n_1_simplices)
        The weights of how much we wish to sample each 1-simplex.

    n_epochs: int
        The total number of epochs we want to train for.

    Returns
    -------
    An array of number of epochs per sample, one for each 1-simplex.
    """
    result = np.full(weights.shape[0], -1.0, dtype=np.float64)
    n_samples = n_epochs * (weights / weights.max())
    positive = n_samples > 0  # compute the mask once instead of twice
    result[positive] = float(n_epochs) / np.float64(n_samples[positive])
    return result
```

```python
def noisy_scale_coords(coords, random_state, max_coord=10.0, noise=0.0001):
    expansion = max_coord / np.abs(coords).max()
    coords = (coords * expansion).astype(np.float32)
    return coords + random_state.normal(scale=noise, size=coords.shape).astype(
        np.float32
    )
```

--------------------------------

### Initialize Graph-Based Transformation

Source: https://umap-learn.readthedocs.io/en/latest/_modules/umap/umap_.html

Initializes new point positions using a bipartite graph. Handles exact matches and weighted averages of neighbors.

```python
def init_graph_transform(graph, embedding):
    """Given a bipartite graph representing the 1-simplices and strengths between the
    new points and the original data set along with an embedding of the original points
    initialize the positions of new points relative to the strengths (of their neighbors in the source data).

    If a point is in our original data set it embeds at the original points coordinates.
    If a point has no neighbours in our original dataset it embeds as the np.nan vector.
    Otherwise a point is the weighted average of it's neighbours embedding locations.

    Parameters
    ----------
    graph: csr_matrix (n_new_samples, n_samples)
        A matrix indicating the 1-simplices and their associated strengths.  These strengths should
        be values between zero and one and not normalized.  One indicating that the new point was identical
        to one of our original points.

    embedding: array of shape (n_samples, dim)
        The original embedding of the source data.

    Returns
    -------
    new_embedding: array of shape (n_new_samples, dim)
        An initial embedding of the new sample points.
    """
    n_new = graph.shape[0]
    result = np.zeros((n_new, embedding.shape[1]), dtype=np.float32)

    row_nnz = np.diff(graph.indptr)
    empty_mask = row_nnz == 0
    result[empty_mask] = np.nan

    has_exact = np.zeros(n_new, dtype=bool)
    exact_data_mask = graph.data == 1.0
    if exact_data_mask.any():
        exact_positions = np.where(exact_data_mask)[0]
        exact_rows = np.searchsorted(graph.indptr, exact_positions, side="right") - 1
        _, first_idx = np.unique(exact_rows, return_index=True)
        unique_rows = exact_rows[first_idx]
        exact_cols = graph.indices[exact_positions[first_idx]]
        has_exact[unique_rows] = True
        result[unique_rows] = embedding[exact_cols]

    avg_mask = ~empty_mask & ~has_exact
    if np.any(avg_mask):
        avg_graph = graph[avg_mask]
        row_sums = np.array(avg_graph.sum(axis=1)).flatten()
        inv_sums = scipy.sparse.diags(1.0 / row_sums)
        normalized = inv_sums @ avg_graph
        result[avg_mask] = (normalized @ embedding).astype(np.float32)

    return result
```

--------------------------------

### Train UMAP model

Source: https://umap-learn.readthedocs.io/en/latest/transform_landmarked_pumap.html

Initialize and fit a UMAP model on the initial data partition.

```python
embedder = UMAP()

emb_x1 = embedder.fit_transform(x1)
```

--------------------------------

### Run UMAP Again for Comparison

Source: https://umap-learn.readthedocs.io/en/latest/_sources/reproducibility.rst.txt

Executes UMAP again on the same data with default settings to observe performance improvements due to Numba's JIT compilation and to highlight the inherent variability in stochastic algorithms.

```python
%%time
mapper2 = umap.UMAP().fit(data)
```

--------------------------------

### Load the pendigits dataset

Source: https://umap-learn.readthedocs.io/en/latest/_sources/aligned_umap_basic_usage.rst.txt

Loads the pendigits dataset from scikit-learn for demonstration purposes.

```python3
digits = sklearn.datasets.load_digits()
```

--------------------------------

### Sampling documents

Source: https://umap-learn.readthedocs.io/en/latest/document_embedding.html

Iterate through the first three documents to display their category and content.

```python
for idx, document in enumerate(dataset.data[:3]):
    category = dataset.target_names[dataset.target[idx]]

    print(f'Category: {category}')
    print('---------------------------')
    # Print the first 500 characters of the post
    print(document[:500])
    print('---------------------------')
```

--------------------------------

### Import animation and display tools

Source: https://umap-learn.readthedocs.io/en/latest/aligned_umap_basic_usage.html

Required imports for rendering animations in a notebook environment.

```python
from IPython.display import display, Image, HTML
from matplotlib import animation
```

--------------------------------

### Run UMAP fitting

Source: https://umap-learn.readthedocs.io/en/latest/reproducibility.html

Execute the UMAP fitting process. Note that this uses multi-threading by default.

```python
%%time
mapper1 = umap.UMAP().fit(data)
```

```python
%%time
mapper2 = umap.UMAP().fit(data)
```

--------------------------------

### Load MNIST and Fashion-MNIST datasets

Source: https://umap-learn.readthedocs.io/en/latest/_sources/densmap_demo.rst.txt

Loads the MNIST and Fashion-MNIST datasets using scikit-learn.

```python
mnist = sklearn.datasets.fetch_openml("mnist_784")
fmnist = sklearn.datasets.fetch_openml("Fashion-MNIST")
```

--------------------------------

### Load Fashion MNIST Dataset

Source: https://umap-learn.readthedocs.io/en/latest/inverse_transform.html

Fetches the Fashion MNIST dataset using scikit-learn.

```python
data, labels = sklearn.datasets.fetch_openml('Fashion-MNIST', version=1, return_X_y=True)
```

--------------------------------

### Run Normal UMAP and Precomputed k-NN UMAP

Source: https://umap-learn.readthedocs.io/en/latest/_sources/precomputed_k-nn.rst.txt

This snippet demonstrates how to run UMAP normally and then with precomputed k-NN. It highlights the importance of using `check_random_state` to ensure the random number generator is mutated in the same way as UMAP's internal fit function, allowing for reproducible results.

```python3
from sklearn.utils import check_random_state

# First we run the normal UMAP to compare with
random_seed3 = 12
normal_umap = umap.UMAP(n_neighbors=30, random_state=random_seed3).fit(X)

# Now we run precomputed_knn UMAP
random_state3 = check_random_state(random_seed3)
# random_state3 = numpy.random.RandomState(random_seed3)
knn5 = nearest_neighbors(
                        X, 
                        n_neighbors=30,
                        metric='euclidean',
                        metric_kwds=None,
                        angular=False,
                        random_state=random_state3,
                        )
# This mutated RNG can now be fed into precompute_knn UMAP to obtain
# the same results as in normal UMAP
knn_umap5 = umap.UMAP(n_neighbors=30, 
                      precomputed_knn=knn5, 
                      random_state=random_state3,  # <--- This is a RNG
                     ).fit(X)
```

--------------------------------

### Compose models using contrast

Source: https://umap-learn.readthedocs.io/en/latest/composing_models.html

Creates a contrast mapper by subtracting the ordinal model from the numeric model.

```python
contrast_mapper = numeric_mapper - ordinal_mapper
```

--------------------------------

### Create Sparse Matrix from Factors

Source: https://umap-learn.readthedocs.io/en/latest/_sources/sparse.rst.txt

Constructs a LIL sparse matrix for UMAP input using prime factorization data. Ensure 'primes' and 'prime_to_column' are defined beforehand.

```python3
factor_matrix = scipy.sparse.lil_matrix((len(lil_matrix_rows), len(primes)), dtype=np.float32)
factor_matrix.rows = np.array(lil_matrix_rows)
factor_matrix.data = np.array(lil_matrix_data)
factor_matrix
```

--------------------------------

### Load and Prepare Galaxy10SDSS Dataset Subset

Source: https://umap-learn.readthedocs.io/en/latest/_sources/densmap_demo.rst.txt

Downloads the Galaxy10SDSS dataset if not present, loads images and labels, and creates a subset for training and testing. This involves flattening image data and preparing labels.

```python3
import numpy as np
import h5py
import matplotlib.pyplot as plt
import umap
import os
import math
import requests

if not os.path.isfile("Galaxy10.h5"):
    url = "http://astro.utoronto.ca/~bovy/Galaxy10/Galaxy10.h5"
    r = requests.get(url, allow_redirects=True)
    open("Galaxy10.h5", "wb").write(r.content)

# To get the images and labels from file
with h5py.File("Galaxy10.h5", "r") as F:
    images = np.array(F["images"])
    labels = np.array(F["ans"])

X_train = np.empty([math.floor(len(labels) / 100), 14283], dtype=np.float64)
y_train = np.empty([math.floor(len(labels) / 100)], dtype=np.float64)
X_test = X_train
y_test = y_train
# Get a subset of the data
for i in range(math.floor(len(labels) / 100)):
    X_train[i, :] = np.array(np.ndarray.flatten(images[i, :, :, :]), dtype=np.float64)
    y_train[i] = labels[i]
    X_test[i, :] = np.array(
        np.ndarray.flatten(images[i + math.floor(len(labels) / 100), :, :, :]),
        dtype=np.float64,
    )
    y_test[i] = labels[i + math.floor(len(labels) / 100)]
```

--------------------------------

### Initialize UMAP with 10 Components

Source: https://umap-learn.readthedocs.io/en/latest/_sources/transform.rst.txt

Initializes and fits a UMAP transformer with 10 components, using specific parameters for neighbors and random state. This is done to explore higher-dimensional embeddings.

```python3
trans = umap.UMAP(n_neighbors=5, n_components=10, random_state=42).fit(X_train)
```

--------------------------------

### UMAP Model Configuration

Source: https://umap-learn.readthedocs.io/en/latest/basic_usage.html

Displays the configuration parameters of the fitted UMAP model.

```python
UMAP(a=None, angular_rp_forest=False, b=None,
     force_approximation_algorithm=False, init='spectral', learning_rate=1.0,
     local_connectivity=1.0, low_memory=False, metric='euclidean',
     metric_kwds=None, min_dist=0.1, n_components=2, n_epochs=None,
     n_neighbors=15, negative_sample_rate=5, output_metric='euclidean',
     output_metric_kwds=None, random_state=42, repulsion_strength=1.0,
     set_op_mix_ratio=1.0, spread=1.0, target_metric='categorical',
     target_metric_kwds=None, target_n_neighbors=-1, target_weight=0.5,
     transform_queue_size=4.0, transform_seed=42, unique=False, verbose=False)
```

--------------------------------

### Configure Keras Callbacks for Early Stopping

Source: https://umap-learn.readthedocs.io/en/latest/_sources/parametric_umap.rst.txt

Sets up Keras callbacks for ParametricUMAP, specifically configuring EarlyStopping to monitor 'loss' and stop training after a specified patience.

```python3
keras_fit_kwargs = {"callbacks": [
    tf.keras.callbacks.EarlyStopping(
        monitor='loss',
        min_delta=10**-2,
        patience=10,
        verbose=1,
    )
]}
```

--------------------------------

### Order and visualize data

Source: https://umap-learn.readthedocs.io/en/latest/aligned_umap_basic_usage.html

Sort digits by total ink amount and visualize the last sample.

```python
ordered_digits = digits.data[np.argsort(digits.data.sum(axis=1))]
ordered_target = digits.target[np.argsort(digits.data.sum(axis=1))]
plt.matshow(ordered_digits[-1].reshape((8,8)))
```

--------------------------------

### Fit DensMAP with high dens_lambda

Source: https://umap-learn.readthedocs.io/en/latest/densmap_demo.html

Initializes and fits a UMAP model with densmap enabled and a dens_lambda of 5.0.

```python
%%time
dens_mapper = umap.UMAP(densmap=True, dens_lambda=5.0, random_state=42).fit(fmnist.data)
```
Tokens:
Raw