scDisInFact

https://github.com/zhanglabgt/scdisinfact
Admin
scDisInFact is a deep learning framework designed for the integration and condition effect...
Tokens:8,207
Snippets:29
Trust Score:6
License:GPL-3.0
Update:2 months ago
Show doc for...
Tokens:
Raw
### Training Output Logs

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Example output showing loss metrics and GPU memory usage during the training process.

```text
Epoch 0, Validating Loss: 3.4188
	 loss reconstruction: 2.16931
	 loss kl comm: 2.87213
	 loss kl diff: 4.66035
	 loss mmd common: 3.34361
	 loss mmd diff: 12.10332
	 loss classification: 0.67965
	 loss group lasso diff: 0.52171
GPU memory usage: 25.123047MB
Epoch 10, Validating Loss: 2.0680
	 loss reconstruction: 1.75014
	 loss kl comm: 21.93596
	 loss kl diff: 8.43641
	 loss mmd common: 5.44845
	 loss mmd diff: 2.78692
	 loss classification: 0.06543
	 loss group lasso diff: 0.16700
GPU memory usage: 25.123047MB
Epoch 20, Validating Loss: 1.9584
	 loss reconstruction: 1.71307
	 loss kl comm: 28.37413
	 loss kl diff: 6.75759
	 loss mmd common: 4.50329
	 loss mmd diff: 2.62253
	 loss classification: 0.04067
	 loss group lasso diff: 0.13609
GPU memory usage: 25.123047MB
Epoch 30, Validating Loss: 1.9047
	 loss reconstruction: 1.69450
	 loss kl comm: 34.15609
	 loss kl diff: 5.56363
	 loss mmd common: 3.97935
	 loss mmd diff: 3.38291
	 loss classification: 0.02864
	 loss group lasso diff: 0.12489
GPU memory usage: 25.123047MB
Epoch 40, Validating Loss: 1.8684
	 loss reconstruction: 1.68159
	 loss kl comm: 38.70638
	 loss kl diff: 4.79726
	 loss mmd common: 3.15349
	 loss mmd diff: 2.04868
	 loss classification: 0.01881
	 loss group lasso diff: 0.11911
GPU memory usage: 25.123047MB
Epoch 50, Validating Loss: 1.8498
	 loss reconstruction: 1.67437
	 loss kl comm: 40.23505
	 loss kl diff: 4.25139
	 loss mmd common: 2.84147
	 loss mmd diff: 1.66778
	 loss classification: 0.01589
	 loss group lasso diff: 0.11621
GPU memory usage: 25.123047MB
Epoch 60, Validating Loss: 1.8374
	 loss reconstruction: 1.66826
	 loss kl comm: 41.90871
	 loss kl diff: 3.84837
	 loss mmd common: 2.62729
	 loss mmd diff: 4.32704
	 loss classification: 0.01390
	 loss group lasso diff: 0.11567
GPU memory usage: 25.123047MB
Epoch 70, Validating Loss: 1.8278
	 loss reconstruction: 1.66392
	 loss kl comm: 43.71992
	 loss kl diff: 3.54066
	 loss mmd common: 2.72978
	 loss mmd diff: 2.49639
	 loss classification: 0.01148
	 loss group lasso diff: 0.11607
GPU memory usage: 25.123047MB
Epoch 80, Validating Loss: 1.8161
	 loss reconstruction: 1.65796
	 loss kl comm: 44.34558
	 loss kl diff: 3.15405
	 loss mmd common: 2.62663
	 loss mmd diff: 1.98116
	 loss classification: 0.01060
	 loss group lasso diff: 0.11513
GPU memory usage: 25.123047MB
Epoch 90, Validating Loss: 1.8102
	 loss reconstruction: 1.65645
	 loss kl comm: 46.59809
	 loss kl diff: 2.78180
	 loss mmd common: 2.46463
	 loss mmd diff: 2.24652
	 loss classification: 0.01034
	 loss group lasso diff: 0.11467
GPU memory usage: 25.123047MB
Epoch 100, Validating Loss: 1.8030
	 loss reconstruction: 1.65312
	 loss kl comm: 45.79342
	 loss kl diff: 2.73456
	 loss mmd common: 2.40877
	 loss mmd diff: 2.41908
	 loss classification: 0.00691
	 loss group lasso diff: 0.11466
GPU memory usage: 25.123047MB
```

--------------------------------

### Install scDisInFact Package

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/README.md

Command to install the package from the root directory.

```bash
pip install .
```

--------------------------------

### Load and Prepare Dataset

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Loads count matrices and metadata, then creates a training dataset by excluding specific test indices.

```python
counts = sp.load_npz("data/demo_data/counts.npz")
meta_cells = pd.read_csv("data/demo_data/meta_cells.csv", index_col = 0)
test_idx = ((meta_cells["condition 1"] == "ctrl") & (meta_cells["condition 2"] == "healthy") & (meta_cells["batch"] == 0)).values
train_idx = ~test_idx

data_dict = create_scdisinfact_dataset(counts[train_idx,:], meta_cells.loc[train_idx,:], condition_key = ["condition 1", "condition 2"], batch_key = "batch")

result_dir = "demo_result/"
if not os.path.exists(result_dir):
    os.makedirs(result_dir)
```

--------------------------------

### Initialize scDisInFact Environment

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Imports necessary libraries and sets the computation device to CUDA if available.

```python
import sys, os
import torch
import numpy as np 
import pandas as pd
from scDisInFact import scdisinfact, create_scdisinfact_dataset
from scDisInFact import utils

import matplotlib.pyplot as plt

from umap import UMAP
from sklearn.decomposition import PCA
import scipy.sparse as sp

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
```

--------------------------------

### Initialize and Train scDisInFact Model

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Sets up hyper-parameters, initializes the model, performs training, and saves the resulting state dictionary.

```python
# default setting of hyper-parameters
reg_mmd_comm = 1e-4
reg_mmd_diff = 1e-4
reg_kl_comm = 1e-5
reg_kl_diff = 1e-2
reg_class = 1
reg_gl = 1

Ks = [8, 2, 2]

batch_size = 64
nepochs = 100
interval = 10
lr = 5e-4
lambs = [reg_mmd_comm, reg_mmd_diff, reg_kl_comm, reg_kl_diff, reg_class, reg_gl]
model = scdisinfact(data_dict = data_dict, Ks = Ks, batch_size = batch_size, interval = interval, lr = lr, 
                    reg_mmd_comm = reg_mmd_comm, reg_mmd_diff = reg_mmd_diff, reg_gl = reg_gl, reg_class = reg_class, 
                    reg_kl_comm = reg_kl_comm, reg_kl_diff = reg_kl_diff, seed = 0, device = device)
model.train()
losses = model.train_model(nepochs = nepochs, recon_loss = "NB")
torch.save(model.state_dict(), result_dir + f"model_{Ks}_{lambs}_{batch_size}_{nepochs}_{lr}.pth")
model.load_state_dict(torch.load(result_dir + f"model_{Ks}_{lambs}_{batch_size}_{nepochs}_{lr}.pth", map_location = device))
_ = model.eval()
```

--------------------------------

### Create scDisInFact Dataset

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/README.md

Initializes the dataset object required for training using count matrices and cell metadata.

```python
from scDisInFact import scdisinfact, create_scdisinfact_dataset
data_dict = create_scdisinfact_dataset(counts, meta_cells, condition_key = ["condition 1", "condition 2"], batch_key = "batch")
```

--------------------------------

### Define scDisInFact Dependencies

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/README.md

List of required software versions for running the scDisInFact framework.

```text
    python >= 3.7
    pytorch >= 1.11.0
    sklearn >= 0.22.1
    numpy >= 1.21.6
    pandas >= 1.3.5
    scipy >- 1.7.3
    matplotlib >= 3.5.2
```

--------------------------------

### Train scDisInFact Model

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/README.md

Configures latent dimensions and device settings before training the model on the prepared dataset.

```python
# declare latent dimensions, we have two condition types, so there are three element corresponding to 
# shared-bio factor, unshared-bio factor for condition 1, unshared-bio factor for condition 2
Ks = [8, 2, 2]
# training device
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = scdisinfact(data_dict = data_dict, Ks = Ks, device = device)
losses = model.train_model(nepochs = 100)
_ = model.eval()
```

--------------------------------

### Visualization output logs

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Displays the figure size outputs generated by the plotting functions.

```text
Result:
<Figure size 3000x1500 with 1 Axes>
```

```text
Result:
<Figure size 2100x1500 with 1 Axes>
```

```text
Result:
<Figure size 2100x1500 with 1 Axes>
```

```text
Result:
<Figure size 2100x1500 with 1 Axes>
```

```text
Result:
<Figure size 2100x1500 with 1 Axes>
```

```text
Result:
<Figure size 2100x1500 with 1 Axes>
```

--------------------------------

### Visualize latent factors with UMAP and PCA

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Applies dimensionality reduction to latent factors and saves visualization plots using utility functions.

```python
# UMAP
umap_op = UMAP(min_dist = 0.1, random_state = 0)
pca_op = PCA(n_components = 2)
z_cs_umap = umap_op.fit_transform(np.concatenate(z_cs, axis = 0))
z_ds_umap = []
z_ds_umap.append(pca_op.fit_transform(np.concatenate([z_d[0] for z_d in z_ds], axis = 0)))
z_ds_umap.append(pca_op.fit_transform(np.concatenate([z_d[1] for z_d in z_ds], axis = 0)))

comment = f'results_{Ks}_{lambs}_{batch_size}_{nepochs}_{lr}/'
if not os.path.exists(result_dir + comment):
    os.makedirs(result_dir + comment)

batch_annos = np.concatenate([x["batch"].values.squeeze() for x in data_dict["meta_cells"]])
batch_annos = np.where(batch_annos == 0, "batch 1", "batch 2")
utils.plot_latent(zs = z_cs_umap, annos = np.concatenate([x["annos"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "annos", axis_label = "UMAP", figsize = (10,5), save = result_dir + comment+"common_dims_annos.png")
# utils.plot_latent(zs = z_cs_umap, annos = np.concatenate([x["annos"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "separate", axis_label = "UMAP", figsize = (10,10), save = result_dir + comment+"common_dims_annos_separate.png")

utils.plot_latent(zs = z_cs_umap, annos = np.concatenate([x["annos"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "batches", axis_label = "UMAP", figsize = (7,5), save = result_dir + comment+"common_dims_batches.png")
# utils.plot_latent(zs = z_cs_umap, annos = np.concatenate([x["condition 1"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "annos", axis_label = "UMAP", figsize = (7,5), save = result_dir + comment+"common_dims_cond1.png")

utils.plot_latent(zs = z_ds_umap[0], annos = np.concatenate([x["condition 1"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "annos", axis_label = "PCA", figsize = (7,5), save = result_dir + comment+"diff_dims1_cond1.png")
utils.plot_latent(zs = z_ds_umap[0], annos = np.concatenate([x["annos"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "batches", axis_label = "PCA", figsize = (7,5), save = result_dir + comment+"diff_dims1_batches.png")

utils.plot_latent(zs = z_ds_umap[1], annos = np.concatenate([x["condition 2"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "annos", axis_label = "PCA", figsize = (7,5), save = result_dir + comment+"diff_dims2_cond1.png")
utils.plot_latent(zs = z_ds_umap[1], annos = np.concatenate([x["annos"].values.squeeze() for x in data_dict["meta_cells"]]), batches = batch_annos, mode = "batches", axis_label = "PCA", figsize = (7,5), save = result_dir + comment+"diff_dims2_batches.png")
```

--------------------------------

### Visualize Predicted and Denoised Counts

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Normalizes, transforms, and visualizes the predicted and denoised counts using UMAP and utility plotting functions.

```python
counts_predict_norm = counts_predict/(np.sum(counts_predict, axis = 1, keepdims = True) + 1e-6) * 100
counts_predict_norm = np.log1p(counts_predict_norm)
counts_test_norm = counts_test_denoised/(np.sum(counts_test_denoised, axis = 1, keepdims = True) + 1e-6) * 100
counts_test_norm = np.log1p(counts_test_norm)
# plot umap
umap_op = UMAP(n_components = 2, n_neighbors = 100, min_dist = 0.4, random_state = 0) 
x_umap_scdisinfact = umap_op.fit_transform(np.concatenate([counts_predict_norm, counts_test_norm], axis = 0))

utils.plot_latent(x_umap_scdisinfact, annos = np.concatenate([meta_input["annos"].values, meta_test["annos"].values], axis = 0), batches = np.array(["predict"] * counts_predict.shape[0] + ["gold-standard"] * counts_test.shape[0]), \
    mode = "annos", save = result_dir + comment + "predict_batches.png", figsize = (12,7))
utils.plot_latent(x_umap_scdisinfact, annos = np.concatenate([meta_input["annos"].values, meta_test["annos"].values], axis = 0), batches = np.array(["predict"] * counts_predict.shape[0] + ["gold-standard"] * counts_test.shape[0]), \
    mode = "batches", save = result_dir + comment + "predict_batches.png", figsize = (12,7))
```

--------------------------------

### Detect Key Genes

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/README.md

Extracts gene scores from the model to identify key genes associated with specific conditions.

```python
gene_scores = model.extract_gene_scores()
# scores for condition type 1
genes_scores[0]
# scores for condition type 2
genes_scores[1]
```

--------------------------------

### Predict Perturbations

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/README.md

Predicts count matrices for specific conditions and batches based on input data.

```python
input_idx = ((meta_cells["condition 1"] == "stim") & (meta_cells["condition 2"] == "severe") & (meta_cells["batch"] == 1)).values
counts_input = counts[input_idx,:].toarray()
meta_input = meta_cells.loc[input_idx,:]

counts_predict = model.predict_counts(input_counts = counts_input, meta_cells = meta_input, condition_keys = ["condition 1", "condition 2"], 
                                      batch_key = "batch", predict_conds = ["ctrl", "healthy"], predict_batch = 0)
```

--------------------------------

### Denoise Count Matrix

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Denoises a count matrix by setting predict_conds and predict_batch to None.

```python
test_idx = ((meta_cells["condition 1"] == "ctrl") & (meta_cells["condition 2"] == "healthy") & (meta_cells["batch"] == 0)).values
counts_test = counts[test_idx,:].toarray()
meta_test = meta_cells.loc[test_idx,:]

counts_test_denoised = model.predict_counts(input_counts = counts_test, meta_cells = meta_test, condition_keys = ["condition 1", "condition 2"], 
                                            batch_key = "batch", predict_conds = None, predict_batch = None)
```

--------------------------------

### Perform forward pass for latent factor extraction

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Iterates through datasets to infer latent factors using the encoder and reconstruct gene expression using the decoder.

```python
# one forward pass
z_cs = []
z_ds = []
zs = []

for dataset in data_dict["datasets"]:
    with torch.no_grad():
        # pass through the encoders
        dict_inf = model.inference(counts = dataset.counts_norm.to(model.device), batch_ids = dataset.batch_id[:,None].to(model.device), print_stat = True)
        # pass through the decoder
        dict_gen = model.generative(z_c = dict_inf["mu_c"], z_d = dict_inf["mu_d"], batch_ids = dataset.batch_id[:,None].to(model.device))
        z_c = dict_inf["mu_c"]
        z_d = dict_inf["mu_d"]
        z = torch.cat([z_c] + z_d, dim = 1)
        mu = dict_gen["mu"]    
        z_ds.append([x.cpu().detach().numpy() for x in z_d])
        z_cs.append(z_c.cpu().detach().numpy())
        zs.append(np.concatenate([z_cs[-1]] + z_ds[-1], axis = 1))
```

--------------------------------

### Visualize Gene Expression Data

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Performs PCA and UMAP dimensionality reduction to visualize batch and condition effects in the latent space.

```python
# Visualize the raw gene expression data
umap_op = UMAP(n_components = 2, n_neighbors = 100, min_dist = 0.4, random_state = 0) 

x_pca = PCA(n_components = 80).fit_transform(np.concatenate([x.counts_norm for x in data_dict["datasets"]], axis = 0))
x_umap = umap_op.fit_transform(x_pca)

utils.plot_latent(x_umap, annos = np.concatenate([x["batch"].values.squeeze() for x in data_dict["meta_cells"]]), mode = "annos", save = result_dir + "batches.png", figsize = (12,7), axis_label = "UMAP", markerscale = 6, s = 2)

utils.plot_latent(x_umap, annos = np.concatenate([x["condition 1"].values.squeeze() for x in data_dict["meta_cells"]]), mode = "annos", save = result_dir + "condition_type_1.png", figsize = (11,7), axis_label = "UMAP", markerscale = 6, s = 2)

utils.plot_latent(x_umap, annos = np.concatenate([x["condition 2"].values.squeeze() for x in data_dict["meta_cells"]]), mode = "annos", save = result_dir + "condition_type_2.png", figsize = (12,7), axis_label = "UMAP", markerscale = 6, s = 2)

utils.plot_latent(x_umap, annos = np.concatenate([x["annos"].values.squeeze() for x in data_dict["meta_cells"]]), mode = "annos", save = result_dir + "celltypes.png", figsize = (11,7), axis_label = "UMAP", markerscale = 6, s = 2)
```

--------------------------------

### Extract Biological Factors

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/README.md

Performs inference and generative passes to extract shared and unshared biological factors from the trained model.

```python
# one forward pass
z_cs = []
z_ds = []
zs = []

# loop through all training count matrices
for dataset in data_dict["datasets"]:
    with torch.no_grad():
        # pass through the encoders
        dict_inf = model.inference(counts = dataset.counts_norm.to(model.device), batch_ids = dataset.batch_id[:,None].to(model.device), print_stat = True)
        # pass through the decoder
        dict_gen = model.generative(z_c = dict_inf["mu_c"], z_d = dict_inf["mu_d"], batch_ids = dataset.batch_id[:,None].to(model.device))
        z_c = dict_inf["mu_c"]
        z_d = dict_inf["mu_d"]
        mu = dict_gen["mu"]    
        z_ds.append([x.cpu().detach().numpy() for x in z_d])
        z_cs.append(z_c.cpu().detach().numpy())

# shared-bio factor, concatenate across all training matrices
z_cs = np.concatenate(z_cs, axis = 0)
# unshared-bio factors for conditions 1 and 2
z_ds_cond1 = np.concatenate([x[0] for x in z_ds], axis = 0)
z_ds_cond2 = np.concatenate([x[1] for x in z_ds], axis = 0)
```

--------------------------------

### Extracting and printing gene scores

Source: https://github.com/zhanglabgt/scdisinfact/blob/main/demo.ipynb

Use extract_gene_scores to retrieve gene scores for different conditions. The returned list contains numpy arrays for each condition type.

```python
gene_scores = model.extract_gene_scores()
# the scores of 500 genes for condition type 1
print(gene_scores[0])
# the scores of 500 genes for condition type 2
print(gene_scores[1])
```

```text
Output:
[1.63669419e-02 9.14421678e-03 1.03906989e-02 2.44847834e-02
 1.10281147e-02 5.04428595e-02 1.99805293e-02 2.87926253e-02
 2.26134062e-02 1.68385115e-02 1.64539106e-02 1.54288597e-02
 1.02815181e-02 1.06106382e-02 1.03216600e-02 2.21145283e-02
 1.20886862e-02 2.03598756e-02 9.06910188e-03 1.94463022e-02
 1.00129182e-02 2.09469944e-02 1.98464487e-02 1.67217739e-02
 9.54557024e-03 1.64563600e-02 1.14200022e-02 1.05871251e-02
 1.11636473e-02 1.32796932e-02 1.70270558e-02 2.98575647e-02
 1.59699973e-02 1.27448607e-02 9.04416014e-03 9.79706272e-03
 9.50883888e-03 3.32097337e-02 1.66252516e-02 1.65118147e-02
 1.34429010e-02 1.88532658e-02 1.07760923e-02 1.86592601e-02
 1.01714404e-02 2.33502220e-02 2.26591676e-02 1.37888743e-02
 1.82681791e-02 1.05414847e-02 1.22234732e-01 1.70774072e-01
 1.29416421e-01 6.95167482e-02 6.00959137e-02 1.07012443e-01
 6.50875494e-02 8.51563737e-02 5.50793707e-02 2.30056405e-01
 9.14855376e-02 5.90917945e-01 1.05939798e-01 5.47770932e-02
 2.53401399e-01 4.81052250e-02 7.53181577e-02 3.02407388e-02
 9.16214958e-02 5.76570146e-02 4.87877876e-02 6.71599805e-02
 4.47623990e-02 1.54475048e-01 8.62992331e-02 4.42959294e-02
 6.39074370e-02 4.25439626e-02 3.10002882e-02 5.34197390e-02
 7.85524696e-02 4.55515645e-02 3.67127359e-02 1.05608895e-01
 5.86714819e-02 2.99524460e-02 2.24323589e-02 3.80827151e-02
 5.40334508e-02 1.06214747e-01 3.82759199e-02 7.74457902e-02
 6.26478344e-02 2.56000217e-02 4.72216681e-02 1.91764101e-01
 7.47343153e-02 7.80757740e-02 2.14260966e-01 7.67388418e-02
 1.66052599e-02 3.04394718e-02 2.31107157e-02 2.48811767e-02
 2.01904867e-02 4.03477661e-02 7.30424151e-02 1.30741186e-02
 2.59846300e-02 1.95017606e-02 3.47877517e-02 1.69440452e-02
 1.54670412e-02 3.54699604e-02 2.52164882e-02 2.54828986e-02
 1.31866867e-02 2.28587221e-02 2.43725609e-02 2.09941808e-02
 2.16768105e-02 1.91361271e-02 1.99311189e-02 2.74303146e-02
 2.36126501e-02 1.80452541e-02 1.77990925e-02 1.82491411e-02
 2.18166281e-02 2.21009925e-02 1.53946774e-02 2.34858673e-02
 2.28862464e-02 1.71462633e-02 2.49691587e-02 1.79383140e-02
 1.50980651e-02 2.84816269e-02 2.36112643e-02 1.16192792e-02
 1.92217603e-02 2.58686114e-02 2.28389259e-02 2.92904917e-02
 3.85771468e-02 1.38778444e-02 2.07012538e-02 1.75946802e-02
 1.59025472e-02 3.47818956e-02 2.32514776e-02 2.34509464e-02
 3.08827199e-02 1.29379472e-02 3.94443013e-02 1.49799483e-02
 3.14951092e-02 1.53286634e-02 1.26605676e-02 1.69340745e-02
 2.24852748e-02 5.44036515e-02 1.99636631e-02 2.89688520e-02
 2.27485765e-02 2.24157348e-02 3.07078753e-02 2.56907064e-02
 3.71009111e-02 1.66844074e-02 1.53273735e-02 1.34411566e-02
 2.65170746e-02 2.60779895e-02 2.43748426e-02 1.84464045e-02
 1.49574978e-02 1.34118423e-02 1.72780678e-02 2.16983277e-02
 1.68407150e-02 2.29700729e-02 1.54608293e-02 1.75490417e-02
 1.85588002e-02 2.63119936e-02 2.67876387e-02 1.73835121e-02
 2.72191335e-02 2.28775796e-02 2.17070375e-02 2.61957143e-02
 1.93082821e-02 3.25720645e-02 1.57930888e-02 2.46775709e-02
 2.36880630e-02 1.32669276e-02 2.06109900e-02 3.81024964e-02
 1.83727778e-02 3.15412432e-02 1.60610937e-02 1.29583757e-02
 1.52418753e-02 1.61016695e-02 1.79219842e-02 1.91934239e-02
 1.43037857e-02 1.92455184e-02 9.56193544e-03 2.44559068e-02
 4.77331094e-02 2.96942275e-02 2.89648622e-02 2.34980360e-02
 3.24502848e-02 2.67059579e-02 1.06456606e-02 2.87704989e-02
 1.73831135e-02 2.17570085e-02 5.20806611e-02 2.54393835e-02
 2.50380505e-02 2.36546136e-02 3.64073925e-02 2.20737234e-02
 2.71441191e-02 2.86978893e-02 5.28513975e-02 2.01960336e-02
 1.89811140e-02 3.04570496e-02 2.00885888e-02 3.54042202e-02
 1.82312876e-02 1.66009981e-02 2.81130653e-02 2.31628977e-02
 1.60533693e-02 2.70113219e-02 2.17466410e-02 2.44031120e-02
 1.50908232e-02 1.61862075e-02 1.98145509e-02 1.83125772e-02
 2.81026773e-02 1.60633195e-02 3.44780013e-02 2.69414745e-02
 1.47514185e-02 2.27426719e-02 2.71328893e-02 1.69861279e-02
 2.30197944e-02 1.33325607e-02 2.12583262e-02 1.69065055e-02
 1.95624530e-02 2.34063491e-02 1.83358379e-02 1.57798417e-02
 1.62240826e-02 1.35129355e-02 2.27162261e-02 1.64201800e-02
 1.72827225e-02 1.68201160e-02 2.98609678e-02 2.24658120e-02
 2.03895997e-02 2.04046145e-02 1.37415826e-02 2.35225148e-02
 2.55785659e-02 2.84883063e-02 1.52104227e-02 1.86115317e-02
 2.54867282e-02 1.52508924e-02 1.97465699e-02 2.47462615e-02
 3.12469397e-02 2.89558843e-02 1.63457710e-02 2.41394602e-02
 2.15466581e-02 1.86309759e-02 2.01000161e-02 1.54214203e-02]
```
Tokens:
Raw