### Print setup information Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Outputs the versions of PyTorch and CUDA, and the selected device (CPU or GPU) to confirm the setup. ```python print("Seeded. Torch:", torch.__version__, "CUDA:", torch.version.cuda, "Using device:", device) ``` -------------------------------- ### Create Dataset and DataLoader Source: https://context7.com/prov-gigatime/gigatime/llms.txt Demonstrates how to instantiate the HECOMETDataset_roi and create a PyTorch DataLoader for it. Includes setup for Albumentations transforms and specifies batch size, shuffling, and number of workers. ```python from torch.utils.data import DataLoader import albumentations as A from albumentations.augmentations import transforms val_transform = A.Compose([ A.Resize(512, 512), transforms.Normalize() ], is_check_shapes=False) dataset = HECOMETDataset_roi( all_tile_pair=metadata, tile_pair_df=metadata, transform=val_transform, dir_path="./data/sample_test_data/data/", window_size=256, split="test", mask_noncell=True, cell_mask_label=True ) dataloader = DataLoader(dataset, batch_size=32, shuffle=False, num_workers=4) ``` -------------------------------- ### Evaluation Example Source: https://context7.com/prov-gigatime/gigatime/llms.txt Example of evaluating a model using get_box_metrics. Sets the model to evaluation mode and iterates through a dataloader. ```python # Evaluation example model.eval() with torch.no_grad(): for images, masks, info in dataloader: logits = model(images.cuda()) preds = (torch.sigmoid(logits) > 0.5).float() mse, pearson, spearman = get_box_metrics(preds, masks.cuda(), box_size=8) print(f"Mean Pearson correlation: {np.nanmean(pearson):.4f}") for i, channel in enumerate(channel_names): if channel not in ['TRITC', 'Cy5']: # Skip background channels print(f" {channel}: {pearson[i]:.4f}") break ``` -------------------------------- ### Create Conda Environment Source: https://github.com/prov-gigatime/gigatime/blob/main/README.md Creates a Conda environment named 'gigatime' using the provided 'environment.yml' file. Ensure your PyTorch version matches your GPU and CUDA setup. ```bash conda env create -f environment.yml ``` -------------------------------- ### Setup Conda Environment for GigaTIME Source: https://context7.com/prov-gigatime/gigatime/llms.txt Create and activate a conda environment with Python 3.11 and PyTorch 2.0+. Ensure your HuggingFace token is set as an environment variable for model access. ```bash conda env create -f environment.yml conda activate gigatime export HF_TOKEN= ``` -------------------------------- ### Visualize Sample Data with GigaTIME Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb This function visualizes a specific sample from the batch, applying channel-specific colors and titles. Ensure 'matplotlib' and 'numpy' are installed. ```python colored_mask = colorize_channel(mask[idx, ch].cpu().numpy(), channel_colors[channel_name]) axes[plot_idx + 1].imshow(colored_mask) axes[plot_idx + 1].set_title(channel_name, color=channel_colors[channel_name]) axes[plot_idx + 1].axis('off') for ax in axes[len(filtered_indices) + 1:]: ax.axis('off') plt.tight_layout() plt.show() # Usage: change idx to visualize different samples in the batch plot_sample(idx=10) ``` -------------------------------- ### Get Box Metrics Source: https://context7.com/prov-gigatime/gigatime/llms.txt Splits images into boxes and computes cell-count based metrics including MSE and correlations. Requires input matrices and a box size. ```python def get_box_metrics(pred, mask, box_size=8): """Split images into boxes and compute cell-count based metrics.""" b, c, h, w = pred.shape # Split into non-overlapping boxes pred_boxes = pred.unfold(2, box_size, box_size).unfold(3, box_size, box_size) mask_boxes = mask.unfold(2, box_size, box_size).unfold(3, box_size, box_size) # Count cells in each box pred_counts = pred_boxes.sum(dim=(4, 5)) mask_counts = mask_boxes.sum(dim=(4, 5)) # Calculate MSE mse = ((pred_counts.float() - mask_counts.float()) ** 2).mean() # Calculate correlations pearson, spearman = calculate_correlations(pred_counts, mask_counts) return mse, pearson, spearman ``` -------------------------------- ### Usage of AverageMeter in Training Loop Source: https://context7.com/prov-gigatime/gigatime/llms.txt Example of using AverageMeter to track loss and Pearson correlation during a training loop. Requires 'model', 'criterion', 'train_loader', and 'get_box_metrics' to be defined. Assumes CUDA availability for model and target. ```python # Usage in training loop loss_meter = AverageMeter() pearson_meter = AverageMeter() for batch_idx, (input, target, _) in enumerate(train_loader): output = model(input.cuda()) loss = criterion(output, target.cuda()) loss_meter.update(loss.item(), input.size(0)) _, pearson, _ = get_box_metrics(output, target.cuda(), box_size=8) pearson_meter.update(np.nanmean(pearson), input.size(0)) print(f"Batch {batch_idx}: Loss={loss_meter.avg:.4f} (std={loss_meter.std:.4f}), " f"Pearson={pearson_meter.avg:.4f} (std={pearson_meter.std:.4f})") ``` -------------------------------- ### Test Model and Export Results Source: https://context7.com/prov-gigatime/gigatime/llms.txt This Python function runs model evaluation on a test loader and exports per-channel Pearson correlation results to a CSV file. It requires a trained model, test data loader, channel names, and an output directory. Ensure PyTorch and pandas are installed. ```python # Testing with results export from datetime import datetime import pandas as pd def test_and_export(model, test_loader, channel_names, output_dir): """Run testing and export per-channel Pearson correlation results.""" pearson_meters = [AverageMeter() for _ in range(23)] model.eval() with torch.no_grad(): for input, target, _ in test_loader: # Preprocess target (same as training) target_down = F.interpolate(target, scale_factor=1/8, mode='bilinear') target = F.interpolate(target_down, size=(512, 512), mode='bilinear').cuda() output = model(input.cuda()) preds = (torch.sigmoid(output) > 0.5).float() _, pearson, _ = get_box_metrics(preds, target, box_size=8) for idx, p in enumerate(pearson): if not np.isnan(p): pearson_meters[idx].update(p, target.size(0)) # Export results to CSV results = { 'Channel': channel_names, 'Pearson_Avg': [m.avg for m in pearson_meters], 'Pearson_Std': [m.std for m in pearson_meters] } df = pd.DataFrame(results) df = df.sort_values('Pearson_Avg', ascending=False) date_str = datetime.now().strftime("%Y-%m-%d") df.to_csv(f'{output_dir}/results_{date_str}.csv', index=False) return df results_df = test_and_export(model, test_loader, channel_names, "./output/") print(results_df) ``` -------------------------------- ### Download and Extract Sample Data Source: https://context7.com/prov-gigatime/gigatime/llms.txt Download the sample test dataset containing paired H&E and mIF patches for evaluation, then extract it to the data directory. ```bash wget -O sample_test_data.zip "https://www.dropbox.com/scl/fi/8ampg43fs2yowt9y6vvr1/sample_test_data.zip?rlkey=bkg4w183qnvkh2dudqy3d8lsg&st=j2l463ug&dl=1" unzip sample_test_data.zip -d ./data/ ``` -------------------------------- ### Get HECOMET Dataset Length Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Returns the total number of tile pairs in the dataset. ```python def __len__(self): return len(self.tile_pair_df) ``` -------------------------------- ### Unzip Sample Data Source: https://github.com/prov-gigatime/gigatime/blob/main/README.md Unzips the downloaded sample data into the './data/' directory. Ensure the extracted folders are correctly placed. ```bash unzip sample_test_data.zip -d ./data/ ``` -------------------------------- ### Training Configuration and Loop Source: https://context7.com/prov-gigatime/gigatime/llms.txt Sets up data augmentation, optimizer, learning rate scheduler, and the training loop. Includes validation and checkpoint saving based on best validation Pearson correlation. ```python # Training configuration example import torch.optim as optim from torch.optim.lr_scheduler import CosineAnnealingLR import albumentations as A from albumentations.augmentations import transforms from albumentations.core.composition import Compose, OneOf # Data augmentation for training train_transform = Compose([ A.RandomRotate90(), A.Flip(), OneOf([ transforms.HueSaturationValue(), transforms.RandomBrightnessContrast(brightness_limit=0, contrast_limit=0.2), transforms.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0), ], p=1), A.RandomCrop(512, 512), transforms.Normalize() ], is_check_shapes=False) # Optimizer and scheduler optimizer = optim.Adam(model.parameters(), lr=1e-3, weight_decay=1e-4) scheduler = CosineAnnealingLR(optimizer, T_max=300, eta_min=1e-5) criterion = BCEDiceLoss().cuda() # Training loop best_pearson = 0 for epoch in range(300): model.train() for input, target, _ in train_loader: # Downsample target to handle registration errors target_down = F.interpolate(target, scale_factor=1/8, mode='bilinear') target = F.interpolate(target_down, size=(512, 512), mode='bilinear').cuda() output = model(input.cuda()) loss = criterion(output, target) optimizer.zero_grad() loss.backward() optimizer.step() scheduler.step() # Validation and checkpointing model.eval() val_pearson = validate(model, val_loader) if val_pearson > best_pearson: torch.save(model.state_dict(), f'models/{name}/model.pth') best_pearson = val_pearson ``` -------------------------------- ### Gigatime Model Configuration and Initialization Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb This code configures and initializes the Gigatime model, including defining channel names, selecting a loss function, setting up the model architecture, and configuring the optimizer and learning rate scheduler based on provided configuration parameters. ```python # channel names common_channel_list=['DAPI', 'TRITC', #background channel not used in analysis 'Cy5', #background channel not used in analysis 'PD-1', 'CD14', 'CD4', 'T-bet', 'CD34', 'CD68', 'CD16', 'CD11c', 'CD138', 'CD20', 'CD3', 'CD8', 'PD-L1', 'CK', 'Ki67', 'Tryptase', 'Actin-D', 'Caspase3-D', 'PHH3-B', 'Transgelin'] # loss if config['loss'] == 'MSELoss': criterion = nn.MSELoss().cuda() elif config['loss'] == 'BCEWithLogitsLoss': criterion = nn.BCEWithLogitsLoss().cuda() elif config['loss'] == 'BCEDiceLoss': criterion = BCEDiceLoss().cuda() else: criterion = losses.__dict__[config['loss']]().cuda() # model model = gigatime(num_classes=config['num_classes'], sigmoid=config["sigmoid"], loss_type=config["loss"], input_channels=config['input_channels']).cuda() if config["gpu_ids"] and len(config["gpu_ids"]) > 1: model = nn.DataParallel(model, device_ids=config["gpu_ids"]) print("using multiple GPUs", config["gpu_ids"]) # optimizer params = filter(lambda p: p.requires_grad, model.parameters()) if config['optimizer'] == 'Adam': optimizer = optim.Adam(params, lr=config['lr'], weight_decay=config['weight_decay']) elif config['optimizer'] == 'SGD': optimizer = optim.SGD(params, lr=config['lr'], momentum=config['momentum'], nesterov=config['nesterov'], weight_decay=config['weight_decay']) # scheduler if config['scheduler'] == 'CosineAnnealingLR': scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=config['epochs'], eta_min=config['min_lr']) elif config['scheduler'] == 'ReduceLROnPlateau': scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, factor=config['factor'], patience=config['patience'], verbose=1, min_lr=config['min_lr']) elif config['scheduler'] == 'MultiStepLR': scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[int(e) for e in config['milestones'].split(',')], gamma=config['gamma']) elif config['scheduler'] == 'ConstantLR': scheduler = None ``` -------------------------------- ### Initialize and Print GigaTIME Model Parameters Source: https://context7.com/prov-gigatime/gigatime/llms.txt Initializes the GigaTIME model with specified number of classes and input channels, then prints the total number of trainable parameters. This is useful for understanding model complexity. ```python # Initialize model for 23 mIF channels from 3-channel H&E input model = gigatime(num_classes=23, input_channels=3) print(f"Model parameters: {sum(p.numel() for p in model.parameters() if p.requires_grad):,}") ``` -------------------------------- ### Train GigaTIME Model Source: https://github.com/prov-gigatime/gigatime/blob/main/README.md Initiates the training process for the GigaTIME model. This command requires specifying various parameters such as architecture, data paths, batch size, and output directories. ```bash python scripts/db_train.py --arch gigatime --tiling_dir "gigatime_training_path" --window_size 256 --batch_size 32 --sampling_prob 1 --name GigaTIME_model --output_dir "Output_Directory" --epoch 300 --input_h 512 --input_w 512 --lr 0.001 --loss BCEDiceLoss --val_sampling_prob 1 --num_workers 12 --gpu_ids 0 1 2 3 4 5 6 7 --crop True --metadata "Gigatime metadata file" ``` -------------------------------- ### Get Box Metrics Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Calculates Mean Squared Error (MSE) and correlation metrics (Pearson, Spearman) for predicted and masked boxes. This function aggregates performance across localized regions of the data. ```python def get_box_metrics(pred, mask, box_size): # Split the images into boxes pred_boxes = split_into_boxes(pred, box_size) mask_boxes = split_into_boxes(mask, box_size) # Count the number of ones in each box pred_counts = count_ones(pred_boxes) mask_counts = count_ones(mask_boxes) # Calculate precision and MSE for the matrices mse = ((pred_counts.float() - mask_counts.float()) ** 2).mean(dim=0) mean_mse_per_channel = mse.mean(dim=(1,2)) mean_mse = mse.mean().item() pearson, spearman = calculate_correlations(pred_counts, mask_counts) return mean_mse_per_channel, pearson, spearman ``` -------------------------------- ### Initialize HECOMETDataset_roi and DataLoader Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Sets up the HECOMETDataset_roi for testing and creates a PyTorch DataLoader for batch processing. Configures dataset parameters like window size and split. ```python test_dataset = HECOMETDataset_roi( all_tile_pair=metadata, tile_pair_df=metadata, transform=val_transform, dir_path = data_root, window_size = 256, split="test", mask_noncell=True, cell_mask_label=True, ) test_loader = torch.utils.data.DataLoader( test_dataset, batch_size=config['batch_size'], shuffle=False, num_workers=0, drop_last=False) ``` -------------------------------- ### Get HECOMET Dataset Item Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Retrieves and preprocesses a single data item (image patch, comet mask, cell masks) from the dataset for a given index. Includes optional masking and labeling steps. ```python def __getitem__(self, idx): # Get the tile pair data for the current index pair = self.tile_pair_df.iloc[idx] # Extract the input patch for the current tile image_input_patch = get_image_roi(pair["pair_name"].split("_")[0], pair["pair_name"].split("_")[1], self.all_tiles, self.dir_path, pair) # Load and unpack binary comet data from the pickle file pkl_data = unpack_and_load(os.path.join(pair["dir_name"], pair["pair_name"] + "_comet_binary_thres_labels.pkl.gz")) # Extract the binary comet array mask from the loaded data mask = pkl_data["comet_array_binary"] # Create cell masks for each channel based on whether it's nuclear or not cell_masks = [pkl_data["labels_dapi"] if channel in ["DAPI", "TRITC", "Cy5", "Ki67_1:150 - TRITC"] else pkl_data["labels_dapi_expanded"] for channel in common_channel_list] # Stack the cell masks along the last axis to create a 3D array (H, W, C) cell_masks = np.stack(cell_masks, axis=-1) ### Note that the following processing was done after careful consideration with clinical domain experts # Extract nuclear and expanded nuclear labels for further processing labels_dapi = pkl_data['labels_dapi'] # Nuclear segmentation labels labels_dapi_expanded = pkl_data['labels_dapi_expanded'] # Expanded nuclear labels for rest # Mask out non-cell regions if specified if self.mask_noncell: mask[cell_masks==0]=0 # Shape: (H, W, C) # Apply cell-level mask labeling if specified if self.cell_mask_label: # Label connected components in nuclear segmentation labeled_nuclei = label(labels_dapi) # Shape: (H, W) labeled_nuclei_props = regionprops(labeled_nuclei) # Label connected components in expanded nuclear segmentation labeled_cell = label(labels_dapi_expanded) # Shape: (H, W) labeled_cell_props = regionprops(labeled_cell) # Initialize new mask with same shape as original mask_new=np.zeros_like(mask) # Shape: (H, W, C) # Process both nuclear and cellular regions for label_prop_mode, label_props in (("nuclei", labeled_nuclei_props), ("cell", labeled_cell_props)): # Iterate through each labeled region for region_idx, region in enumerate(label_props): # Get convex hull mask for the region region_mask = region.convex_image # Shape: (region_height, region_width) # Get bounding box coordinates (minr, minc, maxr, maxc) region_bbox= region.bbox region_area_bbox=region.area_bbox # Extract mask values within the bounding box mask_bbox = mask[region_bbox[0]:region_bbox[2], region_bbox[1]:region_bbox[3], :] # Shape: (region_height, region_width, C) # Calculate ratio of positive pixels for each channel within the region region_ratio_list=(mask_bbox[region_mask].sum(axis=0)/region_mask.sum()) # Shape: (C,) # Select channels based on region type and threshold criteria channel_idx_select=[] for channel_idx, (channel, region_ratio) in enumerate(zip(common_channel_list, region_ratio_list)): valid=False # Determine if channel is valid for current region type ``` -------------------------------- ### Train GigaTIME Model Script Source: https://context7.com/prov-gigatime/gigatime/llms.txt Command-line arguments for training the GigaTIME model. Includes parameters for architecture, data directories, training configuration, and hardware. ```bash python scripts/db_train.py \ --arch gigatime \ --tiling_dir "./data/gigatime_training_tiles/" \ --metadata "./data/full_metadata.csv" \ --window_size 256 \ --batch_size 32 \ --sampling_prob 1 \ --name GigaTIME_model \ --output_dir "./output/" \ --epochs 300 \ --input_h 512 \ --input_w 512 \ --lr 0.001 \ --loss BCEDiceLoss \ --val_sampling_prob 1 \ --num_workers 12 \ --gpu_ids 0 1 2 3 4 5 6 7 \ --crop True ``` -------------------------------- ### Load GigaTIME Model from Hugging Face Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Initializes the GigaTIME model architecture and loads weights from a specified Hugging Face repository. Requires 'torch' and 'huggingface_hub'. Ensure the 'device' is correctly set. ```python # Initialize the model architecture using the specified config parameters model = archs.__dict__[config['arch']](config['num_classes'],config['input_channels']).to(device) # Load it using the huggingface model card from huggingface_hub import snapshot_download repo_id = "prov-gigatime/GigaTIME" # Download the repo snapshot local_dir = snapshot_download(repo_id=repo_id) weights_path = os.path.join(local_dir, "model.pth") state_dict = torch.load(weights_path, map_location=device) model.load_state_dict(state_dict) print("=> loaded model %s" % config['arch']) # Create output directory for saving model artifacts os.makedirs(config['output_dir'] + "models/" +config['name'] +"/", exist_ok=True) ``` -------------------------------- ### Sample Data Loader Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Creates a subset of a PyTorch DataLoader for testing purposes. It allows sampling a fraction of the dataset with options for deterministic or random selection. ```python def sample_data_loader(data_loader, config, sample_fraction=0.1, deterministic=False, what_split="train"): # this just samples some fraction of the data in the dataloader so that we can train on a smaller subset for quick testing dataset = data_loader.dataset total_size = len(dataset) sample_size = int(total_size * sample_fraction) if deterministic: sample_indices = [i for i in range(sample_size)] else: sample_indices = random.sample(range(total_size), sample_size) subset = Subset(dataset, sample_indices) if what_split == "train": sample_loader = DataLoader(subset, batch_size=data_loader.batch_size, shuffle=True, num_workers=config['num_workers'], prefetch_factor=6, drop_last=True) else: sample_loader = DataLoader(subset, batch_size=data_loader.batch_size, shuffle=False, num_workers=config['num_workers'], prefetch_factor=6, drop_last=False) return sample_loader ``` -------------------------------- ### Parse Command-Line Arguments for Gigatime Training Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Sets up argument parsing for Gigatime training, defining parameters for model architecture, data paths, training epochs, and batch size. Uses `argparse` and `edict` for configuration. ```python import argparse def str2bool(v): return v.lower() in ('true', '1') def parse_args(): parser = argparse.ArgumentParser() parser.add_argument('--name', default="gigatime_training") parser.add_argument('--output_dir', default="./scratch/") ## Place to save output files parser.add_argument('--gpu_ids', nargs='+', type=int) ## List of GPU ids to use, e.g. --gpu_ids 0 1 2 parser.add_argument('--metadata', default="./../data/sample_metadata.csv") ## Change these if you download and place the data in another location parser.add_argument('--tiling_dir', default="./../data/sample_test_data/data/") ## Change these if you download and place the data in another location parser.add_argument('--epochs', default=1, type=int) ## Number of training epochs parser.add_argument('--batch_size', default=32, type=int) ## Batch size for training # model parser.add_argument('--arch', default='gigatime') ## Model architecture parser.add_argument('--input_channels', default=3, type=int) ## Number of input channels parser.add_argument('--num_classes', default=23, type=int) ## Number of output classes parser.add_argument('--input_w', default=512, type=int) ## Input width to resize to parser.add_argument('--input_h', default=512, type=int) ## Input height to resize to # loss parser.add_argument('--loss', default='BCEDiceLoss') ## Loss function return edict(vars(parser.parse_args([]))) # use [] so notebook runs config = parse_args() config ``` -------------------------------- ### Create Training Dataset with Augmentation Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Creates the training dataset using HECOMETDataset_roi with specified tile pairs, training transformations, directory path, window size, and masking options. ```python train_dataset = HECOMETDataset_roi( all_tile_pair=tile_pair_df, # Complete tile pair dataframe tile_pair_df=tile_pair_df_filtered_dicefilter, # Filtered tile pairs based on quality metrics transform=train_transform, # Apply training augmentations dir_path = config["tiling_dir"], # Path to tiled image directory window_size = config["window_size"], # Size of image windows to extract split="train", # Specify training split mask_noncell=True, # Mask non-cellular regions cell_mask_label=True, # Use cell mask labels ) ``` -------------------------------- ### Import necessary libraries for Gigatime project Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Imports essential libraries for data handling (pandas, numpy), deep learning (torch, torchvision), image augmentation (albumentations), optimization (torch.optim), and configuration (yaml). ```python import pandas as pd import torch import torch.backends.cudnn as cudnn import torch.nn as nn import torch.optim as optim import yaml from albumentations.augmentations import transforms from albumentations.core.composition import Compose, OneOf from sklearn.model_selection import train_test_split from torch.optim import lr_scheduler from tqdm import tqdm import torchvision from torchvision.utils import save_image from prov_data import * from collections import OrderedDict from datetime import datetime import numpy as np import torch from scipy import stats import random import archs import losses from metrics import iou_score from utils import AverageMeter, str2bool import torch.nn.functional as F from torch.utils.data import DataLoader, Subset import torch import torch.distributed as dist import os from torch.utils.data import DataLoader, DistributedSampler from scipy.stats import pearsonr, spearmanr from easydict import EasyDict as edict import warnings, os, sys ``` -------------------------------- ### Create Training Data Loader Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Creates a DataLoader for the training dataset, enabling shuffling and parallel loading with specified batch size, number of workers, and prefetch factor. ```python train_loader = DataLoader(train_dataset, batch_size=config['batch_size'], shuffle=True, num_workers=config['num_workers'], prefetch_factor=6, drop_last=True) ``` -------------------------------- ### Load Pre-trained GigaTIME Model from HuggingFace Source: https://context7.com/prov-gigatime/gigatime/llms.txt Downloads and loads pre-trained GigaTIME model weights from the HuggingFace Hub. Requires the `HF_TOKEN` environment variable to be set. Ensures the model is moved to the appropriate device (CUDA or CPU). ```python import os import torch from huggingface_hub import snapshot_download from archs import gigatime # Set device device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') # Initialize model architecture model = gigatime(num_classes=23, input_channels=3).to(device) # Download model from HuggingFace (requires HF_TOKEN environment variable) repo_id = "prov-gigatime/GigaTIME" local_dir = snapshot_download(repo_id=repo_id) # Load pre-trained weights weights_path = os.path.join(local_dir, "model.pth") state_dict = torch.load(weights_path, map_location=device) model.load_state_dict(state_dict) model.eval() print(f"Model loaded successfully on {device}") ``` -------------------------------- ### Run Model Testing Script Source: https://context7.com/prov-gigatime/gigatime/llms.txt Execute the database testing script to evaluate model performance on a test set, providing detailed per-channel metrics. Ensure all specified paths and parameters are correctly configured. ```bash python scripts/db_test.py \ --arch gigatime \ --tiling_dir "./data/sample_test_data/data/" \ --metadata "./data/sample_metadata.csv" \ --name gigatime_eval \ --output_dir "./output/" \ --set silver \ --batch_size 16 \ --input_h 512 \ --input_w 512 \ --num_classes 23 \ --val_sampling_prob 1.0 ``` -------------------------------- ### Scikit-learn and EasyDict Imports Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Imports for machine learning utilities from scikit-learn, specifically for data splitting, and EasyDict for convenient configuration management. ```python from sklearn.model_selection import train_test_split from easydict import EasyDict as edict ``` -------------------------------- ### Epoch Summary Output Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Prints the training and validation loss and IoU metrics at the end of an epoch. Also includes a concluding message about further training. ```text Train -> Loss: 0.7882, IoU: 0.2874 Val -> Loss: 0.8626, IoU: 0.2393 End of Epoch 1 Change settings (epochs) to train fully or use the train.py script to train the model ``` -------------------------------- ### Initialize New Columns for Metrics Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Initializes new columns in a dictionary based on the metric keys found in the first entry of the first directory's segmentation metrics. This prepares the structure for populating the metrics into the DataFrame. ```python new_columns = {col: [] for col in next(iter(segment_metric_dict[dir_names[0]].values())).keys()} ``` -------------------------------- ### Convert Tiling Directory Path Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Converts the tiling directory path from the configuration into a Path object for easier file system operations. This requires the 'tiling_dir' key in the config dictionary. ```python tiliting_dir = Path(config["tiling_dir"]) ``` -------------------------------- ### Load Experiment Metadata Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Reads the experiment metadata from a CSV file specified in the configuration. Ensure the 'metadata' key in the config dictionary points to a valid CSV file path. ```python metadata = pd.read_csv(config["metadata"]) ``` -------------------------------- ### Initialize HECOMETDataset_roi Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Initializes the HECOMETDataset_roi class with dataset parameters. Requires paths to tile pairs, transformation functions, and labeling options. ```python class HECOMETDataset_roi(torch.utils.data.Dataset): def __init__(self,all_tile_pair, tile_pair_df, transform,mask_noncell, cell_mask_label, dir_path , window_size , split, ): self.all_tile_pair = all_tile_pair self.tile_pair_df = tile_pair_df self.mask_noncell=mask_noncell self.transform = transform self.cell_mask_label=cell_mask_label self.dir_path = dir_path self.tile_pair_df = self.tile_pair_df self.all_tiles = [self.all_tile_pair.iloc[i]["pair_name"] for i in range(len(self.all_tile_pair))] ``` -------------------------------- ### Progress Bar Output Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Displays the progress of the training and validation steps using a progress bar format. This output is typically generated by libraries like 'tqdm'. ```text 100%|██████████| 879/879 [47:08<00:00, 3.22s/it, loss=0.788, pearson=0.287] 100%|██████████| 4/4 [00:26<00:00, 6.60s/it, loss=0.863, pearson=0.239] ``` -------------------------------- ### Define Training Augmentation with Cropping Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Defines the training augmentation pipeline when random cropping is enabled. Includes random rotations, flips, color adjustments, random cropping to target size, and normalization. ```python train_transform = Compose([ geometric.RandomRotate90(), # Randomly rotate images by 90 degrees geometric.Flip(), # Random horizontal/vertical flips OneOf([ transforms.HueSaturationValue(), # Adjust hue, saturation, and value transforms.RandomBrightnessContrast(brightness_limit=0, contrast_limit=0.2), # Adjust contrast only transforms.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0), # Adjust brightness only ], p=1), geometric.RandomCrop(config['input_h'], config['input_w']), # Random crop to target size transforms.Normalize() # Normalize pixel values ], is_check_shapes=False) ``` -------------------------------- ### Torchvision and Image Utilities Imports Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Imports for PyTorch's computer vision library, including image transformations and utilities for saving images. ```python import torchvision from torchvision.utils import save_image ``` -------------------------------- ### Save Configuration to YAML Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Saves the configuration dictionary to a YAML file for reproducibility. Ensure the output directory and model name are correctly set. ```python with open(config['output_dir'] +'models/%s/config.yml' % config['name'], 'w') as f: yaml.dump(config, f) ``` -------------------------------- ### Load Metadata and Define Transformations Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Loads metadata from a CSV file and defines image transformations using Albumentations. Ensure `pd`, `config`, `data_root`, and `transforms` are properly imported and defined. ```python metadata = pd.read_csv(config['metadata']) import albumentations as geometric val_transform = Compose([ geometric.Resize(config['input_h'], config['input_w']), transforms.Normalize(), ]) metadata['dir_name'] = data_root metadata ``` -------------------------------- ### Training Loop with Metrics Update Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Implements a training loop for a PyTorch model, calculating loss and Pearson correlation metrics. It includes data preprocessing, forward/backward passes, and metric updates. ```python def train(config, train_loader, model, criterion, optimizer): # Initialize average meters to track loss and Pearson correlation metrics avg_meters = {'loss': AverageMeter(), 'pearson': AverageMeter()} pearson_per_class_meters = [AverageMeter() for _ in range(config['num_classes'])] window_size = config['window_size'] # Set model to training mode model.train() # Initialize progress bar for training loop pbar = tqdm.tqdm(total=len(train_loader)) for input, target, name in train_loader: # Downsample target by factor of 8, then resize to input dimensions to make the target coarse to discount for any pixel level registration error downsampled_image = F.interpolate(target, scale_factor=1/8, mode='bilinear', align_corners=False) target = F.interpolate(downsampled_image, size=(config["input_h"],config["input_h"])), mode='bilinear', align_corners=False) target = target.cuda() # Forward pass through model output_image = model(input.cuda()).cuda() # Calculate loss between predicted and target images loss = criterion(output_image, target) # Backpropagation and parameter update optimizer.zero_grad() loss.backward() optimizer.step() # Calculate IoU metrics for overall and per-class evaluation _, pearson, _ = get_box_metrics(output_image, target, box_size=8) # Update per-class Pearson meters for class_idx, pearson_value in enumerate(pearson): pearson_per_class_meters[class_idx].update(pearson_value, input.size(0)) # Update average meters with current batch metrics avg_meters['loss'].update(loss.item(), input.size(0)) avg_meters['pearson'].update(np.nanmean(pearson), input.size(0)) # Update progress bar with current metrics pbar.set_postfix({'loss': avg_meters['loss'].avg, 'pearson': avg_meters['pearson'].avg}) pbar.update(1) pbar.close() ``` -------------------------------- ### Sample DataFrame Output Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb This is a sample representation of the 'result_df' DataFrame, showing columns for 'Channel', 'Pearson Correlation', and the assigned 'Color' in hex format. ```text Channel Pearson Correlation Color 0 DAPI 0.695919 #1f77b4 1 CK 0.532076 #aec7e8 2 CD11c 0.528967 #ff7f0e 3 CD68 0.515629 #ffbb78 4 CD138 0.512969 #2ca02c 5 PD-L1 0.507495 #98df8a 6 CD16 0.487124 #d62728 7 CD3 0.458749 #ff9896 8 CD4 0.454227 #9467bd 9 PHH3-B 0.425760 #c5b0d5 10 CD34 0.349822 #8c564b 11 Ki67 0.335504 #c49c94 12 T-bet 0.328383 #e377c2 13 CD14 0.285896 #f7b6d2 14 CD8 0.224613 #7f7f7f 15 Caspase3-D 0.221382 #c7c7c7 16 PD-1 0.165211 #bcbd22 17 Transgelin 0.145652 #dbdb8d 18 Tryptase 0.131974 #17becf 19 CD20 0.128870 #9edae5 20 Actin-D 0.108910 #9edae5 ``` -------------------------------- ### Define Training Augmentation with Resizing Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Defines the training augmentation pipeline when random cropping is disabled. It includes random rotations, flips, color adjustments, resizing to target dimensions, and normalization. ```python train_transform = Compose([ geometric.RandomRotate90(), # Randomly rotate images by 90 degrees geometric.Flip(), # Random horizontal/vertical flips OneOf([ transforms.HueSaturationValue(), # Adjust hue, saturation, and value transforms.RandomBrightnessContrast(brightness_limit=0, contrast_limit=0.2), # Adjust contrast only transforms.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0), # Adjust brightness only ], p=1), geometric.Resize(config['input_h'], config['input_w']), # Resize to target dimensions transforms.Normalize() # Normalize pixel values ], is_check_shapes=False) ``` -------------------------------- ### PyTorch Core and Utilities Imports Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Imports for PyTorch's core functionalities, neural network modules, functional API, optimizers, and utilities for data loading and management. Includes imports for CUDA backend optimization. ```python import torch import torch.nn as nn import torch.nn.functional as F import torch.optim as optim import torch.backends.cudnn as cudnn from torch.utils.data import DataLoader, Subset ``` -------------------------------- ### Plot Sample Image and Channels Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Visualizes a sample H&E image patch and its corresponding segmented channels using the defined color map. Handles image normalization and channel filtering for display. ```python def plot_sample(idx=0): img, mask, info = next(iter(test_loader)) he_img = img[idx, :3].cpu().numpy() he_img = he_img * std[:, None, None] + mean[:, None, None] he_img = np.clip(he_img, 0, 1) he_img = np.transpose(he_img, (1, 2, 0)) filtered_indices, filtered_names = get_display_channels() channel_colors = build_channel_color_map(filtered_names) n_cols = 4 n_rows = (len(filtered_indices) + 1 + n_cols - 1) // n_cols fig, axes = plt.subplots(n_rows, n_cols, figsize=(4 * n_cols, 4 * n_rows)) axes = axes.flatten() axes[0].imshow(he_img) axes[0].set_title("H&E patch") axes[0].axis('off') for plot_idx, ch in enumerate(filtered_indices): channel_name = filtered_names[plot_idx] ``` -------------------------------- ### Load Pre-trained GigaTIME Model Source: https://github.com/prov-gigatime/gigatime/blob/main/README.md Loads the GigaTIME model weights from HuggingFace. Requires the HF_TOKEN environment variable to be set. The model state dictionary is loaded to the CPU. ```python from huggingface_hub import snapshot_download import torch repo_id = "prov-gigatime/GigaTIME" local_dir = snapshot_download(repo_id=repo_id) weights_path = os.path.join(local_dir, "model.pth") state_dict = torch.load(weights_path, map_location="cpu") model.load_state_dict(state_dict) ``` -------------------------------- ### Activate Conda Environment Source: https://github.com/prov-gigatime/gigatime/blob/main/README.md Activates the 'gigatime' Conda environment. This should be done before running any project scripts. ```bash conda activate gigatime ``` -------------------------------- ### PyTorch Learning Rate Scheduler Imports Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Imports for learning rate scheduling functionalities within PyTorch's optimization module. ```python from torch.optim import lr_scheduler ``` -------------------------------- ### Execute Test Log for Silver Data Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb This snippet executes the test function, likely for evaluating a model's performance on a specific dataset ('silver'). It requires configuration, data loader, model, criterion, and a list of common channels. ```python test_log_silver = test(config, test_loader, model, criterion, common_channel_list=common_channel_list) ``` -------------------------------- ### Define Channel Filtering and Color Mapping Utilities Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_testing.ipynb Provides functions to filter display channels by excluding specific ones and to build a color map for visualizing channels. Handles channel name parsing and color hex conversion. ```python import matplotlib.pyplot as plt import matplotlib.colors as mcolors def get_display_channels(): exclude_channels = ['TRITC', 'Cy5'] filtered_indices = [i for i, name in enumerate(common_channel_list) if name not in exclude_channels] filtered_names = [] for i in filtered_indices: name = common_channel_list[i] if ' - ' in name: name = name.split(' - ')[0].strip() filtered_names.append(name) return filtered_indices, filtered_names def build_channel_color_map(channel_names): palette = plt.cm.get_cmap('tab20', max(len(channel_names), 1)) return { channel_name: mcolors.to_hex(palette(idx)) for idx, channel_name in enumerate(channel_names) } ``` -------------------------------- ### Training and Validation Loop Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Iterates through specified epochs, performing training and validation steps. Logs loss and IoU metrics for both training and validation sets. Use 'train.py' for full model training. ```python for epoch in range(config['epochs']): print(f"Epoch [{epoch+1}/{config['epochs']}]") # --- Train --- train_log = train(config, train_loader, model, criterion, optimizer) # --- Validate --- val_log = validate(config, val_loader, model, criterion) print(f"Train -> Loss: {train_log['loss']:.4f}, IoU: {train_log['pearson']:.4f}") print(f"Val -> Loss: {val_log['loss']:.4f}, IoU: {val_log['pearson']:.4f}") print("End of Epoch 1") print("Change settings (epochs) to train fully or use the train.py script to train the model") ``` -------------------------------- ### Sample Validation Data Source: https://github.com/prov-gigatime/gigatime/blob/main/scripts/gigatime_training.ipynb Samples a subset of validation data for faster evaluation during training. Ensure 'config' and 'val_loader' are properly initialized. ```python val_loader = sample_data_loader(val_loader, config, config['val_sampling_prob'], deterministic=True, what_split="valid") ```