### CellViT Metadata Handover for Single WSI Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Provides an example of manually setting metadata for a WSI, such as microns per pixel (mpp) and magnification. This is useful when the WSI file lacks this information or when you need to override existing metadata. This example processes a single TIFF file. ```bash cellvit-inference \ --model HIPT \ --outdir ./test_results/BRACS/minimal/HIPT \ process_wsi \ --wsi_path ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff \ --wsi_mpp 0.25 \ --wsi_magnification 40 ``` ```yaml model: HIPT output_format: outdir: ./test_results/BRACS/minimal/HIPT process_wsi: wsi_path: ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff wsi_mpp: 0.25 wsi_magnification: 40 ``` -------------------------------- ### Check CellViT-Inference Installation Source: https://github.com/tio-ikim/cellvit-inference/blob/main/README.md Verifies the installation of CellViT-Inference and its dependencies. This command checks for required and optional libraries, printing warnings for missing components. ```bash cellvit-check ``` -------------------------------- ### Install CellViT-Inference from PyPI Source: https://github.com/tio-ikim/cellvit-inference/blob/main/README.md Installs the CellViT-Inference package from the Python Package Index (PyPI). This is the standard method for installing the latest stable release of the package. ```bash pip install cellvit ``` -------------------------------- ### Run CellViT CLI Utilities (Bash) Source: https://context7.com/tio-ikim/cellvit-inference/llms.txt This snippet showcases essential command-line interface (CLI) utilities for CellViT. These commands help verify installation, download example datasets, and manage model weights, facilitating setup and experimentation. ```bash # Verify CellViT installation and system configuration cellvit-check # Output: System info, GPU availability, optional dependencies status # Download test database for experimentation cellvit-download-examples # Downloads sample WSIs to current directory: # - x40_svs/: High-magnification slides # - x20_svs/: Low-magnification slides # - BRACS/: Breast cancer TIFF images # - Philips/: Alternative scanner format # Pre-download model weights cellvit-download-models # Caches model weights to ~/.cache/cellvit (or $CELLVIT_CACHE) # Set custom cache directory export CELLVIT_CACHE=/path/to/custom/cache cellvit-download-models ``` -------------------------------- ### Configure CPU Core Count for CellViT Inference (YAML) Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Configures the number of CPU cores for CellViT inference via a YAML file. This setup includes the model, output directory, input dataset folder, and the desired system CPU core count. ```yaml model: SAM output_format: outdir: ./test_results/x40_svs/cpu_count/SAM process_dataset: wsi_folder: ./test_database/x40_svs system: cpu_count: 16 ``` -------------------------------- ### Install CellViT-Inference in Development Mode Source: https://github.com/tio-ikim/cellvit-inference/blob/main/README.md Installs the CellViT-Inference package in editable (development) mode from a local Git repository. Changes made to the source code will be immediately reflected without reinstallation. ```bash pip install -e . ``` -------------------------------- ### Install PyTorch with CUDA support Source: https://github.com/tio-ikim/cellvit-inference/blob/main/README.md Installs a specific version of PyTorch (2.2.2) with corresponding torchvision and torchaudio, compiled for CUDA 12.1. This is a critical dependency for GPU acceleration in CellViT-Inference. ```bash pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2 --index-url https://download.pytorch.org/whl/cu121 ``` -------------------------------- ### Ocelot Classification with SAM Model Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md This example demonstrates Ocelot classification using the SAM model with cellvit-inference. It sets the model, nuclei taxonomy, output directory, and WSI path. The wsi_mpp and wsi_magnification parameters are included due to the absence of metadata in the example file. ```bash cellvit-inference \ --model SAM \ --nuclei_taxonomy ocelot \ --outdir ./test_results/BRACS/ocelot/SAM \ process_wsi \ --wsi_path ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff \ --wsi_mpp 0.25 \ --wsi_magnification 40 ``` ```yaml model: SAM nuclei_taxonomy: ocelot output_format: outdir: ./test_results/BRACS/ocelot/SAM process_wsi: wsi_path: ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff wsi_mpp: 0.25 wsi_magnification: 40 ``` -------------------------------- ### Configure Ray Workers and CPU Cores for CellViT Inference (YAML) Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Defines the Ray multiprocessing setup for CellViT inference using YAML. This configuration includes the total CPU cores, number of Ray workers, model, output directory, and input dataset. ```yaml model: SAM output_format: outdir: ./test_results/x40_svs/cpu_count_ray_count/SAM process_dataset: wsi_folder: ./test_database/x40_svs system: cpu_count: 16 ray_worker: 4 ``` -------------------------------- ### Download CellViT Example Database Source: https://github.com/tio-ikim/cellvit-inference/blob/main/README.md Downloads a sample dataset for testing and demonstration purposes. This includes whole slide images in various formats and magnifications. ```bash cellvit-download-examples ``` -------------------------------- ### MIDOG Classification with SAM Model Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md This example shows how to perform MIDOG classification using the SAM model with cellvit-inference. It sets the model, nuclei taxonomy, output directory, and WSI path. The wsi_mpp and wsi_magnification parameters are included as the example file lacks metadata. Note that a HIPT classifier for MIDOG is not provided. ```bash cellvit-inference \ --model SAM \ --nuclei_taxonomy midog \ --outdir ./test_results/BRACS/midog/SAM \ process_wsi \ --wsi_path ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff \ --wsi_mpp 0.25 \ --wsi_magnification 40 ``` ```yaml model: SAM nuclei_taxonomy: midog output_format: outdir: ./test_results/BRACS/midog/SAM process_wsi: wsi_path: ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff wsi_mpp: 0.25 wsi_magnification: 40 ``` -------------------------------- ### Check CellViT Installation Source: https://github.com/tio-ikim/cellvit-inference/blob/main/CellViT_Inference.ipynb Verifies that the cellvit installation was successful and the command-line interface is accessible. This command should be run after the installation. ```shell !cellvit-check ``` -------------------------------- ### Download Example Data for CellViT Source: https://github.com/tio-ikim/cellvit-inference/blob/main/CellViT_Inference.ipynb Downloads a sample dataset to be used for testing the cellvit inference functionality. This is useful for verifying the pipeline before using custom data. ```shell !cellvit-download-examples ``` -------------------------------- ### Basic CellViT-SAM-H-Backbone (ViT-H) Inference Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Performs a basic inference run using the CellViT-SAM-H-Backbone model. Similar to the HIPT example, this processes a single WSI and outputs results to a designated directory. It showcases the configuration for the SAM model. ```bash cellvit-inference \ --model SAM \ --outdir ./test_results/x40_svs/minimal/SAM \ process_wsi \ --wsi_path ./test_database/x40_svs/JP2K-33003-2.svs ``` ```yaml model: SAM output_format: outdir: ./test_results/x40_svs/minimal/SAM process_wsi: wsi_path: ./test_database/x40_svs/JP2K-33003-2.svs ``` -------------------------------- ### Install CellViT and Dependencies Source: https://github.com/tio-ikim/cellvit-inference/blob/main/CellViT_Inference.ipynb Installs the necessary openslide-tools and the cellvit Python package. This is a prerequisite for using the cellvit inference capabilities. ```shell !apt update && apt install -y openslide-tools !pip install cellvit ``` -------------------------------- ### Install OpenSlide using Conda Source: https://github.com/tio-ikim/cellvit-inference/blob/main/README.md Installs the OpenSlide library, a prerequisite for CellViT-Inference, using the conda-forge channel. This ensures that the necessary tools for handling whole slide images are available. ```bash conda install -c conda-forge openslide ``` -------------------------------- ### CellViT Advanced Output Formats (GeoJSON, Graph, Compression) Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Demonstrates advanced output customization by enabling GeoJSON, cell graph, and snappy compression. This example processes a WSI and saves the output in multiple formats, including compressed files and a cell graph representation. It's useful for comprehensive data export and analysis. ```bash cellvit-inference \ --model SAM \ --outdir ./test_results/x40_svs/compression_graph_geojson/SAM \ --geojson \ --graph \ --compression \ process_wsi \ --wsi_path ./test_database/x40_svs/JP2K-33003-2.svs ``` ```yaml model: SAM output_format: outdir: ./test_results/x40_svs/compression_graph_geojson/SAM geojson: true graph: true compression: true process_wsi: wsi_path: ./test_database/x40_svs/JP2K-33003-2.svs ``` -------------------------------- ### CellViT QuPath-Compatible GeoJSON Output Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Enables the generation of QuPath-compatible GeoJSON output for processed WSIs. This example directs the output to a specific directory and includes the '--geojson' flag to activate this format. It's beneficial for integrating CellViT results with QuPath for further analysis. ```bash cellvit-inference \ --model SAM \ --outdir ./test_results/x40_svs/geojson/SAM \ --geojson \ process_wsi \ --wsi_path ./test_database/x40_svs/JP2K-33003-2.svs ``` ```yaml model: SAM output_format: outdir: ./test_results/x40_svs/geojson/SAM geojson: true process_wsi: wsi_path: ./test_database/x40_svs/JP2K-33003-2.svs ``` -------------------------------- ### PanOpTILS Classification with SAM Model Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md This snippet demonstrates how to perform PanOpTILS classification using the SAM model with cellvit-inference. It specifies the model, nuclei taxonomy, output directory, and paths to the whole-slide image. The wsi_mpp and wsi_magnification parameters are included as the example file lacks metadata. ```bash cellvit-inference \ --model SAM \ --nuclei_taxonomy panoptils \ --outdir ./test_results/BRACS/panoptils/SAM \ process_wsi \ --wsi_path ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff \ --wsi_mpp 0.25 \ --wsi_magnification 40 ``` ```yaml model: SAM nuclei_taxonomy: panoptils output_format: outdir: ./test_results/BRACS/panoptils/SAM process_wsi: wsi_path: ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff wsi_mpp: 0.25 wsi_magnification: 40 ``` -------------------------------- ### PanOpTILS Classification with HIPT Model Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md This snippet shows how to perform PanOpTILS classification using the HIPT model with cellvit-inference. It configures the model, nuclei taxonomy, output directory, and WSI path. The wsi_mpp and wsi_magnification parameters are provided because the example file lacks metadata. ```bash cellvit-inference \ --model HIPT \ --nuclei_taxonomy panoptils \ --outdir ./test_results/BRACS/panoptils/HIPT \ process_wsi \ --wsi_path ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff \ --wsi_mpp 0.25 \ --wsi_magnification 40 ``` ```yaml model: HIPT nuclei_taxonomy: panoptils output_format: outdir: ./test_results/BRACS/panoptils/HIPT process_wsi: wsi_path: ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff wsi_mpp: 0.25 wsi_magnification: 40 ``` -------------------------------- ### Basic CellViT-HIPT-Backbone (ViT-S) Inference Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Performs a basic inference run using the CellViT-HIPT-Backbone model. It processes a single Whole Slide Image (WSI) and saves the results to a specified output directory. This example demonstrates the minimal configuration required for a standard run. ```bash cellvit-inference \ --model HIPT \ --outdir ./test_results/x40_svs/minimal/HIPT \ process_wsi \ --wsi_path ./test_database/x40_svs/JP2K-33003-2.svs ``` ```yaml model: HIPT output_format: outdir: ./test_results/x40_svs/minimal/HIPT process_wsi: wsi_path: ./test_database/x40_svs/JP2K-33003-2.svs ``` -------------------------------- ### Ocelot Classification with HIPT Model Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md This snippet illustrates Ocelot classification using the HIPT model with cellvit-inference. It configures the model, nuclei taxonomy, output directory, and WSI path. The wsi_mpp and wsi_magnification parameters are provided because the example file lacks metadata. ```bash cellvit-inference \ --model HIPT \ --nuclei_taxonomy ocelot \ --outdir ./test_results/BRACS/ocelot/HIPT \ process_wsi \ --wsi_path ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff \ --wsi_mpp 0.25 \ --wsi_magnification 40 ``` ```yaml model: HIPT nuclei_taxonomy: ocelot output_format: outdir: ./test_results/BRACS/ocelot/HIPT process_wsi: wsi_path: ./test_database/BRACS/BRACS_1640_N_3_cropped.tiff wsi_mpp: 0.25 wsi_magnification: 40 ``` -------------------------------- ### Install CellViT and Dependencies Source: https://context7.com/tio-ikim/cellvit-inference/llms.txt Installs system dependencies, OpenSlide, PyTorch with CUDA, and the CellViT package. Requires Linux environment for system dependencies and specific PyTorch version with CUDA support. ```bash sudo apt-get install libvips openslide gcc g++ libopencv-core-dev \ libopencv-imgproc-dev libsnappy-dev libgeos-dev llvm \ libjpeg-dev libpng-dev libtiff-dev conda install -c conda-forge openslide pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2 \ --index-url https://download.pytorch.org/whl/cu121 pip install cellvit cellvit-check ``` -------------------------------- ### Inspect module buffers and parameters Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.cell_segmentation.md Examples for accessing named buffers and parameters within a model. These methods are useful for filtering specific layers or weights during inference or training. ```python # Iterating over named buffers for name, buf in self.named_buffers(): if name in ['running_var']: print(buf.size()) # Iterating over named parameters for name, param in self.named_parameters(): if name in ['bias']: print(param.size()) ``` -------------------------------- ### Block Module Methods Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.md This section outlines the methods available for the Block module, covering initialization, forward pass, parameter and buffer management, and device/type conversions. ```APIDOC ## Block Module Methods ### Description Details the various methods and attributes associated with the Block module, including its constructor, forward pass, and comprehensive state management capabilities. ### Methods - `T_destination`: Attribute representing the destination type. - `__init__()`: Constructor for the Block module. - `add_module()`: Adds a submodule to the current module. - `apply()`: Applies a function to all modules in the network. - `bfloat16()`: Converts the module to bfloat16 precision. - `buffers()`: Returns an iterator over the module's buffers. - `call_super_init()`: Calls the superclass's initialization method. - `children()`: Returns an iterator over the module's direct children. - `compile()`: Compiles the module for performance optimization. - `cpu()`: Moves the module to the CPU. - `cuda()`: Moves the module to the GPU. - `double()`: Converts the module to double precision. - `dump_patches()`: Dumps patches for debugging. - `eval()`: Sets the module to evaluation mode. - `extra_repr()`: Returns a string representation of the module's extra information. - `float()`: Converts the module to single precision. - `forward()`: Defines the computation performed at every call. - `get_buffer()`: Gets a buffer by its name. - `get_extra_state()`: Gets the extra state of the module. - `get_parameter()`: Gets a parameter by its name. - `get_submodule()`: Gets a submodule by its name. - `half()`: Converts the module to half precision. - `ipu()`: Moves the module to an IPU device. - `load_state_dict()`: Loads the module's state from a dictionary. - `modules()`: Returns an iterator over all modules in the network. - `mtia()`: Moves the module to an MTIA device. - `named_buffers()`: Returns an iterator over the module's named buffers. - `named_children()`: Returns an iterator over the module's named direct children. - `named_modules()`: Returns an iterator over all named modules in the network. - `named_parameters()`: Returns an iterator over the module's named parameters. - `parameters()`: Returns an iterator over the module's parameters. - `register_backward_hook()`: Registers a backward hook. - `register_buffer()`: Registers a buffer. - `register_forward_hook()`: Registers a forward hook. - `register_forward_pre_hook()`: Registers a forward pre-hook. ``` -------------------------------- ### Initialize SystemConfiguration in Python Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.utils.md Initializes the SystemConfiguration class to detect hardware resources. This class validates GPU availability and provides metadata about the runtime environment, CPU counts, and memory. ```python from cellvit.utils.ressource_manager import SystemConfiguration # Initialize configuration for a specific GPU index config = SystemConfiguration(gpu=0) # Access hardware information print(f"GPU available: {config.has_gpu}") print(f"CPU count: {config.cpu_count}") ``` -------------------------------- ### Process WSI Folder with Custom Extension (.tiff) Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Processes a folder containing WSI files with a custom extension (e.g., '.tiff') using the SAM model. Outputs results to a specified directory. Requires the WSI folder path and the custom WSI extension. ```bash cellvit-inference \ --model SAM \ --outdir ./test_results/Philips/folder/SAM \ process_dataset \ --wsi_folder ./test_database/Philips \ --wsi_extension tiff ``` ```yaml model: SAM output_format: outdir: ./test_results/Philips/folder/SAM process_dataset: wsi_folder: ./test_database/Philips wsi_extension: tiff ``` -------------------------------- ### ViTCellViT Class Methods Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.md Documentation for the ViTCellViT class, a backbone for cell segmentation, detailing its initialization and various PyTorch-related methods. ```APIDOC ## ViTCellViT Class Methods ### Description Details the methods and properties of the `ViTCellViT` class, a Vision Transformer-based backbone for cell segmentation tasks. Includes initialization, module registration, and device/data type conversions. ### Class `ViTCellViT` ### Methods - `__init__()`: Initializes the ViTCellViT model. - `add_module(name, module)`: Adds a named module to the current module. - `apply(fn)`: Applies a function to all modules in the network. - `bfloat16()`: Converts all parameters and buffers to bfloat16. - `buffers(recurse=True)`: Returns an iterator over module buffers. - `children()`: Returns an iterator over immediate children modules. - `compile(mode='default', *, fullgraph=False)`: Compiles the model. - `cpu()`: Moves all model parameters and buffers to the CPU. - `cuda(device=None)`: Moves all model parameters and buffers to the GPU. - `double()`: Converts all parameters and buffers to double precision. - `float()`: Converts all parameters and buffers to float. - `get_buffer(name)`: Returns the buffer with the given name. - `get_parameter(name)`: Returns the parameter with the given name. - `half()`: Converts all parameters and buffers to half precision. - `load_state_dict(state_dict, strict=True)`: Copies parameters and buffers from `state_dict` into this module. - `modules()`: Returns an iterator over all modules in the network. - `named_buffers(prefix='', recurse=True)`: Returns an iterator over module buffers, yielding both the name and the buffer. - `named_children()`: Returns an iterator over immediate children modules, yielding both the name and the module. - `named_modules(memo=None, prefix='')`: Returns an iterator over all modules in the network, yielding both the name and the module. - `named_parameters(memo=None, prefix='')`: Returns an iterator over all parameters in the network, yielding both the name and the parameter. - `parameters(recurse=True)`: Returns an iterator over module parameters. - `register_backward_hook(hook)`: Registers a backward hook. - `register_buffer(name, tensor, persistent=True)`: Adds a buffer to the module. - `register_forward_hook(hook)`: Registers a forward hook. - `register_forward_pre_hook(hook)`: Registers a forward pre-hook. - `register_load_state_dict_post_hook(hook)`: Registers a post-load state_dict hook. - `register_load_state_dict_pre_hook(hook)`: Registers a pre-load state_dict hook. - `register_module(name, module)`: Adds a child module to the current module. - `register_parameter(name, param, requires_grad=True)`: Adds a parameter to the module. - `register_state_dict_post_hook(hook)`: Registers a post-state_dict hook. - `register_state_dict_pre_hook(hook)`: Registers a pre-state_dict hook. - `requires_grad_(requires_grad=True)`: Convenience method to set `requires_grad` for all parameters in the module. - `set_extra_state(new_extra_state)`: Sets the extra state. - `set_submodule(target_key, module)`: Sets a submodule. - `share_memory()`: Moves all model parameters and buffers to the shared memory. - `state_dict(destination=None, prefix='', keep_vars=False)`: Returns a dictionary containing a whole state of the module. - `to(device=None, dtype=None, non_blocking=False)`: Moves model to the specified device and dtype. - `to_empty(device=None, recurse=True)`: Clears the parameters and buffers of the module. - `train(mode=True)`: Sets the module in training mode. - `type(dst_type, non_blocking=False)`: Converts all parameters and buffers to the specified type. - `to_padded_tensor(pad_input=False, output_device=None)`: Converts the module to a padded tensor. - `xpu(device=None)`: Moves all model parameters and buffers to the XPU. - `zero_grad(set_to_none=False)`: Zeros out the gradients of all parameters. ### Properties - `T_destination`: Type hint for destination. - `call_super_init`: Flag to call super init. - `dump_patches`: Flag to dump patches. ### Module Structure - `cellvit.models.cell_segmentation.backbones.ViTCellViT` ``` -------------------------------- ### Process WSI Folder with Default Extension (.svs) Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Processes a folder containing WSI files with the default '.svs' extension using the SAM model. Outputs results to a specified directory. Requires the WSI folder path. ```bash cellvit-inference \ --model SAM \ --outdir ./test_results/x40_svs/folder/SAM \ process_dataset \ --wsi_folder ./test_database/x40_svs ``` ```yaml model: SAM output_format: outdir: ./test_results/x40_svs/folder/SAM process_dataset: wsi_folder: ./test_database/x40_svs ``` -------------------------------- ### Process WSI Folder with Metadata (MPP and Magnification) Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Processes a folder containing WSI files, explicitly providing metadata such as Mean Pixelum (MPP) and magnification. This is useful for WSIs with specific characteristics or when metadata is not embedded. Requires WSI folder, extension, MPP, and magnification. ```bash cellvit-inference \ --model SAM \ --outdir ./test_results/BRACS/folder/SAM \ process_dataset \ --wsi_folder ./test_database/BRACS \ --wsi_extension tiff \ --wsi_mpp 0.25 \ --wsi_magnification 40 ``` ```yaml model: SAM output_format: outdir: ./test_results/BRACS/folder/SAM process_dataset: wsi_folder: ./test_database/BRACS wsi_extension: tiff wsi_mpp: 0.25 wsi_magnification: 40 ``` -------------------------------- ### Initialize and Create Logger in Python Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.utils.md Demonstrates how to instantiate the Logger class and generate a logger instance. The Logger supports file-based rotation and console output with configurable levels and timestamps. ```python from cellvit.utils.logger import Logger # Initialize the logger with specific directory and timestamp settings logger = Logger(level="INFO", log_dir="/path/to/log_dir", comment="my_logs", use_timestamp=True) # Create and return the logger instance logger_instance = logger.create_logger() ``` -------------------------------- ### CellViT Binary Classification Output Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Configures the CellViT inference to perform binary classification on nuclei. This example specifies the 'binary' nuclei taxonomy and directs the output to a specific directory. It's useful for tasks requiring a simple binary categorization of nuclei. ```bash cellvit-inference \ --model SAM \ --nuclei_taxonomy binary \ --outdir ./test_results/x40_svs/binary/SAM \ process_wsi \ --wsi_path ./test_database/x40_svs/JP2K-33003-2.svs ``` ```yaml model: SAM nuclei_taxonomy: binary output_format: outdir: ./test_results/x40_svs/binary/SAM process_wsi: wsi_path: ./test_database/x40_svs/JP2K-33003-2.svs ``` -------------------------------- ### ViTCellViTDeit Model Methods Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.md Methods for the ViTCellViTDeit backbone, including initialization, forward pass, and module configuration. ```APIDOC ## [METHOD] ViTCellViTDeit Methods ### Description Interface for the ViTCellViTDeit model, supporting standard PyTorch operations like forward passes, device migration, and module registration. ### Method N/A (Class Methods) ### Endpoint cellvit.models.cell_segmentation.backbones.ViTCellViTDeit ### Parameters #### Path Parameters - **method_name** (string) - Required - The method to execute (e.g., forward, eval, cuda, cpu) ### Request Example { "action": "forward", "params": { "input": "tensor_data" } } ### Response #### Success Response (200) - **output** (tensor) - The result of the forward pass or module operation. #### Response Example { "status": "success", "output": "[...tensor_data...]" } ``` -------------------------------- ### GET /named_modules Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Retrieves an iterator over all modules in the network. ```APIDOC ## GET /named_modules ### Description Return an iterator over all modules in the network, yielding both the name of the module and the module instance. ### Method GET ### Endpoint /named_modules ### Parameters #### Query Parameters - **memo** (set) - Optional - Set of modules already added to the result. - **prefix** (str) - Optional - Prefix added to the module name. - **remove_duplicate** (bool) - Optional - Whether to remove duplicated module instances. ### Response #### Success Response (200) - **iterator** (tuple[str, Module]) - Iterator yielding (name, module) tuples. ``` -------------------------------- ### Get Segmentation Template Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.config.md Retrieves a template for a MultiPolygon GeoJSON object. ```APIDOC ## GET /config/templates/segmentation ### Description Retrieves a template for a MultiPolygon GeoJSON object. ### Method GET ### Endpoint /config/templates/segmentation ### Parameters #### Path Parameters None #### Query Parameters None ### Request Example None ### Response #### Success Response (200) - **template** (dict) - A dictionary representing the MultiPolygon GeoJSON template. #### Response Example ```json { "type": "MultiPolygon", "coordinates": [ [ [ [0.0, 0.0], [1.0, 1.0], [1.0, 0.0], [0.0, 0.0] ] ] ] } ``` ``` -------------------------------- ### Get Point Template Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.config.md Retrieves a template for a Point GeoJSON object. ```APIDOC ## GET /config/templates/point ### Description Retrieves a template for a Point GeoJSON object. ### Method GET ### Endpoint /config/templates/point ### Parameters #### Path Parameters None #### Query Parameters None ### Request Example None ### Response #### Success Response (200) - **template** (dict) - A dictionary representing the Point GeoJSON template. #### Response Example ```json { "type": "Point", "coordinates": [0.0, 0.0] } ``` ``` -------------------------------- ### LinearClassifier Initialization Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.classifier.md Documentation for the LinearClassifier class, detailing its parameters for initialization. ```APIDOC ## LinearClassifier Class ### Description Provides a Linear Classifier model for classification tasks. ### Class `cellvit.models.classifier.linear_classifier.LinearClassifier` ### Parameters * **embed_dim** (*int*) - Embedding dimension (input dimension). * **hidden_dim** (*int*, optional) - Hidden layer dimension. Defaults to 100. * **num_classes** (*int*, optional) - Number of output classes. Defaults to 2. * **drop_rate** (*float*, optional) - Dropout rate. Defaults to 0. ### Methods #### `__init__(self, embed_dim: int, hidden_dim: int = 100, num_classes: int = 2, drop_rate: float = 0)` Initializes the LinearClassifier model with the specified parameters. #### `add_module(self, name: str, module: Module | None)` Adds a child module to the current module. #### `apply(self, fn: Callable[[Module], None])` Applies a function recursively to every submodule and self. #### `bfloat16(self)` Casts all floating point parameters and buffers to `bfloat16` datatype in-place. #### `buffers(self, recurse: bool = True)` Returns an iterator over module buffers. #### `children(self)` Returns an iterator over immediate children modules. #### `compile(self, *args, **kwargs)` Compiles the module's forward pass using `torch.compile()`. #### `cpu(self)` Moves all model parameters and buffers to the CPU in-place. #### `cuda(self, device: int | device | None = None)` Moves all model parameters and buffers to the GPU in-place. #### `double(self)` Casts all floating point parameters and buffers to `double` datatype in-place. ``` -------------------------------- ### Configure Ray Workers and CPU Cores for CellViT Inference (Bash) Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Sets up Ray for multiprocessing in CellViT inference, specifying the total CPU cores and the number of Ray workers for parallel post-processing. This command also defines the model, output directory, and input dataset. ```bash cellvit-inference \ --model SAM \ --outdir ./test_results/x40_svs/cpu_count_ray_worker/SAM \ --cpu_count 16 \ --ray_worker 4 \ process_dataset \ --wsi_folder ./test_database/x40_svs ``` -------------------------------- ### Implement ColoredFormatter for Logging Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.utils.md Demonstrates how to instantiate the ColoredFormatter and apply it to a logging handler. This enables colorized output for log records processed by the handler. ```python formatter = ColoredFormatter() log_handler.setFormatter(formatter) ``` -------------------------------- ### Process WSI using CSV Filelist Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/examples.md Processes WSI files listed in a CSV file. The CSV must contain at least a 'path' column. Optional columns 'wsi_mpp' and 'wsi_magnification' can be provided per WSI. If not present, the tool attempts to read them from WSI metadata. Uses the HIPT model. ```bash cellvit-inference \ --model HIPT \ --outdir ./test_results/all/HIPT \ process_dataset \ --wsi_filelist ./test_database/filelist.csv ``` ```yaml model: HIPT output_format: outdir: /test_results/all/HIPT process_dataset: wsi_filelist: ./test_database/filelist.csv ``` -------------------------------- ### GET /named_parameters Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Retrieves an iterator over module parameters with their names. ```APIDOC ## GET /named_parameters ### Description Return an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself. ### Method GET ### Endpoint /named_parameters ### Parameters #### Query Parameters - **prefix** (str) - Optional - Prefix to prepend to parameter names. - **recurse** (bool) - Optional - If True, yields parameters of this module and submodules. - **remove_duplicate** (bool) - Optional - Whether to remove duplicated parameters. ### Response #### Success Response (200) - **iterator** (tuple[str, Parameter]) - Iterator yielding (name, parameter) tuples. ``` -------------------------------- ### ViTCellViT Model Methods Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.md This section details various methods available for the ViTCellViT model, including evaluation, state manipulation, and hook registrations. ```APIDOC ## ViTCellViT Model Methods This section details various methods available for the ViTCellViT model, including evaluation, state manipulation, and hook registrations. ### Methods - **eval()**: Sets the model to evaluation mode. - **extra_repr()**: Returns a string representation of the module's extra parameters. - **float()**: Converts all floating point parameters and buffers to half precision. - **forward()**: Defines the computation performed at every call. - **get_buffer()**: Returns a buffer by name. - **get_extra_state()**: Returns the extra state of the module. - **get_intermediate_layers()**: Retrieves intermediate layers of the model. - **get_last_selfattention()**: Gets the last self-attention weights. - **get_parameter()**: Returns a parameter by name. - **get_submodule()**: Returns a submodule by name. - **half()**: Converts all floating point parameters and buffers to half precision. - **interpolate_pos_encoding()**: Interpolates positional encoding. - **ipu()**: Configures the model for IPU execution. - **load_state_dict()**: Loads a state dictionary into the module. - **modules()**: Returns an iterator over all modules in the network. - **mtia()**: Configures the model for MTIA execution. - **named_buffers()**: Returns an iterator over module buffers, yielding both the name and the buffer. - **named_children()**: Returns an iterator over immediate children modules, yielding both the name and the module. - **named_modules()**: Returns an iterator over all the modules in the network, yielding both the name of the module and the module itself. - **named_parameters()**: Returns an iterator over module parameters, yielding both the name and the parameter. - **parameters()**: Returns an iterator over module parameters. - **prepare_tokens()**: Prepares input tokens for the model. - **register_backward_hook()**: Registers a backward hook. - **register_buffer()**: Adds a buffer to the module. - **register_forward_hook()**: Registers a forward hook. - **register_forward_pre_hook()**: Registers a forward pre-hook. - **register_full_backward_hook()**: Registers a full backward hook. - **register_full_backward_pre_hook()**: Registers a full backward pre-hook. - **register_load_state_dict_post_hook()**: Registers a post hook for load_state_dict. - **register_load_state_dict_pre_hook()**: Registers a pre hook for load_state_dict. - **register_module()**: Adds a submodule to the current module. - **register_parameter()**: Adds a parameter to the module. - **register_state_dict_post_hook()**: Registers a post hook for state_dict. ### Request Example ```json { "input_data": "example_data" } ``` ### Response #### Success Response (200) - **output** (any) - The result of the model's operation. #### Response Example ```json { "output": "processed_data" } ``` ``` -------------------------------- ### GET /named_buffers Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Retrieves an iterator over module buffers with their names. ```APIDOC ## GET /named_buffers ### Description Return an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself. ### Method GET ### Endpoint /named_buffers ### Parameters #### Query Parameters - **prefix** (str) - Optional - Prefix to prepend to all buffer names. - **recurse** (bool) - Optional - If True, yields buffers of this module and all submodules. - **remove_duplicate** (bool) - Optional - Whether to remove duplicated buffers. ### Response #### Success Response (200) - **iterator** (tuple[str, Tensor]) - Iterator yielding (name, buffer) tuples. ``` -------------------------------- ### GET /modules Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Introspection methods for module hierarchy and parameters. ```APIDOC ## GET /modules ### Description Provides various methods to inspect the module hierarchy, including children, all modules, and parameters. ### Method GET ### Endpoint /modules/{type} ### Parameters #### Path Parameters - **type** (string) - Required - One of: 'children', 'modules', 'parameters'. ### Response #### Success Response (200) - **iterator** (Iterator) - Returns an iterator of the requested module components. ``` -------------------------------- ### GET /buffers Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Retrieves an iterator over the module's buffers. ```APIDOC ## GET /buffers ### Description Return an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself. ### Method GET ### Endpoint /buffers ### Parameters #### Query Parameters - **prefix** (str) - Optional - Prefix to prepend to all buffer names. - **recurse** (bool) - Optional - If True, yields buffers of this module and all submodules. - **remove_duplicate** (bool) - Optional - Whether to remove duplicated buffers. ### Response #### Success Response (200) - **iterator** (tuple[str, Tensor]) - Iterator yielding (name, buffer) pairs. ``` -------------------------------- ### GET /modules Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.cell_segmentation.md Retrieves an iterator over all modules within the network architecture. ```APIDOC ## GET /modules ### Description Returns an iterator over all modules in the network. Duplicate modules are returned only once. ### Method GET ### Endpoint /modules ### Response #### Success Response (200) - **modules** (Iterator) - An iterator yielding each module in the network. ``` -------------------------------- ### Initialize model weights using apply Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.cell_segmentation.md Demonstrates how to recursively apply a weight initialization function to all submodules within a PyTorch model using the apply method. ```python @torch.no_grad() def init_weights(m): print(m) if type(m) is nn.Linear: m.weight.fill_(1.0) print(m.weight) net = nn.Sequential(nn.Linear(2, 2), nn.Linear(2, 2)) net.apply(init_weights) ``` -------------------------------- ### Configure and Run CellViT Inference Source: https://context7.com/tio-ikim/cellvit-inference/llms.txt Demonstrates how to define inference parameters via a YAML configuration file and execute the pipeline from the command line. ```yaml model: SAM nuclei_taxonomy: pannuke inference: gpu: 0 enforce_amp: true batch_size: 8 output_format: outdir: ./results/output geojson: true graph: true compression: true process_wsi: wsi_path: ./slides/sample.svs wsi_mpp: 0.25 wsi_magnification: 40 system: cpu_count: 16 ray_worker: 4 ray_remote_cpus: 4 memory: 32768 debug: false ``` ```bash cellvit-inference --config config.yaml ``` -------------------------------- ### GET /state_dict Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.cell_segmentation.md Retrieves the state dictionary of the module containing all parameters and buffers. ```APIDOC ## GET /state_dict ### Description Returns a dictionary containing references to the whole state of the module, including parameters and persistent buffers. ### Method GET ### Endpoint /state_dict ### Parameters #### Query Parameters - **prefix** (string) - Optional - Prefix added to parameter and buffer names. - **keep_vars** (boolean) - Optional - If True, returned tensors are not detached from autograd. Default: False. ### Response #### Success Response (200) - **state** (dict) - A dictionary containing the module state. #### Response Example { "bias": [0.1, 0.2], "weight": [0.5, 0.9] } ``` -------------------------------- ### GET /named_buffers Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.cell_segmentation.md Returns an iterator over module buffers, yielding both the name and the buffer tensor. ```APIDOC ## GET /named_buffers ### Description Iterates over module buffers, providing access to both the buffer name and the tensor data. ### Method GET ### Endpoint /named_buffers ### Parameters #### Query Parameters - **prefix** (str) - Optional - Prefix to prepend to buffer names. - **recurse** (bool) - Optional - Whether to include submodules. Default: True. - **remove_duplicate** (bool) - Optional - Whether to remove duplicate buffers. Default: True. ### Response #### Success Response (200) - **buffer_data** (tuple) - A tuple containing (name: str, buffer: Tensor). ``` -------------------------------- ### GET /module/state_dict Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Retrieves the current state of the module, including parameters and persistent buffers. ```APIDOC ## GET /module/state_dict ### Description Returns a dictionary containing references to the module's parameters and buffers. ### Method GET ### Parameters #### Query Parameters - **prefix** (string) - Optional - Prefix for keys. - **keep_vars** (bool) - Optional - Whether to keep autograd variables. ### Response #### Success Response (200) - **state_dict** (dict) - A dictionary containing the module's state. ``` -------------------------------- ### GET /submodule Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Retrieves a specific submodule from the model hierarchy using a dot-separated path string. ```APIDOC ## GET /submodule ### Description Returns the submodule given by the target string path if it exists. This is an efficient way to check for the existence of nested modules. ### Method GET ### Parameters #### Query Parameters - **target** (string) - Required - The fully-qualified string name of the submodule (e.g., "net_b.linear"). ### Response #### Success Response (200) - **module** (object) - The requested torch.nn.Module instance. ### Errors - **AttributeError**: Thrown if the path does not resolve to an existing module. ``` -------------------------------- ### Attention Module Methods Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.md This section details the available methods for the Attention module, including state management, submodule manipulation, and device/type conversions. ```APIDOC ## Attention Module Methods ### Description Provides access to various methods for managing the state, submodules, and device/type configurations of the Attention module. ### Methods - `set_extra_state()`: Sets the extra state of the Attention module. - `set_submodule()`: Sets a submodule within the Attention module. - `share_memory()`: Shares the memory of the Attention module's parameters and buffers. - `state_dict()`: Returns a dictionary containing the module's state. - `to()`: Moves the module's parameters and buffers to a specified device (e.g., CPU, GPU). - `to_empty()`: Clears the module's parameters and buffers, freeing up memory. - `train()`: Sets the module to training mode. - `training`: A boolean indicating whether the module is in training mode. - `type()`: Returns the type of the module. - `xpu()`: Moves the module to an XPU device. - `zero_grad()`: Zeros out the gradients of the module's parameters. ``` -------------------------------- ### GET /module/parameter/{target} Source: https://github.com/tio-ikim/cellvit-inference/blob/main/docs/source/cellvit.models.utils.md Retrieves a specific parameter from the module using its fully-qualified string name. ```APIDOC ## GET /module/parameter/{target} ### Description Returns the parameter given by the target string if it exists. Throws an AttributeError if the target is not a valid parameter. ### Method GET ### Endpoint /module/parameter/{target} ### Parameters #### Path Parameters - **target** (string) - Required - The fully-qualified string name of the parameter. ### Response #### Success Response (200) - **parameter** (torch.nn.Parameter) - The requested parameter. #### Error Response (404) - **error** (AttributeError) - If the target string references an invalid path or non-parameter object. ```