### Install HistomicsTK Locally Source: https://github.com/digitalslidearchive/histomicstk/blob/master/CONTRIBUTING.rst Install HistomicsTK and its dependencies into a virtual environment for local development. ```shell mkvirtualenv HistomicsTK cd HistomicsTK/ python -m pip install setuptools-scm Cython>=0.25.2 scikit-build>=0.8.1 cmake>=0.6.0 numpy>=1.12.1 python -m pip install . ``` -------------------------------- ### Install HistomicsTK on Windows Source: https://github.com/digitalslidearchive/histomicstk/blob/master/README.rst Steps to install dependencies and the HistomicsTK package from source on Windows. ```bash $ pip install large-image $ pip install cmake $ git clone https://github.com/DigitalSlideArchive/HistomicsTK/ $ cd HistomicsTK/ $ python -m pip install setuptools-scm "Cython>=0.25.2" "scikit-build>=0.8.1" "cmake>=0.6.0" "numpy>=1.12.1" ``` ```bash $ pip install libtiff ``` -------------------------------- ### Install HistomicsTK and Girder Client Source: https://github.com/digitalslidearchive/histomicstk/blob/master/README.rst Standard installation commands for the HistomicsTK package and the Girder client. ```bash $ python -m pip install -e . $ pip install girder-client ``` -------------------------------- ### Install HistomicsTK dependencies Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/contributing.md Commands to initialize a virtual environment and install the necessary build dependencies and the package itself. ```shell $ mkvirtualenv HistomicsTK $ cd HistomicsTK/ $ python -m pip install setuptools-scm Cython>=0.25.2 scikit-build>=0.8.1 cmake>=0.6.0 numpy>=1.12.1 $ python -m pip install . ``` -------------------------------- ### Manually Start DSA Docker Server Source: https://github.com/digitalslidearchive/histomicstk/blob/master/CONTRIBUTING.rst Use these commands to start the HistomicsTK test environment manually for prototyping or persistent database state. ```bash $ cd HistomicsTK/tests/ $ docker compose up --build ``` -------------------------------- ### Install HistomicsTK from Source Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/installation.md Clone the HistomicsTK repository and install it locally using pip. This method is suitable for non-Linux systems or when system libraries are preferred for image format reading after installing large_image. ```shell git clone https://github.com/DigitalSlideArchive/HistomicsTK.git cd HistomicsTK pip install -e . ``` -------------------------------- ### Configure Environment and Import Modules Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/positive_pixel_count.ipynb Initial setup for HistomicsTK positive pixel count analysis, including matplotlib configuration and library imports. ```python # Configuration and imports of other libraries from __future__ import print_function import large_image %matplotlib inline import matplotlib.pyplot as plt import skimage.io #Some nice default configuration for plots plt.rcParams['figure.figsize'] = 15, 15 plt.rcParams['image.cmap'] = 'gray' # Import and alias positive_pixel_count import histomicstk.segmentation.positive_pixel_count as ppc ``` -------------------------------- ### Install HistomicsTK on Linux Source: https://github.com/digitalslidearchive/histomicstk/blob/master/README.rst Commands to install the package via PyPI or from a local source clone on Linux systems. ```bash $ python -m pip install histomicstk --find-links https://girder.github.io/large_image_wheels ``` ```bash $ git clone https://github.com/DigitalSlideArchive/HistomicsTK/ $ cd HistomicsTK/ $ python -m pip install -e . --find-links https://girder.github.io/large_image_wheels ``` -------------------------------- ### Import Libraries and Setup Colormap Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/semantic_segmentation_superpixel_approach.ipynb Imports necessary libraries for HistomicsTK, girder_client, numpy, and matplotlib. Sets up a colormap for visualization. ```python import tempfile import girder_client import numpy as np from histomicstk.annotations_and_masks.annotation_and_mask_utils import ( delete_annotations_in_slide) from histomicstk.saliency.cellularity_detection_superpixels import ( Cellularity_detector_superpixels) import matplotlib.pylab as plt from matplotlib.colors import ListedColormap %matplotlib inline # color map vals = np.random.rand(256,3) vals[0, ...] = [0.9, 0.9, 0.9] cMap = ListedColormap(1 - vals) ``` -------------------------------- ### Install HistomicsTK on OSX Source: https://github.com/digitalslidearchive/histomicstk/blob/master/README.rst Installation command for HistomicsTK and common tile source libraries on macOS. ```bash $ python -m pip install histomicstk large-image-source-tiff large-image-source-openslide ``` -------------------------------- ### Simple Backup Syntax Example Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotation_database_backup_and_sql_parser.ipynb Illustrates the basic syntax for backing up girder folder, item, and annotation information as JSON files, replicating the girder folder structure locally. Optionally, folder and item information can be saved as tables in an SQLite database. ```python ``` -------------------------------- ### Load a whole-slide image Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/using_large_image.ipynb Initializes a tile source object from a file path. Requires appropriate extras like openslide to be installed for specific formats. ```python ts = large_image.getTileSource(wsi_path) ``` -------------------------------- ### Get ROI Documentation Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/creating_gallery_images_review.ipynb Prints the documentation for the get_all_rois_from_folder_v2 function. This function retrieves all ROIs from a specified folder on the server. ```python print(get_all_rois_from_folder_v2.__doc__) ``` -------------------------------- ### Install HistomicsTK via Pip (Linux) Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/installation.md Use this command to install HistomicsTK on Linux systems using pip. The --find-links option helps in obtaining prebuilt libraries for image format reading. ```shell pip install histomicstk --find-links https://girder.github.io/large_image_wheels ``` -------------------------------- ### Annotation Generation Output Log Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/segmentation_masks_to_annotations.ipynb Example console output showing the progress of processing contours into annotation documents. ```text TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 1 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 2 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 3 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 4 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 5 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 6 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 7 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 8 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 9 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 10 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 11 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 12 of 13 TCGA-A2-A0YE: annotation docs: mostly_lymphocytic_infiltrate: doc 1 of 1: contour 13 of 13 TCGA-A2-A0YE: annotation docs: exclude: doc 1 of 1: contour 1 of 1 TCGA-A2-A0YE: annotation docs: blood_vessel: doc 1 of 1: contour 1 of 3 TCGA-A2-A0YE: annotation docs: blood_vessel: doc 1 of 1: contour 2 of 3 TCGA-A2-A0YE: annotation docs: blood_vessel: doc 1 of 1: contour 3 of 3 TCGA-A2-A0YE: annotation docs: roi: doc 1 of 1: contour 1 of 1 TCGA-A2-A0YE: annotation docs: normal_acinus_or_duct: doc 1 of 1: contour 1 of 2 TCGA-A2-A0YE: annotation docs: normal_acinus_or_duct: doc 1 of 1: contour 2 of 2 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 1 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 2 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 3 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 4 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 5 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 6 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 7 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 8 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 9 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 1 of 2: contour 10 of 10 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 1 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 2 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 3 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 4 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 5 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 6 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 7 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 8 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 9 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 10 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 11 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 12 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 13 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 14 of 15 TCGA-A2-A0YE: annotation docs: mostly_tumor: doc 2 of 2: contour 15 of 15 ``` -------------------------------- ### Load and Display Input Image Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/positive_pixel_count.ipynb Loads an image from a URL using scikit-image and displays it using Matplotlib. Ensure matplotlib and scikit-image are installed. ```python image_url = ('https://data.kitware.com/api/v1/file/' '598b71ee8d777f7d33e9c1d4/download') # DAB.png im_input = skimage.io.imread(image_url) print('Input image') plt.imshow(im_input) plt.show() ``` -------------------------------- ### Interactive Parameter Tuning with ipywidgets Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/positive_pixel_count.ipynb Sets up interactive sliders for image analysis parameters using `ipywidgets`. This allows for real-time adjustment and visualization of analysis results. Requires `ipywidgets` to be installed and enabled in the environment. ```python import ipywidgets as widgets from ipywidgets import interactive params= { 'hue_value': widgets.FloatSlider(min=0.0,max=1.0,step=0.004,style = {'description_width': 'initial'}, value=0.05), 'hue_width':widgets.FloatSlider(min=0.0,max=1.0,step=0.05,style = {'description_width': 'initial'}, value=0.15), 'saturation_minimum': widgets.FloatSlider(min=0.0,max=1.0,step=0.05,style = {'description_width': 'initial'}, value=0.05), 'intensity_upper_limit': widgets.FloatSlider(min=0.0,max=1.0,step=0.05,style = {'description_width': 'initial'}, value=0.95), 'intensity_weak_threshold': widgets.FloatSlider(min=0.0,max=1.0,step=0.05,style = {'description_width': 'initial'}, value=0.65), 'intensity_strong_threshold':widgets.FloatSlider(min=0.0,max=1.0,step=0.05,style = {'description_width': 'initial'}, value=0.35), 'intensity_lower_limit': widgets.FloatSlider(min=0.0,max=1.0,step=0.05,style = {'description_width': 'initial'}, value=0.05), } def g(**params): count_and_label(ppc.Parameters(**params)) w = interactive(g,**params) display(w) template_params = ppc.Parameters(**w.kwargs) ``` -------------------------------- ### Get Documentation for RGB Perturb Stain Concentration Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/color_normalization_and_augmentation.ipynb Prints the docstring for the rgb_perturb_stain_concentration function, explaining its role as a wrapper for perturbing stain concentrations in RGB images. ```python print(rgb_perturb_stain_concentration.__doc__) ``` -------------------------------- ### Get ROI Documentation (Slide Level) Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/creating_gallery_images_review.ipynb Prints the documentation for the get_all_rois_from_slide_v2 function. This function retrieves ROIs for a specific slide without generating an intermediate mask. ```python print(get_all_rois_from_slide_v2.__doc__) ``` -------------------------------- ### Get Image and Mask - WSI Mode Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_semantic_segmentation_masks.ipynb Generates a miniature version of the entire slide and its annotations. Set 'MPP' to a suitable value (e.g., 5.0) to control the output size. ```python get_kwargs['MPP'] = 5.0 # otherwise it's too large! wsi_out = get_image_and_mask_from_slide( mode='wsi', **get_kwargs) ``` -------------------------------- ### Initialize Cellularity Detector Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/semantic_segmentation_superpixel_approach.ipynb Initializes the Cellularity_detector_superpixels object with specific parameters for cellularity detection. This example uses intensity features from the hematoxylin channel and visualizes superpixels and contiguous regions. ```python # init cellularity detector cds = Cellularity_detector_superpixels( gc, slide_id=SAMPLE_SLIDE_ID, MAG=3.0, compactness=0.1, spixel_size_baseMag=256 * 256, max_cellularity=40, visualize_spixels=True, visualize_contiguous=True, get_tissue_mask_kwargs={ 'deconvolve_first': False, 'n_thresholding_steps': 2, 'sigma': 1.5, 'min_size': 500 }, verbose=2, monitorPrefix='test', logging_savepath=logging_savepath) ``` -------------------------------- ### Build and Install Cython Module with CMake Source: https://github.com/digitalslidearchive/histomicstk/blob/master/histomicstk/segmentation/nuclear/CMakeLists.txt Uses Python::Interpreter to generate C++ code from Cython and compiles it as a Python module. Requires Python, NumPy, and Cython to be available in the environment. ```cmake find_package(Python COMPONENTS Interpreter Development.Module NumPy REQUIRED) add_custom_command( OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/_max_clustering_cython.cpp COMMENT "Making ${CMAKE_CURRENT_BINARY_DIR}/_max_clustering_cython.cpp from ${CMAKE_CURRENT_SOURCE_DIR}/_max_clustering_cython.pyx" COMMAND Python::Interpreter -m cython --cplus -o "${CMAKE_CURRENT_BINARY_DIR}/_max_clustering_cython.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/_max_clustering_cython.pyx" DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/_max_clustering_cython.pyx" VERBATIM) python_add_library(_max_clustering_cython MODULE "${CMAKE_CURRENT_BINARY_DIR}/_max_clustering_cython.cpp" WITH_SOABI) target_link_libraries(_max_clustering_cython PRIVATE Python::Module Python::NumPy) install(TARGETS _max_clustering_cython DESTINATION histomicstk/segmentation/nuclear) ``` -------------------------------- ### Connect to Girder and Set Up Parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/semantic_segmentation_superpixel_approach.ipynb Establishes a connection to the Girder API and authenticates using an API key. Defines the sample slide ID and sets up a temporary directory for logging. Includes color normalization parameters for thumbnail and main image processing. ```python APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/' SAMPLE_SLIDE_ID = '5d586d76bd4404c6b1f286ae' # SAMPLE_SLIDE_ID = "5d8c296cbd4404c6b1fa5572" gc = girder_client.GirderClient(apiUrl=APIURL) gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb') # This is where the run logs will be saved logging_savepath = tempfile.mkdtemp() # color normalization values from TCGA-A2-A3XS-DX1 cnorm_thumbnail = { 'mu': np.array([9.24496373, -0.00966569, 0.01757247]), 'sigma': np.array([0.35686209, 0.02566772, 0.02500282]), } # from the ROI in Amgad et al, 2019 cnorm_main = { 'mu': np.array([8.74108109, -0.12440419, 0.0444982]), 'sigma': np.array([0.6135447, 0.10989545, 0.0286032]), } ``` -------------------------------- ### Initialize Girder client and analysis parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/color_normalization_and_augmentation.ipynb Configures the Girder client connection and defines standard color normalization matrices and visualization settings. ```python APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/' SAMPLE_SLIDE_ID = '5d817f5abd4404c6b1f744bb' gc = girder_client.GirderClient(apiUrl=APIURL) # gc.authenticate(interactive=True) gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb') MAG = 1.0 # color norm. standard (from TCGA-A2-A3XS-DX1, Amgad et al, 2019) cnorm = { 'mu': np.array([8.74108109, -0.12440419, 0.0444982]), 'sigma': np.array([0.6135447, 0.10989545, 0.0286032]), } # TCGA-A2-A3XS-DX1_xmin21421_ymin37486_.png, Amgad et al, 2019) # for macenco (obtained using rgb_separate_stains_macenko_pca() # and reordered such that columns are the order: # Hamtoxylin, Eosin, Null W_target = np.array([ [0.5807549, 0.08314027, 0.08213795], [0.71681094, 0.90081588, 0.41999816], [0.38588316, 0.42616716, -0.90380025], ]) # visualization color map vals = np.random.rand(256, 3) vals[0, ...] = [0.9, 0.9, 0.9] cMap = ListedColormap(1 - vals) # for visualization ymin, ymax, xmin, xmax = 1000, 1500, 2500, 3000 # for reproducibility np.random.seed(0) ``` -------------------------------- ### Configure create_review_galleries Parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/creating_gallery_images_review.ipynb Sets up the keyword arguments for the create_review_galleries function, specifying parameters for visualization assembly and server upload. ```python create_review_galleries_kwargs = { 'tilepath_base': combinedvis_savepath, 'upload_results': True, 'gc': gc, 'url': URL, 'gallery_folderid': POST_FOLDERID, 'gallery_savepath': None, 'padding': 25, 'tiles_per_row': 2, 'tiles_per_column': 5, } ``` -------------------------------- ### Initialize Dask client Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/positive_pixel_count.ipynb Sets up a basic Dask distributed client for parallel computing. ```python # Set up a basic configuration. Change as needed. import dask.distributed dask.distributed.Client() ``` -------------------------------- ### Build Documentation with Sphinx Source: https://github.com/digitalslidearchive/histomicstk/blob/master/CONTRIBUTING.rst Generate documentation locally to check for build errors. ```bash $ cd HistomicsTK $ sphinx-build ./docs/ ~/HistomicsTK_test_build/ 2>&1 | tee out.log ``` -------------------------------- ### Get Docstring for get_all_rois_from_slide Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_semantic_segmentation_masks.ipynb Prints the docstring for the `get_all_rois_from_slide` function to display its documentation. ```python print(get_all_rois_from_slide.__doc__) ``` -------------------------------- ### Get Level for Magnification Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/using_large_image.ipynb Retrieves the slide level corresponding to a specific pixel width in millimeters. ```python print('Level with pixel width closest to 0.0005mm = {}'.format( ts.getLevelForMagnification(mm_x=0.0005))) ``` -------------------------------- ### Workflow_runner initialization documentation Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/workflows.ipynb Displays the documentation for the `Workflow_runner` constructor, outlining its required arguments: `slide_iterator`, `workflow`, and `workflow_kwargs`. These parameters are essential for setting up and executing analysis workflows. ```python print(Workflow_runner.__init__.__doc__) ``` -------------------------------- ### Get Single Tile by Position Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/using_large_image.ipynb Retrieves a specific tile from the slide using its linear position index. ```python pos = 1000 tile_info = ts.getSingleTile( tile_size=dict(width=1000, height=1000), scale=dict(magnification=20), tile_position=pos, ) plt.imshow(tile_info['tile']) ``` -------------------------------- ### Polygon Merger Execution Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/polygon_merger_from_tiled_masks.ipynb Runs the full pipeline to get merged contours from the initialized Polygon Merger object. ```APIDOC ## Polygon_merger.run() ### Description Run full pipeline to get merged contours. ### Returns - pandas DataFrame: has the same structure as output from get_contours_from_mask(). ``` -------------------------------- ### Connect Girder client and configure analysis parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_semantic_segmentation_masks.ipynb Establishes a connection to the Girder API and sets up temporary directories and resolution parameters for image processing. ```python CWD = os.getcwd() APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/' SAMPLE_SLIDE_ID = '5d586d57bd4404c6b1f28640' GTCODE_PATH = os.path.join( CWD, '../../tests/test_files/sample_GTcodes.csv') # connect to girder client gc = girder_client.GirderClient(apiUrl=APIURL) # gc.authenticate(interactive=True) gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb') # just a temp directory to save masks for now BASE_SAVEPATH = tempfile.mkdtemp() SAVEPATHS = { 'ROI': os.path.join(BASE_SAVEPATH, 'masks'), 'rgb': os.path.join(BASE_SAVEPATH, 'rgbs'), 'contours': os.path.join(BASE_SAVEPATH, 'contours'), 'visualization': os.path.join(BASE_SAVEPATH, 'vis'), } for _, savepath in SAVEPATHS.items(): os.mkdir(savepath) # What resolution do we want to get the images at? # Microns-per-pixel / Magnification (either or) MPP = 2.5 # <- this roughly translates to 4x magnification MAG = None ``` -------------------------------- ### Connect to Girder Client and Set Parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/segmentation_masks_to_annotations.ipynb Establishes a connection to the Girder API and authenticates the client. Ensure APIURL and SAMPLE_SLIDE_ID are correctly set for your environment. ```python # APIURL = 'http://demo.kitware.com/histomicstk/api/v1/' # SAMPLE_SLIDE_ID = '5bbdee92e629140048d01b5d' APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/' SAMPLE_SLIDE_ID = '5d586d76bd4404c6b1f286ae' # Connect to girder client gc = girder_client.GirderClient(apiUrl=APIURL) gc.authenticate(interactive=True) # gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb') ``` -------------------------------- ### Get Magnification for All Levels Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/using_large_image.ipynb Iterates through all levels of the image pyramid and prints the magnification associated with each level. Requires a TileSource object `ts`. ```python for i in range(ts.levels): print('Level-{} : {}'.format( i, ts.getMagnificationForLevel(level=i))) ``` -------------------------------- ### CLI parameter placeholders Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/positive_pixel_count.ipynb Commented out code showing how to inspect template parameters and region settings. ```python # pp_namedtuple(template_params) # print('\nRegion:', region) ``` -------------------------------- ### Get Single Annotation Document from Contours Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/segmentation_masks_to_annotations.ipynb Uses the large_image annotation schema to create an annotation document that can be posted to DSA for viewing. ```APIDOC ## GET /api/annotation/document/from_contours ### Description Uses the large_image annotation schema to create an annotation document that can be posted to DSA for viewing. ### Method GET ### Endpoint /api/annotation/document/from_contours ### Parameters #### Query Parameters - **contours_df_slice** (pandas DataFrame) - Required - A slice of the contours dataframe. Must contain 'group', 'color', 'coords_x', and 'coords_y' columns. - **docname** (str) - Required - The name of the annotation document. - **F** (float) - Required - Magnification factor between the mask and the slide scan. - **X_OFFSET** (int) - Required - X offset to add to contours at BASE magnification. - **Y_OFFSET** (int) - Required - Y offset to add to contours at BASE magnification. - **opacity** (float) - Optional - Opacity of annotation elements (in the range [0, 1]). Defaults to 0.7. - **lineWidth** (float) - Optional - Width of borders of annotation elements. Defaults to 3. - **verbose** (bool) - Optional - Print progress to screen? - **monitorPrefix** (str) - Optional - Text to prepend to printed statements. ### Response #### Success Response (200) - **annotation_document** (dict) - A DSA-style annotation document ready to be posted for viewing. #### Response Example ```json { "name": "sample_annotation", "elements": [ { "group": "tumor", "color": "#FF0000", "closed": true, "lines": [ { "x": [100, 200, 150], "y": [50, 100, 120] } ], "type": "polygon" } ] } ``` ``` -------------------------------- ### Get Annotation Documents from Contours Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/segmentation_masks_to_annotations.ipynb Generates formatted annotation documents from contours that can be posted to the DSA server. This function wraps `get_single_annotation_document_from_contours`. ```APIDOC ## GET /api/annotation/documents/from_contours ### Description Generates formatted annotation documents from contours that can be posted to the DSA server. This method parses a dataframe of contours to a list of dictionaries, each of which represents a large_image style annotation. ### Method GET ### Endpoint /api/annotation/documents/from_contours ### Parameters #### Query Parameters - **contours_df** (pandas DataFrame) - Required - Dataframe includes data on contours extracted from input mask. Must contain 'group', 'color', 'coords_x', and 'coords_y' columns. - **separate_docs_by_group** (bool) - Optional - If True, generates one or more annotation documents for each group independently. - **annots_per_doc** (int) - Optional - Maximum number of annotation elements per dictionary. Defaults to 10. - **annprops** (dict) - Optional - Properties of annotation elements. Contains keys 'F', 'X_OFFSET', 'Y_OFFSET', 'opacity', 'lineWidth'. - **docnamePrefix** (str) - Optional - Text to prepend to annotation document name. - **verbose** (bool) - Optional - Print progress to screen? - **monitorPrefix** (str) - Optional - Text to prepend to printed statements. ### Response #### Success Response (200) - **annotation_documents** (list of dicts) - DSA-style annotation documents. #### Response Example ```json { "annotation_documents": [ { "name": "annotation_doc_1", "elements": [ { "group": "tumor", "color": "#FF0000", "closed": true, "lines": [ { "x": [100, 200, 150], "y": [50, 100, 120] } ], "type": "polygon" } ] } ] } ``` ``` -------------------------------- ### Initialize Girder Client and Fetch Slide ID Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/simple_tissue_detection.ipynb Initializes the Girder client with the API URL and authenticates using an API key. It then sets a sample slide ID for fetching the thumbnail. ```python APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/' # SAMPLE_SLIDE_ID = '5d586d57bd4404c6b1f28640' SAMPLE_SLIDE_ID = '5d817f5abd4404c6b1f744bb' gc = girder_client.GirderClient(apiUrl=APIURL) # gc.authenticate(interactive=True) _ = gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb') ``` -------------------------------- ### Connect to Girder API and set backup parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotation_database_backup_and_sql_parser.ipynb Establishes a connection to the remote server and defines the target folder ID and local storage path for the backup. ```python gc = connect_to_api( apiurl='http://candygram.neurology.emory.edu:8080/api/v1/', apikey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb') # This is the girder ID of the folder we would like to backup and parse locally SAMPLE_FOLDER_ID = '5e24c20dddda5f8398695671' # This is where the annotations and sqlite database will be dumped locally savepath = '/home/mtageld/Desktop/tmp/concordance/' ``` -------------------------------- ### Access Polygon_merger initialization documentation Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/polygon_merger_from_tiled_masks.ipynb Retrieve the docstring for the Polygon_merger constructor, detailing required arguments like maskpaths and GTCodes_df. ```python print(Polygon_merger.__init__.__doc__) ``` -------------------------------- ### Import large_image library Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/using_large_image.ipynb Initializes the large_image library for use in the environment. ```python import large_image ``` -------------------------------- ### Get Documentation for Perturb Stain Concentration Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/color_normalization_and_augmentation.ipynb Prints the docstring for the perturb_stain_concentration function, which details its parameters and functionality for augmenting stain concentrations in SDA space. ```python print(perturb_stain_concentration.__doc__) ``` -------------------------------- ### Get Bounding Boxes from Slide Annotations Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_semantic_segmentation_masks.ipynb Retrieves bounding box information for all provided slide annotations. This is often a prerequisite for other mask generation functions. ```python element_infos = get_bboxes_from_slide_annotations(slide_annotations) ``` -------------------------------- ### Import necessary libraries for HistomicsTK Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/semantic_segmentation_color_thresholding_approach.ipynb Imports required libraries for HistomicsTK, including utilities for annotations, cellularity detection, and visualization. Ensure these libraries are installed before use. ```python import tempfile import girder_client import numpy as np from pandas import read_csv from histomicstk.annotations_and_masks.annotation_and_mask_utils import ( delete_annotations_in_slide) from histomicstk.saliency.cellularity_detection_thresholding import ( Cellularity_detector_thresholding) import matplotlib.pylab as plt from matplotlib.colors import ListedColormap %matplotlib inline ``` -------------------------------- ### Simple Backup Case Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotation_database_backup_and_sql_parser.ipynb Demonstrates the simplest case of backing up folder, item, and annotation information as JSON files, replicating the local folder structure. Optionally, folder and item information can be saved as tables in an SQLite database. ```APIDOC ## Simple Backup Case ### Description This is the simplest case to backup the information about the girder folders, items, and annotations as `.json` files, with a folder structure replicated locally as it is in the girder database. The user may also elect to save the folder and item/slide information (but not the annotations) as tables in a SQLite database. ### Tables in SQLite Database: - **folders**: all girder folders contained within the folder that the user wants to backup. This includes an 'absolute girder path' convenience column. The column '\_id' is the unique girder ID. - **items**: all items (slide). The column '\_id' is the unique girder ID, and is linked to the folders table by the 'folderId' column. ### Syntax ```python # Example syntax for simple backup # dump_annotations_locally(gc, folderid, local, save_json=True, save_sqlite=True) ``` ``` -------------------------------- ### Connect to Girder Client and Set Parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_object_segmentation_masks.ipynb Establishes a connection to the Girder API, sets the sample slide ID, and defines paths for saving masks, RGB images, contours, and visualizations. It also sets the desired resolution for image retrieval. ```python CWD = os.getcwd() APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/' SAMPLE_SLIDE_ID = '5d586d57bd4404c6b1f28640' GTCODE_PATH = os.path.join(CWD, '../../tests/test_files/sample_GTcodes.csv') # connect to girder client gc = girder_client.GirderClient(apiUrl=APIURL) # gc.authenticate(interactive=True) gc.authenticate(apiKey='kri19nTIGOkWH01TbzRqfohaaDWb6kPecRqGmemb') # just a temp directory to save masks for now BASE_SAVEPATH = tempfile.mkdtemp() SAVEPATHS = { 'mask': os.path.join(BASE_SAVEPATH, 'masks'), 'rgb': os.path.join(BASE_SAVEPATH, 'rgbs'), 'contours': os.path.join(BASE_SAVEPATH, 'contours'), 'visualization': os.path.join(BASE_SAVEPATH, 'vis'), } for _, savepath in SAVEPATHS.items(): os.mkdir(savepath) # What resolution do we want to get the images at? # Microns-per-pixel / Magnification (either or) MPP = 2.5 # <- this roughly translates to 4x magnification MAG = None ``` -------------------------------- ### Import necessary libraries for HistomicsTK Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/creating_gallery_images_review.ipynb Imports required libraries for image processing, data handling, and Girder client interaction. Ensure these libraries are installed in your environment. ```python import os import tempfile import shutil from imageio import imread from pandas import read_csv import girder_client from histomicstk.annotations_and_masks.review_gallery import \ get_all_rois_from_folder_v2, get_all_rois_from_slide_v2, \ _plot_rapid_review_vis, create_review_galleries import matplotlib.pylab as plt %matplotlib inline ``` -------------------------------- ### Connect to SQLite Database Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotation_database_backup_and_sql_parser.ipynb Establishes a connection to the SQLite database file located at the specified path. This is the initial step before performing any database operations. ```python # Connect to the database sql_engine = db.create_engine( 'sqlite:///%s/Concordance.sqlite' % savepath) dbcon = sql_engine.connect() ``` -------------------------------- ### Create Review Galleries Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/creating_gallery_images_review.ipynb Use this function to create review galleries. Ensure `create_review_galleries_kwargs` are properly defined. ```python resps = create_review_galleries(**create_review_galleries_kwargs) ``` -------------------------------- ### Get Annotations for Slide Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_object_segmentation_masks.ipynb Fetches all annotations for a given slide ID from the Grand Central server. This is the initial step to retrieve annotation data before further processing or scaling. ```python # get annotations for slide slide_annotations = gc.get('/annotation/item/' + SAMPLE_SLIDE_ID) ``` -------------------------------- ### Get Image and Mask - Manual Bounds Mode Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_semantic_segmentation_masks.ipynb Generates an image and mask for a specified region by manually providing bounds. Requires pre-calculated 'slide_annotations' and 'element_infos'. ```python roi_out = get_image_and_mask_from_slide( mode='manual_bounds', slide_annotations=slide_annotations, element_infos=element_infos, **get_kwargs) ``` -------------------------------- ### Print Cellularity Detector Thresholding Init Docstring Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/semantic_segmentation_color_thresholding_approach.ipynb Prints the docstring for the __init__ method of the Cellularity_detector_thresholding class, outlining its arguments and default behaviors. ```python print(Cellularity_detector_thresholding.__init__.__doc__) ``` -------------------------------- ### Import necessary libraries for HistomicsTK workflows Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/workflows.ipynb Imports required modules for Girder client interaction, data handling, and HistomicsTK workflow components. Ensure these libraries are installed. ```python import tempfile import girder_client # import numpy as np from pandas import read_csv from histomicstk.workflows.workflow_runner import Slide_iterator # from histomicstk.saliency.cellularity_detection import ( # Cellularity_detector_superpixels) from histomicstk.saliency.cellularity_detection_thresholding import ( Cellularity_detector_thresholding) from histomicstk.workflows.workflow_runner import ( Workflow_runner, Slide_iterator) from histomicstk.workflows.specific_workflows import ( cellularity_detection_workflow) ``` -------------------------------- ### Initialize HistomicsTK environment Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_semantic_segmentation_masks.ipynb Imports necessary libraries and sets default matplotlib configuration for visualization. ```python import os import girder_client from pandas import DataFrame, read_csv import tempfile from imageio import imread import matplotlib.pyplot as plt %matplotlib inline from histomicstk.annotations_and_masks.annotation_and_mask_utils import ( get_bboxes_from_slide_annotations, scale_slide_annotations, get_scale_factor_and_appendStr) from histomicstk.annotations_and_masks.annotations_to_masks_handler import ( get_image_and_mask_from_slide, get_all_rois_from_slide) #Some nice default configuration for plots plt.rcParams['figure.figsize'] = 7, 7 titlesize = 16 ``` -------------------------------- ### Get Level for Specific Magnification Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/using_large_image.ipynb Finds the image pyramid level whose magnification is closest to a specified value (e.g., 10x). Requires a TileSource object `ts`. ```python # get level whose magnification is closest to 10x print('Level with magnification closest to 10x = {}'.format( ts.getLevelForMagnification(10))) ``` -------------------------------- ### Configure ROI Processing Parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_semantic_segmentation_masks.ipynb Sets up a dictionary of detailed keyword arguments for ROI processing, including MPP, MAG, and specific kwargs for mask and contour generation. ```python detailed_kwargs = { 'MPP': MPP, 'MAG': None, 'get_roi_mask_kwargs': get_roi_mask_kwargs, 'get_contours_kwargs': get_contours_kwargs, 'get_rgb': True, 'get_contours': True, 'get_visualization': True, } ``` -------------------------------- ### Get ROI Bounds Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_object_segmentation_masks.ipynb Retrieves the bounding box coordinates (XMIN, XMAX, YMIN, YMAX) for the specified region of interest. This output is typically generated after processing annotations. ```python roi_out['bounds'] ``` -------------------------------- ### Repeatedly Apply and Visualize Augmentation Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/color_normalization_and_augmentation.ipynb Demonstrates applying the color augmentation multiple times in a loop and visualizing the results after each application. ```python for _ in range(5): augmented_rgb = rgb_perturb_stain_concentration(rgb, mask_out=exclude) vis_augmentation() ``` -------------------------------- ### Get Bounding Box Information Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_object_segmentation_masks.ipynb Retrieves bounding box information for all elements within slide annotations. This is a precursor to processing annotations, especially when dealing with scaled annotations. ```python # get bounding box information for all annotations element_infos = get_bboxes_from_slide_annotations(slide_annotations) ``` -------------------------------- ### Get ROI Keys Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/annotations_to_object_segmentation_masks.ipynb Retrieves the keys from the 'roi_out' dictionary, which typically include 'rgb', 'contours', 'bounds', and 'visualization'. This helps in understanding the available data components after annotation processing. ```python roi_out.keys() ``` -------------------------------- ### Execute PositivePixelCount CLI Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/positive_pixel_count.ipynb Runs the PositivePixelCount script via command line with specified parameters and output file arguments. ```bash # %%script sh # # Stats and label image output must be specified via file arguments to # # --returnparameterfile and --outputLabelImage, respectively. # python ../../server/PositivePixelCount/PositivePixelCount.py \ # 'TCGA-DX-A6BG-01Z-00-DX2.34763958-0613-4069-9ACC-13D6633FE415.svs' \ # 0.05 0.15 0.05 0.95 0.65 0.35 0.05 --region 50000,35000,1600,900 \ # --returnparameterfile stats.txt --outputLabelImage labelImage.png 2>/dev/null ``` -------------------------------- ### Get and Pretty-Print Statistics Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/positive_pixel_count.ipynb Retrieves statistics and the label image from `ppc.count_image` using the original parameter values. It then pretty-prints the statistics using a helper function `pp_namedtuple` for better readability. ```python stats, label_image = ppc.count_image(im_input, template_params) def pp_namedtuple(t): """Pretty-print a namedtuple by printing each field on its own line and left-aligning all values""" print(type(t).__name__) maxlen = max(map(len, t._fields)) for f in t._fields: print(f, getattr(t, f), sep=':' + ' ' * (maxlen - len(f)) + ' ') pp_namedtuple(stats) ``` -------------------------------- ### Download sample whole-slide image Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/using_large_image.ipynb Downloads a sample SVS file from a remote URL if it does not already exist locally. ```python wsi_url = 'https://data.kitware.com/api/v1/file/5899dd6d8d777f07219fcb23/download' wsi_path = 'TCGA-02-0010-01Z-00-DX4.07de2e55-a8fe-40ee-9e98-bcb78050b9f7.svs' if not os.path.isfile(wsi_path): !curl -OJ "$wsi_url" ``` -------------------------------- ### Connect to Girder and set analysis parameters Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/workflows.ipynb Establishes a connection to the Girder API and defines source/destination folder IDs and parameters for cellularity thresholding. An API key is used for authentication. ```python APIURL = 'http://candygram.neurology.emory.edu:8080/api/v1/' SAMPLE_SOURCE_FOLDER_ID = '5d5c28c6bd4404c6b1f3d598' SAMPLE_DESTINATION_FOLDER_ID = '5d9246f6bd4404c6b1faaa89' # girder client gc = girder_client.GirderClient(apiUrl=APIURL) # gc.authenticate(interactive=True) gc.authenticate(apiKey='kri19nTidorWH01TbzRqfohaaDWb6kPecRqGmemb') # This is where the run logs will be saved logging_savepath = tempfile.mkdtemp() # params for cellularity thresholding cdt_params = { 'gc': gc, 'slide_id': '', # this will be handled by the slide iterator 'GTcodes': read_csv('../../histomicstk/saliency/tests/saliency_GTcodes.csv'), 'MAG': 3.0, 'visualize': True, 'verbose': 2, 'logging_savepath': logging_savepath, } ``` -------------------------------- ### POST /create_review_galleries Source: https://github.com/digitalslidearchive/histomicstk/blob/master/docs/examples/creating_gallery_images_review.ipynb Assembles visualization tiles into a gallery image and posts the result to a specified Digital Slide Archive folder. ```APIDOC ## POST /create_review_galleries ### Description Creates a gallery image from visualization tiles and optionally uploads it to a Digital Slide Archive (DSA) instance. ### Parameters #### Request Body - **tilepath_base** (str) - Required - Directory where combined visualizations are stored. - **upload_results** (bool) - Required - Whether to upload results to DSA. - **gc** (girder_client.Girder_Client) - Optional - Authenticated girder client (required if upload_results is True). - **gallery_savepath** (str) - Optional - Directory to save gallery locally (required if upload_results is True). - **gallery_folderid** (str) - Optional - Girder ID of folder to post galleries (required if upload_results is True). - **padding** (int) - Optional - Padding in pixels between tiles in the gallery. - **tiles_per_row** (int) - Optional - Number of visualization tiles per row. - **tiles_per_column** (int) - Optional - Number of visualization tiles per column. - **annprops** (dict) - Optional - Properties of annotations to be posted to DSA. - **url** (str) - Optional - URL of the Digital Slide Archive instance. ### Response #### Success Response (200) - **list** (list) - A list of dictionaries, where each entry represents the server response for the gallery upload request. ```