### Install SCALLOPS via PyPI Source: https://github.com/genentech/scallops/blob/main/docs/install.md Installs the latest stable release of the SCALLOPS package from the Python Package Index using pip. ```bash pip install scallops ``` -------------------------------- ### Clone and Setup SCALLOPS Development Environment Source: https://github.com/genentech/scallops/blob/main/docs/install.md Provides commands to clone the SCALLOPS repository, create a dedicated Python 3.12 environment using mamba, and install the package in editable mode with dependencies. ```bash git clone https://github.com/Genentech/scallops.git cd scallops mamba create --name scallops python=3.12 mamba activate scallops pip install -r requirements.txt -e . ``` -------------------------------- ### scallops.datasets.example_feature_summary_stats() Source: https://github.com/genentech/scallops/blob/main/docs/scallops.datasets.example_feature_summary_stats.md Retrieves example phenotype feature summary statistics. This function returns the path to a Parquet file containing the statistics. ```APIDOC ## GET /genentech/scallops/datasets/example_feature_summary_stats ### Description Retrieves example phenotype feature summary statistics. This function returns the path to a Parquet file containing the statistics. ### Method GET ### Endpoint /genentech/scallops/datasets/example_feature_summary_stats ### Parameters #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) - **return** (Path) - Path to Parquet file containing the summary statistics. #### Response Example { "return": "/path/to/your/scallops_example_feature_summary_stats.parquet" } ``` -------------------------------- ### Install SCALLOPS with custom temporary directory Source: https://github.com/genentech/scallops/blob/main/docs/faq.md Resolves ImportError issues during installation by setting the TMPDIR environment variable to the current directory, ensuring the installer has necessary execution permissions. ```bash TMPDIR=. pip install scallops ``` -------------------------------- ### Create Multi-Scene Hyperstack with Python Source: https://github.com/genentech/scallops/blob/main/docs/scallops.io.create_multiscene_hyperstack.md This example demonstrates how to initialize multiple xarray DataArrays representing image scenes and save them as a multi-scene hyperstack using the scallops.io library. It requires the xarray and numpy libraries to prepare the input data dictionary. ```python import scallops.io import xarray as xr import numpy as np # Create example image scenes scene1 = xr.DataArray( np.random.rand(1, 3, 10, 512, 512), dims=("c", "t", "z", "y", "x") ) scene2 = xr.DataArray( np.random.rand(1, 3, 10, 512, 512), dims=("c", "t", "z", "y", "x") ) # Create a dictionary of image scenes images = {"Scene1": scene1, "Scene2": scene2} # Define output URI output_uri = "path/to/output/multiscene_hyperstack.tiff" # Create the multi-scene hyperstack image scallops.io.create_multiscene_hyperstack(output_uri, images) ``` -------------------------------- ### Write data to Zarr group using scallops Source: https://github.com/genentech/scallops/blob/main/docs/scallops.zarr_io.write_zarr.md Demonstrates how to initialize an OME-Zarr group and write a NumPy array to it using the write_zarr utility. This example includes setting image attributes and defining dimensions for the target dataset. ```python import numpy as np import dask.array as da import zarr from scallops.zarr_io import write_zarr, open_ome_zarr # Create a Zarr group root = open_ome_zarr("example.zarr") grp = root.create_group("test_group") # Write data to the Zarr group write_zarr( grp, np.random.rand(100, 100), image_attrs={"description": "Test data"}, coords=None, dims=["y", "x"], ) ``` -------------------------------- ### Visualize image montages with montage_plot Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.composite.montage_plot.md Demonstrates how to use the montage_plot function to visualize an XArray image dataset. The example shows the setup of synthetic data and the application of percentile-based thresholds and figure sizing. ```python import numpy as np import xarray as xr import matplotlib.pyplot as plt from scallops.visualize import montage_plot # Generate synthetic data for testing image_data = np.random.rand(1, 3, 1, 100, 100) image = xr.DataArray(image_data, dims=("t", "c", "z", "y", "x")) # Plot the image montage montage_plot( image.isel(t=0, z=0), percentile_min=0, percentile_max=1, pad_min=0.01, pad_max=0.99, figsize=(12, 8) ) plt.show() ``` -------------------------------- ### Visualize 2D image array with imshow_plane Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.imshow.imshow_plane.md This example demonstrates how to use imshow_plane to visualize a synthetic 2D numpy array. It shows the initialization of a matplotlib figure and the application of custom colormaps and titles. ```python import numpy as np import matplotlib.pyplot as plt from scallops.visualize.imshow import imshow_plane # Generate a synthetic 2D image array image_array = np.random.rand(512, 512) # Plot the image using imshow_plane fig, ax = plt.subplots() imshow_plane(image_array, ax=ax, title="Synthetic Image", cmap="viridis") plt.show() ``` -------------------------------- ### Visualize image registration with diagnose_registration Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.registration.diagnose_registration.md This example demonstrates how to use diagnose_registration to overlay images from an xarray DataArray. It requires defining selectors for the specific time and channel dimensions to be compared. ```python import numpy as np import xarray as xr import matplotlib.pyplot as plt from scallops.visualize.registration import diagnose_registration # Create synthetic DataArray image_shape = (3, 4, 1, 100, 100) # (t, c, z, y, x) imagestack = xr.DataArray( np.random.rand(*image_shape), dims=("t", "c", "z", "y", "x") ) # Define selectors selectors = [ {"t": 0, "c": 0}, {"t": 1, "c": 1}, {"t": 2, "c": 2}, ] # Plot the registration diagnosis axes = diagnose_registration( imagestack.squeeze(), *selectors, title="Registration Diagnosis" ) plt.show() ``` -------------------------------- ### GET scallops.visualize.distribution.cdf_plot Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.distribution.cdf_plot.md Creates a Cumulative Distribution Function (CDF) plot to compare experimental conditions against a reference group. ```APIDOC ## GET scallops.visualize.distribution.cdf_plot ### Description A utility function to create Cumulative Distribution Function (CDF) plots comparing a set of conditions against a reference condition. The CDF plots are shaded to highlight the differences between the reference and target groups. ### Method GET ### Endpoint scallops.visualize.distribution.cdf_plot ### Parameters #### Request Body (Function Arguments) - **df** (DataFrame) - Required - DataFrame containing the experimental data. - **feature** (str) - Required - Column representing the feature for CDF plotting. - **targets** (Sequence[str] | None) - Optional - Target values in groupby_column to plot. - **groupby_column** (str) - Optional - Column used for grouping the data (default: 'gene_symbol_0'). - **reference_group** (str | None) - Optional - Reference group for comparison (default: 'NTC'). - **line_width** (int | None) - Optional - Width of the CDF plot lines. - **col** (str | None) - Optional - Variable that defines subsets to plot on different columns. - **col_order** (Sequence[Any] | None) - Optional - Specify the order column order for categorical levels of col. - **hue** (str | None) - Optional - Variable used to color CDF plots. - **reference_color** (str) - Optional - Color for the reference group (default: 'grey'). - **shade** (bool | None) - Optional - Shade the area between target and reference. - **height** (int) - Optional - Figure height (default: 8). - **include_n** (bool) - Optional - Include the sample size per target in legend (default: True). ### Request Example { "df": "", "feature": "intensity", "groupby_column": "gene", "reference_group": "NTC" } ### Response #### Success Response (200) - **axes** (Sequence[Axes]) - The matplotlib axes object containing the plot. #### Response Example { "axes": "" } ``` -------------------------------- ### Generate aggregated well heatmaps with plot_well_aggregated_heatmaps Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.heatmap.plot_well_aggregated_heatmaps.md Demonstrates how to prepare a DataFrame and use the plot_well_aggregated_heatmaps function to visualize spatial data. The example includes defining an aggregation function and rendering the resulting Matplotlib figure. ```python import numpy as np import pandas as pd import matplotlib.pyplot as plt # Example DataFrame df = pd.DataFrame( { "well": np.random.randint(1, 7, size=1000), "x": np.random.rand(1000) * 10000, "y": np.random.rand(1000) * 10000, "value": np.random.rand(1000), } ) # Function to aggregate values def mean_value(group): return group["value"].mean() # Generate heatmaps fig = plot_well_aggregated_heatmaps(df, mean_value) # Display the figure plt.show() ``` -------------------------------- ### Define Custom Registration Workflow (WDL) Source: https://github.com/genentech/scallops/blob/main/docs/workflows.md An example of a custom WDL file that imports Scallops tasks to perform only image registration. It defines a workflow that takes input images and an output directory, then calls the `register_elastix` task from the Scallops library. ```wdl version 1.0 # Import the existing Scallops tasks import "scallops/wdl/ops_tasks.wdl" as tasks workflow my_custom_registration { input { String moving_image String fixed_image String output_dir String docker } # Call the existing Scallops registration task call tasks.register_elastix { input: moving = [moving_image], fixed = fixed_image, transform_output_directory = output_dir + "/transforms", moving_output_directory = output_dir + "/registered_images", # Pass through required runtime parameters docker = docker, cpu = 4, memory = "16 GiB", # ... (other required inputs like zones, disks, etc.) } } ``` -------------------------------- ### Generate a ridge plot using scallops.visualize.ridge_plot Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.distribution.ridge_plot.md This example demonstrates how to use the ridge_plot function to visualize a numeric variable grouped by categorical features. It requires a pandas DataFrame and utilizes matplotlib for rendering the final figure. ```python import pandas as pd import seaborn as sns import numpy as np import matplotlib.pyplot as plt from scallops.visualize import ridge_plot # Create a sample DataFrame np.random.seed(42) data = pd.DataFrame( { "category": np.random.choice(["A", "B", "C"], size=300), "group": np.random.choice(["X", "Y"], size=300), "value": np.random.normal(size=300), } ) # Generate a ridge plot ridge_plot( data=data, feature="value", row="category", col="group", scale="standard", title="Ridge Plot Example", ) # Show the plot plt.show() ``` -------------------------------- ### Visualize Plate Heatmap with scallops Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.heatmap.plate_heatmap.md This example demonstrates how to use the plate_heatmap function to visualize data grouped by well and tile. It requires a pandas DataFrame indexed by well and tile, and returns a Matplotlib Figure object. ```python import pandas as pd import numpy as np from itertools import product from scallops.visualize import plate_heatmap # Create a sample DataFrame np.random.seed(42) wells = ["A1", "A2", "B1", "B2"] tiles = list(range(16)) w, t = zip(*product(wells, tiles)) data = pd.DataFrame( { "well": w, "tile": t, "value": np.random.normal(size=len(w)), } ).set_index(["well", "tile"]) # Generate a heatmap plate_heatmap(data, column="value", tile_shape=(4, 4), cmap="viridis") ``` -------------------------------- ### Custom Pairwise Scatter Plot with Base Call Coloring Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.crosstalk.pairwise_channel_scatter_plot.md This example shows how to create a customized pairwise channel scatter plot where points are colored based on their base calls. It defines a custom `plot_func` that uses `matplotlib.pyplot.scatter` to color points according to a provided mapping of bases to colors. This allows for visualizing how base calls correlate with channel intensities. ```python import xarray as xr import matplotlib.pyplot as plt import matplotlib.patches as mpatches from scallops.visualize import pairwise_channel_scatter_plot import pandas as pd import numpy as np # Create a sample DataArray channels = ["ChannelA", "ChannelT", "ChannelG", "ChannelC"] data = xr.DataArray( np.random.rand(100, 10, len(channels)), dims=("read", "t", "c"), coords={"c": channels}, ) bases = ["G", "T", "A", "C"] base_calls = pd.Series(np.random.choice(bases, size=100)) base_colors = {"G": "green", "T": "red", "A": "magenta", "C": "cyan"} def plot_func(x, y, i, j, ax, **plot_args): ax.scatter( x, y, c=base_calls.apply(lambda x: base_colors[x]), **plot_args ) fig, axes = pairwise_channel_scatter_plot( data.isel(t=0), plot_func=plot_func ) patches = [ mpatches.Patch(color=color, label=label) for label, color in base_colors.items() ] plt.legend(handles=patches) plt.show() ``` -------------------------------- ### Download and stage deep learning models Source: https://github.com/genentech/scallops/blob/main/docs/faq.md Demonstrates how to manually download model files and upload them to an accessible storage location like S3. This is required for environments that restrict direct internet access for model fetching. ```bash wget https://github.com/stardist/stardist-models/releases/download/v0.1/python_2D_versatile_fluo.zip aws s3 cp python_2D_versatile_fluo.zip s3://my-bucket/model/ ``` -------------------------------- ### GET /scallops/segmentation/util/area_overlap Source: https://github.com/genentech/scallops/blob/main/docs/scallops.segmentation.util.area_overlap.md Calculates the overlap between two sets of labels, typically used for comparing nuclei and cell segmentation masks. ```APIDOC ## GET /scallops/segmentation/util/area_overlap ### Description Find labels in y that overlap with labels in x. This function is used to quantify the spatial relationship between two label arrays. ### Method GET ### Endpoint /scallops/segmentation/util/area_overlap ### Parameters #### Query Parameters - **x** (Array) - Required - Reference labels, typically nuclei. - **y** (Array) - Required - Query labels, typically cells. ### Request Example { "x": [1, 0, 1], "y": [0, 2, 2] } ### Response #### Success Response (200) - **x** (int) - Reference label ID - **y** (int) - Query label ID - **area** (float) - Overlapping area - **overlap** (float) - Overlap ratio #### Response Example { "data": [ {"x": 1, "y": 2, "area": 150.5, "overlap": 0.85} ] } ``` -------------------------------- ### View Experiment in Napari Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.napari.experiment_napari.md This Python code snippet demonstrates how to load an experiment using `scallops.io.read_experiment` and then visualize it in Napari using the `expnapari` function. It assumes the existence of an experiment file at the specified path. ```python from scallops.visualize.napari import expnapari import scallops # Load an exp exp = scallops.io.read_experiment("path/to/exp") # View the exp in Napari napari_viewer = expnapari(exp) ``` -------------------------------- ### GET /scallops/io/pixel-positions Source: https://github.com/genentech/scallops/blob/main/docs/scallops.io.get_pixel_positions.md Retrieves stage pixel positions from the metadata of an image file located at the specified path. ```APIDOC ## GET /scallops/io/pixel-positions ### Description Extracts stage pixel positions from the metadata of an image file. This function is useful for mapping image coordinates to physical stage positions. ### Method GET ### Endpoint /scallops/io/pixel-positions ### Parameters #### Query Parameters - **path** (str) - Required - The file system path to the image file (e.g., .nd2). ### Request Example GET /scallops/io/pixel-positions?path=path/to/image.nd2 ### Response #### Success Response (200) - **positions** (ndarray) - An array containing the extracted pixel positions. #### Response Example { "positions": [[0, 0], [10, 20], [50, 100]] } ``` -------------------------------- ### Manage Experiment objects Source: https://context7.com/genentech/scallops/llms.txt Demonstrates creating an Experiment object with images and labels, and saving the resulting structure to Zarr format for persistent storage. ```python from scallops.experiment.elements import Experiment import xarray as xr import numpy as np # Create an experiment with images and labels images = { "A1-102": xr.DataArray(np.random.rand(5, 4, 1, 1024, 1024), dims=("t", "c", "z", "y", "x")), "A1-103": xr.DataArray(np.random.rand(5, 4, 1, 1024, 1024), dims=("t", "c", "z", "y", "x")) } labels = { "nuclei": np.random.randint(0, 100, (1024, 1024), dtype=np.uint16), "cells": np.random.randint(0, 100, (1024, 1024), dtype=np.uint16) } experiment = Experiment(images=images, labels=labels) print(experiment) # Experiment with 2 images and 2 labels # Save experiment to Zarr format experiment.save("output/experiment.zarr") ``` -------------------------------- ### SCALLOPS pooled-sbs spot-detect Usage Source: https://github.com/genentech/scallops/blob/main/docs/command_line.md Details the command-line usage for the 'spot-detect' sub-command of scallops pooled-sbs. This command is used for identifying sequencing spots in pooled SBS images and specifies arguments for input images, output paths, channel configuration, filtering, and saving options. ```bash scallops pooled-sbs spot-detect [-h] -i IMAGES [IMAGES ...] -o OUTPUT -c CHANNELS [CHANNELS ...] [--image-pattern IMAGE_PATTERN] [--max-filter-width MAX_FILTER_WIDTH] [--sigma-log SIGMA_LOG [SIGMA_LOG ...]] [--peak-neighborhood-size PEAK_NEIGHBORHOOD_SIZE] [--cycles CYCLES [CYCLES ...]] [--chunks CHUNKS] [--z-index Z_INDEX] [-g [GROUPBY ...]] [-s [SUBSET ...]] [--force] [--client CLIENT] [--dask-cluster DASK_CLUSTER] [--save [{log,std} ...]] [--expected-cycles EXPECTED_CYCLES] [--verbose] [--no-version] ``` -------------------------------- ### SCALLOPS pooled-sbs Usage Source: https://github.com/genentech/scallops/blob/main/docs/command_line.md Shows the main command-line usage for the scallops pooled-sbs command, which is a pipeline for processing pooled in-situ sequencing (SBS) images. It lists the available sub-commands: spot-detect, reads, and merge. ```bash usage: scallops pooled-sbs [-h] {spot-detect,reads,merge} ... ``` -------------------------------- ### GET scallops.io.read_in_situ_reads_and_barcodes Source: https://github.com/genentech/scallops/blob/main/docs/scallops.io.read_in_situ_reads_and_barcodes.md Loads reads and barcode information from specified directories and files, matching them to produce a combined DataFrame. ```APIDOC ## GET scallops.io.read_in_situ_reads_and_barcodes ### Description This function loads reads and barcode information from the specified directories and files. It matches reads with barcodes and adds a 'barcode_match' column to the resulting reads DataFrame. ### Method GET (Function Call) ### Endpoint scallops.io.read_in_situ_reads_and_barcodes(reads_directory, barcodes_path, barcode_indices, reads_pattern) ### Parameters #### Path Parameters - **reads_directory** (str) - Required - Path to the directory containing reads information. - **barcodes_path** (str) - Required - Path to the file containing barcodes information. - **barcode_indices** (Sequence[int]) - Optional - List of indices (0-based) to extract from the barcode column. - **reads_pattern** (str) - Optional - Pattern to filter read paths. ### Request Example ```python import scallops.io reads_directory = "path/to/reads/directory" barcodes_path = "path/to/barcodes/file.csv" barcode_indices = [0, 1, 2] reads_pattern = "*.csv" reads, barcodes = scallops.io.read_in_situ_reads_and_barcodes( reads_directory, barcodes_path, barcode_indices, reads_pattern ) ``` ### Response #### Success Response (200) - **reads** (DataFrame) - The loaded reads data with barcode matches. - **barcodes** (DataFrame) - The loaded barcodes data. #### Response Example ```python # Returns a tuple of (DataFrame, DataFrame) (reads_df, barcodes_df) ``` ``` -------------------------------- ### Visualize images in Napari using imnapari Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.napari.imnapari.md This function initializes or updates a Napari viewer instance with provided image data, labels, and optional point markers. It accepts xarray DataArrays or dictionaries of images and returns a napari.Viewer instance for interactive exploration. ```python import xarray as xr import numpy as np from napari import Viewer from scallops.visualize.napari import imnapari # Create a synthetic image width = 512 height = 512 channels = 3 data = np.random.randint( 0, 255, size=(channels, height, width), dtype=np.uint8 ) coords = { "c": np.arange(channels), "y": np.arange(height), "x": np.arange(width), } synthetic_image = xr.DataArray(data, coords=coords, dims=("c", "y", "x")) # Create a Napari viewer viewer = Viewer() # View the synthetic image in Napari imnapari(synthetic_image, title_attribute="Synthetic Image", viewer=viewer) # Run the Napari event loop viewer.show() ``` -------------------------------- ### SCALLOPS illum-corr Usage Source: https://github.com/genentech/scallops/blob/main/docs/command_line.md Displays the command-line usage for the scallops illum-corr command, which performs illumination correction on images. It outlines the required and optional arguments for aggregation methods, output formats, and filtering. ```bash usage: scallops illum-corr agg [-h] -i IMAGES [IMAGES ...] -o OUTPUT [-g [GROUPBY ...]] [-s [SUBSET ...]] [--image-pattern IMAGE_PATTERN] [--smooth SMOOTH] [--agg-method {mean,median,min}] [--no-rescale] [--output-image-format {tiff,zarr}] [--z-index Z_INDEX] [--channel CHANNEL] [--force] [--verbose] [--expected-images EXPECTED_IMAGES] [--no-version] [--client CLIENT] [--dask-cluster DASK_CLUSTER] ``` -------------------------------- ### Load Barcodes from CSV using Python Source: https://github.com/genentech/scallops/blob/main/docs/scallops.io.read_barcodes.md This Python function loads barcodes from a CSV file. It can optionally select specific indices from the barcode column and validates that all barcodes have the same length. It requires the 'scallops' library to be installed. ```python import scallops.io # Define the path to the barcodes CSV file barcodes_path = "path/to/barcodes.csv" # Load barcodes without selecting specific indices barcodes_df = scallops.io.read_barcodes(barcodes_path) print(barcodes_df.head()) # Load barcodes and select specific indices selected_indices = [0, 1, 2] barcodes_df_selected = scallops.io.read_barcodes( barcodes_path, barcode_indices=selected_indices ) print(barcodes_df_selected.head()) ``` -------------------------------- ### Run In-Situ Sequencing Pipeline via CLI Source: https://github.com/genentech/scallops/blob/main/docs/outputs.md Executes the pooled-sbs sequencing pipeline using the Scallops CLI. It requires input data paths, a barcode CSV file, and phenotype data as arguments. ```bash scallops pooled-sbs pipeline scallops/tests/data/experimentC/input --barcodes scallops/tests/data/experimentC/barcodes.csv --pheno=scallops/tests/data/experimentC/10X_c0-DAPI-p65ab ``` -------------------------------- ### Visualization Utilities Source: https://github.com/genentech/scallops/blob/main/docs/api.md Helper functions for contrast adjustment and axis labeling. ```APIDOC ## GET scallops.visualize.utils.channel_thresholds ### Description Compute thresholds per channel for contrast adjustment in visualization. ### Method GET ### Parameters #### Query Parameters - **image** (array) - Required - Input image data. ### Response #### Success Response (200) - **thresholds** (list) - Computed thresholds for each channel. ``` -------------------------------- ### Napari Visualization API Source: https://github.com/genentech/scallops/blob/main/docs/api.md Integrates with Napari for interactive visualization of images and experimental data, including adding bases and performing radial distortion estimation. ```APIDOC ## Napari Visualization Functions ### Description This section details functions for interactive visualization using Napari. ### Functions - **scallops.visualize.napari.imnapari** - **Description**: View image in Napari. - **Parameters**: - `image` (Image) - Required - The image data to view. - **Method**: Not applicable (Python function) - **Endpoint**: Not applicable (Python function) - **scallops.visualize.napari.experiment_napari** - **Description**: View an experiment in Napari. - **Method**: Not applicable (Python function) - **Endpoint**: Not applicable (Python function) - **scallops.visualize.napari.add_bases** - **Description**: Add bases to Napari viewer. - **Parameters**: - `viewer` (NapariViewer) - Required - The Napari viewer object. - **Method**: Not applicable (Python function) - **Endpoint**: Not applicable (Python function) - **scallops.visualize.napari.radial_distortion_estimation** - **Description**: Run radial distortion estimation. - **Method**: Not applicable (Python function) - **Endpoint**: Not applicable (Python function) ``` -------------------------------- ### Read experiments with read_experiment Source: https://context7.com/genentech/scallops/llms.txt Reads entire experiments from directories or Zarr files, allowing for pattern-based organization of images by wells, tiles, and channels. ```python import scallops.io # Read experiment from a directory with pattern matching experiment = scallops.io.read_experiment( image_path="path/to/images/", pattern="{well}_Tile-{tile}_Channel{channel}.tif", groupby=["well", "tile"] ) # Access images in the experiment for key in experiment.images: image = experiment.images[key] print(f"Image {key}: shape={image.shape}") # Read from Zarr format experiment = scallops.io.read_experiment("path/to/experiment.zarr") # Read with subset filtering experiment = scallops.io.read_experiment( image_path="s3://bucket/images/", pattern="Well{well}_Point{tile}_Channel{channel}.nd2", subset=["A1-*", "B2-*"] ) ``` -------------------------------- ### Run radial distortion estimation in Napari Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.napari.radial_distortion_estimation.md Initializes a Napari viewer to display four adjacent images and provides an interactive slider to estimate radial distortion parameters. The function requires paths to image files and an overlap proportion to correctly position the images. ```python from scallops.visualize.napari import radial_distortion_estimation radial_distortion_estimation( top_left="s3://your-bucket/path/to/image_top_left.nd2", top_right="s3://your-bucket/path/to/image_top_right.nd2", bottom_left="s3://your-bucket/path/to/image_bottom_left.nd2", bottom_right="s3://your-bucket/path/to/image_bottom_right.nd2", channel=0, proportion_overlap=0.05 ) ``` -------------------------------- ### Get Pixel Positions from Image Metadata (Python) Source: https://github.com/genentech/scallops/blob/main/docs/scallops.io.get_pixel_positions.md This Python function retrieves stage pixel positions from the metadata of an image file. It takes the file path as input and returns an ndarray of pixel positions. Ensure the image file contains the necessary metadata. ```python import scallops.io import numpy as np # Define the path to the image file image_path = "path/to/image.nd2" # Get pixel positions from the image positions = scallops.io.get_pixel_positions(image_path) print(positions) ``` -------------------------------- ### POST /scallops.io.to_label_crops Source: https://github.com/genentech/scallops/blob/main/docs/scallops.io.to_label_crops.md Exports individual label crops as tiff or npy files based on provided intensity and label images. ```APIDOC ## POST /scallops.io.to_label_crops ### Description Export individual label crops as tiff or npy files. If a label image is supplied, the image is masked by the corresponding segmentation mask. ### Method POST ### Endpoint /scallops.io.to_label_crops ### Parameters #### Request Body - **intensity_image** (Array) - Required - Image data - **label_image** (Array) - Optional - Label data for masking - **objects_df** (DataFrame) - Required - DataFrame containing objects from find-objects - **output_dir** (str) - Required - Crop output directory - **crop_size** (tuple[int, int]) - Optional - Size of label crop (default: (224, 224)) - **output_format** (Literal['tiff', 'npy']) - Optional - Crop output format (default: 'tiff') - **centroid_cols** (Sequence[str]) - Optional - Columns in objects_df containing y and x centroids - **gaussian_sigma** (float) - Optional - If provided, applies gaussian-smoothed mask to isolate target mask ### Request Example { "intensity_image": "[array_data]", "label_image": "[array_data]", "objects_df": "[dataframe_data]", "output_dir": "/path/to/output", "crop_size": [224, 224], "output_format": "tiff" } ### Response #### Success Response (200) - **objects_df** (DataFrame) - Objects dataframe with objects at well edges removed #### Response Example { "status": "success", "data": "[processed_dataframe]" } ``` -------------------------------- ### Generate a Volcano Plot using scallops Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.distribution.volcano_plot.md This example demonstrates how to prepare a pandas DataFrame with effect sizes and FDR values, and subsequently generate a volcano plot using the volcano_plot function. It includes custom configuration for vertical and horizontal significance lines and legend labels. ```python import pandas as pd import numpy as np import matplotlib.pyplot as plt from scallops.visualize import volcano_plot # Generate sample data np.random.seed(42) # Creating a DataFrame with 200 genes num_genes = 200 data = { "∆ AUC": np.random.uniform(-0.5, 0.5, num_genes), "FDR-BH pval": np.random.uniform(0.05, 1, num_genes), } # Marking two genes as down-regulated and two as up-regulated data["∆ AUC"][:2] = np.random.uniform(-2, -1, 2) # Down-regulated data["FDR-BH pval"][:2] = np.random.uniform(0, 0.01, 2) data["∆ AUC"][-2:] = np.random.uniform(1, 2, 2) # Up-regulated data["FDR-BH pval"][-2:] = np.random.uniform(0, 0.01, 2) df = pd.DataFrame(data) df["-log2FDR"] = df["FDR-BH pval"].apply(lambda x: -np.log(x)) # Generate volcano plot volcano_plot( df, effect_size_col="∆ AUC", ycol="-log2FDR", fdr_col="FDR-BH pval", vbar_std=2, hbar_value=np.log10(-np.log2(0.05)), legend=("Down-regulated", "Up-regulated"), ) ``` -------------------------------- ### Generate Pairwise Channel Scatter Plot with Scallops Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.crosstalk.pairwise_channel_scatter_plot.md This example demonstrates how to generate a basic pairwise channel intensity scatter plot using the `pairwise_channel_scatter_plot` function. It takes an xarray DataArray as input and visualizes intensity values across different channels. The plot can be customized with parameters like `q` for quantile fitting and `col_wrap` for layout. ```python import xarray as xr import matplotlib.pyplot as plt from scallops.visualize import pairwise_channel_scatter_plot import numpy as np # Create a sample DataArray channels = ["ChannelA", "ChannelT", "ChannelG", "ChannelC"] data = xr.DataArray( np.random.rand(100, 10, len(channels)), dims=("read", "t", "c"), coords={"c": channels}, ) # Generate a pairwise channel scatter plot fig, axes = pairwise_channel_scatter_plot( data, q=(0.1, 0.9), col_wrap=2 ) # Show the plot plt.show() ``` -------------------------------- ### Complete Pooled SBS Analysis Pipeline Source: https://context7.com/genentech/scallops/llms.txt Executes the full pooled in-situ sequencing (SBS) analysis pipeline in three steps: spot detection, read calling with crosstalk correction, and merging SBS data with phenotype data. Requires input images, segmentation labels, and barcode information. ```bash # Step 1: Spot detection scallops pooled-sbs spot-detect \ --images "path/to/sbs_images.zarr" \ --output "output/spots/" \ --channels 1 2 3 4 \ --cycles 1 2 3 4 5 6 7 8 9 \ --groupby plate well # Step 2: Read calling with crosstalk correction scallops pooled-sbs reads \ --spots "output/spots/" \ --labels "output/segment.zarr" \ --label-name cell \ --barcodes "barcodes.csv" \ --output "output/reads/" \ --crosstalk-correction-method li_and_speed \ --mismatches 1 # Step 3: Merge SBS with phenotype data scallops pooled-sbs merge \ --sbs "output/reads/labels" \ --phenotype "output/features-nuclei" "output/features-cell" \ --barcodes "barcodes.csv" \ --output "output/merged/" ``` -------------------------------- ### Stitching Utilities Source: https://github.com/genentech/scallops/blob/main/docs/api.md Utilities for handling tile stitching operations. ```APIDOC ## GET scallops.stitch.utils.tile_overlap_mask ### Description Create a binary mask where zeros indicate locations where tiles overlap. ### Method GET ### Parameters #### Query Parameters - **df** (DataFrame) - Required - Dataframe containing tile coordinates. ### Response #### Success Response (200) - **mask** (array) - Binary mask representing overlap regions. ``` -------------------------------- ### Visualize image composites with imcomposite Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.composite.imcomposite.md This snippet demonstrates how to use imcomposite to render a composite image from XArray data. It includes generating synthetic data and applying the function with specific dimensions and figure settings. ```python from matplotlib import pyplot as plt import numpy as np import xarray as xr from scallops.visualize import imcomposite # Generate synthetic data for testing t, c, z, y, x = (1, 3, 1, 10, 10) image_data = np.random.rand(t, c, z, y, x) labels_data = np.random.randint(0, 2, size=(y, x)) image = xr.DataArray(image_data, dims=("t", "c", "z", "y", "x")) labels = xr.DataArray(labels_data, dims=("y", "x")) # Plot the composite image ax = imcomposite(image.isel(t=0, z=0), labels, figsize=(8, 8), dim="c") plt.show() ``` -------------------------------- ### Stitch images using SCALLOPS CLI Source: https://github.com/genentech/scallops/blob/main/docs/faq.md Configures the image stitching process by defining a file pattern that includes well, tile, channel, and z-index placeholders. This command processes images from a source directory and outputs a Zarr file and report. ```bash scallops stitch --image-pattern "e100_1_t0_{well}_s{tile}_w{c}_z{z}.tif" \ --images my-images/ --groupby well --image-output stitch.zarr --report-output reports/ ``` -------------------------------- ### Execute CLI Illumination Correction Source: https://context7.com/genentech/scallops/llms.txt Use the command-line interface to calculate flat-field correction images and apply them during the stitching process to normalize illumination across tiles. ```bash scallops illum-corr agg \ --images "path/to/images/" \ --image-pattern "Well{well}_Tile{tile}_Channel{channel}.tif" \ --output "output/illumination/" \ --groupby well channel \ --agg-method mean \ --smooth 50 scallops stitch \ --images "path/to/images/" \ --ffp "output/illumination/{well}-{channel}.tiff" \ --output "output/stitched/" ``` -------------------------------- ### POST scallops.visualize.composite.imcomposite Source: https://github.com/genentech/scallops/blob/main/docs/scallops.visualize.composite.imcomposite.md Plots an image composite using additive blending, supporting various colormaps, label overlays, and contour rendering. ```APIDOC ## POST scallops.visualize.composite.imcomposite ### Description Plot an image composite using additive blending. This function allows for the visualization of multi-dimensional image arrays with optional segmentation labels and contour overlays. ### Method POST ### Endpoint scallops.visualize.composite.imcomposite ### Parameters #### Request Body - **image** (DataArray | ndarray | Array) - Optional - XArray with dimensions y, x and optionally dim. - **labels** (DataArray | ndarray | Array) - Optional - Segmentation labels to overlay. - **vmin** (float | DataArray | ndarray | Array) - Optional - Lower color limit for colormap bounds. - **vmax** (float | DataArray | ndarray | Array) - Optional - Upper color limit for colormap bounds. - **cmap** (Sequence | str | Colormap) - Optional - Colormap mapping for dimension index. - **labels_cmap** (str | ListedColormap) - Optional - Colormap for label data. - **labels_alpha** (float) - Optional - Alpha value for labels (default: 0.5). - **labels_contour** (bool | ndarray | DataArray) - Optional - If true, show label contours. - **dim** (str) - Optional - Image dimension to blend (default: 'c'). - **figsize** (tuple[int, int]) - Optional - Figure size. - **ax** (Axes) - Optional - Matplotlib axes to plot to. - **rgb** (bool) - Optional - Whether the image is RGB or RGBA. - **mask** (ndarray | DataArray | Array) - Optional - Mask pixels where mask == 0. ### Request Example { "image": "", "labels": "", "dim": "c", "figsize": [8, 8] } ### Response #### Success Response (200) - **axes** (Axes) - The Matplotlib axes object containing the plot. #### Response Example { "status": "success", "object": "matplotlib.axes.Axes" } ``` -------------------------------- ### Zarr Data Structure Overview Source: https://github.com/genentech/scallops/blob/main/docs/outputs.md Illustrates the nested directory and file layout for chunked data storage in the Zarr format. It shows how arrays are composed of chunks, with directories representing all but the last chunk element, and the terminal chunk being a file. The 'labels' group stores segmentation information following the OME-ZARR schema. ```bash ./images.zarr ├── .zgroup ├── A1-102 │ ├── .zattrs │ ├── .zgroup │ ├── 0 │ │ ├── .zarray │ │ ├── 0 │ │ ├── 1 │ │ └── 2 │ └── labels │ ├── .zattrs │ ├── .zgroup │ ├── cell │ ├── cytosol │ ├── iss-spots │ └── nuclei ├── A1-102-cell-mask │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ ├── 2 │ └── 3 ├── A1-102-log │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ └── 2 ├── A1-102-max │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ └── 2 ├── A1-102-peaks │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ ├── 2 │ └── 3 ├── A1-102-phenotype │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ └── 1 ├── A1-102-std │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ ├── 2 │ └── 3 ├── A1-103 │ ├── .zattrs │ ├── .zgroup │ ├── 0 │ │ ├── .zarray │ │ ├── 0 │ │ ├── 1 │ │ └── 2 │ └── labels │ ├── .zattrs │ ├── .zgroup │ ├── cell │ ├── cytosol │ ├── iss-spots │ └── nuclei ├── A1-103-cell-mask │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ ├── 2 │ └── 3 ├── A1-103-log │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ └── 2 ├── A1-103-max │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ └── 2 ├── A1-103-peaks │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ ├── 1 │ ├── 2 │ └── 3 ├── A1-103-phenotype │ ├── .zattrs │ ├── .zgroup │ └── 0 │ ├── .zarray │ ├── 0 │ └── 1 └── A1-103-std ├── .zattrs ├── .zgroup └── 0 ├── .zarray ├── 0 ├── 1 ├── 2 └── 3 ``` -------------------------------- ### Illumination Correction Output Structure (BaSiCPy) Source: https://github.com/genentech/scallops/blob/main/docs/outputs.md Details the directory structure generated by BaSiCPy for illumination correction. It includes a 'model' directory containing subdirectories for each channel, each holding 'profiles.npy' (models) and 'settings.json' (correction settings). TIFF files for flatfield and darkfield may also be generated. ```bash model/ ├── c0 │   ├── profiles.npy │   └── settings.json ├── c1 │   ├── profiles.npy │   └── settings.json └── c2 ├── profiles.npy └── settings.json ```