### Install traccuracy Source: https://traccuracy.readthedocs.io/en/stable/index.html Use pip to install the library. ```bash pip install traccuracy ``` -------------------------------- ### Setup Imports and Environment Source: https://traccuracy.readthedocs.io/en/stable/_sources/test_cases/divisions.md.txt Configures the Python path and imports necessary libraries for graph visualization. ```python import sys sys.path.append("../../../tests") import matplotlib.pyplot as plt from matplotlib.patches import Patch import examples.graphs as ex_graphs from traccuracy import TrackingGraph ``` -------------------------------- ### Execute Visualization for Larger Example 1 Source: https://traccuracy.readthedocs.io/en/stable/_sources/test_cases/larger_examples.md.txt Generates a plot for the first larger example graph using the defined visualization utilities. ```python plot_matched([ex_graphs.larger_example_1()], "Larger Example 1") ``` -------------------------------- ### Import Dependencies for Tracking Examples Source: https://traccuracy.readthedocs.io/en/stable/_sources/test_cases/larger_examples.md.txt Initializes the environment by appending the test directory to the system path and importing necessary libraries for graph manipulation and visualization. ```python import sys sys.path.append("../../../tests") import matplotlib.pyplot as plt import numpy as np from matplotlib.colors import ListedColormap from matplotlib.patches import Patch import examples.larger_examples as ex_graphs from traccuracy import TrackingGraph ``` -------------------------------- ### Visualize segmentation test cases Source: https://traccuracy.readthedocs.io/en/stable/_sources/test_cases/matchers.md.txt Generates plots for various segmentation matching scenarios using predefined example data. ```python plot_pair(*ex_segs.good_segmentation_2d(), "Good Segmentation") ``` ```python plot_pair(*ex_segs.false_positive_segmentation_2d(), "False Positive") ``` ```python plot_pair(*ex_segs.false_negative_segmentation_2d(), "False Negative") ``` ```python plot_pair(*ex_segs.oversegmentation_2d(), "Oversegmentation") ``` ```python plot_pair(*ex_segs.undersegmentation_2d(), "Undersegmentation") ``` ```python plot_pair(*ex_segs.no_overlap_2d(), "No Overlap") ``` ```python plot_pair(*ex_segs.multicell_2d(), "Multiple cells") ``` -------------------------------- ### View Tracking Benchmark Results Structure Source: https://traccuracy.readthedocs.io/en/stable/examples/ctc.html Example of the JSON structure returned after running tracking benchmarks, including matcher configuration and division metrics. ```json [ { 'gt': 'Hela-01_GT', 'matcher': { '_matching_type': 'one-to-one', 'iou_threshold': 0.1, 'matching type': 'one-to-one', 'name': 'IOUMatcher', 'one_to_one': True}, 'metric': { 'frame_buffer': 2, 'name': 'DivisionMetrics', 'relax_skips_gt': False, 'relax_skips_pred': False, 'valid_match_types': ['one-to-one']}, 'pred': 'Hela-01_RES', 'results': { 'Frame Buffer 0': { 'Division F1': nan, 'Division Precision': 0.0, 'Division Recall': 0.0, 'False Negative Divisions': 19, 'False Positive Divisions': 31, 'Mitotic Branching Correctness': 0.0, 'Total GT Divisions': 94, 'Total Predicted Divisions': 106, 'True Positive Divisions': 0, 'Wrong Children Divisions': 75}, 'Frame Buffer 1': { 'Division F1': nan, 'Division Precision': 0.0, 'Division Recall': 0.0, 'False Negative Divisions': 19, 'False Positive Divisions': 31, 'Mitotic Branching Correctness': 0.0, 'Total GT Divisions': 94, 'Total Predicted Divisions': 106, 'True Positive Divisions': 0, 'Wrong Children Divisions': 75}, 'Frame Buffer 2': { 'Division F1': nan, 'Division Precision': 0.0, 'Division Recall': 0.0, 'False Negative Divisions': 19, 'False Positive Divisions': 31, 'Mitotic Branching Correctness': 0.0, 'Total GT Divisions': 94, 'Total Predicted Divisions': 106, 'True Positive Divisions': 0, 'Wrong Children Divisions': 75}}, 'version': '0.3.1.dev47+g0c591a4'}] ``` -------------------------------- ### Get lineages Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/_tracking_graph.html Retrieves a list of DiGraph objects representing all lineages in the current graph. ```python def get_lineages(self) -> list[nx.DiGraph]: """Gets a list of new nx.DiGraph objects containing all lineages of the current graph. ``` -------------------------------- ### Retrieve Node Location Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/_tracking_graph.html Get the spatial coordinates of a node based on configured location keys. ```python def get_location(self, node_id: Hashable) -> list[float] | tuple[float] | np.ndarray: """Get the spatial location of the node with node_id using self.location_keys. Args: node_id (hashable): The node_id to get the location of Returns: list of float: A list of location values in the same order as self.location_keys Raises: ValueError if location keys were not provided """ if self.location_keys is None: raise ValueError("Must provide location key(s) to access node locations") if isinstance(self.location_keys, str): return self.graph.nodes[node_id][self.location_keys] else: return [self.graph.nodes[node_id][key] for key in self.location_keys] ``` -------------------------------- ### Initialize TrackOverlapMetrics Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/metrics/_track_overlap.html Instantiate the metric calculator with an option to include division edges. ```python class TrackOverlapMetrics(Metric): """Calculate metrics for longest track overlaps. - Target Effectiveness: fraction of longest overlapping prediction tracklets on each GT tracklet - Track Purity : fraction of longest overlapping GT tracklets on each prediction tracklet Args: matched_data (traccuracy.matchers.Matched): Matched object for set of GT and Pred data include_division_edges (bool, optional): If True, include edges at division. """ def __init__(self, include_division_edges: bool = True): valid_match_types = ["many-to-one", "one-to-one", "one-to-many"] super().__init__(valid_match_types) self.include_division_edges = include_division_edges ``` -------------------------------- ### Get Edge Errors Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/track_errors/_ctc.html Calculates and annotates edge errors. Requires node errors to be annotated first. ```python def get_edge_errors(matched_data: Matched) -> None: comp_graph = matched_data.pred_graph gt_graph = matched_data.gt_graph node_mapping = matched_data.mapping if comp_graph.ctc_edge_errors + gt_graph.ctc_edge_errors == 1: graph_with_errors = "pred graph" if comp_graph.ctc_edge_errors else "GT graph" raise ValueError( f"Only {graph_with_errors} has edge errors annotated. " "Please ensure either both or neither " + "of the graphs have traccuracy annotations before running metrics." ) if comp_graph.ctc_edge_errors and gt_graph.ctc_edge_errors: warnings.warn("Edge errors already calculated. Skipping graph annotation", stacklevel=2) return # Node errors must already be annotated if not comp_graph.ctc_node_errors and not gt_graph.ctc_node_errors: logger.info("Node errors have not been annotated. Running node annotation.", stacklevel=2) get_vertex_errors(matched_data) comp_tp_nodes = comp_graph.get_nodes_with_flag(NodeFlag.CTC_TRUE_POS) induced_graph = comp_graph.graph.subgraph(comp_tp_nodes) gt_comp_mapping = {gt: comp for gt, comp in node_mapping if comp in induced_graph} comp_gt_mapping = {comp: gt for gt, comp in node_mapping if comp in induced_graph} # intertrack edges = connection between parent and daughter for graph in [comp_graph, gt_graph]: for parent in graph.get_divisions(): for daughter in graph.graph.successors(parent): graph.set_flag_on_edge((parent, daughter), EdgeFlag.INTERTRACK_EDGE, True) for merge in graph.get_merges(): for parent in graph.graph.predecessors(merge): graph.set_flag_on_edge((parent, merge), EdgeFlag.INTERTRACK_EDGE, True) # fp edges - edges in induced_graph that aren't in gt_graph for edge in tqdm(induced_graph.edges, "Evaluating FP edges"): source, target = edge[0], edge[1] source_gt_id = comp_gt_mapping[source] target_gt_id = comp_gt_mapping[target] ``` -------------------------------- ### TrackingGraph.__init__ Source: https://traccuracy.readthedocs.io/en/stable/_sources/autoapi/traccuracy/index.rst.txt Initializes a new TrackingGraph instance with a networkx directed graph. ```APIDOC ## TrackingGraph(graph, segmentation=None, frame_key='t', label_key='segmentation_id', location_keys=None, name=None, validate=True) ### Description Creates a directed graph representing a tracking solution where edges go forward in time. ### Parameters - **graph** (networkx.DiGraph) - Required - The underlying networkx graph. - **segmentation** (numpy.ndarray) - Optional - Optional segmentation data. - **frame_key** (str) - Optional - Attribute name for the time frame (default 't'). - **label_key** (str) - Optional - Attribute name for the label (default 'segmentation_id'). - **location_keys** (str | tuple) - Optional - Keys used to access spatial location. - **name** (str) - Optional - Name of the graph. - **validate** (bool) - Optional - Whether to validate the graph structure. ``` -------------------------------- ### Importing required libraries Source: https://traccuracy.readthedocs.io/en/stable/examples/ctc.html Initial imports for data handling, downloading, and traccuracy metric modules. ```python import os import pprint import urllib.request import zipfile from tqdm import tqdm from traccuracy import run_metrics from traccuracy.loaders import load_ctc_data from traccuracy.matchers import CTCMatcher, IOUMatcher from traccuracy.metrics import CTCMetrics, DivisionMetrics from traccuracy.utils import export_graphs_to_geff pp = pprint.PrettyPrinter(indent=4) ``` -------------------------------- ### Get Skip Path Maps Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/metrics/_track_overlap.html Retrieves mappings for equivalent paths associated with skip edges in a matched graph. ```python def _get_skip_path_maps( matched: Matched, skips: set[tuple[Any, Any]], skip_to_other_map: dict[Any, list[Any]], ) -> tuple[dict[tuple[Any, Any], int], dict[Any, dict[str, list[tuple[Any, Any]]]]]: """Get information about equivalent paths for skip edges. For each skip edge, find the equivalent path in the matched graph and return a mapping of skip edges to their equivalent path lengths. Also returns a mapping from nodes along equivalent paths to the edge they are part of and the skip edge they cover. Args: matched (traccuracy.matchers.Matched): The matched object containing the graphs. skips (set[tuple[Any, Any]]): Set of skip edges to process. skip_to_other_map (dict[Any, list[Any]]): Mapping of nodes in graph with skips to nodes in the other graph. ``` -------------------------------- ### Results.__init__ Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/metrics/_results.html Initializes a new Results object to store pipeline metric data and configuration. ```APIDOC ## Results(results, matcher_info, metric_info, gt_name=None, pred_name=None) ### Description Initializes the Results object with metric outputs and configuration details. ### Parameters - **results** (dict) - Required - Dictionary containing the metric output. - **matcher_info** (dict) - Required - Dictionary containing the matcher name and parameters. - **metric_info** (dict) - Required - Dictionary containing the metric name and parameters. - **gt_name** (str) - Optional - Name of the ground truth data. - **pred_name** (str) - Optional - Name of the predicted data. ``` -------------------------------- ### Get Relevant Skip Edges Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/metrics/_track_overlap.html Extracts skip edges from a tracking graph, with an option to filter out division edges. ```python def _get_relevant_skip_edges( graph: TrackingGraph, include_division_edges: bool ) -> set[tuple[Any, Any]]: """Get relevant skip edges from the graph, potentially including division edges. Args: graph (TrackingGraph): graph to extract skip edges from include_division_edges (bool): True if parent-daughter edges should be included, otherwise False. Returns: set[tuple[Any, Any]]: skip edges on graph with/without division edges. """ skips = graph.get_skip_edges() if include_division_edges: return skips # if division edges are not included, we only consider skips that are not division edges for skip_src, skip_tgt in skips.copy(): if graph.graph.out_degree(skip_src) > 1: # type: ignore skips.remove((skip_src, skip_tgt)) return skips ``` -------------------------------- ### Get Vertex Errors Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/track_errors/_ctc.html Counts vertex errors and assigns classes to nodes in the provided matched data object. ```python def get_vertex_errors(matched_data: Matched) -> None: """Count vertex errors and assign class to each comp/gt node. Parameters ---------- matched_data: traccuracy.matchers.Matched Matched data object containing gt and pred graphs with their associated mapping """ comp_graph = matched_data.pred_graph gt_graph = matched_data.gt_graph dict_mapping = matched_data.pred_gt_map if comp_graph.ctc_node_errors + gt_graph.ctc_node_errors == 1: graph_with_errors = "pred graph" if comp_graph.ctc_node_errors else "GT graph" raise ValueError( f"Only {graph_with_errors} has node errors annotated. " "Please ensure either both or neither " + "of the graphs have traccuracy annotations before running metrics." ) if comp_graph.ctc_node_errors and gt_graph.ctc_node_errors: warnings.warn("Node errors already calculated. Skipping graph annotation", stacklevel=2) return # will flip this when we come across the vertex in the mapping comp_graph.set_flag_on_all_nodes(NodeFlag.CTC_FALSE_POS, True) gt_graph.set_flag_on_all_nodes(NodeFlag.CTC_FALSE_NEG, True) ns_count = 0 for pred_id in tqdm(dict_mapping, desc="Evaluating nodes"): gt_ids = dict_mapping[pred_id] if len(gt_ids) == 1: gid = gt_ids[0] comp_graph.set_flag_on_node(pred_id, NodeFlag.CTC_TRUE_POS, True) comp_graph.remove_flag_from_node(pred_id, NodeFlag.CTC_FALSE_POS) gt_graph.remove_flag_from_node(gid, NodeFlag.CTC_FALSE_NEG) gt_graph.set_flag_on_node(gid, NodeFlag.CTC_TRUE_POS, True) elif len(gt_ids) > 1: comp_graph.set_flag_on_node(pred_id, NodeFlag.NON_SPLIT, True) comp_graph.remove_flag_from_node(pred_id, NodeFlag.CTC_FALSE_POS) # number of split operations that would be required to correct the vertices ns_count += len(gt_ids) - 1 for gt_id in gt_ids: gt_graph.remove_flag_from_node(gt_id, NodeFlag.CTC_FALSE_NEG) # Record presence of annotations on the TrackingGraph comp_graph.ctc_node_errors = True gt_graph.ctc_node_errors = True ``` -------------------------------- ### Defining download parameters Source: https://traccuracy.readthedocs.io/en/stable/examples/ctc.html Sets the URL and local directory for the CTC dataset. ```python url = "http://data.celltrackingchallenge.net/training-datasets/Fluo-N2DL-HeLa.zip" data_dir = "downloads" if not os.path.exists(data_dir): os.mkdir(data_dir) filename = url.split("/")[-1] file_path = os.path.join(data_dir, filename) ds_name = filename.split(".")[0] ``` -------------------------------- ### Initialize DivisionMetrics Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/metrics/_divisions.html Instantiate the DivisionMetrics class with an optional frame buffer to handle shifted division events. ```python class DivisionMetrics(Metric): """Computes division summary metrics with an optional frame tolerance. Computes the following metrics: - Division Recall - Division Precision - Division F1 Score (also Branching Correctness) - Mitotic Branching Correctness: TP / (TP + FP + FN) as defined by *Ulicna, K., Vallardi, G., Charras, G. & Lowe, A. R. Automated deep lineage tree analysis using a Bayesian single cell tracking approach. Frontiers in Computer Science 3, 734559 (2021).* These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives. Args: max_frame_buffer (int, optional): Maximum value of frame buffer to use in correcting shifted divisions. Divisions will be evaluated for all integer values of frame buffer between 0 and max_frame_buffer """ def __init__(self, max_frame_buffer: int = 0) -> None: valid_matching_types = ["one-to-one"] super().__init__(valid_matching_types) self.frame_buffer = max_frame_buffer ``` -------------------------------- ### Numba JIT acceleration configuration Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/matchers/_compute_overlap.html Attempts to import Numba for JIT compilation, falling back to standard NumPy if unavailable. ```python try: import numba except ImportError: import os import warnings if not os.getenv("NO_JIT_WARNING", False): warnings.warn( "Numba not installed, falling back to slower numpy implementation. " "Install numba for a significant speedup. Set the environment " "variable NO_JIT_WARNING=1 to disable this warning.", stacklevel=2, ) else: # compute_overlap 2d and 3d have the same signature signature = [ "f8[:,::1](f8[:,::1], f8[:,::1])", numba.types.Array(numba.float64, 2, "C", readonly=True)( numba.types.Array(numba.float64, 2, "C", readonly=True), numba.types.Array(numba.float64, 2, "C", readonly=True), ), ] # variables that appear in the body of each function common_locals = { "N": numba.uint64, "K": numba.uint64, "overlaps": numba.types.Array(numba.float64, 2, "C"), "iw": numba.float64, "ih": numba.float64, "ua": numba.float64, "n": numba.uint64, "k": numba.uint64, } compute_overlap = numba.njit( signature, locals={**common_locals, "box_area": numba.float64}, fastmath=True, nogil=True, boundscheck=False, )(compute_overlap) ``` -------------------------------- ### Initialize TrackingGraph class Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/_tracking_graph.html Represents a directed graph for tracking solutions where edges flow forward in time. Nodes require time frame and location attributes. ```python class TrackingGraph: """A directed graph representing a tracking solution where edges go forward in time. Nodes represent cell detections and edges represent links between detections in the same track. Nodes in the graph must have an attribute that represents time frame (default to 't') and location (defaults to 'x' and 'y'). As in networkx, every cell must have a unique id, but these can be of any (hashable) type. Edges typically connect nodes across consecutive frames, but gap closing or frame skipping edges are valid, which connect nodes in frame t to nodes in frames beyond t+1. """ ``` -------------------------------- ### Initialize TrackingGraph attributes Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/_tracking_graph.html Sets up internal mappings for nodes and edges based on flags and validates node attributes if requested. ```python def _set_attrs(self, validate: bool) -> None: """Set TrackingGraph attributes that are dependent on the input graph or potentially changed during annotations Args: validate (bool): Validate that nodes have required attributes: frame_key, location_key and label_key (if segmentation provided). """ # construct dictionaries from attributes to nodes/edges for easy lookup self.nodes_by_frame: defaultdict[int, set[Hashable]] = defaultdict(set) self.nodes_by_flag: dict[str, set[Hashable]] = { # We do not include MIN_BUFFER_CORRECT here, as it is not a boolean # "flag" but rather an integer. In future, if we decide to store # more "flags" that take on integer values, we may wish to make a # separate enum for them. flag.value: set() for flag in NodeFlag if flag != NodeFlag.MIN_BUFFER_CORRECT } self.edges_by_flag: dict[str, set[tuple[Hashable, Hashable]]] = { flag.value: set() for flag in EdgeFlag } for node, attrs in self.graph.nodes.items(): node = cast("int", node) if validate: self._validate_node(node, attrs) # store node id in nodes_by_frame mapping frame = attrs[self.frame_key] if frame not in self.nodes_by_frame.keys(): self.nodes_by_frame[frame] = {node} else: self.nodes_by_frame[frame].add(node) # store node id in nodes_by_flag mapping for node_flag in NodeFlag: if attrs.get(node_flag.value): self.nodes_by_flag[node_flag.value].add(node) # store edge id in edges_by_flag for edge, attrs in self.graph.edges.items(): for edge_flag in EdgeFlag: if attrs.get(edge_flag.value): self.edges_by_flag[edge_flag.value].add(edge) # Record types of annotations that have been calculated self.division_annotations = False self.division_skip_gt_relaxed = False self.division_skip_pred_relaxed = False self.basic_node_errors = False self.basic_edge_errors = False self.ctc_node_errors = False self.ctc_edge_errors = False self.skip_edges_gt_relaxed = False self.skip_edges_pred_relaxed = False ``` -------------------------------- ### Visualize Default Skip Edge Behavior Source: https://traccuracy.readthedocs.io/en/stable/_sources/track_errors/basic.md.txt Demonstrates how skip edges are annotated as FP or FN by default when they do not match exactly. ```python plot_matched( [ ex_graphs.gap_close_matched_gap(), ex_graphs.gap_close_gt_gap(), ex_graphs.gap_close_pred_gap(), ], [ {}, {(1, 3): "FN", (5, 6): "FP", (6, 7): "FP"}, {(2, 3): "FN", (3, 4): "FN", (6, 8): "FP"}, ], "", ["Ex. 1", "Ex. 2", "Ex. 3"], ) ``` -------------------------------- ### Visualize Relaxed Skip Edge Handling Source: https://traccuracy.readthedocs.io/en/stable/_sources/track_errors/basic.md.txt Demonstrates skip edge annotations when using permissive matching settings. ```python plot_matched( [ ex_graphs.gap_close_gt_gap(), ex_graphs.gap_close_pred_gap(), ex_graphs.div_parent_gap() ], [ {(1, 3): "STP", (5, 6): "STP", (6, 7): "STP"}, {(2, 3): "STP", (3, 4): "STP", (6, 8): "STP"}, {(9, 11): "STP", (9, 12): "STP", (2, 3): "STP", (3, 4): "STP", (3, 5): "STP"} ], "", ["relax_skips_gt=True", "relax_skips_pred=True", "relax_skips_pred=True"], ) ``` -------------------------------- ### Compute Basic Performance Metrics Source: https://traccuracy.readthedocs.io/en/stable/_sources/metrics/basic.md.txt Calculates tracking metrics by running a matcher and basic metrics on ground truth and predicted data. ```python from traccuracy.matchers import PointMatcher from traccuracy.metrics import BasicMetrics # Data loaded using a function from traccuracy.loaders or constructed explicitly using a networkx graph and associated information gt_data: TrackingGraph pred_data: TrackingGraph results, matched = run_metrics( gt_data=gt_data, pred_data=pred_data, matcher=PointMatcher(), # Choose a matcher that produces a one-to-one mapping metrics=[BasicMetrics()] ) ``` -------------------------------- ### Numba JIT configuration for compute_overlap_3D Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/matchers/_compute_overlap.html Applies Numba's just-in-time compilation to the 3D overlap function with specific performance optimizations. ```python compute_overlap_3D = numba.njit( signature, locals={**common_locals, "id_": numba.float64, "box_volume": numba.float64}, fastmath=True, nogil=True, boundscheck=False, )(compute_overlap_3D) ``` -------------------------------- ### Downloading and extracting data Source: https://traccuracy.readthedocs.io/en/stable/examples/ctc.html Downloads the dataset with a progress bar and extracts the zip file. ```python # Add a utility to make a progress bar when downloading the file class DownloadProgressBar(tqdm): def update_to(self, b=1, bsize=1, tsize=None): if tsize is not None: self.total = tsize self.update(b * bsize - self.n) if not os.path.exists(file_path): print(f"Downloading {ds_name} data from the CTC website") # Downloading data with DownloadProgressBar(unit="B", unit_scale=True, miniters=1, desc=url.split("/")[-1]) as t: urllib.request.urlretrieve(url, file_path, reporthook=t.update_to) # Unzip the data # TODO add a progress bar to zip as well with zipfile.ZipFile(file_path, "r") as zip_ref: zip_ref.extractall(data_dir) ``` -------------------------------- ### PointMatcher Implementation Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/matchers/_point.html The PointMatcher class uses Euclidean distance and a KDTree to perform one-to-one matching between ground truth and prediction graphs. ```python from __future__ import annotations from typing import TYPE_CHECKING import pylapy from scipy.spatial import KDTree from ._base import Matcher if TYPE_CHECKING: from collections.abc import Hashable from typing import Any import numpy as np from scipy.sparse import coo_matrix from traccuracy._tracking_graph import TrackingGraph [docs] class PointMatcher(Matcher): """A one-to-one matcher that uses Hungarian matching to minimize global distance of node pairs with a maximum distance threshold beyond which nodes will not be matched. Note: this matcher computes the Euclidean distance based on the location on the points. If the data is not isotropic, the scale parameter can be used to rescale the locations in each dimension to reflect "real-world" distances. Args: threshold (float): The maximum distance to allow node matchings (inclusive), in (potentially rescaled) pixels. scale_factor (tuple[float, ...] | list[float] | None, optional): If provided, multiply the node locations by the scale factor in each dimension before computing the distance. Useful if the data is not isotropic to ensure that distances are computed in a space that reflects real world distances. """ def __init__( self, threshold: float, scale_factor: tuple[float, ...] | list[float] | None = None, ): self.threshold = threshold self.scale_factor = scale_factor # this matching is always one-to-one self._matching_type = "one-to-one" # Lap solver using scipy self._solver = pylapy.LapSolver(implementation="scipy", sparse_implementation="csgraph") def _compute_mapping( self, gt_graph: TrackingGraph, pred_graph: TrackingGraph ) -> list[tuple[Any, Any]]: mapping: list[tuple[Any, Any]] = [] if gt_graph.start_frame is None or gt_graph.end_frame is None: return mapping for frame in range(gt_graph.start_frame, gt_graph.end_frame): # Sorting node ids to ensure deterministic solution when there are ties # Ignoring typing because technically "Hashable" node ids are not # always sortable, but we don't anticipate non-sortable types gt_nodes = sorted(gt_graph.nodes_by_frame.get(frame, [])) # type: ignore pred_nodes = sorted(pred_graph.nodes_by_frame.get(frame, [])) # type: ignore gt_locations = [gt_graph.get_location(node) for node in gt_nodes] pred_locations = [pred_graph.get_location(node) for node in pred_nodes] if self.scale_factor is not None: assert len(self.scale_factor) == len(gt_locations[0]), ( f"scale factor {self.scale_factor} has different length than " f"location {gt_locations[0]}" ) gt_locations = [ [loc[d] * self.scale_factor[d] for d in range(len(loc))] for loc in gt_locations ] pred_locations = [ [loc[d] * self.scale_factor[d] for d in range(len(loc))] for loc in pred_locations ] matches = self._match_frame( gt_nodes, gt_locations, pred_nodes, pred_locations, ) mapping.extend(matches) return mapping def _match_frame( self, gt_nodes: list[Hashable], gt_locations: list[list[float] | tuple[float] | np.ndarray], pred_nodes: list[Hashable], pred_locations: list[list[float] | tuple[float] | np.ndarray], ) -> list[tuple[Any, Any]]: mapping: list[tuple[Any, Any]] = [] if len(gt_nodes) == 0 or len(pred_nodes) == 0: return mapping gt_kdtree = KDTree(gt_locations) pred_kdtree = KDTree(pred_locations) # indices correspond to indices in the gt_nodes, pred_nodes lists sdm: coo_matrix = gt_kdtree.sparse_distance_matrix( pred_kdtree, max_distance=self.threshold, output_type="coo_matrix" ) # Let's keep threshold * 4 for compatibility. But one could probably do # hard thresholding instead (using hard=True) which is indeed what we want to do links = self._solver.sparse_solve(sdm, self.threshold).T # Go back to matched node ids from matrix indices for row, col in links.T.tolist(): gt_id = gt_nodes[row] pred_id = pred_nodes[col] mapping.append((gt_id, pred_id)) return mapping ``` -------------------------------- ### Visualize False Negative Edges Source: https://traccuracy.readthedocs.io/en/stable/_sources/track_errors/ctc.md.txt Illustrates various scenarios resulting in false negative edge annotations. ```python plot_matched([ex_graphs.fn_node_matched(0)], [{1: "FN", (1, 2): "FN"}], "") ``` ```python plot_matched([ex_graphs.fn_edge_matched(0)], [{(1, 2): "FN"}], "") ``` ```python plot_matched([ex_graphs.node_two_to_one(t) for t in [0, 1]], [{4: "NS", (1, 2): "FN"}, {5: "NS", (1, 7): "FN", (2, 3): "FN"}], "") ``` -------------------------------- ### Run traccuracy CLI Source: https://traccuracy.readthedocs.io/en/stable/cli.html Executes the traccuracy tool by specifying the directories for ground truth and predicted data. ```bash traccuracy [OPTIONS] GT_DIR PRED_DIR ``` -------------------------------- ### Calculate Division Metrics Source: https://traccuracy.readthedocs.io/en/stable/_sources/metrics/divisions.md.txt Initializes and runs division metrics using a TrackingGraph and a PointMatcher. The frame_buffer parameter defines the temporal tolerance for matching divisions. ```python from traccuracy.matchers import PointMatcher from traccuracy.metrics import DivisionMetrics # Load data using a function of your choice from traccuracy.loaders # Or initialize a TrackingGraph with a nx.DiGraph and optional segmentation arrays gt_data: TrackingGraph pred_data: TrackingGraph # frame_buffer determines the size of a window in which a division can occur but not # be in the same frame while still being counted as correct frame_buffer = 2 results, matched = run_metrics( gt_data=gt_data, pred_data=pred_data, matcher=PointMatcher(), # Select any matcher that returns a one-to-one mapping metrics=[DivisionMetrics(max_frame_buffer=frame_buffer)] ) ``` -------------------------------- ### load_point_data(path=None, df=None, parent_column='parent', id_column='node_id', pos_columns=('z', 'y', 'x'), time_column='t', seg_id_column=None, name=None, sep=None) Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/loaders/_point.html Loads point-based tracking data into a TrackingGraph. Requires either a file path or a pandas DataFrame as input. ```APIDOC ## load_point_data ### Description Loads point-based tracking data into a TrackingGraph from a csv-like file or an existing pandas DataFrame. ### Parameters - **path** (str | None) - Optional - Path to the csv-like file to load. - **df** (pd.DataFrame | None) - Optional - A dataframe that has already been loaded. - **parent_column** (str) - Optional - Column name for the parent node reference. Defaults to 'parent'. - **id_column** (str) - Optional - Column name for unique node IDs. Defaults to 'node_id'. - **pos_columns** (tuple[str, ...]) - Optional - Tuple of column names to use for position. Defaults to ('z', 'y', 'x'). - **time_column** (str) - Optional - Column name to use for time. Defaults to 't'. - **seg_id_column** (str | None) - Optional - Column name for an optional segmentation label ID. - **name** (str | None) - Optional - Optional string to name/describe the dataset. - **sep** (str | None) - Optional - Separator passed to pd.read_csv. ### Returns - **TrackingGraph** - The resulting graph object containing the loaded tracking data. ``` -------------------------------- ### Import dependencies for segmentation testing Source: https://traccuracy.readthedocs.io/en/stable/_sources/test_cases/matchers.md.txt Configures the system path and imports necessary libraries for plotting and segmentation handling. ```python import sys sys.path.append("../../../tests") import matplotlib.pyplot as plt import numpy as np from matplotlib.colors import ListedColormap from matplotlib.patches import Patch import examples.segs as ex_segs ``` -------------------------------- ### TrackingGraph Loader Source: https://traccuracy.readthedocs.io/en/stable/_sources/autoapi/traccuracy/loaders/index.rst.txt Initializes a TrackingGraph by loading data from a CSV file or an existing pandas DataFrame. ```APIDOC ## TrackingGraph Loader ### Description Loads tracking data from a file or DataFrame to create a TrackingGraph object. Requires either a file path or a pre-loaded DataFrame. ### Parameters - **path** (str | None) - Optional - Path to the csv-like file to load. - **df** (pd.DataFrame | None) - Optional - A dataframe that has already been loaded. - **parent_column** (str | None) - Optional - A reference to the parent node in the previous time frame. Defaults to "parent". - **id_column** (str) - Optional - Column used to specify node ids. Defaults to 'node_id'. - **pos_columns** (tuple[str]) - Optional - A tuple of columns to use for position. Defaults to ("z", "y", "x"). - **time_column** (str) - Optional - The column to use for time. Defaults to "t". - **seg_id_column** (str | None) - Optional - Name of an optional column containing a segmentation label id. - **name** (str | None) - Optional - Optional string to name/describe the dataset. - **sep** (str | None) - Optional - Passed to pd.read_csv to set the sep kwarg. ### Errors - **ValueError**: Raised if neither path nor df is provided, or if required columns are missing/invalid. ``` -------------------------------- ### Running IOU metrics with division evaluation Source: https://traccuracy.readthedocs.io/en/stable/examples/ctc.html Executes metrics using an IOU matcher with a specific threshold and evaluates division events. ```python iou_results, iou_matched = run_metrics( gt_data=gt_data, pred_data=pred_data, matcher=IOUMatcher(iou_threshold=0.1, one_to_one=True), metrics=[DivisionMetrics(max_frame_buffer=2)], ) pp.pprint(iou_results) ``` -------------------------------- ### run_metrics(gt_data, pred_data, matcher, metrics, relax_skips_gt=False, relax_skips_pred=False) Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/_run_metrics.html Computes given metrics on tracking graph data using the specified matcher. ```APIDOC ## run_metrics ### Description Compute given metrics on data using the given matcher. The returned result dictionary will contain all metrics computed by the given Metric classes, as well as general summary numbers e.g. false positive/false negative detection and edge counts. ### Parameters - **gt_data** (traccuracy.TrackingGraph) - Required - Ground truth graph and optionally segmentation. - **pred_data** (traccuracy.TrackingGraph) - Required - Predicted graph and optionally segmentation. - **matcher** (traccuracy.matchers._base.Matcher) - Required - Instantiated matcher object. - **metrics** (list[traccuracy.metrics._base.Metric]) - Required - List of instantiated metrics objects to compute. - **relax_skips_gt** (bool) - Optional - If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph. Defaults to False. - **relax_skips_pred** (bool) - Optional - If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph. Defaults to False. ### Returns - **tuple[list[dict], Matched]** - A tuple containing a list of dictionaries with one dictionary per Metric object and the Matched data which includes annotated graphs. ``` -------------------------------- ### Running CTC metrics Source: https://traccuracy.readthedocs.io/en/stable/examples/ctc.html Executes standard CTC metrics using the CTCMatcher. ```python ctc_results, ctc_matched = run_metrics( gt_data=gt_data, pred_data=pred_data, matcher=CTCMatcher(), metrics=[CTCMetrics()], ) pp.pprint(ctc_results) ``` -------------------------------- ### Evaluate tracking data Source: https://traccuracy.readthedocs.io/en/stable/index.html Load ground truth and predicted data, then compute metrics using a matcher. ```python from traccuracy.loaders import load_ctc_data from traccuracy.matchers import PointMatcher from traccuracy.metrics import DivisionMetrics, BasicMetrics # Load data in TrackingGraph objects gt_data = load_ctc_data( "path/to/GT/TRA", "path/to/GT/TRA/man_track.txt", name="GT" ) pred_data = load_ctc_data( "path/to/prediction", "path/to/prediction/track.txt", name="prediction" ) results, matched = run_metrics( gt_data=gt_data, pred_data=pred_data, matcher=PointMatcher(), metrics=[DivisionMetrics(), BasicMetrics()] ) ``` -------------------------------- ### load_geff_data(geff_path, load_geff_seg=False, seg_path=None, seg_property=None, name=None, load_all_props=False) Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/loaders/_geff.html Loads a graph into memory from a GEFF file. Segmentations can be optionally loaded either from a related object specified in the GEFF or via a path to a Zarr array. ```APIDOC ## load_geff_data ### Description Loads a graph into memory from a GEFF file. Segmentations can be optionally loaded either from a related object specified in the GEFF or via a path to a Zarr array. ### Parameters - **geff_path** (str) - Required - Path to a geff group inside of a zarr. - **load_geff_seg** (bool) - Optional - Load segmentation based on a geff metadata of related segmentation. Defaults to False. - **seg_path** (str | None) - Optional - Path to a zarr array containing segmentation data. Defaults to None. - **seg_property** (str | None) - Optional - If seg_path provided, this is the corresponding property on the geff graph that contains the segmentation key. Defaults to None. - **name** (str | None) - Optional - Optional name to store on TrackingGraph for identification. Defaults to None. - **load_all_props** (bool) - Optional - If True, load all node and edge properties on the graph. Defaults to False. ### Returns - **TrackingGraph** - The loaded tracking graph object. ``` -------------------------------- ### CTCMatcher Implementation Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/matchers/_ctc.html The CTCMatcher class inherits from Matcher and implements the _compute_mapping method to perform frame-by-frame overlap analysis between ground truth and predicted tracking graphs. ```python from __future__ import annotations from typing import TYPE_CHECKING from tqdm import tqdm if TYPE_CHECKING: from collections.abc import Hashable from traccuracy._tracking_graph import TrackingGraph from ._base import Matcher from ._compute_overlap import get_labels_with_overlap, graph_bbox_and_labels [docs] class CTCMatcher(Matcher): """Match graph nodes based on measure used in cell tracking challenge benchmarking. A computed marker (segmentation) is matched to a reference marker if the computed marker covers a majority of the reference marker. Each reference marker can therefore only be matched to one computed marker, but multiple reference markers can be assigned to a single computed marker. See https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0144959 for complete details. """ # CTC can return many-to-one or one-to-one _matching_type = None def _compute_mapping( self, gt_graph: TrackingGraph, pred_graph: TrackingGraph ) -> list[tuple[Hashable, Hashable]]: """Run ctc matching Args: gt_graph (TrackingGraph): Tracking graph object for the gt pred_graph (TrackingGraph): Tracking graph object for the pred Returns: traccuracy.matchers.Matched: Matched data object containing the CTC mapping Raises: ValueError: if GT and pred segmentations are None or are not the same shape """ gt = gt_graph pred = pred_graph gt_label_key = gt_graph.label_key pred_label_key = pred_graph.label_key G_gt, mask_gt = gt, gt.segmentation G_pred, mask_pred = pred, pred.segmentation if mask_gt is None or mask_pred is None: raise ValueError("Segmentation is None, cannot perform matching") if mask_gt.shape != mask_pred.shape: raise ValueError("Segmentation shapes must match between gt and pred") mapping: list[tuple] = [] # Get overlaps for each frame if gt.start_frame is None or gt.end_frame is None: return mapping for i, t in enumerate( tqdm( range(gt.start_frame, gt.end_frame), desc="Matching frames", ) ): gt_frame = mask_gt[i] pred_frame = mask_pred[i] gt_frame_nodes = gt.nodes_by_frame[t] pred_frame_nodes = pred.nodes_by_frame[t] # get the labels for this frame gt_label_to_id = { G_gt.graph.nodes[node][gt_label_key]: node for node in gt_frame_nodes if gt_label_key in G_gt.graph.nodes[node] } pred_label_to_id = { G_pred.graph.nodes[node][pred_label_key]: node for node in pred_frame_nodes if pred_label_key in G_pred.graph.nodes[node] } gt_boxes, gt_labels = graph_bbox_and_labels(gt.graph, gt_frame_nodes, gt_label_key) pred_boxes, pred_labels = graph_bbox_and_labels( pred.graph, pred_frame_nodes, pred_label_key ) overlaps = get_labels_with_overlap( gt_frame, pred_frame, gt_boxes=gt_boxes, res_boxes=pred_boxes, gt_labels=gt_labels, res_labels=pred_labels, overlap="iogt", ) # Switch from segmentation ids to node ids for gt_label, pred_label, iogt in overlaps: if iogt > 0.5: mapping.append((gt_label_to_id[gt_label], pred_label_to_id[pred_label])) return mapping ``` -------------------------------- ### Compute metrics with run_metrics Source: https://traccuracy.readthedocs.io/en/stable/_modules/traccuracy/_run_metrics.html Executes a list of metrics on ground truth and predicted tracking graphs using a specified matcher. ```python from __future__ import annotations from typing import TYPE_CHECKING from traccuracy._tracking_graph import TrackingGraph from traccuracy.matchers._base import Matcher from traccuracy.metrics._base import Metric if TYPE_CHECKING: from traccuracy.matchers._matched import Matched [docs] def run_metrics( gt_data: TrackingGraph, pred_data: TrackingGraph, matcher: Matcher, metrics: list[Metric], relax_skips_gt: bool = False, relax_skips_pred: bool = False, ) -> tuple[list[dict], Matched]: """Compute given metrics on data using the given matcher. The returned result dictionary will contain all metrics computed by the given Metric classes, as well as general summary numbers e.g. false positive/false negative detection and edge counts. Args: gt_data (traccuracy.TrackingGraph): ground truth graph and optionally segmentation pred_data (traccuracy.TrackingGraph): predicted graph and optionally segmentation matcher (traccuracy.matchers._base.Matcher): instantiated matcher object metrics (List[traccuracy.metrics._base.Metric]): list of instantiated metrics objects to compute relax_skips_gt (bool): If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph relax_skips_pred (bool): If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph Returns: List[Dict]: List of dictionaries with one dictionary per Metric object Matched: Matched data which includes annotated graphs """ if not isinstance(gt_data, TrackingGraph) or not isinstance(pred_data, TrackingGraph): raise TypeError("gt_data and pred_data must be TrackingGraph objects") if not isinstance(matcher, Matcher): raise TypeError("matcher must be an instantiated Matcher object") if not all(isinstance(m, Metric) for m in metrics): raise TypeError("metrics must be a list of instantiated Metric objects") matched = matcher.compute_mapping(gt_data, pred_data) results = [] for _metric in metrics: result = _metric.compute( matched, relax_skips_gt=relax_skips_gt, relax_skips_pred=relax_skips_pred ) results.append(result.to_dict()) return results, matched ```