### Install pointnet2 Module Source: https://github.com/graspnet/anygrasp_sdk/blob/main/README.md Install the 'pointnet2' module by navigating to its directory and running the setup script. ```bash cd pointnet2 python setup.py install ``` -------------------------------- ### Example License Configuration Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/license_system.md An example of a 'licenseCfg.json' file demonstrating permanent and time-limited toolbox modes. ```json { "version": "2.8", "feature_id": "15189294542147360485", "public_key": "ai_123.public_key", "signature": "ai_123.signature", "license": "ai_123.lic", "toolbox": [ { "name": "Production", "type": "PERMANENT", "data": "" }, { "name": "Research", "type": "PERMANENT", "data": "" }, { "name": "Demo", "type": "PERMANENT", "data": "" }, { "name": "Example_DO_NO_MODIFY", "type": "TIME_LIMITED", "data": "2021-01-01_00-00-00 2021-05-31_23-59-59" } ] } ``` -------------------------------- ### Configure and Initialize AnyGraspTracker Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_tracking_api.md Example of how to configure and initialize the AnyGraspTracker using argparse for command-line arguments. This setup is necessary before creating an instance of the tracker. ```python import argparse parser = argparse.ArgumentParser() parser.add_argument('--checkpoint_path', required=True, help='Model checkpoint path') parser.add_argument('--filter', type=str, default='oneeuro', help='Filter to smooth grasp parameters [oneeuro/kalman/none]') parser.add_argument('--debug', action='store_true', help='Enable visualization') cfgs = parser.parse_args() tracker = AnyGraspTracker(cfgs) ``` -------------------------------- ### Install Other Requirements Source: https://github.com/graspnet/anygrasp_sdk/blob/main/README.md Install the remaining project dependencies using pip. ```bash pip install -r requirements.txt ``` -------------------------------- ### PointnetSAModuleMSG Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_modules.md Illustrates the setup and application of PointnetSAModuleMSG with multiple radii and corresponding MLP configurations. This example demonstrates how to aggregate features from different neighborhood scales. ```python # Multi-scale: 3 radii with different sampling densities sa_msg = PointnetSAModuleMSG( npoint=512, radii=[0.1, 0.2, 0.4], nsamples=[16, 32, 128], mlps=[[3+6, 64, 64], [3+6, 64, 96], [3+6, 64, 128]], bn=True, use_xyz=True ) xyz = torch.randn(2, 4096, 3) features = torch.randn(2, 4096, 6) new_xyz, new_features = sa_msg(xyz, features) # new_xyz: (2, 512, 3) # new_features: (2, 64+96+128, 512) = (2, 288, 512) ``` -------------------------------- ### PointnetSAModule Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_modules.md Demonstrates the instantiation and usage of PointnetSAModule for processing point cloud data. This example shows how to set up the layer with specific parameters and apply it to input XYZ and features. ```python import torch import torch.nn as nn from pointnet2.pointnet2_modules import PointnetSAModule # Layer 1: sample 512 points within 0.1m radius, sample up to 32 neighbors sa1 = PointnetSAModule( mlp=[3, 64, 64], npoint=512, radius=0.1, nsample=32, bn=True, use_xyz=True ) xyz = torch.randn(2, 4096, 3) # (batch=2, points=4096) features = torch.randn(2, 4096, 6) # (batch=2, points=4096, channels=6) new_xyz, new_features = sa1(xyz, features) # new_xyz: (2, 512, 3) # new_features: (2, 128, 512) after mlp=[3+6, 64, 128] ``` -------------------------------- ### Install MinkowskiEngine Source: https://github.com/graspnet/anygrasp_sdk/blob/main/README.md Follow these steps to install a modified version of MinkowskiEngine. Ensure CUDA is set up correctly and select the appropriate git checkout for your CUDA version. ```bash mkdir dependencies && cd dependencies conda install openblas-devel -c anaconda export CUDA_HOME=/path/to/cuda git clone git@github.com:chenxi-wang/MinkowskiEngine.git cd MinkowskiEngine # Uncomment the following line if you are using CUDA 12.x. # git checkout cuda-12-1 # Uncomment the following line if you are using CUDA 12.8. # sed -i 's/\bauto __raw = __to_address(__r.get());/auto __raw = __std::__to_address(__r.get());/ /usr/include/c++/11/bits/shared_ptr_base.h python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas_library_dirs=${CONDA_PREFIX}/lib --blas=openblas cd ../.. ``` -------------------------------- ### Example Workspace Limits Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/data_structures.md Provides an example of defining workspace limits for a table, specifying ranges in meters for X, Y, and Z axes. ```python # Table workspace: ±0.2m in X, ±0.15m in Y, 0-1m in Z lims = [-0.2, 0.2, -0.15, 0.15, 0.0, 1.0] ``` -------------------------------- ### Setup AnyGrasp Configuration and Instance Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_detection_api.md Configure AnyGrasp parameters using argparse and instantiate the AnyGrasp class. Ensure max_gripper_width is clamped between 0 and 0.1 meters. ```python import argparse parser = argparse.ArgumentParser() parser.add_argument('--checkpoint_path', required=True, help='Model checkpoint path') parser.add_argument('--max_gripper_width', type=float, default=0.1, help='Maximum gripper width (<=0.1m)') parser.add_argument('--gripper_height', type=float, default=0.03, help='Gripper height') parser.add_argument('--top_down_grasp', action='store_true', help='Output top-down grasps.') parser.add_argument('--debug', action='store_true', help='Enable debug mode') cfgs = parser.parse_args() cfgs.max_gripper_width = max(0, min(0.1, cfgs.max_gripper_width)) anygrasp = AnyGrasp(cfgs) ``` -------------------------------- ### Check License Status Source: https://github.com/graspnet/anygrasp_sdk/blob/main/license_registration/README.md Use this command to verify the validity and status of your installed license by pointing to the license configuration file. ```bash ./license_checker -c license/licenseCfg.json ``` -------------------------------- ### AnyGrasp SDK Detection Configurations Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/integration_guide.md Provides example configurations for high-quality, real-time, and dense grasp detection. Adjust parameters based on your specific application needs for accuracy, speed, or density. ```python # High-quality detection (slower, higher accuracy) config_high_quality = { 'max_gripper_width': 0.1, 'gripper_height': 0.03, 'dense_grasp': False, 'apply_object_mask': True, 'collision_detection': True, 'nms_threshold': 0.3, 'min_score': 0.5, } # Real-time detection (faster, lower latency) config_realtime = { 'max_gripper_width': 0.1, 'gripper_height': 0.03, 'dense_grasp': False, 'apply_object_mask': True, 'collision_detection': True, 'nms_threshold': 0.5, # Larger threshold = fewer outputs 'min_score': 0.3, } # Dense mode (specialized scenarios) config_dense = { 'max_gripper_width': 0.1, 'gripper_height': 0.03, 'dense_grasp': True, # Extremely dense output 'apply_object_mask': False, # Predict on background 'collision_detection': False, # Disabled for performance 'nms_threshold': 0.1, 'min_score': 0.1, } ``` -------------------------------- ### BNMomentumScheduler Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Demonstrates how to instantiate and use BNMomentumScheduler to decay batch normalization momentum over training epochs. ```python from pointnet2.pytorch_utils import BNMomentumScheduler # Decay momentum from 0.5 to 0.99 over epochs def bn_lambda(epoch): return max(0.99, 0.5 + 0.45 * (1 - epoch / 100.0)) scheduler = BNMomentumScheduler(model, bn_lambda) for epoch in range(100): # Training loop... scheduler.step(epoch) ``` -------------------------------- ### Usage Example for AnyGrasp get_grasp Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_detection_api.md Demonstrates loading depth and color data, creating a point cloud, configuring AnyGrasp, and performing grasp detection with specified parameters. Includes post-processing and visualization of results. ```python import numpy as np import open3d as o3d from PIL import Image from gsnet import AnyGrasp # Load data colors = np.array(Image.open('color.png'), dtype=np.float32) / 255.0 depths = np.array(Image.open('depth.png')) # Create point cloud from depth image fx, fy, cx, cy = 927.17, 927.37, 651.32, 349.62 scale = 1000.0 xmap, ymap = np.arange(depths.shape[1]), np.arange(depths.shape[0]) xmap, ymap = np.meshgrid(xmap, ymap) points_z = depths / scale points_x = (xmap - cx) / fx * points_z points_y = (ymap - cy) / fy * points_z points = np.stack([points_x, points_y, points_z], axis=-1) # Filter valid points mask = (points_z > 0) & (points_z < 1.0) points = points[mask].astype(np.float32) colors = colors[mask].astype(np.float32) # Run detection cfgs.max_gripper_width = 0.1 cfgs.checkpoint_path = './log/model.pth' anygrasp = AnyGrasp(cfgs) anygrasp.load_net() gg, cloud = anygrasp.get_grasp( points, colors, lims=[-0.19, 0.12, 0.02, 0.15, 0.0, 1.0], apply_object_mask=True, dense_grasp=False, collision_detection=True ) # Process results if len(gg) > 0: gg = gg.nms().sort_by_score() top_grasps = gg[0:20] print(f"Top grasp score: {top_grasps[0].score}") # Visualize grippers = gg.to_open3d_geometry_list() o3d.visualization.draw_geometries([*grippers, cloud]) ``` -------------------------------- ### PointNet2 SAModule Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Example usage of a PointNet2 Set Abstraction (SA) module for downsampling and feature learning in point clouds. ```python sa_module = PointnetSAModule(radius=0.2, nsample=32, mlp=[64, 64, 128]) new_xyz, new_points = sa_module(xyz, points) ``` -------------------------------- ### PointNet2 FPModule Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Example usage of a PointNet2 Feature Propagation (FP) module for upsampling features from a lower resolution to a higher resolution point cloud. ```python fp_module = PointnetFPModule(mlp=[128, 128, 64]) new_points = fp_module(xyz1, xyz2, points1, points2) ``` -------------------------------- ### PointNet2 QueryAndGroup Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Demonstrates the QueryAndGroup operation, which finds points within a specified radius and groups them for further processing in PointNet2 architectures. ```python query_and_group = QueryAndGroup(radius=0.3, nsample=48) indices, grouped_points = query_and_group(xyz, ref_xyz, points) ``` -------------------------------- ### Conv2d Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Shows an example of using Conv2d for processing point cloud features. The input tensor shape is (batch, features, npoints, nsample). ```python from pointnet2.pytorch_utils import Conv2d # Processing grouped point features: (batch, channels, npoints, nsample) conv = Conv2d( 64, 128, kernel_size=(1, 1), bn=True, activation=nn.ReLU() ) x = torch.randn(2, 64, 512, 32) # (batch, features, npoints, nsample) y = conv(x) # y: (2, 128, 512, 32) ``` -------------------------------- ### PointnetSAModuleVotes Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_modules.md Demonstrates initializing and using the PointnetSAModuleVotes module with specific MLP layers, sampling parameters, and pooling configurations. Shows how to pass input tensors and handle the output. ```python sa_votes = PointnetSAModuleVotes( mlp=[3+6, 64, 128], npoint=256, radius=0.2, nsample=64, pooling='rbf', sigma=0.1, normalize_xyz=True ) xyz = torch.randn(2, 2048, 3) features = torch.randn(2, 6, 2048) new_xyz, new_features, inds = sa_votes(xyz, features) # Can specify indices to override FPS # inds_override = torch.tensor([[0, 10, 20, ...]]) # new_xyz, new_features, inds = sa_votes(xyz, features, inds_override) ``` -------------------------------- ### PointNet2 SAModuleMSG Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Example usage of a PointNet2 Set Abstraction (SA) module with multiple message passing radii for richer feature extraction. ```python sa_module_msg = PointnetSAModuleMSG(radii=[0.1, 0.2, 0.4], nsample=[16, 32, 64], mlps=[[64, 64, 128], [128, 128, 256], [256, 256, 512]]) new_xyz, new_points = sa_module_msg(xyz, points) ``` -------------------------------- ### PointnetFPModule Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_modules.md Demonstrates how to use the PointnetFPModule to propagate features from a sparse set of points to a denser set. Ensure input tensors match the expected dimensions. ```python fp = PointnetFPModule(mlp=[256+64, 256, 256], bn=True) # Propagate from sparse set back to denser set unknown_xyz = torch.randn(2, 1024, 3) # Upsampled points known_xyz = torch.randn(2, 256, 3) # Sparse points from SA layer known_feats = torch.randn(2, 256, 256) # Features at sparse points unknown_feats = torch.randn(2, 64, 1024) # Skip connection features new_feats = fp(unknown_xyz, known_xyz, unknown_feats, known_feats) # new_feats: (2, 256, 1024) ``` -------------------------------- ### Verify License Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt This command-line tool verifies the integrity and validity of a license file. It is crucial for ensuring the license is correctly installed and active. ```bash ./license_checker -c ``` -------------------------------- ### set_bn_momentum_default Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Shows how to use set_bn_momentum_default to create a momentum setter and apply it to all Batch Normalization layers in a model. ```python from pointnet2.pytorch_utils import set_bn_momentum_default setter = set_bn_momentum_default(0.9) model.apply(setter) # Apply 0.9 momentum to all BN layers ``` -------------------------------- ### PointnetLFPModuleMSG Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_modules.md Illustrates the usage of PointnetLFPModuleMSG for propagating features from a sparse to a denser point cloud level. This module handles multi-scale information effectively. ```python lfp_msg = PointnetLFPModuleMSG( mlps=[[256, 64], [256, 96], [256, 128]], radii=[0.1, 0.2, 0.4], nsamples=[8, 16, 32], post_mlp=[64+96+128, 256, 256], bn=True ) # Propagate from sparse to denser level sparse_xyz = torch.randn(2, 256, 3) dense_xyz = torch.randn(2, 1024, 3) sparse_feats = torch.randn(2, 256, 256) dense_feats = torch.randn(2, 64, 1024) result = lfp_msg(dense_xyz, sparse_xyz, dense_feats, sparse_feats) # result: (2, 256, 1024) ``` -------------------------------- ### License Checker Binary File Structure Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/license_system.md This shows the file structure for the license checker binary, including the executable, version-specific license libraries, and an example license structure. ```text license_registration/ ├── license_checker # Executable for feature ID extraction ├── lib_cxx_versions/ # Version-specific license libraries │ ├── lib_cxx.cpython-36m-x86_64-linux-gnu.so │ ├── lib_cxx.cpython-37m-x86_64-linux-gnu.so │ └── ... └── sample_license/ # Example license structure ``` -------------------------------- ### Generate and Deploy Licenses for Multiple Robots Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/license_system.md On each robot, run the feature checker to get a unique ID. Submit this ID to obtain a separate license file for each robot and deploy it to the respective license directory. ```bash # On Robot 1 ./license_checker -f # Returns ID_1 # Deploy license_1.zip contents to grasp_detection/license/ # On Robot 2 ./license_checker -f # Returns ID_2 # Deploy license_2.zip contents to grasp_detection/license/ ``` -------------------------------- ### PointNet2 GroupAll Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Illustrates the GroupAll operation, which groups all points in a point cloud into a single group, typically used at the end of a feature extraction pipeline. ```python group_all = GroupAll() indices, grouped_points = group_all(xyz, points) ``` -------------------------------- ### Create and Use SharedMLP Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Instantiate a SharedMLP layer with specified channel dimensions, batch normalization, and activation. This example demonstrates applying the MLP to a random tensor representing point cloud data. ```python from pointnet2.pytorch_utils import SharedMLP # MLP: 64 -> 128 -> 256 -> 256 mlp = SharedMLP([64, 128, 256, 256], bn=True, activation=nn.ReLU(inplace=True)) x = torch.randn(2, 64, 512, 32) # (batch, channels, height, width) y = mlp(x) # y: (2, 256, 512, 32) ``` -------------------------------- ### Weight Initialization Strategies in PyTorch Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Demonstrates various weight initialization functions from torch.nn.init, such as Kaiming and Xavier initializations. Useful for setting initial weights in neural networks. ```python from torch.nn import init init_options = [ init.kaiming_normal_, # Default (ReLU networks) init.kaiming_uniform_, # Uniform variant init.xavier_normal_, # Xavier (sigmoid networks) init.xavier_uniform_, # Xavier uniform init.orthogonal_, # Orthogonal None # PyTorch default ] ``` -------------------------------- ### Initialize and Load AnyGraspTracker Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_tracking_api.md Initializes the AnyGraspTracker with specified configurations and loads the pre-trained model checkpoint. Ensure the checkpoint path is correct. ```python import numpy as np import open3d as o3d from PIL import Image from tracker import AnyGraspTracker class CameraInfo: def __init__(self, width, height, fx, fy, cx, cy, scale): self.width = width self.height = height self.fx = fx self.fy = fy self.cx = cx self.cy = cy self.scale = scale def create_point_cloud_from_depth(depth, camera, organized=True): xmap = np.arange(camera.width) ymap = np.arange(camera.height) xmap, ymap = np.meshgrid(xmap, ymap) points_z = depth / camera.scale points_x = (xmap - camera.cx) * points_z / camera.fx points_y = (ymap - camera.cy) * points_z / camera.fy points = np.stack([points_x, points_y, points_z], axis=-1) if not organized: points = points.reshape([-1, 3]) return points def load_frame(data_dir, index): colors = np.array(Image.open(f'{data_dir}/color_{index:03d}.png'), dtype=np.float32) / 255.0 depths = np.load(f'{data_dir}/depth_{index:03d}.npy') width, height = depths.shape[1], depths.shape[0] fx, fy = 927.17, 927.37 cx, cy = 651.32, 349.62 scale = 1000.0 camera = CameraInfo(width, height, fx, fy, cx, cy, scale) points = create_point_cloud_from_depth(depths, camera) mask = (points[:,:,2] > 0) & (points[:,:,2] < 1.5) points = points[mask] colors = colors[mask] return points, colors # Initialize tracker cfgs.checkpoint_path = './log/tracker_model.pth' cfgs.filter = 'oneeuro' cfgs.debug = False tracker = AnyGraspTracker(cfgs) tracker.load_net() ``` -------------------------------- ### Executing the Demo Script Source: https://github.com/graspnet/anygrasp_sdk/blob/main/grasp_detection/README.md This command executes the demo script. Ensure that `gsnet.so` is correctly placed and accessible, and that model weights are in the `log/` directory. ```bash sh demo.sh ``` -------------------------------- ### BNMomentumScheduler Constructor Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Initializes the BNMomentumScheduler with a model, a momentum lambda function, and an optional starting epoch and setter function. ```python BNMomentumScheduler( model: nn.Module, bn_lambda: Callable[[int], float], last_epoch: int = -1, setter=set_bn_momentum_default ) ``` -------------------------------- ### AnyGrasp Configuration Parameters Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/INDEX.md Configuration options for the AnyGrasp class, including model weights path, gripper dimensions, and filtering options. ```python cfgs.checkpoint_path # str: Model weights path cfgs.max_gripper_width # float: Max width (≤0.1m) cfgs.gripper_height # float: Jaw length (default 0.03m) cfgs.top_down_grasp # bool: Filter to top-down approaches cfgs.debug # bool: Enable visualization ``` -------------------------------- ### AnyGrasp Data Pipeline Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/data_structures.md This snippet shows the end-to-end process of loading sensor data, preparing it for the AnyGrasp model, detecting grasps, and visualizing the results. Ensure necessary libraries and configurations are imported and set up. ```python import torch import numpy as np from gsnet import AnyGrasp from graspnetAPI import GraspGroup # 1. Load and prepare point cloud raw_depth = load_depth_image('depth.png') raw_colors = load_color_image('color.png') points = depth_to_points(raw_depth, fx, fy, cx, cy, scale) colors = raw_colors / 255.0 # 2. Filter invalid points valid = (points[:, 2] > 0) & (points[:, 2] < 1.0) points = points[valid].astype(np.float32) colors = colors[valid].astype(np.float32) # 3. Run detection detector = AnyGrasp(cfgs) detector.load_net() gg, cloud = detector.get_grasp(points, colors, lims=workspace_lims) # 4. Post-process results if len(gg) > 0: gg = gg.nms().sort_by_score() # 5. Access results best_grasp = gg[0] print(f"Best grasp score: {best_grasp.score:.3f}") print(f"Position: {best_grasp.translation}") print(f"Width: {best_grasp.width:.3f}m") # 6. Visualize grippers = gg[0:10].to_open3d_geometry_list() visualize(cloud, grippers) else: print("No grasps detected") ``` -------------------------------- ### Get Machine Feature ID Source: https://github.com/graspnet/anygrasp_sdk/blob/main/license_registration/README.md Run this command to retrieve the unique feature ID of your machine, which is required for license application. ```bash ./license_checker -f ``` -------------------------------- ### AnyGrasp Constructor Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_detection_api.md Initializes the AnyGrasp object with a configuration namespace. The configuration object specifies parameters for grasp detection, such as model checkpoint path, gripper dimensions, and debug settings. ```APIDOC ## AnyGrasp Constructor ### Description Initializes the AnyGrasp object with a configuration namespace. The configuration object specifies parameters for grasp detection, such as model checkpoint path, gripper dimensions, and debug settings. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **cfgs** (Namespace) - Required - Configuration object with detection parameters. **Configuration Fields (cfgs):** - **checkpoint_path** (str) - Required - Path to model weights (.pth or checkpoint directory) - **max_gripper_width** (float) - Optional - Maximum gripper width in meters (clamped to ≤0.1m). Default: 0.1 - **gripper_height** (float) - Optional - Gripper height in meters. Default: 0.03 - **top_down_grasp** (bool) - Optional - If True, filter output to top-down approach grasps. Default: False - **debug** (bool) - Optional - Enable visualization and debug mode. Default: False ### Request Example ```python import argparse parser = argparse.ArgumentParser() parser.add_argument('--checkpoint_path', required=True, help='Model checkpoint path') parser.add_argument('--max_gripper_width', type=float, default=0.1, help='Maximum gripper width (<=0.1m)') parser.add_argument('--gripper_height', type=float, default=0.03, help='Gripper height') parser.add_argument('--top_down_grasp', action='store_true', help='Output top-down grasps.') parser.add_argument('--debug', action='store_true', help='Enable debug mode') cfgs = parser.parse_args() cfgs.max_gripper_width = max(0, min(0.1, cfgs.max_gripper_width)) anygrasp = AnyGrasp(cfgs) ``` ### Response None ### Errors None explicitly documented for constructor. ``` -------------------------------- ### AnyGraspTracker Constructor Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_tracking_api.md Initializes the AnyGraspTracker with a configuration object. The configuration specifies parameters for temporal smoothing and model loading. ```APIDOC ## AnyGraspTracker(cfgs) ### Description Initializes the AnyGraspTracker with a configuration object. The configuration specifies parameters for temporal smoothing and model loading. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **cfgs** (Namespace) - Required - Configuration object with tracking parameters. ### Configuration Fields (cfgs): - **checkpoint_path** (str) - Required - Path to tracking model weights (.pth) - **filter** (str) - Optional - Temporal smoothing filter type. Options: 'oneeuro' (default), 'kalman', 'none'. - **debug** (bool) - Optional - Enable visualization during tracking. Default: False ### Request Example ```python import argparse parser = argparse.ArgumentParser() parser.add_argument('--checkpoint_path', required=True, help='Model checkpoint path') parser.add_argument('--filter', type=str, default='oneeuro', help='Filter to smooth grasp parameters [oneeuro/kalman/none]') parser.add_argument('--debug', action='store_true', help='Enable visualization') cfgs = parser.parse_args() tracker = AnyGraspTracker(cfgs) ``` ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### PointNet2 CylinderQueryAndGroup Example Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Shows the CylinderQueryAndGroup operation, which queries points within a cylindrical region, useful for specific geometric feature extraction. ```python cylinder_query = CylinderQueryAndGroup(radius=0.2, height=0.4, nsample=32) indices, grouped_points = cylinder_query(xyz, ref_xyz, points) ``` -------------------------------- ### Get Grasp Predictions Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Performs grasp detection using the loaded AnyGrasp model. This method returns potential grasp poses and scores. ```python grasps = grasp_net.get_grasp(point_cloud) ``` -------------------------------- ### Load Network for AnyGraspTracker Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Initializes the AnyGraspTracker by loading the specified neural network model. This prepares the tracker for subsequent updates. ```python tracker = AnyGraspTracker() tracker.load_net(model_path) ``` -------------------------------- ### Conv3d Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Illustrates the usage of Conv3d for 3D feature processing. The input tensor shape is (batch, channels, depth, height, width). ```python from pointnet2.pytorch_utils import Conv3d # 3D feature processing conv = Conv3d( 64, 128, kernel_size=(3, 3, 3), padding=(1, 1, 1), bn=True ) x = torch.randn(2, 64, 16, 16, 16) # (batch, channels, d, h, w) y = conv(x) # y: (2, 128, 16, 16, 16) ``` -------------------------------- ### AnyGrasp SDK File Structure Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/README.md This snippet shows the directory structure of the AnyGrasp SDK, outlining the location of key documentation files. ```text output/ ├── INDEX.md # Navigation guide (start here) ├── README.md # This file ├── overview.md # Project overview ├── grasp_detection_api.md # Detection API ├── grasp_tracking_api.md # Tracking API ├── pointnet2_modules.md # PointNet2 modules ├── pointnet2_utils.md # PointNet2 utilities ├── pointnet2_pytorch_utils.md # PyTorch wrappers ├── data_structures.md # Data format reference ├── license_system.md # Licensing details └── integration_guide.md # Integration patterns ``` -------------------------------- ### Conv1d Example Usage Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_pytorch_utils.md Demonstrates how to instantiate and use the Conv1d class for 1D convolution on a tensor. Input tensor shape is (batch, channels, length). ```python from pointnet2.pytorch_utils import Conv1d conv = Conv1d(64, 128, kernel_size=3, padding=1, bn=True, activation=nn.ReLU()) x = torch.randn(2, 64, 1024) # (batch, channels, length) y = conv(x) # y: (2, 128, 1024) ``` -------------------------------- ### AnyGrasp SDK File Organization Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/INDEX.md This snippet shows the directory structure of the AnyGrasp SDK, highlighting the main modules for grasp detection, grasp tracking, and utility functions. ```text anygrasp_sdk/ ├── grasp_detection/ # Detection module │ ├── demo.py │ ├── gsnet.*.so │ ├── lib_cxx.*.so │ └── example_data/ ├── grasp_tracking/ # Tracking module │ ├── demo.py │ ├── tracker.*.so │ ├── lib_cxx.*.so │ └── example_data/ ├── pointnet2/ # Pure Python utilities │ ├── pointnet2_modules.py │ ├── pointnet2_utils.py │ ├── pytorch_utils.py │ └── setup.py ├── license_registration/ # Licensing utilities │ ├── license_checker │ └── lib_cxx_versions/ ├── README.md └── requirements.txt ``` -------------------------------- ### SharedMLP Layer Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Example of a Shared Multi-Layer Perceptron (MLP) layer, commonly used in PointNet architectures for applying the same weights across different points. ```python shared_mlp = SharedMLP([64, 64, 128]) output_features = shared_mlp(input_features) ``` -------------------------------- ### Load Tracking Model Weights Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_tracking_api.md Demonstrates loading the tracking model weights for the AnyGraspTracker. This method must be called after initializing the tracker and before calling the update method. ```python tracker = AnyGraspTracker(cfgs) tracker.load_net() # Must be called before update() ``` -------------------------------- ### Temporal Grasp Tracking with AnyGraspTracker Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/README.md This snippet demonstrates how to use AnyGraspTracker for temporal grasp tracking. Import the tracker, load weights, and then call the update method with point cloud data. ```python # Tracking from tracker import AnyGraspTracker tracker = AnyGraspTracker(cfgs) tracker.load_net() # Load weights target_gg, curr_gg, ids, _ = tracker.update(points, colors, grasp_ids) ``` -------------------------------- ### AnyGrasp SDK Repository Structure Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/overview.md Illustrates the directory layout of the AnyGrasp SDK, highlighting the organization of code for grasp detection, grasp tracking, PointNet++ modules, and license registration. ```text anygrasp_sdk/ ├── grasp_detection/ # Detection demo and binary modules │ ├── demo.py # Example usage of AnyGrasp detector │ ├── gsnet_versions/ # Version-specific .so files (Python 3.6-3.9) │ └── example_data/ # Sample point cloud and image data ├── grasp_tracking/ # Tracking demo and binary modules │ ├── demo.py # Example usage of AnyGraspTracker │ └── example_data/ # Tracking sequence data ├── pointnet2/ # Pure Python PointNet++ layers │ ├── pointnet2_modules.py # Set abstraction and propagation layers │ ├── pointnet2_utils.py # CUDA operations (ball query, grouping, etc.) │ ├── pytorch_utils.py # Conv/FC layer wrappers │ └── setup.py # C++ extension building ├── license_registration/ # License system │ ├── license_checker # Binary for feature ID extraction │ └── lib_cxx_versions/ # Version-specific .so files └── requirements.txt # External dependencies ``` -------------------------------- ### Fully Connected Layer Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Example of a standard Fully Connected (FC) layer, used for transforming features into a desired output dimension, often in classification or regression heads. ```python fc_layer = FC(in_features=512, out_features=10) output = fc_layer(input_features) ``` -------------------------------- ### Configuration Objects Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/MANIFEST.txt Documentation for configuration objects. ```APIDOC ## Configuration Objects ### AnyGrasp Configuration - Description: Configuration settings for AnyGrasp. - Options: Detailed specifications available in reference tables. ### AnyGraspTracker Configuration - Description: Configuration settings for AnyGraspTracker. - Options: Detailed specifications available in reference tables. ``` -------------------------------- ### Real-time Detection Loop with AnyGrasp Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/integration_guide.md Continuously detects grasps from RGB and depth streams, processes them, and optionally visualizes the results. Ensure your camera intrinsics are calibrated and the workspace limits are set appropriately for your environment. The visualization requires Open3D. ```python import cv2 import numpy as np import open3d as o3d from gsnet import AnyGrasp def detection_loop(rgb_stream, depth_stream, checkpoint_path, visualization=True): """ Continuous grasp detection from sensor streams. Args: rgb_stream: Generator yielding RGB images depth_stream: Generator yielding depth images checkpoint_path: Path to model weights visualization: Show Live visualizations """ # Setup import argparse parser = argparse.ArgumentParser() parser.add_argument('--checkpoint_path', default=checkpoint_path) parser.add_argument('--max_gripper_width', type=float, default=0.1) parser.add_argument('--gripper_height', type=float, default=0.03) cfgs = parser.parse_args([]) detector = AnyGrasp(cfgs) detector.load_net() # Camera intrinsics (calibrate for your camera) fx, fy = 927.17, 927.37 cx, cy = 651.32, 349.62 scale = 1000.0 workspace_lims = [-0.2, 0.2, -0.15, 0.15, 0.0, 1.0] if visualization: vis = o3d.visualization.Visualizer() vis.create_window(height=720, width=1280) frame_idx = 0 while True: try: # Get frame color = next(rgb_stream) depth = next(depth_stream) # Convert to point cloud height, width = depth.shape xmap, ymap = np.meshgrid(np.arange(width), np.arange(height)) color_f32 = color.astype(np.float32) / 255.0 points_z = depth.astype(np.float32) / scale points_x = (xmap - cx) / fx * points_z points_y = (ymap - cy) / fy * points_z mask = (points_z > 0.01) & (points_z < 2.0) points = np.stack([points_x, points_y, points_z], axis=-1)[mask].astype(np.float32) colors = color_f32[mask].astype(np.float32) # Detect gg, cloud = detector.get_grasp( points, colors, lims=workspace_lims, apply_object_mask=True, collision_detection=True ) # Process if len(gg) > 0: gg = gg.nms().sort_by_score() top_grasp = gg[0] print(f"Frame {frame_idx}: Best grasp score {top_grasp.score:.3f}") if visualization: # Update visualization vis.clear() # Add point cloud o3d_cloud = o3d.geometry.PointCloud() o3d_cloud.points = o3d.utility.Vector3dVector(points) o3d_cloud.colors = o3d.utility.Vector3dVector(colors) vis.add_geometry(o3d_cloud) # Add top grasps for i, grasp in enumerate(gg[0:5]): gripper = grasp.to_open3d_geometry() vis.add_geometry(gripper) vis.poll_events() vis.update_renderer() else: print(f"Frame {frame_idx}: No grasps detected") frame_idx += 1 except StopIteration: break if visualization: vis.destroy_window() ``` -------------------------------- ### Initialize GraspGroup Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/data_structures.md Create a new GraspGroup instance. Can be initialized with existing grasp data. ```python from graspnetAPI import GraspGroup # Initialize an empty GraspGroup gg = GraspGroup() # Initialize with existing grasp data (e.g., from a numpy array) grasps_data = [...] # Assume grasps_data is a list or array of grasp information gg = GraspGroup(grasps=grasps_data) ``` -------------------------------- ### Detection Module File Structure Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/license_system.md This outlines the expected file structure for the detection module, including the detection binary, license checker library, and the license directory with its components. ```text grasp_detection/ ├── gsnet.so # Detection binary ├── lib_cxx.so # License checker library └── license/ # License directory (user creates) ├── licenseCfg.json ├── user_name.public_key ├── user_name.signature └── user_name.lic ``` -------------------------------- ### Get Machine Feature ID for Licensing Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/overview.md Command to extract the machine feature ID required for AnyGrasp SDK license registration. This ID is unique to the machine and used in the licensing process. ```bash license_checker -f ``` -------------------------------- ### Optimized Grasp Detection Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/integration_guide.md Use this class for faster grasp detection by reducing input resolution and skipping computationally expensive steps like collision detection. Ensure PyTorch and NumPy are installed. ```python import numpy as np import torch class OptimizedDetector: def __init__(self, checkpoint_path, device='cuda:0'): """Initialize with optimization.""" import argparse # Clear GPU cache if 'cuda' in device: torch.cuda.empty_cache() parser = argparse.ArgumentParser() parser.add_argument('--checkpoint_path', default=checkpoint_path) parser.add_argument('--max_gripper_width', type=float, default=0.1) cfgs = parser.parse_args([]) self.detector = AnyGrasp(cfgs) self.detector.load_net() self.device = device def detect_fast(self, depth, color, fx, fy, cx, cy, scale=1000.0, downsample=1, max_points=50000): """ Fast detection with reduced input size. Args: downsample: Downsample factor (2 = quarter resolution) max_points: Limit total points to reduce memory """ # Downsample if downsample > 1: depth = depth[::downsample, ::downsample] color = color[::downsample, ::downsample] fx /= downsample fy /= downsample cx /= downsample cy /= downsample # Convert to point cloud height, width = depth.shape xmap, ymap = np.meshgrid(np.arange(width), np.arange(height)) color_f32 = color.astype(np.float32) / 255.0 points_z = depth.astype(np.float32) / scale points_x = (xmap - cx) / fx * points_z points_y = (ymap - cy) / fy * points_z mask = (points_z > 0.01) & (points_z < 2.0) points = np.stack([points_x, points_y, points_z], axis=-1)[mask].astype(np.float32) colors = color_f32[mask].astype(np.float32) # Limit point count if len(points) > max_points: indices = np.random.choice(len(points), max_points, replace=False) points = points[indices] colors = colors[indices] # Run detection (skip expensive steps) gg, _ = self.detector.get_grasp( points, colors, apply_object_mask=True, dense_grasp=False, # Don't use dense mode collision_detection=False # Skip collision detection for speed ) return gg if len(gg) > 0 else [] ``` -------------------------------- ### PointnetSAModuleMSG Constructor Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/pointnet2_modules.md Initializes a multi-scale set abstraction layer. Use this when you need to capture features at different neighborhood scales simultaneously. ```python class PointnetSAModuleMSG(_PointnetSAModuleBase) PointnetSAModuleMSG( *, npoint: int, radii: List[float], nsamples: List[int], mlps: List[List[int]], bn: bool = True, use_xyz: bool = True, sample_uniformly: bool = False ) ``` -------------------------------- ### Update Grasp Tracking with Frame Data Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/grasp_tracking_api.md Example of updating the AnyGraspTracker with new frame data, including point cloud, colors, and grasp IDs from the previous frame. This method performs frame-to-frame association and temporal smoothing. ```python # Assuming 'tracker' is an initialized AnyGraspTracker instance # Assuming 'points', 'colors', and 'grasp_ids' are numpy arrays # target_gg, curr_gg, target_grasp_ids, corres_preds = tracker.update(points, colors, grasp_ids) ``` -------------------------------- ### Command Line License Check Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/license_system.md Use the license_checker executable to verify license validity before running applications. Ensure the license configuration file is correctly specified. ```bash ./license_checker -c license/licenseCfg.json ``` -------------------------------- ### AnyGraspTracker Configuration Parameters Source: https://github.com/graspnet/anygrasp_sdk/blob/main/_autodocs/INDEX.md Configuration options for the AnyGraspTracker class, specifying model path, filtering method, and debug mode. ```python cfgs.checkpoint_path # str: Model weights path cfgs.filter # str: 'oneeuro', 'kalman', or 'none' cfgs.debug # bool: Enable visualization ```