### motion_config YAML example

Source: https://genmanip.com/datagen/advancedgen/generation-config

Example of `motion_config` with distances for pre/post grasp and place. `null` values skip the corresponding motion primitive.

```yaml
motion_config:
  pre_grasp_distance: 0.08
  post_grasp_distance: 0.16
  pre_place_distance: null
  post_place_distance: 0.08
```

--------------------------------

### Start server and run client for GenManipBench

Source: https://genmanip.com/evaluation/basic_usage

Shows the two commands to start the evaluation server and run the client for a benchmark. The server command downloads assets and starts the server; the client command runs the standalone client with worker ID 0, letting the server create an Isaac Sim test instance.

```bash
# Download assets and start server
python ray_eval_server.py -cfg GenManipSuite/GenManip-Package-Basic


# Run client
python standalone_tools/client.py --worker_ids 0 # Let the server create an Isaac Sim test instance
```

--------------------------------

### Start Server for GenManip Scaling Pick-and-Place Benchmark

Source: https://genmanip.com/evaluation/benchmarks

Starts the evaluation server for the GenManip Scaling Pick-and-Place Benchmark using the specified configuration. Run this before executing the client evaluation.

```bash
python ray_eval_server.py -cfg GenManipSuite/GenManip-Package-OOC_Bench
```

--------------------------------

### Install rerun.io support (optional)

Source: https://genmanip.com/introduction/installation

Installs rerun-sdk version 0.28.4 and pins numpy to 1.26.4 because rerun-sdk installs numpy>2. This is optional and used for synchronized visualization of state/action and frames.

```bash
pip install rerun-sdk==0.28.4
pip install numpy==1.26.4 # rerun-sdk installs numpy>2; pin back to 1.26.4
```

--------------------------------

### Install required Python packages

Source: https://genmanip.com/introduction/installation

Runs the installation script and installs the genmanip_client package. The script may need modification if not using CUDA 12.1. The genmanip_client package contains only common dependencies and is used to communicate with the GenManip Server.

```bash
bash scripts/install.sh # modify the script if you are not using CUDA 12.1


# Install genmanip_client (your model environment should also install this)
# It contains only common deps and is unlikely to cause conflicts; it helps you communicate with the GenManip Server.
cd standalone_tools/packages/genmanip_client
pip install -e .
cd ../../..
```

--------------------------------

### Example .env file for API keys

Source: https://genmanip.com/baselines/modular-framework

Example environment file content for configuring API keys and base URLs. Includes proxy support with X_URL and X_API_KEY.

```dotenv
BASE_URL=https://api.openai.com/v1
API_KEY=sk-proj-1234567890
X_URL=http://api.XXX.com/v1
X_API_KEY=sk-proj-1234567890
```

--------------------------------

### Start Server for GenManip Tabletop10 Benchmark

Source: https://genmanip.com/evaluation/benchmarks

Starts the evaluation server for the GenManip Tabletop10 Benchmark using the specified configuration. This benchmark is part of the IROS 2025 Challenge.

```bash
python ray_eval_server.py -cfg GenManipSuite/GenManip-Package-TableTop10Aloha
```

--------------------------------

### Start evaluation server and download assets

Source: https://genmanip.com/introduction/quickstart

Starts the evaluation server with ray_eval_server.py, which automatically downloads assets from the specified GenManip Package repo id.

```bash
python ray_eval_server.py -cfg GenManipSuite/GenManip-Package-Basic

```

--------------------------------

### Install MiniConda

Source: https://genmanip.com/introduction/installation

Commands to download and install the latest MiniConda on Linux, as required by the prerequisites.

```bash
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm ~/miniconda3/miniconda.sh
```

--------------------------------

### Run Modular Framework with run.py

Source: https://genmanip.com/baselines/modular-framework

Starts the Modular Framework services. Run from the deploy_sim directory after services are up.

```bash
cd baselines/modular_framework/deploy_sim
python run.py -r 10000 -s 10001 --model_name gpt-4o --P2P True --CtoF False
```

--------------------------------

### Evaluation Usage Workflow Commands

Source: https://genmanip.com/introduction/quickstart

Run these commands in order to start the server and then run the client. Replace 'your-assets-package' with your actual package name.

```bash
python ray_eval_server.py your-assets-package
python standalone_tools/client.py --worker_ids 0
```

--------------------------------

### Install dependencies via git clone and pip

Source: https://genmanip.com/baselines/modular-framework

Clones the SAM2 repository and installs required Python packages. Run after activating the conda environment.

```bash
git clone https://github.com/facebookresearch/sam2.git && cd sam2
pip install -e .
pip install huggingface_hub flask opencv-python python-dotenv
```

--------------------------------

### Object Config Example for Scaling Data Generation

Source: https://genmanip.com/datagen/simplegen

This YAML snippet shows an example object config that loads 50 objects and 50 containers from respective folders, enabling random activation of one object and one container per sample for the 'place object into container' task. It replaces the minimal config's object mappings.

```yaml
"0":
  filter_rule: []
  max_cached_num: 50
  option: []
  path: object_usds/objects
  type: load_object_from_path
"1":
  filter_rule: []
  max_cached_num: 50
  option: []
  path: object_usds/containers
  type: load_object_from_path
```

--------------------------------

### Verify installation with render test

Source: https://genmanip.com/introduction/installation

Runs the demo script to load a basic scene and render an image. If successful, it saves a rendered image at tmp/test.png, indicating Isaac Sim is installed and configured correctly.

```bash
python demos/example_case/test_demoscene.py
```

--------------------------------

### Create and activate Conda environment

Source: https://genmanip.com/introduction/installation

Creates a Conda environment named 'genmanip' with Python 3.10 and activates it, as recommended by the installation guide.

```bash
conda create -n genmanip python=3.10
conda activate genmanip
```

--------------------------------

### Launch AnyGrasp server command

Source: https://genmanip.com/datagen/anygrasp

Command to start the AnyGrasp server. The -p flag sets the listening port and -d selects the GPU. The server can be exposed on the local network for multiple clients.

```bash
python anygrasp_sdk/grasp_detection/server.py -p 5000 -d 0
```

--------------------------------

### Download scene assets

Source: https://genmanip.com/introduction/installation

Logs into Hugging Face and downloads the basic scene assets needed for installation verification.

```bash
huggingface-cli login


# Download assets you need
python standalone_tools/download_assets.py --dataset basic # assets for installation verification
```

--------------------------------

### Install GR1 dependencies

Source: https://genmanip.com/baselines/learning-based-framework

Installs required Python packages for the GR1 environment, including the requirements file, the OpenAI CLIP repository, and the roboticstoolbox-python package. Run these commands after activating the 'gr1' environment.

```bash
pip install -r requirements.txt
pip install git+https://github.com/openai/CLIP.git
pip3 install roboticstoolbox-python
```

--------------------------------

### Extended goal YAML example

Source: https://genmanip.com/datagen/advancedgen/generation-config

Example of an extended goal configuration with fields like `obj1_uid`, `obj2_uid`, `position`, `fixed_position`, `fixed_grasp_config`, `ignored_uid`, `arm`, and `motion_config`.

```yaml
obj1_uid:
- 'bread4'
obj2_uid:
- 'beef'
position:
- top
fixed_position: true
fixed_position_config:
  translation: [0.0, 0.02, -0.01]
  orientation: [0.924, -0.383, 0.0, 0.0]
force_fixed_grasp: true
fixed_grasp_config:
  translation: [0.0, 0.0, -0.03]
allow_fixed_grasp: true
mesh_top_only: true
ignored_uid:
- 'plate'
- 'bread2'
arm: auto
motion_config:
  pre_grasp_distance: 0.08
  post_grasp_distance: 0.16
  pre_place_distance: null
  post_place_distance: 0.08
```

--------------------------------

### fixed_global_range placement example

Source: https://genmanip.com/datagen/advancedgen/layout-config

Example of a fixed_global_range placement for an object named 'bread', setting a fixed XY position and zero rotation range.

```yaml
"bread":
  pos_x: -0.17217
  pos_y: -0.45642
  random_range_angle: 0
  type: fixed_global_range
```

--------------------------------

### random_custom_tableset layout_config example

Source: https://genmanip.com/datagen/advancedgen/layout-config

Example of a layout_config using type random_custom_tableset with custom_tableset defining per-object placement for apple, plate, and banana using global_range placement.

```yaml
layout_config:
  ignored_objects: []
  type: random_custom_tableset
  in_order: true  # whether to place objects in the order of custom_tableset
  custom_tableset:
    "apple":
      type: global_range
      random_range_angle: 360
      random_range_h: 0.7
      random_range_w: 0.4
      random_range_x: -0.2
      random_range_y: -0.35
    "plate":
      type: global_range
      random_range_angle: 360
      random_range_h: 0.7
      random_range_w: 0.4
      random_range_x: -0.2
      random_range_y: -0.35
    "banana":
      type: global_range
      random_range_angle: 360
      random_range_h: 20
      random_range_w: 20
      random_range_x: -10
      random_range_y: -10
```

--------------------------------

### fixed_grasp_config YAML example

Source: https://genmanip.com/datagen/advancedgen/generation-config

Configuration for a fixed grasp pose, specifying relative translation and absolute orientation as a quaternion in wxyz order.

```yaml
fixed_grasp_config:
  translation: [0.0, 0.0, -0.03] # relative translation
  orientation: [0.924, -0.383, 0.0, 0.0] # absolute orientation, quaternion wxyz
```

--------------------------------

### Random Environment Configuration Example

Source: https://genmanip.com/datagen/advancedgen/domain-randomization

YAML configuration block showing all available randomization options for the environment, including wall, HDR lighting, robot base and end-effector pose, table and wall textures, and per-camera noise parameters. The `camera_randomization` section specifies `max_translation_noise` in meters and `max_orientation_noise` in degrees for each camera.

```yaml
random_environment:
  has_wall: false
  hdr: false
  robot_base_position: false
  robot_eepose: false
  table_texture: false
  table_type: false
  wall_texture: false
  camera_randomization:
    realsense:
      max_translation_noise: 0.02
      max_orientation_noise: 2.5
    obs_camera:
      max_translation_noise: 0.05
      max_orientation_noise: 10.0
```

--------------------------------

### centric_range placement example

Source: https://genmanip.com/datagen/advancedgen/layout-config

Example of a centric_range placement for an object named 'box', defining a bounding box and rotation range.

```yaml
"box":
  w: 0.05
  h: 0.05
  angle: 0
  angle_bilateral: false
  type: centric_range
```

--------------------------------

### Create conda environment and activate it

Source: https://genmanip.com/baselines/modular-framework

Creates a conda environment named 'prompt' with Python 3.10 and activates it. Use before installing dependencies.

```bash
conda create -n prompt python=3.10
conda activate prompt
```

--------------------------------

### fixed_position_config YAML example

Source: https://genmanip.com/datagen/advancedgen/generation-config

Configuration for a fixed placement position, specifying relative translation and absolute orientation as a quaternion in wxyz order.

```yaml
fixed_position_config:
  translation: [0.0, 0.02, -0.01] # relative translation
  orientation: [0.924, -0.383, 0.0, 0.0] # absolute orientation, quaternion wxyz
```

--------------------------------

### Object Config example with three Meta Object types

Source: https://genmanip.com/datagen/advancedgen/object-config

Shows the structure of object_config with three Meta Objects: apple (existed_object), banana (load_object_from_path), and container (add_additional_object_from_path). Each value is a dictionary with type-specific keys.

```yaml
object_config:
  apple:
    type: existed_object
    uid_list:
    - 1fdc84a7be2c4348b281490c89d76062
  banana:
    filter_rule: []
    max_cached_num: 50
    option: []
    path: object_usds/bananas
    type: load_object_from_path
  container:
    option: []
    path: object_usds/objaverse_usd/f8a2cdc9970846da95585a428697d173.usd
    type: add_additional_object_from_path
```

--------------------------------

### GenManip-style camera config YAML

Source: https://genmanip.com/datagen/advancedgen/domain-randomization

Example GenManip-style camera configuration. Uses separate top-level parameters like focal_length, horizontal_aperture, and position. The camera is placed under /Cameras<prim_path> if exists is false. Includes rendering options for distance, semantic, bbox2d, bbox3d, and motion vectors.

```yaml
camera_name:
  clipping_range_max: 10000.0
  clipping_range_min: 0.001
  exists: false
  focal_length: 5.0
  frequency: 60
  horizontal_aperture: 10.0
  name: camera_name
  orientation:
  - -0.0071865058262247095
  - -0.5792785229595211
  - -0.005107428795172445
  - 0.815081996576386
  position:
  - 0.28070902824401855
  - -0.02326910011470318
  - 1.6857764720916748
  prim_path: /camera_name
  resolution:
  - 1280
  - 720
  vertical_aperture: 5.625
  with_distance: true
  with_semantic: true
  with_bbox2d: true
  with_bbox3d: true
  with_motion_vector: true

```

--------------------------------

### Run evaluation with eval.py

Source: https://genmanip.com/baselines/modular-framework

Starts the evaluation process in a separate terminal. Uses the specified config file and ports.

```bash
python eval.py -cfg configs/tasks/genmanipbench.yml -r 10001 -s 10000
```

--------------------------------

### Simbox-style camera config YAML

Source: https://genmanip.com/datagen/advancedgen/domain-randomization

Example Simbox-style camera configuration. Uses a nested camera_params array [fx, fy, cx, cy], as well as additional simulation-specific parameters like pixel_size, f_number, and focus_distance. Note the orientation is scalar-first [w, x, y, z].

```yaml
obs_camera:
  exists: false
  frequency: 60
  name: obs_camera
  camera_params:
  - 605.451
  - 605.137
  - 320.778
  - 255.816
  orientation: [0.83980864, 0.4807688, -0.10643996, -0.2285899]
  position:
  - -0.54772
  - -0.79264
  - 1.64032
  prim_path: /obs_camera
  resolution:
  - 640
  - 480
  pixel_size: 3.0
  f_number: 2.0
  focus_distance: 0.5
  with_distance: false
  with_semantic: false
  with_bbox2d: false
  with_bbox3d: false
  with_motion_vector: false
  camera_axes: usd

```

--------------------------------

### Auto-generated Instruction Template

Source: https://genmanip.com/datagen/advancedgen/domain-randomization

Example of an auto-generated instruction string when `rewrite_instruction` is enabled, using placeholders like `{obj10_name}` and `{position0}` to reference objects and positions.

```text
put {obj10_name} to the {position0} of {obj20_name}, put {obj11_name} to the {position1} of {obj21_name} and put {obj12_name} to the {position2} of {obj22_name}
```

--------------------------------

### Create Conda environment for GR1

Source: https://genmanip.com/baselines/learning-based-framework

Creates and activates a Conda environment named 'gr1' with Python 3.10, as recommended for training and evaluating the GR1 model. Use this before installing dependencies.

```bash
conda create -n gr1 python=3.10
conda activate gr1
```

--------------------------------

### Minimal GenManip YAML Config

Source: https://genmanip.com/datagen/simplegen

Minimal YAML configuration for GenManip data generation and evaluation. It defines a demonstration config and an evaluation config, each with domain randomization, generation settings, object definitions, and robot setup. The camera configuration references a config file relative to the project root.

```yaml

demonstration_configs:
- domain_randomization:
    cameras:
      config_path: configs/cameras/fixed_camera.yml
      type: fixed
    random_environment:
      has_wall: false
      hdr: false
      robot_base_position: false
      robot_eepose: false
      table_texture: false
      table_type: false
      wall_texture: false
    rewrite_instruction: false
  generation_config:
    action_path:
      mode: auto
      robot: 0
    articulation: []
    goal:
    - - obj1_uid:
        - '0'
        obj2_uid:
        - '1'
        position:
        - top
        fixed_position: true
        allow_fixed_grasp: true
        force_fixed_grasp: true
    mode: manual
    planner: curobo
  instruction: put the banana on the top of the vintage rusty frying pan.
  layout_config:
    ignored_objects: []
    type: None
  mode: manual
  num_episode: 10
  object_config:
    '0':
      type: existed_object
      uid_list:
      - fb1b6fc41f7e49adbf467e5e5988d190
    '1':
      type: existed_object
      uid_list:
      - 1fdc84a7be2c4348b281490c89d76062
  preprocess_config:
  - config:
      config: default
      type: convexDecomposition
    type: collider
  - type: ccd
  robots:
  - config:
      gripper_type: panda_hand
    type: franka
  table_uid: aa49db8a801d402dac6cf1579536502c
  task_name: Banana/banana_plate_4035_none-wow-wot-woh-now
  usd_name: scene_usds/debug_scenes/banana_plate_scenes/base
evaluation_configs:
- domain_randomization:
    cameras:
      config_path: configs/cameras/fixed_camera.yml
      type: fixed
    random_environment:
      has_wall: false
      hdr: false
      robot_base_position: false
      robot_eepose: false
      table_texture: false
      table_type: false
      wall_texture: false
    rewrite_instruction: false
  generation_config:
    action_path:
      mode: auto
      robot: 0
    articulation: []
    goal:
    - - obj1_uid:
        - '0'
        obj2_uid:
        - '1'
        position:
        - top
        fixed_position: true
        allow_fixed_grasp: true
        force_fixed_grasp: true
    mode: manual
    planner: curobo
  instruction: put the banana on the top of the vintage rusty frying pan.
  layout_config:
    ignored_objects: []
    type: None
  mode: manual
  num_test: 10
  object_config:
    '0':
      type: existed_object
      uid_list:
      - fb1b6fc41f7e49adbf467e5e5988d190
    '1':
      type: existed_object
      uid_list:
      - 1fdc84a7be2c4348b281490c89d76062
  preprocess_config:
  - config:
      config: default
      type: convexDecomposition
    type: collider
  - type: ccd
  robots:
  - config:
      gripper_type: panda_hand
    type: franka
  table_uid: aa49db8a801d402dac6cf1579536502c
  task_name: Banana/banana_plate_4035_none-wow-wot-woh-now
  usd_name: scene_usds/debug_scenes/banana_plate_scenes/base

```

--------------------------------

### preprocess_config: rigid_body

Source: https://genmanip.com/datagen/advancedgen/misc

Adds rigid body properties so objects respond to physics. Useful when creating scenes manually and needing automatic physics setup.

```yaml
preprocess_config:
- type: rigid_body

```

--------------------------------

### Integrate model inference into client.py main loop

Source: https://genmanip.com/evaluation/basic_usage

Shows how to modify the main loop of standalone_tools/client.py to integrate your own model's inference logic. The script is completely standalone with no external dependencies. The example creates workers on the server, wraps the eval loop in try-finally for cleanup, and calls model.inference on processed observations.

```python
...
if __name__ == "__main__":
    args = parse_args()
    host = args.host
    port = args.port
    worker_ids = args.worker_ids
    base_url = f"http://{host}:{port}"


    # Create workers on server here, make sure they are created before stepping
    client = EvalClient(base_url, worker_ids)
    print(f"Created workers {worker_ids} on server {base_url}.")


    # wrap the eval loop in a try-finally to ensure cleanup
    try:


        obs = client.reset()


        while True:


            action = {
                i: fake_action(args.arm_type, args.gripper_type, args.control_type)
                i: model.inference(process_data(obs[i])) # Or use batch inference
                for i in worker_ids
            }


            start = time.time()
            obs, done = client.step(action)
            print(f"workers {worker_ids} Step time: {time.time() - start:.4f} seconds")


            if done:
                # finished all evaluations
                break
            if obs is None:
                break
            if obs[worker_ids[0]]["obs"]["reset"]:  # type: ignore
                # model.reset()
                model.reset()
                pass
    finally:
        client.kill_workers()
        print("Client cleaned.")
```

--------------------------------

### preprocess_config: collider with convexDecomposition

Source: https://genmanip.com/datagen/advancedgen/misc

Adds collision shapes via convex decomposition to existing meshes. Useful when creating scenes manually and needing automatic physics setup.

```yaml
preprocess_config:
- config:
    config: default
    type: convexDecomposition
  type: collider

```

--------------------------------

### Render data samples with render.py

Source: https://genmanip.com/datagen/diy

Runs render.py to iterate over each dictionary in demonstration_configs and render num_episode data samples. Use this command after generating data.

```bash
# Iterate over each dictionary in demonstration_configs and render num_episode data samples
python render.py -cfg configs/tasks/xxx.yml
```

--------------------------------

### Benchmark Creation Workflow Commands

Source: https://genmanip.com/introduction/quickstart

Run these commands in order to download assets, generate test cases, and package/upload to Hugging Face. Replace 'your-assets-package', 'your-task-name', and 'your-task-package' with your actual names.

```bash
python standalone_tools/download_assets.py --dataset your-assets-package
python demogen.py -cfg configs/tasks/your-task-name.yml --eval
python standalone_tools/collect_benchmark_assets.py --asset_path saved/tasks/your-task-name --dataset_id your-task-package --upload_to_huggingface
```

--------------------------------

### Generate data samples with demogen.py

Source: https://genmanip.com/datagen/diy

Runs demogen.py to iterate over each dictionary in demonstration_configs and generate num_episode data samples. Use this command after writing the Config file.

```bash
# Iterate over each dictionary in demonstration_configs and generate num_episode data samples
python demogen.py -cfg configs/tasks/xxx.yml
```

--------------------------------

### Run Evaluation for GenManip Scaling Pick-and-Place Benchmark

Source: https://genmanip.com/evaluation/benchmarks

Runs the client evaluation for the GenManip Scaling Pick-and-Place Benchmark with worker ID 0 and a robotiq gripper. Requires the server to be running.

```bash
python standalone_tools/client.py --worker_ids 0 --gripper_type robotiq
```

--------------------------------

### Deploy AnyGrasp as a Flask server

Source: https://genmanip.com/datagen/anygrasp

Python script that sets up a Flask server with a /process endpoint. It loads the AnyGrasp model, processes incoming color/depth images with camera intrinsics, and returns grasp groups as JSON. The script uses CUDA_VISIBLE_DEVICES from the -d argument and runs on the port from -p. It includes optional visualization and clears GPU cache after each request.

```python
import argparse
import os


parser = argparse.ArgumentParser()
parser.add_argument("-p", "--port", type=str)
parser.add_argument("-d", "--device", type=str)
args = parser.parse_args()


os.environ["CUDA_VISIBLE_DEVICES"] = args.device


import numpy as np
import open3d as o3d
from flask import Flask, request, jsonify
import torch
import gc


from gsnet import AnyGrasp
from graspnetAPI import GraspGroup


VISUALIZED = False
app = Flask(__name__)


class Configs:
    checkpoint_path = "log/checkpoint_detection.tar"
    max_gripper_width = 0.1
    gripper_height = 0.03
    top_down_grasp = True
    debug = True


cfgs = Configs()


print("Loading AnyGrasp model into memory...")
anygrasp = AnyGrasp(cfgs)
anygrasp.load_net()
print("Model loaded successfully.")


def handle_grasp(data):
    try:
        dense_mode = data.get("dense_mode", False)
        print(dense_mode)
        colors = np.array(data["colors"])
        depths = np.array(data["depths"])
        fx, fy = float(data["fx"]), float(data["fy"])
        cx, cy = float(data["cx"]), float(data["cy"])
        scale = float(data["scale"])


        intrinsic = o3d.camera.PinholeCameraIntrinsic()
        intrinsic.set_intrinsics(
            width=colors.shape[1], height=colors.shape[0], fx=fx, fy=fy, cx=cx, cy=cy
        )
        colors = colors.astype(np.float32) / 255.0
        xmap, ymap = np.arange(depths.shape[1]), np.arange(depths.shape[0])
        xmap, ymap = np.meshgrid(xmap, ymap)
        points_z = depths / scale
        points_x = (xmap - cx) / fx * points_z
        points_y = (ymap - cy) / fy * points_z


        mask = (points_z > -2.0) & (points_z <= 2.0)
        points = np.stack([points_x, points_y, points_z], axis=-1)
        points = points[mask].astype(np.float32)
        colors = colors[mask].astype(np.float32)


        lims = [-20.0, 20.0, -20.0, 20.0, -2.0, 2.0]


        with torch.no_grad():
            gg, cloud = anygrasp.get_grasp(
                points,
                colors,
                lims=lims,
                apply_object_mask=True,
                dense_grasp=dense_mode,
                collision_detection=True,
            )


        if gg is None or len(gg) == 0:
            return None


        gg = gg.nms().sort_by_score()
        gg_pick = gg[:]  # top-K or all


        if VISUALIZED:
            trans_mat = np.array([[1, 0, 0, 0],
                                  [0, 1, 0, 0],
                                  [0, 0, -1, 0],
                                  [0, 0, 0, 1]])
            cloud.transform(trans_mat)
            grippers = gg_pick.to_open3d_geometry_list()
            for gripper in grippers:
                gripper.transform(trans_mat)
            o3d.visualization.draw_geometries([*grippers, cloud])


        return gg_pick
    finally:
        gc.collect()
        torch.cuda.empty_cache()




@app.route("/process", methods=["POST"])
def process_data():
    try:
        data = request.get_json()
        data["colors"] = np.array(data["colors"], dtype=np.uint8)
        data["depths"] = np.array(data["depths"], dtype=np.uint16)
        result = handle_grasp(data)


        if result is None:
            return jsonify({"message": "No Grasp detected!"}), 400


        grasp_list = []
        for grasp in result:
            grasp_list.append({
                "translation": grasp.translation.tolist(),
                "rotation_matrix": grasp.rotation_matrix.tolist(),
                "depth": grasp.depth,
                "score": grasp.score,
            })


        return jsonify({"grasp_groups": grasp_list}), 200


    except Exception as e:
        print("Error:", e)
        return jsonify({"error": str(e)}), 500




if __name__ == "__main__":
    app.run(host="0.0.0.0", port=int(args.port), threaded=False)
```

--------------------------------

### Generate data by config: python demogen.py and python render.py

Source: https://genmanip.com/datagen/introduction

Run these two commands sequentially to generate data via a config file. The first command runs the planning stage, and the second runs the rendering stage.

```bash
python demogen.py -cfg configs/tasks/xxx.yml
python render.py -cfg configs/tasks/xxx.yml
```

--------------------------------

### Download Assets for Dual-Arm Generation

Source: https://genmanip.com/datagen/dualarm_datagen

Downloads the necessary assets for dual-arm data generation. Run this command before proceeding with the generation steps.

```bash
python standalone_tools/download_assets.py --dataset all-robot
```

--------------------------------

### Define GR1 training dataset with stat_utils_args.py

Source: https://genmanip.com/baselines/learning-based-framework

Run this Python script to generate metadata files for the GR1 training dataset. Before running, set the root_path variable to the absolute path of your local GenManip repository. After execution, verify that baselines/learning_based_framework/GR1/data_info/ contains the expected .json and .pkl metadata files.

```python
if __name__ == "__main__":
    args = parse_args()
    root_path = "YOUR_GENMANIP_PATH"
    make_dir(f"{root_path}/baselines/learning_based_framework/GR1/data_info/")

```

--------------------------------

### Collect benchmark assets with collect_benchmark_assets.py

Source: https://genmanip.com/datagen/diy

Runs collect_benchmark_assets.py to collect benchmark assets and package them into a GenManip Package. Use this command after generating test cases.

```bash
# Collect benchmark assets and package them into a GenManip Package
python standalone_tools/collect_benchmark_assets.py --asset_path saved/tasks/<task_name> --dataset_id <task_package> --upload_to_huggingface
```

--------------------------------

### Run Evaluation for GenManip Tabletop10 Benchmark

Source: https://genmanip.com/evaluation/benchmarks

Runs the client evaluation for the GenManip Tabletop10 Benchmark with worker ID 0, piper gripper, and aloha arm type. Requires the server to be running.

```bash
python standalone_tools/client.py --worker_ids 0 --gripper_type piper --arm_type aloha
```

--------------------------------

### Commands to run generation and rendering

Source: https://genmanip.com/datagen/scaling_datagen

Commands to run the generation and rendering scripts. Ensure the AnyGrasp service is running before executing. The generation command uses demogen.py and the rendering command uses render.py with the specified config files.

```bash
# Make sure the AnyGrasp service is running
python demogen.py -cfg configs/tasks/scaling_up/system1_scaling_14kobj_object_container_fgrandpos_bgrandpos_fgclip_NOrobotbaserandpos_bgcache10_bgclip_3L2obsAlign_refineActionSpace.yml
python render.py -cfg configs/tasks/scaling_up/system1_scaling_14kobj_object_container_fgrandpos_bgrandpos_fgclip_NOrobotbaserandpos_bgcache10_bgclip_3L2obsAlign_refineActionSpace.yml
```

--------------------------------

### Basic load_object_from_path configuration

Source: https://genmanip.com/datagen/advancedgen/object-config

Configures a banana object with type load_object_from_path, loading from object_usds/bananas. Up to 50 objects are preloaded, and one is randomly activated each run. If max_cached_num exceeds the folder's file count, all are loaded.

```yaml
banana:
  filter_rule: []
  max_cached_num: 50
  option: []
  path: object_usds/bananas
  type: load_object_from_path

```

--------------------------------

### Data generation pipeline: download assets, generate planning data, generate rendering data

Source: https://genmanip.com/introduction/quickstart

Runs the minimal data generation pipeline: downloads assets for the banana dataset, generates planning data with demogen.py, and generates rendering data with render.py.

```bash
# Download assets
python standalone_tools/download_assets.py --dataset banana


# Generate planning data
python demogen.py


# Generate rendering data
python render.py

```

--------------------------------

### Download all robot assets (optional)

Source: https://genmanip.com/introduction/installation

Downloads all robot assets and corresponding cuRobo configurations using the standalone download tool.

```bash
python standalone_tools/download_assets.py --dataset all-robot # download all robot assets and corresponding cuRobo configurations
```

--------------------------------

### Data Generation Workflow Commands

Source: https://genmanip.com/introduction/quickstart

Run these commands in order to download assets, generate data, and render data. Replace 'your-assets-package' and 'your-task-name' with your actual package and task names.

```bash
python standalone_tools/download_assets.py --dataset your-assets-package
python demogen.py -cfg configs/tasks/your-task-name.yml
python render.py -cfg configs/tasks/your-task-name.yml
```

--------------------------------

### Additional configuration options for generation

Source: https://genmanip.com/datagen/scaling_datagen

Sets additional options including mode, num_episode, restart_per_success, preprocess_config, robots with default joint positions, table_uid, task_name, and usd_name. Uses RoboTiq gripper and restarts preloading every 100 successes to ensure diversity.

```yaml
mode: manual
num_episode: 1000000
restart_per_success: 100
preprocess_config:
- config:
    config: default
    type: convexDecomposition
  type: collider
robots:
- config:
    gripper_type: robotiq
  default_joint_positions:
  - 0.0
  - -0.785
  - 0.0
  - -2.356
  - 0.0
  - 1.57079
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  type: franka
table_uid: aa49db8a801d402dac6cf1579536502c
task_name: system1_scaling_14kobj_object_container_fgrandpos_bgrandpos_fgclip_NOrobotbaserandpos_bgcache10_bgclip_3L2obsAlign_refineActionSpace
usd_name: scene_usds/sim2real_scenes_align_for_scaling/base
```

--------------------------------

### Run Dual-Arm Data Generation Commands

Source: https://genmanip.com/datagen/dualarm_datagen

Commands to run the dual-arm data generation pipeline. Ensure the AnyGrasp service is running before executing these commands.

```bash
# Make sure the AnyGrasp service is running
python demogen.py -cfg configs/tasks/scaling_up/system1_scaling_14kobj_object_container_fgrandpos_bgrandpos_fgclip_NOrobotbaserandpos_bgcache10_bgclip_3L2obsAlign_refineActionSpace_aloha.yml
python render.py -cfg configs/tasks/scaling_up/system1_scaling_14kobj_object_container_fgrandpos_bgrandpos_fgclip_NOrobotbaserandpos_bgcache10_bgclip_3L2obsAlign_refineActionSpace_aloha.yml
```

--------------------------------

### Download assets for large-scale generation

Source: https://genmanip.com/datagen/scaling_datagen

Downloads the objaverse-subset dataset assets required for large-scale data generation. Run this command before proceeding with generation.

```bash
python standalone_tools/download_assets.py --dataset objaverse-subset
```

--------------------------------

### Run evaluation client

Source: https://genmanip.com/introduction/quickstart

Runs the client script standalone_tools/client.py with worker id 0, letting the server create an Isaac Sim test instance.

```bash
python standalone_tools/client.py --worker_ids 0 # Let the server create an Isaac Sim test instance

```

--------------------------------

### robots: Franka with robotiq gripper

Source: https://genmanip.com/datagen/advancedgen/misc

Configures a Franka robot with the robotiq gripper type.

```yaml
robots:
- config:
    gripper_type: robotiq
  type: franka

```

--------------------------------

### Launch distributed GR1 training with torchrun bash script

Source: https://genmanip.com/baselines/learning-based-framework

Use this bash script to run distributed training of the GR1 model. Customize the placeholders for model name, wandb project, dataset names, and paths. The script sets NCCL networking, activates the gr1 conda environment, and launches torchrun with the specified training hyperparameters.

```bash
#!/bin/bash


# -------- Networking & NCCL Config --------
export NCCL_DEBUG=INFO
export NCCL_IB_DISABLE=0
export NCCL_IB_HCA=mlx5_bond_0
export NCCL_SOCKET_IFNAME=eth0
export NCCL_IB_GID_INDEX=3
# Using async gradient all reduce
export CUDA_DEVICE_MAX_CONNECTIONS=1


# -------- Environment -------
source /<your_path>/miniconda3/etc/profile.d/conda.sh
conda activate gr1


# -------- Training Configuration --------
NODES=1
NPROC_PER_NODE=8
NODE_RANK=0
MASTER_ADDR=127.0.0.1
MASTER_PORT="29501"
BATCH_JOB_ID=$(date +"%Y%m%d_%H%M%S")


ROOT_PATH="/<your_path>/GenManip-Sim/"
save_checkpoint_path="${ROOT_PATH}/saved/GR1"
vit_checkpoint_path="${ROOT_PATH}/saved/vit_mae/mae_pretrain_vit_base.pth" # downloaded from https://drive.google.com/file/d/1bSsvRI4dmDMG3gC51x6O0l9CbojYwO3Et/view?usp=sharing
clip_checkpoint_path="${ROOT_PATH}/saved/public/ViT-B-32.pt"
data_dir="${ROOT_PATH}/saved/demonstrations/"
model_name="<model_name>"
wandb_project="<wandb_project_name>"
banana_dataset_name="<banana_dataset_name>"
banana_dataset_info_name="<banana_dataset_info_name>"
OUTPUT_LOG="${ROOT_PATH}/saved/Training_log/${model_name}/${BATCH_JOB_ID}_${NODE_RANK}.log"
mkdir -p "$(dirname "$OUTPUT_LOG")"
OBS_CAMERA_TYPE="obs_camera"


# -------- Launch Training --------
torchrun --nnodes="${NODES}" \
        --node_rank="${NODE_RANK}" \
        --nproc_per_node="${NPROC_PER_NODE}" \
        --master_addr="${MASTER_ADDR}" \
        --master_port="${MASTER_PORT}" \
        train.py \
        --traj_cons \
        --rgb_pad 10 \
        --gripper_pad 4 \
        --gradient_accumulation_steps 1 \
        --bf16_module "vision_encoder" \
        --vit_checkpoint_path ${vit_checkpoint_path} \
        --calvin_dataset "" \
        --workers 8 \
        --clip_checkpoint_path ${clip_checkpoint_path} \
        --lr_scheduler cosine \
        --save_every_iter 100000 \
        --num_epochs 40 \
        --seed 42 \
        --batch_size 32 \
        --precision fp32 \
        --learning_rate 1e-3 \
        --save_checkpoint \
        --finetune_type banana \
        --root_dir ${data_dir} \
        --wandb_project ${wandb_project} \
        --report_to_wandb \
        --weight_decay 1e-4 \
        --num_resampler_query 6 \
        --run_name ${model_name} \
        --save_checkpoint_path ${save_checkpoint_path} \
        --except_lang \
        --transformer_layers 24 \
        --phase "finetune" \
        --action_pred_steps 1 \
        --sequence_length 10 \
        --future_steps 3 \
        --window_size 13 \
        --obs_pred \
        --loss_action \
        --loss_image \
        --save_checkpoint_seq 1 \
        --start_save_checkpoint -1 \
        --warmup_epochs 5 \
        --gripper_width \
        --banana_dataset_names ${banana_dataset_name} \
        --dataset_info_names ${banana_dataset_info_name} \
        --action_type "delta_qpos" \
        --use_aug_data \
        --obs_type ${OBS_CAMERA_TYPE}

```

--------------------------------

### robots: Franka with panda_hand gripper

Source: https://genmanip.com/datagen/advancedgen/misc

Configures a Franka robot with the panda_hand gripper type.

```yaml
robots:
- config:
    gripper_type: panda_hand
  type: franka

```

--------------------------------

### Camera config using config_path and type (fixed)

Source: https://genmanip.com/datagen/advancedgen/domain-randomization

Specifies a camera in the domain_randomization config. Currently only the 'fixed' type is supported; config_path points to the camera's YAML file.

```yaml
cameras:
  config_path: configs/cameras/fixed_camera.yml
  type: fixed

```

--------------------------------

### Add Additional Object from Path Configuration

Source: https://genmanip.com/datagen/advancedgen/object-config

YAML configuration for loading a USD object from a path. Set 'type' to 'add_additional_object_from_path' and provide the 'path' to the USD file. If 'path' is a folder, use 'max_cached_num' to control the number of objects loaded.

```yaml
container:
  option: []
  path: object_usds/objaverse_usd/f8a2cdc9970846da95585a428697d173.usd
  type: add_additional_object_from_path

```

--------------------------------

### Layout configuration for large-scale generation

Source: https://genmanip.com/datagen/scaling_datagen

Sets up random placement using random_custom_tableset. Target objects (obj1_0, obj2) are placed within a 0.7 height, 0.4 width, and limited x/y range; background objects are placed in a much larger range (20x20) to scatter them widely.

```yaml
layout_config:
  ignored_objects: []
  type: random_custom_tableset
  in_order: true
  custom_tableset:
    obj1_0: &id001
      type: global_range
      random_range_angle: 360
      random_range_h: 0.7
      random_range_w: 0.4
      random_range_x: -0.2
      random_range_y: -0.35
    obj2: *id001
    background_0: &id002
      type: global_range
      random_range_angle: 360
      random_range_h: 20
      random_range_w: 20
      random_range_x: -10
      random_range_y: -10
    background_1: *id002
    background_2: *id002
    background_3: *id002
    background_4: *id002
    background_5: *id002
    background_6: *id002
    background_7: *id002
    background_8: *id002
```

--------------------------------

### Generate evaluation test cases with demogen.py --eval

Source: https://genmanip.com/datagen/diy

Runs demogen.py with the --eval flag to iterate over each dictionary in evaluation_configs and generate num_test test cases. Use this command to create evaluation test cases.

```bash
# Iterate over each dictionary in evaluation_configs and generate num_test test cases
python demogen.py -cfg configs/tasks/xxx.yml --eval
```

--------------------------------

### Run deprecated benchmark script

Source: https://genmanip.com/introduction/quickstart

In a second terminal, activates the genmanip environment and runs eval.py with the GenManip-Package-Basic configuration.

```bash
conda activate genmanip
python eval.py -cfg configs/tasks/GenManip-Package-Basic.yml

```

--------------------------------

### Object configuration for large-scale generation

Source: https://genmanip.com/datagen/scaling_datagen

Defines 11 objects (9 background, 2 target) loaded from object_usds/objaverse_usd. Background objects are non-rigid; obj1 must be graspable and obj2 must be a container. clip_range controls size sampling, max_cached_num limits memory, and adjust_thickness is applied to obj1.

```yaml
object_config:
  background_0: &id003
    clip_range:
      max: 0.2
      min: 0.05
    filter_rule: []
    max_cached_num: 30
    is_not_rigid: true
    path: object_usds/objaverse_usd
    type: load_object_from_path
  background_1: *id003
  background_2: *id003
  background_3: *id003
  background_4: *id003
  background_5: *id003
  background_6: *id003
  background_7: *id003
  background_8: *id003
  obj1_0:
    clip_range:
      max: 0.15
      min: 0.05
    filter_rule:
    - can_grasp
    max_cached_num: 30
    option:
    - adjust_thickness
    path: object_usds/objaverse_usd
    type: load_object_from_path
  obj2:
    clip_range:
      max: 0.35
      min: 0.2
    filter_rule:
    - is_container
    max_cached_num: 30
    option:
    - force_top
    path: object_usds/objaverse_usd
    type: load_object_from_path
```

--------------------------------

### Config class for distributed deployment

Source: https://genmanip.com/baselines/modular-framework

Configuration class with service addresses and ports for distributed deployment. Modify config.py to set pipeline addresses and ports.

```python
class Config:
    PROMPT_PIPELINE_ADDRESS = "127.0.0.1"
    PROMPT_PIPELINE_PORT = "5004"
    CHECK_FINISHED_PIPELINE_ADDRESS = "127.0.0.1"
    CHECK_FINISHED_PIPELINE_PORT = "5009"
    TASK_SPLIT_PIPELINE_ADDRESS = "127.0.0.1"
    TASK_SPLIT_PIPELINE_PORT = "5008"
    debug_text = True
```