NVIDIA Isaac Sim

https://docs.isaacsim.omniverse.nvidia.com/5.1.0/
Admin
NVIDIA Isaac Sim is a reference application for developing, simulating, and testing AI-driven robots...
Tokens:1,777,195
Snippets:15,177
Trust Score:10
Update:1 month ago
Show doc for...
Tokens:
Raw
### Example Execution Setup

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_cosmos.html

Asynchronous wrapper to initialize and run the simulation example.

```python
asyncio.ensure_future(run_example_async(
    num_clips=NUM_CLIPS,
    num_frames_per_clip=NUM_FRAMES_PER_CLIP,
    capture_interval=CAPTURE_INTERVAL,
    start_delay=START_DELAY,
    use_instance_id=True,
))
```

--------------------------------

### Copy Example Files

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/utilities/custom_interactive_examples.html

Use these commands to copy existing example files into the user_examples directory as a starting point.

```bash
cd exts/isaacsim.examples.interactive/isaacsim/examples/interactive
cp hello_world/hello_world* user_examples/
```

--------------------------------

### Example Async Setup and Asset Loading

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_cosmos.html

Initializes the stage, configures settings for DLSS and script nodes, and positions assets for the simulation.

```python
async def run_example_async(
    num_clips,
    num_frames_per_clip,
    capture_interval,
    start_delay=0.0,
    use_instance_id=True,
    segmentation_mapping=None,
):
    assets_root_path = await get_assets_root_path_async()
    stage_path = assets_root_path + STAGE_URL
    print(f"Opening stage: '{stage_path}'")
    omni.usd.get_context().open_stage(stage_path)
    stage = omni.usd.get_context().get_stage()

    # Enable script nodes
    carb.settings.get_settings().set_bool("/app/omni.graph.scriptnode/opt_in", True)

    # Disable capture on play on the new stage, data is captured manually using the step function
    rep.orchestrator.set_capture_on_play(False)

    # Set DLSS to Quality mode (2) for best SDG results (Options: 0 (Performance), 1 (Balanced), 2 (Quality), 3 (Auto)
    carb.settings.get_settings().set("rtx/post/dlss/execMode", 2)

    # Load carter nova asset with its navigation graph
    carter_url_path = assets_root_path + CARTER_NAV_ASSET_URL
    print(f"Loading carter nova asset: '{carter_url_path}' at prim path: '{CARTER_NAV_PATH}'")
    carter_nav_prim = add_reference_to_stage(usd_path=carter_url_path, prim_path=CARTER_NAV_PATH)

    if not carter_nav_prim.GetAttribute("xformOp:translate"):
        UsdGeom.Xformable(carter_nav_prim).AddTranslateOp()
    carter_nav_prim.GetAttribute("xformOp:translate").Set(CARTER_NAV_POSITION)

    # Set the navigation target position
    carter_navigation_target_prim = stage.GetPrimAtPath(CARTER_NAV_TARGET_PATH)
    if not carter_navigation_target_prim.IsValid():
        print(f"Carter navigation target prim not found at path: {CARTER_NAV_TARGET_PATH}, exiting")
        return
    if not carter_navigation_target_prim.GetAttribute("xformOp:translate"):
        UsdGeom.Xformable(carter_navigation_target_prim).AddTranslateOp()
    carter_navigation_target_prim.GetAttribute("xformOp:translate").Set(CARTER_NAV_TARGET_POSITION)

    # Use the carter nova front hawk camera for capturing data
    camera_prim = stage.GetPrimAtPath(CARTER_CAMERA_PATH)
    if not camera_prim.IsValid():
        print(f"Camera prim not found at path: {CARTER_CAMERA_PATH}, exiting")
        return
```

--------------------------------

### Run follow example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/cortex_tutorials/tutorial_cortex_2_decider_networks.html

Command to launch the initial robot follow example.

```bash
isaac_python follow_example_main.py
```

--------------------------------

### Run Pick and Place Example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/robot_setup_tutorials/tutorial_pickplace_example.html

Command to execute the standalone pick and place example script.

```bash
./python.sh standalone_examples/api/isaacsim.robot.manipulators/ur10e/pick_up_example.py
```

--------------------------------

### onStart()

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/api/classisaacsim_1_1core_1_1includes_1_1_prim_manager_base.html

Handles application start event. Optional callback that runs when the application starts. Override this to implement custom start behavior.

```APIDOC
## onStart()

### Description
Handles application start event. Optional callback that runs when the application starts. Override this to implement custom start behavior.

### Method
`inline virtual void onStart()`

### Parameters
None
```

--------------------------------

### Register Example in Init

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/utilities/custom_interactive_examples.html

Expose the new example and extension classes by adding them to the __init__.py file.

```python
from isaacsim.examples.interactive.user_examples.hello_world import HelloWorld
from isaacsim.examples.interactive.user_examples.hello_world_extension import HelloWorldExtension
```

--------------------------------

### BaseIsaacBenchmarkAsync Setup

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.benchmark.services/docs/index.html

Asynchronous setup for derived benchmarks. Must be awaited. Initializes the benchmark backend.

```python
await benchmark.setUp(backend_type='JSONFileMetrics')

```

--------------------------------

### Start Application

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/python_scripting/manual_standalone_python.html

Starts the application using the configured arguments.

```python
app = omni.kit.app.get_app()
app.startup("Isaac-Sim", os.environ["CARB_APP_PATH"], sys.argv)
```

--------------------------------

### onStart

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/api/classisaacsim_1_1sensors_1_1physics_1_1_isaac_sensor_manager.html

Handles the application start event. This is an optional callback that can be overridden to implement custom start behavior.

```APIDOC
## onStart

### Description
Handles application start event. Optional callback that runs when the application starts. Override this to implement custom start behavior.
```

--------------------------------

### onStart

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/api/classisaacsim_1_1sensors_1_1physx_1_1_range_sensor_manager.html

Handles the application start event. This is an optional callback that can be overridden to implement custom start behavior.

```APIDOC
## onStart

### Description
Handles application start event. Optional callback that runs when the application starts. Override this to implement custom start behavior.

### Method
`void`

### Parameters
None

### Response Example
None
```

--------------------------------

### Install and Run Teleop Keyboard

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/ros2_tutorials/tutorial_ros2_drive_turtlebot.html

Install the teleop package and launch the keyboard controller node.

```bash
sudo apt-get install ros-$ROS_DISTRO-teleop-twist-keyboard
```

```bash
ros2 run teleop_twist_keyboard teleop_twist_keyboard
```

--------------------------------

### Running Standalone Examples

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_getting_started.html

Command to execute a pre-built Replicator standalone example script from the terminal.

```bash
./python.sh standalone_examples/api/isaacsim.replicator.examples/sdg_getting_started_04.py
```

--------------------------------

### UR10 Trajectory Generation Example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/manipulators/manipulators_lula_trajectory_generator.html

Example class demonstrating the setup of Lula trajectory generators and advanced path specification using CompositePathSpec.

```python
  1class UR10TrajectoryGenerationExample():
  2    def __init__(self):
  3        self._c_space_trajectory_generator = None
  4        self._taskspace_trajectory_generator = None
  5        self._kinematics_solver = None
  6
  7        self._action_sequence = []
  8        self._action_sequence_index = 0
  9
 10        self._articulation = None
 11
 12    def load_example_assets(self):
 13        # Add the Franka and target to the stage
 14        # The position in which things are loaded is also the position in which they
 15
 16        robot_prim_path = "/ur10"
 17        path_to_robot_usd = get_assets_root_path() + "/Isaac/Robots/UniversalRobots/ur10/ur10.usd"
 18
 19        add_reference_to_stage(path_to_robot_usd, robot_prim_path)
 20        self._articulation = Articulation(robot_prim_path)
 21
 22        # Return assets that were added to the stage so that they can be registered with the core.World
 23        return [self._articulation]
 24
 25    def setup(self):
 26        # Config files for supported robots are stored in the motion_generation extension under "/motion_policy_configs"
 27        mg_extension_path = get_extension_path_from_name("isaacsim.robot_motion.motion_generation")
 28        rmp_config_dir = os.path.join(mg_extension_path, "motion_policy_configs")
 29
 30        #Initialize a LulaCSpaceTrajectoryGenerator object
 31        self._c_space_trajectory_generator = LulaCSpaceTrajectoryGenerator(
 32            robot_description_path = rmp_config_dir + "/universal_robots/ur10/rmpflow/ur10_robot_description.yaml",
 33            urdf_path = rmp_config_dir + "/universal_robots/ur10/ur10_robot.urdf"
 34        )
 35
 36        self._taskspace_trajectory_generator = LulaTaskSpaceTrajectoryGenerator(
 37            robot_description_path = rmp_config_dir + "/universal_robots/ur10/rmpflow/ur10_robot_description.yaml",
 38            urdf_path = rmp_config_dir + "/universal_robots/ur10/ur10_robot.urdf"
 39        )
 40
 41        self._kinematics_solver = LulaKinematicsSolver(
 42            robot_description_path = rmp_config_dir + "/universal_robots/ur10/rmpflow/ur10_robot_description.yaml",
 43            urdf_path = rmp_config_dir + "/universal_robots/ur10/ur10_robot.urdf"
 44        )
 45
 46        self._end_effector_name = "ee_link"
 47
 48    def setup_advanced_trajectory(self):
 49        # The following code demonstrates how to specify a complicated cspace and taskspace path
 50        # using the lula.CompositePathSpec object
 51
 52        initial_c_space_robot_pose = np.array([0,0,0,0,0,0])
 53
 54        # Combine a cspace and taskspace trajectory
 55        composite_path_spec = lula.create_composite_path_spec(initial_c_space_robot_pose)
 56
 57        #############################################################################
```

--------------------------------

### Configure Application Start Arguments

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/python_scripting/manual_standalone_python.html

Sets up experience configuration, extension paths, and headless mode before application startup.

```python
# Inject a experience config
sys.argv.insert(1, f'{os.environ["EXP_PATH"]}/isaacsim.exp.base.python.kit')

# Add paths to extensions
sys.argv.append(f"--ext-folder")
sys.argv.append(f'{os.path.abspath(os.environ["ISAAC_PATH"])}/exts')

# Run headless
sys.argv.append("--no-window")
```

--------------------------------

### Get Guide Radii of Prims

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.experimental.objects/docs/index.html

Retrieves the guide radii used for visualizing dome lights. Can process all prims or a specified subset by index.

```python
>>> # get the guide radii of all prims
>>> radii = prims.get_guide_radii()
>>> radii.shape
(3, 1)
>>>
>>> # get the guide radii of the first and last prims
>>> radii = prims.get_guide_radii(indices=[0, 2])
>>> radii.shape
(2, 1)
```

--------------------------------

### get_default_state

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.robot.manipulators/docs/index.html

Gets the default state of the gripper, indicating if it should start in an opened state.

```APIDOC
## get_default_state

### Description
Gets the default state of the gripper.

### Returns
key is “opened” and value would be true if the surface gripper should start in an opened state. False otherwise.

### Return type
dict
```

--------------------------------

### HelloWorld Sample Scene Setup

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/core_api_tutorials/tutorial_core_multiple_tasks.html

Implementation of a BaseSample class to initialize the scene, controllers, and physics callbacks.

```python
102class HelloWorld(BaseSample):
103    def __init__(self) -> None:
104        super().__init__()
105        return
106
107    def setup_scene(self):
108        world = self.get_world()
109        world.add_task(RobotsPlaying(name="awesome_task", offset=np.array([0, -1.0, 0])))
110        # For visualization purposes only, so we don't need to add it to the scene
111        # Its there to only see where Franka was supposed to be vs after adding the offset
112        VisualCuboid(prim_path="/new_cube_1",
113                    name="visual_cube",
114                    position=np.array([1.0, 0, 0.05]),
115                    scale=np.array([0.1, 0.1, 0.1]))
116        return
117
118    async def setup_post_load(self):
119        self._world = self.get_world()
120        task_params = self._world.get_task("awesome_task").get_params()
121        self._franka = self._world.scene.get_object(task_params["franka_name"]["value"])
122        self._jetbot = self._world.scene.get_object(task_params["jetbot_name"]["value"])
123        self._cube_name = task_params["cube_name"]["value"]
124        self._franka_controller = PickPlaceController(name="pick_place_controller",
125                                                    gripper=self._franka.gripper,
126                                                    robot_articulation=self._franka)
127        self._jetbot_controller = WheelBasePoseController(name="cool_controller",
128                                                        open_loop_wheel_controller=
129                                                            DifferentialController(name="simple_control",
130                                                                                    wheel_radius=0.03, wheel_base=0.1125))
131        self._world.add_physics_callback("sim_step", callback_fn=self.physics_step)
132        await self._world.play_async()
133        return
134
135    async def setup_post_reset(self):
136        self._franka_controller.reset()
137        self._jetbot_controller.reset()
138        await self._world.play_async()
139        return
140
141    def physics_step(self, step_size):
142        current_observations = self._world.get_observations()
143        if current_observations["task_event"] == 0:
144            self._jetbot.apply_wheel_actions(
145                self._jetbot_controller.forward(
146                    start_position=current_observations[self._jetbot.name]["position"],
147                    start_orientation=current_observations[self._jetbot.name]["orientation"],
148                    goal_position=current_observations[self._jetbot.name]["goal_position"]))
149        elif current_observations["task_event"] == 1:
150            self._jetbot.apply_wheel_actions(ArticulationAction(joint_velocities=[-8.0, -8.0]))
151        elif current_observations["task_event"] == 2:
152            self._jetbot.apply_wheel_actions(ArticulationAction(joint_velocities=[0.0, 0.0]))
153            actions = self._franka_controller.forward(
154                picking_position=current_observations[self._cube_name]["position"],
155                placing_position=current_observations[self._cube_name]["target_position"],
156                current_joint_positions=current_observations[self._franka.name]["joint_positions"])
157            self._franka.apply_action(actions)
158        if self._franka_controller.is_done():
159            self._world.pause()
160        return
```

--------------------------------

### onStart

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/api/classisaacsim_1_1sensors_1_1physics_1_1_contact_sensor.html

Callback executed when the sensor component starts, handling initialization and state setup.

```APIDOC
## onStart

### Description
Called when the sensor starts. Handles sensor initialization when the component is started. Triggers onComponentChange to ensure proper initial state.
```

--------------------------------

### Launch H1 Fullbody Controller

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/ros2_tutorials/tutorial_ros2_rl_controller.html

Starts the ROS 2 policy controller in an environment with PyTorch installed.

```bash
ros2 launch h1_fullbody_controller h1_fullbody_controller.launch.py
```

--------------------------------

### Run Manual Camera Image Publishing Sample

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/ros2_tutorials/tutorial_ros2_python.html

Executes the camera manual publishing example.

```bash
./python.sh standalone_examples/api/isaacsim.ros2.bridge/camera_manual.py
```

--------------------------------

### Initialize Example Stage

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_modular_scripting.html

Opens a USD stage and validates the existence of required prims.

```python
async def run_example_async():
    STAGE_URL = "/Isaac/Samples/Replicator/Stage/warehouse_pallets_behavior_scripts.usd"
    PALLETS_ROOT_PATH = "/Root/Pallets"
    LIGHTS_ROOT_PATH = "/Root/Lights"
    CAMERA_PATH = "/Root/Camera_01"
    TARGET_ASSET_URL = "/Isaac/Props/YCB/Axis_Aligned/035_power_drill.usd"
    TARGET_ASSET_PATH = "/Root/Target"
    TARGET_ASSET_LABEL = "power_drill"
    TARGET_ASSET_LOCATION = (-1.5, 5.5, 1.5)

    # Open stage
    assets_root_path = await get_assets_root_path_async()
    print(f"Opening stage from {assets_root_path + STAGE_URL}")
    await omni.usd.get_context().open_stage_async(assets_root_path + STAGE_URL)
    stage = omni.usd.get_context().get_stage()

    # Check if all required prims exist in the stage
    pallets_root_prim = stage.GetPrimAtPath(PALLETS_ROOT_PATH)
    lights_root_prim = stage.GetPrimAtPath(LIGHTS_ROOT_PATH)
    camera_prim = stage.GetPrimAtPath(CAMERA_PATH)
    if not all([pallets_root_prim.IsValid(), lights_root_prim.IsValid(), camera_prim.IsValid()]):
        print(f"Not all required prims exist in the stage.")
        return
```

--------------------------------

### Build ROS Workspace Dependencies

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/installation/install_ros.html

Installs dependencies, builds the workspace, and sources the environment setup files.

```bash
cd /${ROS_DISTRO}_ws
apt-get update
git submodule update --init --recursive # If using docker, perform this step outside the container and relaunch the container
rosdep install --from-paths src --ignore-src --rosdistro=$ROS_DISTRO -y
source /opt/ros/$ROS_DISTRO/setup.sh
colcon build
source install/local_setup.bash
```

--------------------------------

### Get Default Cspace Seeds

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.robot_motion.motion_generation/docs/index.html

Retrieves a list of cspace seeds that the solver can use as starting points for solutions.

```APIDOC
## Get Default Cspace Seeds

### Description
Get a list of cspace seeds that the solver may use as starting points for solutions.

### Returns
- **cspace_seeds** (List[np.array]) - An N x num_dof list of cspace seeds.
```

--------------------------------

### Run Hello World Standalone Example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/introduction/examples.html

Commands to execute the hello_world example script from the root directory on Linux and Windows platforms.

```bash
./python.sh standalone_examples/api/isaacsim.simulation_app/hello_world.py
```

```batch
python.bat standalone_examples\api\isaacsim.simulation_app\hello_world.py
```

--------------------------------

### Get All Matching Child Prims

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.utils/docs/index.html

Retrieves a list of prims matching a specific predicate starting from a root path.

```python
>>> import isaacsim.core.utils.prims as prims_utils
>>>
>>> # get all hidden prims
>>> predicate = lambda path: prims_utils.is_prim_hidden_in_stage(path)  # True if the prim at path is hidden
>>> prims_utils.get_all_matching_child_prims("/", predicate)
[Usd.Prim(</OmniverseKit_Persp>),
 Usd.Prim(</OmniverseKit_Front>),
 Usd.Prim(</OmniverseKit_Top>),
 Usd.Prim(</OmniverseKit_Right>),
 Usd.Prim(</Render>)]
```

--------------------------------

### Get Collision Approximation

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.prims/docs/index.html

Retrieves the collision approximation type used for a prim. The example shows the 'none' approximation.

```python
>>> prim.get_collision_approximation()
none
```

--------------------------------

### Launch Isaac Sim Streaming via PIP

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/installation/manual_livestream_clients.html

Start the streaming application using the installed Python package.

```bash
isaacsim isaacsim.exp.full.streaming --no-window
```

--------------------------------

### Setup SDG Configuration

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_amr_navigation.html

Configures global settings for rendering, DLSS, and camera properties, and initializes the BasicWriter.

```python
    def _setup_sdg(self):
        # Disable capture on play and async rendering
        carb.settings.get_settings().set("/omni/replicator/captureOnPlay", False)
        carb.settings.get_settings().set("/omni/replicator/asyncRendering", False)
        carb.settings.get_settings().set("/app/asyncRendering", False)

        # Set DLSS to Quality mode (2) for best SDG results (Options: 0 (Performance), 1 (Balanced), 2 (Quality), 3 (Auto)
        carb.settings.get_settings().set("rtx/post/dlss/execMode", 2)

        # Set camera sensors fStop to 0.0 to get well lit sharp images
        left_camera_prim = self._stage.GetPrimAtPath(self.LEFT_CAMERA_PATH)
        left_camera_prim.GetAttribute("fStop").Set(0.0)
        right_camera_prim = self._stage.GetPrimAtPath(self.RIGHT_CAMERA_PATH)
        right_camera_prim.GetAttribute("fStop").Set(0.0)

        self._writer = rep.WriterRegistry.get("BasicWriter")
        self._writer.initialize(output_dir=self._out_dir, rgb=True)
        self._setup_sdg_render_products()
```

--------------------------------

### setup_writer

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.replicator.writers/docs/index.html

Initializes a writer and attaches a render product. It accepts general script configurations and writer-specific configurations.

```APIDOC
## setup_writer

### Description
Initializes writer and attach render product.

### Parameters
#### Request Body
- **writer_config** (dict) - Required - A dictionary containing writer-specific configurations.
- **config_data** (dict) - Required - A dictionary containing the general configurations for the script.
```

--------------------------------

### Get Current Time Step Index

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.api/docs/index.html

Returns the total number of physics steps elapsed since the simulation started.

```python
>>> # given a running Isaac Sim instance and after approximately 15 seconds of physics simulation at 60 Hz
>>> simulation_context.current_time_step_index
911
```

--------------------------------

### CommandResponse Example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/action_and_event_data_generation/ext_replicator-agent/actor_control.html

A sample configuration for a response that triggers all actors to perform a LookAround command 5 seconds after play starts.

```yaml
CommandResponse:
    name: "MyCommandResponse"
    priority: 1
    pick_agent: all
    resume: True
    position: (0, 0, 0)
    trigger:
        type: time
        time: 5.0
    commands:
        - LookAround 10
```

--------------------------------

### onStart()

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/api/classisaacsim_1_1robot_1_1surface__gripper_1_1_surface_gripper_manager.html

Callback function invoked when the system starts.

```APIDOC
## onStart()

### Description
Callback function invoked when the system starts.

### Method
Callback
```

--------------------------------

### Run Pre-ISP Camera Pipeline Example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/sensors/isaacsim_sensors_camera.html

Executes the standalone example to render and save pre-ISP camera pipeline outputs.

```bash
./python.sh standalone_examples/api/isaacsim.sensors.camera/camera_pre_isp_pipeline.py --draw-output
```

--------------------------------

### Compute Inverse Kinematics with LulaKinematicsSolver

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/manipulators/manipulators_lula_kinematics.html

Example demonstrating the setup and usage of LulaKinematicsSolver and ArticulationKinematicsSolver to move a Franka robot to a target pose.

```python
 1import numpy as np
 2import os
 3import carb
 4
 5from isaacsim.core.utils.extensions import get_extension_path_from_name
 6from isaacsim.core.utils.stage import add_reference_to_stage
 7from isaacsim.core.prims import Articulation
 8from isaacsim.core.utils.nucleus import get_assets_root_path
 9from isaacsim.core.prims import XFormPrim
10from isaacsim.core.utils.numpy.rotations import euler_angles_to_quats
11
12from isaacsim.robot_motion.motion_generation import ArticulationKinematicsSolver, LulaKinematicsSolver
13from isaacsim.robot_motion.motion_generation import interface_config_loader
14
15class FrankaKinematicsExample():
16    def __init__(self):
17        self._kinematics_solver = None
18        self._articulation_kinematics_solver = None
19
20        self._articulation = None
21        self._target = None
22
23    def load_example_assets(self):
24        # Add the Franka and target to the stage
25
26        robot_prim_path = "/panda"
27        path_to_robot_usd = get_assets_root_path() + "/Isaac/Robots/FrankaRobotics/FrankaPanda/franka.usd"
28
29        add_reference_to_stage(path_to_robot_usd, robot_prim_path)
30        self._articulation = Articulation(robot_prim_path)
31
32        add_reference_to_stage(get_assets_root_path() + "/Isaac/Props/UIElements/frame_prim.usd", "/World/target")
33        self._target = XFormPrim("/World/target")
34        self._target.set_local_scales([0.04, 0.04, 0.04])
35        self._target.set_default_state(np.array([.3,0,.5]),euler_angles_to_quats([0,np.pi,0]))
36
37        # Return assets that were added to the stage so that they can be registered with the core.World
38        return self._articulation, self._target
39
40    def setup(self):
41        # Load a URDF and Lula Robot Description File for this robot:
42        mg_extension_path = get_extension_path_from_name("isaacsim.robot_motion.motion_generation")
43        kinematics_config_dir = os.path.join(mg_extension_path, "motion_policy_configs")
44
45        self._kinematics_solver = LulaKinematicsSolver(
46            robot_description_path = kinematics_config_dir + "/franka/rmpflow/robot_descriptor.yaml",
47            urdf_path = kinematics_config_dir + "/franka/lula_franka_gen.urdf"
48        )
49
50        # Kinematics for supported robots can be loaded with a simpler equivalent
51        # print("Supported Robots with a Lula Kinematics Config:", interface_config_loader.get_supported_robots_with_lula_kinematics())
52        # kinematics_config = interface_config_loader.load_supported_lula_kinematics_solver_config("Franka")
53        # self._kinematics_solver = LulaKinematicsSolver(**kinematics_config)
54
55        print("Valid frame names at which to compute kinematics:", self._kinematics_solver.get_all_frame_names())
56
57        end_effector_name = "right_gripper"
58        self._articulation_kinematics_solver = ArticulationKinematicsSolver(self._articulation,self._kinematics_solver, end_effector_name)
59
60
61    def update(self, step: float):
62        target_position, target_orientation = self._target.get_world_poses()
63
64        #Track any movements of the robot base
65        robot_base_translation,robot_base_orientation = self._articulation.get_world_poses()
66        self._kinematics_solver.set_robot_base_pose(robot_base_translation,robot_base_orientation)
67
68        action, success = self._articulation_kinematics_solver.compute_inverse_kinematics(target_position, target_orientation)
69
70        if success:
71            self._articulation.apply_action(action)
72        else:
73            carb.log_warn("IK did not converge to a solution.  No action is being taken")
74
75        # Unused Forward Kinematics:
76        # ee_position,ee_rot_mat = articulation_kinematics_solver.compute_end_effector_pose()
77
78    def reset(self):
79        # Kinematics is stateless
80        pass
```

--------------------------------

### Source ROS 2 Jazzy environment from source

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/installation/install_ros.html

Sources the local setup script for a source-built ROS 2 Jazzy installation.

```bash
. ~/ros2_jazzy/install/local_setup.bash
```

--------------------------------

### get_physics_num_steps

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.nodes/docs/index.html

Gets the number of physics steps completed since simulation start. This counter is reset to zero when the timeline is stopped. This method is deprecated.

```APIDOC
## get_physics_num_steps

### Description
Gets the number of physics steps completed since simulation start. This counter is reset to zero when the timeline is stopped.

### Returns
- **int**: The number of physics steps completed.

### Deprecated
This method is deprecated and will be removed in a future release. Please use isaacsim.core.simulation_manager.get_num_physics_steps() instead.
```

--------------------------------

### Launch simple state machine example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/cortex_tutorials/tutorial_cortex_2_decider_networks.html

Executes the simple state machine behavior using the franka_examples_main.py script.

```bash
isaac_python franka_examples_main.py --behavior=simple_state_machine
```

--------------------------------

### Implement SimulationApp and Carb Settings Check

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/utilities/debugging/tutorial_advanced_python_debugging.html

Example script demonstrating the initialization of SimulationApp and retrieving settings via the carb interface.

```python
# The most basic usage for creating a simulation app
kit = SimulationApp()
import carb
server_check = carb.settings.get_settings().get_as_string("/persistent/isaac/asset_root/default")
print(server_check)
for i in range(100):
    kit.update()
kit.close()  # Cleanup application
```

--------------------------------

### Get Physics Steps Completed

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.nodes/docs/index.html

Retrieves the number of physics steps completed since the simulation started. This count resets when the timeline is stopped.

```python
steps = core_nodes.get_physics_num_steps()
```

--------------------------------

### Run Standalone Example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/examples/manipulation_franka_pick_place.html

Commands to execute the standalone Franka pick-and-place script from the command line on Linux and Windows.

```bash
./python.sh standalone_examples/api/isaacsim.robot.manipulators/franka/pick_place.py
```

```batch
.\python.bat standalone_examples\api\isaacsim.robot.manipulators\franka\pick_place.py
```

--------------------------------

### Initialize and Control RMPFlow Controller

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/importer_exporter/import_urdf.html

Example setup for an RMPFlow controller within an Isaac Sim extension, including physics callbacks and action application.

```python
 86        self._world = self.get_world()
 87        self._franka = self._world.scene.get_object("fancy_franka")
 88
 89        # Initialize the RMPFlow controller for the robot
 90        self._controller = RMPFlowController(
 91            name="target_follower_controller",
 92            robot_articulation=self._franka
 93        )
 94
 95        # Add a physics callback for simulation steps
 96        self._world.add_physics_callback("sim_step", callback_fn=self.physics_step)
 97        await self._world.play_async()
 98        return
 99
100    async def setup_post_reset(self):
101        # Reset the controller to its initial state
102        self._controller.reset()
103        await self._world.play_async()
104        return
105
106    def physics_step(self, step_size):
107        # Perform a simulation step and compute actions for the robot
108        world = self.get_world()
109        observations = world.get_observations()
110
111        # Compute actions for the robot to follow the target's position and orientation
112        actions = self._controller.forward(
113            target_end_effector_position=observations["target"]["position"],
114            target_end_effector_orientation=observations["target"]["orientation"]
115        )
116
117        # Apply the computed actions to the robot
118        self._franka.apply_action(actions)
119        return
```

--------------------------------

### Follow Target Example Script

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/robot_setup_tutorials/tutorial_pickplace_example.html

Initializes the simulation environment, sets up the FollowTarget task, and runs the inverse kinematics solver loop.

```python
 1from isaacsim import SimulationApp
 2
 3simulation_app = SimulationApp({"headless": False})
 4
 5import numpy as np
 6from controller.ik_solver import KinematicsSolver
 7from isaacsim.core.api import World
 8from tasks.follow_target import FollowTarget
 9
10my_world = World(stage_units_in_meters=1.0)
11# Initialize the Follow Target task with a target location for the cube to be followed by the end effector
12my_task = FollowTarget(name="ur10e_follow_target", target_position=np.array([0.5, 0, 0.5]))
13my_world.add_task(my_task)
14my_world.reset()
15task_params = my_world.get_task("ur10e_follow_target").get_params()
16target_name = task_params["target_name"]["value"]
17ur10e_name = task_params["robot_name"]["value"]
18my_ur10e = my_world.scene.get_object(ur10e_name)
19
20# initialize the ik solver
21ik_solver = KinematicsSolver(my_ur10e)
22articulation_controller = my_ur10e.get_articulation_controller()
23
24# run the simulation
25while simulation_app.is_running():
26    my_world.step(render=True)
27    if my_world.is_playing():
28        if my_world.current_time_step_index == 0:
29            my_world.reset()
30
31        observations = my_world.get_observations()
32        actions, succ = ik_solver.compute_inverse_kinematics(
33            target_position=observations[target_name]["position"],
34            target_orientation=observations[target_name]["orientation"],
35        )
36        if succ:
37            articulation_controller.apply_action(actions)
38        else:
39            print("IK did not converge to a solution.  No action is being taken.")
40simulation_app.close()
```

--------------------------------

### Script Editor Implementation

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_object_based_sdg.html

Full code example for use within the Isaac Sim Script Editor, including helper functions for transformations and collision setup.

```python
import asyncio
import os
import random
import time
from itertools import chain

import carb.settings
import numpy as np
import omni.kit.app
import omni.replicator.core as rep
import omni.timeline
import omni.usd
import usdrt
from isaacsim.core.utils.semantics import add_labels, remove_labels, upgrade_prim_semantics_to_labels
from isaacsim.storage.native import get_assets_root_path
from omni.kit.viewport.utility import get_active_viewport
from omni.physx import get_physx_interface, get_physx_scene_query_interface
from pxr import Gf, PhysxSchema, Sdf, Usd, UsdGeom, UsdPhysics


# Add transformation properties to the prim (if not already present)
def set_transform_attributes(prim, location=None, orientation=None, rotation=None, scale=None):
    if location is not None:
        if not prim.HasAttribute("xformOp:translate"):
            UsdGeom.Xformable(prim).AddTranslateOp()
        prim.GetAttribute("xformOp:translate").Set(location)
    if orientation is not None:
        if not prim.HasAttribute("xformOp:orient"):
            UsdGeom.Xformable(prim).AddOrientOp()
        prim.GetAttribute("xformOp:orient").Set(orientation)
    if rotation is not None:
        if not prim.HasAttribute("xformOp:rotateXYZ"):
            UsdGeom.Xformable(prim).AddRotateXYZOp()
        prim.GetAttribute("xformOp:rotateXYZ").Set(rotation)
    if scale is not None:
        if not prim.HasAttribute("xformOp:scale"):
            UsdGeom.Xformable(prim).AddScaleOp()
        prim.GetAttribute("xformOp:scale").Set(scale)


# Enables collisions with the asset (without rigid body dynamics the asset will be static)
def add_colliders(root_prim):
    # Iterate descendant prims (including root) and add colliders to mesh or primitive types
    for desc_prim in Usd.PrimRange(root_prim):
        if desc_prim.IsA(UsdGeom.Mesh) or desc_prim.IsA(UsdGeom.Gprim):
            # Physics
            if not desc_prim.HasAPI(UsdPhysics.CollisionAPI):
                collision_api = UsdPhysics.CollisionAPI.Apply(desc_prim)
            else:
                collision_api = UsdPhysics.CollisionAPI(desc_prim)
            collision_api.CreateCollisionEnabledAttr(True)
            # PhysX
            if not desc_prim.HasAPI(PhysxSchema.PhysxCollisionAPI):
                physx_collision_api = PhysxSchema.PhysxCollisionAPI.Apply(desc_prim)
            else:
                physx_collision_api = PhysxSchema.PhysxCollisionAPI(desc_prim)
            # Set PhysX specific properties
            physx_collision_api.CreateContactOffsetAttr(0.001)
            physx_collision_api.CreateRestOffsetAttr(0.0)

        # Add mesh specific collision properties only to mesh types
        if desc_prim.IsA(UsdGeom.Mesh):
            # Add mesh collision properties to the mesh (e.g. collider approximation type)
            if not desc_prim.HasAPI(UsdPhysics.MeshCollisionAPI):
                mesh_collision_api = UsdPhysics.MeshCollisionAPI.Apply(desc_prim)
            else:
                mesh_collision_api = UsdPhysics.MeshCollisionAPI(desc_prim)
            mesh_collision_api.CreateApproximationAttr().Set("convexHull")


# Check if prim (or its descendants) has colliders
def has_colliders(root_prim):
    for desc_prim in Usd.PrimRange(root_prim):
        if desc_prim.HasAPI(UsdPhysics.CollisionAPI):
            return True
    return False
```

--------------------------------

### Get Fixed Tendon Offsets

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.prims/docs/index.html

Retrieves the offset values for fixed tendons of specified prims. These offsets can influence the starting position or reference point for tendon movements.

```python
>>> # get the fixed tendon offsets
>>> # for the ShadowHand articulation that has 4 fixed tendons (prims.num_fixed_tendons)
>>> prims.get_fixed_tendon_offsets()
[[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]]

```

--------------------------------

### Setup PoseWriter

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_pose_estimation.html

Retrieves the PoseWriter from the registry and attaches it to the render product.

```python
1    # setup writer
2    self.writer = rep.WriterRegistry.get("PoseWriter")
3    self.writer.initialize(...)
4    self.writer.attach([self.render_product])
```

--------------------------------

### Get Enabled Gravities

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.experimental.prims/docs/index.html

Checks if gravity is enabled for prims. Allows specifying prims with 'indices'. The example shows modifying gravity for a specific prim and then querying the state.

```python
>>> # get the gravity enabled state of all prims after disabling it for the second prim
>>> prims.set_enabled_gravities([False], indices=[1])
>>> print(prims.get_enabled_gravities())
[[ True]
 [False]
 [ True]]
```

--------------------------------

### Method Naming Conventions

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/docs/overview/guidelines.html

Method names must start with a verb to clearly indicate their action. Examples show correct usage for accessing attributes and performing actions.

```c++
myVector.getLength();
myObject.applyForce(x, y, z);
myObject.isDynamic();
texture.getFormat();
```

--------------------------------

### Run Periodic Camera Image Publishing Sample

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/ros2_tutorials/tutorial_ros2_python.html

Executes the camera periodic publishing example.

```bash
./python.sh standalone_examples/api/isaacsim.ros2.bridge/camera_periodic.py
```

--------------------------------

### ScopedTimer Example Usage

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/api/classisaacsim_1_1core_1_1includes_1_1_scoped_timer.html

Demonstrates how to use ScopedTimer to measure the execution time of a code block. The timer automatically starts on construction and prints the duration when the scope ends.

```cpp
{
    ScopedTimer timer("Operation X");
    // Code to measure...
} // Timer automatically prints duration when scope ends

```

--------------------------------

### Run Multi-Camera Replicator Example

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/replicator_tutorials/tutorial_replicator_isaac_snippets.html

Executes the multi-camera standalone example script. Use python.bat on Windows instead of python.sh.

```bash
./python.sh standalone_examples/api/isaacsim.replicator.examples/multi_camera.py
```

--------------------------------

### Enable PhysX Examples Extension via CLI

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.sensors.physx.examples/docs/index.html

Enable the isaacsim.sensors.physx.examples extension using the command-line interface.

```bash
APP_SCRIPT.(sh|bat) --enable isaacsim.sensors.physx.examples

```

--------------------------------

### Get Robot Setup Wizard Extension Setting Programmatically (Python)

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.robot_setup.wizard/docs/index.html

Use the carb framework in Python to retrieve the value of an extension-specific setting. Ensure the carb module is imported.

```python
import carb

settings = carb.settings.get_settings()
value = settings.get("/exts/isaacsim.robot_setup.wizard/SETTING_NAME")

```

--------------------------------

### initialize

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/api/classisaacsim_1_1sensors_1_1physx_1_1_range_sensor_manager.html

Initializes the application with a USD stage. Sets up the stage reference and creates the notice listener.

```APIDOC
## initialize

### Description
Initializes the application with a USD stage. Sets up the stage reference and creates the notice listener.

### Parameters
#### Path Parameters
- **stage** (pxr::UsdStageWeakPtr) - in - Weak pointer to the USD stage to be managed

### Post
Application is initialized with the stage and notice listener
```

--------------------------------

### Get Stage Time Code

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.experimental.utils/docs/index.html

Retrieves the start time, end time, and time codes per second for the current USD stage. This is essential for animation playback and timing control.

```python
import isaacsim.core.experimental.utils.stage as stage_utils

stage_utils.get_stage_time_code()

```

--------------------------------

### Get Enabled Rigid Bodies

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.experimental.prims/docs/index.html

Retrieves the enabled state of rigid body dynamics for prims. Supports selection via 'indices'. The example shows the default enabled state for all prims.

```python
>>> # get the rigid body dynamics enabled state of all prims
>>> print(prims.get_enabled_rigid_bodies())
[[ True]
 [ True]
 [ True]]
```

--------------------------------

### SimulationManager

Source: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/source/extensions/isaacsim.core.simulation_manager/docs/index.html

This class provides functions that take care of many time-related events such as warm starting simulation in order for the physics data to be retrievable. It also allows for adding/removing callback functions that get triggered with certain events such as a physics step, on post reset, on physics ready, etc.

```APIDOC
## SimulationManager

### Description
This class provides functions that take care of many time-related events such as warm starting simulation in order for the physics data to be retrievable. Adding/ removing callback functions that gets triggered with certain events such as a physics step, on post reset, on physics ready..etc.

### Methods

* `_assets_loading() -> bool`
  Checks if textures are loaded.

* `_deregister_callback(_callback_id_)`
  Deregisters a callback which was previously registered using register_callback.
  Parameters:
    * `callback_id` (any) - The ID of the callback returned by register_callback to deregister.

* `_enable_all_default_callbacks(_enable: bool = True_) -> None`
  Enable or disable all default callbacks. Default callbacks are: warm_start, on_stop, post_warm_start, stage_open.
  Parameters:
    * `enable` (bool) - Whether to enable all callbacks.

* `_enable_ccd(_flag: bool_, `_physics_scene: str = None_`) -> None`
  Enables Continuous Collision Detection (CCD).
  Parameters:
    * `flag` (bool) - enables or disables CCD on the PhysicsScene.
    * `physics_scene` (str, optional) - physics scene prim path.
  Raises:
    * Exception - If the prim path registered in context doesn’t correspond to a valid prim path currently.

* `_enable_fabric(_enable_) -> None`
  Enables or disables physics fabric integration and associated settings.
  Parameters:
    * `enable` (bool) - Whether to enable or disable fabric.

* `_enable_fabric_usd_notice_handler(_stage_id_, `_flag_) -> None`
  Enables or disables the fabric usd notice handler.
  Parameters:
    * `stage_id` (any) - The stage ID to enable or disable the handler for.
    * `flag` (any) - Whether to enable or disable the handler.

* `_enable_gpu_dynamics(_flag: bool_, `_physics_scene: str = None_`) -> None`
  Enables gpu dynamics pipeline, required for deformables for instance.
  Parameters:
    * `flag` (bool) - enables or disables gpu dynamics on the PhysicsScene. If flag is true, CCD is disabled.
    * `physics_scene` (str, optional) - physics scene prim path.
  Raises:
    * Exception - If the prim path registered in context doesn’t correspond to a valid prim path currently.

* `_enable_on_stop_callback(_enable: bool = True_) -> None`
  Enable or disable the on stop callback.
  Parameters:
    * `enable` (bool) - Whether to enable the callback.

* `_enable_post_warm_start_callback(_enable: bool = True_) -> None`
  Enable or disable the post warm start callback.
  Parameters:
    * `enable` (bool) - Whether to enable the callback.

* `_enable_stablization(_flag: bool_, `_physics_scene: str = None_`) -> None`
  Enables additional stabilization pass in the solver.
  Parameters:
    * `flag` (bool) - enables or disables stabilization on the PhysicsScene
    * `physics_scene` (str, optional) - physics scene prim path.
  Raises:
    * Exception - If the prim path registered in context doesn’t correspond to a valid prim path currently.

* `_enable_stage_open_callback(_enable: bool = True_) -> None`
  Enable or disable the stage open callback. Note: This also enables/disables the assets loading and loaded callbacks. If disabled, assets_loading() will always return True.
  Parameters:
    * `enable` (bool) - Whether to enable the callback.

* `_enable_usd_notice_handler(_flag_) -> None`
  Enables or disables the usd notice handler.
  Parameters:
    * `flag` (any) - Whether to enable or disable the handler.

* `_enable_warm_start_callback(_enable: bool = True_) -> None`
  Enable or disable the warm start callback.
  Parameters:
    * `enable` (bool) - Whether to enable the callback.

* `_get_backend() -> str`
  Returns the current backend.

* `_get_broadphase_type(_physics_scene: str = None_) -> str`
  Gets current broadcast phase algorithm type.
  Parameters:
    * `physics_scene` (str, optional) - physics scene prim path.
  Returns:
    * Broadcast phase algorithm used.
  Return type:
    * str
  Raises:
    * Exception - If the prim path registered in context doesn’t correspond to a valid prim path currently.

* `_get_default_callback_status() -> dict`
  Get the status of all default callbacks. Default callbacks are: warm_start, on_stop, post_warm_start, stage_open.
  Returns:
    * Dictionary with callback names and their enabled status.

* `_get_default_physics_scene() -> str`
  Returns the default physics scene.

* `_get_num_physics_steps() -> int`
  Returns the number of physics steps.

```
Tokens:
Raw