### Install wget on MacOS using Homebrew Source: https://github.com/castacks/tartanairpy/blob/main/docs/installation.md If wget is not installed by default on MacOS, use Homebrew to install it. ```bash brew install wget ``` -------------------------------- ### Install wget on Ubuntu Source: https://github.com/castacks/tartanairpy/blob/main/docs/installation.md If downloading fails on Ubuntu due to a missing wget utility, use this command to install it. ```bash sudo apt install wget ``` -------------------------------- ### Install TartanAir Python Package Source: https://github.com/castacks/tartanairpy/blob/main/docs/index.md Install the TartanAir dataset toolkit using pip. This is the first step to using the dataset. ```bash pip install tartanair ``` -------------------------------- ### Download All Modalities for Specific Environments and Robots Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Use this example to download all specified sensor modalities for a list of environments and robot versions. Set `unzip=False` to keep downloaded files compressed. ```python import tartanair as ta ta.init('/path/to/tartanground/root') env = ["AbandonedFactory", "ConstructionSite", "Hospital"] ta.download_ground( env = env, version = ['omni', 'diff', 'anymal'], traj = [], modality = [ 'image', 'meta', 'depth', 'seg', 'lidar', 'imu', 'rosbag', 'sem_pcd', 'seg_labels', 'rgb_pcd' ], camera_name = ['lcam_front', 'lcam_right', 'lcam_left', 'lcam_back'], unzip = False ) ``` -------------------------------- ### Install TartanGround Dataset Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Steps to create a conda environment, clone the repository, update submodules, and install the TartanGround package. ```bash # 1. Create and activate conda environment conda create -n tartanground python=3.9 conda activate tartanground # 2. Clone repository with all submodules git clone --recursive git@github.com:castacks/tartanairpy.git cd tartanairpy # 3. Ensure submodules are up to date git submodule update --init --recursive # 4. Install the package pip install -e . ``` -------------------------------- ### Install pytransform3d on MacOS Source: https://github.com/castacks/tartanairpy/blob/main/docs/installation.md If pytransform3d fails to install via pip on MacOS, use this command for manual installation. ```bash python3 -m pip install pytransform3d ``` -------------------------------- ### Download Entire TartanGround Dataset Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md This example downloads the complete TartanGround dataset, approximately 15 TB in size. Ensure sufficient disk space and network bandwidth before running. ```python import tartanair as ta ta.init('/path/to/tartanground/root') ta.download_ground( env = [], version = [], traj = [], modality = [], camera_name = [], unzip = False ) ``` -------------------------------- ### Install PyYAML on MacOS Source: https://github.com/castacks/tartanairpy/blob/main/docs/installation.md If PyYAML fails to install via pip on MacOS, use this command for manual installation. ```bash python3 -m pip install pyyaml ``` -------------------------------- ### Load Occupancy Maps in Python Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Example of how to load generated occupancy map data using NumPy. This code snippet demonstrates the initial import and preparation for data loading. ```python import numpy as np ``` -------------------------------- ### Multi-threaded Download of TartanGround Dataset Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md This example utilizes multi-threading for faster dataset downloads. Specify the number of worker threads using `num_workers` for improved download speeds. ```python import tartanair as ta ta.init('/path/to/tartanground/root') ta.download_ground_multi_thread( env = env, version = ['omni', 'diff', 'anymal'], traj = [], modality = [ 'image', 'meta', 'depth', 'seg', 'lidar', 'imu', 'rosbag', 'sem_pcd', 'seg_labels', 'rgb_pcd' ], camera_name = ['lcam_front', 'lcam_right', 'lcam_left', 'lcam_back'], unzip = False, num_workers = 8 ) ``` -------------------------------- ### Download One Trajectory for Omnidirectional Robot Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md This example downloads a single specified trajectory for the omnidirectional robot version. Ensure `modality` and `camera_name` are set appropriately for the desired data. ```python import tartanair as ta ta.init('/path/to/tartanground/root') ta.download_ground( env = [], version = ['omni'], traj = ['P0000'], modality = [], camera_name = ['lcam_front'], unzip = False ) ``` -------------------------------- ### Install opencv-contrib-python on MacOS Source: https://github.com/castacks/tartanairpy/blob/main/docs/installation.md If opencv-contrib-python fails to install via pip on MacOS, use this command for manual installation. This may take a while. ```bash python3 -m pip install opencv-contrib-python ``` -------------------------------- ### TartanAir Evaluation Setup Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md This snippet shows the basic imports for TartanAir evaluation, including NumPy. It sets the stage for using ATE and RPE metrics for trajectory analysis. ```python import tartanair as ta import numpy as np ``` -------------------------------- ### Reinstall Pillow and Torchvision on MacOS Source: https://github.com/castacks/tartanairpy/blob/main/docs/installation.md If Pillow causes issues with older torchvision versions on MacOS, manually install Pillow, uninstall torchvision, and then reinstall torchvision. ```bash python3 -m pip install Pillow python3 -m pip uninstall torchvision python3 -m pip install torchvision ``` -------------------------------- ### Visualize Semantic Occupancy Maps (Basic) Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Launch the interactive viewer to inspect generated semantic occupancy maps. Use navigation controls to explore the 3D environment. ```bash python examples/visualize_semantic_occ_local.py \ --root_dir /path/to/tartanground/dataset \ --env ConstructionSite \ --traj P0000 \ --skip_samples 100 ``` -------------------------------- ### Visualize Semantic Occupancy Maps (Customized) Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Customize the visualization appearance by adjusting point size and background color. Control which occupancy maps are displayed using skip_samples. ```bash python examples/visualize_semantic_occ_local.py \ --root_dir /path/to/dataset \ --env ConstructionSite \ --traj P0000 \ --skip_samples 50 \ --point_size 12.0 \ --background white ``` -------------------------------- ### Generate Semantic Occupancy Maps (Customized) Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Customize occupancy map generation by setting resolution, bounds, and pose subsampling. This allows fine-tuning the output for specific analysis needs. ```bash python examples/subsample_semantic_pcd_gpu.py \ --root_dir /path/to/dataset \ --env ConstructionSite \ --traj P0000 \ --resolution 0.1 \ --x_bounds -20 20 \ --y_bounds -20 20 \ --z_bounds -3 5 \ --subsample_poses 10 ``` -------------------------------- ### Generate Semantic Occupancy Maps (Basic) Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Use this script to extract local semantic occupancy maps from global point clouds. Specify the dataset root, environment, and trajectory to process. ```bash python examples/subsample_semantic_pcd_gpu.py \ --root_dir /path/to/tartanground/dataset \ --env ConstructionSite \ --traj P0000 ``` -------------------------------- ### Initialize TartanAir and Specify Dataloader Parameters Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md Initializes TartanAir and sets up parameters for loading data, including environments, difficulties, and trajectory IDs. This prepares the dataloader for efficient batch processing. ```python import tartanair as ta import numpy as np import cv2 # Initialize TartanAir. tartanair_data_root = '/my/path/to/root/folder/for/tartanair-v2' ta.init(tartanair_data_root) # Specify the environments, difficulties, and trajectory ids to load. envs = ['ArchVizTinyHouseDay'] difficulties = ['easy'] trajectory_ids = ['P000', 'P001'] ``` -------------------------------- ### Download Semantic Occupancy Data Only Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md This snippet demonstrates how to download only the semantic segmentation labels and point clouds. Leave `env`, `version`, and `camera_name` empty to avoid downloading other data types. ```python import tartanair as ta ta.init('/path/to/tartanground/root') ta.download_ground( env = [], version = [], traj = [], modality = ['seg_labels', 'sem_pcd'], camera_name = [], unzip = False ) ``` -------------------------------- ### Run All TartanAir-V2 Tests Source: https://github.com/castacks/tartanairpy/blob/main/tests/README.md Execute all sanity tests for the TartanAir-V2 Python package. Ensure you are in the root of the repository. Data download tests require `self.azure_token` to be set. ```bash python3 tartanairpy_test.py ``` -------------------------------- ### Customize TartanAir V2 Dataset with New Camera Models Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md Initializes TartanAir and customizes the dataset by adding new camera models with specified parameters and intrinsic/extrinsic properties. Supports GPU acceleration. ```python import tartanair as ta # For help with rotations. from scipy.spatial.transform import Rotation # Initialize TartanAir. tartanair_data_root = '/my/path/to/root/folder/for/tartanair-v2' ta.init(tartanair_data_root) # Create your camera model(s). R_raw_new0 = Rotation.from_euler('y', 90, degrees=True).as_matrix().tolist() cam_model_0 = {'name': 'pinhole', 'raw_side': 'left', # TartanAir has two cameras, one on the left and one on the right. This parameter specifies which camera to use. 'params': {'fx': 320, 'fy': 320, 'cx': 320, 'cy': 320, 'width': 640, 'height': 640}, 'R_raw_new': R_raw_new0} R_raw_new1 = Rotation.from_euler('xyz', [45, 0, 0], degrees=True).as_matrix().tolist() cam_model_1 = {'name': 'doublesphere', 'raw_side': 'left', 'params': {'fx': 300, 'fy': 300, 'cx': 500, 'cy': 500, 'width': 1000, 'height': 1000, 'alpha': 0.6, 'xi': -0.2, 'fov_degree': 195}, 'R_raw_new': R_raw_new1} # Customize the dataset. ta.customize(env = 'ArchVizTinyHouseDay', difficulty = 'easy', trajectory_id = ['P000'], modality = ['image', 'depth'], new_camera_models_params=[cam_model_1, cam_model_0], num_workers = 4, device = "cuda") # or cpu ``` -------------------------------- ### Create TartanAir Data Iterator Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md This snippet demonstrates how to initialize TartanAir and create an iterator to fetch samples for a specific modality and camera. The iterator can be used to loop through a dataset. ```python import tartanair as ta # Initialize TartanAir. tartanair_data_root = '/my/path/to/root/folder/for/tartanair-v2' ta.init(tartanair_data_root) # Create iterator. ta_iterator = ta.iterator(env = ['ArchVizTinyHouseDay'], difficulty = 'easy', trajectory_id = [], modality = 'image', camera_name = ['lcam_left']) for i in range(100): sample = next(ta_iterator) ``` -------------------------------- ### Download TartanAir V2 Data via Python API Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md Initializes TartanAir and downloads specified trajectories with various modalities and cameras. Supports unzipping and parallel downloads. ```python import tartanair as ta # Initialize TartanAir. tartanair_data_root = '/my/path/to/root/folder/for/tartanair-v2' ta.init(tartanair_data_root) # Download a trajectory. ta.download(env = "ArchVizTinyHouseDay", difficulty = ['easy'], # this can be 'easy', and/or 'hard' modality = ['image', 'depth', 'seg', 'imu'], # available modalities are: image', 'depth', 'seg', 'imu', 'lidar', 'flow' camera_name = ['lcam_front', 'lcam_left', 'lcam_right', 'lcam_back', 'lcam_top', 'lcam_bottom'], unzip = True, delete_zip = False, num_workers = 4) # unzip files autonomously after download # To download the entire dataset all_data = ta.get_all_data() # this will fill in all available TAv2 data for env, difficulty, modality and camera_name ta.download(**all_data, unzip = True, num_workers = 8) ``` -------------------------------- ### Load and Inspect Occupancy Map Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Load a single occupancy map from a .npz file using NumPy. Access and print the shape of the occupancy map, the resolution in meters per voxel, and the count of occupied voxels. ```python import numpy as np # Load a single occupancy map data = np.load('semantic_occupancy_000000.npz', allow_pickle=True) occupancy_map = data['occupancy_map'] # 3D array with class IDs pose = data['pose'] # Robot pose [x,y,z,qx,qy,qz,qw] bounds = data['bounds'] # Local bounds [x_min,x_max,y_min,y_max,z_min,z_max] resolution = data['resolution'] # Voxel size in meters class_mapping = data['class_mapping'] # Class ID to RGB mapping print(f"Occupancy map shape: {occupancy_map.shape}") print(f"Resolution: {resolution}m per voxel") print(f"Occupied voxels: {np.sum(occupancy_map > 0)}") ``` -------------------------------- ### Load and Visualize TartanAir Data Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md This snippet shows how to create a dataloader with specified modalities and parameters, iterate through batches, and visualize image data along with pose and IMU information. Ensure all necessary libraries like OpenCV and NumPy are imported. ```python import tartanair as ta import numpy as np import cv2 # Assume envs, difficulties, trajectory_ids are defined elsewhere # Example placeholder definitions: envs = ['ArchVizTinyHouseDay'] difficulties = ['easy'] trajectory_ids = [] # Specify the modalities to load. modalities = ['image', 'pose', 'imu'] camnames = ['lcam_front', 'lcam_left', 'lcam_right', 'lcam_back', 'lcam_top', 'lcam_bottom'] # Specify the dataloader parameters. new_image_shape_hw = [640, 640] # If None, no resizing is performed. If a value is passed, then the image is resized to this shape. subset_framenum = 200 # This is the number of frames in a subset. Notice that this is an upper bound on the batch size. Ideally, make this number large to utilize your RAM efficiently. Information about the allocated memory will be provided in the console. seq_length = {'image': 2, 'pose': 2, 'imu': 10} # This is the length of the data-sequences. For example, if the sequence length is 2, then the dataloader will load pairs of images. seq_stride = 1 # This is the stride between the data-sequences. For example, if the sequence length is 2 and the stride is 1, then the dataloader will load pairs of images [0,1], [1,2], [2,3], etc. If the stride is 2, then the dataloader will load pairs of images [0,1], [2,3], [4,5], etc. frame_skip = 0 # This is the number of frames to skip between each frame. For example, if the frame skip is 2 and the sequence length is 3, then the dataloader will load frames [0, 3, 6], [1, 4, 7], [2, 5, 8], etc. batch_size = 8 # This is the number of data-sequences in a mini-batch. num_workers = 4 # This is the number of workers to use for loading the data. shuffle = True # Whether to shuffle the data. Let's set this to False for now, so that we can see the data loading in a nice video. Yes it is nice don't argue with me please. Just look at it! So nice. :) # Create a dataloader object. dataloader = ta.dataloader(env = envs, difficulty = difficulties, trajectory_id = trajectory_ids, modality = modalities, camera_name = camnames, new_image_shape_hw = new_image_shape_hw, seq_length = seq_length, subset_framenum = subset_framenum, seq_stride = seq_stride, frame_skip = frame_skip, batch_size = batch_size, num_workers = num_workers, shuffle = shuffle, verbose = True) # Iterate over the batches. for i in range(100): # Get the next batch. batch = dataloader.load_sample() # Visualize some images. # The shape of an image batch is (B, S, H, W, C), where B is the batch size, S is the sequence length, H is the height, W is the width, and C is the number of channels. print("Batch number: {}".format(i+1), "Loaded {} samples so far.".format((i+1) * batch_size)) for b in range(batch_size): # Create image cross. left = batch['image_lcam_left'][b][0].numpy().transpose(1,2,0) front = batch['image_lcam_front'][b][0].numpy().transpose(1,2,0) right = batch['image_lcam_right'][b][0].numpy().transpose(1,2,0) back = batch['image_lcam_back'][b][0].numpy().transpose(1,2,0) top = batch['image_lcam_top'][b][0].numpy().transpose(1,2,0) bottom = batch['image_lcam_bottom'][b][0].numpy().transpose(1,2,0) cross_mid = np.concatenate([left, front, right, back], axis=1) cross_top = np.concatenate([np.zeros_like(top), top, np.zeros_like(top), np.zeros_like(top)], axis=1) cross_bottom = np.concatenate([np.zeros_like(bottom), bottom, np.zeros_like(bottom), np.zeros_like(bottom)], axis=1) cross = np.concatenate([cross_top, cross_mid, cross_bottom], axis=0) pose = batch['pose_lcam_front'].numpy() imu = batch['imu'].numpy() # Resize. cross = cv2.resize(cross, (cross.shape[1]//4, cross.shape[0]//4)) # Show the image cross. cv2.imshow('cross', cross) cv2.waitKey(100) print(" Pose: ", pose[0][0]) print(" IMU: ", imu[0][0]) dataloader.stop_cachers() ``` -------------------------------- ### Download TartanGround Data Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md Initializes TartanAir and downloads TartanGround trajectories with specified versions, modalities, and cameras. Supports different data sources and parallel downloads. ```python import tartanair as ta # Initialize TartanAir. tartanground_data_root = '/my/path/to/root/folder/for/tartanair-v2' ta.init(tartanground_data_root) # Download a trajectory. ta.download_ground(env = ['OldTownSummer', 'DesertGasStation'], version = ['omni', 'diff', 'anymal'], # available versions are: 'v1', 'v2', 'v3_anymal' modality = ['image', 'depth', 'seg', 'lidar', 'imu'], # available modalities are: image', 'depth', 'seg', 'imu', 'lidar' camera_name = ['lcam_front', 'lcam_bottom'], # available cameras are: 'lcam_front', 'lcam_left', 'lcam_right', 'lcam_back', 'lcam_top', 'lcam_bottom' unzip = True, # unzip files autonomously after download delete_zip = False, num_workers = 4, data_source = 'huggingface') # data_source can be 'huggingface' or 'airlab' ``` -------------------------------- ### Initialize TartanAir for Flow Sampling Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md Initializes TartanAir and customizes parameters for flow sampling. This is used for calculating dense correspondences between camera views. ```python import tartanair as ta # For help with rotations. from scipy.spatial.transform import Rotation # Initialize TartanAir. tartanair_data_root = 'your/path/to_tav2' ta.init(tartanair_data_root) # Customize the dataset. ta.customize_flow(env = 'ArchVizTinyHouseDay', difficulty = 'easy', trajectory_id = ['P000'], camera_name=["lcam_left", "lcam_back"], num_workers = 4, frame_sep = 1, device = "cuda") # or cpu ``` -------------------------------- ### Initialize TartanAir and Evaluate Trajectory Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md Initializes TartanAir and evaluates an estimated trajectory against ground truth. Supports plotting and alignment options. ```python import tartanair as ta import numpy as np # Initialize TartanAir. tartanair_data_root = '/my/path/to/root/folder/for/tartanair-v2' ta.init(tartanair_data_root) # Create an example trajectory. This is a noisy version of the ground truth trajectory. env = 'ArchVizTinyHouseDay' difficulty = 'easy' trajectory_id = 'P002' camera_name = 'lcam_front' gt_traj = ta.get_traj_np(env, difficulty, trajectory_id, camera_name) est_traj = gt_traj + np.random.normal(0, 0.1, gt_traj.shape) # Pass the ground truth trajectory directly to the evaluation function. results = ta.evaluate_traj(est_traj, gt_traj = gt_traj, enforce_length = True, plot = True, plot_out_path = plot_out_path, do_scale = True, do_align = True) # Or pass the environment, difficulty, and trajectory id to the evaluation function. plot_out_path = "evaluator_example_plot.png" results = ta.evaluate_traj(est_traj, env = env, difficulty = difficulty, trajectory_id = trajectory_id, camera_name = camera_name, enforce_length = True, plot = True, plot_out_path = plot_out_path, do_scale = True, do_align = True) ``` -------------------------------- ### TartanGround Dataset Directory Layout Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Illustrates the hierarchical organization of the TartanGround dataset, including environments, robot data, and sensor modalities. ```text TartanGround_Root/ ├── AbandonedCable/ │ ├── AbandonedCable_rgb.pcd # Global RGB point cloud │ ├── AbandonedCable_sem.pcd # Global Semantic point cloud │ ├── seg_label_map.json # Semantic segmentation label map │ ├── Data_omni/ # Omnidirectional robot data │ │ ├── P0000/ │ │ │ ├── image_lcam_front/ │ │ │ ├── depth_lcam_front/ │ │ │ ├── seg_lcam_front/ │ │ │ ├── imu/ │ │ │ ├── lidar/ │ │ │ ├── pose_lcam_front.txt │ │ │ ├── P0000_metadata.json │ │ │ ├── image_lcam_left/ │ │ │ └── ... │ │ └── P00XX/ │ ├── Data_diff/ # Differential drive robot data │ │ ├── P1000/ │ │ └── P10XX/ │ └── Data_anymal/ # Quadrupedal robot data │ ├── P2000/ │ └── P20XX/ ├── AbandonedFactory/ │ └── (same structure as above) └── ... ``` -------------------------------- ### Available Left Camera Names Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Lists the names for the left-side cameras in the stereo configuration. ```python ['lcam_front', 'lcam_right', 'lcam_left', 'lcam_back', 'lcam_top', 'lcam_bottom'] ``` -------------------------------- ### Available Right Camera Names Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md Lists the names for the right-side cameras in the stereo configuration. ```python ['rcam_front', 'rcam_right', 'rcam_left', 'rcam_back', 'rcam_top', 'rcam_bottom'] ``` -------------------------------- ### List Available Modalities Source: https://github.com/castacks/tartanairpy/blob/main/docs/tartanground.md This snippet shows the complete list of sensor modalities available in the TartanGround dataset. It is useful for understanding the data types that can be downloaded. ```python # Complete list of available modalities ['image', 'meta', 'depth', 'seg', 'lidar', 'imu', 'rosbag', 'sem_pcd', 'seg_labels', 'rgb_pcd'] ``` -------------------------------- ### Depth Interpolation Formula Source: https://github.com/castacks/tartanairpy/blob/main/docs/flow_sampling.md Formula for obtaining depth values at fractional pixel locations using bilinear interpolation. ```latex \tilde{D}_2[i_2] = \text{bilinear}(\tilde{D}_2, i_2) ``` -------------------------------- ### Download TartanAir V2 Data via YAML Config Source: https://github.com/castacks/tartanairpy/blob/main/docs/examples.md Downloads TartanAir V2 data using a configuration specified in a YAML file. This is useful for managing download parameters. ```python ta.download(config = 'download_config.yaml') ``` ```yaml env: ['ArchVizTinyHouseDay'] difficulty: ['easy'] modality: ['image', 'depth'] camera_name: ['lcam_front', 'lcam_left', 'lcam_right', 'lcam_back', 'lcam_top', 'lcam_bottom'] unzip: True ``` -------------------------------- ### Run Specific TartanAir-V2 Test Source: https://github.com/castacks/tartanairpy/blob/main/tests/README.md Execute a specific test case within the TartanAir-V2 Python package tests. Replace `TartanAirTest.test_customization` with the desired test name. ```bash python3 tartanairpy_test.py TartanAirTest.test_customization ``` -------------------------------- ### Fix SSL Certificate Verification on MacOS Source: https://github.com/castacks/tartanairpy/blob/main/docs/installation.md If URLLib encounters SSL certificate verification errors on MacOS, run this command to fix certificate issues (adapt the Python version as needed). ```bash /Applications/Python\ 3.8/Install\ Certificates.command ``` -------------------------------- ### Decode Depth Image Source: https://github.com/castacks/tartanairpy/blob/main/docs/modalities.md Reads a compressed 4-channel PNG depth image and decodes it into a float32 depth map. This is useful for handling the lossless compression applied to depth data. ```python import cv2 import numpy as np def read_decode_depth(depthpath): depth_rgba = cv2.imread(depthpath, cv2.IMREAD_UNCHANGED) depth = depth_rgba.view("