### Install Dependencies Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Install the necessary dependencies for Monodepth2 using Anaconda and pip. Ensure you use the specified versions for compatibility. ```shell conda install pytorch=0.4.1 torchvision=0.2.1 -c pytorch pip install tensorboardX==1.4 conda install opencv=3.3.1 # just needed for evaluation ``` -------------------------------- ### View training options Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Displays the help menu for training script arguments. ```shell python train.py -h ``` -------------------------------- ### Initialize Network and Load Weights Source: https://github.com/nianticlabs/monodepth2/blob/master/depth_prediction_example.ipynb Downloads the pretrained model and loads the encoder and decoder state dictionaries into the network. ```python model_name = "mono_640x192" download_model_if_doesnt_exist(model_name) encoder_path = os.path.join("models", model_name, "encoder.pth") depth_decoder_path = os.path.join("models", model_name, "depth.pth") # LOADING PRETRAINED MODEL encoder = networks.ResnetEncoder(18, False) depth_decoder = networks.DepthDecoder(num_ch_enc=encoder.num_ch_enc, scales=range(4)) loaded_dict_enc = torch.load(encoder_path, map_location='cpu') filtered_dict_enc = {k: v for k, v in loaded_dict_enc.items() if k in encoder.state_dict()} encoder.load_state_dict(filtered_dict_enc) loaded_dict = torch.load(depth_decoder_path, map_location='cpu') depth_decoder.load_state_dict(loaded_dict) encoder.eval() depth_decoder.eval(); ``` -------------------------------- ### Initialize KITTI Datasets and DataLoader Source: https://context7.com/nianticlabs/monodepth2/llms.txt Configures data loading for KITTI RAW and Odometry datasets, including frame indexing and image dimensions. ```python # Load filenames from split file train_filenames = readlines("splits/eigen_zhou/train_files.txt") # Format: "2011_09_26/2011_09_26_drive_0001_sync 0 l" # (folder, frame_index, side) # Create KITTI RAW dataset dataset = KITTIRAWDataset( data_path="kitti_data/", filenames=train_filenames, height=192, width=640, frame_idxs=[0, -1, 1], # Current frame and neighbors num_scales=4, is_train=True, img_ext='.jpg' ) # DataLoader dataloader = DataLoader( dataset, batch_size=12, shuffle=True, num_workers=12, pin_memory=True, drop_last=True ) # Batch structure for inputs in dataloader: # Color images at different scales color_0_0 = inputs[("color", 0, 0)] # [B, 3, 192, 640] current frame color_m1_0 = inputs[("color", -1, 0)] # Previous frame color_p1_0 = inputs[("color", 1, 0)] # Next frame # Augmented versions for training color_aug = inputs[("color_aug", 0, 0)] # With color jitter # Camera intrinsics K = inputs[("K", 0)] # [B, 4, 4] inv_K = inputs[("inv_K", 0)] # Ground truth depth (if available) if "depth_gt" in inputs: depth_gt = inputs["depth_gt"] # [B, 1, H, W] break # Odometry dataset for pose evaluation odom_filenames = readlines("splits/odom/train_files.txt") odom_dataset = KITTIOdomDataset( data_path="kitti_odom/", filenames=odom_filenames, height=192, width=640, frame_idxs=[0, 1], num_scales=4, is_train=False ) ``` -------------------------------- ### Finetune a pretrained model Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Loads weights from a specific folder to continue training or perform finetuning. ```shell python train.py --model_name finetuned_mono --load_weights_folder ~/tmp/mono_model/models/weights_19 ``` -------------------------------- ### Import Dependencies Source: https://github.com/nianticlabs/monodepth2/blob/master/depth_prediction_example.ipynb Initial imports for image processing, visualization, and PyTorch model handling. ```python from __future__ import absolute_import, division, print_function %matplotlib inline import os import numpy as np import PIL.Image as pil import matplotlib.pyplot as plt import torch from torchvision import transforms import networks from utils import download_model_if_doesnt_exist ``` -------------------------------- ### Train depth estimation models Source: https://context7.com/nianticlabs/monodepth2/llms.txt Use the train.py script to initiate training with various configurations, including monocular, stereo, or combined training modes. ```bash # Basic monocular training python train.py --model_name mono_model # Stereo-only training with full Eigen split python train.py --model_name stereo_model \ --frame_ids 0 --use_stereo --split eigen_full # Combined monocular + stereo training python train.py --model_name mono+stereo_model \ --frame_ids 0 -1 1 --use_stereo # Training with custom parameters python train.py --model_name custom_model \ --data_path /path/to/kitti_data \ --log_dir /path/to/logs \ --height 192 \ --width 640 \ --batch_size 12 \ --num_epochs 20 \ --learning_rate 1e-4 \ --num_layers 18 # Finetuning from pretrained weights python train.py --model_name finetuned_mono \ --load_weights_folder ~/tmp/mono_model/models/weights_19 # Training with specific GPU CUDA_VISIBLE_DEVICES=2 python train.py --model_name mono_model ``` -------------------------------- ### Load and Preprocess Test Image Source: https://github.com/nianticlabs/monodepth2/blob/master/depth_prediction_example.ipynb Loads an image from disk and resizes it to match the model's expected input dimensions. ```python image_path = "assets/test_image.jpg" input_image = pil.open(image_path).convert('RGB') original_width, original_height = input_image.size feed_height = loaded_dict_enc['height'] feed_width = loaded_dict_enc['width'] input_image_resized = input_image.resize((feed_width, feed_height), pil.LANCZOS) input_image_pytorch = transforms.ToTensor()(input_image_resized).unsqueeze(0) ``` -------------------------------- ### Initialize and Use PoseDecoder for Camera Pose Estimation Source: https://context7.com/nianticlabs/monodepth2/llms.txt Initializes a ResnetEncoder and PoseDecoder for predicting 6-DoF camera pose between consecutive frames. Requires stacking input frames and outputs axis-angle rotation and translation vectors. ```python import torch import networks from layers import transformation_from_parameters # Create pose encoder (for 2 input images) and decoder pose_encoder = networks.ResnetEncoder(18, pretrained=True, num_input_images=2) pose_decoder = networks.PoseDecoder( num_ch_enc=pose_encoder.num_ch_enc, num_input_features=1, num_frames_to_predict_for=2 ) # Stack two consecutive frames frame_t = torch.randn(4, 3, 192, 640) # Current frame frame_t1 = torch.randn(4, 3, 192, 640) # Next frame stacked = torch.cat([frame_t, frame_t1], dim=1) # [4, 6, 192, 640] # Get pose prediction features = [pose_encoder(stacked)] axisangle, translation = pose_decoder(features) # axisangle: [B, num_frames, 1, 3] rotation in axis-angle representation # translation: [B, num_frames, 1, 3] translation vector # Convert to 4x4 transformation matrix T = transformation_from_parameters( axisangle[:, 0], # [B, 1, 3] translation[:, 0], # [B, 1, 3] invert=False ) # T: [B, 4, 4] transformation matrix from frame t to frame t+1 ``` -------------------------------- ### Train Monodepth2 models Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Commands for initiating training for different modalities. ```shell python train.py --model_name mono_model ``` ```shell python train.py --model_name stereo_model \ --frame_ids 0 --use_stereo --split eigen_full ``` ```shell python train.py --model_name mono+stereo_model \ --frame_ids 0 -1 1 --use_stereo ``` -------------------------------- ### Initialize KITTI Dataset Loaders Source: https://context7.com/nianticlabs/monodepth2/llms.txt Imports and initializes dataset loader classes for KITTI raw, odometry, and depth benchmark data. These classes handle data loading and support data augmentation. ```python from datasets import KITTIRAWDataset, KITTIOdomDataset from torch.utils.data import DataLoader from utils import readlines ``` -------------------------------- ### Initialize DepthDecoder Source: https://context7.com/nianticlabs/monodepth2/llms.txt Sets up a U-Net style decoder to process encoder features into multi-scale disparity maps. ```python import torch import networks from layers import disp_to_depth # Create encoder and decoder encoder = networks.ResnetEncoder(18, pretrained=True) depth_decoder = networks.DepthDecoder( num_ch_enc=encoder.num_ch_enc, # [64, 64, 128, 256, 512] scales=range(4), # Output at scales 0, 1, 2, 3 num_output_channels=1, # Single channel disparity use_skips=True # Use skip connections ) # Forward pass input_image = torch.randn(1, 3, 192, 640) features = encoder(input_image) outputs = depth_decoder(features) # Outputs dictionary with multi-scale disparities disp_0 = outputs[("disp", 0)] # [B, 1, 192, 640] - full resolution disp_1 = outputs[("disp", 1)] # [B, 1, 96, 320] disp_2 = outputs[("disp", 2)] # [B, 1, 48, 160] disp_3 = outputs[("disp", 3)] # [B, 1, 24, 80] # Convert disparity to depth min_depth, max_depth = 0.1, 100.0 scaled_disp, depth = disp_to_depth(disp_0, min_depth, max_depth) # depth: values in [min_depth, max_depth] meters ``` -------------------------------- ### Download Pretrained Models Source: https://context7.com/nianticlabs/monodepth2/llms.txt Utility function to download pretrained model weights. ```python from utils import download_model_if_doesnt_exist ``` -------------------------------- ### Download Pretrained Model Source: https://context7.com/nianticlabs/monodepth2/llms.txt Downloads the specified model weights if they do not already exist locally, verifying the integrity via MD5 hash. ```python model_name = "mono+stereo_640x192" download_model_if_doesnt_exist(model_name) ``` -------------------------------- ### Download KITTI dataset Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Downloads the raw KITTI dataset using the provided split file. ```shell wget -i splits/kitti_archives_to_download.txt -P kitti_data/ ``` -------------------------------- ### Configure GPU usage Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Specifies the GPU device index for training using environment variables. ```shell CUDA_VISIBLE_DEVICES=2 python train.py --model_name mono_model ``` -------------------------------- ### Initialize ResNetEncoder Source: https://context7.com/nianticlabs/monodepth2/llms.txt Configures a ResNet-based feature encoder for depth or pose estimation tasks. ```python import torch import networks # Create encoder with ImageNet pretrained weights encoder = networks.ResnetEncoder(num_layers=18, pretrained=True) encoder.eval() # Input: RGB image tensor [B, 3, H, W], values in [0, 1] input_image = torch.randn(1, 3, 192, 640) # Output: list of 5 feature maps at different scales features = encoder(input_image) # features[0]: [B, 64, H/2, W/2] # features[1]: [B, 64, H/4, W/4] # features[2]: [B, 128, H/8, W/8] # features[3]: [B, 256, H/16, W/16] # features[4]: [B, 512, H/32, W/32] print(encoder.num_ch_enc) # [64, 64, 128, 256, 512] # Encoder for pose network (takes 2 stacked images) pose_encoder = networks.ResnetEncoder( num_layers=18, pretrained=True, num_input_images=2 ) # Input: 2 stacked RGB images [B, 6, H, W] stacked_images = torch.randn(1, 6, 192, 640) pose_features = pose_encoder(stacked_images) ``` -------------------------------- ### Evaluate Odometry Model (Split 10) Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Evaluate odometry predictions for split 10. Ensure the KITTI odometry dataset is downloaded and unzipped, and specify the data path and model weights folder. ```shell python evaluate_pose.py --eval_split odom_10 --load_weights_folder ./odom_split.M/models/weights_29 --data_path kitti_odom/ ``` -------------------------------- ### Prepare KITTI Ground Truth Depth Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Run these commands to export ground truth depth maps for KITTI dataset evaluation. Ensure the KITTI dataset is located in the specified data path. ```shell python export_gt_depth.py --data_path kitti_data --split eigen ``` ```shell python export_gt_depth.py --data_path kitti_data --split eigen_benchmark ``` -------------------------------- ### Evaluate Depth Models Source: https://context7.com/nianticlabs/monodepth2/llms.txt Runs evaluation on monocular or stereo models with various configuration options like post-processing or custom splits. ```bash python evaluate_depth.py \ --load_weights_folder ~/tmp/mono_model/models/weights_19/ \ --eval_mono ``` ```bash python evaluate_depth.py \ --load_weights_folder ~/tmp/stereo_model/models/weights_19/ \ --eval_stereo ``` ```bash python evaluate_depth.py \ --load_weights_folder ~/tmp/mono_model/models/weights_19/ \ --eval_mono \ --eval_split eigen_benchmark ``` ```bash python evaluate_depth.py \ --load_weights_folder ~/tmp/mono_model/models/weights_19/ \ --eval_mono \ --post_process ``` ```bash python evaluate_depth.py --ext_disp_to_eval ~/other_method_disp.npy ``` ```bash python evaluate_depth.py \ --load_weights_folder ~/tmp/mono_model/models/weights_19/ \ --eval_mono \ --eval_split benchmark \ --no_eval ``` -------------------------------- ### Backproject Depth and Project 3D Points for View Synthesis Source: https://context7.com/nianticlabs/monodepth2/llms.txt Utilizes BackprojectDepth to convert depth maps to 3D points and Project3D to project these points into a target camera view. These layers enable differentiable view synthesis. ```python from layers import BackprojectDepth, Project3D import torch batch_size, height, width = 4, 192, 640 # Initialize projection layers backproject = BackprojectDepth(batch_size, height, width) project = Project3D(batch_size, height, width) # Inputs depth = torch.rand(batch_size, 1, height, width) * 10 + 0.1 # Depth in meters K = torch.eye(4).unsqueeze(0).repeat(batch_size, 1, 1) # Intrinsics inv_K = torch.inverse(K) T = torch.eye(4).unsqueeze(0).repeat(batch_size, 1, 1) # Transformation # Backproject to 3D points cam_points = backproject(depth, inv_K) # cam_points: [B, 4, H*W] homogeneous 3D points # Project to target view pix_coords = project(cam_points, K, T) # pix_coords: [B, H, W, 2] sampling coordinates in range [-1, 1] # Use for view synthesis with grid_sample source_image = torch.randn(batch_size, 3, height, width) warped_image = torch.nn.functional.grid_sample( source_image, pix_coords, padding_mode="border", align_corners=True ) ``` -------------------------------- ### Evaluate Odometry Model (Split 9) Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Evaluate odometry predictions for split 9. Ensure the KITTI odometry dataset is downloaded and unzipped, and specify the data path and model weights folder. ```shell python evaluate_pose.py --eval_split odom_9 --load_weights_folder ./odom_split.M/models/weights_29 --data_path kitti_odom/ ``` -------------------------------- ### Convert KITTI images to JPEG Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Converts PNG images to JPEG format to match training defaults, deleting the original PNG files. ```shell find kitti_data/ -name '*.png' | parallel 'convert -quality 92 -sampling-factor 2x2,1x1,1x1 {.}.png {.}.jpg && rm {}' ``` -------------------------------- ### Predict depth for a single image Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Use these commands to run inference on a single image. The first run will automatically download the specified model to the models/ directory. ```shell python test_simple.py --image_path assets/test_image.jpg --model_name mono+stereo_640x192 ``` ```shell python test_simple.py --image_path assets/test_image.jpg --model_name mono+stereo_640x192 --pred_metric_depth ``` -------------------------------- ### Evaluate Mono Depth Model Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Evaluate a monocular depth model using its trained weights. Specify the folder containing the weights and the evaluation split. ```shell python evaluate_depth.py --load_weights_folder ~/tmp/mono_model/models/weights_19/ --eval_mono ``` -------------------------------- ### Unzip KITTI dataset Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Extracts all downloaded zip files within the kitti_data directory. ```shell cd kitti_data unzip "*.zip" cd .. ``` -------------------------------- ### CLI: Pose Evaluation Source: https://context7.com/nianticlabs/monodepth2/llms.txt Evaluate camera pose estimation accuracy on KITTI odometry sequences. ```APIDOC ## CLI evaluate_pose.py ### Description Computes Absolute Trajectory Error (ATE) on KITTI odometry sequences by comparing predicted transformations against ground truth. ### Parameters #### Query Parameters - **--eval_split** (string) - Required - Split to evaluate (e.g., odom_9, odom_10) - **--load_weights_folder** (string) - Required - Path to model weights - **--data_path** (string) - Required - Path to KITTI odometry data ``` -------------------------------- ### Evaluate External Disparities Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Evaluate raw disparities from external methods using the `--ext_disp_to_eval` flag. Provide the path to the .npy file containing the disparities. ```shell python evaluate_depth.py --ext_disp_to_eval ~/other_method_disp.npy ``` -------------------------------- ### Define Available Monodepth2 Models Source: https://context7.com/nianticlabs/monodepth2/llms.txt List of available model identifiers for monocular and stereo depth estimation, including resolution and training source. ```python models = [ "mono_640x192", # Monocular, 640x192, ImageNet pretrained "stereo_640x192", # Stereo, 640x192, ImageNet pretrained "mono+stereo_640x192", # Mono+Stereo, 640x192, ImageNet pretrained "mono_1024x320", # Monocular, 1024x320, ImageNet pretrained "stereo_1024x320", # Stereo, 1024x320, ImageNet pretrained "mono+stereo_1024x320", # Mono+Stereo, 1024x320, ImageNet pretrained "mono_no_pt_640x192", # Monocular, 640x192, trained from scratch "stereo_no_pt_640x192", # Stereo, 640x192, trained from scratch "mono+stereo_no_pt_640x192", # Mono+Stereo, 640x192, trained from scratch ] ``` -------------------------------- ### Compute Depth Evaluation Metrics Source: https://context7.com/nianticlabs/monodepth2/llms.txt Calculates standard KITTI depth metrics including absolute relative error and accuracy thresholds. ```python import numpy as np from evaluate_depth import compute_errors # Ground truth and predicted depths (NumPy arrays) gt_depth = np.random.rand(375, 1242) * 80 # GT depth in meters pred_depth = np.random.rand(375, 1242) * 80 # Predicted depth # Mask valid depth values MIN_DEPTH, MAX_DEPTH = 1e-3, 80 mask = np.logical_and(gt_depth > MIN_DEPTH, gt_depth < MAX_DEPTH) # Apply Eigen crop crop = np.array([ 0.40810811 * 375, 0.99189189 * 375, 0.03594771 * 1242, 0.96405229 * 1242 ]).astype(np.int32) crop_mask = np.zeros(mask.shape) crop_mask[crop[0]:crop[1], crop[2]:crop[3]] = 1 mask = np.logical_and(mask, crop_mask) gt_masked = gt_depth[mask] pred_masked = pred_depth[mask] # Median scaling for monocular evaluation ratio = np.median(gt_masked) / np.median(pred_masked) pred_scaled = pred_masked * ratio # Compute metrics abs_rel, sq_rel, rmse, rmse_log, a1, a2, a3 = compute_errors(gt_masked, pred_scaled) print(f"abs_rel: {abs_rel:.3f}") # Absolute relative error print(f"sq_rel: {sq_rel:.3f}") # Squared relative error print(f"rmse: {rmse:.3f}") # Root mean squared error print(f"rmse_log: {rmse_log:.3f}")# RMSE of log depths print(f"a1: {a1:.3f}") # % of pixels with delta < 1.25 print(f"a2: {a2:.3f}") # % of pixels with delta < 1.25^2 print(f"a3: {a3:.3f}") # % of pixels with delta < 1.25^3 ``` -------------------------------- ### Compute Reprojection and Smoothness Losses Source: https://context7.com/nianticlabs/monodepth2/llms.txt Calculates photometric reprojection loss using SSIM and L1, alongside edge-aware disparity smoothness. ```python from layers import SSIM, get_smooth_loss import torch import torch.nn.functional as F # Initialize SSIM module ssim = SSIM() # Predicted and target images pred = torch.randn(4, 3, 192, 640) target = torch.randn(4, 3, 192, 640) # Compute reprojection loss (L1 + SSIM) abs_diff = torch.abs(target - pred) l1_loss = abs_diff.mean(1, True) ssim_loss = ssim(pred, target).mean(1, True) reprojection_loss = 0.85 * ssim_loss + 0.15 * l1_loss # Edge-aware smoothness loss disp = torch.sigmoid(torch.randn(4, 1, 192, 640)) color = torch.randn(4, 3, 192, 640) # Normalize disparity mean_disp = disp.mean(2, True).mean(3, True) norm_disp = disp / (mean_disp + 1e-7) smooth_loss = get_smooth_loss(norm_disp, color) # Total loss with smoothness weight disparity_smoothness = 1e-3 total_loss = reprojection_loss.mean() + disparity_smoothness * smooth_loss ``` -------------------------------- ### Perform single image depth prediction Source: https://context7.com/nianticlabs/monodepth2/llms.txt Use test_simple.py to run inference on images or folders, or integrate the model programmatically using PyTorch. ```bash # Predict disparity for a single image python test_simple.py --image_path assets/test_image.jpg \ --model_name mono+stereo_640x192 # Predict metric depth (only for stereo-trained models) python test_simple.py --image_path assets/test_image.jpg \ --model_name mono+stereo_640x192 \ --pred_metric_depth # Process entire folder of images python test_simple.py --image_path /path/to/images/ \ --model_name mono_1024x320 \ --ext png # Run on CPU without CUDA python test_simple.py --image_path assets/test_image.jpg \ --model_name mono_640x192 \ --no_cuda ``` ```python import torch import networks from layers import disp_to_depth from utils import download_model_if_doesnt_exist from PIL import Image from torchvision import transforms # Load pretrained model model_name = "mono+stereo_640x192" download_model_if_doesnt_exist(model_name) encoder = networks.ResnetEncoder(18, False) depth_decoder = networks.DepthDecoder(num_ch_enc=encoder.num_ch_enc, scales=range(4)) encoder_dict = torch.load(f"models/{model_name}/encoder.pth") encoder.load_state_dict({k: v for k, v in encoder_dict.items() if k in encoder.state_dict()}) depth_decoder.load_state_dict(torch.load(f"models/{model_name}/depth.pth")) encoder.eval() depth_decoder.eval() # Process image input_image = Image.open("test.jpg").convert('RGB') input_image = input_image.resize((encoder_dict['width'], encoder_dict['height'])) input_tensor = transforms.ToTensor()(input_image).unsqueeze(0) with torch.no_grad(): features = encoder(input_tensor) outputs = depth_decoder(features) disp = outputs[("disp", 0)] scaled_disp, depth = disp_to_depth(disp, 0.1, 100) # Output: disp has shape [1, 1, H, W], depth in range [0.1, 100] meters ``` -------------------------------- ### CLI: Depth Evaluation Source: https://context7.com/nianticlabs/monodepth2/llms.txt Evaluate monocular or stereo depth models using the evaluate_depth.py script. ```APIDOC ## CLI evaluate_depth.py ### Description Evaluates depth estimation models on KITTI datasets. Supports monocular and stereo evaluation, post-processing, and external disparity evaluation. ### Parameters #### Query Parameters - **--load_weights_folder** (string) - Required - Path to model weights folder - **--eval_mono** (flag) - Optional - Evaluate monocular model - **--eval_stereo** (flag) - Optional - Evaluate stereo model - **--eval_split** (string) - Optional - Dataset split to evaluate (e.g., eigen, eigen_benchmark, benchmark) - **--post_process** (flag) - Optional - Enable flip augmentation post-processing - **--ext_disp_to_eval** (string) - Optional - Path to external disparity file (.npy) - **--no_eval** (flag) - Optional - Save predictions without running evaluation ``` -------------------------------- ### Evaluate Stereo Depth Model Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Evaluate a stereo depth model. Use the `--eval_stereo` flag, which automatically disables median scaling and applies a baseline scaling factor of 5.4. ```shell python evaluate_depth.py --load_weights_folder ~/tmp/stereo_model/models/weights_19/ --eval_stereo ``` -------------------------------- ### Visualize Disparity Output Source: https://github.com/nianticlabs/monodepth2/blob/master/depth_prediction_example.ipynb Interpolates the disparity map to the original image size and displays the result using matplotlib. ```python disp_resized = torch.nn.functional.interpolate(disp, (original_height, original_width), mode="bilinear", align_corners=False) # Saving colormapped depth image disp_resized_np = disp_resized.squeeze().cpu().numpy() vmax = np.percentile(disp_resized_np, 95) plt.figure(figsize=(10, 10)) plt.subplot(211) plt.imshow(input_image) plt.title("Input", fontsize=22) plt.axis('off') plt.subplot(212) plt.imshow(disp_resized_np, cmap='magma', vmax=vmax) plt.title("Disparity prediction", fontsize=22) plt.axis('off'); ``` -------------------------------- ### Convert Disparity Output to Metric Depth Source: https://context7.com/nianticlabs/monodepth2/llms.txt Converts sigmoid-activated disparity output (range 0-1) to metric depth values within specified minimum and maximum bounds. For stereo-trained models, an additional scale factor is applied. ```python from layers import disp_to_depth import torch # Disparity output from network (sigmoid activated, range 0-1) disp = torch.sigmoid(torch.randn(1, 1, 192, 640)) # Convert to depth with bounds min_depth = 0.1 # 10cm minimum depth max_depth = 100.0 # 100m maximum depth scaled_disp, depth = disp_to_depth(disp, min_depth, max_depth) # scaled_disp: disparity scaled to [1/max_depth, 1/min_depth] # depth: metric depth in [min_depth, max_depth] # For stereo-trained models, multiply by STEREO_SCALE_FACTOR = 5.4 # to convert from training baseline (0.1 units) to KITTI baseline (0.54m) STEREO_SCALE_FACTOR = 5.4 metric_depth = STEREO_SCALE_FACTOR * depth ``` -------------------------------- ### Evaluate Camera Pose Estimation Source: https://context7.com/nianticlabs/monodepth2/llms.txt Computes Absolute Trajectory Error (ATE) for camera pose estimation on specific KITTI odometry sequences. ```bash python evaluate_pose.py \ --eval_split odom_9 \ --load_weights_folder ./odom_split.M/models/weights_29 \ --data_path kitti_odom/ ``` ```bash python evaluate_pose.py \ --eval_split odom_10 \ --load_weights_folder ./odom_split.M/models/weights_29 \ --data_path kitti_odom/ ``` -------------------------------- ### Generate 4x4 Transformation Matrices from Pose Parameters Source: https://context7.com/nianticlabs/monodepth2/llms.txt Converts axis-angle rotation and translation vectors, typically from a pose network, into 4x4 rigid transformation matrices. Supports both forward and inverse transformations. ```python from layers import transformation_from_parameters import torch # Pose network outputs axisangle = torch.randn(4, 1, 3) # Batch of 4 axis-angle rotations translation = torch.randn(4, 1, 3) # Batch of 4 translation vectors # Forward transformation (camera at t to camera at t+1) T_forward = transformation_from_parameters(axisangle, translation, invert=False) # T_forward: [4, 4, 4] transformation matrices # Inverse transformation (for warping from t+1 to t) T_inverse = transformation_from_parameters(axisangle, translation, invert=True) # Usage in view synthesis # T transforms points from coordinate system at time t to time t+1 # T_inverse transforms points from t+1 back to t ``` -------------------------------- ### Module: ResnetEncoder Source: https://context7.com/nianticlabs/monodepth2/llms.txt Feature encoder based on pretrained ResNet architectures. ```APIDOC ## Class networks.ResnetEncoder ### Description Extracts hierarchical features from input images using ResNet backbones. Returns features at 5 scales. ### Parameters - **num_layers** (int) - Required - ResNet depth (18, 34, 50, 101, 152) - **pretrained** (bool) - Optional - Use ImageNet weights - **num_input_images** (int) - Optional - Number of stacked input images (default 1) ``` -------------------------------- ### Run Model Prediction Source: https://github.com/nianticlabs/monodepth2/blob/master/depth_prediction_example.ipynb Performs inference using the encoder and decoder to obtain the disparity map. ```python with torch.no_grad(): features = encoder(input_image_pytorch) outputs = depth_decoder(features) disp = outputs[("disp", 0)] ``` -------------------------------- ### Module: DepthDecoder Source: https://context7.com/nianticlabs/monodepth2/llms.txt Decoder network that converts encoder features to disparity predictions. ```APIDOC ## Class networks.DepthDecoder ### Description U-Net style decoder producing multi-scale disparity maps from encoder features. ### Parameters - **num_ch_enc** (list) - Required - Channel counts from encoder - **scales** (range) - Required - Output scales to generate - **num_output_channels** (int) - Optional - Number of output channels - **use_skips** (bool) - Optional - Enable skip connections ``` -------------------------------- ### Monodepth2 Citation Source: https://github.com/nianticlabs/monodepth2/blob/master/README.md Use this BibTeX entry when citing the Monodepth2 paper in your research. ```bibtex @article{monodepth2, title = {Digging into Self-Supervised Monocular Depth Prediction}, author = {Cl{\'e}ment Godard and Oisin {Mac Aodha} and Michael Firman and Gabriel J. Brostow}, booktitle = {The International Conference on Computer Vision (ICCV)}, month = {October}, year = {2019} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.