### Install Dependencies using pip Source: https://github.com/rickgroen/cov-weighting/blob/main/README.md Installs all necessary project dependencies from a requirements file. Ensure you have a Python environment set up before running this command. ```shell pip install -r requirements.txt ``` -------------------------------- ### Download and Setup KITTI Dataset (Bash) Source: https://context7.com/rickgroen/cov-weighting/llms.txt Provides bash commands to download the KITTI raw dataset and outlines the expected directory structure. It also shows how to initiate training on a subset of the data for quick experiments using the `train.py` script. ```bash # Download KITTI raw data (approximately 175GB) wget -i utils/kitti_archives_to_download.txt -P /data/kitti_raw/ # Expected directory structure after extraction: # /data/kitti_raw/ # ├── 2011_09_26/ # ├── 2011_09_28/ # ├── 2011_09_29/ # ├── 2011_09_30/ # └── 2011_10_03/ # For CityScapes dataset (requires registration): # /data/cityscapes/ # ├── cs_camera/ # Camera parameters # ├── cs_disparity/ # Ground truth disparities # ├── cs_leftImg8bit/ # Left stereo images # └── cs_rightImg8bit/ # Right stereo images # Train on subset for testing (10% of data) python train.py \ --data_dir /data/kitti_raw \ --model_name quick_test \ --method cov-weighting \ --train_ratio 0.1 \ --epochs 5 ``` -------------------------------- ### Download KITTI Dataset Archives Source: https://github.com/rickgroen/cov-weighting/blob/main/README.md Downloads KITTI dataset archives using a provided list of URLs. The downloaded files will be saved to the specified output folder. ```shell wget -i utils/kitti_archives_to_download.txt -P ~/my/output/folder/ ``` -------------------------------- ### Training Configuration Options Source: https://context7.com/rickgroen/cov-weighting/llms.txt Provides a comprehensive set of configuration options for training experiments using the `TrainOptions` class. It allows setting essential parameters like data directory, model name, method, dataset, backbone, and device, as well as training hyperparameters such as epochs, learning rate, batch size, optimizer, and learning rate adjustment modes. Specific weights for loss components and parameters for the CoV-Weighting method can also be configured. ```python from options import TrainOptions parser = TrainOptions() args = parser.parse() # Essential parameters args.data_dir = '/data/kitti' args.model_name = 'experiment_cov' args.method = 'cov-weighting' # or 'gradnorm', 'uncertainty', 'multi-objective', 'static' args.dataset = 'kitti' # or 'cityscapes' args.backbone = 'resnet18' args.device = 'cuda:0' # Training hyperparameters args.epochs = 30 args.learning_rate = 1e-4 args.batch_size = 8 args.optimizer = 'adam' # or 'sgd', 'rmsprop' args.adjust_lr = True args.lr_mode = 'plateau' # or 'polynomial', 'step' # Loss component weights (for static method) args.img_loss_l1_w = 0.25 args.img_loss_ssim_w = 0.25 args.lr_loss_w = 0.25 args.disp_grad_loss_w = 0.25 # CoV-Weighting specific args.mean_sort = 'full' # 'full' or 'decay' args.mean_decay_param = 0.95 ``` -------------------------------- ### Test Depth Estimation Model Performance (Bash) Source: https://context7.com/rickgroen/cov-weighting/llms.txt Command-line interface for evaluating trained depth estimation models. Allows testing of the best model checkpoint or the final epoch model, with options for post-processing. Requires specifying data directory, model name, and dataset. ```bash python test.py \ --data_dir /path/to/kitti/data \ --model_name my_cov_model \ --dataset kitti \ --postprocessing True python test.py \ --data_dir /path/to/kitti/data \ --model_name my_cov_model \ --load_final True \ --postprocessing False ``` -------------------------------- ### Train Model with CoV-Weighting Source: https://github.com/rickgroen/cov-weighting/blob/main/README.md Initiates model training using the CoV-Weighting method. Requires specifying the data directory, a model name, and the method. Various other training options are available. ```shell python train.py --data_dir data/ --model_name [MODEL_NAME] --method cov-weighting ``` -------------------------------- ### Test Model on KITTI Eigen Split Source: https://github.com/rickgroen/cov-weighting/blob/main/README.md Executes testing on the pre-defined Eigen split of the KITTI dataset. Requires the data directory and model name as input. ```shell python test.py --data_dir data/ --model_name [MODEL_NAME] ``` -------------------------------- ### Train Depth Estimation Model with CoV-Weighting (Bash) Source: https://context7.com/rickgroen/cov-weighting/llms.txt Command-line interface for training a depth estimation model using the CoV-Weighting method. Requires specifying data directory, model name, method, dataset, backbone, epochs, batch size, and learning rate. Advanced options include mean statistics and decay parameters. ```bash python train.py \ --data_dir /path/to/kitti/data \ --model_name my_cov_model \ --method cov-weighting \ --dataset kitti \ --backbone resnet18 \ --epochs 30 \ --batch_size 8 \ --learning_rate 1e-4 python train.py \ --data_dir /path/to/data \ --model_name cov_full_mean \ --method cov-weighting \ --dataset cityscapes \ --backbone resnet50 \ --mean_sort full \ --mean_decay_param 0.95 \ --train_ratio 0.8 \ --do_augmentation True ``` -------------------------------- ### DataLoader: Create Dataset Loaders for KITTI and CityScapes Source: https://context7.com/rickgroen/cov-weighting/llms.txt A factory function to create dataset loaders for KITTI and CityScapes datasets. It allows customization of data directory, dataset type, input dimensions, batch size, data augmentation, and training ratio. Requires 'data_loaders' and 'options' libraries. The loader provides data in a dictionary format including images, ground truth depth, and camera parameters. ```python from data_loaders import create_dataloader from options import TrainOptions args = TrainOptions().parse() args.data_dir = '/data/kitti_raw' args.dataset = 'kitti' args.input_height = 256 args.input_width = 512 args.batch_size = 8 args.num_workers = 4 args.train_ratio = 1.0 # Use 100% of training data args.augment_parameters = [0.8, 1.2, 0.8, 1.2, 0.8, 1.2] # Gamma, brightness, color ranges # Create loaders for different splits train_loader = create_dataloader(args, 'train') # Shuffled, batched val_loader = create_dataloader(args, 'val') # Sequential, batch_size=1 test_loader = create_dataloader(args, 'test') # Sequential, batch_size=1 # Iterate through training data for batch_idx, data in enumerate(train_loader): left_image = data['left_image'] # Shape: [8, 3, 256, 512] right_image = data['right_image'] # Shape: [8, 3, 256, 512] gt_depth = data['gt_depth'] # Shape: [8, 1, 256, 512] (if available) camera = data['camera'] # [focal_length, baseline] ``` -------------------------------- ### GradNormMethod: Implement Gradient Normalization for Task Balancing Source: https://context7.com/rickgroen/cov-weighting/llms.txt This method implements gradient normalization for balancing tasks during training. It computes a weighted total loss, extracts gradients, balances them based on loss ratios and a gamma parameter, and updates loss weights via gradient descent. Requires 'methods' library and a dataloader. The backward pass handles the gradient balancing and weight updates. ```python from methods import GradNormMethod args.method = 'gradnorm' args.init_gamma = 1.5 # Asymmetry parameter for gradient magnitude targets train_loader = create_dataloader(args, 'train') model = GradNormMethod(args, train_loader) # Training with GradNorm for epoch in range(args.epochs): model.to_train() for data in train_loader: model.set_input(data) model.forward() # GradNorm backward pass: # 1. Computes weighted loss L_total = sum(w[i] * L[i]) # 2. Extracts gradients G[i] from shared network layer # 3. Balances gradients: G[i] should equal G_avg * (L[i]/L0[i])^gamma # 4. Updates loss weights w via gradient descent on ||G - target|| loss = model.backward() model.optimizer.step() model.optimizer_params.step() # Separate optimizer for weights # Weights automatically renormalized to sum to 1 print("Current task weights:", model.params.data) ``` -------------------------------- ### CoVWeightingMethod Class for Training (Python) Source: https://context7.com/rickgroen/cov-weighting/llms.txt Core Python class for implementing the CoV-Weighting training method. Initializes training configuration, data loaders, and the CoVWeightingMethod. The training loop iteratively optimizes parameters, automatically updating alpha weights based on CoV statistics. ```python from methods import CoVWeightingMethod from options import TrainOptions from data_loaders import create_dataloader # Initialize training configuration parser = TrainOptions() args = parser.parse() args.method = 'cov-weighting' args.data_dir = '/path/to/data' args.model_name = 'experiment_1' args.dataset = 'kitti' # Create data loader and method train_loader = create_dataloader(args, 'train') model = CoVWeightingMethod(args, train_loader) # Training loop for epoch in range(args.epochs): model.to_train() for data in train_loader: model.set_input(data) # Sets left and right stereo images loss = model.optimize_parameters() # Forward, backward, update weights # Alpha weights automatically updated based on CoV statistics # Access learned alpha weights after epoch print(f"Epoch {epoch} mean weights:", model.mean_weights) model.save_network('checkpoint') ``` -------------------------------- ### Compute Depth Estimation Metrics on KITTI Test Set (Python) Source: https://context7.com/rickgroen/cov-weighting/llms.txt Calculates seven standard error metrics for depth estimation using predicted disparity, ground truth depth, focal length, and baseline. Requires the Tester class and TestOptions from the project's utilities. Outputs include Absolute Relative Error and various accuracy measures. ```python from test import Tester from options import TestOptions import numpy as np parser = TestOptions() args = parser.parse() args.model_name = 'my_cov_model' args.data_dir = '/data/kitti' args.dataset = 'kitti' args.min_depth = 1e-3 args.max_depth = 80 args.postprocessing = True tester = Tester(args) # Manual evaluation for single image predicted_disparity = np.random.rand(375, 1242) # Example disparity map gt_depth = np.random.rand(375, 1242) * 50 + 1 # Ground truth depth focal_length = 721.5377 baseline = 0.54 # Compute 7 error metrics abs_rel, sq_rel, rmse, log_rmse, a1, a2, a3 = tester.evaluate_kitti( predicted_disparity, gt_depth, focal_length, baseline ) # Metrics interpretation: # abs_rel: Mean absolute relative error (lower is better) # sq_rel: Mean squared relative error (lower is better) # rmse: Root mean squared error (lower is better) # log_rmse: Log RMSE (lower is better) # a1, a2, a3: Accuracy under thresholds 1.25, 1.25^2, 1.25^3 (higher is better) print(f"Absolute Relative Error: {abs_rel:.4f}") print(f"δ < 1.25 Accuracy: {a1:.4f}") ``` -------------------------------- ### BaseLoss: Compute Stereo Depth Estimation Losses Source: https://context7.com/rickgroen/cov-weighting/llms.txt The BaseLoss class computes 32 individual stereo depth estimation loss components. It takes predicted disparities and a stereo image pair as input and outputs a list of loss tensors, including L1 loss, SSIM loss, left-right consistency loss, and disparity smoothness loss at multiple scales. Requires 'losses' and 'torch' libraries. ```python from losses import BaseLoss import torch args = Args() base_loss = BaseLoss(args) # Input: predicted disparities and stereo pair left_img = torch.randn(8, 3, 256, 512).cuda() # Batch of 8 images right_img = torch.randn(8, 3, 256, 512).cuda() pred_disps = [torch.randn(8, 2, 256//2**i, 512//2**i).cuda() for i in range(4)] # Compute all 32 unweighted losses losses = base_loss.forward(pred_disps, [left_img, right_img]) # Returns list of 32 tensors: # losses[0-3]: L1 loss at scales 0-3 (left image) # losses[4-7]: L1 loss at scales 0-3 (right image) # losses[8-15]: SSIM loss at scales 0-3 (left + right) # losses[16-23]: Left-right consistency loss (left + right) # losses[24-31]: Disparity smoothness loss (left + right) print(f"L1 loss scale 0 (left): {losses[0].item():.4f}") print(f"SSIM loss scale 1 (left): {losses[9].item():.4f}") ``` -------------------------------- ### CoVWeightingLoss Class for Loss Computation (Python) Source: https://context7.com/rickgroen/cov-weighting/llms.txt Python class implementing the CoV-Weighting loss function. Utilizes Welford's algorithm for online coefficient of variation computation. It calculates 32 unweighted loss components, updates running statistics, computes dynamic alpha weights, and returns the weighted loss. ```python import torch from losses import CoVWeightingLoss # Initialize loss with configuration class Args: device = 'cuda:0' mean_sort = 'full' # or 'decay' mean_decay_param = 0.95 args = Args() criterion = CoVWeightingLoss(args).to(args.device) criterion.to_train() # During training - computes weighted loss from 32 components disps = model(left_image) # Predicted disparities at 4 scales loss = criterion.forward(disps, [left_image, right_image]) # Access dynamic alpha weights print("Current alpha weights:", criterion.alphas) print("Running mean losses:", criterion.running_mean_L) print("Running std of loss ratios:", criterion.running_std_l) # Loss computation flow: # 1. Compute 32 unweighted losses L (L1, SSIM, LR, DISP at 4 scales) # 2. Calculate loss ratios l = L / L0 (L0 = initial loss) # 3. Update running statistics (mean, variance) via Welford's algorithm # 4. Compute alphas = (std(l) / mean(l)) / sum(std(l) / mean(l)) # 5. Return weighted_loss = sum(alpha[i] * L[i]) ``` -------------------------------- ### Network Definition: Define Generator Networks (ResNet-based) Source: https://context7.com/rickgroen/cov-weighting/llms.txt Defines generator networks using ResNet-18 or ResNet-50 based encoders. The `define_G` function takes network configuration arguments, including the backbone and normalization layer type, and returns a CUDA-enabled model. The forward pass produces multi-scale disparity maps for both left and right images. ```python from networks import define_G import torch class Args: backbone = 'resnet18' # or 'resnet50' norm_layer = 'batch' # 'batch', 'instance', or '' method = 'cov-weighting' args = Args() model = define_G(args).cuda() # Forward pass produces multi-scale disparity maps input_img = torch.randn(1, 3, 256, 512).cuda() output_disps = model(input_img) # Output: list of 4 tensors at different scales # Each tensor has shape [batch, 2, height, width] # Channel 0: left disparity, Channel 1: right disparity for i, disp in enumerate(output_disps): print(f"Scale {i}: {disp.shape}") # Scale 0: torch.Size([1, 2, 256, 512]) # Scale 1: torch.Size([1, 2, 128, 256]) # Scale 2: torch.Size([1, 2, 64, 128]) # Scale 3: torch.Size([1, 2, 32, 64]) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.