### Install Docker-Compose and Start Docker Service Source: https://github.com/hirotomusiker/clrernet/blob/main/docs/INSTALL.md Installs docker-compose using apt and starts the docker service. Use this if you do not have a docker-compose environment set up. ```bash sudo apt install docker-compose sudo service docker start ``` -------------------------------- ### Build and Run Docker Environment Source: https://context7.com/hirotomusiker/clrernet/llms.txt Sets up a Docker environment for CLRerNet development, including PyTorch, mmdetection, and CUDA. Build the image using `docker compose build` and run the container with `docker compose run`. Verify installations inside the container. ```bash # Build the Docker image with NVIDIA runtime docker compose build --build-arg UID="`id -u`" dev # Run the development container docker compose run --rm dev # Inside container, verify installation python -c "from mmdet.apis import init_detector; print('mmdet OK')" python -c "from nms import nms; print('NMS extension OK')" # If NMS build fails during Docker build, build manually inside container: cd libs/models/layers/nms/ python setup.py install cd /work ``` -------------------------------- ### Install NMS Inside Container Source: https://github.com/hirotomusiker/clrernet/blob/main/docs/INSTALL.md Installs the NMS extension manually within the container if the initial build fails. This involves navigating to the NMS directory and running the setup script. ```bash cd libs/models/layers/nms/ python setup.py install cd /work ``` -------------------------------- ### Run Development Environment Source: https://github.com/hirotomusiker/clrernet/blob/main/docs/INSTALL.md Starts the clrernet development environment using docker-compose. The --rm flag ensures the container is removed upon exit. ```bash docker-compose run --rm dev ``` -------------------------------- ### Configure CLRerNet Model Source: https://context7.com/hirotomusiker/clrernet/llms.txt Defines the model configuration using a hierarchical system. This example shows base configurations for architecture, dataset, and runtime, along with custom imports and EMA training hooks for performance improvement. ```python # configs/clrernet/culane/clrernet_culane_dla34_ema.py _base_ = [ "../base_clrernet.py", # Model architecture "dataset_culane_clrernet.py", # Dataset and pipeline "../../_base_/default_runtime.py", # Training runtime ] # Custom imports for CLRerNet components custom_imports = dict( imports=[ "libs.models", # CLRerNet, CLRerHead, CLRerNetFPN, DLANet "libs.datasets", # CulaneDataset "libs.core.bbox", # DynamicTopkAssigner, LaneIoUCost "libs.core.anchor", # CLRerNetAnchorGenerator "libs.core.hook", # Custom logging hooks ], ) # EMA training configuration (improves performance) custom_hooks = [ dict( type='EMAHook', ema_type='ExpMomentumEMA', momentum=0.0001, update_buffers=True, priority=49 ) ] ``` -------------------------------- ### Download CULane Dataset Files Source: https://github.com/hirotomusiker/clrernet/blob/main/docs/DATASETS.md Use gdown to download the necessary CULane dataset files from Google Drive. Ensure you have gdown installed. ```bash gdown "https://drive.google.com/uc?id=1LTdUXzUWcnHuEEAiMoG42oAGuJggPQs8" gdown "https://drive.google.com/uc?id=1daWl7XVzH06GwcZtF4WD8Xpvci5SZiUV" gdown "https://drive.google.com/uc?id=1Z6a463FQ3pfP54HMwF3QS5h9p2Ch3An7" gdown "https://drive.google.com/uc?id=1QbB1TOk9Fy6Sk0CoOsR3V8R56_eG6Xnu" gdown "https://drive.google.com/uc?id=18alVEPAMBA9Hpr3RDAAchqSj5IxZNRKd" ``` -------------------------------- ### Visualize Lane Detections Source: https://context7.com/hirotomusiker/clrernet/llms.txt Overlays detected lanes on an image, optionally showing ground truth and IoU scores. Use `visualize_lanes` for multiple lanes and `draw_lane` for a single lane on a blank image. Ensure OpenCV and NumPy are installed. ```python import cv2 import numpy as np from libs.utils.visualizer import visualize_lanes, draw_lane # Load source image src = cv2.imread('demo/demo.jpg') # Lane predictions as list of (N, 2) arrays with (x, y) coordinates predictions = [ np.array([[100, 400], [150, 350], [200, 300], [250, 250]]), np.array([[500, 400], [480, 350], [460, 300], [440, 250]]), ] # Ground truth annotations (optional) annotations = [ np.array([[105, 400], [155, 350], [205, 300], [255, 250]]), ] # IoU scores for each prediction (optional) pred_ious = [0.85, 0.45] # Visualize with IoU-based coloring result = visualize_lanes( src=src, preds=predictions, annos=annotations, pred_ious=pred_ious, iou_thr=0.5, # Threshold for hit/miss classification concat_src=True, # Stack original and annotated vertically save_path='vis_result.png' ) # Draw a single lane on blank image lane = np.array([[100, 400], [150, 350], [200, 300]]) lane_img = draw_lane( lane=lane, img=None, img_shape=(590, 1640, 3), width=30, color=(0, 255, 0) # BGR green ) ``` -------------------------------- ### Build and Run Docker Environment Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md Commands to build and execute the development container environment. ```bash docker compose build --build-arg UID="`id -u`" dev docker compose run --rm dev ``` -------------------------------- ### Run Image Demo via CLI Source: https://context7.com/hirotomusiker/clrernet/llms.txt Executes lane detection on images using the command-line interface with support for custom thresholds and devices. ```bash # Basic lane detection demo python demo/image_demo.py demo/demo.jpg \ configs/clrernet/culane/clrernet_culane_dla34_ema.py \ clrernet_culane_dla34_ema.pth \ --out-file=result.png # With custom confidence threshold and device python demo/image_demo.py input_image.jpg \ configs/clrernet/culane/clrernet_culane_dla34_ema.py \ clrernet_culane_dla34_ema.pth \ --out-file=output.png \ --device=cuda:0 \ --score-thr=0.4 ``` -------------------------------- ### Initialize and Run Inference in Python Source: https://context7.com/hirotomusiker/clrernet/llms.txt Loads a CLRerNet model using mmdet utilities and performs lane detection on a single image. ```python from mmdet.apis import init_detector from libs.api.inference import inference_one_image from libs.utils.visualizer import visualize_lanes import cv2 # Initialize the model with config and checkpoint model = init_detector( config='configs/clrernet/culane/clrernet_culane_dla34_ema.py', checkpoint='clrernet_culane_dla34_ema.pth', device='cuda:0' ) # Run inference on an image img_path = 'demo/demo.jpg' src_img, lane_predictions = inference_one_image(model, img_path) # lane_predictions is a list of numpy arrays, each containing (x, y) coordinates # Example output: [array([[x1, y1], [x2, y2], ...]), array([[x1, y1], ...]), ...] # Visualize and save the results result_img = visualize_lanes(src_img, lane_predictions, save_path='result.png') ``` -------------------------------- ### Download Pretrained Weights Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md Commands to download pretrained model weights for CLRerNet and the EMA variant. ```bash wget https://github.com/hirotomusiker/CLRerNet/releases/download/v0.1.0/clrernet_culane_dla34.pth ``` ```bash wget https://github.com/hirotomusiker/CLRerNet/releases/download/v0.1.0/clrernet_culane_dla34_ema.pth ``` -------------------------------- ### Train CLRerNet on CULane Source: https://context7.com/hirotomusiker/clrernet/llms.txt Commands for preparing the dataset and executing training, including distributed training and configuration overrides. ```bash # Prepare frame difference file first (required for training) python tools/calculate_frame_diff.py $HOME/dataset/culane # This creates: dataset/culane/list/train_diffs.npz # Basic training with default settings python tools/train.py configs/clrernet/culane/clrernet_culane_dla34.py # Training with custom work directory and AMP python tools/train.py configs/clrernet/culane/clrernet_culane_dla34.py \ --work-dir=./work_dirs/my_experiment \ --amp # Resume training from checkpoint python tools/train.py configs/clrernet/culane/clrernet_culane_dla34.py \ --resume=work_dirs/clrernet_culane_dla34/epoch_30.pth # Distributed training with PyTorch launcher python -m torch.distributed.launch --nproc_per_node=4 \ tools/train.py configs/clrernet/culane/clrernet_culane_dla34.py \ --launcher=pytorch # Override config options python tools/train.py configs/clrernet/culane/clrernet_culane_dla34.py \ --cfg-options train_dataloader.batch_size=16 total_epochs=100 ``` -------------------------------- ### Benchmark Inference Speed Source: https://context7.com/hirotomusiker/clrernet/llms.txt Run the speed test script to measure FPS with configurable warmup and test iterations. ```bash # Run speed test with default iterations python tools/speed_test.py \ configs/clrernet/culane/clrernet_culane_dla34.py \ clrernet_culane_dla34.pth \ --filename=demo/demo.jpg # Custom warmup and test iterations for accurate benchmarking python tools/speed_test.py \ configs/clrernet/culane/clrernet_culane_dla34.py \ clrernet_culane_dla34.pth \ --filename=demo/demo.jpg \ --n_iter_warmup=1000 \ --n_iter_test=10000 ``` -------------------------------- ### Restart Docker and Build Development Environment Source: https://github.com/hirotomusiker/clrernet/blob/main/docs/INSTALL.md Restarts the Docker service to apply changes and then builds the development environment using docker-compose. The UID argument ensures the container runs with the host user's ID. ```bash sudo systemctl restart docker docker-compose build --build-arg UID="`id -u`" dev ``` -------------------------------- ### Train Model on CULane Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md Command to initiate model training, requiring the frame difference file to be present. ```bash python tools/train.py configs/clrernet/culane/clrernet_culane_dla34.py ``` -------------------------------- ### Load and Preprocess CULane Dataset Source: https://context7.com/hirotomusiker/clrernet/llms.txt Initializes the CulaneDataset for training or testing. For training, it includes frame difference filtering to remove redundant frames. Ensure the data paths are correctly set. ```python from libs.datasets.culane_dataset import CulaneDataset # Dataset configuration for training train_dataset = CulaneDataset( data_root='$HOME/dataset/culane', data_list='$HOME/dataset/culane/list/train_gt.txt', pipeline=[ dict(type='Resize', size=(800, 320)), dict(type='Normalize', mean=[0, 0, 0], std=[255, 255, 255]), dict(type='LaneFormatting'), ], diff_file='$HOME/dataset/culane/list/train_diffs.npz', # Frame difference filter diff_thr=15, # Threshold for filtering similar frames test_mode=False ) # Dataset configuration for testing test_dataset = CulaneDataset( data_root='$HOME/dataset/culane', data_list='$HOME/dataset/culane/list/test.txt', pipeline=[ dict(type='Resize', size=(800, 320)), dict(type='Normalize', mean=[0, 0, 0], std=[255, 255, 255]), dict(type='LaneFormatting'), ], test_mode=True ) # Access a training sample sample = train_dataset[0] # Returns dict with: 'inputs' (tensor), 'data_samples' (DetDataSample with lanes) ``` -------------------------------- ### Configure CLRerNet Training Hyperparameters Source: https://context7.com/hirotomusiker/clrernet/llms.txt Defines the training loop, batch size, optimizer, and randomness settings for the model. ```python total_epochs = 50 train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=total_epochs, val_interval=10) train_dataloader = dict(batch_size=24) # Single GPU setting optim_wrapper = dict( type='OptimWrapper', optimizer=dict(type="AdamW", lr=6e-4), ) randomness = dict(seed=0, deterministic=True) ``` -------------------------------- ### Perform Speed Test Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md Command to measure inference speed in frames per second. ```bash python tools/speed_test.py configs/clrernet/culane/clrernet_culane_dla34.py clrernet_culane_dla34.pth --filename demo/demo.jpg --n_iter_warmup 1000 --n_iter_test 10000 ``` -------------------------------- ### Configure Docker for NVIDIA Runtime Source: https://github.com/hirotomusiker/clrernet/blob/main/docs/INSTALL.md Configures the Docker daemon to use the NVIDIA container runtime by default. This is required for enabling the NMS extension during build. ```json { "runtimes": { "nvidia": { "path": "nvidia-container-runtime", "runtimeArgs": [] } }, "default-runtime": "nvidia" } ``` -------------------------------- ### Generate Lane Anchors Source: https://context7.com/hirotomusiker/clrernet/llms.txt Initialize the anchor generator and compute anchor coordinates based on learned prior embeddings. ```python import torch from libs.core.anchor.anchor_generator import CLRerNetAnchorGenerator # Initialize anchor generator anchor_gen = CLRerNetAnchorGenerator( num_priors=192, # Number of anchor priors num_points=72 # Number of row points per anchor ) # Access learned anchor parameters (y0, x0, theta) # Shape: (192, 3) - start_y, start_x, angle anchor_params = anchor_gen.prior_embeddings.weight print(f"Anchor params shape: {anchor_params.shape}") # Generate anchor x-coordinates for given y-coordinates prior_ys = torch.linspace(1, 0, steps=72) # Row positions from bottom to top sample_indices = (torch.linspace(0, 1, steps=36) * 71).long() # Sampling indices anchor_xs, sampled_xs = anchor_gen.generate_anchors( anchor_params=anchor_params, prior_ys=prior_ys, sample_x_indices=sample_indices, img_w=800, img_h=320 ) ``` -------------------------------- ### Run Inference on Image Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md Command to perform lane detection on a single image and save the visualization. ```bash python demo/image_demo.py demo/demo.jpg configs/clrernet/culane/clrernet_culane_dla34_ema.py clrernet_culane_dla34_ema.pth --out-file=result.png ``` -------------------------------- ### Calculate Frame Differences Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md Command to generate an npz file containing frame difference values for training. ```bash python tools/calculate_frame_diff.py [culane_root_path] ``` -------------------------------- ### Define CLRerNet Architecture Source: https://context7.com/hirotomusiker/clrernet/llms.txt Configure the CLRerNet model using the mmdetection registry, specifying backbone, neck, and head components. ```python from mmdet.registry import MODELS # Model configuration dictionary model_cfg = dict( type="CLRerNet", data_preprocessor=dict( type='DetDataPreprocessor', mean=[0, 0, 0], std=[255.0, 255.0, 255.0], bgr_to_rgb=False, ), backbone=dict( type="DLANet", dla="dla34", pretrained=True, ), neck=dict( type="CLRerNetFPN", in_channels=[128, 256, 512], out_channels=64, num_outs=3, ), bbox_head=dict( type="CLRerHead", anchor_generator=dict( type="CLRerNetAnchorGenerator", num_priors=192, num_points=72, ), img_w=800, img_h=320, prior_feat_channels=64, fc_hidden_dim=64, num_fc=2, refine_layers=3, sample_points=36, attention=dict(type="ROIGather"), loss_cls=dict(type="KorniaFocalLoss", alpha=0.25, gamma=2, loss_weight=2.0), loss_bbox=dict(type="SmoothL1Loss", reduction="none", loss_weight=0.2), loss_iou=dict(type="LaneIoULoss", lane_width=7.5/800, loss_weight=4.0), loss_seg=dict(type="CLRNetSegLoss", loss_weight=1.0, num_classes=5), ), test_cfg=dict( conf_threshold=0.41, use_nms=True, as_lanes=True, nms_thres=50, nms_topk=4, ), ) # Build model from config model = MODELS.build(model_cfg) ``` -------------------------------- ### Extract CULane Dataset Files Source: https://github.com/hirotomusiker/clrernet/blob/main/docs/DATASETS.md Extract the downloaded tar.gz files into your dataset directory. This command assumes you are in the directory where the files were downloaded. ```bash tar xf driver_37_30frame.tar.gz tar xf driver_100_30frame.tar.gz tar xf driver_193_90frame.tar.gzt tar xf annotations_new.tar.gz tar xf list.tar.gz ``` -------------------------------- ### Citation BibTeX Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md BibTeX entry for citing the CLRerNet paper. ```BibTeX @inproceedings{honda2024clrernet, title={Clrernet: improving confidence of lane detection with laneiou}, author={Honda, Hiroto and Uchida, Yusuke}, booktitle={Proceedings of the IEEE/CVF winter conference on applications of computer vision}, pages={1176--1185}, year={2024} } ``` -------------------------------- ### Evaluate Model on CULane Source: https://github.com/hirotomusiker/clrernet/blob/main/README.md Command to run model evaluation on the CULane dataset. ```bash python tools/test.py configs/clrernet/culane/clrernet_culane_dla34_ema.py clrernet_culane_dla34_ema.pth ``` -------------------------------- ### Implement LaneIoU Loss Source: https://context7.com/hirotomusiker/clrernet/llms.txt Calculate angle-aware lane intersection-over-union loss using the custom LaneIoU implementation. ```python import torch from libs.models.losses.iou_loss import LaneIoULoss, CLRNetIoULoss # Initialize LaneIoU loss (CLRerNet's novel contribution) lane_iou_loss = LaneIoULoss( loss_weight=4.0, lane_width=7.5 / 800, # Half virtual lane width relative to image width img_h=320, img_w=1640 ) # Initialize standard LineIoU loss (CLRNet baseline) line_iou_loss = CLRNetIoULoss( loss_weight=1.0, lane_width=15 / 800 ) # Example usage with prediction and target lane coordinates # pred shape: (num_lanes, num_rows), values in [0, 1] relative coords # target shape: (num_lanes, num_rows), values in [0, 1] relative coords pred = torch.rand(4, 72) # 4 lanes, 72 row points each target = torch.rand(4, 72) # Compute LaneIoU loss (considers local lane angles) loss = lane_iou_loss(pred, target) print(f"LaneIoU Loss: {loss.item():.4f}") ``` -------------------------------- ### Evaluate Model Performance Source: https://context7.com/hirotomusiker/clrernet/llms.txt Evaluates trained models on the CULane test set with options for visualization and outputting predictions. ```bash # Basic evaluation on CULane test set python tools/test.py \ configs/clrernet/culane/clrernet_culane_dla34_ema.py \ clrernet_culane_dla34_ema.pth # Save visualizations to directory python tools/test.py \ configs/clrernet/culane/clrernet_culane_dla34_ema.py \ clrernet_culane_dla34_ema.pth \ --show-dir=visualizations/ # Dump predictions to pickle file python tools/test.py \ configs/clrernet/culane/clrernet_culane_dla34_ema.py \ clrernet_culane_dla34_ema.pth \ --out=predictions.pkl # Test with custom work directory python tools/test.py \ configs/clrernet/culane/clrernet_culane_dla34_ema.py \ clrernet_culane_dla34_ema.pth \ --work-dir=./test_results ``` -------------------------------- ### Calculate Frame Differences Source: https://context7.com/hirotomusiker/clrernet/llms.txt Computes pixel-wise differences between consecutive training frames using `calculate_frame_diff.py` to filter redundant data. The output is saved as a NumPy file. Alternatively, pre-computed differences can be downloaded. ```bash # Calculate frame differences for CULane training set python tools/calculate_frame_diff.py $HOME/dataset/culane # Output: $HOME/dataset/culane/list/train_diffs.npz # Contains numpy array 'data' with difference values per frame # Alternative: download pre-computed differences wget https://github.com/hirotomusiker/CLRerNet/releases/download/v0.2.0/train_diffs.npz mv train_diffs.npz $HOME/dataset/culane/list/ ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.