### Install ScaleLSD Environment Source: https://context7.com/ant-research/scalelsd/llms.txt Instructions for setting up the Conda environment and installing necessary dependencies including PyTorch and the local package. ```bash git clone https://github.com/ant-research/scalelsd.git conda create -n scalelsd python=3.10 pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu121 pip install -r requirements.txt pip install -e . ``` -------------------------------- ### Launch Gradio Inference Demo Source: https://context7.com/ant-research/scalelsd/llms.txt Provides methods to start the interactive Gradio web interface for real-time line segment detection testing. ```python from gradio_demo.inference import run_demo, process_image # Programmatic launch run_demo() ``` -------------------------------- ### Instantiate and Configure ScaleLSD Model Source: https://context7.com/ant-research/scalelsd/llms.txt Shows how to initialize the ScaleLSD neural network model and configure inference parameters either programmatically or via command-line arguments. ```python import torch from scalelsd.ssl.models.detector import ScaleLSD model = ScaleLSD(gray_scale=True, use_layer_scale=False, enable_attention_hooks=False) ScaleLSD.configure(opts=type('opts', (), {'num_junctions': 512, 'junction_hm': 0.008})()) import argparse parser = argparse.ArgumentParser() ScaleLSD.cli(parser) ``` -------------------------------- ### Junction Detection with ScaleLSD (Python) Source: https://context7.com/ant-research/scalelsd/llms.txt Shows how to load a ScaleLSD model and use its `detect_junctions` method to find corner points in an image. It includes image preprocessing, model inference, and accessing junction parameters. Manual non-maximum suppression is also demonstrated. ```python import torch import cv2 from scalelsd.ssl.misc.train_utils import load_scalelsd_model # Load model and prepare image model = load_scalelsd_model('models/scalelsd-vitbase-v1-train-sa1b.pt', device='cuda') image = cv2.imread('path/to/image.jpg', 0) image_tensor = torch.from_numpy(cv2.resize(image, (512, 512))).float() / 255.0 image_tensor = image_tensor[None, None].to('cuda') # Detect junctions only (faster than full wireframe detection) with torch.no_grad(): junctions_batch = model.detect_junctions(image_tensor) # Results for first image in batch junctions = junctions_batch[0] # Shape: [N, 2] print(f"Detected {len(junctions)} junctions") # Access model junction parameters print(f"Junction threshold: {model.junction_threshold_hm}") print(f"Max junctions: {model.num_junctions_inference}") # Apply non-maximum suppression manually jloc = torch.rand(1, 1, 128, 128).cuda() # Example heatmap jloc_nms = ScaleLSD.non_maximum_suppression(jloc, kernel_size=3) ``` -------------------------------- ### Model Attributes and Usage Source: https://context7.com/ant-research/scalelsd/llms.txt Demonstrates how to access model attributes and use the model for training and inference. ```APIDOC ## Model Attributes and Usage ### Description Access model configuration parameters and utilize the model for both training and inference modes. ### Method N/A (Code examples) ### Endpoint N/A ### Parameters N/A ### Request Example ```python # Access model attributes print(f"Distance threshold: {model.distance_threshold}") # 5.0 print(f"J2L threshold: {model.j2l_threshold}") # 10 print(f"Backbone stride: {model.stride}") # For training mode model.train() loss_dict, extra_info = model(images, annotations=training_annotations) # loss_dict contains: loss_md, loss_dis, loss_res, loss_jloc, loss_joff # For inference mode model.eval() with torch.no_grad(): outputs, _ = model(images, annotations=meta) ``` ### Response N/A ``` -------------------------------- ### Run Single Image Inference with ScaleLSD Source: https://context7.com/ant-research/scalelsd/llms.txt Demonstrates how to perform programmatic image inference using the ScaleLSD model. It includes image preprocessing, parameter configuration for detection, and launching an interactive demo. ```python import cv2 input_image = cv2.imread('path/to/image.jpg') input_image = cv2.cvtColor(input_image, cv2.COLOR_BGR2RGB) result_image, json_file, image_file = process_image( input_image=input_image, model_name='scalelsd-vitbase-v1-train-sa1b.pt', save_name='my_detection', threshold=10, junction_threshold_hm=0.008, num_junctions_inference=512, width=512, height=512, line_width=2, juncs_size=4, whitebg=0.7, draw_junctions_only=False, use_lsd=False, use_nms=True, edge_color='orange', vertex_color='Cyan', output_format='png', seed=42, randomize_seed=False ) demo = run_demo() demo.launch(share=True) ``` -------------------------------- ### Manage Wireframe Graphs with WireframeGraph Source: https://context7.com/ant-research/scalelsd/llms.txt Demonstrates how to initialize a WireframeGraph from model outputs, perform rescaling, filter by confidence, and serialize the structure to JSON. This class is essential for post-processing detection results into a structured graph format. ```python from scalelsd.base import WireframeGraph import json indices = WireframeGraph.xyxy2indices(juncs_pred, lines_pred) wireframe = WireframeGraph(vertices=juncs_pred, v_confidences=juncs_score, edges=indices, edge_weights=lines_score, frame_width=width, frame_height=height) line_segments = wireframe.line_segments(threshold=10.0, to_np=True) wireframe.rescale(image_width=1920, image_height=1080) json_data = wireframe.jsonize() with open('wireframe.json', 'w') as f: json.dump(json_data, f) loaded_wireframe = WireframeGraph.load_json('wireframe.json') ``` -------------------------------- ### Load ScaleLSD Model Programmatically Source: https://context7.com/ant-research/scalelsd/llms.txt Initialize the pre-trained ScaleLSD model using the utility function. This automatically handles versioning and sets the model to evaluation mode. ```python import torch from scalelsd.ssl.misc.train_utils import load_scalelsd_model device = 'cuda' if torch.cuda.is_available() else 'cpu' model = load_scalelsd_model('models/scalelsd-vitbase-v1-train-sa1b.pt', device=device) model.junction_threshold_hm = 0.008 model.num_junctions_inference = 512 print(f"Model loaded on {device}") ``` -------------------------------- ### Utility Functions: Seeds and Data Parsing (Python) Source: https://context7.com/ant-research/scalelsd/llms.txt Illustrates the use of utility functions for ensuring reproducibility and handling data. `fix_seeds` sets random seeds for various libraries, and `parse_h5_data` reads data from HDF5 files. This ensures that model training and inference can yield consistent results. ```python from scalelsd.ssl.misc.train_utils import fix_seeds, parse_h5_data import h5py # Set random seeds for reproducibility fix_seeds(42) # Sets: random, numpy, torch, cuda, cudnn deterministic mode # Parse HDF5 dataset files (for training data) with h5py.File('dataset.h5', 'r') as f: data = parse_h5_data(f) # Returns dict with numpy arrays for each key # Reproducible inference from scalelsd.ssl.misc.train_utils import fix_seeds, load_scalelsd_model fix_seeds(42) model = load_scalelsd_model('models/scalelsd-vitbase-v1-train-sa1b.pt', device='cuda') # Results will be deterministic ``` -------------------------------- ### Visualize Detections with HAWPainter Source: https://context7.com/ant-research/scalelsd/llms.txt Shows how to use the HAWPainter utility to overlay wireframes and junctions onto images. It supports custom styling, confidence thresholding, and saving outputs to various file formats like PNG or PDF. ```python from scalelsd.base import show painter = show.painters.HAWPainter() painter.confidence_threshold = 10 with show.image_canvas(image_path, fig_file='output.png') as ax: painter.draw_wireframe(ax, result, edge_color='orange', vertex_color='Cyan') with show.image_canvas(image_path, fig_file='junctions_only.pdf') as ax: painter.draw_junctions(ax, result, vertex_color='deeppink') ``` -------------------------------- ### Model Attributes and Modes (Python) Source: https://context7.com/ant-research/scalelsd/llms.txt Demonstrates how to access model attributes like distance threshold and stride, and how to switch between training and inference modes. In training mode, the model processes images and annotations to compute losses. In inference mode, it performs forward passes without gradient computation. ```python print(f"Distance threshold: {model.distance_threshold}") # 5.0 print(f"J2L threshold: {model.j2l_threshold}") # 10 print(f"Backbone stride: {model.stride}") # For training mode model.train() loss_dict, extra_info = model(images, annotations=training_annotations) # loss_dict contains: loss_md, loss_dis, loss_res, loss_jloc, loss_joff # For inference mode model.eval() with torch.no_grad(): outputs, _ = model(images, annotations=meta) ``` -------------------------------- ### JSON Output Format Source: https://context7.com/ant-research/scalelsd/llms.txt Details the structure of the JSON output for line detection and how to use it. ```APIDOC ## JSON Output Format ### Description The JSON output from line detection contains the complete wireframe structure. ### Method N/A (Code examples) ### Endpoint N/A ### Parameters N/A ### Request Example ```json { "vertices": [[x1, y1], [x2, y2], ...], "vertices-score": [0.95, 0.87, ...], "edges": [[0, 1], [1, 2], ...], "edges-weights": [15.2, 12.8, ...], "height": 1080, "width": 1920 } ``` ### Response #### Success Response (200) - **vertices** (list of lists) - Junction coordinates. - **vertices-score** (list) - Junction confidences. - **edges** (list of lists) - Connectivity as index pairs. - **edges-weights** (list) - Line confidence scores. - **height** (int) - Image height. - **width** (int) - Image width. #### Response Example ```python import json from scalelsd.base import WireframeGraph # Load and use JSON output with open('detection.json', 'r') as f: data = json.load(f) # Reconstruct wireframe wireframe = WireframeGraph.load_json('detection.json') # Access data vertices = data['vertices'] # Junction coordinates vertex_scores = data['vertices-score'] # Junction confidences edges = data['edges'] # Connectivity as index pairs edge_weights = data['edges-weights'] # Line confidence scores # Convert edges to line segments import numpy as np vertices = np.array(vertices) edges = np.array(edges) lines = np.hstack([vertices[edges[:, 0]], vertices[edges[:, 1]]]) # lines shape: [N, 4] with format [x1, y1, x2, y2] ``` ``` -------------------------------- ### Utility Functions Source: https://context7.com/ant-research/scalelsd/llms.txt Provides helper functions for reproducibility and data handling. ```APIDOC ## Utility Functions ### Description Helper functions for reproducibility and data handling. ### Method N/A (Code examples) ### Endpoint N/A ### Parameters N/A ### Request Example ```python from scalelsd.ssl.misc.train_utils import fix_seeds, parse_h5_data import h5py # Set random seeds for reproducibility fix_seeds(42) # Sets: random, numpy, torch, cuda, cudnn deterministic mode # Parse HDF5 dataset files (for training data) with h5py.File('dataset.h5', 'r') as f: data = parse_h5_data(f) # Returns dict with numpy arrays for each key # Reproducible inference from scalelsd.ssl.misc.train_utils import fix_seeds, load_scalelsd_model fix_seeds(42) model = load_scalelsd_model('models/scalelsd-vitbase-v1-train-sa1b.pt', device='cuda') # Results will be deterministic ``` ### Response N/A ``` -------------------------------- ### Perform Command Line Inference Source: https://context7.com/ant-research/scalelsd/llms.txt Execute line segment detection on images via the command line. Supports single images or directories with configurable thresholds and output formats. ```bash python -m predictor.predict --img path/to/image.jpg python -m predictor.predict --img path/to/image_folder/ python -m predictor.predict --ckpt models/scalelsd-vitbase-v1-train-sa1b.pt --img path/to/image.jpg --ext json --threshold 10 --junction-hm 0.008 --num-junctions 512 --device cuda --disable-show ``` -------------------------------- ### Perform Line Matching with GlueStick Source: https://context7.com/ant-research/scalelsd/llms.txt Configures a two-view pipeline to match line segments between two images. It utilizes ScaleLSD for feature extraction and GlueStick for matching, outputting matched keypoints and lines. ```python import torch import cv2 import numpy as np from gluestick import batch_to_np, numpy_image_to_torch, GLUESTICK_ROOT from line_matching.two_view_pipeline import TwoViewPipeline pipeline_conf = {'name': 'two_view_pipeline', 'use_lines': True, 'extractor': {'name': 'wireframe', 'sp_params': {'force_num_keypoints': False, 'max_num_keypoints': 2048}, 'wireframe_params': {'merge_points': True, 'merge_line_endpoints': True}, 'max_n_lines': 512}, 'matcher': {'name': 'gluestick', 'weights': str(GLUESTICK_ROOT / 'resources/weights/checkpoint_GlueStick_MD.tar'), 'trainable': False}, 'ground_truth': {'from_pose_depth': False}} device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') pipeline = TwoViewPipeline(pipeline_conf).to(device).eval() pipeline.extractor.update_conf({'model_name': 'scalelsd-vitbase-v1-train-sa1b.pt', 'threshold': 5, 'junction_threshold_hm': 0.008, 'num_junctions_inference': 4096, 'use_lsd': False, 'use_nms': True, 'width': 512, 'height': 512}) gray0 = cv2.imread('image1.jpg', 0) gray1 = cv2.imread('image2.jpg', 0) torch_gray0 = numpy_image_to_torch(gray0).to(device)[None] torch_gray1 = numpy_image_to_torch(gray1).to(device)[None] with torch.no_grad(): pred = pipeline({'image0': torch_gray0, 'image1': torch_gray1}) pred = batch_to_np(pred) kp0, kp1 = pred['keypoints0'], pred['keypoints1'] m0 = pred['matches0'] valid_matches = m0 != -1 matched_kps0 = kp0[valid_matches] matched_kps1 = kp1[m0[valid_matches]] line_seg0, line_seg1 = pred['lines0'], pred['lines1'] line_matches = pred['line_matches0'] valid_line_matches = line_matches != -1 matched_lines0 = line_seg0[valid_line_matches] matched_lines1 = line_seg1[line_matches[valid_line_matches]] ``` -------------------------------- ### JSON Output Format for Wireframe Detection (JSON & Python) Source: https://context7.com/ant-research/scalelsd/llms.txt Details the structure of the JSON output generated by the line detection process, including vertices, scores, edges, and weights. It also provides Python code to load this JSON data and reconstruct the wireframe graph using the `WireframeGraph` class. ```json { "vertices": [[x1, y1], [x2, y2], ...], "vertices-score": [0.95, 0.87, ...], "edges": [[0, 1], [1, 2], ...], "edges-weights": [15.2, 12.8, ...], "height": 1080, "width": 1920 } ``` ```python import json from scalelsd.base import WireframeGraph # Load and use JSON output with open('detection.json', 'r') as f: data = json.load(f) # Reconstruct wireframe wireframe = WireframeGraph.load_json('detection.json') # Access data vertices = data['vertices'] # Junction coordinates vertex_scores = data['vertices-score'] # Junction confidences edges = data['edges'] # Connectivity as index pairs edge_weights = data['edges-weights'] # Line confidence scores # Convert edges to line segments import numpy as np vertices = np.array(vertices) edges = np.array(edges) lines = np.hstack([vertices[edges[:, 0]], vertices[edges[:, 1]]]) # lines shape: [N, 4] with format [x1, y1, x2, y2] ``` -------------------------------- ### Execute ScaleLSD Forward Pass Source: https://context7.com/ant-research/scalelsd/llms.txt Process an image through the model to extract lines, junctions, and connectivity graphs. Requires image normalization and metadata preparation. ```python import torch import cv2 import copy from scalelsd.ssl.misc.train_utils import load_scalelsd_model image_path = 'path/to/image.jpg' image = cv2.imread(image_path, 0) ori_shape = image.shape[:2] image_resized = cv2.resize(copy.deepcopy(image), (512, 512)) image_tensor = torch.from_numpy(image_resized).float() / 255.0 image_tensor = image_tensor[None, None].to('cuda') meta = {'width': ori_shape[1], 'height': ori_shape[0], 'filename': image_path, 'use_lsd': False, 'use_nms': True} model = load_scalelsd_model('models/scalelsd-vitbase-v1-train-sa1b.pt', device='cuda') with torch.no_grad(): outputs, _ = model(image_tensor, meta) result = outputs[0] lines = result['lines_pred'] junctions = result['juncs_pred'] ``` -------------------------------- ### Junction Detection API Source: https://context7.com/ant-research/scalelsd/llms.txt API for detecting junctions (corner points) in images using the ScaleLSD model. ```APIDOC ## Junction Detection ### Description Detect junctions (corner points) in images using the model's junction detector. The `detect_junctions` method extracts junction locations from the model's junction heatmap output with optional non-maximum suppression. ### Method N/A (Code examples) ### Endpoint N/A ### Parameters N/A ### Request Example ```python import torch import cv2 from scalelsd.ssl.misc.train_utils import load_scalelsd_model # Load model and prepare image model = load_scalelsd_model('models/scalelsd-vitbase-v1-train-sa1b.pt', device='cuda') image = cv2.imread('path/to/image.jpg', 0) image_tensor = torch.from_numpy(cv2.resize(image, (512, 512))).float() / 255.0 image_tensor = image_tensor[None, None].to('cuda') # Detect junctions only (faster than full wireframe detection) with torch.no_grad(): junctions_batch = model.detect_junctions(image_tensor) # Results for first image in batch junctions = junctions_batch[0] # Shape: [N, 2] print(f"Detected {len(junctions)} junctions") # Access model junction parameters print(f"Junction threshold: {model.junction_threshold_hm}") print(f"Max junctions: {model.num_junctions_inference}") # Apply non-maximum suppression manually # Assuming ScaleLSD class is available for static method call # jloc = torch.rand(1, 1, 128, 128).cuda() # Example heatmap # jloc_nms = ScaleLSD.non_maximum_suppression(jloc, kernel_size=3) ``` ### Response N/A ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.