### Load and Parse Camera Calibration (KRT Format) in Python Source: https://context7.com/facebookresearch/eyefultower/llms.txt This Python script shows how to load camera calibration data from a 'cameras.json' file and access specific camera parameters, such as the KRT (K, R, T) matrix, for a given image. ```python import json import numpy as np with open('apartment/cameras.json', 'r') as f: data = json.load(f) cameras = data['KRT'] camera = cameras[0] ``` -------------------------------- ### Download Specific Resolution/Format using AWS CLI Source: https://context7.com/facebookresearch/eyefultower/llms.txt Demonstrates downloading specific data types, such as 2K HDR EXR images for a scene, or only camera calibration and split files for all scenes, using AWS CLI commands. ```bash aws s3 cp --recursive --no-sign-request \ s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/apartment/images-2k/ \ apartment/images-2k/ for dataset in apartment kitchen office1a office1b office2 \ office_view1 office_view2 riverview seating_area \ table workshop; do mkdir -p $dataset aws s3 cp --no-sign-request \ s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/$dataset/cameras.json \ $dataset/cameras.json aws s3 cp --no-sign-request \ s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/$dataset/splits.json \ $dataset/splits.json done ``` -------------------------------- ### Sync a Single Scene's 1K JPEG Images using AWS CLI Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md This AWS CLI command synchronizes images from a specific scene's 1K JPEG directory recursively. The 'sync' command is useful as it avoids re-downloading files that already exist locally, making subsequent downloads faster. ```bash aws s3 sync --no-sign-request s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/apartment/images-jpeg-1k/ apartment/images-jpeg-1k/ ``` -------------------------------- ### Download a Single Scene's 1K JPEG Images using AWS CLI Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md This AWS CLI command downloads all images from a specific scene's 1K JPEG directory recursively. It utilizes the 'cp' command for copying files and '--no-sign-request' for accessing public S3 buckets. ```bash aws s3 cp --recursive --no-sign-request s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/apartment/images-jpeg-1k/ apartment/images-jpeg-1k/ ``` -------------------------------- ### Download All Scenes' 1K JPEG Images using Bash Loop and AWS CLI Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md This bash script iterates through a list of scene names, creates a corresponding directory for each, and then uses the AWS CLI to download the 1K JPEG images for each scene recursively. It's designed for downloading the main image data for all scenes. ```bash for dataset in apartment kitchen office1a office1b office2 office_view1 office_view2 riverview seating_area table workshop; do mkdir -p $dataset/images-jpeg-1k; aws s3 cp --recursive --no-sign-request s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/$dataset/images-jpeg-1k/ $dataset/images-jpeg-1k/; done ``` -------------------------------- ### Create Custom Train/Test Data Splits (Python) Source: https://context7.com/facebookresearch/eyefultower/llms.txt This script demonstrates creating custom data splits for training and testing. It offers two strategies: holding out specific cameras entirely for testing, and performing a random frame-based split (e.g., 80/20). The script loads camera information from a JSON file, processes it to identify unique camera and frame IDs, and then generates lists of image identifiers for training and testing sets according to the chosen strategy. The custom split can optionally be saved to a new JSON file. ```python import json import random from pathlib import Path # Load all camera IDs with open('apartment/cameras.json', 'r') as f: cameras = json.load(f)['KRT'] # Extract unique camera IDs and frameIds camera_ids = list(set(c['cameraId'] for c in cameras)) all_images = [c['cameraId'] for c in cameras] # Strategy 1: Hold out specific cameras for testing test_camera_sensors = [17, 18] # Hold out 2 cameras test_images = [img for img in all_images if any(img.startswith(f"{cam}/") for cam in test_camera_sensors)] train_images = [img for img in all_images if img not in test_images] print(f"Camera holdout split:") print(f" Train: {len(train_images)} images") print(f" Test: {len(test_images)} images") # Strategy 2: Random frame-based split (80/20) frame_ids = list(set(c['frameId'] for c in cameras)) random.shuffle(frame_ids) split_idx = int(0.8 * len(frame_ids)) train_frames = set(frame_ids[:split_idx]) test_frames = set(frame_ids[split_idx:]) train_by_frame = [c['cameraId'] for c in cameras if c['frameId'] in train_frames] test_by_frame = [c['cameraId'] for c in cameras if c['frameId'] in test_frames] print(f"\nFrame-based split:") print(f" Train: {len(train_by_frame)} images ({len(train_frames)} frames)") print(f" Test: {len(test_by_frame)} images ({len(test_frames)} frames)") # Save custom split custom_split = { 'train': train_images, 'test': test_images } with open('apartment/splits_custom.json', 'w') as f: json.dump(custom_split, f, indent=2) ``` -------------------------------- ### Download All Scenes (1K JPEG) using AWS CLI Loop Source: https://context7.com/facebookresearch/eyefultower/llms.txt This script downloads 1K JPEG images for all 11 primary scenes of the Eyeful Tower dataset by iterating through a list of scene names and using AWS CLI commands. It outlines the directory structure created. ```bash for dataset in apartment kitchen office1a office1b office2 \ office_view1 office_view2 riverview seating_area \ table workshop; do mkdir -p $dataset/images-jpeg-1k aws s3 cp --recursive --no-sign-request \ s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/$dataset/images-jpeg-1k/ \ $dataset/images-jpeg-1k/ done ``` -------------------------------- ### Read EXR and Convert to JPEG using Python Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md This Python script demonstrates how to read High Dynamic Range (HDR) EXR images using OpenCV, apply white-balance scaling, perform tonemapping to a sRGB curve, and save the result as a JPEG file. It requires the OpenCV library and the 'OPENCV_IO_ENABLE_OPENEXR' environment variable to be set for EXR support. The script takes an EXR image path as input and outputs a JPEG image. ```python import os, cv2, numpy as np # Enable OpenEXR support in OpenCV (https://github.com/opencv/opencv/issues/21326). # This environment variable needs to be defined before the first EXR image is opened. os.environ["OPENCV_IO_ENABLE_OPENEXR"] = "1" # Read an EXR image using OpenCV. img = cv2.imread("apartment/images-2k/17/17_DSC0316.exr", cv2.IMREAD_UNCHANGED) # Apply white-balance scaling (Note: OpenCV uses BGR colors). coeffs = np.array([0.726097, 1.0, 1.741252]) # apartment [RGB] img = np.einsum("ijk,k->ijk", img, coeffs[::-1]) # Tonemap using sRGB curve. linear_part = 12.92 * img exp_part = 1.055 * (np.maximum(img, 0.0) ** (1 / 2.4)) - 0.055 img = np.where(img <= 0.0031308, linear_part, exp_part) # Write resulting image as JPEG. img = np.clip(255 * img, 0.0, 255.0).astype(np.uint8) cv2.imwrite("apartment-17_DSC0316.jpg", img, params=[cv2.IMWRITE_JPEG_QUALITY, 100]) ``` -------------------------------- ### Download Single Scene (1K JPEG) using AWS CLI Source: https://context7.com/facebookresearch/eyefultower/llms.txt This code snippet demonstrates how to download a single scene's 1K JPEG images from the Eyeful Tower dataset using the AWS CLI. It shows both direct copy and sync commands, explaining the resulting file structure. ```bash aws s3 cp --recursive --no-sign-request \ s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/apartment/images-jpeg-1k/ \ apartment/images-jpeg-1k/ aws s3 sync --no-sign-request \ s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/apartment/images-jpeg-1k/ \ apartment/images-jpeg-1k/ ``` -------------------------------- ### Load Predefined Train/Test Data Splits from JSON (Python) Source: https://context7.com/facebookresearch/eyefultower/llms.txt This Python snippet shows how to load pre-defined training and testing data splits from a JSON file. It uses the `json` library to parse the file and extracts lists of image identifiers for training and testing sets. The code then prints the counts and the first few image identifiers for each split, along with explanatory notes on specific camera exclusions. ```python import json from pathlib import Path # Load splits configuration with open('apartment/splits.json', 'r') as f: splits = json.load(f) train_images = splits['train'] test_images = splits['test'] print(f"Training images: {len(train_images)}") print(f"Testing images: {len(test_images)}") print(f"\nFirst 5 training images:") for img in train_images[:5]: print(f" {img}") print(f"\nFirst 5 testing images:") for img in test_images[:5]: print(f" {img}") # Output: # Training images: 3564 # Testing images: 396 # # First 5 training images: # 10/10_DSC0001 # 10/10_DSC0010 # 10/10_DSC0019 # 10/10_DSC0028 # 10/10_DSC0037 # # First 5 testing images: # 17/17_DSC0001 # 17/17_DSC0010 # 17/17_DSC0019 # 17/17_DSC0028 # 17/17_DSC0037 # # Note: Camera 17 is held out for testing in v2 scenes # Camera 5 is held out for testing in v1 scenes ``` -------------------------------- ### Extract Camera Parameters using Python Source: https://context7.com/facebookresearch/eyefultower/llms.txt Extracts camera intrinsic matrix (K), extrinsic transformation matrix (T), rotation (R), translation (t), and distortion parameters from camera calibration data. It reshapes matrices from column-major format and prints them. ```python import numpy as np # Assuming 'camera' is a dictionary loaded with calibration data # Example structure: # camera = { # 'width': 5784, # 'height': 8660, # 'cameraId': '17/17_DSC0316', # 'frameId': 10, # 'sensorId': 1, # 'K': [4891.23, 0.0, 2892.00, 0.0, 4891.23, 4330.00, 0.0, 0.0, 1.0], # 'T': [ ... ], # 16 float values for 4x4 matrix # 'distortionModel': 'RadialAndTangential', # 'distortion': [k1, k2, p1, p2, k3] # } width = camera['width'] # 5784 pixels height = camera['height'] # 8660 pixels camera_id = camera['cameraId'] # e.g., "17/17_DSC0316" frame_id = camera['frameId'] # Position index during capture sensor_id = camera['sensorId'] # Camera ID # K: 3x3 intrinsic matrix (column-major format) K_flat = camera['K'] K = np.array(K_flat).reshape(3, 3, order='F') # Reshape column-major print(f"Intrinsic matrix K:\n{K}") # T: 4x4 world-to-camera transformation matrix (column-major) T_flat = camera['T'] T = np.array(T_flat).reshape(4, 4, order='F') # Reshape column-major print(f"Extrinsic matrix T (world-to-camera):\n{T}") # Extract rotation and translation R = T[:3, :3] # 3x3 rotation matrix t = T[:3, 3] # 3x1 translation vector # Distortion parameters distortion_model = camera['distortionModel'] # "RadialAndTangential" or "Fisheye" distortion = camera['distortion'] # [k1, k2, p1, p2, k3] for pinhole print(f"Camera: {camera_id}") print(f"Distortion model: {distortion_model}") print(f"Distortion coefficients: {distortion}") ``` -------------------------------- ### Configure AWS CLI for Increased Downloads Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md This command configures the AWS Command Line Interface (CLI) to increase the maximum number of concurrent S3 download requests from the default of 10 to 100. This can significantly speed up bulk downloads from AWS S3 buckets. ```bash aws configure set default.s3.max_concurrent_requests 100 ``` -------------------------------- ### Camera Calibration in Metashape XML Format Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md Camera calibration data exported directly from Metashape using its proprietary XML format. This file is essential for understanding camera intrinsic and extrinsic parameters. ```xml 1920 1080 1200.0 1200.0 960.0 540.0 pinhole ``` -------------------------------- ### Download Entire Dataset using AWS CLI Sync Source: https://context7.com/facebookresearch/eyefultower/llms.txt This command synchronizes the entire Eyeful Tower dataset (3.5 TB) from an S3 bucket to the local directory. It also includes an optional configuration step to increase AWS CLI's concurrent download requests for potentially faster transfers. ```bash aws configure set default.s3.max_concurrent_requests 100 aws s3 sync --no-sign-request \ s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/ . ``` -------------------------------- ### Download the Entire Eyeful Tower Dataset using AWS CLI Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md This AWS CLI command synchronizes the entire Eyeful Tower dataset from the specified S3 bucket to the current local directory. It's a comprehensive download command that will transfer all available data. ```bash aws s3 sync --no-sign-request s3://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15/EyefulTower/ . ``` -------------------------------- ### Batch Process HDR EXR Images to Tone-Mapped JPEG (OpenCV/NumPy) Source: https://context7.com/facebookresearch/eyefultower/llms.txt This Python script processes all EXR (HDR) images within a specified directory, applies white balance and sRGB gamma correction (tonemapping), and saves the results as JPEG files. It utilizes OpenCV for image reading/writing and NumPy for numerical operations. The script preserves the directory structure and allows customization of white balance coefficients and JPEG quality. ```python import os import cv2 import numpy as np from pathlib import Path from tqdm import tqdm os.environ["OPENCV_IO_ENABLE_OPENEXR"] = "1" def tonemap_exr_to_jpeg(exr_path, jpeg_path, white_balance_rgb): """Convert HDR EXR image to tone-mapped JPEG.""" # Read EXR img = cv2.imread(str(exr_path), cv2.IMREAD_UNCHANGED) if img is None: return False # White balance (BGR order for OpenCV) coeffs = np.array(white_balance_rgb[::-1]) img = np.einsum("ijk,k->ijk", img, coeffs) # sRGB gamma correction linear = 12.92 * img exp = 1.055 * (np.maximum(img, 0.0) ** (1 / 2.4)) - 0.055 img = np.where(img <= 0.0031308, linear, exp) # Save JPEG img = np.clip(255 * img, 0.0, 255.0).astype(np.uint8) cv2.imwrite(str(jpeg_path), img, [cv2.IMWRITE_JPEG_QUALITY, 95]) return True # Process all EXR images in apartment scene scene = 'apartment' exr_dir = Path(f'{scene}/images-2k') jpeg_dir = Path(f'{scene}/images-jpeg-2k-custom') white_balance = [0.726097, 1.0, 1.741252] # Create output directory structure jpeg_dir.mkdir(exist_ok=True) # Process all cameras exr_files = list(exr_dir.rglob('*.exr')) print(f"Processing {len(exr_files)} EXR images...") for exr_path in tqdm(exr_files): # Maintain directory structure relative_path = exr_path.relative_to(exr_dir) jpeg_path = jpeg_dir / relative_path.with_suffix('.jpg') jpeg_path.parent.mkdir(parents=True, exist_ok=True) tonemap_exr_to_jpeg(exr_path, jpeg_path, white_balance) print(f"Completed! Output saved to: {jpeg_dir}") ``` -------------------------------- ### Apply White Balance and sRGB Gamma Correction to an Image (OpenCV/NumPy) Source: https://context7.com/facebookresearch/eyefultower/llms.txt This snippet demonstrates applying white balance correction using specified coefficients and then performing sRGB gamma correction (tonemapping) on an image. It handles the color order differences between RGB and BGR as used by OpenCV. The processed image is then converted to 8-bit format and saved as a JPEG file with high quality. ```python # Apply white balance for apartment scene (OpenCV uses BGR color order) coeffs = np.array(white_balance_coeffs['apartment']) img = np.einsum("ijk,k->ijk", img, coeffs[::-1]) # Reverse RGB to BGR # Apply sRGB gamma correction (tonemapping) linear_part = 12.92 * img exp_part = 1.055 * (np.maximum(img, 0.0) ** (1 / 2.4)) - 0.055 img = np.where(img <= 0.0031308, linear_part, exp_part) # Convert to 8-bit and save as JPEG img = np.clip(255 * img, 0.0, 255.0).astype(np.uint8) cv2.imwrite("apartment-17_DSC0316.jpg", img, params=[cv2.IMWRITE_JPEG_QUALITY, 100]) print("Saved: apartment-17_DSC0316.jpg") ``` -------------------------------- ### Training and Testing Data Splits Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md A JSON file containing lists of images designated for training and testing. All images from a single camera are held out for testing purposes, with specific camera IDs (5 for Eyeful v1, 17 for Eyeful v2) noted. ```json { "train": [ "image_0001.jpg", "image_0002.jpg", "image_0003.jpg" ], "test": [ "camera5_image_001.jpg", "camera5_image_002.jpg" ] } ``` -------------------------------- ### Read HDR EXR Images with OpenCV Source: https://context7.com/facebookresearch/eyefultower/llms.txt Reads High Dynamic Range (HDR) images in EXR format using OpenCV and applies white-balance correction. It requires setting an environment variable to enable OpenEXR support and uses scene-specific coefficients for white balancing. ```python import os import cv2 import numpy as np # Enable OpenEXR support in OpenCV (must be set before first EXR read) os.environ["OPENCV_IO_ENABLE_OPENEXR"] = "1" # Read HDR EXR image img = cv2.imread("apartment/images-2k/17/17_DSC0316.exr", cv2.IMREAD_UNCHANGED) print(f"HDR image shape: {img.shape}, dtype: {img.dtype}") # Apply white-balance correction (scene-specific RGB coefficients) # White balance coefficients for each scene (from ColorChecker calibration) white_balance_coeffs = { 'apartment': [0.726097, 1.0, 1.741252], 'kitchen': [0.628143, 1.0, 2.212346], 'office1a': [0.740846, 1.0, 1.750224], 'office1b': [0.725535, 1.0, 1.839938], 'office2': [0.707729, 1.0, 1.747833], 'office_view1': [1.029089, 1.0, 1.145235], 'office_view2': [0.939620, 1.0, 1.273549], 'riverview': [1.077719, 1.0, 1.145992], 'seating_area': [0.616093, 1.0, 2.426888], 'table': [0.653298, 1.0, 2.139514], 'workshop': [0.709929, 1.0, 1.797705], 'raf_emptyroom': [0.718776, 1.0, 1.787020], 'raf_furnishedroom': [0.721494, 1.0, 1.793423], } # Example of applying white balance (assuming scene is 'apartment') # scene = 'apartment' # if scene in white_balance_coeffs: # wb_coeffs = white_balance_coeffs[scene] # img_wb = img * wb_coeffs # else: # img_wb = img # No white balance applied if scene not found # For demonstration, we'll just print the shape and dtype. # To convert to JPEG, you would typically tone map the HDR image first. # For example: # tone_mapped_img = cv2.createTonemapDurand()(img.astype(np.float32)) # jpeg_img = (tone_mapped_img * 255).astype(np.uint8) # cv2.imwrite('output.jpg', jpeg_img) ``` -------------------------------- ### Load COLMAP Reconstruction Data - Python Source: https://context7.com/facebookresearch/eyefultower/llms.txt Parses COLMAP's 'cameras.txt' and 'images.txt' files to extract camera intrinsics and image extrinsics (pose information). It reads camera parameters like model, width, height, and focal length, and image data including quaternions and translation vectors. This data is crucial for understanding the 3D scene structure captured by COLMAP. ```python import numpy as np from pathlib import Path def read_colmap_cameras(cameras_txt_path): """Parse COLMAP cameras.txt file.""" cameras = {} with open(cameras_txt_path, 'r') as f: for line in f: if line.startswith('#'): continue parts = line.strip().split() if len(parts) >= 5: cam_id = int(parts[0]) model = parts[1] width = int(parts[2]) height = int(parts[3]) params = [float(p) for p in parts[4:]] cameras[cam_id] = { 'model': model, 'width': width, 'height': height, 'params': params } return cameras def read_colmap_images(images_txt_path): """Parse COLMAP images.txt file.""" images = {} with open(images_txt_path, 'r') as f: lines = [l.strip() for l in f if not l.startswith('#') and l.strip()] for i in range(0, len(lines), 2): parts = lines[i].split() img_id = int(parts[0]) qw, qx, qy, qz = map(float, parts[1:5]) tx, ty, tz = map(float, parts[5:8]) cam_id = int(parts[8]) name = parts[9] images[img_id] = { 'camera_id': cam_id, 'name': name, 'qvec': np.array([qw, qx, qy, qz]), 'tvec': np.array([tx, ty, tz]) } return images # Load COLMAP data colmap_dir = Path('apartment/colmap/sparse') cameras = read_colmap_cameras(colmap_dir / 'cameras.txt') images = read_colmap_images(colmap_dir / 'images.txt') print(f"Loaded {len(cameras)} cameras and {len(images)} images from COLMAP") print(f"\nFirst camera:") print(cameras[list(cameras.keys())[0]]) print(f"\nFirst image:") print(images[list(images.keys())[0]]) ``` -------------------------------- ### Undistort Images using OpenCV Source: https://context7.com/facebookresearch/eyefultower/llms.txt Undistorts images based on camera calibration data (intrinsic matrix and distortion coefficients). It supports both standard pinhole ('RadialAndTangential') and fisheye ('Fisheye') camera models. ```python import cv2 import numpy as np import json # Load camera calibration from JSON file with open('apartment/cameras.json', 'r') as f: cameras = json.load(f)['KRT'] # Find camera parameters for a specific image using cameraId camera = next(c for c in cameras if c['cameraId'] == '17/17_DSC0316') # Extract intrinsic matrix and distortion coefficients, width, and height K = np.array(camera['K']).reshape(3, 3, order='F') distortion = np.array(camera['distortion']) width = camera['width'] height = camera['height'] # Load the distorted image img = cv2.imread('apartment/images-jpeg/17/17_DSC0316.jpg') # Undistort the image based on the camera's distortion model if camera['distortionModel'] == 'RadialAndTangential': # Standard pinhole camera model undistorted = cv2.undistort(img, K, distortion) elif camera['distortionModel'] == 'Fisheye': # Fisheye camera model # Adjust K for optimal new camera matrix if needed K_new = cv2.getOptimalNewCameraMatrix(K, distortion[:8], (width, height), 1)[0] undistorted = cv2.fisheye.undistortImage(img, K, distortion[:4], K_new=K_new) # Save the undistorted image cv2.imwrite('undistorted_17_DSC0316.jpg', undistorted) print(f"Saved undistorted image: undistorted_17_DSC0316.jpg") ``` -------------------------------- ### BibTeX Citation for VR-NeRF Paper Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md The BibTeX entry for citing the VR-NeRF paper, which should be used when referencing data or code from this repository. Includes author, title, booktitle, year, DOI, and URL. ```bibtex @InProceedings{VRNeRF, author = {Linning Xu and Vasu Agrawal and William Laney and Tony Garcia and Aayush Bansal and Changil Kim and Rota Bulò, Samuel and Lorenzo Porzi and Peter Kontschieder and Aljaž Božič and Dahua Lin and Michael Zollhöfer and Christian Richardt}, title = {{VR-NeRF}: High-Fidelity Virtualized Walkable Spaces}, booktitle = {SIGGRAPH Asia Conference Proceedings}, year = {2023}, doi = {10.1145/3610548.3618139}, url = {https://vr-nerf.github.io}, } ``` -------------------------------- ### COLMAP Reconstruction Exported with Undistorted Images Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md COLMAP reconstructions exported from Metashape, featuring automatically undistorted images with principal points at the image center. These images are in Pinhole projection and are cropped, potentially leading to varying image sizes. Downsampled versions are also provided, similar to the Mip-NeRF 360 dataset format. ```bash # Example COLMAP structure # colmap/ # ├── images/ # │ ├── image001.png # │ └── image002.png # ├── images-1/ # │ ├── image001.png # │ └── image002.png # ├── sparse/ # │ └── 0/ # │ ├── cameras.bin # │ ├── images.bin # │ └── points3D.bin # └── dense/ # └── 0/ # ├── dense.ply # └── images.bin ``` -------------------------------- ### Group Images by Capture Time (Frame ID) Source: https://context7.com/facebookresearch/eyefultower/llms.txt Groups camera data by `frameId` to identify images captured simultaneously. It first finds cameras for a specific `frameId` and then aggregates all cameras into synchronized capture sets. ```python import json from collections import defaultdict # Load camera data from JSON file with open('apartment/cameras.json', 'r') as f: cameras = json.load(f)['KRT'] # Find all cameras that captured at the same moment (same frameId) target_frame_id = 10 synchronized_cameras = [ c for c in cameras if c['frameId'] == target_frame_id ] print(f"Found {len(synchronized_cameras)} cameras at frame {target_frame_id}:") for cam in synchronized_cameras: print(f" - Camera ID: {cam['cameraId']}, Sensor: {cam['sensorId']}") # Group all images by frameId to get synchronized capture sets frames = defaultdict(list) for cam in cameras: frames[cam['frameId']].append(cam['cameraId']) print(f"\nTotal synchronized capture moments: {len(frames)}") print(f"Images per moment: {len(frames[0])}") ``` -------------------------------- ### 3D Textured Mesh in OBJ Format Source: https://github.com/facebookresearch/eyefultower/blob/main/README.md A textured 3D mesh in OBJ format, exported from Metashape and created from full-resolution JPEG images. The mesh uses a right-handed, y-up world coordinate system where y=0 is the ground plane, with units in meters. ```obj # OBJ file generated from Metashape v 1.0 2.0 3.0 v 4.0 5.0 6.0 v 7.0 8.0 9.0 f 1 2 3 usemtl material_0 ``` -------------------------------- ### Convert COLMAP to NeRF Format - Python Source: https://context7.com/facebookresearch/eyefultower/llms.txt Converts COLMAP camera and image data into the 'transforms.json' format required by NeRF models. It transforms camera poses from COLMAP's world-to-camera convention to camera-to-world and calculates camera intrinsics (like focal length and principal point) for NeRF. This output file is essential for training NeRF. ```python import numpy as np import json from pathlib import Path def qvec2rotmat(qvec): """Convert quaternion to rotation matrix.""" qw, qx, qy, qz = qvec return np.array([ [1 - 2*qy**2 - 2*qz**2, 2*qx*qy - 2*qz*qw, 2*qx*qz + 2*qy*qw], [2*qx*qy + 2*qz*qw, 1 - 2*qx**2 - 2*qz**2, 2*qy*qz - 2*qx*qw], [2*qx*qz - 2*qy*qw, 2*qy*qz + 2*qx*qw, 1 - 2*qx**2 - 2*qy**2] ]) # Load COLMAP data (using functions from previous example) colmap_dir = Path('apartment/colmap/sparse') cameras = read_colmap_cameras(colmap_dir / 'cameras.txt') images = read_colmap_images(colmap_dir / 'images.txt') # Convert to NeRF format (transforms.json) frames = [] for img_id, img_data in images.items(): cam = cameras[img_data['camera_id']] # Get camera-to-world transformation R = qvec2rotmat(img_data['qvec']) t = img_data['tvec'] # COLMAP uses world-to-camera, convert to camera-to-world R_w2c = R.T t_w2c = -R.T @ t # Create 4x4 transformation matrix transform = np.eye(4) transform[:3, :3] = R_w2c transform[:3, 3] = t_w2c frames.append({ 'file_path': f"images/{img_data['name']}", 'transform_matrix': transform.tolist() }) # Get camera intrinsics (assuming PINHOLE model) cam = cameras[1] fx, fy, cx, cy = cam['params'] transforms = { 'camera_angle_x': 2 * np.arctan(cam['width'] / (2 * fx)), 'camera_angle_y': 2 * np.arctan(cam['height'] / (2 * fy)), 'fl_x': fx, 'fl_y': fy, 'cx': cx, 'cy': cy, 'w': cam['width'], 'h': cam['height'], 'frames': frames } # Save NeRF-compatible transforms with open('apartment/transforms.json', 'w') as f: json.dump(transforms, f, indent=2) print(f"Converted {len(frames)} images to NeRF format") print(f"Saved to: apartment/transforms.json") ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.