MVSNet and R-MVSNet

https://github.com/yoyo000/mvsnet
Admin
MVSNet and R-MVSNet are deep learning architectures for depth map inference and scalable...
Tokens:11,721
Snippets:90
Trust Score:8.6
License:MIT
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install Dependencies

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Install TensorFlow and other required Python packages using pip. Ensure CUDA 9.0 and cuDNN 7.0 are installed.

```bash
sudo pip install -r requirements.txt
```

--------------------------------

### Resume Training from Checkpoint

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Use this command to resume training for either MVSNet or R-MVSNet from a previously saved checkpoint. Ensure the model folder and checkpoint step are correctly specified.

```bash
python mvsnet/train.py \
    --regularization 'GRU' \
    --train_blendedmvs \
    --blendedmvs_data_root /data/BlendedMVS/dataset_low_res \
    --use_pretrain \
    --model_folder /data/tf_model \
    --ckpt_step 50000
```

--------------------------------

### Visualize .pfm Depth Map Output

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Use this script to visualize the generated depth maps in `.pfm` format. Provide the path to the specific depth map file.

```bash
python mvsnet/visualize.py /data/scan9/depths_mvsnet/00000000_init.pfm
```

--------------------------------

### View Selection File Format

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

The `pair.txt` file stores the top 10 best source image IDs and their scores for each reference image.

```text
TOTAL_IMAGE_NUM
IMAGE_ID0                       # index of reference image 0 
10 ID0 SCORE0 ID1 SCORE1 ...    # 10 best source images for reference image 0 
IMAGE_ID1                       # index of reference image 1
10 ID0 SCORE0 ID1 SCORE1 ...    # 10 best source images for reference image 1 
...

```

--------------------------------

### Compile Fusibile

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Instructions to compile the fusibile library after cloning the repository. This generates the necessary executable for post-processing.

```bash
cmake .
```

```bash
make
```

--------------------------------

### Run Post-Processing with Fusibile

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Perform depth map filtering and fusion to generate a 3D point cloud using the fusibile tool. Ensure fusibile is compiled and its executable path is provided. The probability threshold can be adjusted based on the regularization method used.

```python
python depthfusion.py --dense_folder TEST_DATA_FOLDER --fusibile_exe_path FUSIBILE_EXE_PATH --prob_threshold 0.3
```

--------------------------------

### Camera Parameter File Format

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

The `cam.txt` file stores camera extrinsics (E), intrinsics (K), and depth range information.

```text
extrinsic
E00 E01 E02 E03
E10 E11 E12 E13
E20 E21 E22 E23
E30 E31 E32 E33

intrinsic
K00 K01 K02
K10 K11 K12
K20 K21 K22

DEPTH_MIN DEPTH_INTERVAL (DEPTH_NUM DEPTH_MAX) 

```

--------------------------------

### Navigate to MVSNet Script Folder

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Change the current directory to the mvsnet script folder to execute training commands.

```bash
cd MVSNet/mvsnet
```

--------------------------------

### Visualize PFM Format Results

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Command to visualize depth and probability maps stored in `.pfm` format using the provided visualization script.

```python
python visualize.py .pfm

```

--------------------------------

### load_cam() / write_cam() — Camera File I/O

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Read and write MVSNet camera format files, which include extrinsic and intrinsic matrices, and depth range parameters.

```APIDOC
## load_cam() / write_cam() — Camera File I/O

**`mvsnet/preprocess.py`** — Read and write the MVSNet camera format (extrinsic E, intrinsic K, depth range).

Camera files are plain-text `.txt` files containing the 4×4 extrinsic matrix, 3×3 intrinsic matrix, and depth range parameters `(DEPTH_MIN, DEPTH_INTERVAL, DEPTH_NUM, DEPTH_MAX)`.

```python
from preprocess import load_cam, write_cam

# Load a camera file
with open('/data/scan9/cams/00000000_cam.txt') as f:
    cam = load_cam(f, interval_scale=1.06)
# cam: numpy array [2, 4, 4]
# cam[0]       -> 4x4 extrinsic matrix [R|t]
# cam[1, :3, :3] -> 3x3 intrinsic matrix K
# cam[1, 3, 0]   -> DEPTH_MIN  (e.g. 425.0)
# cam[1, 3, 1]   -> DEPTH_INTERVAL (e.g. 2.65)
# cam[1, 3, 2]   -> DEPTH_NUM  (e.g. 192.0)
# cam[1, 3, 3]   -> DEPTH_MAX  (e.g. 934.6)

print(f"Depth range: {cam[1,3,0]:.1f} to {cam[1,3,3]:.1f}")
print(f"Intrinsic K:\n{cam[1, :3, :3]}")

# Modify and write back
cam[1, 3, 2] = 256  # update depth sample count
write_cam('/data/scan9/cams/00000000_cam.txt', cam)
```
```

--------------------------------

### Run MVSNet Testing

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Execute MVSNet for depth map generation. Specify the test data folder and regularization method. Adjust max width, height, and depth for different hardware or desired output resolution. Interval scale affects the sampling density of depth hypotheses.

```python
python test.py --dense_folder TEST_DATA_FOLDER  --regularization '3DCNNs' --max_w 1152 --max_h 864 --max_d 192 --interval_scale 1.06
```

--------------------------------

### Load and Write MVSNet Camera Files

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Handles reading and writing camera intrinsic, extrinsic, and depth parameters in MVSNet's text file format. Adjusts depth interval scale during loading.

```python
from preprocess import load_cam, write_cam

# Load a camera file
with open('/data/scan9/cams/00000000_cam.txt') as f:
    cam = load_cam(f, interval_scale=1.06)
# cam: numpy array [2, 4, 4]
# cam[0]       -> 4x4 extrinsic matrix [R|t]
# cam[1, :3, :3] -> 3x3 intrinsic matrix K
# cam[1, 3, 0]   -> DEPTH_MIN  (e.g. 425.0)
# cam[1, 3, 1]   -> DEPTH_INTERVAL (e.g. 2.65)
# cam[1, 3, 2]   -> DEPTH_NUM  (e.g. 192.0)
# cam[1, 3, 3]   -> DEPTH_MAX  (e.g. 934.6)

print(f"Depth range: {cam[1,3,0]:.1f} to {cam[1,3,3]:.1f}")
print(f"Intrinsic K:\n{cam[1, :3, :3]}")

# Modify and write back
cam[1, 3, 2] = 256  # update depth sample count
write_cam('/data/scan9/cams/00000000_cam.txt', cam)
```

--------------------------------

### Run MVSNet Inference

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Execute depth map inference using a trained MVSNet model. Provide the dense folder path, regularization type, pre-trained model path, checkpoint step, and image/depth dimensions. GTX1080Ti recommended settings are shown.

```bash
python mvsnet/test.py \
    --dense_folder /data/scan9 \
    --regularization '3DCNNs' \
    --pretrained_model_ckpt_path /data/tf_model/3DCNNs/model.ckpt \
    --ckpt_step 150000 \
    --max_w 1152 --max_h 864 --max_d 192 \
    --interval_scale 1.06
```

--------------------------------

### Clone Repository

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Clone the MVSNet source code from the GitHub repository.

```bash
git clone https://github.com/YoYo000/MVSNet
```

--------------------------------

### Generate Point Cloud with Depth Fusion

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Performs post-processing on depth maps, including probability filtering and fusion into a dense 3D point cloud using Fusibile. Supports MVSNet and Gipuma formats.

```bash
# Full post-processing pipeline to generate a point cloud
python mvsnet/depthfusion.py \
    --dense_folder /data/scan9 \
    --fusibile_exe_path /opt/fusibile/fusibile \
    --prob_threshold 0.3 \
    --disp_threshold 0.25 \
    --num_consistent 3

# For MVSNet (3DCNNs) use a higher confidence threshold:
python mvsnet/depthfusion.py \
    --dense_folder /data/scan9 \
    --fusibile_exe_path /opt/fusibile/fusibile \
    --prob_threshold 0.8 \
    --disp_threshold 0.25 \
    --num_consistent 3

# Output:
# /data/scan9/points_mvsnet/<timestamp>/final3d_model.ply
```

```python
# Programmatic usage
from depthfusion import probability_filter, mvsnet_to_gipuma, depth_map_fusion
import os

dense_folder = '/data/scan9'
point_folder = os.path.join(dense_folder, 'points_mvsnet')
os.makedirs(point_folder, exist_ok=True)

probability_filter(dense_folder, prob_threshold=0.3)       # creates *_prob_filtered.pfm
mvsnet_to_gipuma(dense_folder, point_folder)               # converts to Gipuma format
depth_map_fusion(point_folder,                             # runs fusibile
                 fusibile_exe_path='/opt/fusibile/fusibile',
                 disp_thresh=0.25,
                 num_consistent=3)
```

--------------------------------

### Visualize Depth Maps

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Visualize depth maps and probability maps generated by MVSNet. This script takes a path to .pfm files as input.

```python
python visualize.py .pfm
```

--------------------------------

### Run MVSNet Inference

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Performs MVSNet inference using a pre-trained model checkpoint. Requires the converted dense folder, regularization type, and checkpoint path.

```python
python mvsnet/test.py \
    --dense_folder /data/myproject/dense \
    --regularization 'GRU' \
    --pretrained_model_ckpt_path /data/tf_model/GRU/model.ckpt \
    --ckpt_step 150000
```

--------------------------------

### Depth Map Refinement

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Use this function to refine an initial depth map using the reference image with a lightweight `RefineNet`. It requires an initial depth map and the corresponding reference image.

```python
from model import depth_refine

# Requires an initial depth map and the reference image
ref_image   = tf.placeholder(tf.float32, [1, 512, 640, 3])
init_depth  = tf.placeholder(tf.float32, [1, 128, 160, 1])
depth_start    = tf.constant([425.0])
depth_interval = tf.constant([2.5])
depth_num      = 128

refined_depth = depth_refine(
    init_depth_map=init_depth,
    image=ref_image,
    depth_num=depth_num,
    depth_start=depth_start,
    depth_interval=depth_interval,
    is_master_gpu=True
)
# refined_depth: same shape as init_depth — [B, H/4, W/4, 1]

with tf.Session(config=config) as sess:
    sess.run(tf.global_variables_initializer())
    refined = sess.run(refined_depth, feed_dict={
        ref_image: image_data,
        init_depth: initial_depth_data
    })
    print(f"Refined depth shape: {refined.shape}")  # (1, 128, 160, 1)
```

--------------------------------

### Train R-MVSNet on ETH3D

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Train the R-MVSNet model on the ETH3D dataset with 'GRU' regularization. Specify the root directories for your training data and desired log folder.

```python
python train.py --regularization 'GRU' --train_eth3d --max_w 896 --max_h 480 --max_d 128
```

--------------------------------

### Train R-MVSNet (GRU regularization) on DTU

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Use this command to train the R-MVSNet model with GRU regularization on the DTU dataset. Specify data roots, image dimensions, logging, and training parameters.

```bash
python mvsnet/train.py \
    --regularization 'GRU' \
    --train_dtu \
    --dtu_data_root /data/dtu \
    --max_w 640 --max_h 512 --max_d 128 \
    --log_folder /data/tf_log \
    --model_folder /data/tf_model
```

--------------------------------

### Run R-MVSNet Testing

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Execute R-MVSNet for depth map generation. This variant uses a GRU regularization. Parameters for resolution and interval scale can be adjusted similarly to MVSNet.

```python
python test.py --dense_folder TEST_DATA_FOLDER  --regularization 'GRU' --max_w 1600 --max_h 1200 --max_d 256 --interval_scale 0.8
```

--------------------------------

### Run R-MVSNet Inference (Higher Resolution)

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Execute depth map inference using a trained R-MVSNet model at higher resolution. Adjust image/depth dimensions, interval scale, and consider using `--inverse_depth` for specific datasets.

```bash
python mvsnet/test.py \
    --dense_folder /data/scan9 \
    --regularization 'GRU' \
    --pretrained_model_ckpt_path /data/tf_model/GRU/model.ckpt \
    --ckpt_step 150000 \
    --max_w 1600 --max_h 1200 --max_d 256 \
    --interval_scale 0.8 \
    --inverse_depth
```

--------------------------------

### Train MVSNet (3D CNN regularization) on BlendedMVS

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Use this command to train the MVSNet model with 3D CNN regularization on the BlendedMVS dataset. Specify data roots, image dimensions, augmentation, logging, and training parameters.

```bash
python mvsnet/train.py \
    --regularization '3DCNNs' \
    --train_blendedmvs \
    --blendedmvs_data_root /data/BlendedMVS/dataset_low_res \
    --max_w 768 --max_h 576 --max_d 128 \
    --online_augmentation \
    --log_folder /data/tf_log \
    --model_folder /data/tf_model \
    --base_lr 0.001 \
    --epoch 6 \
    --snapshot 5000
```

--------------------------------

### Train MVSNet on ETH3D

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Train the MVSNet model using the ETH3D dataset. This command uses '3DCNNs' regularization. Customize data and log folder paths as required.

```python
python train.py --regularization '3DCNNs' --train_eth3d --max_w 896 --max_h 480 --max_d 128
```

--------------------------------

### Train MVSNet on DTU

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Command to train the MVSNet model on the DTU dataset. Requires '3DCNNs' regularization. Adjust data and log folder paths if necessary.

```python
python train.py --regularization '3DCNNs' --train_dtu --max_w 640 --max_h 512 --max_d 128
```

--------------------------------

### Convert COLMAP SfM to MVSNet Input

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Use this script to convert COLMAP SfM results to the R/MVSNet input format. The depth sample number is computed automatically or can be specified.

```python
python colmap2mvsnet.py --dense_folder COLMAP/dense

```

```python
python colmap2mvsnet.py --dense_folder COLMAP/dense --max_d 256

```

--------------------------------

### Convert COLMAP SfM to MVSNet Format

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Converts COLMAP sparse reconstructions into MVSNet's input format, including camera files and pair.txt. It computes depth ranges and selects source views.

```bash
# COLMAP SfM to MVSNet Conversion
python mvsnet/colmap2mvsnet.py \
    --sfm_folder /data/scan9/sfm \
    --output_folder /data/scan9/mvsnet_cameras \
    --dense_folder /data/scan9 \
    --num_src 10
```

--------------------------------

### Train MVSNet on BlendedMVS

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Use this command to train the MVSNet model on the BlendedMVS dataset. Ensure '3DCNNs' regularization is selected. Specify data and log folders as needed.

```python
python train.py --regularization '3DCNNs' --train_blendedmvs --max_w 768 --max_h 576 --max_d 128 --online_augmentation
```

--------------------------------

### Train R-MVSNet on DTU

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Command for training the R-MVSNet model on the DTU dataset. Uses 'GRU' regularization. Ensure correct paths for data and logs are provided.

```python
python train.py --regularization 'GRU' --train_dtu --max_w 640 --max_h 512 --max_d 128
```

--------------------------------

### Load and Write Depth Maps in PFM Format

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Reads and writes depth and probability maps using the PFM (Portable Float Map) format. Outputs are single-channel float32 .pfm files.

```python
from preprocess import load_pfm, write_pfm
import numpy as np

# Load a depth map
with open('/data/scan9/depths_mvsnet/00000000_init.pfm', 'rb') as f:
    depth_map = load_pfm(f)
# depth_map: numpy float32 array of shape (H, W)

print(f"Depth map shape: {depth_map.shape}")   # e.g. (216, 288)
print(f"Valid depths: {(depth_map > 0).sum()}")
print(f"Depth range: {depth_map[depth_map>0].min():.2f} - {depth_map.max():.2f}")

# Apply probability threshold and save
with open('/data/scan9/depths_mvsnet/00000000_prob.pfm', 'rb') as f:
    prob_map = load_pfm(f)

filtered_depth = depth_map.copy()
filtered_depth[prob_map < 0.3] = 0   # zero out low-confidence pixels

write_pfm('/data/scan9/depths_mvsnet/00000000_filtered.pfm',
          filtered_depth.astype(np.float32))
print(f"Retained {(filtered_depth > 0).sum()} valid pixels after filtering.")
```

--------------------------------

### ConvGRUCell for Cost Volume Regularization (TensorFlow)

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Implements a ConvGRUCell using 2D convolutions and group normalization for cost volume regularization in MVSNet. Initializes hidden states to zero and processes cost features slice by slice.

```python
import tensorflow as tf
from convgru import ConvGRUCell

# Input feature maps from the cost volume: [B, H, W, C]
batch_size = 1
height, width = 128, 160
input_channels = 32   # cost feature dimension

# Create a 3-level GRU stack (as used in R-MVSNet)
gru_input_shape = [height, width]
conv_gru1 = ConvGRUCell(shape=gru_input_shape, kernel=[3, 3], filters=16)
conv_gru2 = ConvGRUCell(shape=gru_input_shape, kernel=[3, 3], filters=4)
conv_gru3 = ConvGRUCell(shape=gru_input_shape, kernel=[3, 3], filters=2)

# Initialize hidden states to zero
state1 = tf.zeros([batch_size, height, width, 16])
state2 = tf.zeros([batch_size, height, width, 4])
state3 = tf.zeros([batch_size, height, width, 2])

# Process one depth slice of cost features
cost_slice = tf.placeholder(tf.float32, [batch_size, height, width, input_channels])
reg1, state1 = conv_gru1(-cost_slice, state1, scope='conv_gru1')  # negate cost
reg2, state2 = conv_gru2(reg1, state2, scope='conv_gru2')
reg3, state3 = conv_gru3(reg2, state3, scope='conv_gru3')

# Final 1-channel probability logit for this depth
prob_logit = tf.layers.conv2d(reg3, 1, 3, padding='same', name='prob_conv')
# prob_logit shape: [B, H, W, 1]

# Stack logits from all depth planes then apply softmax
# prob_volume shape after stacking: [B, D, H, W, 1]
```

--------------------------------

### Train R-MVSNet on BlendedMVS

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Train the R-MVSNet model on the BlendedMVS dataset. This command utilizes 'GRU' regularization. Specify your data and log folders.

```python
python train.py --regularization 'GRU' --train_blendedmvs --max_w 768 --max_h 576 --max_d 128 --online_augmentation
```

--------------------------------

### Convert COLMAP to MVSNet Input

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

A utility script to convert COLMAP Structure-from-Motion output into the input format required by R/MVSNet. This is useful for applying the network to custom datasets.

```python
python colmap2mvsnet.py
```

--------------------------------

### Convert COLMAP Dense Output to MVSNet Format

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Converts COLMAP's dense reconstruction output to the format required by MVSNet. Supports auto depth sampling or fixed depth sampling. Use the --test flag for a dry run.

```python
python mvsnet/colmap2mvsnet.py \
    --dense_folder /data/myproject/dense
```

```python
python mvsnet/colmap2mvsnet.py \
    --dense_folder /data/myproject/dense \
    --max_d 256 \
    --interval_scale 1.0
```

```python
python mvsnet/colmap2mvsnet.py \
    --dense_folder /data/myproject/dense \
    --test
```

--------------------------------

### colmap2mvsnet.py — COLMAP SfM to MVSNet Conversion

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Convert a COLMAP Structure-from-Motion reconstruction into the MVSNet input format, including camera files and pair.txt.

```APIDOC
## colmap2mvsnet.py — COLMAP SfM to MVSNet Conversion

**`mvsnet/colmap2mvsnet.py`** — Convert a COLMAP SfM reconstruction into the MVSNet input format (camera files + `pair.txt`).

Reads COLMAP text/binary model files, computes per-image depth ranges from 3D points, selects the 10 best source views per reference image using a baseline-angle scoring function, and writes MVSNet-format camera files.

```bash

```

--------------------------------

### R-MVSNet Depth Inference (GRU)

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Use this function for memory-efficient depth inference with R-MVSNet's GRU. It processes the cost volume depth-by-depth. Note that `depth_end` is required, unlike `inference()`.

```python
from model import inference_winner_take_all

# depth_end is required (unlike inference() which uses depth_interval)
depth_start = tf.constant([425.0])
depth_end   = tf.constant([935.0])
depth_num   = tf.constant(256)

# Inverse depth sampling (recommended for outdoor scenes)
depth_map, prob_map = inference_winner_take_all(
    images, cams,
    depth_num=depth_num,
    depth_start=depth_start,
    depth_end=depth_end,
    is_master_gpu=True,
    reg_type='GRU',
    inverse_depth=True   # linearly sample in inverse-depth space
)
# depth_map: [B, H/4, W/4, 1] — winner-take-all argmax depth
# prob_map:  [B, H/4, W/4, 1] — normalized max probability (confidence)

with tf.Session(config=config) as sess:
    sess.run(tf.global_variables_initializer())
    depth, prob = sess.run(
        [depth_map, prob_map],
        feed_dict={images: img_data, cams: cam_data}
    )
    print(f"Depth map shape: {depth.shape}")       # (1, 300, 400, 1)
    print(f"Confidence > 0.8: {(prob > 0.8).mean()*100:.1f}%")
```

--------------------------------

### Calculate Homographies for Depth Estimation

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Calculates homographies for a range of depths using camera parameters. Requires TensorFlow and specific camera data.

```python
left_cam  = tf.placeholder(tf.float32, [1, 2, 4, 4])
right_cam = tf.placeholder(tf.float32, [1, 2, 4, 4])
depth_num      = 128
depth_start    = tf.constant([425.0])
depth_interval = tf.constant([2.5])

homographies = get_homographies(
    left_cam, right_cam,
    depth_num=depth_num,
    depth_start=depth_start,
    depth_interval=depth_interval
)
# homographies: [B, depth_num, 3, 3]

with tf.Session() as sess:
    H = sess.run(homographies, feed_dict={
        left_cam: ref_cam_data,
        right_cam: src_cam_data
    })
    print(f"Homographies shape: {H.shape}")  # (1, 128, 3, 3)
    print(f"H at depth 0:\n{H[0, 0]}")
```

--------------------------------

### load_pfm() / write_pfm() — Depth Map File I/O

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Read and write depth or probability maps in the PFM (Portable Float Map) format, commonly used for MVSNet outputs.

```APIDOC
## load_pfm() / write_pfm() — Depth Map File I/O

**`mvsnet/preprocess.py`** — Read and write depth/probability maps in the PFM (Portable Float Map) format.

All depth map and probability map outputs are stored as single-channel `.pfm` files. This format stores raw float32 values and is the primary output format of `test.py`.

```python
from preprocess import load_pfm, write_pfm
import numpy as np

# Load a depth map
with open('/data/scan9/depths_mvsnet/00000000_init.pfm', 'rb') as f:
    depth_map = load_pfm(f)
# depth_map: numpy float32 array of shape (H, W)

print(f"Depth map shape: {depth_map.shape}")   # e.g. (216, 288)
print(f"Valid depths: {(depth_map > 0).sum()}")
print(f"Depth range: {depth_map[depth_map>0].min():.2f} - {depth_map.max():.2f}")

# Apply probability threshold and save
with open('/data/scan9/depths_mvsnet/00000000_prob.pfm', 'rb') as f:
    prob_map = load_pfm(f)

filtered_depth = depth_map.copy()
filtered_depth[prob_map < 0.3] = 0   # zero out low-confidence pixels

write_pfm('/data/scan9/depths_mvsnet/00000000_filtered.pfm',
          filtered_depth.astype(np.float32))
print(f"Retained {(filtered_depth > 0).sum()} valid pixels after filtering.")
```
```

--------------------------------

### MVSNet Training Loss Functions (TensorFlow)

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Computes training losses and accuracy metrics for MVSNet (regression) and R-MVSNet (classification). MVSNet uses masked L1 loss, while R-MVSNet uses per-pixel cross-entropy.

```python
import tensorflow as tf
from loss import mvsnet_regression_loss, mvsnet_classification_loss

depth_interval = tf.constant([2.5])   # shape [B]

# --- MVSNet regression loss ---
estimated_depth = tf.placeholder(tf.float32, [1, 128, 160, 1])
gt_depth        = tf.placeholder(tf.float32, [1, 128, 160, 1])  # 0 = invalid pixel

loss, less_one_acc, less_three_acc = mvsnet_regression_loss(
    estimated_depth, gt_depth, depth_interval
)
# loss:           scalar — masked mean absolute error / depth_interval
# less_one_acc:   scalar — fraction of valid pixels with |pred-gt| < 1 interval
# less_three_acc: scalar — fraction of valid pixels with |pred-gt| < 3 intervals

# --- R-MVSNet classification loss ---
depth_num   = 256
depth_start = tf.constant([425.0])
prob_volume = tf.placeholder(tf.float32, [1, depth_num, 128, 160, 1])

cross_entropy, mae, less_one, less_three, wta_depth = mvsnet_classification_loss(
    prob_volume, gt_depth, depth_num, depth_start, depth_interval
)
# cross_entropy: scalar — masked cross-entropy loss
# mae:           scalar — mean absolute error of winner-take-all depth map
# wta_depth:     [B, H, W, 1] — argmax depth map for visualization

with tf.Session() as sess:
    out = sess.run([loss, less_one_acc, less_three_acc], feed_dict={
        estimated_depth: pred_depth_data,
        gt_depth: gt_depth_data
    })
    print(f"Loss: {out[0]:.4f}, <1px: {out[1]:.4f}, <3px: {out[2]:.4f}")
```

--------------------------------

### Depth Range Calculation

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Formula for calculating the maximum depth based on minimum depth, interval, and sample number.

```text
DEPTH_MAX = DEPTH_MIN + (interval_scale * DEPTH_INTERVAL) * (max_d - 1)

```

--------------------------------

### inference_winner_take_all() — R-MVSNet Depth Inference (GRU)

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Memory-efficient depth inference using a recurrent Convolutional GRU to process the cost volume depth-by-depth. Supports both linear and inverse depth sampling.

```APIDOC
## inference_winner_take_all()

### Description
Memory-efficient depth inference using a recurrent Convolutional GRU to process the cost volume depth-by-depth. Supports both linear and inverse depth sampling.

### Method
`inference_winner_take_all(images, cams, depth_num, depth_start, depth_end, is_master_gpu=True, reg_type='GRU', inverse_depth=False)`

### Parameters
- **images** (tf.Tensor) - Input images tensor with shape [B, V, H, W, 3].
- **cams** (tf.Tensor) - Camera intrinsic and extrinsic parameters with shape [B, V, 2, 4, 4].
- **depth_num** (tf.Tensor) - The number of depth hypotheses to consider.
- **depth_start** (tf.Tensor) - Starting depth value for the hypothesis range [B].
- **depth_end** (tf.Tensor) - Ending depth value for the hypothesis range [B].
- **is_master_gpu** (bool, optional) - Whether this is the master GPU. Defaults to True.
- **reg_type** (str, optional) - Regularization type. Defaults to 'GRU'.
- **inverse_depth** (bool, optional) - Whether to sample in inverse-depth space. Defaults to False.

### Returns
- **depth_map** (tf.Tensor) - Winner-take-all argmax depth map with shape [B, H/4, W/4, 1].
- **prob_map** (tf.Tensor) - Normalized maximum probability (confidence) map with shape [B, H/4, W/4, 1].
```

--------------------------------

### Validate R-MVSNet on ETH3D

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Validate the R-MVSNet model on the ETH3D dataset. This command uses 'GRU' regularization. Specify input data folders, pretrained model checkpoint, and the output validation result path.

```python
python validate.py --regularization 'GRU' --validate_set eth3d --max_w 896 --max_h 480 --max_d 128
```

--------------------------------

### Validate MVSNet (3DCNNs) on DTU

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Evaluate a trained MVSNet model on the DTU validation set. Specify the validation set, data root, model parameters, pre-trained checkpoint, and output path for results.

```bash
python mvsnet/validate.py \
    --regularization '3DCNNs' \
    --validate_set dtu \
    --dtu_data_root /data/dtu \
    --max_w 640 --max_h 512 --max_d 128 \
    --pretrained_model_ckpt_path /data/tf_model/3DCNNs/model.ckpt \
    --ckpt_step 150000 \
    --validation_result_path /data/tf_model/validation_results.txt
```

--------------------------------

### Validate R-MVSNet on BlendedMVS

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Evaluate a trained R-MVSNet model on the BlendedMVS validation set. Specify the validation set, data root, model parameters, pre-trained checkpoint, and output path for results.

```bash
python mvsnet/validate.py \
    --regularization 'GRU' \
    --validate_set blendedmvs \
    --blendedmvs_data_root /data/BlendedMVS/dataset_low_res \
    --max_w 768 --max_h 576 --max_d 128 \
    --pretrained_model_ckpt_path /data/tf_model/GRU/model.ckpt \
    --ckpt_step 150000 \
    --validation_result_path /data/tf_model/validation_results.txt
```

--------------------------------

### Validate MVSNet on ETH3D

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Validate the MVSNet model on the ETH3D dataset. This command employs '3DCNNs' regularization. Specify input data folders and the path to the pretrained model checkpoint.

```python
python validate.py --regularization '3DCNNs' --validate_set eth3d --max_w 896 --max_h 480 --max_d 128
```

--------------------------------

### Validate R-MVSNet on DTU

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Command to validate the R-MVSNet model on the DTU dataset. Uses 'GRU' regularization. Ensure correct paths for data, pretrained models, and output results are specified.

```python
python validate.py --regularization 'GRU' --validate_set dtu --max_w 640 --max_h 512 --max_d 128
```

--------------------------------

### probability_filter() / depth_map_fusion() — Post-Processing & Point Cloud Generation

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Filter depth maps by confidence and fuse all views into a dense 3D point cloud using Fusibile.

```APIDOC
## probability_filter() / depth_map_fusion() — Post-Processing & Point Cloud Generation

**`mvsnet/depthfusion.py`** — Filter depth maps by confidence and fuse all views into a dense 3D point cloud using Fusibile.

The pipeline: (1) zero-out low-confidence depth values, (2) convert MVSNet format to Gipuma/Fusibile format, (3) run multi-view depth fusion to produce a `.ply` point cloud.

```bash
# Full post-processing pipeline to generate a point cloud
python mvsnet/depthfusion.py \
    --dense_folder /data/scan9 \
    --fusibile_exe_path /opt/fusibile/fusibile \
    --prob_threshold 0.3 \
    --disp_threshold 0.25 \
    --num_consistent 3

# For MVSNet (3DCNNs) use a higher confidence threshold:
python mvsnet/depthfusion.py \
    --dense_folder /data/scan9 \
    --fusibile_exe_path /opt/fusibile/fusibile \
    --prob_threshold 0.8 \
    --disp_threshold 0.25 \
    --num_consistent 3

# Output:
# /data/scan9/points_mvsnet/<timestamp>/final3d_model.ply

# Programmatic usage
from depthfusion import probability_filter, mvsnet_to_gipuma, depth_map_fusion
import os

dense_folder = '/data/scan9'
point_folder = os.path.join(dense_folder, 'points_mvsnet')
os.makedirs(point_folder, exist_ok=True)

probability_filter(dense_folder, prob_threshold=0.3)       # creates *_prob_filtered.pfm
mvsnet_to_gipuma(dense_folder, point_folder)               # converts to Gipuma format
depth_map_fusion(point_folder,                             # runs fusibile
                 fusibile_exe_path='/opt/fusibile/fusibile',
                 disp_thresh=0.25,
                 num_consistent=3)
```
```

--------------------------------

### Validate R-MVSNet on BlendedMVS

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Validate the R-MVSNet model on the BlendedMVS dataset. This command uses 'GRU' regularization. Provide paths for data, pretrained models, and specify the validation result file.

```python
python validate.py --regularization 'GRU' --validate_set blendedmvs --max_w 768 --max_h 576 --max_d 128
```

--------------------------------

### MVSNet Depth Inference (3D CNN)

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Use this function for depth inference with MVSNet's core 3D CNN cost-volume regularization. Ensure correct shapes for images and camera parameters, and provide depth parameters.

```python
import tensorflow as tf
from model import inference

# Shapes: images [B, V, H, W, 3], cams [B, V, 2, 4, 4]
# depth_start, depth_interval: [B] tensors
images = tf.placeholder(tf.float32, [1, 3, 512, 640, 3])  # batch=1, views=3
cams   = tf.placeholder(tf.float32, [1, 3, 2, 4, 4])
depth_start    = tf.constant([425.0])
depth_interval = tf.constant([2.5])
depth_num      = 128

# Run inference
estimated_depth_map, prob_map = inference(
    images, cams,
    depth_num=depth_num,
    depth_start=depth_start,
    depth_interval=depth_interval,
    is_master_gpu=True
)
# estimated_depth_map: [B, H/4, W/4, 1]  float32 depth values
# prob_map:            [B, H/4, W/4, 1]  per-pixel confidence in [0, 1]

config = tf.ConfigProto()
config.gpu_options.allow_growth = True
with tf.Session(config=config) as sess:
    sess.run(tf.global_variables_initializer())
    depth, prob = sess.run(
        [estimated_depth_map, prob_map],
        feed_dict={images: img_data, cams: cam_data}
    )
    print(f"Depth map shape: {depth.shape}")   # (1, 128, 160, 1)
    print(f"Depth range: {depth.min():.2f} - {depth.max():.2f}")
    print(f"Mean confidence: {prob.mean():.4f}")
```

--------------------------------

### depth_refine() — Depth Map Refinement

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Refine an initial depth map using the reference image with a lightweight residual RefineNet. Normalizes the depth map, concatenates with the image, and passes through conv layers.

```APIDOC
## depth_refine()

### Description
Refine an initial depth map using the reference image with a lightweight residual RefineNet. Normalizes the depth map, concatenates with the image, and passes through conv layers.

### Method
`depth_refine(init_depth_map, image, depth_num, depth_start, depth_interval, is_master_gpu=True)`

### Parameters
- **init_depth_map** (tf.Tensor) - The initial depth map to refine, shape [B, H/4, W/4, 1].
- **image** (tf.Tensor) - The reference image, shape [B, H, W, 3].
- **depth_num** (int) - The number of depth hypotheses used during initial inference.
- **depth_start** (tf.Tensor) - Starting depth value for the hypothesis range [B].
- **depth_interval** (tf.Tensor) - Interval between depth hypotheses [B].
- **is_master_gpu** (bool, optional) - Whether this is the master GPU. Defaults to True.

### Returns
- **refined_depth** (tf.Tensor) - The refined depth map, same shape as `init_depth_map`.
```

--------------------------------

### inference() — MVSNet Depth Inference (3D CNN)

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Core TensorFlow function for depth inference using 3D CNN cost-volume regularization. Extracts features, builds a cost volume, regularizes it, and decodes depth via soft-argmin.

```APIDOC
## inference()

### Description
Core TensorFlow function for depth inference using 3D CNN cost-volume regularization. Extracts features, builds a cost volume, regularizes it, and decodes depth via soft-argmin.

### Method
`inference(images, cams, depth_num, depth_start, depth_interval, is_master_gpu=True)`

### Parameters
- **images** (tf.Tensor) - Input images tensor with shape [B, V, H, W, 3].
- **cams** (tf.Tensor) - Camera intrinsic and extrinsic parameters with shape [B, V, 2, 4, 4].
- **depth_num** (int) - The number of depth hypotheses to consider.
- **depth_start** (tf.Tensor) - Starting depth value for the hypothesis range [B].
- **depth_interval** (tf.Tensor) - Interval between depth hypotheses [B].
- **is_master_gpu** (bool, optional) - Whether this is the master GPU. Defaults to True.

### Returns
- **estimated_depth_map** (tf.Tensor) - Estimated depth map with shape [B, H/4, W/4, 1].
- **prob_map** (tf.Tensor) - Per-pixel confidence map with shape [B, H/4, W/4, 1] in the range [0, 1].
```

--------------------------------

### Validate MVSNet on DTU

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Command to validate the MVSNet model on the DTU dataset. Uses '3DCNNs' regularization. Ensure the correct paths for data and pretrained models are provided.

```python
python validate.py --regularization '3DCNNs' --validate_set dtu --max_w 640 --max_h 512 --max_d 128
```

--------------------------------

### Validate MVSNet on BlendedMVS

Source: https://github.com/yoyo000/mvsnet/blob/master/README.md

Validate the MVSNet model on the BlendedMVS dataset. This command uses '3DCNNs' regularization. Specify data and model checkpoint paths.

```python
python validate.py --regularization '3DCNNs' --validate_set blendedmvs --max_w 768 --max_h 576 --max_d 128
```

--------------------------------

### get_homographies() — Differentiable Homography Computation

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Compute plane-sweep homographies between a reference camera and a source camera across all depth hypotheses. Returns a batch of 3x3 homography matrices.

```APIDOC
## get_homographies()

### Description
Compute plane-sweep homographies between a reference camera and a source camera across all depth hypotheses. Returns a batch of 3x3 homography matrices that warp source-view features to the reference view at each sampled depth plane.

### Method
`get_homographies(ref_cam, src_cams, depth_values)`

### Parameters
- **ref_cam** (tf.Tensor) - Intrinsic and extrinsic parameters of the reference camera.
- **src_cams** (tf.Tensor) - Intrinsic and extrinsic parameters of the source cameras.
- **depth_values** (tf.Tensor) - The depth values for which to compute homographies.

### Returns
- **homographies** (tf.Tensor) - A batch of 3x3 homography matrices.
```

--------------------------------

### Differentiable Homography Computation

Source: https://context7.com/yoyo000/mvsnet/llms.txt

Computes plane-sweep homographies between a reference camera and source cameras for all depth hypotheses. This function is part of the `homography_warping` module.

```python
from homography_warping import get_homographies
```
Tokens:
Raw