ColorMNet

https://github.com/yyang181/colormnet
Admin
ColorMNet is a memory-based deep spatial-temporal feature propagation network for exemplar-based...
Tokens:60,075
Snippets:366
Trust Score:7
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### Install ninja-build

Source: https://github.com/yyang181/colormnet/blob/main/inference/interact/fbrs/model/syncbn/README.md

Install ninja-build, required by PyTorch JIT to compile the SyncBN extension seamlessly.

```bash
sudo apt-get install ninja-build
```

--------------------------------

### Install Python dependencies

Source: https://github.com/yyang181/colormnet/blob/main/inference/interact/fbrs/model/syncbn/README.md

Install all Python dependencies for the project.

```bash
pip install -U -r requirements.txt
```

--------------------------------

### Launch Gradio demo with app.py

Source: https://github.com/yyang181/colormnet/blob/main/README.md

Starts the Gradio demo application on the default GPU. Use this command to run the interactive demo after setting CUDA_VISIBLE_DEVICES as needed.

```bash
CUDA_VISIBLE_DEVICES=0 python app.py
```

--------------------------------

### Install ColorMNet dependencies and environment

Source: https://github.com/yyang181/colormnet/blob/main/README.md

Sets up a conda environment named 'colormnet' with Python 3.10, installs PyTorch 2.0.1 with CUDA 11.8, and builds the required dependencies: py-thin-plate-spline and Pytorch-Correlation-extension. Run these commands in a terminal after cloning the repository. Requires a CUDA-capable NVIDIA GPU.

```shell
# git clone this repository

conda create -n colormnet python=3.10 -y
conda activate colormnet 

pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --index-url https://download.pytorch.org/whl/cu118

# install py-thin-plate-spline
git clone https://github.com/cheind/py-thin-plate-spline.git
cd py-thin-plate-spline && pip install -e . && cd ..

# install Pytorch-Correlation-extension
git clone https://github.com/ClementPinard/Pytorch-Correlation-extension.git 
cd Pytorch-Correlation-extension && python setup.py install && cd ..

pip install -r requirements.txt
```

--------------------------------

### Train ColorMNet with torch.distributed.run

Source: https://github.com/yyang181/colormnet/blob/main/README.md

Launches distributed training with one GPU. Set --davis_root and --validation_root to your data paths, and adjust --savepath to your desired output directory. The command uses master port 25205 and one process per node.

```bash
CUDA_VISIBLE_DEVICES=0 python -m torch.distributed.run \
    --master_port 25205 \
    --nproc_per_node=1 \
    train.py \
    --exp_id DINOv2FeatureV6_LocalAtten_DAVISVidevo \
    --davis_root /path/to/your/training/data/\
    --validation_root /path/to/your/validation/data\
    --savepath ./wandb_save_dir
```

--------------------------------

### Dataset directory structure for training and validation

Source: https://github.com/yyang181/colormnet/blob/main/README.md

Shows the expected folder layout for the training and validation sets. Each sequence folder (e.g., 001, 002) contains PNG frames named sequentially (00000.png, 00001.png, ...). Specify --davis_root and --validation_root when training.

```text
# Specify --davis_root and --validation_root
data_root/
├── 001/
│   ├── 00000.png
│   ├── 00001.png
│   ├── 00002.png
│   └── ...
├── 002/
│   ├── 00000.png
│   ├── 00001.png
│   ├── 00002.png
│   └── ...
└── ...
```

--------------------------------

### Run inference on images with test.py

Source: https://github.com/yyang181/colormnet/blob/main/README.md

Runs the test script on a GPU. Add --FirstFrameIsNotExemplar if the reference frame is not exactly the first input image; ensure the reference frame and input frames are the same size. Windows users should follow the linked RuntimeError workaround to avoid multiprocessor data loader errors.

```bash
CUDA_VISIBLE_DEVICES=0 python test.py 
# Add --FirstFrameIsNotExemplar if the reference frame is not exactly the first input image. Please make sure the ref frame and the input frames are of the same size. 
```

--------------------------------

### Test custom BatchNorm2d with DataParallel

Source: https://github.com/yyang181/colormnet/blob/main/inference/interact/fbrs/model/syncbn/README.md

Creates a model with custom BatchNorm2d layers, wraps it in DataParallel, and runs a forward pass on random input. Requires multiple GPUs (num_gpu) and CUDA.

```python
import torch
from modules import nn as NN
num_gpu = torch.cuda.device_count()
model = nn.Sequential(
    nn.Conv2d(3, 3, 1, 1, bias=False),
    NN.BatchNorm2d(3),
    nn.ReLU(inplace=True),
    nn.Conv2d(3, 3, 1, 1, bias=False),
    NN.BatchNorm2d(3),
).cuda()
model = nn.DataParallel(model, device_ids=range(num_gpu))
x = torch.rand(num_gpu, 3, 2, 2).cuda()
z = model(x)
```
Tokens:
Raw