VideoFlow

https://github.com/xiaoyushi97/videoflow
Admin
VideoFlow is a project for multi-frame optical flow estimation that exploits temporal cues. It...
Tokens:1,604
Snippets:22
Trust Score:7.1
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install VideoFlow Dependencies

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Use Conda to create an environment and install PyTorch, torchvision, and other necessary Python packages for VideoFlow.

```shell
conda create --name videoflow
conda activate videoflow
conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.1 matplotlib tensorboard scipy opencv-python -c pytorch
pip install yacs loguru einops timm==0.4.12 imageio
```

--------------------------------

### Compile and Install Efficient RAFT CUDA Extension

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Compile and install the alternative CUDA extension for RAFT. This implementation uses less GPU memory but is slower and does not support backward computation, making it suitable only for inference.

```shell
cd alt_cuda_corr && python setup.py install && cd ..

```

--------------------------------

### Run Inference with VideoFlow (MOF Model)

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Execute the inference script using the MOF model for optical flow estimation. Specify the directory containing input images and the directory for visualization output.

```shell
python -u inference.py --mode MOF --seq_dir demo_input_images --vis_dir demo_flow_vis
```

--------------------------------

### Run Inference with VideoFlow (BOF Model)

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Execute the inference script using the BOF model, recommended for inputs with only three frames. Specify the input image directory and the visualization output directory.

```shell
python -u inference.py --mode BOF --seq_dir demo_input_images_three_frames --vis_dir demo_flow_vis_three_frames
```

--------------------------------

### Evaluate MOFNet Models

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Run these commands to evaluate MOFNet models on different datasets (Sintel, Things, KITTI) and for submission benchmarks. Ensure the correct checkpoints are loaded via configuration files.

```shell
python -u evaluate_MOFNet.py --dataset=sintel
python -u evaluate_MOFNet.py --dataset=things
python -u evaluate_MOFNet.py --dataset=kitti
python -u evaluate_MOFNet.py --dataset=sintel_submission_stride1
python -u evaluate_MOFNet.py --dataset=kitti_submission
```

--------------------------------

### Evaluate BOFNet Models

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Execute these commands to evaluate BOFNet models on various datasets and for submission benchmarks. Configuration files should point to the appropriate checkpoints.

```shell
python -u evaluate_BOFNet.py --dataset=sintel
python -u evaluate_BOFNet.py --dataset=things
python -u evaluate_BOFNet.py --dataset=kitti
python -u evaluate_BOFNet.py --dataset=sintel_submission
python -u evaluate_BOFNet.py --dataset=kitti_submission
```

--------------------------------

### Train MOFNet Model

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Use these commands to train the MOFNet model for different stages (things, sintel, kitti). The --name flag specifies the experiment name, and --stage defines the training dataset. --validation is set to sintel.

```shell
python -u train_MOFNet.py --name MOF-things --stage things --validation sintel
```

```shell
python -u train_MOFNet.py --name MOF-sintel --stage sintel --validation sintel
```

```shell
python -u train_MOFNet.py --name MOF-kitti --stage kitti --validation sintel
```

--------------------------------

### Train BOFNet Model

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

Use these commands to train the BOFNet model for different stages (things, sintel, kitti). The --name flag specifies the experiment name, and --stage defines the training dataset. --validation is set to sintel.

```shell
python -u train_BOFNet.py --name BOF-things --stage things --validation sintel
```

```shell
python -u train_BOFNet.py --name BOF-sintel --stage sintel --validation sintel
```

```shell
python -u train_BOFNet.py --name BOF-kitti --stage kitti --validation sintel
```

--------------------------------

### Dataset Directory Structure

Source: https://github.com/xiaoyushi97/videoflow/blob/main/README.md

This is the expected directory structure for datasets used by VideoFlow. Ensure your downloaded datasets are organized accordingly or create symbolic links in the 'datasets' folder.

```shell
├── datasets
    ├── Sintel
        ├── test
        ├── training
    ├── KITTI
        ├── testing
        ├── training
        ├── devkit
    ├── FlyingChairs_release
        ├── data
    ├── FlyingThings3D
        ├── frames_cleanpass
        ├── frames_finalpass
        ├── optical_flow
```
Tokens:
Raw