### Install Sphinx and Sphinxcontrib-Bibtex

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Install the necessary dependencies for building the documentation.

```bash
pip install sphinx
pip install sphinxcontrib-bibtex
```

--------------------------------

### Install Training Dependencies

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Installs necessary packages for training Co-Tracker. Ensure pip is at version 24.0.

```bash
pip install pip==24.0
pip install pytorch_lightning==1.6.0 tensorboard opencv-python
```

--------------------------------

### Install imageio with FFmpeg support

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Install the imageio library with FFmpeg support, which is required for video processing.

```bash
pip install imageio[ffmpeg]
```

--------------------------------

### Install Development Version of CoTracker

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Installs CoTracker from a cloned GitHub repository and its dependencies. Ensure PyTorch and TorchVision are installed with CUDA support if possible.

```bash
git clone https://github.com/facebookresearch/co-tracker
cd co-tracker
pip install -e .
pip install matplotlib flow_vis tqdm tensorboard
```

--------------------------------

### Install Evaluation Dependencies

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Installs the necessary Python packages for running evaluations. Ensure you have hydra-core and mediapy installed.

```bash
pip install hydra-core==1.1.0 mediapy
```

--------------------------------

### Load and run Online CoTracker

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Loads a pretrained online CoTracker model and demonstrates processing video in chunks for memory efficiency. This example assumes the video length is known.

```python
cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker3_online").to(device)

# Run Online CoTracker, the same model with a different API:
# Initialize online processing
cotracker(video_chunk=video, is_first_step=True, grid_size=grid_size)  

# Process the video
for ind in range(0, video.shape[1] - cotracker.step, cotracker.step):
    pred_tracks, pred_visibility = cotracker(
        video_chunk=video[:, ind : ind + cotracker.step * 2]
    )  # B T N 2,  B T N 1
```

--------------------------------

### Install CoTracker and Dependencies

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Installs CoTracker from source, along with necessary libraries like opencv-python, matplotlib, and moviepy. It also downloads the pre-trained model checkpoint.

```python
!git clone https://github.com/facebookresearch/co-tracker
%cd co-tracker
!pip install -e .
!pip install opencv-python matplotlib moviepy flow_vis
!mkdir checkpoints
%cd checkpoints
!wget https://huggingface.co/facebook/cotracker3/resolve/main/scaled_offline.pth
```

--------------------------------

### Launch Online Model Training on Kubric

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Command to start training the online Co-Tracker model on the Kubric dataset. Modify dataset_root and ckpt_path before running. Assumes 4 nodes with 8 GPUs each.

```bash
python train_on_kubric.py --batch_size 1 --num_steps 50000 \
 --ckpt_path ./ --model_name cotracker_three --save_freq 200 --sequence_len 64 \
  --eval_datasets tapvid_davis_first tapvid_stacking --traj_per_sample 384 \
  --sliding_window_len 16 --train_datasets kubric --save_every_n_epoch 5 \
  --evaluate_every_n_epoch 5 --model_stride 4 --dataset_root ${path_to_your_dataset} \
   --num_nodes 4 --num_virtual_tracks 64 --mixed_precision --corr_radius 3 \
   --wdecay 0.0005 --linear_layer_for_vis_conf --validate_at_start --add_huber_loss
```

--------------------------------

### Launch Offline Model Training on Kubric

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Command to start training the offline Co-Tracker model on the Kubric dataset. Adjust dataset_root and ckpt_path. Requires 4 nodes with 8 GPUs each.

```bash
python train_on_kubric.py --batch_size 1 --num_steps 50000 \
 --ckpt_path ./ --model_name cotracker_three --save_freq 200 --sequence_len 60 \
 --eval_datasets tapvid_davis_first tapvid_stacking --traj_per_sample 512 \
 --sliding_window_len 60 --train_datasets kubric --save_every_n_epoch 5 \
 --evaluate_every_n_epoch 5 --model_stride 4 --dataset_root ${path_to_your_dataset} \
 --num_nodes 4 --num_virtual_tracks 64 --mixed_precision --offline_model \
 --random_frame_rate --query_sampling_method random --corr_radius 3 \
 --wdecay 0.0005 --random_seq_len --linear_layer_for_vis_conf \
 --validate_at_start --add_huber_loss
```

--------------------------------

### Forward Tracking from Grid

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Performs forward tracking starting from points sampled on a regular grid in a specified frame. Requires the video and model to be initialized.

```python
pred_tracks, pred_visibility = model(video, grid_size=grid_size, grid_query_frame=grid_query_frame)
```

--------------------------------

### Visualize predicted tracks

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Visualizes the predicted tracks and visibility using the Visualizer utility. Ensure CoTracker is installed and specify a save directory for the output video.

```python
from cotracker.utils.visualizer import Visualizer

vis = Visualizer(save_dir="./saved_videos", pad_value=120, linewidth=3)
vis.visualize(video, pred_tracks, pred_visibility)
```

--------------------------------

### Visualize Predicted Tracks

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Visualizes the predicted tracks from the CoTracker model. The visualization saves a video file and uses color to encode time. Tracks leave traces from the frame where tracking starts.

```python
vis = Visualizer(
    save_dir='./videos',
    linewidth=6,
    mode='cool',
    tracks_leave_trace=-1
)
vis.visualize(
    video=video,
    tracks=pred_tracks,
    visibility=pred_visibility,
    filename='queries')
```

--------------------------------

### Generate HTML Documentation

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Use the make command to generate the HTML documentation in the specified folder.

```bash
make -C docs html
```

--------------------------------

### Run online demo script

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Executes the online demo script. A GPU is recommended for optimal performance.

```bash
python online_demo.py
```

--------------------------------

### Run Gradio Demo Locally

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Instructions on how to run the Gradio demo for CoTracker locally. This allows for interactive testing of the model's capabilities.

```python
cd gradio_demo/
python app.py
```

--------------------------------

### Run offline demo script

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Executes the offline demo script with a specified grid size. Results will be saved to './saved_videos/demo.mp4'. A GPU is recommended.

```bash
python demo.py --grid_size 10
```

--------------------------------

### Initialize Online CoTracker v2

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Load the online version of CoTracker v2 and initialize it for processing.

```python
cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker2_online").to(device)

# Run Online CoTracker, the same model with a different API:
# Initialize online processing
cotracker(video_chunk=video, is_first_step=True, grid_size=grid_size)
```

--------------------------------

### Visualize and Save Tracking Results

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Initializes a Visualizer object to save results to a specified directory. It then visualizes the predicted tracks and visibility, saving the output as a video file named 'teaser'.

```python
vis = Visualizer(save_dir='./videos', pad_value=100)
vis.visualize(video=video, tracks=pred_tracks, visibility=pred_visibility, filename='teaser');
```

--------------------------------

### Download Baseline CoTracker3 Checkpoints

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Downloads pre-trained checkpoints for the baseline CoTracker3 models (online and offline), trained on Kubric. Place these in the 'checkpoints' directory.

```bash
# download the online (sliding window) model
wget https://huggingface.co/facebook/cotracker3/resolve/main/baseline_online.pth
# download the offline (single window) model
wget https://huggingface.co/facebook/cotracker3/resolve/main/baseline_offline.pth
```

--------------------------------

### Download CoTracker v1 Checkpoints

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Download pre-trained checkpoint files for different configurations of CoTracker v1.

```bash
wget https://dl.fbaipublicfiles.com/cotracker/cotracker_stride_4_wind_8.pth
wget https://dl.fbaipublicfiles.com/cotracker/cotracker_stride_4_wind_12.pth
wget https://dl.fbaipublicfiles.com/cotracker/cotracker_stride_8_wind_16.pth
```

--------------------------------

### Download CoTracker v2 Checkpoint

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Download the pre-trained checkpoint file for CoTracker v2.

```bash
wget https://huggingface.co/facebook/cotracker/resolve/main/cotracker2.pth
```

--------------------------------

### Download Scaled CoTracker3 Checkpoints

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Downloads pre-trained checkpoints for the scaled CoTracker3 models (online and offline). Place these in the 'checkpoints' directory.

```bash
mkdir -p checkpoints
cd checkpoints
# download the online (multi window) model
wget https://huggingface.co/facebook/cotracker3/resolve/main/scaled_online.pth
# download the offline (single window) model
wget https://huggingface.co/facebook/cotracker3/resolve/main/scaled_offline.pth
cd ..
```

--------------------------------

### Initialize Grid Size and Query Frame

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Sets the parameters for grid sampling and the frame from which tracking originates.

```python
grid_size = 30
grid_query_frame = 20
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Imports core libraries for video processing, PyTorch, and visualization utilities. This sets up the environment for subsequent operations.

```python
%cd ..
import os
import torch

from base64 import b64encode
from cotracker.utils.visualizer import Visualizer, read_video_from_path
from IPython.display import HTML
```

--------------------------------

### Load and run Offline CoTracker

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Loads a pretrained offline CoTracker model from torch.hub and processes a video. Ensure you have a CUDA-enabled GPU for optimal performance.

```python
import torch
# Download the video
url = 'https://github.com/facebookresearch/co-tracker/raw/refs/heads/main/assets/apple.mp4'

import imageio.v3 as iio
frames = iio.imread(url, plugin="FFMPEG")  # plugin="pyav"

device = 'cuda'
grid_size = 10
video = torch.tensor(frames).permute(0, 3, 1, 2)[None].float().to(device)  # B T C H W

# Run Offline CoTracker:
cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker3_offline").to(device)
pred_tracks, pred_visibility = cotracker(video, grid_size=grid_size) # B T N 2,  B T N 1
```

--------------------------------

### Download Video for Offline Mode

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Download a sample video using imageio for offline CoTracker processing.

```python
import torch
# Download the video
url = 'https://github.com/facebookresearch/co-tracker/blob/main/assets/apple.mp4'

import imageio.v3 as iio
frames = iio.imread(url, plugin="FFMPEG")  # plugin="pyav"

device = 'cuda'
grid_size = 10
video = torch.tensor(frames).permute(0, 3, 1, 2)[None].float().to(device)  # B T C H W
```

--------------------------------

### Import Libraries and Set Grid Size

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Imports necessary libraries like numpy and PIL, and sets a grid size for point sampling.

```python
import numpy as np
from PIL import Image
grid_size = 100
```

--------------------------------

### Load Video for Tracking

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Reads a video file from a specified path and prepares it for model input by converting it to a PyTorch tensor with the correct dimensions and data type.

```python
video = read_video_from_path('./assets/apple.mp4')
video = torch.from_numpy(video).permute(0, 3, 1, 2)[None].float()
```

--------------------------------

### Load CoTracker v1 via PyTorch Hub

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Load the CoTracker v1 model directly using torch.hub.

```python
import torch
import einops
import timm
import tqdm

cotracker = torch.hub.load("facebookresearch/co-tracker:v1.0", "cotracker_w8")
```

--------------------------------

### Fine-tune Offline Model with Pseudo Labels

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Command to fine-tune the offline Co-Tracker model using pseudo-labeled real-world videos. Requires a pre-trained Kubric model. Adjust dataset_root and restore_ckpt paths.

```bash
python train_on_real_data.py --batch_size 1 --num_steps 15000 \
 --ckpt_path ./ --model_name cotracker_three --save_freq 200 --sequence_len 80 \
 --eval_datasets tapvid_stacking tapvid_davis_first --traj_per_sample 384 --save_every_n_epoch 15 \
 --evaluate_every_n_epoch 15 --model_stride 4 --dataset_root ${path_to_your_dataset} \
  --num_nodes 4 --real_data_splits 0 --num_virtual_tracks 64 --mixed_precision \
  --random_frame_rate --restore_ckpt ./checkpoints/baseline_offline.pth --lr 0.00005 \
  --real_data_filter_sift --validate_at_start --offline_model --limit_samples 15000
```

--------------------------------

### Fine-tune Online Model with Pseudo Labels

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Command to fine-tune the online Co-Tracker model using pseudo-labeled real-world videos. Requires a pre-trained Kubric model. Adjust dataset_root and restore_ckpt paths.

```bash
python ./train_on_real_data.py --batch_size 1 --num_steps 15000 \
 --ckpt_path ./ --model_name cotracker_three --save_freq 200 --sequence_len 64 \
 --eval_datasets tapvid_stacking tapvid_davis_first --traj_per_sample 384 \
 --save_every_n_epoch 15 --evaluate_every_n_epoch 15 --model_stride 4 \
 --dataset_root ${path_to_your_dataset} --num_nodes 4 --real_data_splits 0 \
 --num_virtual_tracks 64 --mixed_precision --random_frame_rate \
 --restore_ckpt ./checkpoints/baseline_online.pth \
 --lr 0.00005 --real_data_filter_sift --validate_at_start \
 --sliding_window_len 16 --limit_samples 15000
```

--------------------------------

### Evaluate Online Model on TAP-Vid DAVIS

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Command to evaluate the online CoTracker model on the TAP-Vid DAVIS dataset. Replace 'your/tapvid/path' with the actual dataset directory.

```bash
python ./cotracker/evaluation/evaluate.py --config-name eval_tapvid_davis_first exp_dir=./eval_outputs dataset_root=your/tapvid/path
```

--------------------------------

### Visualize Forward Tracking Results

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Visualizes the results of the forward tracking, saving the video output. Ensure the Visualizer is configured with a save directory.

```python
vis = Visualizer(save_dir='./videos', pad_value=100)
vis.visualize(
    video=video,
    tracks=pred_tracks,
    visibility=pred_visibility,
    filename='grid_query_20',
    query_frame=grid_query_frame);
```

--------------------------------

### Perform Point Tracking

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Executes the CoTracker model to predict tracks and visibility for points sampled on a regular grid (30x30) on the first frame of the video.

```python
pred_tracks, pred_visibility = model(video, grid_size=30)
```

--------------------------------

### Perform Dense Tracking

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Executes the CoTracker model for dense tracking, enabling backward tracking.

```python
pred_tracks, pred_visibility = model(video_interp, grid_query_frame=grid_query_frame, backward_tracking=True)
```

--------------------------------

### Display Saved Tracking Video

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Displays the generated 'teaser.mp4' video file, which contains the visualization of the tracked points, within the notebook output.

```python
show_video("./videos/teaser.mp4")
```

--------------------------------

### Initialize CoTracker Predictor

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Initializes the CoTrackerPredictor model, loading the pre-trained weights from the specified checkpoint path. This object is used for performing the actual tracking.

```python
from cotracker.predictor import CoTrackerPredictor

model = CoTrackerPredictor(
    checkpoint=os.path.join(
        './checkpoints/scaled_offline.pth'
    )
)
```

--------------------------------

### Run Offline CoTracker v2

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Load and run the CoTracker v2 model in offline mode for video processing.

```python
# Run Offline CoTracker:
cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker2").to(device)
pred_tracks, pred_visibility = cotracker(video, grid_size=grid_size) # B T N 2,  B T N 1
```

--------------------------------

### Move Model and Video to GPU

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Checks if a CUDA-enabled GPU is available and moves the CoTracker model and the input video tensor to the GPU for accelerated processing.

```python
if torch.cuda.is_available():
    model = model.cuda()
    video = video.cuda()
```

--------------------------------

### Bidirectional Tracking from Grid

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Performs tracking both forward and backward from points sampled on a regular grid. This enables tracking from the middle of the video outwards.

```python
pred_tracks, pred_visibility = model(video, grid_size=grid_size, grid_query_frame=grid_query_frame, backward_tracking=True)
vis.visualize(
    video=video,
    tracks=pred_tracks,
    visibility=pred_visibility,
    filename='grid_query_20_backward');
```

--------------------------------

### Track Points Forward in Time

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Passes the video and query points to the CoTracker model to predict tracks. The model tracks points in the forward direction only by default.

```python
pred_tracks, pred_visibility = model(video, queries=queries[None])
```

--------------------------------

### Evaluate Offline Model on TAP-Vid DAVIS

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Command to evaluate the offline CoTracker model on the TAP-Vid DAVIS dataset. This command specifies the dataset root, enables offline model evaluation, sets the window length, and points to the checkpoint file.

```bash
python ./cotracker/evaluation/evaluate.py --config-name eval_tapvid_davis_first exp_dir=./eval_outputs dataset_root=/fsx-repligen/shared/datasets/tapvid offline_model=True window_len=60 checkpoint=./checkpoints/scaled_offline.pth
```

--------------------------------

### Display Dense Tracking Video

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Shows the generated video of the dense tracking results.

```python
show_video("./videos/dense.mp4")
```

--------------------------------

### Process Video in Online Mode

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

Iterate through video chunks and process them using the online CoTracker v2 API.

```python
# Process the video
for ind in range(0, video.shape[1] - cotracker.step, cotracker.step):
    pred_tracks, pred_visibility = cotracker(
        video_chunk=video[:, ind : ind + cotracker.step * 2]
    )  # B T N 2,  B T N 1
```

--------------------------------

### Display Forward Tracking Video

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Displays the saved video of the forward tracking results. Assumes the video has been saved to the specified path.

```python
show_video("./videos/grid_query_20.mp4")
```

--------------------------------

### Visualize Dense Tracking with Optical Flow

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Visualizes the dense tracking results using an optical flow color encoding and saves the video.

```python
vis = Visualizer(
    save_dir='./videos',
    pad_value=20,
    linewidth=1,
    mode='optical_flow'
)
vis.visualize(
    video=video_interp,
    tracks=pred_tracks,
    visibility=pred_visibility,
    filename='dense');
```

--------------------------------

### Display Tracking Video

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Shows the generated video of the tracking results.

```python
show_video("./videos/segm_grid.mp4")
```

--------------------------------

### Display Video in Notebook

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Defines a utility function to display MP4 videos within the Jupyter Notebook environment using HTML5 video tags. It encodes the video file in base64.

```python
def show_video(video_path):
    video_file = open(video_path, "r+b").read()
    video_url = f"data:video/mp4;base64,{b64encode(video_file).decode()}"
    return HTML(f"<video width=\"640\" height=\"480\" autoplay loop controls><source src=\"{video_url}\"></video>")
 
show_video("./assets/apple.mp4")
```

--------------------------------

### Track Points with Segmentation Mask

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Uses the CoTracker model to predict tracks and visibility based on the video and a segmentation mask. It then visualizes these tracks.

```python
pred_tracks, pred_visibility = model(video, grid_size=grid_size, segm_mask=torch.from_numpy(segm_mask)[None, None])
vis = Visualizer(
    save_dir='./videos',
    pad_value=100,
    linewidth=2,
)
vis.visualize(
    video=video,
    tracks=pred_tracks,
    visibility=pred_visibility,
    filename='segm_grid');
```

--------------------------------

### Visualize Query Points on Video Frames

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Visualizes the defined query points on their respective video frames using Matplotlib. Each subplot shows a frame with a red dot indicating the query point.

```python
import matplotlib.pyplot as plt
# Create a list of frame numbers corresponding to each point
frame_numbers = queries[:,0].int().tolist()

fig, axs = plt.subplots(2, 2)
axs = axs.flatten()

for i, (query, frame_number) in enumerate(zip(queries, frame_numbers)):
    ax = axs[i]
    ax.plot(query[1].item(), query[2].item(), 'ro') 
    
    ax.set_title("Frame {}".format(frame_number))
    ax.set_xlim(0, video.shape[4])
    ax.set_ylim(0, video.shape[3])
    ax.invert_yaxis()
    
plt.tight_layout()
plt.show()
```

--------------------------------

### Visualize Segmentation Mask

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Displays the segmentation mask overlaid on the first frame of the video.

```python
plt.imshow((segm_mask[...,None]/255.*video[0,0].permute(1,2,0).cpu().numpy()/255.))
```

--------------------------------

### Display Bidirectionally Tracked Video

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Displays the video generated from bidirectional tracking. This allows for visual verification of the corrected tracking results.

```python
show_video("./videos/queries_backward.mp4")
```

--------------------------------

### Display Tracked Video

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Displays the generated video of the tracked points. This function is used to preview the output of the visualization.

```python
show_video("./videos/queries.mp4")
```

--------------------------------

### Display Bidirectional Tracking Video

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Displays the saved video of the bidirectional tracking results. Assumes the video has been saved to the specified path.

```python
show_video("./videos/grid_query_20_backward.mp4")
```

--------------------------------

### Define Query Points for Tracking

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Defines a tensor of query points, each represented as [time, x_coordinate, y_coordinate]. Supports CUDA if available.

```python
queries = torch.tensor([
    [0., 400., 350.],  # point tracked from the first frame
    [10., 600., 500.], # frame number 10
    [20., 750., 600.], # ...
    [30., 900., 200.]
])
if torch.cuda.is_available():
    queries = queries.cuda()
```

--------------------------------

### Set Grid Query Frame for Dense Tracking

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Specifies the frame number to be used as the query frame for dense tracking.

```python
grid_query_frame=20
```

--------------------------------

### Load Segmentation Mask

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Loads a segmentation mask image and converts it into a numpy array for use with the model.

```python
input_mask = './assets/apple_mask.png'
segm_mask = np.array(Image.open(input_mask))
```

--------------------------------

### Downsample Video for Dense Tracking

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Downsamples the input video using bilinear interpolation to reduce resolution for faster dense tracking.

```python
import torch.nn.functional as F
video_interp = F.interpolate(video[0], [200, 360], mode="bilinear")[None]
```

--------------------------------

### Track Points Bidirectionally

Source: https://github.com/facebookresearch/co-tracker/blob/main/notebooks/demo.ipynb

Tracks points in both forward and backward directions by setting the `backward_tracking` argument to True. This corrects incorrect tracking before the query frame.

```python
pred_tracks, pred_visibility = model(video, queries=queries[None], backward_tracking=True)
vis.visualize(
    video=video,
    tracks=pred_tracks,
    visibility=pred_visibility,
    filename='queries_backward')
```

--------------------------------

### CoTracker BibTeX Citation

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

BibTeX entry for citing the primary CoTracker paper. Use this when referencing the main CoTracker research.

```bibtex
@inproceedings{karaev23cotracker,
  title     = {CoTracker: It is Better to Track Together},
  author    = {Nikita Karaev and Ignacio Rocco and Benjamin Graham and Natalia Neverova and Andrea Vedaldi and Christian Rupprecht},
  booktitle = {Proc. {ECCV}},
  year      = {2024}
}
```

--------------------------------

### CoTracker3 BibTeX Citation

Source: https://github.com/facebookresearch/co-tracker/blob/main/README.md

BibTeX entry for citing the CoTracker3 paper. Use this when referencing the CoTracker3 research, which focuses on simpler and better point tracking.

```bibtex
@inproceedings{karaev24cotracker3,
  title     = {CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos},
  author    = {Nikita Karaev and Iurii Makarov and Jianyuan Wang and Natalia Neverova and Andrea Vedaldi and Christian Rupprecht},
  booktitle = {Proc. {arXiv:2410.11831}},
  year      = {2024}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.