### Install PyTorchVideo

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Installs PyTorchVideo from a local clone of the repository. Ensure you are in the PyTorchVideo directory.

```bash
cd pytorchvideo
python -m pip install -e .
```

--------------------------------

### Install PyTorchVideo from Local Clone (Development)

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Install PyTorchVideo from a local clone with development and testing dependencies included.

```bash
git clone https://github.com/facebookresearch/pytorchvideo.git
cd pytorchvideo
pip install -e . [test,dev]
```

--------------------------------

### Install PyTorchVideo from Local Clone

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Install PyTorchVideo from a local clone of the GitHub repository. This is useful for development.

```bash
git clone https://github.com/facebookresearch/pytorchvideo.git
cd pytorchvideo
pip install -e .
```

--------------------------------

### Install PyTorchVideo from GitHub (Stable Branch)

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Install PyTorchVideo from the 'stable' branch of the GitHub repository.

```bash
pip install "git+https://github.com/facebookresearch/pytorchvideo.git@stable"
```

--------------------------------

### Install PyTorchVideo

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_Model_Transmuter.ipynb

Install the PyTorchVideo library using pip. This is a prerequisite for using the model transmuter.

```python
!pip install pytorchvideo
```

--------------------------------

### Install PyTorchVideo Trainer with Development Dependencies

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/pytorchvideo_trainer/README.md

Install the PyTorchVideo Trainer with additional dependencies for development and testing. This is recommended for users contributing to the project.

```bash
pip install -e . [test,dev]
```

--------------------------------

### Install and Import Core Modules

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/video_detection_example/video_detection_inference_tutorial.ipynb

Installs necessary libraries like torch, torchvision, opencv-python, and pytorchvideo if they are not already present. It includes conditional installation for pytorchvideo from GitHub if needed.

```python
try:
    import torch
except ModuleNotFoundError:
    !pip install torch torchvision
    import os
    import sys
    import torch

try:
    import cv2
except ModuleNotFoundError:
    !pip install opencv-python
    
if torch.__version__=='1.6.0+cu101' and sys.platform.startswith('linux'):
    !pip install pytorchvideo
else:
    need_pytorchvideo=False
    try:
        # Running notebook locally
        import pytorchvideo
    except ModuleNotFoundError:
        need_pytorchvideo=True
    if need_pytorchvideo:
        # Install from GitHub
        !pip install "git+https://github.com/facebookresearch/pytorchvideo.git"
```

--------------------------------

### Install PyTorchVideo Trainer

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/pytorchvideo_trainer/README.md

Clone the PyTorchVideo repository and install the PyTorchVideo Trainer in editable mode. This allows for local development and testing.

```bash
git clone https://github.com/facebookresearch/pytorchvideo.git
cd pytorchvideo/pytorchvideo_trainer
pip install -e .
```

--------------------------------

### Load Example Video File

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/torchhub_inference_tutorial.ipynb

Specifies the file path for the downloaded example video, preparing it for loading and processing.

```python
# Load the example video
video_path = "archery.mp4"  
```

--------------------------------

### Install Documentation Dependencies

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/README.md

Installs the required Python packages for building the documentation using pip.

```bash
pip install -U recommonmark mock sphinx sphinx_rtd_theme sphinx_markdown_tables
```

--------------------------------

### Setup Conda Environment for PyTorch and Torchvision

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Instructions to set up a conda environment with PyTorch and Torchvision, a prerequisite for installing PyTorchVideo.

```bash
conda create -n pytorchvideo python=3.7
conda activate pytorchvideo
conda install -c pytorch pytorch=1.8.0 torchvision cudatoolkit=10.2
```

--------------------------------

### Install Recipes Dependency

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/pytorchvideo_trainer/README.md

Install the required 'recipes' dependency using pip. This is a prerequisite for installing PyTorchVideo Trainer.

```bash
pip install "git+https://github.com/facebookresearch/recipes.git"
```

--------------------------------

### Install PyTorchVideo from GitHub using pip

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Install PyTorchVideo directly from its GitHub repository using pip.

```bash
pip install "git+https://github.com/facebookresearch/pytorchvideo.git"
```

--------------------------------

### Install PyTorchVideo from PyPI (Nightly)

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Install the most recent nightly build of PyTorchVideo for the latest features and fixes.

```bash
pip install pytorchvideo-nightly
```

--------------------------------

### Install PyTorchVideo and Dependencies

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/torchhub_inference_tutorial.ipynb

Installs PyTorch, torchvision, and PyTorchVideo. Includes conditional installation for specific PyTorch versions and local environments.

```python
try:
    import torch
except ModuleNotFoundError:
    !pip install torch torchvision
    import os
    import sys
    import torch
    
if torch.__version__=='1.6.0+cu101' and sys.platform.startswith('linux'):
    !pip install pytorchvideo
else:
    need_pytorchvideo=False
    try:
        # Running notebook locally
        import pytorchvideo
    except ModuleNotFoundError:
        need_pytorchvideo=True
    if need_pytorchvideo:
        # Install from GitHub
        !pip install "git+https://github.com/facebookresearch/pytorchvideo.git"
```

--------------------------------

### Serve Documentation Locally

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/README.md

Starts a simple Python HTTP server to view the generated HTML documentation locally. Navigate to http://0.0.0.0:8000/ in your browser.

```bash
> python -m http.server
```

--------------------------------

### Set up Conda Environment for PyTorchVideo

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Recommended steps to create and activate a conda environment, installing PyTorch, torchvision, and necessary CUDA toolkits.

```bash
conda create -n pytorchvideo python=3.7
conda activate pytorchvideo
conda install -c pytorch pytorch=1.8.0 torchvision cudatoolkit=10.2
conda install -c conda-forge -c fvcore -c iopath fvcore=0.1.4 iopath
```

--------------------------------

### Run Development Server

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/website/README.md

Starts the Docusaurus development server to preview the website. Use either yarn or npm.

```sh
# Start the site
$ yarn start

or
$ ./node_modules/docusaurus/lib/start-server.js
```

--------------------------------

### Install Website Dependencies

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/website/README.md

Installs the necessary dependencies for the Docusaurus website. Use either yarn or npm.

```sh
# Install dependencies
$ yarn

or

$ npm install docusaurus-init
```

--------------------------------

### Download Example Video

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/torchhub_inference_tutorial.ipynb

Downloads a sample video file ('archery.mp4') from a public URL for testing video classification.

```bash
!wget https://dl.fbaipublicfiles.com/pytorchvideo/projects/archery.mp4
```

--------------------------------

### Install PyTorchVideo from PyPI (Stable)

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/INSTALL.md

Use this command to install the latest stable release of PyTorchVideo from the Python Package Index.

```bash
pip install pytorchvideo
```

--------------------------------

### Download Example Video Data

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Downloads an example video of a chair and its associated metadata from the Objectron dataset using a provided script. The data is saved to `nerf/data/objectron`.

```bash
python download_objectron_data.py
```

--------------------------------

### Download Example Video

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/video_detection_example/video_detection_inference_tutorial.ipynb

Downloads a sample video file ('theatre.webm') from Wikimedia for testing video detection.

```bash
!wget https://dl.fbaipublicfiles.com/pytorchvideo/projects/theatre.webm
```

--------------------------------

### Launch Visdom Server

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Starts the Visdom server for visualizing predictions and metrics during and after training. Access visualizations at `https://localhost:8097`.

```bash
python -m visdom.server
```

--------------------------------

### Install PyTorch3D

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Installs PyTorch3D and its dependencies using conda. Ensure you are in a new conda environment.

```bash
conda create -n 3ddemo
conda activate 3ddemo

conda install -c pytorch pytorch=1.7.1 torchvision cudatoolkit=10.1
conda install -c conda-forge -c fvcore -c iopath fvcore iopath
conda install pytorch3d -c pytorch3d-nightly
```

--------------------------------

### Download AVA Action List and Setup Visualizer

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_torchhub_detection_inference.md

Downloads the AVA dataset's action list for label mapping and initializes a VideoVisualizer for displaying detection results.

```python
# Dowload the action text to id mapping
!wget https://dl.fbaipublicfiles.com/pytorchvideo/data/class_names/ava_action_list.pbtxt

# Create an id to label name mapping
label_map, allowed_class_ids = AvaLabeledVideoFramePaths.read_label_map('ava_action_list.pbtxt')
# Create a video visualizer that can plot bounding boxes and visualize actions on bboxes.
video_visualizer = VideoVisualizer(81, label_map, top_k=3, mode="thres",thres=0.5)
```

--------------------------------

### Download and Load Example Video

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_torchhub_detection_inference.md

Downloads a sample video file from Wikimedia and loads it using PyTorchVideo's EncodedVideo utility.

```python
# Download the demo video.
!wget https://dl.fbaipublicfiles.com/pytorchvideo/projects/theatre.webm

# Load the video
encoded_vid = pytorchvideo.data.encoded_video.EncodedVideo.from_path('theatre.webm')
print('Completed loading encoded video.')
```

--------------------------------

### Run Unit Tests

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/pytorchvideo_trainer/README.md

Execute the unit tests for PyTorchVideo Trainer from the current directory. Ensure all test dependencies are installed beforehand.

```bash
# From the current directory
python -m unittest discover -v -s ./tests
```

--------------------------------

### Download Objectron Dataset and Protobuf

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Clones the Objectron dataset repository and installs the protobuf library for parsing metadata files. The data will be saved to `nerf/data/objectron`.

```bash
git clone https://github.com/google-research-datasets/Objectron.git

# Also install protobuf for parsing the metadata
pip install protobuf
```

--------------------------------

### Run Linter Script

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/dev/README.md

Execute the linter script from the project root. Ensure necessary linter dependencies are installed beforehand.

```bash
./dev/linter.sh
```

--------------------------------

### Install NeRF Extras

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Installs additional Python libraries required for the NeRF implementation, including visdom, Pillow, matplotlib, tqdm, plotly, and hydra-core.

```bash
pip install visdom Pillow matplotlib tqdm plotly
pip install hydra-core --upgrade
```

--------------------------------

### Convert Model for Quantization

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_accelerator_use_accelerator_model_zoo.md

Perform calibration (skipped in this example) and then convert the prepared model to a quantized model using `torch.quantization.convert`. This step applies the quantization to the model weights and activations.

```python
# calibration is skipped here.
model_efficient_x3d_xs_deploy_quant_stub_wrapper_quantized = torch.quantization.convert(model_efficient_x3d_xs_deploy_quant_stub_wrapper_prepared)
```

--------------------------------

### Example Clip Dictionary Structure

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/data.md

Illustrates the structure of a clip dictionary returned by PyTorchVideo datasets, containing video, audio, label, and metadata.

```default
{
        'video': <video_tensor>,
        'audio': <audio_tensor>,
        'label': <action_label>,
        'video_name': <video_path>,
        'video_index': <video_id>,
        'clip_index': <clip_id>
     }
```

--------------------------------

### Convert model for quantization

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Build_your_model_with_PytorchVideo_Accelerator.ipynb

Performs the calibration (skipped in this example) and conversion steps for quantization. This results in an int8 quantized model ready for tracing.

```python
# calibration is skipped here.
net_inst_quant_stub_wrapper_quantized = torch.quantization.convert(net_inst_quant_stub_wrapper_prepared)
```

--------------------------------

### Clone and Setup PyTorch3D NeRF Implementation

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Clones the PyTorch3D NeRF project and copies the necessary files into the current directory. It then removes the rest of the PyTorch3D repository to save space.

```bash
cd pytorchvideo/tutorials/video_nerf
git clone https://github.com/facebookresearch/pytorch3d.git
cp -r pytorch3d/projects/nerf .

# Remove the rest of the PyTorch3D repo
rm -r pytorch3d
```

--------------------------------

### Define Example Input Tensor

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_Model_Transmuter.ipynb

Create a sample input tensor with the expected dimensions for the model. This tensor is used later for converting the model into a deployable form.

```python
# Define example input tensor
input_blob_size = (1, 3, 4, 6, 6)
input_tensor = torch.randn(input_blob_size)
```

--------------------------------

### Training the Video Classification Model

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_classification.md

Initiates the training process for the video classification model using PyTorch Lightning's Trainer. This example uses default local CPU training settings.

```python
  def train():
    classification_module = VideoClassificationLightningModule()
    data_module = KineticsDataModule()
    trainer = pytorch_lightning.Trainer()
    trainer.fit(classification_module, data_module)
```

--------------------------------

### Generate Coverage Reports

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tests/README.md

Generate a coverage report for the unit tests. This requires installing the 'coverage' package first, then running the tests with coverage enabled.

```bash
#Install Coverage using 
pip install coverage

# From root of the project
coverage run -m unittest discover -v -s tests
```

--------------------------------

### Create Nonlocal, Swish, and 2+1 Conv Layers

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/layers.md

Constructs instances of Nonlocal, Swish, and 2+1 Conv layers with random weights. Ensure PyTorchVideo is installed and imported.

```python
import pytorchvideo.layers as layers

nonlocal = layers.create_nonlocal(dim_in=256, dim_inner=128)
swish = layers.Swish()
conv_2plus1d = layers.create_conv_2plus1d(in_channels=256, out_channels=512)
```

--------------------------------

### Loading Kinetics Dataset with DataLoader

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/data.md

Example of initializing a Kinetics dataset and using it with PyTorch's DataLoader for batch loading. Ensure the data path points to your Kinetics dataset file.

```python
dataset = pytorchvideo.data.Kinetics(
        data_path="path/to/kinetics_root/train.csv",
        clip_sampler=pytorchvideo.data.make_clip_sampler("random", duration=2),
)
data_loader = torch.utils.data.DataLoader(dataset, batch_size=8)
```

--------------------------------

### Build Standard Model Heads

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/models.md

Construct standard model heads with random weights. Examples include basic residual heads and X3D heads, used for classification or detection tasks.

```python
import pytorchvideo.models as models

res_head = models.head.create_res_basic_head(in_features, out_features)
x3d_head = models.x3d.create_x3d_head(dim_in=1024, dim_inner=512, dim_out=2048, num_classes=400)
```

--------------------------------

### Prepare Model for Quantization

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_accelerator_build_your_model.md

Configure the quantization backend and prepare the model for quantization. Ensure you select the appropriate backend and qconfig for your target device.

```python
net_inst_quant_stub_wrapper.qconfig = torch.quantization.torch.quantization.get_default_qconfig("qnnpack")
torch.backends.quantized.engine = "qnnpack"
net_inst_quant_stub_wrapper_prepared = torch.quantization.prepare(net_inst_quant_stub_wrapper)
```

--------------------------------

### Build HTML Documentation

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/README.md

Command to build the HTML version of the documentation from the docs directory using make.

```bash
> make html
```

--------------------------------

### Verify Nonlocal Layer Functionality

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/layers.md

Tests the Nonlocal layer by passing a dummy tensor through it. Requires PyTorch and PyTorchVideo to be installed.

```python
import pytorchvideo.layers as layers
import torch

nonlocal = layers.create_nonlocal(dim_in=256, dim_inner=128)
B, C, T, H, W = 2, 256, 4, 14, 14
input_tensor = torch.zeros(B, C, T, H, W)
output = nonlocal(input_tensor)
```

--------------------------------

### Initialize AVA Label Map and Video Visualizer

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/video_detection_example/video_detection_inference_tutorial.ipynb

Creates a mapping from action IDs to labels and initializes a VideoVisualizer for plotting detection results. Adjust `top_k` and `thres` for visualization.

```python
# Create an id to label name mapping
label_map, allowed_class_ids = AvaLabeledVideoFramePaths.read_label_map('ava_action_list.pbtxt')
# Create a video visualizer that can plot bounding boxes and visualize actions on bboxes.
video_visualizer = VideoVisualizer(81, label_map, top_k=3, mode="thres",thres=0.5)
```

--------------------------------

### Get Model Predictions

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_torchhub_inference.md

Pass the prepared input video data through the loaded PyTorchVideo model to obtain action predictions.

```python
preds = model(inputs)
```

--------------------------------

### Verify Swish Layer Functionality

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/layers.md

Tests the Swish activation layer by passing a dummy tensor through it. Requires PyTorch and PyTorchVideo to be installed.

```python
import pytorchvideo.layers as layers
import torch

swish = layers.Swish()
B, C, T, H, W = 2, 256, 4, 14, 14
input_tensor = torch.zeros(B, C, T, H, W)
output = swish(input_tensor)
```

--------------------------------

### Verify 2+1 Conv Layer Functionality

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/layers.md

Tests the 2+1 Conv layer by passing a dummy tensor through it. Requires PyTorch and PyTorchVideo to be installed.

```python
import pytorchvideo.layers as layers
import torch

conv_2plus1d = layers.create_conv_2plus1d(in_channels=256, out_channels=512)
B, C, T, H, W = 2, 256, 4, 14, 14
input_tensor = torch.zeros(B, C, T, H, W)
output = conv_2plus1d(input_tensor)
```

--------------------------------

### Export JIT Trace and Optimize

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_Model_Transmuter.ipynb

Export a TorchScript JIT trace of the deployable model and apply `optimize_for_mobile` for final optimization. This prepares the model for efficient mobile deployment.

```python
from torch.utils.mobile_optimizer import (
    optimize_for_mobile,
)
traced_model = torch.jit.trace(model_transmute_deploy, input_tensor, strict=False)
traced_model_opt = optimize_for_mobile(traced_model)
# Here we can save the traced_model_opt to JIT file using traced_model_opt.save(<file_path>)
```

--------------------------------

### Instantiate quantization stub wrapper

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Build_your_model_with_PytorchVideo_Accelerator.ipynb

Instantiates the `quant_stub_wrapper` class with the deployable model. This prepares the model for quantization by adding the necessary stub modules.

```python
net_inst_quant_stub_wrapper = quant_stub_wrapper(net_inst_deploy)
```

--------------------------------

### Prepare Model for Quantization

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_PytorchVideo_Accelerator_Model_Zoo.ipynb

Sets the quantization configuration and prepares the model for quantization using `torch.quantization.prepare`. Fusion of efficient blocks is assumed to be done automatically during the deployment conversion.

```python
model_efficient_x3d_xs_deploy_quant_stub_wrapper.qconfig = torch.quantization.default_qconfig
model_efficient_x3d_xs_deploy_quant_stub_wrapper_prepared = torch.quantization.prepare(model_efficient_x3d_xs_deploy_quant_stub_wrapper)
```

--------------------------------

### Prepare model for quantization

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Build_your_model_with_PytorchVideo_Accelerator.ipynb

Sets the quantization configuration and prepares the model for quantization using `torch.quantization.prepare`. Fusion of efficient blocks is assumed to be done automatically during the `convert_to_deployable_form` step.

```python
net_inst_quant_stub_wrapper.qconfig = torch.quantization.default_qconfig
net_inst_quant_stub_wrapper_prepared = torch.quantization.prepare(net_inst_quant_stub_wrapper)
```

--------------------------------

### Export and Optimize for Mobile

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_accelerator_use_model_transmuter.md

Trace the deployable model using `torch.jit.trace` and then apply `optimize_for_mobile` for final optimization. This prepares the model for efficient execution on mobile devices.

```python
from torch.utils.mobile_optimizer import (\
    optimize_for_mobile,
)
traced_model = torch.jit.trace(model_transmute_deploy, input_tensor, strict=False)
traced_model_opt = optimize_for_mobile(traced_model)
# Here we can save the traced_model_opt to JIT file using traced_model_opt.save(<file_path>)
```

--------------------------------

### Define a User Model

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_Model_Transmuter.ipynb

Define a sample PyTorch nn.Module that will be used as input for the model transmuter. This example defines a residual block with several convolutional layers.

```python
import torch
import torch.nn as nn

class user_model_residual_block(nn.Module):
    def __init__(self):
        super().__init__()
        self.stem0 = nn.Conv3d(3, 3, kernel_size=(3, 1, 1), padding=(1, 0, 0))
        self.stem1 = nn.Conv3d(3, 3, kernel_size=(5, 1, 1), padding=(2, 0, 0))
        self.pw = nn.Conv3d(3, 6, kernel_size=1)
        self.relu = nn.ReLU()
        self.dw = nn.Conv3d(6, 6, kernel_size=3, padding=1, groups=6)
        self.relu1 = nn.ReLU()
        self.pwl = nn.Conv3d(6, 3, kernel_size=1)
        self.relu2 = nn.ReLU()

    def forward(self, x):
        out = self.stem0(x)
        out = self.stem1(out)
        out = self.pw(out)
        out = self.relu(out)
        out = self.dw(out)
        out = self.relu1(out)
        out = self.pwl(out)
        return self.relu2(out + x)
```

--------------------------------

### Run Unit Tests

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tests/README.md

Execute all unit tests within the project using the unittest module. Ensure you are in the root directory of the project.

```bash
# From root of the project
python -m unittest discover -v -s ./tests
```

--------------------------------

### ClipSampler Initialization

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/api/data/data.md

Initializes a clip sampler to define how clips are extracted from videos.

```APIDOC
## ClipSampler

### Description
Initializes a clip sampler to define how clips are extracted from videos.

### Parameters
#### Parameters
- **clip_duration** (Union[float, Fraction]) - Required - The length of the clip to sample (in seconds).
- **stride** (Union[float, Fraction], optional) - Optional - The amount of seconds to offset the next clip by. Defaults to None, which is equivalent to clip_duration.
- **eps** (float) - Required - Epsilon for floating point comparisons. Used to check the last clip.
- **backpad_last** (bool) - Required - Whether to include the last frame(s) by “back padding”.

### Example
```python
from fractions import Fraction

# Example with float
clip_sampler_float = ClipSampler(clip_duration=1.0, stride=0.5)

# Example with Fraction for precision
clip_sampler_fraction = ClipSampler(clip_duration=Fraction(1, 1), stride=Fraction(1, 2))
```
```

--------------------------------

### Standard Video Transform Pipeline

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/transforms.md

Example of a standard transform pipeline for a video model using PyTorchVideo and TorchVision transforms. This pipeline is suitable for use with PyTorchVideo datasets.

```python
transform = torchvision.transforms.Compose([
  pytorchvideo.transforms.ApplyTransformToKey(
    key="video",
    transform=torchvision.transforms.Compose([
      pytorchvideo.transforms.UniformTemporalSubsample(8),
      pytorchvideo.transforms.Normalize((0.45, 0.45, 0.45), (0.225, 0.225, 0.225)),
      pytorchvideo.transforms.RandomShortSideScale(min_size=256, max_size=320),
      torchvision.transforms.RandomCrop(244),
      torchvision.transforms.RandomHorizontalFlip(p=0.5),
    ])
  )
])
dataset = pytorchvideo.data.Kinetics(
  data_path="path/to/kinetics_root/train.csv",
  clip_sampler=pytorchvideo.data.make_clip_sampler("random", duration=2),
  transform=transform
)
```

--------------------------------

### Instantiate Quantization Wrapper

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_PytorchVideo_Accelerator_Model_Zoo.ipynb

Instantiates the `quant_stub_wrapper` with the deployable model, adding quantization stubs.

```python
model_efficient_x3d_xs_deploy_quant_stub_wrapper = quant_stub_wrapper(model_efficient_x3d_xs_deploy)
```

--------------------------------

### Import PyTorchVideo and Utilities

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_torchhub_inference.md

Imports necessary libraries for PyTorchVideo model loading, data handling, and transformations.

```python
import torch
import json
from torchvision.transforms import Compose, Lambda
from torchvision.transforms._transforms_video import (
    CenterCropVideo,
    NormalizeVideo,
)
from pytorchvideo.data.encoded_video import EncodedVideo
from pytorchvideo.transforms import (
    ApplyTransformToKey,
    ShortSideScale,
    UniformTemporalSubsample,
    UniformCropVideo
)
from typing import Dict
```

--------------------------------

### Export and Optimize for Mobile (Floating Point)

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_PytorchVideo_Accelerator_Model_Zoo.ipynb

Exports the deployable model to a TorchScript traced model and then applies `optimize_for_mobile` for final optimization. This is for deploying a floating-point model.

```python
from torch.utils.mobile_optimizer import (
    optimize_for_mobile,
)
traced_model = torch.jit.trace(model_efficient_x3d_xs_deploy, input_tensor, strict=False)
traced_model_opt = optimize_for_mobile(traced_model)
# Here we can save the traced_model_opt to JIT file using traced_model_opt.save(<file_path>)
```

--------------------------------

### Define Function to Get Person Bounding Boxes

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/video_detection_example/video_detection_inference_tutorial.ipynb

A helper function that takes an image and a Detectron2 predictor, returning bounding boxes specifically for detected people with a score threshold of 0.75.

```python
# This method takes in an image and generates the bounding boxes for people in the image.
def get_person_bboxes(inp_img, predictor):
    predictions = predictor(inp_img.cpu().detach().numpy())['instances'].to('cpu')
    boxes = predictions.pred_boxes if predictions.has("pred_boxes") else None
    scores = predictions.scores if predictions.has("scores") else None
    classes = np.array(predictions.pred_classes.tolist() if predictions.has("pred_classes") else None)
    predicted_boxes = boxes[np.logical_and(classes==0, scores>0.75 )].tensor.cpu() # only person
    return predicted_boxes
```

--------------------------------

### Import Detection and Video Processing Libraries

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/video_detection_example/video_detection_inference_tutorial.ipynb

Imports essential libraries for detection, video processing, and model loading, including Detectron2 components and PyTorchVideo's functional transforms.

```python
from functools import partial
import numpy as np

import cv2
import torch

import detectron2
from detectron2.config import get_cfg
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor

import pytorchvideo
from pytorchvideo.transforms.functional import (
    uniform_temporal_subsample,
    short_side_scale_with_boxes,
    clip_boxes_to_image,
)
from torchvision.transforms._functional_video import normalize
from pytorchvideo.data.ava import AvaLabeledVideoFramePaths
from pytorchvideo.models.hub import slow_r50_detection # Another option is slowfast_r50_detection

from visualization import VideoVisualizer
```

--------------------------------

### Import Model Transmuter Utilities

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_Model_Transmuter.ipynb

Import the necessary modules for model transmutation, including the specific transmuter for the target device (e.g., mobile_cpu) and the top-level wrapper function.

```python
import pytorchvideo.accelerator.deployment.mobile_cpu.transmuter  # mobile cpu model transmuter
from pytorchvideo.accelerator.deployment.common.model_transmuter import transmute_model  # top-level wrapper of model transmuter
```

--------------------------------

### Verify model arithmetic equivalence

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Build_your_model_with_PytorchVideo_Accelerator.ipynb

Checks if the converted model is arithmetically equivalent to the original model by comparing their outputs on an example input tensor. Small differences due to floating-point operations are expected.

```python
net_inst.eval()
out_ref = net_inst(input_tensor)
out = net_inst_deploy(input_tensor)

max_err = float(torch.max(torch.abs(out_ref - out)))
print(f"max error is {max_err}")
```

--------------------------------

### Convert Model to Deployable Form

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_PytorchVideo_Accelerator_Model_Zoo.ipynb

Converts the loaded model into a deployable form using `convert_to_deployable_form`. This process includes kernel and graph optimizations. The model's input size after conversion must match the example input tensor size.

```python
import torch
from pytorchvideo.accelerator.deployment.mobile_cpu.utils.model_conversion import (
    convert_to_deployable_form,
)
input_blob_size = (1, 3, 4, 160, 160)
input_tensor = torch.randn(input_blob_size)
model_efficient_x3d_xs_deploy = convert_to_deployable_form(model_efficient_x3d_xs, input_tensor)
```

--------------------------------

### Build Standard Video Models

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/models.md

Construct standard video understanding models with random weights using provided builders. These include ResNet, Acoustic ResNet, SlowFast, X3D, R2+1D, and CSN.

```python
import pytorchvideo.models as models

resnet = models.create_resnet()
acoustic_resnet = models.create_acoustic_resnet()
slowfast = models.create_slowfast()
x3d = models.create_x3d()
r2plus1d = models.create_r2plus1d()
csn = models.create_csn()
```

--------------------------------

### Load Pretrained PyTorchVideo Model via TorchHub

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/pytorchvideo/models/hub/README.md

Demonstrates how to load a specific PyTorchVideo model with pre-trained weights using torch.hub.load. Ensure you specify the correct repository and model name.

```Python
model_name = "slowfast_r50"
model = torch.hub.load("facebookresearch/pytorchvideo:main", model=model_name, pretrained=True)
```

--------------------------------

### Load and Prepare Video Clip for Inference

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/torchhub_inference_tutorial.ipynb

Load a specific segment of a video, apply necessary transformations, and move the data to the appropriate device for model input.

```python
# Select the duration of the clip to load by specifying the start and end duration
# The start_sec should correspond to where the action occurs in the video
start_sec = 0
end_sec = start_sec + clip_duration 

# Initialize an EncodedVideo helper class
video = EncodedVideo.from_path(video_path)

# Load the desired clip
video_data = video.get_clip(start_sec=start_sec, end_sec=end_sec)

# Apply a transform to normalize the video input
video_data = transform(video_data)

# Move the inputs to the desired device
inputs = video_data["video"]
inputs = [i.to(device)[None, ...] for i in inputs]
```

--------------------------------

### Print deployable model

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Build_your_model_with_PytorchVideo_Accelerator.ipynb

Prints the network graph of the model after it has been converted to a deployable form, showing optimizations made.

```python
print(net_inst_deploy)
```

--------------------------------

### Train NeRF Model

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/projects/video_nerf/README.md

Initiates the NeRF model training process using the specified configuration. Ensure you are in the `nerf` directory.

```bash
cd nerf
python ./train_nerf.py --config-name objectron
```

--------------------------------

### Build Standard Models

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/models.md

Construct standard video understanding models with random weights using provided builders.

```APIDOC
## Build standard models

PyTorchVideo provide default builders to construct state-of-the-art video understanding models, layers, heads, and losses.

### Models

You can construct a model with random weights by calling its constructor:

```python
import pytorchvideo.models as models

resnet = models.create_resnet()
acoustic_resnet = models.create_acoustic_resnet()
slowfast = models.create_slowfast()
x3d = models.create_x3d()
r2plus1d = models.create_r2plus1d()
csn = models.create_csn()
```

You can verify whether you have built the model successfully by:

```python
import pytorchvideo.models as models
import torch

resnet = models.create_resnet()
B, C, T, H, W = 2, 3, 8, 224, 224
input_tensor = torch.zeros(B, C, T, H, W)
output = resnet(input_tensor)
```
```

--------------------------------

### Add Symlink to Root README

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/README.md

Creates a symbolic link to the root README.md file within the docs directory to include it as an overview. This is a prerequisite before generating the documentation.

```bash
cd <pytorchvideo>/docs
ln -s ../README.md  overview.md
```

--------------------------------

### Create New Blog Post

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/website/README.md

Markdown frontmatter for a new blog post. Includes author details and the post title. The filename must follow the `YYYY-MM-DD-Title.md` format.

```markdown
---
author: Frank Li
authorURL: https://twitter.com/foobarbaz
authorFBID: 503283835
title: New Blog Post
---

Lorem Ipsum...
```

--------------------------------

### Ava Dataset

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/api/data/index.md

Initializes the Ava dataset.

```APIDOC
## Ava()

### Description
Initializes the Ava dataset.

### Parameters
None explicitly documented.
```

--------------------------------

### Instantiate MyNet Model

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Build_your_model_with_PytorchVideo_Accelerator.ipynb

Instantiates the MyNet model. When instantiated, the efficient blocks within the model are in their original form, suitable for training.

```python
net_inst = MyNet()
print(net_inst)
```

--------------------------------

### Move Inputs to Device

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/website/docs/tutorial_torchhub_inference.md

Prepare video data by moving it to the appropriate device (e.g., GPU). This is a prerequisite for model inference.

```python
inputs = video_data["video"]
inputs = [i.to(device)[None, ...] for i in inputs]
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/torchhub_inference_tutorial.ipynb

Imports essential modules from torchvision, pytorchvideo, and typing for video processing and model transformations.

```python
import json 
from torchvision.transforms import Compose, Lambda
from torchvision.transforms._transforms_video import (
    CenterCropVideo,
    NormalizeVideo,
)
from pytorchvideo.data.encoded_video import EncodedVideo
from pytorchvideo.transforms import (
    ApplyTransformToKey,
    ShortSideScale,
    UniformTemporalSubsample,
    UniformCropVideo
) 
from typing import Dict
```

--------------------------------

### Convert Model to Deployable Form

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_Model_Transmuter.ipynb

Import and use the `convert_to_deployable_form` utility to prepare the transmuted model for deployment. This function converts efficient blocks to their deployable representation.

```python
from pytorchvideo.accelerator.deployment.mobile_cpu.utils.model_conversion import (
    convert_to_deployable_form,
)
model_transmute_deploy = convert_to_deployable_form(
    model_transmute,
    input_tensor
)
```

--------------------------------

### Download AVA Annotations

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/docs/source/data_preparation.md

Downloads the official AVA v2.2 annotation zip file and extracts its contents into the specified annotations directory. It checks if the directory exists and creates it if necessary.

```shell
DATA_DIR="../../data/ava/annotations"

if [[ ! -d "${DATA_DIR}" ]]; then
  echo "${DATA_DIR} doesn't exist. Creating it.";
  mkdir -p ${DATA_DIR}
fi

wget https://research.google.com/ava/download/ava_v2.2.zip -P ${DATA_DIR}
unzip -q ${DATA_DIR}/ava_v2.2.zip -d ${DATA_DIR}
```

--------------------------------

### Trace and Optimize Quantized Model for Mobile

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_PytorchVideo_Accelerator_Model_Zoo.ipynb

Trace the quantized model to create a TorchScript representation and then optimize it for mobile deployment. The traced model can be saved to a JIT file.

```python
traced_model_int8 = torch.jit.trace(model_efficient_x3d_xs_deploy_quant_stub_wrapper_quantized, input_tensor, strict=False)
traced_model_int8_opt = optimize_for_mobile(traced_model_int8)
# Here we can save the traced_model_opt to JIT file using traced_model_int8_opt.save(<file_path>)
```

--------------------------------

### Load Pre-trained Checkpoint

Source: https://github.com/facebookresearch/pytorchvideo/blob/main/tutorials/accelerator/Use_PytorchVideo_Accelerator_Model_Zoo.ipynb

Loads a pre-trained checkpoint for the EfficientX3D-XS model from a URL. This checkpoint is in the original form, suitable for further training.

```python
from torch.hub import load_state_dict_from_url
checkpoint_path = 'https://dl.fbaipublicfiles.com/pytorchvideo/model_zoo/kinetics/efficient_x3d_xs_original_form.pyth'
checkpoint = load_state_dict_from_url(checkpoint_path)

model_efficient_x3d_xs.load_state_dict(checkpoint)
```