### Start Jupyter Notebook Server with Colab Origin Allowed

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/README.md

Start a Jupyter notebook server that allows connections from Google Colab. This is necessary for running the 'OpticalFlow-Inference.ipynb' notebook.

```bash
> jupyter notebook --NotebookApp.allow_origin='https://colab.research.google.com' --port=8888   -NotebookApp.port_retries=0
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/README.md

Install all required Python packages for running the optical flow inference notebook. This command reads from the 'requirements.txt' file.

```bash
> pip install -r requirements.txt
```

--------------------------------

### Create Directory for Pre-trained Models

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/README.md

Create a directory to store large pre-trained model files. Ensure this directory is separate from the project's source code.

```bash
> mkdir <pretrained_models>
```

--------------------------------

### Run PWC-Net on DAVIS Dataset

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Loads image pair from the DAVIS dataset and runs the PWC-Net model to compute optical flow. Requires the `run_pwc_it` function and pre-defined paths.

```python
# setup data
DATA_DIR = os.path.join(BASE_DIR, 'examples')

# load data and run optical flow network
im0_path = os.path.join(DATA_DIR, 'davis/frame_0000.png')
im1_path = os.path.join(DATA_DIR, 'davis/frame_0001.png')
model_optflow = run_pwc_it(model, im0_path, im1_path)
```

--------------------------------

### Run PWC-Net on Middlebury Dataset

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Loads image pair from the Middlebury dataset and runs the PWC-Net model to compute optical flow. Requires the `run_pwc_it` function and pre-defined paths.

```python
# load data and run optical flow network
im0_path = os.path.join(DATA_DIR, 'middlebury_dogdance/frame_0007.png')
im1_path = os.path.join(DATA_DIR, 'middlebury_dogdance/frame_0008.png')
model_optflow = run_pwc_it(model, im0_path, im1_path)
```

--------------------------------

### Run PWC Inference and Display Results

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

This function orchestrates the optical flow calculation and visualization. It reads two images, computes the optical flow using PWC-Net, and displays the input images along with the resulting flow visualization. Ensure 'media' and 'flow_to_img' are correctly imported and available.

```python
def run_pwc_it(pwc_net, im0_path, im1_path):
    im0 = media.read_image(im0_path)
    im1 = media.read_image(im1_path)
    optflow = calculate_optical_flow_pwc(pwc_net.signatures['serving_default'], im0, im1)
    media.show_images([im0, im1, flow_to_img(optflow)])
    return optflow
```

--------------------------------

### Run RAFT Inference and Display

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Loads images from paths, calculates optical flow using the RAFT model, and displays the input images along with the computed flow visualization. This function assumes `media.read_image`, `flow_to_img`, and `media.show_images` are available.

```python
def run_raft_it(model, im0_path, im1_path):
    im0 = media.read_image(im0_path)
    im1 = media.read_image(im1_path)
    optflow = calculate_optical_flow_raft(model.signatures['serving_default'], im0, im1)
    media.show_images([im0, im1, flow_to_img(optflow)])
    return optflow
```

--------------------------------

### Run RAFT on Middlebury Dataset

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Loads an image pair from the Middlebury dataset and runs the RAFT model to compute optical flow. This utilizes the `run_raft_it` function and assumes the model and data paths are correctly configured.

```python
# load data and run optical flow network
im0_path = os.path.join(DATA_DIR, 'middlebury_dogdance/frame_0007.png')
im1_path = os.path.join(DATA_DIR, 'middlebury_dogdance/frame_0008.png')
model_optflow = run_raft_it(model, im0_path, im1_path)
```

--------------------------------

### Run RAFT on DAVIS Dataset

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Loads an image pair from the DAVIS dataset and runs the RAFT model to compute optical flow. This utilizes the `run_raft_it` function and assumes the model and data paths are correctly configured.

```python
# setup data
DATA_DIR = os.path.join(BASE_DIR, 'examples')

# load data and run optical flow network
im0_path = os.path.join(DATA_DIR, 'davis/frame_0000.png')
im1_path = os.path.join(DATA_DIR, 'davis/frame_0001.png')
model_optflow = run_raft_it(model, im0_path, im1_path)
```

--------------------------------

### Pad Image Batch to Align Dimensions

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Pads an image batch so that its width and height are divisible by a specified alignment factor. Returns the padded image and a bounding box for undoing the padding.

```python
def _pad_to_align(x, align):
  """Pad image batch x so width and height divide by align.

  Args:
    x: Image batch to align.
    align: Number to align to.

  Returns:
    1) An image padded so width % align == 0 and height % align == 0.
    2) A bounding box that can be fed readily to tf.image.crop_to_bounding_box
      to undo the padding.
  """
  # Input checking.
  assert np.ndim(x) == 4
  assert align > 0, 'align must be a positive number.'

  height, width = x.shape[-3:-1]
  height_to_pad = (align - height % align) if height % align != 0 else 0
  width_to_pad = (align - width % align) if width % align != 0 else 0

  bbox_to_pad = {
      'offset_height': height_to_pad // 2,
      'offset_width': width_to_pad // 2,
      'target_height': height + height_to_pad,
      'target_width': width + width_to_pad,
  }
  #  print(bbox_to_pad)
  padded_x = tf.image.pad_to_bounding_box(x, **bbox_to_pad)
  bbox_to_crop = {
      'offset_height': height_to_pad // 2,
      'offset_width': width_to_pad // 2,
      'target_height': height,
      'target_width': width,
  }
  return padded_x, bbox_to_crop
```

--------------------------------

### Import Libraries for Optical Flow

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Imports necessary libraries for TensorFlow, NumPy, time, OS operations, media handling, and OpenCV.

```python
import tensorflow as tf
import numpy as np
import time 
import os
import mediapy as media
import cv2
```

--------------------------------

### Convert Flow to Viewable Image (OpenCV)

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Converts optical flow to a viewable RGB image using OpenCV. It encodes flow vector orientation with color hue and vector length with color saturation. Optionally, it can normalize the flow magnitude and superimpose text information.

```python
def flow_to_img(flow, normalize=True, info=None, flow_mag_max=None):
  """Convert flow to viewable image, using color hue to encode flow vector orientation, and color saturation to

    encode vector length. This is similar to the OpenCV tutorial on dense
    optical flow, except that they map vector
    length to the value plane of the HSV color model, instead of the saturation
    plane, as we do here.
    Args:
        flow: optical flow
        normalize: Normalize flow to 0..255
        info: Text to superimpose on image (typically, the epe for the predicted
          flow)
        flow_mag_max: Max flow to map to 255

    Returns:
        img: viewable representation of the dense optical flow in RGB format
        flow_avg: optionally, also return average flow magnitude
    Ref:
        - OpenCV 3.0.0-dev documentation » OpenCV-Python Tutorials » Video
        Analysis »
        https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_video/py_lucas_kanade/py_lucas_kanade.html
        https://github.com/philferriere/tfoptflow/blob/master/tfoptflow/optflow.py#L190
        https://github.com/philferriere/tfoptflow/blob/master/LICENSE
    """
  hsv = np.zeros((flow.shape[0], flow.shape[1], 3), dtype=np.uint8)
  flow_magnitude, flow_angle = cv2.cartToPolar(flow[..., 0].astype(np.float32),
                                               flow[..., 1].astype(np.float32))

  # A couple times, we've gotten NaNs out of the above...
  nans = np.isnan(flow_magnitude)
  if np.any(nans):
    nans = np.where(nans)
    flow_magnitude[nans] = 0.

  # Normalize
  hsv[..., 0] = flow_angle * 180 / np.pi / 2
  if normalize is True:
    if flow_mag_max is None:
      hsv[..., 1] = cv2.normalize(flow_magnitude, None, 0, 255, cv2.NORM_MINMAX)
    else:
      hsv[..., 1] = flow_magnitude * 255 / flow_mag_max
  else:
    hsv[..., 1] = flow_magnitude
  hsv[..., 2] = 255
  img = cv2.cvtColor(hsv, cv2.COLOR_HSV2RGB)

  # Add text to the image, if requested
  if info is not None:
    font = cv2.FONT_HERSHEY_SIMPLEX
```

--------------------------------

### Set Base Directory for Data

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Defines the base directory for accessing data. This variable should be updated to reflect the correct path on your system.

```python
BASE_DIR = '/usr/local/google/home/irwinherrmann/Documents/returnpwc-eccv/eccv' ## TODO: change this
```

--------------------------------

### Load PWC-Net Model

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Loads a pre-trained PWC-Net model from a specified directory using TensorFlow's SavedModel API. Ensure the model path is correctly set.

```python
model_key = 'pwc_it_ft'
MODEL_DIR = os.path.join(BASE_DIR, 'pretrained_models')
MODEL_PATH = os.path.join(MODEL_DIR, model_key)
model = tf.compat.v2.saved_model.load(MODEL_PATH, options=tf.saved_model.LoadOptions(allow_partial_checkpoint=True))
print('Loaded: %s' % model_key)
```

--------------------------------

### Calculate Optical Flow using PWC-Net

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

This function calculates optical flow between two images using a loaded PWC-Net model. It handles image preprocessing, padding, model inference, and cropping of the output flow. Ensure the model is loaded and images are in the correct format.

```python
def calculate_optical_flow_pwc(loaded_model, image0, image1):
  image0 = tf.cast(image0, tf.float32) / 127.5 - 1
  image1 = tf.cast(image1, tf.float32) / 127.5 - 1
    
  image0_batch = np.reshape(image0, (1,) + image0.shape)
  image1_batch = np.reshape(image1, (1,) + image1.shape)
  image0_batch = tf.convert_to_tensor(image0_batch, dtype=tf.float32)
  image1_batch = tf.convert_to_tensor(image1_batch, dtype=tf.float32)
  
  image0_batch, bbox_to_crop = _pad_to_align(image0_batch, align=64)
  image1_batch, bbox_to_crop = _pad_to_align(image1_batch, align=64)

  input_batch = tf.concat([image0_batch, image1_batch], axis=0)
  input_batch = tf.expand_dims(input_batch, axis=0)

  flow_output = loaded_model(input_batch)
  
  # input_batch = tf.concat([tf.expand_dims(image0_batch, axis=1), tf.expand_dims(image1_batch, axis=1)], axis=1)
  # flow_output = loaded_model(input=input_batch,max_rec_iters=tf.convert_to_tensor(12))

  flow_output = tf.image.crop_to_bounding_box(flow_output['output_1'], **bbox_to_crop)

  height, width, _ = image0.shape
  flow_shape = flow_output.shape
  np.testing.assert_array_equal(flow_shape, (1,) + (height, width, 2))
  
  return flow_output.numpy()[0]
```

--------------------------------

### Convert Flow to Color Image (TensorFlow)

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Converts an optical flow tensor into a 3-channel RGB color image using the HSV color space. Supports an optional mask for flow validity and a maximum flow value for normalization.

```python
def flow_to_color(flow, mask=None, max_flow=None):
  """Converts flow to 3-channel color image.

  Args:
      flow: tensor of shape [num_batch, height, width, 2].
      mask: flow validity mask of shape [num_batch, height, width, 1].
      max_flow: unused argument.

  Returns:
    flow in 3-channel RGB image.

  Snippet Reference:
    Code: https://github.com/ppliuboy/SelFlow/blob/master/flowlib.py#L109
    MIT License: https://github.com/ppliuboy/SelFlow/blob/master/LICENSE
  """
  n = 8
  height, width, _ = tf.unstack(tf.shape(flow))
  mask = tf.ones([height, width, 1]) if mask is None else mask
  flow_u, flow_v = tf.unstack(flow, axis=2)
  if max_flow is not None:
    max_flow = tf.maximum(tf.to_float(max_flow), 1.)
  else:
    max_flow = tf.reduce_max(tf.abs(flow * mask))
  mag = tf.sqrt(tf.reduce_sum(tf.square(flow), 2))
  angle = tf.atan2(flow_v, flow_u)

  im_h = tf.math.mod(angle / (2 * np.pi) + 1.0, 1.0)
  im_s = tf.clip_by_value(mag * n / max_flow, 0, 1)
  im_v = tf.clip_by_value(n - im_s, 0, 1)
  im_hsv = tf.stack([im_h, im_s, im_v], 2)
  im = tf.image.hsv_to_rgb(im_hsv)
  return im * mask
```

--------------------------------

### Load RAFT Model

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Loads a pre-trained RAFT model from a specified directory using TensorFlow's SavedModel API. Ensure the `BASE_DIR` and `model_key` are correctly set.

```python
BASE_DIR = '/usr/local/google/home/irwinherrmann/Documents/returnpwc-eccv/eccv' ## TODO: change this

# load model
model_key = 'raft_it_ft'
MODEL_DIR = os.path.join(BASE_DIR, 'pretrained_models')
MODEL_PATH = os.path.join(MODEL_DIR, model_key)
model = tf.compat.v2.saved_model.load(MODEL_PATH, options=tf.saved_model.LoadOptions(allow_partial_checkpoint=True))
print('Loaded: %s' % model_key)
```

--------------------------------

### RAFT Optical Flow Calculation

Source: https://github.com/google-research/opticalflow-autoflow/blob/main/colabs/OpticalFlow-Inference.ipynb

Calculates optical flow using a loaded RAFT model. This function handles image preprocessing, padding, model inference, and output cropping. It expects images to be in RGB format and normalizes them to [-1, 1].

```python
def calculate_optical_flow_raft(loaded_model, image0, image1):
  image0 = tf.cast(image0, tf.float32) / 127.5 - 1
  image1 = tf.cast(image1, tf.float32) / 127.5 - 1
    
  image0_batch = np.reshape(image0, (1,) + image0.shape)
  image1_batch = np.reshape(image1, (1,) + image1.shape)
  image0_batch = tf.convert_to_tensor(image0_batch, dtype=tf.float32)
  image1_batch = tf.convert_to_tensor(image1_batch, dtype=tf.float32)
  
  image0_batch, bbox_to_crop = _pad_to_align(image0_batch, align=64)
  image1_batch, bbox_to_crop = _pad_to_align(image1_batch, align=64)

  input_batch = tf.concat([image0_batch, image1_batch], axis=0)
  input_batch = tf.expand_dims(input_batch, axis=0)

  flow_output = loaded_model(input_1=input_batch, input_2=tf.constant(12))

  flow_output = tf.image.crop_to_bounding_box(flow_output['output_1'], **bbox_to_crop)

  height, width, _ = image0.shape
  flow_shape = flow_output.shape
  np.testing.assert_array_equal(flow_shape, (1,) + (height, width, 2))
  
  return flow_output.numpy()[0]
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.