### Build and install NvFRUCSample (Linux) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Build the executable and install it into the ubuntu directory. ```bash make && make install ``` -------------------------------- ### Build and install samples on Linux (cmake, make, sudo make install) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Build and install the Optical Flow SDK basic samples on Linux in release mode. Run these commands in the 'build' directory inside NvOFBasicSamples; the binaries are copied to 'build/bin/x64'. ```bash cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=. .. ``` ```bash make ``` ```bash sudo make install ``` -------------------------------- ### Build and install NvFRUCSample (Windows) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Build the project in release configuration and install it. This copies the executable and FreeImage.dll to the appropriate win32 or win64 directory. ```bash cmake --build . --target install --config release ``` -------------------------------- ### Run NvOFFRUCSample with YUV Input Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Example command line for processing a YUV420 video file. The input path must not contain whitespaces. ```bash NvOFFRUCSample.exe --input=C:\fruc\inputfile.yuv --width=1920 --height=1080 --output=C:\fruc\outputdir ``` -------------------------------- ### NvOFFRUCSample Help Menu Parameters Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Run the application without parameters to view the help menu. This list details the mandatory and optional command-line arguments required for execution. ```text Mandatory Parameters --input Input name , Path must be Absolute --width Input frame width --height Input frame height --output Output Directory to store results, Path could be Absolute or Relative Optional Parameters --surfaceformat Surface format : Set 0 for NV12, 1 for ABGR --startframe Start processing from this frame index --endframe Process till this frame index --allocationtype Specify 0 to create CUDA and 1 to create DX allocations --cudasurfacetype Specify 0 to create cuDevicePtr and 1 for cuArray, please note this option takes effect only if client wants to use CUDA code path ``` -------------------------------- ### Run NvOFFRUCSample with PNG Sequence Input Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Example command line for processing a sequence of PNG frames. The input path uses a wildcard pattern to identify the sequence. ```bash NvOFFRUCSample.exe --input=C:\fruc\wonders\rainbow_*.png --width=1920 --height=1080 --output=C:\fruc\outputdir ``` -------------------------------- ### Create build directory for NvFRUCSample (Windows) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Navigate to the NvFRUCSample directory and create a build directory. This is the first step in the Windows build process. ```bash cd C:/Users/TestPC/Downloads/OpticalFlowSDK/NvFRUCSample && mkdir build && cd build ``` -------------------------------- ### Navigate to NvOFFRUCSample binary directory (Windows 64-bit) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Change to the directory containing the 64-bit NvOFFRUCSample executable and its dependencies. ```bash C:/Users/TestPC/Downloads/OpticalFlowSDK/NvOFFRUC/NvOFFRUCSample/bin/win64 ``` -------------------------------- ### Interpolate intermediate frame with NvFRUCProcess Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/nvfruc-programming-guide/index.html Call NvFRUCProcess in a loop with consecutive input frames to get interpolated frames. Fill stInParams and stOutParams before each call. The first call returns the input frame itself; subsequent calls return interpolated frames at the requested timestamps. The function uses the current and cached previous frame. ```cpp NvFRUC_PROCESS_IN_PARAMS stInParams = { 0 }; NvFRUC_PROCESS_OUT_PARAMS stOutParams = { 0 }; status = NvFRUCProcess( hFRUC, &stInParams, &stOutParams); ``` -------------------------------- ### Create build directory for NvFRUCSample (Linux) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Navigate to the NvFRUCSample directory and create a build directory. This is the first step in the Linux build process. ```bash cd /home/Downloads/OpticalFlowSDK/NvFRUCSample && mkdir build && cd build ``` -------------------------------- ### Navigate to NvOFFRUCSample binary directory (Windows 32-bit) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Change to the directory containing the 32-bit NvOFFRUCSample executable and its dependencies. ```bash C:/Users/TestPC/Downloads/OpticalFlowSDK/NvOFFRUC/NvOFFRUCSample/bin/win32 ``` -------------------------------- ### Build NvOFTSample and NvOFTracker with CMake Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Commands to configure and build the NvOFTSample and NvOFTracker projects. Replace opencvDir and videocodecsdkroot with actual paths. ```bash cd /home/Downloads/OpticalFlowSDK/NvOFTracker && mkdir build && cd build ``` ```bash cmake -DOpenCV_DIR=opencvDir -DVIDEOCODEC_SDK_ROOT=videocodecsdkroot .. ``` ```bash cmake -DOpenCV_DIR=opencvDir -DVIDEOCODEC_SDK_ROOT=videocodecsdkroot -DTRT_ROOT=trtRoot .. ``` ```bash make install ``` -------------------------------- ### Navigate to NvOFFRUCSample binary directory (Linux) Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Change to the directory containing the Linux NvOFFRUCSample executable and its dependencies. ```bash /home/Downloads/OpticalFlowSDK/NvOFFRUC/NvOFFRUCSample/bin/ubuntu ``` -------------------------------- ### Load libNvFRUC.so on Linux using dlopen Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/nvfruc-programming-guide/index.html Loads the FRUC library on Linux using dlopen with RTLD_LAZY. ```cpp hDLL = dlopen("libNvFRUC.so", RTLD_LAZY); ``` -------------------------------- ### NvOFTSample help menu output Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Output of running NvOFTSample to display its help menu, showing mandatory and optional parameters. ```text 123456789 Mandatory Parameters -i Input video file -e TensorRT Engine file for Detector Optional Parameters: -o Output video file -fT Filename to dump tracked objects -dC Dump tracked objects to Console -sI Detection skip interval. Must be 0 or greater -g GPU Id on which the tracker needs to run. Default is 0 ``` -------------------------------- ### Vulkan Interface Initialization Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/nvofa-programming-guide/index.html This snippet documents the initialization steps for the Vulkan interface of the NVIDIA Optical Flow SDK, including the required extensions, features, and the parameters for NV_OF_INIT_PARAMS. ```APIDOC ## [Vulkan] Initialize Vulkan Interface for Optical Flow ### Description Initializes the Vulkan interface for the NVIDIA Optical Flow SDK. This involves loading the optical flow module, creating a Vulkan instance, selecting a physical device, creating a logical device, and populating the API function pointer list. ### Steps 1. Load the optical flow module `nvofapi.dll`/`nvofapi64.dll`. 2. Initialize Vulkan by creating a `VkInstance` with the `VK_KHR_get_physical_device_properties2` extension enabled, and ensure that `VkApplicationInfo::apiVersion` is set to `VK_API_VERSION_1_3` or higher. 3. Select a physical device which supports `VK_KHR_timeline_semaphore` and `VK_NV_optical_flow` extensions. 4. Query the queue family which supports `VK_QUEUE_OPTICAL_FLOW_BIT_NV`. 5. Create a logical device which enables: - `VK_KHR_timeline_semaphore`, `VK_NV_optical_flow` extensions. - `VkPhysicalDeviceTimelineSemaphoreFeatures`, `VkPhysicalDeviceSynchronization2Features` and `VkPhysicalDeviceOpticalFlowFeaturesNV` features. - A device queue for optical flow family along with other device queues for graphics, compute and/or other tasks. 6. Retrieve the address of exported function `NvOFAPICreateInstanceVk` from the loaded optical flow module. 7. Populate API function pointer list: - Call `NvOFAPICreateInstanceVk` to populate Vulkan API function pointer list. - The API accepts `NV_OF_API_VERSION` and a pointer `NV_OF_VK_API_FUNCTION_LIST*` to a memory block of function pointer list `NV_OF_VK_API_FUNCTION_LIST` that receives the API addresses. - The function pointers enable the access to the optical flow functionality. 8. Create optical flow instance: - Call `nvCreateOpticalFlowVk` and pass the `VkInstance`, `VkPhysicalDevice`, `VkDevice` created in the previous step. - The API returns an `NvOFHandle` which should be preserved and used throughout the session. 9. Call `NvOFGetCaps` to query the capabilities supported by the NVIDIA display driver and the GPU. 10. Call `NvOFInit` after filling up `NV_OF_INIT_PARAMS`. ### Parameters for NV_OF_INIT_PARAMS - **Image height** (uint32_t) - Required - The height of the image/video sequence for which the vectors will be evaluated. - **Image width** (uint32_t) - Required - The width of the image/video sequence for which the vectors will be evaluated. - **Flag to enable external hints** (bool) - Optional - NVOF API provides clients the flexibility for feeding external flow vectors which will be used as hints by NVOFA while performing the motion search. - **Output grid size** (NV_OF_OUTPUT_GRID_SIZE) - Required - Specify `NV_OF_INIT_PARAMS::outGridSize`. The flow vectors will be generated for the specified grid size. - **Hint grid size** (NV_OF_HINT_GRID_SIZE) - Optional - In case the client is using external hints, the hint grid size should be specified. - **Preset** (NV_OF_PRESET_TYPE) - Required - The NVOF API exposes three presets to trade quality vs performance. - **Operating mode** (NV_OF_MODE) - Required - The NVOFA can be used for generating optical flow vectors (which comprise X and Y components and is referred to as optical flow mode) or stereo vectors (which comprise only the X component and is referred to as stereo mode). - **Enable output cost** (bool) - Optional - Set this flag to generate confidence associated with the flow vectors. Higher cost value implies the flow vector to be less accurate and vice-versa. It is recommended to use the 8-bit cost (by allocating a buffer of type `NV_OF_BUFFER_FORMAT_UINT8`) instead of 32-bit cost as that saves CUDA bandwidth. - **Disparity Range** (uint32_t) - Optional - This is valid from Ampere and later GPUs and applicable in `NV_OF_MODE_STEREODISPARITY` mode. It specifies maximum stereo disparity range value. - **Enable ROI** (bool) - Optional - Set the flag to estimate the flow for a rectangular region within the image. Available on Ampere and later GPUs. - **Prediction direction** (NV_OF_PREDICTION_DIRECTION) - Required - Direction of the prediction. Supports either forward or forward-backward directions. - **Enable Global Flow** (bool) - Optional - Global flow estimation is enabled when this flag is set. - **Input buffer format** (NV_OF_BUFFER_FORMAT) - Required - `NV_OF_BUFFER_FORMAT` for the input buffer. ``` -------------------------------- ### Run CMake to configure NvOFTracker build Source: https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html Configure the build with CMake, specifying paths to OpenCV, TensorRT, and Video Codec SDK. Replace opencvDir, trtRoot, and videocodecsdkroot with actual paths. ```bash cmake -DOpenCV_DIR=opencvDir -DTRT_ROOT=trtRoot -DVIDEOCODEC_SDK_ROOT=videocodecsdkroot .. ```