### C/C++ Usage Example Source: https://docs.nvidia.com/vpi/html/algo_laplacian_pyramid_generator.html Provides a step-by-step guide for using the VPI Laplacian Pyramid Generator in C/C++. ```APIDOC ## C/C++ Usage ### 1. Initialization Phase 1. **Include Header:** Include the header that defines the Laplacian pyramid generator function. ```c #include ``` * `LaplacianPyramid.h`: Declares functions that handle Laplacian pyramids. 2. **Define Input Image:** Define the input image object. ```c VPIImage input = /*...*/; ``` * `VPIImage`: A handle to an image. **Definition:** Types.h:256 3. **Create Output Pyramid:** Create the output pyramid with the desired number of levels (e.g., 4) and scale factor (fixed at 0.5). ```c int32_t w, h; vpiImageGetSize(input, &w, &h); VPIPyramid output; vpiPyramidCreate(w, h, VPI_IMAGE_FORMAT_F32, 4, 0.5, 0, &output); ``` * `VPI_IMAGE_FORMAT_F32`: Single plane with one 32-bit floating point channel. **Definition:** ImageFormat.h:130 * `vpiImageGetSize`: Get the image dimensions in pixels. * `vpiPyramidCreate`: Create an empty image pyramid instance with the specified flags. * `VPIPyramid`: A handle to an image pyramid. **Definition:** Types.h:262 4. **Create Optional Gaussian Pyramid:** Optionally, create the Gaussian pyramid with the same dimensions as the Laplacian pyramid output. Its image format must be the same as the input image. ```c VPIImageFormat type; vpiImageGetFormat(input, &type); VPIPyramid gaussianPyr; vpiPyramidCreate(w, h, type, 4, 0.5, 0, &gaussianPyr); ``` * `VPIImageFormat`: Pre-defined image formats. **Definition:** ImageFormat.h:94 * `vpiImageGetFormat`: Get the image format. 5. **Create Stream:** Create the stream to which the algorithm is to be submitted for execution. ```c VPIStream stream; vpiStreamCreate(0, &stream); ``` * `VPIStream`: A handle to a stream. **Definition:** Types.h:250 * `vpiStreamCreate`: Create a stream instance. ### 2. Processing Phase 1. **Submit Algorithm:** Submit the algorithm to the stream, along with the input image, output pyramid, and optional Gaussian pyramid. The algorithm is executed by the CPU backend. ```c vpiSubmitLaplacianPyramidGenerator(stream, VPI_BACKEND_CPU, input, output, gaussianPyr, VPI_BORDER_CLAMP); ``` * `vpiSubmitLaplacianPyramidGenerator`: Computes the Laplacian pyramid from the input image. * `VPI_BACKEND_CPU`: CPU backend. **Definition:** Types.h:92 * `VPI_BORDER_CLAMP`: Border pixels are repeated indefinitely. **Definition:** Types.h:279 2. **Wait for Processing (Optional):** Optionally, wait until the processing is done. ```c vpiStreamSync(stream); ``` * `vpiStreamSync`: Blocks the calling thread until all submitted commands in this stream queue are done. ### 3. Cleanup Phase 1. **Free Resources:** Free resources held by the stream, the input image, the output pyramid, and the optional Gaussian pyramid. ```c vpiStreamDestroy(stream); vpiImageDestroy(input); vpiPyramidDestroy(output); vpiPyramidDestroy(gaussianPyr); ``` * `vpiStreamDestroy`: Destroy a stream instance and deallocate all HW resources. * `vpiImageDestroy`: Destroy an image instance. * `vpiPyramidDestroy`: Destroy an image pyramid instance as well as all resources it owns. For more information, see Laplacian Pyramid Generator in the "C API Reference" section of _VPI - Vision Programming Interface_. ``` -------------------------------- ### Install VPI Host Packages Source: https://docs.nvidia.com/vpi/installation.html Installs the VPI package, development headers, and sample applications on the host system. ```bash sudo apt install libnvvpi4 vpi4-dev vpi4-samples ``` -------------------------------- ### Install NVIDIA Container Toolkit Source: https://docs.nvidia.com/vpi/install_in_docker.html Install the NVIDIA container toolkit, ensuring version >= 1.15. ```bash curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \ | sudo tee /etc/apt/trusted.gpg.d/nvidia-container-toolkit-keyring.asc curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \ | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list cat <<-EOF | sudo tee /etc/apt/preferences.d/nvidia-container-toolkit-version Package: * Pin: release o=https://nvidia.github.io/libnvidia-container/stable Pin-Priority: 1001 EOF sudo apt-get update sudo apt-get install nvidia-container-toolkit ``` -------------------------------- ### Install pva-allow Source: https://docs.nvidia.com/vpi/install_in_docker.html Install the pva-allow package on the target system. ```bash sudo apt-get install pva-allow-2 ``` -------------------------------- ### Install apt Repository Utilities Source: https://docs.nvidia.com/vpi/installation.html Installs software-properties-common, which is needed to manage additional repositories. ```bash sudo apt install software-properties-common ``` -------------------------------- ### Install VPI in Docker Source: https://docs.nvidia.com/vpi/install_in_docker.html Install the VPI development package and samples within the Docker container. ```bash # Install VPI apt-get install libnvvpi3 vpi-dev vpi-samples ``` -------------------------------- ### Add VPI Repository and Install Host Components Source: https://docs.nvidia.com/vpi/html/installation.html Steps to add the NVIDIA VPI repository and install necessary host packages for VPI on Linux. This includes installing gnupg, software-properties-common, and the repository itself. ```bash sudo apt install gnupg sudo apt-key adv --fetch-key https://repo.download.nvidia.com/jetson/jetson-ota-public.asc ``` ```bash sudo apt install software-properties-common ``` ```bash sudo add-apt-repository 'deb https://repo.download.nvidia.com/jetson/x86_64/bionic r34.1 main' ``` ```bash sudo add-apt-repository 'deb https://repo.download.nvidia.com/jetson/x86_64/focal r34.1 main' ``` ```bash sudo apt update sudo apt install libnvvpi2 vpi2-dev vpi2-samples ``` ```bash sudo apt install vpi2-demos ``` -------------------------------- ### CMakeLists.txt Example Source: https://docs.nvidia.com/vpi/html/sample_cross_aarch64.html A basic CMakeLists.txt file that can be used with the provided toolchain file. ```APIDOC ## Basic CMakeLists.txt This is a minimal `CMakeLists.txt` file. Cross-compilation is handled by the toolchain file. ```cmake cmake_minimum_required(VERSION 3.5) # To cross-compile for aarch64-l4t target from x86, # pass -DCMAKE_TOOLCHAIN_FILE=Toolchain_aarch64_l4t.cmake ``` ``` -------------------------------- ### Python Usage Example Source: https://docs.nvidia.com/vpi/html/algo_laplacian_pyramid_generator.html Demonstrates how to create and use a Laplacian pyramid with the VPI module in Python. ```APIDOC ## Python Usage 1. **Import VPI module** ```python import vpi ``` 2. **(Optional) Create a 4-level VPI pyramid** This pyramid will store the Gaussian pyramid by-product. It uses the VPI image input to get its dimensions and format. ```python gaussian = vpi.Pyramid(input.size, input.format, 4) ``` 3. **Generate Laplacian and Gaussian pyramids** Returns the 4-level Laplacian pyramid created from the input image using the CUDA backend. It also returns the corresponding Gaussian pyramid. Input is a VPI image, and outputs are VPI pyramids. ```python with vpi.Backend.CUDA: output = input.laplacian_pyramid(4, out_gaussian=gaussian) ``` ``` -------------------------------- ### Install VPI Python Packages and Numpy Source: https://docs.nvidia.com/vpi/install_in_docker.html Execute these commands to install the required VPI Python packages and the numpy library. This setup is necessary for running VPI-based applications and samples inside a Docker container. ```bash apt-get install python3-pip ``` ```bash python3.12 -m pip install numpy ``` -------------------------------- ### Install VPI Samples Source: https://docs.nvidia.com/vpi/html/samples.html Copies VPI sample applications to your home directory for development. Ensure you have write access to the target directory. ```bash vpi2_install_samples.sh $HOME ``` -------------------------------- ### Initialize VPI Backend and Quality Settings (Python) Source: https://docs.nvidia.com/vpi/sample_optflow_dense.html Sets up the VPI backend and optical flow quality based on parsed command-line arguments. Asserts that the backend is 'ofa'. ```Python assert args.backend == 'ofa' if args.backend == 'ofa': backend = vpi.Backend.OFA if args.quality == "low": quality = vpi.OptFlowQuality.LOW elif args.quality == "medium": quality = vpi.OptFlowQuality.MEDIUM else: assert args.quality == "high" quality = vpi.OptFlowQuality.HIGH ``` -------------------------------- ### Initialize Temporal Noise Reduction Parameters Source: https://docs.nvidia.com/vpi/html/sample_tnr.html Initializes a VPITNRParams structure with default values for the vpiSubmitTemporalNoiseReduction function. Use this to get a starting point for TNR configuration. ```cpp VPIStatus vpiInitTemporalNoiseReductionParams(VPITNRParams *params) ``` -------------------------------- ### Main Application Logic Source: https://docs.nvidia.com/vpi/sample_stereo.html Sets up the VPI backend, confidence type, and processing parameters based on parsed arguments. Prints configuration details if verbose mode is enabled. ```Python def main(): args = process_arguments() if args.backend == 'cuda': backend = vpi.Backend.CUDA elif args.backend == 'ofa': backend = vpi.Backend.OFA elif args.backend == 'ofa-pva-vic': backend = vpi.Backend.OFA|vpi.Backend.PVA|vpi.Backend.VIC else: raise ValueError(f'E Invalid backend: {args.backend}') conftype = None if args.conf_type == 'best': conftype = vpi.ConfidenceType.INFERENCE if args.backend == 'ofa-pva-vic' else vpi.ConfidenceType.ABSOLUTE elif args.conf_type == 'absolute': conftype = vpi.ConfidenceType.ABSOLUTE elif args.conf_type == 'relative': conftype = vpi.ConfidenceType.RELATIVE elif args.conf_type == 'inference': conftype = vpi.ConfidenceType.INFERENCE else: raise ValueError(f'E Invalid confidence type: {args.conf_type}') scaleFactor = args.scale_factor minDisparity = args.min_disparity maxDisparity = args.max_disparity includeDiagonals = not args.skip_diagonal numPasses = args.num_passes calcConf = not args.skip_confidence downscale = args.downscale windowSize = args.window_size quality = 6 if args.verbose: print(f'I Backend: {backend}\nI Left image: {args.left}\nI Right image: {args.right}\n' f'I Disparities (min, max): {(minDisparity, maxDisparity)}\n' f'I Input size scale factor: {scaleFactor}\nI Output downscale factor: {downscale}\n' f'I Window size: {windowSize}\nI Quality: {quality}\n' f'I Calculate confidence: {calcConf}\nI Confidence threshold: {args.conf_threshold}\n' f'I Confidence type: {conftype}\nI Uniqueness ratio: {args.uniqueness}\n' f'I Penalty P1: {args.p1}\nI Penalty P2: {args.p2}\nI Adaptive P2 alpha: {args.p2_alpha}\n' f'I Include diagonals: {includeDiagonals}\nI Number of passes: {numPasses}\n') ``` -------------------------------- ### Install VPI Samples Source: https://docs.nvidia.com/vpi/samples.html Copies VPI sample applications to a user-writable directory. Use this command to easily access and modify the provided samples. ```bash vpi4_install_samples.sh $HOME ``` -------------------------------- ### Get Channel Type Source: https://docs.nvidia.com/vpi/PixelType_8h_source.html Retrieves the pixel type of a specific channel within a multi-channel VPI pixel type. Channels are typically indexed starting from 0. ```c VPI_PUBLIC VPIPixelType vpiPixelTypeGetChannelType(VPIPixelType type, int channel); ``` -------------------------------- ### Initialize VPI Objects and Parse Arguments Source: https://docs.nvidia.com/vpi/html/sample_optflow_dense.html Sets up VPI stream, images, and parses command-line arguments for backend, input video, and quality. ```cpp int main(int argc, char *argv[]) { // OpenCV image that will be wrapped by a VPIImage. // Define it here so that it's destroyed *after* wrapper is destroyed cv::Mat cvPrevFrame, cvCurFrame; // VPI objects that will be used VPIStream stream = NULL; VPIImage imgPrevFramePL = NULL; VPIImage imgPrevFrameTmp = NULL; VPIImage imgPrevFrameBL = NULL; VPIImage imgCurFramePL = NULL; VPIImage imgCurFrameTmp = NULL; VPIImage imgCurFrameBL = NULL; VPIImage imgMotionVecBL = NULL; VPIPayload payload = NULL; int retval = 0; try { if (argc != 4) { throw std::runtime_error(std::string("Usage: ") + argv[0] + " "); } // Parse input parameters std::string strBackend = argv[1]; std::string strInputVideo = argv[2]; std::string strQuality = argv[3]; VPIOpticalFlowQuality quality; if (strQuality == "low") { quality = VPI_OPTICAL_FLOW_QUALITY_LOW; } else if (strQuality == "medium") { quality = VPI_OPTICAL_FLOW_QUALITY_MEDIUM; } else if (strQuality == "high") { quality = VPI_OPTICAL_FLOW_QUALITY_HIGH; } else { throw std::runtime_error("Unknown quality provided"); } VPIBackend backend; if (strBackend == "nvenc") { backend = VPI_BACKEND_NVENC; } else { throw std::runtime_error("Backend '" + strBackend + "' not recognized, it must be nvenc."); } // Load the input video cv::VideoCapture invid; if (!invid.open(strInputVideo)) { throw std::runtime_error("Can't open '" + strInputVideo + "'"); } // Create the stream where processing will happen. We'll use user-provided backend // for Optical Flow, and CUDA/VIC for image format conversions. CHECK_STATUS(vpiStreamCreate(backend | VPI_BACKEND_CUDA | VPI_BACKEND_VIC, &stream)); // Fetch the first frame if (!invid.read(cvPrevFrame)) { throw std::runtime_error("Cannot read frame from input video"); } // Create the previous and current frame wrapper using the first frame. This wrapper will // be set to point to every new frame in the main loop. CHECK_STATUS(vpiImageCreateWrapperOpenCVMat(cvPrevFrame, 0, &imgPrevFramePL)); CHECK_STATUS(vpiImageCreateWrapperOpenCVMat(cvPrevFrame, 0, &imgCurFramePL)); // Define the image formats we'll use throughout this sample. VPIImageFormat imgFmt = VPI_IMAGE_FORMAT_NV12_ER; VPIImageFormat imgFmtBL = VPI_IMAGE_FORMAT_NV12_ER_BL; int32_t width = cvPrevFrame.cols; int32_t height = cvPrevFrame.rows; // Create Dense Optical Flow payload to be executed on the given backend CHECK_STATUS(vpiCreateOpticalFlowDense(backend, width, height, imgFmtBL, quality, &payload)); // The Dense Optical Flow on NVENC backend expects input to be in block-linear format. // Since Convert Image Format algorithm doesn't currently support direct BGR // pitch-linear (from OpenCV) to NV12 block-linear conversion, it must be done in two // passes, first from BGR/PL to NV12/PL using CUDA, then from NV12/PL to NV12/BL using VIC. // The temporary image buffer below will store the intermediate NV12/PL representation. CHECK_STATUS(vpiImageCreate(width, height, imgFmt, 0, &imgPrevFrameTmp)); CHECK_STATUS(vpiImageCreate(width, height, imgFmt, 0, &imgCurFrameTmp)); // Now create the final block-linear buffer that'll be used as input to the // algorithm. CHECK_STATUS(vpiImageCreate(width, height, imgFmtBL, 0, &imgPrevFrameBL)); CHECK_STATUS(vpiImageCreate(width, height, imgFmtBL, 0, &imgCurFrameBL)); // Motion vector image width and height, align to be multiple of 4 int32_t mvWidth = (width + 3) / 4; int32_t mvHeight = (height + 3) / 4; // The output video will be heatmap of motion vector image int fourcc = cv::VideoWriter::fourcc('M', 'P', 'E', 'G'); double fps = invid.get(cv::CAP_PROP_FPS); cv::VideoWriter outVideo("denseoptflow_mv_" + strBackend + ".mp4", fourcc, fps, cv::Size(mvWidth, mvHeight)); ``` -------------------------------- ### Get Swizzle Channels Source: https://docs.nvidia.com/vpi/html/group__VPI__DataLayout.html Use this function to retrieve the channel order from a VPI swizzle configuration. For example, given VPI_SWIZZLE_YZWX, it returns VPI_CHANNEL_Y, VPI_CHANNEL_Z, VPI_CHANNEL_W, and VPI_CHANNEL_X. ```c #include void vpiSwizzleGetChannels(VPISwizzle _swizzle, VPIChannel *_channels) ``` -------------------------------- ### VPI Perspective Warp C++ Example Source: https://docs.nvidia.com/vpi/html/sample_perspwarp.html This C++ code demonstrates perspective warping using VPI. It includes error checking for VPI status, matrix multiplication for transformations, and setup for input/output videos. ```cpp #include #include #include #include #include #include #include #include #include #include #include #include // for memset #include #include #include #include #define CHECK_STATUS(STMT) \ do \ {\ VPIStatus status = (STMT); \ if (status != VPI_SUCCESS) \ { \ char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \ vpiGetLastStatusMessage(buffer, sizeof(buffer)); \ std::ostringstream ss; ss << vpiStatusGetName(status) << ": " << buffer; throw std::runtime_error(ss.str()); \ } \ } while (0); static void MatrixMultiply(VPIPerspectiveTransform &r, const VPIPerspectiveTransform &a, const VPIPerspectiveTransform &b) { for (int i = 0; i < 3; ++i) { for (int j = 0; j < 3; ++j) { r[i][j] = a[i][0] * b[0][j]; for (int k = 1; k < 3; ++k) { r[i][j] += a[i][k] * b[k][j]; } } } } int main(int argc, char *argv[]) { // OpenCV image that will be wrapped by a VPIImage. // Define it here so that it's destroyed *after* wrapper is destroyed cv::Mat cvFrame; // VPI objects that will be used VPIStream stream = NULL; VPIImage imgInput = NULL, imgOutput = NULL; VPIImage frameBGR = NULL; int retval = 0; try { // ============================= // Parse command line parameters if (argc != 3) { throw std::runtime_error(std::string("Usage: ") + argv[0] + " "); } std::string strBackend = argv[1]; std::string strInputVideo = argv[2]; // Now parse the backend VPIBackend backend; if (strBackend == "cpu") { backend = VPI_BACKEND_CPU; } else if (strBackend == "cuda") { backend = VPI_BACKEND_CUDA; } else if (strBackend == "vic") { backend = VPI_BACKEND_VIC; } else { throw std::runtime_error("Backend '" + strBackend + "' not recognized, it must be either cpu, cuda or vic."); } // =============================== // Prepare input and output videos // Load the input video cv::VideoCapture invid; ``` -------------------------------- ### Main Function for KLT Tracking Example Source: https://docs.nvidia.com/vpi/html/sample_klt_tracker.html Sets up and runs the KLT tracking pipeline. It handles command-line arguments for backend selection, input video, and bounding box descriptions, initializes VPI and OpenCV objects, and processes video frames. ```cpp int main(int argc, char *argv[]) { // OpenCV image that will be wrapped by a VPIImage. // Define it here so that it's destroyed *after* wrapper is destroyed cv::Mat cvTemplate, cvReference; // Arrays that will store our input bboxes and predicted transform. VPIArray inputBoxList = NULL, inputPredList = NULL; // Other VPI objects that will be used VPIStream stream = NULL; VPIArray outputBoxList = NULL; VPIArray outputEstimList = NULL; VPIPayload klt = NULL; VPIImage imgReference = NULL; VPIImage imgTemplate = NULL; int retval = 0; try { if (argc != 4) { throw std::runtime_error(std::string("Usage: ") + argv[0] + " "); } std::string strBackend = argv[1]; std::string strInputVideo = argv[2]; std::string strInputBBoxes = argv[3]; // Load the input video cv::VideoCapture invid; if (!invid.open(strInputVideo)) { throw std::runtime_error("Can't open '" + strInputVideo + "'"); } // Open the output video for writing using input's characteristics int w = invid.get(cv::CAP_PROP_FRAME_WIDTH); int h = invid.get(cv::CAP_PROP_FRAME_HEIGHT); int fourcc = cv::VideoWriter::fourcc('M', 'P', 'E', 'G'); double fps = invid.get(cv::CAP_PROP_FPS); cv::VideoWriter outVideo("klt_" + strBackend + ".mp4", fourcc, fps, cv::Size(w, h)); if (!outVideo.isOpened()) { throw std::runtime_error("Can't create output video"); } // Load the bounding boxes // Format is: // Important assumption: bboxes must be sorted with increasing frame numbers. ``` -------------------------------- ### Build the Application with CMake and Make Source: https://docs.nvidia.com/vpi/html/tutorial_c.html Execute these commands in the project directory to build the application. Ensure CMake and Make are installed and the project is set up correctly. ```bash cmake . make ``` -------------------------------- ### C++ Background Subtractor Initialization and Processing Source: https://docs.nvidia.com/vpi/html/sample_background_subtractor.html Initializes and uses the VPI Background Subtractor in C++. This example demonstrates error checking with a CHECK_STATUS macro and setting up OpenCV and VPI objects for video processing. Ensure VPI and OpenCV are correctly installed and linked. ```C++ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define CHECK_STATUS(STMT) \ do \ {\ VPIStatus status = (STMT); \ if (status != VPI_SUCCESS) \ { \ char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \ vpiGetLastStatusMessage(buffer, sizeof(buffer)); \ std::ostringstream ss; \ ss << vpiStatusGetName(status) << ": " << buffer; \ throw std::runtime_error(ss.str()); \ } \ } while (0); int main(int argc, char *argv[]) { // OpenCV image that will be wrapped by a VPIImage. // Define it here so that it's destroyed *after* wrapper is destroyed cv::Mat cvCurFrame; // VPI objects that will be used VPIStream stream = NULL; VPIImage imgCurFrame = NULL; VPIImage bgimage = NULL; VPIImage fgmask = NULL; VPIPayload payload = NULL; int retval = 0; try { if (argc != 3) { throw std::runtime_error(std::string("Usage: ") + argv[0] + " "); } // Parse input parameters std::string strBackend = argv[1]; std::string strInputVideo = argv[2]; VPIBackend backend; if (strBackend == "cpu") { backend = VPI_BACKEND_CPU; } else if (strBackend == "cuda") { backend = VPI_BACKEND_CUDA; } else { throw std::runtime_error("Backend '" + strBackend + "' not recognized."); } // Load the input video cv::VideoCapture invid; if (!invid.open(strInputVideo)) { throw std::runtime_error("Can't open '" + strInputVideo + "'"); } int32_t width = invid.get(cv::CAP_PROP_FRAME_WIDTH); int32_t height = invid.get(cv::CAP_PROP_FRAME_HEIGHT); // Create the stream where processing will happen. We'll use user-provided backend. CHECK_STATUS(vpiStreamCreate(backend, &stream)); // Create background subtractor payload to be executed on the given backend // OpenCV delivers us BGR8 images, so the algorithm is configured to accept that. CHECK_STATUS(vpiCreateBackgroundSubtractor(backend, width, height, VPI_IMAGE_FORMAT_BGR8, &payload)); // Create foreground image CHECK_STATUS(vpiImageCreate(width, height, VPI_IMAGE_FORMAT_U8, 0, &fgmask)); // Create background image CHECK_STATUS(vpiImageCreate(width, height, VPI_IMAGE_FORMAT_BGR8, 0, &bgimage)); int fourcc = cv::VideoWriter::fourcc('M', 'P', 'E', 'G'); ``` -------------------------------- ### Install Dependencies on RedHat/Fedora Source: https://docs.nvidia.com/vpi/rift.html Installs necessary packages for VPI on RedHat 9, Fedora 40, or CentOS 9 systems. Ensure these are installed before proceeding with VPI installation. ```bash yum install xz libglvnd-egl python3.10 ``` -------------------------------- ### Initialize Video and VPI Resources Source: https://docs.nvidia.com/vpi/html/sample_perspwarp.html Opens input video, sets up output video writer, and allocates VPI resources like streams and images. Ensure input video is opened successfully and output video can be created. ```cpp 130 if (!invid.open(strInputVideo)) 131 { 132 throw std::runtime_error("Can't open '" + strInputVideo + "'"); 133 } 134 135 // Open the output video for writing using input's characteristics 136 int w = invid.get(cv::CAP_PROP_FRAME_WIDTH); 137 int h = invid.get(cv::CAP_PROP_FRAME_HEIGHT); 138 int fourcc = cv::VideoWriter::fourcc('M', 'P', 'E', 'G'); 139 double fps = invid.get(cv::CAP_PROP_FPS); 140 141 cv::VideoWriter outVideo("perspwarp_" + strBackend + ".mp4", fourcc, fps, cv::Size(w, h)); 142 if (!outVideo.isOpened()) 143 { 144 throw std::runtime_error("Can't create output video"); 145 } 146 147 // ================================= 148 // Allocate all VPI resources needed 149 150 // Create the stream for the given backend. We'll be using CUDA for image format conversion. 151 CHECK_STATUS(vpiStreamCreate(backend | VPI_BACKEND_CUDA, &stream)); 152 153 CHECK_STATUS(vpiImageCreate(w, h, VPI_IMAGE_FORMAT_NV12_ER, 0, &imgInput)); 154 CHECK_STATUS(vpiImageCreate(w, h, VPI_IMAGE_FORMAT_NV12_ER, 0, &imgOutput)); 155 156 VPIPerspectiveTransform xform; 157 memset(&xform, 0, sizeof(xform)); ``` -------------------------------- ### Install VPI Python Bindings Source: https://docs.nvidia.com/vpi/html/installation.html Install Python 3.8 or Python 3.9 bindings for VPI. Ensure the numpy module is installed separately. ```bash sudo apt install python3.8-vpi2 ``` ```bash sudo apt install python3.9-vpi2 ``` -------------------------------- ### Install Python 3.12 VPI Bindings Source: https://docs.nvidia.com/vpi/installation.html Installs the VPI Python 4 bindings for Python 3.12. The numpy module must also be installed separately. ```bash sudo apt install python3.12-vpi4 ``` -------------------------------- ### Setting VPI Backend Source: https://docs.nvidia.com/vpi/html/python/build/vpi.Backend.html Demonstrates how to set a backend for VPI algorithms, either directly or using a 'with' statement. ```APIDOC ## Setting VPI Backend ### Description Algorithms require a backend for execution. You can set the backend in two ways: 1. As an argument to the algorithm call. 2. Using a `with` statement, which defines the backend for all algorithms within that statement. ### Example 1: Setting backend as an argument ```python # Assuming 'algorithm' is a VPI algorithm function result = algorithm(..., backend=vpi.Backend.CUDA) ``` ### Example 2: Setting backend with a `with` statement ```python with vpi.Backend.CUDA: # All algorithms called here will use the CUDA backend result1 = algorithm1(...) result2 = algorithm2(...) ``` ``` -------------------------------- ### Install Python 3.10 VPI Bindings Source: https://docs.nvidia.com/vpi/installation.html Installs the VPI Python 4 bindings for Python 3.10. The numpy module must also be installed separately. ```bash sudo apt install python3.10-vpi4 ``` -------------------------------- ### Run the vpi_blur Application Source: https://docs.nvidia.com/vpi/html/tutorial_c.html After building, run the executable with an image file as an argument. Replace with the actual path to your image. ```bash ./vpi_blur ``` -------------------------------- ### Run VPI Background Subtractor Sample (C++) Source: https://docs.nvidia.com/vpi/html/sample_background_subtractor.html Example command to run the VPI background subtractor sample using the CPU backend with a specified input video. ```bash ./vpi_sample_14_background_subtractor cpu ../assets/pedestrians.mp4 ``` -------------------------------- ### Run Image View Sample Application (C++) Source: https://docs.nvidia.com/vpi/html/sample_image_view.html Executes the Image View sample application using C++. Requires the input image file as a command-line argument. ```bash ./vpi_sample_15_image_view ../assets/kodim08.png ``` -------------------------------- ### Initialize VPI Stream (C/C++) Source: https://docs.nvidia.com/vpi/algo_imageconv.html This C/C++ snippet demonstrates the initialization phase for image format conversion. It includes including the necessary header, creating a stream, defining input and output images, and getting image dimensions. Ensure proper resource management by destroying them later. ```c++ #include VPIStream stream; vpiStreamCreate(0, &stream); VPIImage input; vpiImageCreate(w, h, VPI_IMAGE_FORMAT_NV12_ER, 0, &input); int w, h; vpiImageGetSize(input, &w, &h); VPIImage output; vpiImageCreate(w, h, VPI_IMAGE_FORMAT_S16, 0, &output); ``` -------------------------------- ### C/C++: Initialize Mix Channels Algorithm Source: https://docs.nvidia.com/vpi/algo_mix_channels.html This C/C++ snippet covers the initialization phase for the Mix Channels algorithm. It includes header inclusion, input/output image creation, and stream creation. Ensure VPIImage and VPIStream types are correctly handled. ```c++ #include VPIImage input = /*...*/; int32_t w, h; vpiImageGetSize(input, &w, &h); VPIImage outputs[3]; for (int i = 0; i < 3; ++i) { vpiImageCreate(w, h, VPI_IMAGE_FORMAT_RGB8, 0, &outputs[i]); } VPIStream stream; vpiStreamCreate(0, &stream); ``` -------------------------------- ### Configure Library Path and Install pva-allow-2 in Docker Source: https://docs.nvidia.com/vpi/install_in_docker.html Add CUDA packages to the library path and install pva-allow-2 within the Docker container. The pva-allow-2 installation includes a workaround for potential post-installation script issues. ```bash # Add CUDA packages to library path export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-13-0/targets/aarch64-linux/lib/ # This is a temporary workaround required to install pva-allow-2 in docker which will not be necessary next release apt-get install pva-allow-2 || true rm /var/lib/dpkg/info/pva-allow-2.post* dpkg --configure pva-allow-2 ``` -------------------------------- ### Run VPI Rescale Application (C++) Source: https://docs.nvidia.com/vpi/html/sample_rescale.html Example of how to run the VPI Rescale application using the CUDA backend. Ensure you have the necessary assets available. ```bash ./vpi_sample_04_rescale cuda ../assets/kodim08.png ``` -------------------------------- ### Check NVIDIA Driver Installation Source: https://docs.nvidia.com/vpi/install_in_docker.html Verify that a working NVIDIA CUDA driver is installed on the device. ```bash nvidia-smi ``` -------------------------------- ### Initialize LK Parameters (C/C++) Source: https://docs.nvidia.com/vpi/html/algo_optflow_lk.html C/C++ example for initializing the VPIOpticalFlowPyrLKParams structure with default values to configure the LK tracking process. ```C/C++ VPIOpticalFlowPyrLKParams lkParams; vpiInitOpticalFlowPyrLKParams(&lkParams); ``` -------------------------------- ### CMake Toolchain File Example Source: https://docs.nvidia.com/vpi/html/sample_cross_aarch64.html Example CMake toolchain file for cross-compiling for aarch64 Linux. ```APIDOC ## CMake Toolchain File for aarch64 Linux This file configures CMake for cross-compilation targeting aarch64 Linux. ```cmake set(CMAKE_SYSTEM_NAME Linux) set(CMAKE_SYSTEM_PROCESSOR aarch64) set(target_arch aarch64-linux-gnu) set(CMAKE_LIBRARY_ARCHITECTURE ${target_arch} CACHE STRING "" FORCE) # Configure CMake to look for libraries, include directories, and packages # inside the target root prefix. set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY) set(CMAKE_FIND_ROOT_PATH "/usr/${target_arch}") # Avoid strict compiler checks that fail during cross-compilation set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY) # Specify the toolchain programs find_program(CMAKE_C_COMPILER ${target_arch}-gcc) find_program(CMAKE_CXX_COMPILER ${target_arch}-g++) if(NOT CMAKE_C_COMPILER OR NOT CMAKE_CXX_COMPILER) message(FATAL_ERROR "Can't find suitable C/C++ cross compiler for ${target_arch}") endif() set(CMAKE_AR ${target_arch}-ar CACHE FILEPATH "" FORCE) set(CMAKE_RANLIB ${target_arch}-ranlib) set(CMAKE_LINKER ${target_arch}-ld) # Allow undefined shared libraries during linking set(CMAKE_EXE_LINKER_FLAGS_INIT "-Wl,--allow-shlib-undefined") # Instruct nvcc to use our cross-compiler set(CMAKE_CUDA_FLAGS "-ccbin ${CMAKE_CXX_COMPILER} -Xcompiler -fPIC" CACHE STRING "" FORCE) ``` ``` -------------------------------- ### Run VPI Background Subtractor Sample (Python) Source: https://docs.nvidia.com/vpi/html/sample_background_subtractor.html Example command to run the VPI background subtractor sample using Python with the CPU backend and a specified input video. ```bash python3 main.py cpu ../assets/pedestrians.mp4 ``` -------------------------------- ### C/C++: Initialize Remap Algorithm Resources Source: https://docs.nvidia.com/vpi/algo_remap.html Sets up the necessary VPI resources for the Remap algorithm in C/C++. This includes including headers, defining input and output images, creating a stream, and allocating a dense warp map. Ensure VPI library is linked. ```c++ #include #include VPIImage input = /*...*/; VPIImage output; int32_t w, h; vpiImageGetSize(input, &w, &h); VPIImageFormat type; vpiImageGetFormat(input, &type); vpiImageCreate(w, h, type, 0, &output); VPIStream stream; vpiStreamCreate(0, &stream); VPIWarpMap map; memset(&map, 0, sizeof(map)); map.grid.numHorizRegions = 1; map.grid.numVertRegions = 1; map.grid.regionWidth[0] = w; map.grid.regionHeight[0] = h; map.grid.horizInterval[0] = 1; map.grid.vertInterval[0] = 1; vpiWarpMapAllocData(&map); ``` -------------------------------- ### Install VPI Cross-Compilation Packages Source: https://docs.nvidia.com/vpi/html/installation.html Optional step to install development packages for cross-compilation targeting the aarch64-l4t architecture. ```bash sudo apt install vpi2-cross-aarch64-l4t ``` -------------------------------- ### Install Cross-Compilation Packages Source: https://docs.nvidia.com/vpi/installation.html Installs development packages for cross-compiling VPI applications targeting the aarch64-l4t architecture on an x86_64 host. ```bash sudo apt install vpi4-cross-aarch64-l4t ``` -------------------------------- ### VPI KLT Feature Tracker Usage Example Source: https://docs.nvidia.com/vpi/html/algo_klt_tracker.html Example usage of the VPI KLT Feature Tracker in C++ and Python. ```APIDOC ## Usage Language: C++ Python 1. Import VPI module ```python import vpi ``` 2. (optional) Define a function to update the input bounding boxes and predictions for the next tracking given the output bounding boxes and estimations from the previous tracking. This custom update is similar to the default update, added here as an example on how to define a custom update. ``` -------------------------------- ### Run VPI Sample Application Source: https://docs.nvidia.com/vpi/html/sample_cross_aarch64.html Execute the VPI sample application, specifying the backend for processing. The backend can be 'cpu', 'cuda', or 'pva'. ```bash ./vpi_sample_08_cross_aarch64_l4t ``` -------------------------------- ### Get Image Dimensions Source: https://docs.nvidia.com/vpi/html/Image_8h_source.html Retrieves the width and height of a VPIImage object in pixels. Use this to get the image's resolution. ```c VPIStatus vpiImageGetSize(VPIImage img, int32_t *width, int32_t *height) ``` -------------------------------- ### Run DCF Tracker Sample Application Source: https://docs.nvidia.com/vpi/sample_dcf_tracker.html Example of how to run the DCF Tracker sample application using the CUDA backend. This command specifies the input video and bounding box files. ```bash ./vpi_sample_19_dcf_tracker cuda ../assets/pedestrians.mp4 ../assets/pedestrians_bboxes.txt ``` -------------------------------- ### C/C++: Initialize Input and Output Images Source: https://docs.nvidia.com/vpi/algo_sep_convolution.html Prepare input and output VPIImage objects for convolution. This involves getting the size and format of the input image to create a compatible output image. ```c #include VPIImage input = /*...*/; VPIImage output; int32_t w, h; vpiImageGetSize(input, &w, &h); VPIImageFormat type; vpiImageGetFormat(input, &type); vpiImageCreate(w, h, type, 0, &output); ``` -------------------------------- ### Update and Install VPI Package Source: https://docs.nvidia.com/vpi/installation.html Use these commands to update your package list and install the main VPI package and its development files. ```bash sudo apt-get update sudo apt-get install libnvvpi4 ``` -------------------------------- ### C++ Implementation Example Source: https://docs.nvidia.com/vpi/sample_cross_aarch64.html Example C++ code demonstrating the usage of VPI for image processing, specifically applying a box filter. ```APIDOC ## C++ Implementation Example ### Description This C++ code snippet demonstrates how to initialize VPI, create images, submit a box filter operation, and retrieve the results. It includes error handling and resource cleanup. ### Method N/A (This is a C++ program, not an API endpoint) ### Endpoint N/A ### Parameters #### Command Line Arguments - **argv[1]** (string) - Specifies the backend to use: "cpu", "pva", or "cuda". ### Request Body N/A ### Request Example ```bash ./your_program_name cuda ``` ### Response #### Success Response - A PGM image file named `boxfiltered_.pgm` is created in the current directory containing the filtered output. #### Response Example N/A ### Error Handling - Throws `std::runtime_error` for invalid command-line arguments or backend recognition issues. - Uses `CHECK_STATUS` macro for VPI function call error checking. ### Key VPI Functions Used - `vpiStreamCreate`: Creates a VPI stream. - `vpiImageCreateWrapper`: Wraps existing image data into a VPIImage. - `vpiImageCreate`: Creates a new VPI image. - `vpiSubmitBoxFilter`: Submits the box filter operation. - `vpiStreamSync`: Synchronizes the stream to wait for operations to complete. - `vpiImageLockData`: Locks an image to access its data. - `vpiImageUnlock`: Unlocks an image. - `vpiImageDestroy`: Destroys a VPI image. - `vpiStreamDestroy`: Destroys a VPI stream. - `vpiGetLastStatusMessage`: Retrieves the last VPI status message. - `vpiStatusGetName`: Gets the name of a VPI status code. ``` -------------------------------- ### Run Image View Sample Application (Python) Source: https://docs.nvidia.com/vpi/html/sample_image_view.html Executes the Image View sample application using Python. Requires the input image file as a command-line argument. ```python python3 main.py ../assets/kodim08.png ``` -------------------------------- ### Install Dependencies on SUSE Source: https://docs.nvidia.com/vpi/rift.html Installs required packages for VPI on SUSE Linux Enterprise Server 6. Use this command if you are on a SUSE-based system. ```bash zypper install xz Mesa-libEGL1 python310 ``` -------------------------------- ### Install VPI Dependencies in Docker Source: https://docs.nvidia.com/vpi/install_in_docker.html Inside the Docker container, update package lists and install necessary dependencies for VPI, including GNUPG and software-properties-common. ```bash # Install packages required by add-apt-repository apt-get update apt-get install gnupg software-properties-common # Add Jetson public APT repository apt-key adv --fetch-key https://repo.download.nvidia.com/jetson/jetson-ota-public.asc add-apt-repository 'deb https://repo.download.nvidia.com/jetson/common r36.4 main' # Install VPI depedencies apt-get update apt-get install libnpp-13-0 libcufft-13-0 cuda-cudart-13-0 libegl1-mesa ```