### Install Library Target Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Installs the 'vsmigx' library to the appropriate system directories. ```cmake install(TARGETS vsmigx LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} ) ``` -------------------------------- ### Installation Configuration Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/CMakeLists.txt Installs the vsncnn target to the appropriate library and runtime directories. ```cmake install(TARGETS vsncnn LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} ) ``` -------------------------------- ### CMake Minimum Version and Project Setup Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Sets the minimum required CMake version and defines the project name, version, and languages. This is a standard starting point for any CMake project. ```cmake cmake_minimum_required(VERSION 3.20) project(vs-trt VERSION 3.1 LANGUAGES CXX) ``` -------------------------------- ### Project and Version Setup Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Sets the minimum CMake version and defines the project name and version. ```cmake cmake_minimum_required(VERSION 3.20) project(vs-ort VERSION 3.0 LANGUAGES CXX) ``` -------------------------------- ### Installation Rules Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Defines installation rules for the vsort target, specifying library and runtime destinations. ```cmake install(TARGETS vsort LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} ) ``` -------------------------------- ### Basic CMake Project Setup Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/CMakeLists.txt Sets the minimum required CMake version and defines the project name and languages. ```cmake cmake_minimum_required(VERSION 3.20) project(vs-ncnn VERSION 3.0 LANGUAGES CXX) ``` -------------------------------- ### Install TrtExec Executable Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/trtexec/CMakeLists.txt Installs the trtexec executable to the runtime destination directory. ```cmake install(TARGETS trtexec RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}) ``` -------------------------------- ### Tactic Sources Example Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/README.md Example of how to specify tactic sources to disable cuDNN and enable cuBLAS. ```bash --tacticSources=-CUDNN,+CUBLAS ``` -------------------------------- ### Installing vstrt Target Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Installs the `vstrt` target, specifying the destination directories for libraries and runtime components. This makes the built plugin available for use. ```cmake install(TARGETS vstrt LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} ) ``` -------------------------------- ### Installing the vsov Target Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Installs the vsov library to the appropriate library and runtime directories. ```cmake install(TARGETS vsov LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} ) ``` -------------------------------- ### Installing vstrt_rtx Target Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Installs the `vstrt_rtx` target, specifying the destination directories for libraries and runtime components. This makes the built plugin available for use. ```cmake install(TARGETS vstrt_rtx LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} ) ``` -------------------------------- ### CPU Backend Video Sorting Source: https://github.com/amusementclub/vs-mlrt/wiki/vsort Example of using the CPU backend for video sorting with an ONNX model. ```python src = core.std.BlankClip(width=1920, height=1080, format=vs.RGBS) flt = core.ort.Model(src, "upconv_7_anime_style_art_rgb_scale2.0x.onnx") ``` -------------------------------- ### Flexible Output Property Example Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/README.md Demonstrates how to use the `flexible_output_prop` argument with `core.trt.Model` to handle models with an arbitrary number of output planes. The output clip and number of planes are extracted, and then individual planes are converted to clips. ```python from typing import TypedDict class Output(TypedDict): clip: vs.VideoNode num_planes: int prop = "planes" # arbitrary non-empty string output = core.trt.Model(src, engine_path, flexible_output_prop=prop) # type: Output clip = output["clip"] num_planes = output["num_planes"] output_planes = [ clip.std.PropToClip(prop=f"{prop}{i}") for i in range(num_planes) ] # type: list[vs.VideoNode] ``` -------------------------------- ### Version Information from Git Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Executes a Git command to get the current tag and version information, then uses `configure_file` to embed this into a `config.h` file. This is useful for runtime version checking. ```cmake find_package(Git REQUIRED) execute_process( COMMAND ${GIT_EXECUTABLE} describe --tags --long --always WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}" OUTPUT_VARIABLE VCS_TAG ) string(STRIP ${VCS_TAG} VCS_TAG) configure_file(config.h.in config.h) ``` -------------------------------- ### Run MIGraphX Model in VapourSynth Script Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/README.md Applies a compiled MIGraphX model to an input clip within a VapourSynth script. This example shows how to load a model and specify tile size for processing. ```python3 # DPIR src = core.std.BlankClip(src, width=1920, height=1080, format=vs.GRAYS) sigma = 10.0 flt = core.migx.Model([src, core.std.BlankClip(src, color=sigma/255.0)], engine_path="dpir_gray_1080p.mxr", tilesize=[1920, 1080]) ``` -------------------------------- ### Run DPIR Model with vs-tensorrt Source: https://github.com/amusementclub/vs-mlrt/wiki/vstrt Example Python script demonstrating how to use the vs-tensorrt plugin to run a DPIR model. Ensure the input clip is in GRAY format and specify the engine path and block dimensions. ```python # DPIR src = core.std.BlankClip(src, width=640, height=360, format=vs.GRAYS) sigma = 10.0 flt = core.trt.Model([src, core.std.BlankClip(src, color=sigma/255.0)], engine_path="dpir_gray_640_360.engine", block_w=640, block_h=360) ``` -------------------------------- ### Target Properties Setup Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Sets properties for the vsort target, including position-independent code, C++ standard, and extensions. ```cmake set_target_properties(vsort PROPERTIES POSITION_INDEPENDENT_CODE ON CXX_EXTENSIONS OFF CXX_STANDARD 17 CXX_STANDARD_REQUIRED ON ) ``` -------------------------------- ### Find Git and Get VCS Tag Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Finds the Git executable and retrieves the latest tag information. ```cmake find_package(Git REQUIRED) execute_process( COMMAND ${GIT_EXECUTABLE} describe --tags --long --always WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}" OUTPUT_VARIABLE VCS_TAG ) string(STRIP ${VCS_TAG} VCS_TAG) ``` -------------------------------- ### CUDA GPU Backend Video Sorting Source: https://github.com/amusementclub/vs-mlrt/wiki/vsort Example of using the CUDA GPU backend for video sorting with an ONNX model. ```python src = core.std.BlankClip(width=1920, height=1080, format=vs.RGBS) flt = core.ort.Model(src, "upconv_7_anime_style_art_rgb_scale2.0x.onnx", provider="CUDA") ``` -------------------------------- ### Initialize Waifu2x with vsmlrt Wrapper Source: https://github.com/amusementclub/vs-mlrt/wiki/waifu2x Demonstrates initializing the Waifu2x wrapper with various backend options. Choose the backend that best suits your hardware and performance needs. ```python from vsmlrt import Waifu2x, Waifu2xModel, Backend src = core.std.BlankClip(format=vs.RGBS) # backend could be: # - CPU Backend.OV_CPU(): the recommended CPU backend; generally faster than ORT-CPU. # - CPU Backend.ORT_CPU(num_streams=1, verbosity=2): vs-ort cpu backend. # - GPU Backend.ORT_CUDA(device_id=0, cudnn_benchmark=True, num_streams=1, verbosity=2) # - use device_id to select device # - set cudnn_benchmark=False to reduce script reload latency when debugging, but with slight throughput performance penalty. # - GPU Backend.TRT(fp16=True, device_id=0, num_streams=1): TensorRT runtime, the fastest NV GPU runtime. flt = Waifu2x(src, noise=-1, scale=2, model=Waifu2xModel.upconv_7_anime_style_art_rgb, backend=Backend.ORT_CUDA()) ``` -------------------------------- ### Build vs-openvino Plugin Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/README.md Sample cmake commands to build the vs-openvino plugin. Ensure OpenVINO and VapourSynth include directories are correctly specified. ```bash cmake -S . -B build -G Ninja -D CMAKE_BUILD_TYPE=Release -D CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -D InferenceEngine_DIR=openvino/runtime/cmake -D VAPOURSYNTH_INCLUDE_DIRECTORY="path/to/vapoursynth/include" cmake --build build cmake --install build --prefix install ``` -------------------------------- ### RealESRGANv2 Wrapper Usage with vsmlrt Source: https://github.com/amusementclub/vs-mlrt/wiki/RealESRGANv2 Demonstrates how to use the RealESRGANv2 wrapper module for simplified integration. Supports various backends including CPU and GPU options. ```python from vsmlrt import RealESRGANv2, RealESRGANv2Model, Backend src = core.std.BlankClip(format=vs.RGBS) # backend could be: # - CPU Backend.OV_CPU(): the recommended CPU backend; generally faster than ORT-CPU. # - CPU Backend.ORT_CPU(num_streams=1, verbosity=2): vs-ort cpu backend. # - GPU Backend.ORT_CUDA(device_id=0, cudnn_benchmark=True, num_streams=1, verbosity=2) # - use device_id to select device # - set cudnn_benchmark=False to reduce script reload latency when debugging, but with slight throughput performance penalty. # - GPU Backend.TRT(fp16=True, device_id=0, num_streams=1): TensorRT runtime, the fastest NV GPU runtime. flt = RealESRGANv2(src, model=RealESRGANv2Model.animevideo_xsx2, backend=Backend.ORT_CUDA()) ``` -------------------------------- ### Initialize CUGAN with ORT_CUDA Backend Source: https://github.com/amusementclub/vs-mlrt/wiki/CUGAN Demonstrates how to initialize the CUGAN model using the ORT_CUDA backend for GPU acceleration. Ensure the input source is in RGBS format and clamped to the [0,1] range. ```python from vsmlrt import CUGAN, Backend src = core.std.BlankClip(format=vs.RGBS) # only supports RGBS input formats # clamp src to be safe as out of range values will produce large negative output. src = core.akarin.Expr(src, "x 0 1 clamp") # backend could be: # - CPU Backend.OV_CPU(): the recommended CPU backend; generally faster than ORT-CPU. # - CPU Backend.ORT_CPU(num_streams=1, verbosity=2): vs-ort cpu backend. # - GPU Backend.ORT_CUDA(device_id=0, cudnn_benchmark=True, num_streams=1, verbosity=2) # - use device_id to select device # - set cudnn_benchmark=False to reduce script reload latency when debugging, but with slight throughput performance penalty. # - GPU Backend.TRT(fp16=True, device_id=0, num_streams=1): TensorRT runtime, the fastest NV GPU runtime. flt = CUGAN(src, noise=-1, scale=2, backend=Backend.ORT_CUDA()) ``` -------------------------------- ### Raw Model Usage with VapourSynth Source: https://github.com/amusementclub/vs-mlrt/wiki/RealESRGANv2 Shows how to use the RealESRGANv2 model directly with VapourSynth's OV.Model function. Requires specifying the model path. ```python src = core.std.BlankClip(width=1920, height=1080, format=vs.RGBS) flt = core.ov.Model(src, "RealESRGANv2/RealESRGANv2-animevideo-xsx2") ``` -------------------------------- ### Finding Required Packages Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Finds and includes necessary packages like OpenVINO, protobuf, and ONNX. ```cmake find_package(OpenVINO REQUIRED CONFIG) find_package(protobuf REQUIRED CONFIG) find_package(ONNX REQUIRED CONFIG) ``` -------------------------------- ### core.ov.Model Source: https://github.com/amusementclub/vs-mlrt/wiki/vsov Initializes and runs an AI model using the vs-openvino plugin. Supports various configurations for network path, tiling, device, and model quantization. ```APIDOC ## core.ov.Model ### Description Initializes and runs an AI model using the vs-openvino plugin. Supports various configurations for network path, tiling, device, and model quantization. ### Prototype `core.ov.Model(clip[] clips, string network_path[, int[] overlap = None, int[] tilesize = None, string device = "CPU", bint builtin = 0, string builtindir="models", bint fp16 = False, function config = None, bint path_is_serialization = False])` ### Parameters #### Path Parameters - **clips** (clip[]) - Required - The input clips. Only 32-bit floating point RGB or GRAY clips are supported. - **network_path** (string) - Required - The path to the network in ONNX format. - **overlap** (int[]) - Optional - Specifies the overlapping pixels between adjacent tiles to minimize boundary issues. Used when processing networks that support arbitrary input shapes in tiles. - **tilesize** (int[]) - Optional - Specifies the tile size (horizontal and vertical, including overlap) for processing networks that may not work well with full input dimensions. - **device** (string) - Optional - Specifies the device to run inference on. Supported values are "CPU" and "GPU". Defaults to "CPU". - **builtin** (bint) - Optional - Whether to load the model from the VS plugins directory. Defaults to 0 (False). - **builtindir** (string) - Optional - The model directory under VS plugins directory for builtin models. Defaults to "models". - **fp16** (bint) - Optional - Whether to quantize the model to fp16 for faster and memory-efficient computation. Defaults to False. - **config** (function) - Optional - A callable object (e.g., a function) with no positional arguments that returns a dictionary of plugin configuration parameters. Keys must be strings, and values can be int, float, or str. - **path_is_serialization** (bint) - Optional - Whether the `network_path` argument specifies an ONNX serialization of type `bytes`. Defaults to False. ### Notes - When `overlap` and `tilesize` are not specified, the filter attempts to resize the network to fit the input clips. This may fail if the network has specific input dimension requirements (e.g., width divisible by 8). - To ensure proper tiling, either leave `overlap` and `tilesize` unspecified to process the frame in a single tile, or set all three parameters (`overlap`, `tilesize`) to define tile dimensions and overlap. - The `config` parameter can be used to set specific parameters for the CPU or GPU plugins, such as `CPU_THROUGHPUT_STREAMS`. ``` -------------------------------- ### Windows Delay Loading Options Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Configures delay loading for DLLs on Windows based on WIN32_SHARED_OPENVINO. ```cmake if (WIN32) if(WIN32_SHARED_OPENVINO) target_link_options(vsov PRIVATE "/DELAYLOAD:openvino.dll" "delayimp.lib") else() target_link_options(vsov PRIVATE "/DELAYLOAD:tbb.dll" "delayimp.lib") endif() endif() ``` -------------------------------- ### Benchmark App Execution Command Source: https://github.com/amusementclub/vs-mlrt/wiki/Intel-Xeon-Platinum-8480 Command to run the OpenVINO benchmark_app for performance testing. Specifies CPU binding, memory binding, number of threads, streams, model path, input shape, and inference precision. ```bash numactl --cpubind=0 --membind=0 python3 benchmark_app.py -hint none -nthreads 56 -nstreams 1 -m waifu2x/upconv_7_anime_style_art_rgb/scale2.0x_model.onnx -shape "input[1,3,1080,1920]" -ip f32 -op f32 -infer_precision CPU:bf16 ``` -------------------------------- ### Configure Header File Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Generates a config.h file from a template, incorporating VCS tag information. ```cmake configure_file(config.h.in config.h) ``` -------------------------------- ### Dependency Finding Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Finds required configuration packages for protobuf and ONNX. ```cmake find_package(protobuf REQUIRED CONFIG) find_package(ONNX REQUIRED CONFIG) ``` -------------------------------- ### Find Required Packages Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Locates and loads the necessary migraphx and hip packages. ```cmake find_package(migraphx REQUIRED CONFIG) find_package(hip REQUIRED CONFIG) ``` -------------------------------- ### Dependency Finding Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/CMakeLists.txt Finds and configures required packages like Protobuf, ONNX, and ncnn. ```cmake find_package(protobuf REQUIRED CONFIG) find_package(ONNX REQUIRED CONFIG) find_package(ncnn REQUIRED CONFIG) ``` -------------------------------- ### Windows Specific Link Options Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Applies delay-load options for DLLs on Windows systems. ```cmake if (WIN32) target_link_options(vsmigx PRIVATE "/DELAYLOAD:migraphx_c.dll" "/DELAYLOAD:amdhip64_6.dll" "delayimp.lib" ) endif() ``` -------------------------------- ### CUDA Backend Configuration Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Enables and configures the CUDA backend if ENABLE_CUDA is set. Includes finding CUDA toolkit, adding definitions, and linking CUDA libraries. ```cmake if (ENABLE_CUDA) find_package(CUDAToolkit REQUIRED) add_compile_definitions(ENABLE_CUDA) target_include_directories(vsort PRIVATE ${CUDAToolkit_INCLUDE_DIRS}) target_link_libraries(vsort PRIVATE CUDA::cudart_static) if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC") target_link_options(vsort PRIVATE "/DELAYLOAD:onnxruntime.dll" "delayimp.lib") endif() endif() ``` -------------------------------- ### Using Flexible Output Properties in MIGraphX Model Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/README.md Demonstrates how to handle ONNX models with an arbitrary number of output planes using the `flexible_output_prop` argument. This is useful when a model's output is not a single clip but multiple planes. ```python3 from typing import TypedDict class Output(TypedDict): clip: vs.VideoNode num_planes: int prop = "planes" # arbitrary non-empty string output = core.migx.Model(src, program_path, flexible_output_prop=prop) # type: Output clip = output["clip"] num_planes = output["num_planes"] output_planes = [ clip.std.PropToClip(prop=f"{prop}{i}") for i in range(num_planes) ] # type: list[vs.VideoNode] ``` -------------------------------- ### Build MIGraphX Program from ONNX Model Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/README.md Compiles an ONNX model into a MIGraphX program using the `migraphx-driver`. This command specifies the input ONNX file, the output file name, and the input dimensions for the model. ```shell migraphx-driver compile --onnx drunet_gray.onnx --gpu --input-dim @input 1 2 1080 1920 --output dpir_gray_1080p.mxr ``` -------------------------------- ### Public Include Directories Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Sets public include directories for the vsov target, including the project's binary directory. ```cmake target_include_directories(vsov PUBLIC "${PROJECT_BINARY_DIR}" ) ``` -------------------------------- ### Library Creation and Source Files Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/CMakeLists.txt Defines the vsncnn shared library and lists its source files. ```cmake add_library(vsncnn SHARED vs_ncnn.cpp onnx2ncnn.cpp ../common/onnx_utils.cpp) ``` -------------------------------- ### Target Include Directories Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Adds VapourSynth include directories to the 'vsmigx' target. ```cmake target_include_directories(vsmigx PRIVATE ${VAPOURSYNTH_INCLUDE_DIRECTORY}) ``` -------------------------------- ### Add Public Include Directory Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Adds the binary directory to the public include directories for the 'vsmigx' target. ```cmake target_include_directories(vsmigx PUBLIC "${PROJECT_BINARY_DIR}") ``` -------------------------------- ### Public Include Directory Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/CMakeLists.txt Adds the project's binary directory as a public include directory for the vsncnn target. ```cmake target_include_directories(vsncnn PUBLIC "${PROJECT_BINARY_DIR}" ) ``` -------------------------------- ### DPIR Wrapper Usage with vsmlrt Source: https://github.com/amusementclub/vs-mlrt/wiki/DPIR Demonstrates how to use the vsmlrt Python wrapper for DPIR models. It shows input clip creation and backend selection for CPU or GPU execution. GPU backends like ORT_CUDA and TRT are recommended for performance. ```python from vsmlrt import DPIR, DPIRModel, Backend src = core.std.BlankClip(format=vs.RGBS) # or vs.GRAYS for gray only models # backend could be: # - CPU Backend.OV_CPU(): the recommended CPU backend; generally faster than ORT-CPU. # - CPU Backend.ORT_CPU(num_streams=1, verbosity=2): vs-ort cpu backend. # - GPU Backend.ORT_CUDA(device_id=0, cudnn_benchmark=True, num_streams=1, verbosity=2) # - use device_id to select device # - set cudnn_benchmark=False to reduce script reload latency when debugging, but with slight throughput performance penalty. # - GPU Backend.TRT(fp16=True, device_id=0, num_streams=1): TensorRT runtime, the fastest NV GPU runtime. # DPIR is a huge model and GPU backend is highly recommended (use TRT to provide the best performance) # If the model runs out of GPU memory, increase the tiles parameter. flt = DPIR(src, strength=5, model=DPIRModel.drunet_color, tiles=2, backend=Backend.ORT_CUDA()) ``` -------------------------------- ### Load Raw ONNX Model with Pre-scaling Source: https://github.com/amusementclub/vs-mlrt/wiki/waifu2x Loads a raw ONNX model after pre-scaling the input using bicubic interpolation. This is necessary for models that do not include built-in upscaling. ```python src = core.std.BlankClip(width=1920, height=1080, format=vs.RGBS) flt = core.ov.Model(src.fmtc.resample(scale=2, kernel="bicubic", a1=0, a2=0.5), "anime_style_art_rgb_scale2.0x.onnx") ``` -------------------------------- ### Project Definition Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Defines the project name, version, and primary language. ```cmake project(vs-migraphx VERSION 3.1 LANGUAGES CXX) ``` -------------------------------- ### Public Include Directory Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Adds the project's binary directory to the public include directories for the vsort target. ```cmake target_include_directories(vsort PUBLIC "${PROJECT_BINARY_DIR}" ) ``` -------------------------------- ### Run DPIR Model with TensorRT Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/README.md Illustrates how to run a DPIR model using the `core.trt.Model` function in a VapourSynth script. It sets up a grayscale source clip and applies the DPIR model with a specified engine path and tile size. ```python # DPIR src = core.std.BlankClip(src, width=640, height=360, format=vs.GRAYS) sigma = 10.0 flt = core.trt.Model([src, core.std.BlankClip(src, color=sigma/255.0)], engine_path="dpir_gray_1080p_dynamic.engine", tilesize=[640, 360]) ``` -------------------------------- ### Include Directory Configuration Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/CMakeLists.txt Specifies private include directories for the vsncnn library, including VapourSynth and ONNX headers. ```cmake target_include_directories(vsncnn PRIVATE ${VAPOURSYNTH_INCLUDE_DIRECTORY} ${ONNX_INCLUDE_DIRS} ) ``` -------------------------------- ### RIFE Wrapper Usage with vsmlrt Source: https://github.com/amusementclub/vs-mlrt/wiki/RIFE Demonstrates how to initialize the RIFE wrapper with a source clip, specifying the model and backend. Supports various backends including CPU, ORT-CUDA, and TensorRT. ```python from vsmlrt import RIFE, RIFEModel, Backend src = core.std.BlankClip(format=vs.RGBS) # backend could be: # - CPU Backend.OV_CPU(): the recommended CPU backend; generally faster than ORT-CPU. # - CPU Backend.ORT_CPU(num_streams=1, verbosity=2): vs-ort cpu backend. # - GPU Backend.ORT_CUDA(device_id=0, cudnn_benchmark=True, num_streams=1, verbosity=2) # - use device_id to select device # - set cudnn_benchmark=False to reduce script reload latency when debugging, but with slight throughput performance penalty. # - GPU Backend.TRT(fp16=True, device_id=0, num_streams=1): TensorRT runtime, the fastest NV GPU runtime. flt = RIFE(src, model=RIFEModel.v4_4, backend=Backend.ORT_CUDA()) ``` -------------------------------- ### CoreML Backend Definition Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Adds a compile definition for the CoreML backend if ENABLE_COREML is set. ```cmake if(ENABLE_COREML) add_compile_definitions(ENABLE_COREML=1) endif() ``` -------------------------------- ### Setting Include Directories Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Specifies private include directories for the vsov target. ```cmake target_include_directories(vsov PRIVATE ${VAPOURSYNTH_INCLUDE_DIRECTORY} ${ONNX_INCLUDE_DIRS} ) ``` -------------------------------- ### Raw DPIR Model Usage with VapourSynth Source: https://github.com/amusementclub/vs-mlrt/wiki/DPIR Shows how to use DPIR models directly with VapourSynth's ov.Model. This method requires specifying the model file and passing the input clip along with a sigma value (normalized to 0-1) as a separate clip. ```python src = core.std.BlankClip(width=640, height=360, format=vs.GRAYS) sigma = 2.0 flt = core.ov.Model([src, core.std.BlankClip(src, color=sigma/255.0)], "drunet_gray.onnx") ``` -------------------------------- ### Include Directories Configuration Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Configures private include directories for the vsort library, including VapourSynth and ONNX headers. ```cmake target_include_directories(vsort PRIVATE ${VAPOURSYNTH_INCLUDE_DIRECTORY} ${ONNX_INCLUDE_DIRS} ${ONNX_RUNTIME_API_DIRECTORY} ) ``` -------------------------------- ### DML Backend Definition Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Adds a compile definition for the DirectML backend if ENABLE_DML is set. ```cmake if (ENABLE_DML) add_compile_definitions(ENABLE_DML) endif() ``` -------------------------------- ### Setting Cache Variables Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Defines cache variables for VapourSynth include directory, visualization, and OpenVINO build options. ```cmake set(VAPOURSYNTH_INCLUDE_DIRECTORY "" CACHE PATH "Path to VapourSynth headers") set(ENABLE_VISUALIZATION OFF CACHE BOOL "Enable support for network visualization") set(WIN32_SHARED_OPENVINO OFF CACHE BOOL "Build for win32 with shared openvino library") ``` -------------------------------- ### Public Includes for vstrt Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Sets the public include directories for the `vstrt` target, ensuring that build artifacts in the binary directory are accessible to other projects. ```cmake target_include_directories(vstrt PUBLIC "${PROJECT_BINARY_DIR}" ) ``` -------------------------------- ### Linking OpenVINO Runtime Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Links the OpenVINO runtime library to the vsov target. ```cmake target_link_libraries(vsov PRIVATE openvino::runtime) ``` -------------------------------- ### VapourSynth Include Directory Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/CMakeLists.txt Defines a cache variable for the VapourSynth header directory. ```cmake set(VAPOURSYNTH_INCLUDE_DIRECTORY "" CACHE PATH "Path to VapourSynth headers") ``` -------------------------------- ### Building vstrt_rtx Library (with TensorRT RTX) Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Defines the `vstrt_rtx` shared library target when TensorRT RTX is detected. It specifies source files, include directories, and compiler properties. ```cmake add_library(vstrt_rtx SHARED $<$: longpath.manifest> vs_tensorrt.cpp win32.cpp ) target_include_directories(vstrt_rtx PRIVATE ${VAPOURSYNTH_INCLUDE_DIRECTORY} ${CUDAToolkit_INCLUDE_DIRS} ${TENSORRT_HOME}/include ) set_target_properties(vstrt_rtx PROPERTIES CXX_EXTENSIONS OFF POSITION_INDEPENDENT_CODE ON CXX_STANDARD 20 CXX_STANDARD_REQUIRED ON ) target_link_directories(vstrt_rtx PRIVATE ${TENSORRT_HOME}/lib) target_link_libraries(vstrt_rtx PRIVATE CUDA::cudart_static) ``` -------------------------------- ### Link Directories Configuration Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Configures private link directories for the vsort library, specifically for ONNX Runtime libraries. ```cmake target_link_directories(vsort PRIVATE ${ONNX_RUNTIME_LIB_DIRECTORY} ) ``` -------------------------------- ### Setting C++ Standard and Extensions Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsov/CMakeLists.txt Configures C++ standard to 17 and disables C++ extensions for the vsov target. ```cmake set_target_properties(vsov PROPERTIES CXX_EXTENSIONS OFF CXX_STANDARD 17 CXX_STANDARD_REQUIRED ON ) ``` -------------------------------- ### C++ Standard and Cache Variables Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Sets the C++ standard to C++17 and defines cache variables for VapourSynth and TensorRT paths, along with options for nvinfer plugin usage. These variables allow users to customize the build configuration. ```cmake set(CMAKE_CXX_STANDARD 17) set(VAPOURSYNTH_INCLUDE_DIRECTORY "" CACHE PATH "Path to VapourSynth headers") set(TENSORRT_HOME "" CACHE PATH "Path to TensorRT") option(USE_NVINFER_PLUGIN "Initialize nvinfer_plugin" FALSE) option(USE_NVINFER_PLUGIN_STATIC "Use static nvinfer_plugin" FALSE) set(TENSORRT_LIBRARY_SUFFIX "" CACHE STRING "TensorRT library suffix") ``` -------------------------------- ### Compile Definitions and Public Includes for vstrt_rtx Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/CMakeLists.txt Adds a compile definition for `USE_NVINFER_PLUGIN` and sets public include directories for the `vstrt_rtx` target. This makes the build configuration available to dependent projects. ```cmake target_compile_definitions(vstrt_rtx PRIVATE USE_NVINFER_PLUGIN) target_include_directories(vstrt_rtx PUBLIC "${PROJECT_BINARY_DIR}" ) ``` -------------------------------- ### Load Raw ONNX Model Source: https://github.com/amusementclub/vs-mlrt/wiki/waifu2x Loads a raw ONNX model for image processing. This is a more direct approach compared to the wrapper. ```python src = core.std.BlankClip(width=1920, height=1080, format=vs.RGBS) flt = core.ov.Model(src, "upconv_7_anime_style_art_rgb_scale2.0x.onnx") ``` -------------------------------- ### ONNX Library Linking Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Links the ONNX library to the vsort target, handling version differences for ONNX::onnx. ```cmake # https://github.com/onnx/onnx/commit/21bff4e55dcefecc069c679115baae6b00caa0d5 if (ONNX_VERSION VERSION_LESS 1.16.0) target_link_libraries(vsort PRIVATE onnx) else() target_link_libraries(vsort PRIVATE ONNX::onnx) endif() ``` -------------------------------- ### Specify Include Directories Source: https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/trtexec/CMakeLists.txt Adds private include directories for the trtexec target, making headers available during compilation. ```cmake target_include_directories(trtexec PRIVATE ../common .. ../../include ../../shared ) ``` -------------------------------- ### Cache Variable Definitions Source: https://github.com/amusementclub/vs-mlrt/blob/master/vsort/CMakeLists.txt Defines cache variables for VapourSynth and ONNX Runtime paths, and boolean flags for optional backends. ```cmake set(VAPOURSYNTH_INCLUDE_DIRECTORY "" CACHE PATH "Path to VapourSynth headers") set(ONNX_RUNTIME_API_DIRECTORY "" CACHE PATH "Path to ONNX API headers") set(ONNX_RUNTIME_LIB_DIRECTORY "" CACHE PATH "Path to ONNX Runtime libraries") set(ENABLE_CUDA OFF CACHE BOOL "Enable CUDA backend") set(ENABLE_DML OFF CACHE BOOL "Enable DirectML backend") set(ENABLE_COREML OFF CACHE BOOL "Enable CoreML support") ```