### Getting Started Operation Build Method Implementation Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/primer_mlir.md Illustrates the implementation of the build method for the GettingStartedOp, which internally calls inferReturnTypes to determine the operation's output types. This is a common pattern for operations where return types are not explicitly provided during construction. ```C++ void GettingStartedOp::build(::mlir::OpBuilder &odsBuilder, ::mlir::OperationState &odsState, ::mlir::Value input, ::mlir::Value secondInput, mlir::IntegerAttr myInt, ::mlir::ArrayAttr myIntArray, /*optional*/::mlir::FloatAttr myOptionalFloat, mlir::IntegerAttr myDefaultValuedInt, vpux::IE::GettingStartedAttr gettingStarted) { // ... ::llvm::SmallVector<::mlir::Type, 2> inferredReturnTypes; if (::mlir::succeeded(GettingStartedOp::inferReturnTypes(odsBuilder.getContext(), odsState.location, odsState.operands, odsState.attributes.getDictionary(odsState.getContext()), odsState.getRawProperties(), odsState.regions, inferredReturnTypes))) odsState.addTypes(inferredReturnTypes); else ::mlir::detail::reportFatalInferReturnTypesError(odsState); } ``` -------------------------------- ### Install Dependencies Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tools/ir-analyzer/README.md Run this command to install the necessary dependencies before using the IR-Analyzer. ```bash python3 -m pip install -r requirements.in ``` -------------------------------- ### Install pre-commit hooks Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/developer_tools.md Install the project's git hooks into your local clone using the pre-commit framework. Dependencies are installed automatically on first execution. ```sh pre-commit install ``` -------------------------------- ### Install Test Data Files Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tests/functional/CMakeLists.txt Installs the skip configuration file and VM runtime test data to the 'tests' directory for the installed package. ```cmake set(VM_RUNTIME_TEST_DATA_DIR "${CMAKE_CURRENT_SOURCE_DIR}/behavior/vm_runtime/data") # Copies test data to installed package install(FILES ${SKIP_CONFIG_PATH} DESTINATION tests COMPONENT ${VPUX_TESTS_COMPONENT} EXCLUDE_FROM_ALL ) install(DIRECTORY "${VM_RUNTIME_TEST_DATA_DIR}/" DESTINATION tests COMPONENT ${VPUX_TESTS_COMPONENT} EXCLUDE_FROM_ALL ) ``` -------------------------------- ### VPUX-OPT Command Line Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/project_structure.md This is an example of how to run the vpux-opt tool with specific compiler flags and input file. It's used for testing and demonstrating compiler behavior. ```mlir // RUN: vpux-opt --split-input-file --init-compiler="platform=NPU4000 allow-custom-values=true" --unroll-distributed-ops-VPUX40XX %s | FileCheck %s ``` -------------------------------- ### Install OpenVINO and VPUXPlugin Libraries Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_build_static.md Installs the built OpenVINO and VPUXPlugin libraries to a specified directory. Set the INSTALL_DIR environment variable before running. ```bat cmake --install %OPENVINO_HOME%\build-x86_64 --prefix %INSTALL_DIR% ``` -------------------------------- ### Install pre-commit framework Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/developer_tools.md Install the pre-commit framework using pip. This is the recommended way to manage git hooks for code formatting. ```sh pip install pre-commit ``` -------------------------------- ### Example Command to Run Plotter Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tools/memory-usage-plotter/README.md An example command demonstrating how to run the memory usage plotter with specific log and output file names. ```bash python3 plot_memory_usage.py logs.txt memory_usage_plot.png ``` -------------------------------- ### Example Benchmark Output Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tests/micro_benchmarks/README.md This is an example of the output generated by the npuMicroBenchmark binary, showing performance metrics for various benchmarks. ```text ------------------------------------------------------------- Benchmark Time CPU Iterations ------------------------------------------------------------- BM_GetValues 1354994342 ns 1354837630 ns 1 BM_GetTmpBuff 136714917 ns 136682567 ns 4 BM_FuseMulAdd 138881573 ns 138875599 ns 4 BM_FuseMulAddDtype 126526909 ns 126527880 ns 4 ``` -------------------------------- ### Build and Install LLVM Binaries for Cache Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_llvm_cache_linux.md These commands set up a build directory, configure CMake with specific options, and then build the LLVM targets. Finally, they install the LLVM binaries to a designated cache directory for future use. ```sh mkdir ${OPENVINO_HOME}/build_llvm_${CONFIG} cd ${OPENVINO_HOME}/build_llvm_${CONFIG} cmake \ -G Ninja \ -D CMAKE_BUILD_TYPE=${CONFIG} \ -D OPENVINO_EXTRA_MODULES=${NPU_PLUGIN_HOME} \ .. ninja build-modules/vpux_plugin/thirdparty/llvm-project/llvm/all -j${nproc} ``` ```sh mkdir ${LLVMPKG_CACHE_HOME} export LLVM_LIB_PATH=${OPENVINO_HOME}/build_llvm_${CONFIG}/build-modules/vpux_plugin/thirdparty/llvm-project/llvm cmake --install ${LLVM_LIB_PATH} --config ${CONFIG} --prefix ${LLVMPKG_CACHE_HOME} ``` -------------------------------- ### Install CiD Components Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_windows.md Installs all CiD components to a specified location using CMake. This command requires navigating to the build directory and specifying the installation prefix and component. ```bat cd %OPENVINO_HOME%\build_%CONFIG% cmake --install .\ --prefix .\ --component CiD --verbose ``` -------------------------------- ### Install CiD Components Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_cmake_presets_linux.md Install all Driver Compiler (CiD) components to a specified location using CMake's install command. This is an optional step. ```sh # Install all CiD components cd ${OPENVINO_HOME}/build_${CONFIG} cmake --install . --prefix $(pwd)/ --component CiD --verbose ``` -------------------------------- ### Install VTune on Linux Host Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_use_vtune.md Installs VTune and related tools on a Linux host for user-mode sampling. This involves setting up Intel repositories and updating package lists. Ensure you have root privileges or use `sudo -E`. ```bash # Temporarily disregard normal proxy config # You may need to run these commands as root since sudo doesn't preserve # environment variables _or_ you can try `sudo -E`. export no_proxy= # Download and install Intel GPG key wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null # Add Intel repos to APT echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list # Fetch package lists from repos sudo apt update # Install VTune and related tools to enable user-mode sampling sudo apt install intel-oneapi-vtune sudo apt install linux-tools-common linux-tools-generic linux-tools-`uname -r` # Check that everything is working (expect to see failures for hardware based features) /opt/intel/oneapi/vtune/latest/bin64/vtune-self-checker.sh ``` -------------------------------- ### Install Build Dependencies on Linux Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_build.md Automatically install required build tools like GCC, CMake, Ninja, and ccache using the provided script. Manual installation is needed for Clang. ```bash sudo -E ./install_build_dependencies.sh ``` -------------------------------- ### Ccache statistics example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/developer_tools.md Example output of 'ccache -s' command, showing cache statistics such as cacheable calls, hits, misses, and local storage usage. This helps verify if Ccache is being utilized effectively. ```text Cacheable calls: 8980 / 23702 (37.89%) Hits: 1198 / 8980 (13.34%) Direct: 1176 / 1198 (98.16%) Preprocessed: 22 / 1198 ( 1.84%) Misses: 7782 / 8980 (86.66%) Uncacheable calls: 14722 / 23702 (62.11%) Local storage: Cache size (GB): 1.55 / 20.00 ( 7.73%) ``` -------------------------------- ### Build Default HW Mode Pipeline (Good Example) Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/code_style.md Shows an example of building a pipeline where optimization passes are called as a sub-pipeline, maintaining clarity of dependencies. This is preferred over listing individual passes when they form a logical unit. ```cpp void vpux::buildDefaultHWModePipeline(mlir::OpPassManager& pm, const DefaultHWOptions37XX& options, Logger log) { // ... IE::buildOptimizeActivationsPipeline(pm, log) // ... } ``` -------------------------------- ### Return Early Pattern - Good Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/code_style.md Demonstrates the 'return early' pattern for managing fail-cases, preserving the Open-Closed Principle by reducing code nesting. ```cpp mlir::LogicalResult goodExample(VPUIP::SubViewOp origOp) { auto producerSubViewOp = origOp.source().getDefiningOp(); if (producerSubViewOp == nullptr) { return mlir::failure(); } if (origOp.static_strides().hasValue() || producerSubViewOp.static_strides().hasValue()) { return mlir::failure(); } // <...> return mlir::success(); } ``` -------------------------------- ### Install CiD Components Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_llvm_cache_linux.md This command installs all CiD components to a specified location, typically the current directory. It's an optional step for deploying the Driver Compiler. ```sh # Install all CiD components cd ${OPENVINO_HOME}/build_${CONFIG} cmake --install . --prefix $(pwd)/ --component CiD ``` -------------------------------- ### Trace Event Structure Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_get_schedule_trace_and_analysis.md This example illustrates the hierarchical structure of trace events within the schedule trace file, showing how DMA, DPU, and SW tasks are organized across different clusters and engines. ```text DMA: |-- DMA 0 |-- DMA 1 Cluster 0: |-- DPU 0 |-- SW / Shave 1 |-- SW / Shave 2 Cluster 1: |-- DPU 0 |-- SW / Shave 1 |-- SW / Shave 2 ``` -------------------------------- ### Inference with Configuration File Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tools/infer_tool/README.md Use a configuration file to set plugin-specific options. Lines starting with '#' are ignored. ```bash # Inference with a configuration file, where `config.txt` contains: # NPU_COMPILER_TYPE PLUGIN infer_tool -m model.xml -d IMD.5010 -c config.txt ``` -------------------------------- ### Register Interfaces for Platform Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/project_structure.md Example of creating an interfaces registry and registering necessary interfaces for a specific hardware architecture before the compilation pipeline. ```C++ auto interfacesRegistry = createInterfacesRegistry(archKind); // registry is a mlir::DialectRegistry interfacesRegistry->registerInterfaces(registry); ``` -------------------------------- ### Filter IR Printing with Regex Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/how_to_debug.md Use IE_NPU_IR_PRINTING_FILTER with a POSIX Regex to match multiple passes. This example prints IR for 'InitResources' or any pass starting with 'ConvertLayers'. ```bash export IE_NPU_IR_PRINTING_FILTER="InitResources|ConvertLayers.*" ``` -------------------------------- ### Full Procedure for vclExecutableCreate Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/api_reference.md Demonstrates the complete workflow for creating an executable object using `vclExecutableCreate`. This includes compiler and device setup, executable description preparation, creation, optional blob export, and resource cleanup. ```c // 1. Create compiler and device description and instantiate compiler vcl_compiler_desc_t compilerDesc = { ... }; vcl_device_desc_t deviceDesc = { ... }; vcl_compiler_handle_t compiler; vcl_log_handle_t log; vclCompilerCreate(&compilerDesc, &deviceDesc, &compiler, &log); // 2. Prepare model IR (e.g. xml+weights) and optional parameters vcl_executable_desc_t execDesc = { .modelIRData = ..., // pointer to IR data .modelIRSize = ..., // IR data size .options = ..., // compiler options (optional) .optionsSize = ..., // options size }; // 3. Create executable object vcl_executable_handle_t exec; vcl_result_t ret = vclExecutableCreate(compiler, execDesc, &exec); // 4. Export blob (if needed) uint64_t blobSize; vclExecutableGetSerializableBlob(exec, NULL, &blobSize); uint8_t* blob = malloc(blobSize); vclExecutableGetSerializableBlob(exec, blob, &blobSize); // After use, free(blob); // 5. Destroy executable object vclExecutableDestroy(exec); // 6. Destroy compiler object vclCompilerDestroy(compiler); ``` -------------------------------- ### Configure TBBROOT in CMake Presets Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/FAQ.md Example of how to set the TBBROOT path within a CMake preset's cacheVariables. This allows specifying an alternative location for the oneTBB installation. ```json "name": "cid", "cacheVariables": { "TBBROOT": { "type": "FILEPATH", "value": "/path/to/alternative/oneTBB" } } ``` -------------------------------- ### Inferring Return Types for Getting Started Operation Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/primer_mlir.md Demonstrates how to use the adaptor to access operation operands and attributes, verify constraints, and infer return types. This is typically used within the operation's inferReturnTypes method. ```C++ // The adaptor offers an interface similar to the main operation class, with accessors to the operands // and attributes of the operation. Additionally, it contains a verify method that checks whether // the constraints of the operation are satisfied. If the constraints are not satisfied, the return // type might not be inferrable so a failure is returned. IE::GettingStartedOpAdaptor gettingStartedOp(operands, attributes, properties); if (mlir::failed(gettingStartedOp.verify(loc))) { return mlir::failure(); } // `input()` returns a `mlir::Value` object from which we extract the `mlir::Type` // This type is then passed to the output for this particular example operation, but the creation of // the output type varies from operation to operation const auto inputType = gettingStartedOp.getInput().getType(); inferredReturnTypes.push_back(inputType); // Success is returned when the return type has been inferred return mlir::success(); ``` -------------------------------- ### Implement Pass with Top-Down Traversal Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/code_style.md Demonstrates adding rewriter patterns to a pass. It's crucial that the rewriter's traversal direction matches the pass's traversal direction to avoid unexpected behavior. This example shows a correct top-down traversal setup. ```C++ // IR structure to consider: // func.func @foo(...) -> !outType { // %0 = VPUIP.NCEClusterTask(...) // %alloc = memref.alloc(): !outType // %1 = VPUIP.Copy inputs(%0) outputs(%alloc) // return %1 // } struct MyBottomUpRewriter; struct MyTopDownRewriter; void MyTopDownPass::safeRunOnFunc() { auto& ctx = getContext(); mlir::RewritePatternSet patterns(&ctx); patterns.add(&ctx); // BAD: see below patterns.add(&ctx); // OK auto func = getOperation(); auto config = getDefaultGreedyRewriteConfig(); config.useTopDownTraversal = true; if (mlir::failed(mlir::applyPatternsGreedily( func, std::move(patterns), config))) { signalPassFailure(); } } // BAD: the pass would first read the NCEClusterTask and only then CopyOp; // some other rewriter may end up changing the code *before* this // rewriter would be able to process the CopyOp! struct MyBottomUpRewriter : mlir::OpRewritePattern { MyBottomUpRewriter(mlir::MLIRContext* ctx) : mlir::OpRewritePattern(ctx) {} mlir::LogicalResult matchAndRewrite(VPUIP::CopyOp copyOp, mlir::PatternRewriter& rewriter) const final { // Remove a copy: // NCEClusterTask NCEClusterTask // | => | // Copy // | // ... return mlir::success(); } }; // OK: the pass would first read the NCEClusterTask and MyTopDownRewriter // would process this operation immediately. struct MyTopDownRewriter : mlir::OpRewritePattern { MyTopDownRewriter(mlir::MLIRContext* ctx) : mlir::OpRewritePattern(ctx) {} mlir::LogicalResult matchAndRewrite(VPUIP::NCEClusterTaskOp taskOp, mlir::PatternRewriter& rewriter) const final { // Remove a copy: // NCEClusterTask NCEClusterTask // | => | // Copy // | // ... return mlir::success(); } }; ``` -------------------------------- ### Install CiD Components to a Specific Location Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_linux.md Use this command to install all CiD components to a specified directory. The --prefix flag defines the installation location, and --component CiD ensures only CiD components are installed. ```sh cd ${OPENVINO_HOME}/build_${CONFIG} cmake --install . --prefix $(pwd)/ --component CiD --verbose ``` -------------------------------- ### VPUX Bytecode Function Call Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/design/bytecode_format.md Demonstrates the calling convention in practice, showing an entrypoint function calling an inner function. It highlights call frame creation, buffer creation, `call` instruction usage, and `ret`/`retv` instructions for returning control and values. ```mlir function_section: [ 0: @main, num_args=1, num_results=0, { buffer.create r0, rp0, ... call @inner_fn, 1, r1, 1, r0 ret } 1: @inner_fn, num_args=1, num_results=1, { set.imm r0, 5 retv 1, r0 } ] ``` -------------------------------- ### Constructing VCL Options String Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/FAQ.md This example shows the final string format for VCL options, including input/output precisions, layouts, and NPU platform configuration. It is recommended to extract node names from the IR model's XML file. ```bash --inputs_precisions="input:fp16" --inputs_layouts="input:C" --outputs_precisions="output:fp16" --outputs_layouts="output:C" --config NPU_PLATFORM="4000" ``` -------------------------------- ### Install LIT Test Executable Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tests/lit/CMakeLists.txt Installs the LIT test executable to the 'tests' destination. This makes the test runner accessible after installation. ```cmake install(PROGRAMS "${_lit_tests_dir_full_path}" DESTINATION tests COMPONENT tests EXCLUDE_FROM_ALL ) ``` -------------------------------- ### Configure LLDB Initialization File Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/scripts/lldb_scripts/README.md This snippet shows how to configure the ~/.lldbinit file to import LLVM, MLIR, and NPU specific pretty printer scripts for LLDB. ```shell script print("LLDB is using .lldbinit!") script print("LLVMSupport...") command script import /thirdparty/llvm-project/llvm/utils/lldbDataFormatters.py script print("MLIRSupport...") command script import /thirdparty/llvm-project/mlir/utils/lldb-scripts/mlirDataFormatters.py script print("NPUSupport...") command script import /scripts/lldb_scripts/NPUDataFormatters.py script print(" Done") ``` -------------------------------- ### Create and Build Pipeline Strategy Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/project_structure.md Demonstrates how to create a pipeline strategy for a given architecture and build the pipeline using the strategy. ```C++ auto pipelineFactory = createPipelineStrategy(arch); // pm is mlir::PassManager pipelineFactory->buildPipeline(pm, config, rootTiming, log); ``` -------------------------------- ### Configure VSCode launch.json for LLDB Initialization Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/scripts/lldb_scripts/README.md This JSON configuration snippet demonstrates how to set up VSCode's launch.json to ensure that the ~/.lldbinit file is sourced when launching a debug session with the CodeLLDB extension. ```json { "version": "0.2.0", "configurations": [ { "name": "vpuxOpt-lldb", "type": "lldb", "request": "launch", "program": "~/src/openvino/bin/intel64/RelWithDebInfo/vpux-opt", "args": [ "--platform=NPU4000", "--mlir-disable-threading", "--host-compile", "path_to_file.mlir" ], "cwd": "${fileDirname}", "preLaunchTask": "", "postDebugTask": "", "sourceLanguages": [ "cpp" ], "initCommands": [ "command source ${env:HOME}/.lldbinit" ] } ] } ``` -------------------------------- ### Install CiD Components to a Specific Location Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_cmake_presets_windows.md Install all Driver Compiler (CiD) components to a specified directory. This command uses CMake's install functionality with verbose output. ```bat @REM Install all CiD components cd %OPENVINO_HOME%\build_%CONFIG% cmake --install .\ --prefix .\ --component CiD --verbose ``` -------------------------------- ### Create VPU Compiler Instance Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/api_reference.md Initializes a compiler instance for a specific device. Ensure correct API version and device details are provided. ```c vcl_compiler_desc_t compilerDesc = { .version = {7, 4}, // API version .debugLevel = VCL_LOG_INFO // Debug level }; vcl_device_desc_t deviceDesc = { .size = sizeof(vcl_device_desc_t), .deviceID = 0x1234, // PCI Device ID in lower 16 bits .revision = 0, // NPU Revision (0 for first stepping) .tileCount = 1 // Number of slices/tiles }; vcl_compiler_handle_t compiler; vcl_log_handle_t logHandle; vcl_result_t result = vclCompilerCreate(&compilerDesc, &deviceDesc, &compiler, &logHandle); ``` -------------------------------- ### Install Functional Test Target Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/test/functional/CMakeLists.txt Installs the functional test executable and its debug files if the BUILD_COMPILER_FOR_DRIVER option is enabled. ```cmake if(BUILD_COMPILER_FOR_DRIVER) install_target_with_debug_files(vpuxCompilerL0Test ${INSTALL_DESTINATION} ${INSTALL_COMPONENT}) endif() ``` -------------------------------- ### Parameterized RUN Command with Architecture Filtering Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_test.md This example shows how to use parameters like %arch% in the RUN command and filter test execution based on specific architecture requirements using the REQUIRES comment. ```MLIR // RUN: vpux-opt ... %arch% ... // REQUIRES: arch-NPU37XX || arch-NPU40XX ``` -------------------------------- ### Full Model Compilation Workflow Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/how_to_debug.md Demonstrates a complete model compilation process using vpux-translate and vpux-opt. ```bash # Import an OpenVINO IR into IE dialect ./vpux-translate --platform=NPU3720 --import-IE -o net.mlir # Call the DefaultHWMode pipeline over the imported IR ./vpux-opt --platform=NPU3720 --default-hw-mode --lower-VPUIP-to-ELF net.mlir -o net_out.mlir # Export the final IR into an ELF file ./vpux-translate --platform=NPU3720 --export-ELF net_out.mlir > net.blob ``` -------------------------------- ### Install NPU Functional Test Target Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tests/functional/CMakeLists.txt Installs the npuFuncTests target to the 'tests' directory, excluding it from all targets and assigning it to the VPUX_TESTS_COMPONENT. ```cmake install(TARGETS ${TARGET_NAME} RUNTIME DESTINATION tests COMPONENT ${VPUX_TESTS_COMPONENT} EXCLUDE_FROM_ALL ) ``` -------------------------------- ### Install clang-tidy on Ubuntu Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/developer_tools.md Installs clang-tidy using the standard Ubuntu package manager. This is the recommended method for Ubuntu users. ```sh sudo apt install clang-tidy ``` -------------------------------- ### Install Ccache on Ubuntu Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/developer_tools.md Install Ccache, a compiler cache that speeds up recompilation, on Ubuntu using the standard package manager. ```sh sudo apt install ccache ``` -------------------------------- ### gcov Output Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_profile_code.md An example of the output generated by the gcov tool, indicating lines executed and the creation of a '.gcov' file. ```text File 'propagate_quantize_dequantize.cpp Lines executed:3.91% of 128 Creating 'propagate_quantize_dequantize.cpp.gcov' ... ``` -------------------------------- ### Testing Pipelines with --vpu-arch Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_test.md Specify the VPU architecture using the --vpu-arch command-line argument to test pipelines like DefaultHW or ReferenceSW. This example also includes options for splitting input files and eliding large element attributes. ```MLIR // RUN: vpux-opt --vpu-arch=NPU40XX --split-input-file --mlir-elide-elementsattrs-if-larger 8 --default-hw-mode %s | FileCheck %s ``` -------------------------------- ### Install lxml Python Library Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tools/edit-xml/README.md Install the lxml library using pip3. This library is required for the edit XML tool to function. ```bash pip3 install lxml ``` -------------------------------- ### MLIR Module with Custom Operation Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/primer_mlir.md An example MLIR module demonstrating the usage of the IE.GettingStarted operation with its custom assembly format. This shows how the operation is represented in the MLIR IR. ```mlir module { func.func @main(%arg0: tensor<1x16x3x3xf16>) -> tensor<1x16x3x3xf16> { %0 = IE.GettingStarted(%arg0) : tensor<1x16x3x3xf16> -> tensor<1x16x3x3xf16> return %0 : tensor<1x16x3x3xf16> } } ``` -------------------------------- ### Memory Scheduler Task Entry Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/how_to_debug.md An example of an entry in the scheduleTrace.json file generated by the memory scheduler, detailing task information. ```json {"name":"TASK_NAME", "cat":"NCE", "ph":"X", "ts":0, "dur":7175, "pid":0, "tid":0}, ``` -------------------------------- ### vclExecutableCreate Full Procedure Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/api_reference.md This section details the full procedure for creating an executable using `vclExecutableCreate`. ```APIDOC ## vclExecutableCreate Full Procedure ### Description This procedure outlines the steps involved in creating an executable using the `vclExecutableCreate` function, which is part of the network compilation process. ### Workflow 1. **Create a compiler instance**: Use `vclCompilerCreate()` to obtain a compiler handle. 2. **Create an executable**: Call `vclExecutableCreate()` with the compiler handle and other necessary parameters to create an executable object. 3. **Get the compiled blob**: Use `vclExecutableGetSerializableBlob()` to retrieve the compiled model data. 4. **Destroy the executable**: Clean up resources by calling the appropriate destroy function for the executable handle. 5. **Destroy the compiler**: Clean up resources by calling `vclCompilerDestroy()`. ``` -------------------------------- ### IE.Reorder Operation Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/operators/Permute.md An example of an IE.Reorder operation used for layout adjustment in the IE dialect. It specifies the destination order as NHWC. ```mlir %0 = IE.Reorder(%input) { dstOrder = #NHWC } : tensor<1x3x224x224xf16> -> tensor<1x3x224x224xf16, {order = #NHWC}> ``` -------------------------------- ### Testing Passes with --init-compiler Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_test.md Use the --init-compiler pass with the 'vpu-arch' option to test specific passes and sub-pipelines. This example also includes splitting the input file and unrolling distributed operations. ```MLIR // RUN: vpux-opt --split-input-file --init-compiler="vpu-arch=NPU40XX" --unroll-distributed-ops %s | FileCheck %s ``` -------------------------------- ### Run ExpandActivationChannels and Canonicalizer Passes Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/how_to_debug.md Use vpux-opt to apply multiple passes, ExpandActivationChannels and Canonicalizer, on an MLIR file. ```bash vpux-opt --platform=NPU3720 --expand-activation-channels --canonicalizer ``` -------------------------------- ### Installing Python Tools with CMake Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tools/CMakeLists.txt Installs Python tools to specific destinations within the build. This is used for utility scripts and analysis tools. ```cmake set(PYTHON_TOOLS "edit-xml" "ir-analyzer" "model-output-comparator" ) foreach(TOOL IN LISTS PYTHON_TOOLS) install(DIRECTORY "${CMAKE_CURRENT_LIST_DIR}/${TOOL}/" DESTINATION "tools/${TOOL}" COMPONENT tests EXCLUDE_FROM_ALL) # TODO: Remove duplication E#31024 install(DIRECTORY "${CMAKE_CURRENT_LIST_DIR}/${TOOL}/" DESTINATION "tools/${TOOL}" COMPONENT ${VPUX_TESTS_COMPONENT} EXCLUDE_FROM_ALL) endforeach() ``` -------------------------------- ### Install NPU Smoke Test Target Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/tests/smoke/CMakeLists.txt Installs the defined test target to the 'tests' runtime destination, associated with the VPUX_TESTS_COMPONENT, and excludes it from all builds. ```cmake install(TARGETS ${TARGET_NAME} RUNTIME DESTINATION tests COMPONENT ${VPUX_TESTS_COMPONENT} EXCLUDE_FROM_ALL ) ``` -------------------------------- ### Quick Build Commands for VPU Driver Compiler with LLVM Cache Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_llvm_cache_windows.md Execute these commands for a rapid build of the Driver Compiler using LLVM cache. Ensure environment variables like %LLVMPKG_CACHE_HOME%, %OPENVINO_HOME%, %NPU_PLUGIN_HOME%, %CONFIG%, %NUMBER_OF_PROCESSORS%, and %LLVM_LIB_PATH% are set. ```bat md %LLVMPKG_CACHE_HOME% md %OPENVINO_HOME%\build_llvm_%CONFIG% cd %OPENVINO_HOME%\build_llvm_%CONFIG% cmake \ -G Ninja \ -D CMAKE_BUILD_TYPE=%CONFIG% \ -D OPENVINO_EXTRA_MODULES=%NPU_PLUGIN_HOME% \ -D CMAKE_TOOLCHAIN_FILE=%OPENVINO_HOME%\cmake\toolchains\onecoreuap.toolchain.cmake \ .. ninja build-modules\vpux_plugin\thirdparty\llvm-project\llvm\all -j %NUMBER_OF_PROCESSORS% cmake --install %LLVM_LIB_PATH% --config %CONFIG% --prefix %LLVMPKG_CACHE_HOME% md %OPENVINO_HOME%\build_%CONFIG% cd %OPENVINO_HOME%\build_%CONFIG% cmake \ -G Ninja \ -D CMAKE_BUILD_TYPE=%CONFIG% \ -D BUILD_SHARED_LIBS=OFF \ -D OPENVINO_EXTRA_MODULES=%NPU_PLUGIN_HOME% \ -D ENABLE_PREBUILT_LLVM_MLIR_LIBS=ON \ -D MLIR_BINARY_PKG_DIR=%LLVMPKG_CACHE_HOME% \ %CommonBuildOptions% \ .. ninja npu_driver_compiler compilerTest profilingTest vpuxCompilerL0Test loaderTest -j %NUMBER_OF_PROCESSORS% @REM Optional, compress and pack all CiD targets cpack -V -D CPACK_COMPONENTS_ALL=CiD -D CPACK_CMAKE_GENERATOR=Ninja -D CPACK_PACKAGE_FILE_NAME=%CONFIG% -G "ZIP" ``` -------------------------------- ### Install Test Executables Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/test/smoke/CMakeLists.txt Installs the compilerTest, profilingTest, and loaderTest executables along with their debug files if BUILD_COMPILER_FOR_DRIVER is enabled. This is for integration into CI/CD components. ```cmake if(BUILD_COMPILER_FOR_DRIVER) install_target_with_debug_files(compilerTest ${INSTALL_DESTINATION} ${INSTALL_COMPONENT}) install_target_with_debug_files(profilingTest ${INSTALL_DESTINATION} ${INSTALL_COMPONENT}) install_target_with_debug_files(loaderTest ${INSTALL_DESTINATION} ${INSTALL_COMPONENT}) endif() ``` -------------------------------- ### Install Android NDK Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_cmake_presets_android.md Download and install the Android NDK to the specified directory. Ensure the path matches the one used in CMake configuration. ```sh curl -L https://dl.google.com/android/repository/android-ndk-r27c-linux.zip -o /tmp/packages/android-ndk.zip sudo unzip /tmp/packages/android-ndk.zip -d /opt/ ``` -------------------------------- ### Environment Setup for Windows Build Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_llvm_cache_windows.md Set these environment variables in an x64 Native Tools Command Prompt for VS 2022. Ensure paths are correctly set for OpenVINO, NPU plugin, and optionally TBB and LLVM cache. ```bat set OPENVINO_HOME=C:\path\to\cloned\openvino set NPU_PLUGIN_HOME=C:\path\to\cloned\applications.ai.vpu-accelerators.vpux-plugin @REM Or Debug/RelWithDebInfo set CONFIG=Release @REM Optional, only if using custom TBB, but required for enabling sideloading set TBBROOT=C:\path\to\your\onecore\tbb set LLVMPKG_CACHE_HOME=C:\path\to\your\saved\llvm_cache_%CONFIG% set LLVM_LIB_PATH=%OPENVINO_HOME%\build_llvm_%CONFIG%\build-modules\vpux_plugin\thirdparty\llvm-project\llvm ``` -------------------------------- ### Run LowPrecision Pipeline Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/how_to_debug.md Use vpux-opt to execute the LowPrecision pipeline on an MLIR file. ```bash vpux-opt --platform=NPU3720 --low-precision ``` -------------------------------- ### Install CiD Components to a Specific Location Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_llvm_cache_windows.md This command installs all CiD components of the Driver Compiler to a specified location. Use this to deploy the built components after packaging. ```bat @REM Install all CiD components cd %OPENVINO_HOME%\build_%CONFIG% cmake --install .\ --prefix .\ --component CiD ``` -------------------------------- ### Create Profiling Handle Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/api_reference.md Initializes a profiling session by creating a profiling handle. Requires input data including the compiled blob and raw profiling data. Outputs a profiling handle and a log handle for the session. ```c vcl_profiling_input_t profilingInput = { .blobData = blobBuffer, // Compiled blob data .blobSize = blobSize, // Size of blob data .profData = profRawData, // Raw profiling data from execution .profSize = profRawSize // Size of raw profiling data }; vcl_profiling_handle_t profHandle; vcl_log_handle_t profLogHandle; result = vclProfilingCreate(&profilingInput, &profHandle, &profLogHandle); ``` -------------------------------- ### Activation Layer Test Setup (C++) Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/sw_layer_enabling.md Sets up and runs activation layer tests in software mode for NPU3720. Configures absolute threshold and default hardware mode. ```cpp #include "single_op_tests/ctc_greedy_decoder.hpp" #include "vpu_ov2_layer_test.hpp" #include using namespace ov::test::utils; using ov::test::ActivationParamLayerTest; namespace ov::test { class ActivationLayerTestCommon : public ActivationLayerTest, virtual public VpuOv2LayerTest {}; class ActivationLayerTest_SW_FP16 : public ActivationLayerTestCommon {}; class ActivationLayerTest_HW_FP16 : public ActivationLayerTestCommon {}; // SW TESTP(ActivationLayerTest_SW_FP16, NPU3720) { abs_threshold = 0.0056; setReferenceSoftwareMode(); run(Platform::NPU3720); } // HW TEST_P(ActivationLayerTest_HW_FP16, NPU3720) { abs_threshold = 0.0056; setDefaultHardwareMode(); run(Platform::NPU3720); } } // namespace ov::test using namespace ov::test; namespace { const std::vector netPrecisions = {ov::element::f16}; const std::map>> activationTypes = { {Sigmoid, {{1.0f}}}, ... }; std::map, std::vector> basic = {{{{1, 50, 1, 1}}, {}}, {{{1, 128, 1, 1}}, {}}}; auto static_shapes_param_transform = [](const std::vector, ov::Shape>>& original_shapes) { std::vector, ov::Shape>> new_shapes; for (const auto& shape_element : original_shapes) { new_shapes.emplace_back(ov::test::static_shapes_to_test_representation(shape_element.first), shape_element.second); } return new_shapes; }; const auto basicCases = ::testing::Combine(::testing::ValuesIn(::combineParams(activationTypes)), // Activation type and constant ::testing::ValuesIn(netPrecisions), // Model type ::testing::ValuesIn(static_shapes_param_transform( ov::test::utils::combineParams(basic))), // Input shapes and input const shape ::testing::Values(test_utils::TARGET_DEVICE)); // Target device name INSTANTIATE_TEST_SUITE_P(smoke_precommit_Activation, ActivationLayerTest_SW_FP16, basicCases, ActivationLayerTest::getTestCaseName); ``` -------------------------------- ### Install Ninja build generator on Ubuntu Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/developer_tools.md Install the Ninja build generator on Ubuntu using the standard package manager. Ninja is a fast build generator alternative to Makefiles. ```sh sudo apt install ninja-build ``` -------------------------------- ### Execute a Reproducer MLIR File Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/guides/how_to_debug.md Use `vpux-opt` with the `--platform` argument to execute a generated MLIR reproducer file. ```sh vpux-opt --platform=NPU3720 reproducer.mlir ``` -------------------------------- ### Exporting with vpux-opt and vpux-translate Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_test.md This RUN command demonstrates exporting a model using vpux-opt with an initialized compiler (including vpu-arch) and then piping it to vpux-translate for export to VPUIP format. ```MLIR // RUN: vpux-opt --init-compiler="vpu-arch=NPU40XX" %s | vpux-translate --vpu-arch=NPU40XX --export-VPUIP -o %t ``` -------------------------------- ### Build Default HW Mode Pipeline (Bad Example) Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/code_style.md Illustrates a less desirable way to build a pipeline by listing many individual passes, which obscures dependencies and makes isolated testing difficult. This approach can lead to issues when the pipeline is modified. ```cpp void vpux::buildDefaultHWModePipeline(mlir::OpPassManager& pm, const DefaultHWOptions37XX& options, Logger log) { // ... pm.addPass(IE::createSwapOperationsPass(isOptionEnabled(options.enableSEPtrsOperations), log)); pm.addPass(IE::createInsertIdentityPoolBeforeActivationPass(log)); pm.addPass(IE::createFuseActivationOpsPass(log)); pm.addPass(mlir::createCanonicalizerPass(grc)); // ... } ``` -------------------------------- ### vclCompilerCreate Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/api_reference.md Creates a compiler instance for a specific device. This function initializes the compiler with provided device and logging configurations. ```APIDOC ## vclCompilerCreate ### Description Creates a compiler instance for a specific device. ### Function Signature ```c vcl_result_t vclCompilerCreate(vcl_compiler_desc_t* compilerDesc, vcl_device_desc_t* deviceDesc, vcl_compiler_handle_t* compiler, vcl_log_handle_t* logHandle); ``` ### Parameters #### Input Parameters - **compilerDesc** (vcl_compiler_desc_t*): Pointer to compiler descriptor. - **deviceDesc** (vcl_device_desc_t*): Pointer to device descriptor. #### Output Parameters - **compiler** (vcl_compiler_handle_t*): Returns the compiler handle. - **logHandle** (vcl_log_handle_t*): Returns the log handle. ### Usage Example ```c vcl_compiler_desc_t compilerDesc = { .version = {7, 4}, // API version .debugLevel = VCL_LOG_INFO // Debug level }; vcl_device_desc_t deviceDesc = { .size = sizeof(vcl_device_desc_t), .deviceID = 0x1234, // PCI Device ID in lower 16 bits .revision = 0, // NPU Revision (0 for first stepping) .tileCount = 1 // Number of slices/tiles }; vcl_compiler_handle_t compiler; vcl_log_handle_t logHandle; vcl_result_t result = vclCompilerCreate(&compilerDesc, &deviceDesc, &compiler, &logHandle); ``` ### Related Types `vcl_log_level_t` for `debugLevel` field of `vcl_compiler_desc_t` struct: - **VCL_LOG_NONE**: Logging disabled - **VCL_LOG_ERROR**: Error events - **VCL_LOG_WARNING**: Warning events - **VCL_LOG_INFO**: Informational messages - **VCL_LOG_DEBUG**: Debug messages - **VCL_LOG_TRACE**: Trace-level messages ``` -------------------------------- ### Build All OpenVINO Components Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/guides/how_to_build_static.md This command builds all selected OpenVINO components for a static release configuration. It's a general build command for the entire project. ```bash cmake --build . --config Release --parallel ``` -------------------------------- ### MLIR Example for Weights Separation Initialization Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/design/bytecode_format.md This MLIR snippet illustrates the initialization phase for weights separation, showing two init functions that transform constants. It includes options for calling NPU kernels or performing manual transformations. ```mlir functions_section: [ // This example contains two init functions, whose goal is to transform the constants from the original model, such that they can be used by the main function. // init0 will process two constants, while init1 will process three constants. // These functions will only be called once, after which main can be called any number of times to execute the actual inferences. 0: @init0, num_args=2, num_results=0, { // Register rp0 contains the original constants // Register rp1 should be populated with the transformed constants by this init function // Option 1. Call NPU kernel associated with the first init function, which receives the same input / output buffers kernel.create ... // Option 2. Manually process each constant via dedicated NPU kernels / CPU transformations add.i64 r0, rp0, 0 // First constant is found at offset 0 and has 1024 bytes add.i64 r1, rp0, 1024 // Second constant is found at offset 1024 add.i64 r2, rp1, 0 // First transformed constant should be placed at offset 0 and should have 2048 bytes add.i64 r3, rp1, 2048 // Second constant should be placed at offset 2048 // Process constants inside r0, r1 and store the results in r2, r3 ret } 1: @init1, num_args=2, num_results=0, { } // This example assumes that the original model has one input value and one output value. The main function receives the two buffers as arguments, // while also receiving the two buffers with transformed constants, that are populated by the init functions. 2: @main, num_args=4, num_results=0, { } ] ``` -------------------------------- ### IE.PermuteQuantize Operation Example Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_compiler/docs/operators/Permute.md An example of an IE.PermuteQuantize operation in the IE dialect, converting NCHW to NHWC layout and potentially handling channel expansion. It includes parameters for destination element type, order, memory permutation, and padding. ```mlir %0 = IE.PermuteQuantize(%input) { dstElemType = !qElemType, dst_order = #NHWC, mem_perm = #NHWC, pads_begin = [0, 0, 0, 0], pads_end = [0, 1, 0, 0] } : tensor<1x3x224x224xf16> -> tensor<1x4x224x224x!qElemType, {order = #NHWC}> ``` -------------------------------- ### Configure CMake with Presets Source: https://github.com/openvinotoolkit/npu_compiler/blob/develop/src/vpux_driver_compiler/docs/build/build_with_cmake_presets_windows.md Configure the build using the 'cid-windows' CMake preset. The build options for this preset are detailed in the referenced CMakePresets.json file. ```bat cd %OPENVINO_HOME% cmake --preset cid-windows ```