### Quick Start: Basic Usage Source: https://github.com/tencent/ncnn/blob/master/tools/onnx/README.md Install pnnx and run the conversion command. This will generate .ncnn.param and .ncnn.bin files. ```shell pip3 install pnnx # Syntax: pnnx # Example: pnnx my_model.onnx ``` -------------------------------- ### net.param example Source: https://github.com/tencent/ncnn/wiki/developer-guide/param-and-model-file-structure Example structure of a net.param file. ```text 7767517 3 3 Input input 0 1 data 0=4 1=4 2=1 InnerProduct ip 1 1 data fc 0=10 1=1 2=80 Softmax softmax 1 1 fc prob 0=0 ``` -------------------------------- ### Python setup command Source: https://github.com/tencent/ncnn/wiki/faq.en Command to install a Python package in development mode. ```bash python setup.py develop ``` -------------------------------- ### Build and install ncnn from source Source: https://github.com/tencent/ncnn/blob/master/python/README.md Installs ncnn from source using setup.py. ```bash python setup.py install ``` -------------------------------- ### Example Building Logic Source: https://github.com/tencent/ncnn/blob/master/examples/CMakeLists.txt Configures and builds various ncnn examples, with conditional logic for OpenCV availability. ```cmake if(NCNN_PIXEL) if(NOT NCNN_SIMPLEOCV) find_package(OpenCV QUIET COMPONENTS opencv_world) # for opencv 2.4 on ubuntu 16.04, there is no opencv_world but OpenCV_FOUND will be TRUE if("${OpenCV_LIBS}" STREQUAL "") set(OpenCV_FOUND FALSE) endif() if(NOT OpenCV_FOUND) find_package(OpenCV QUIET COMPONENTS core highgui imgproc imgcodecs videoio) endif() if(NOT OpenCV_FOUND) find_package(OpenCV QUIET COMPONENTS core highgui imgproc) endif() endif() if(OpenCV_FOUND OR NCNN_SIMPLEOCV) if(OpenCV_FOUND) message(STATUS "OpenCV library: ${OpenCV_INSTALL_PATH}") message(STATUS " version: ${OpenCV_VERSION}") message(STATUS " libraries: ${OpenCV_LIBS}") message(STATUS " include path: ${OpenCV_INCLUDE_DIRS}") if(${OpenCV_VERSION_MAJOR} GREATER 3) set(CMAKE_CXX_STANDARD 11) endif() endif() include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../src) include_directories(${CMAKE_CURRENT_BINARY_DIR}/../src) ncnn_add_example(arcface) ncnn_add_example(squeezenet) ncnn_add_example(squeezenet_c_api) ncnn_add_example(fasterrcnn) ncnn_add_example(rfcn) ncnn_add_example(yolov2) ncnn_add_example(yolov3) ncnn_add_example(yolov5) ncnn_add_example(yolov5_pnnx) ncnn_add_example(yolov7_pnnx) ncnn_add_example(yolov7) ncnn_add_example(yolov8) ncnn_add_example(yolov8_seg) ncnn_add_example(yolov8_pose) ncnn_add_example(yolov8_cls) ncnn_add_example(yolox) ncnn_add_example(yolo11) ncnn_add_example(yolo11_seg) ncnn_add_example(yolo11_pose) ncnn_add_example(yolo11_cls) ncnn_add_example(yoloworld) ncnn_add_example(mobilenetv2ssdlite) ncnn_add_example(mobilenetssd) ncnn_add_example(squeezenetssd) ncnn_add_example(shufflenetv2) ncnn_add_example(peleenetssd_seg) ncnn_add_example(simplepose) ncnn_add_example(retinaface) ncnn_add_example(yolact) ncnn_add_example(nanodet) ncnn_add_example(nanodetplus_pnnx) ncnn_add_example(scrfd) ncnn_add_example(scrfd_crowdhuman) ncnn_add_example(piper) ncnn_add_example(whisper) if(OpenCV_FOUND) ncnn_add_example(yolov4) ncnn_add_example(yolov8_obb) ncnn_add_example(yolo11_obb) ncnn_add_example(rvm) ncnn_add_example(p2pnet) ncnn_add_example(ppocrv5) endif() else() message(WARNING "OpenCV not found and NCNN_SIMPLEOCV disabled, examples won't be built") endif() else() message(WARNING "NCNN_PIXEL not enabled, examples won't be built") endif() ``` -------------------------------- ### Install CMake Source: https://github.com/tencent/ncnn/wiki/faq Steps to download and install a specific version of CMake. ```shell wget https://github.com/Kitware/CMake/releases/download/v3.18.2/cmake-3.18.2-Linux-x86_64.tar.gz tar zxvf cmake-3.18.2-Linux-x86_64.tar.gz mv cmake-3.18.2-Linux-x86_64 /opt/cmake-3.18.2 ln -sf /opt/cmake-3.18.2/bin/* /usr/bin/ ``` -------------------------------- ### A53 Practical Guide Example Source: https://github.com/tencent/ncnn/wiki/developer-guide/arm-a53-a55-dual-issue Practical guide for A53 dual-issuing 64-bit vector loads with fmla, including pointer updates and interleaved inserts. ```assembly ldr d0, [r0] // 1 cycle, v0 first 64bit fmla ldr x23, [r0, #8] // 0 cycle, v0 second 64bit to temp register fmla add r0, r0, #16 // 0 cycle, update pointer fmla ldr d1, [r0] // 1 cycle, v1 first 64bit ins v0.d[1], x23 // 0 cycle, v0 second 64bit complete fmla ldr x23, [r0, #8] // 0 cycle, v1 second 64bit to temp register fmla add r0, r0, #16 // 0 cycle, update pointer fmla ins v1.d[1], x23 // 1 cycle, v1 second 64bit complete nop fmla fmla fmla nop nop fmla fmla fmla ``` -------------------------------- ### Build and install pnnx from source Source: https://github.com/tencent/ncnn/blob/master/tools/pnnx/python/README.md Navigates to the pnnx Python directory and installs it using setup.py. ```bash cd /pathto/ncnn/tools/pnnx/python python setup.py install ``` -------------------------------- ### A55 Practical Guide Example Source: https://github.com/tencent/ncnn/wiki/developer-guide/arm-a53-a55-dual-issue Practical guide for A55 dual-issuing instructions, including 64-bit vector loads, integer loads, pointer updates, and vector inserts with fmla. ```assembly ldr d0, [r0] // 0 cycle, v0 first 64bit fmla ldr x23, [r0, #8] // 0 cycle, v0 second 64bit to temp register fmla add r0, r0, #16 // 0 cycle, update pointer fmla ldr d1, [r0] // 0 cycle, v1 first 64bit fmla ins v0.d[1], x23 // 0 cycle, v0 second 64bit complete fmla ldr x23, [r0, #8] // 0 cycle, v1 second 64bit to temp register fmla add r0, r0, #16 // 0 cycle, update pointer fmla ins v1.d[1], x23 // 0 cycle, v1 second 64bit complete fmla ``` -------------------------------- ### Verify build by running examples Source: https://github.com/tencent/ncnn/wiki/how-to-build/how-to-build Example command to run a verification test using the squeezenet model. ```shell cd ../examples ../build/examples/squeezenet ../images/256-ncnn.png ``` -------------------------------- ### Natural Assembly Example Source: https://github.com/tencent/ncnn/wiki/developer-guide/arm-a53-a55-dual-issue Example of natural assembly instructions with no register dependency, allowing for dual issue without penalty. ```assembly ld1 {v0.4s}, [r0], #16 fmla v10.4s, v16.4s, v24.s[0] fmla v11.4s, v16.4s, v24.s[1] fmla v12.4s, v16.4s, v24.s[2] fmla v13.4s, v16.4s, v24.s[3] ``` -------------------------------- ### Build for QNX - Setup Environment (Windows) Source: https://github.com/tencent/ncnn/wiki/how-to-build/how-to-build Sets up the QNX environment on Windows by calling a batch script. ```batch call C:\Users\zz\qnx800\qnxsdp-env.bat ``` -------------------------------- ### Export to TorchScript Source: https://github.com/tencent/ncnn/blob/master/tools/pytorch/README.md Example Python script to export a PyTorch model to a TorchScript (.pt) file. ```python import torch import torch.nn as nn # Define or load your model as in the example above class MyModel(nn.Module): def __init__(self): super(MyModel, self).__init__() self.conv1 = nn.Conv2d(3, 16, 3, 1, 1) self.relu = nn.ReLU() self.fc = nn.Linear(16 * 224 * 224, 10) def forward(self, x): x = self.conv1(x) x = self.relu(x) x = x.view(x.size(0), -1) x = self.fc(x) return x model = MyModel() model.eval() # Create a dummy input input_tensor = torch.rand(1, 3, 224, 224) # Trace the model to generate a TorchScript file traced_module = torch.jit.trace(model, input_tensor) traced_module.save("my_model.pt") print("TorchScript model saved to my_model.pt") ``` -------------------------------- ### Example Param File Content Source: https://github.com/tencent/ncnn/wiki/developer-guide/how-to-implement-custom-layer-step-by-step Illustrates the structure of an ncnn parameter file, showing how layers and their parameters are defined. ```text // example param file content Input input 0 1 input Convolution conv2d 1 1 input conv2d 0=32 1=1 2=1 3=1 4=0 5=0 6=768 MyLayer mylayer 1 1 conv2d mylayer0 Pooling maxpool 1 1 mylayer0 maxpool 0=0 1=3 2=2 3=-233 4=0 ``` -------------------------------- ### Direct Conversion with pnnx.export Source: https://github.com/tencent/ncnn/blob/master/tools/pytorch/README.md Example Python script demonstrating direct conversion of a PyTorch model to ncnn format using pnnx.export. ```python import torch import torch.nn as nn import pnnx # 1. Define your pytorch model class MyModel(nn.Module): def __init__(self): super(MyModel, self).__init__() self.conv1 = nn.Conv2d(3, 16, 3, 1, 1) self.relu = nn.ReLU() self.fc = nn.Linear(16 * 224 * 224, 10) def forward(self, x): x = self.conv1(x) x = self.relu(x) x = x.view(x.size(0), -1) x = self.fc(x) return x # 2. Instantiate your model and set it to evaluation mode model = MyModel() model.eval() # 3. Create a dummy input tensor with the correct shape # Format: [batch, channels, height, width] input_tensor = torch.rand(1, 3, 224, 224) # 4. Export the model to ncnn format # The first argument is the model instance. # The second argument is the base path for the output files. # The third argument is a tuple of input tensors. pnnx.export(model, "my_model.pt", (input_tensor,)) print("Conversion finished! Check for my_model.ncnn.param and my_model.ncnn.bin") ``` -------------------------------- ### Benchmark Command Example 2 Source: https://github.com/tencent/ncnn/blob/master/benchmark/README.md Example command to run benchmarks on MediaTek Dimensity 9300 with different parameters. ```bash ../build-android/benchmark/benchncnn 8 1 2 -1 1 ``` -------------------------------- ### Build for QNX - Setup Environment (Linux) Source: https://github.com/tencent/ncnn/wiki/how-to-build/how-to-build Sets up the QNX environment on Linux by sourcing a shell script. ```shell source /home/zz/qnx800/qnxsdp-env.sh ``` -------------------------------- ### Register binding for d registers Source: https://github.com/tencent/ncnn/wiki/developer-guide/armv7-mix-assembly-and-intrinsic Example demonstrating how to bind specific d registers for saving values using the 'register' keyword. This explicitly assigns registers like d0, d2, and d4. ```c // d寄存器声明绑定 // specify concrete d reg which want to save // vmla.f32 d0, d2, d4 register float32x2_t _a asm("d0") = vld1_f32(a); register float32x2_t _b asm("d2") = vld1_f32(b); register float32x2_t _c asm("d4") = vld1_f32(c); asm volatile( "vmla.f32 %P0, %P2, %P3" : "=w"(_a) // %0 : "0"(_a), "w"(_b), // %2 "w"(_c) // %3 : ); ``` -------------------------------- ### d register usage with %P Source: https://github.com/tencent/ncnn/wiki/developer-guide/armv7-mix-assembly-and-intrinsic Example of using the 'vmla.f32' instruction with d registers, where %P is used to match d registers. This performs the operation a += b * c for 2 float32 elements. ```c // d寄存器全部使用 %P // d reg matches %P // a += b * c float32x2_t _a = vld1_f32(a); float32x2_t _b = vld1_f32(b); float32x2_t _c = vld1_f32(c); asm volatile( "vmla.f32 %P0, %P2, %P3" : "=w"(_a) // %0 : "0"(_a), "w"(_b), // %2 "w"(_c) // %3 : ); ``` -------------------------------- ### Register binding for q registers Source: https://github.com/tencent/ncnn/wiki/developer-guide/armv7-mix-assembly-and-intrinsic Example demonstrating how to bind specific q registers for data using the 'register' keyword. This explicitly assigns registers like q0, q1, and q2. ```c // q寄存器声明绑定 // bind q reg with data // vmla.f32 q0, q1, q2 register float32x4_t _a asm("q0") = vld1q_f32(a); register float32x4_t _b asm("q1") = vld1q_f32(b); register float32x4_t _c asm("q2") = vld1q_f32(c); asm volatile( "vmla.f32 %q0, %q2, %q3" : "=w"(_a) // %0 : "0"(_a), "w"(_b), // %2 "w"(_c) // %3 : ); ``` -------------------------------- ### Benchmark Execution Example 2 Source: https://github.com/tencent/ncnn/blob/master/benchmark/README.md Example command to run benchmarks with different settings, showing results for various models. ```bash loop_count = 4 num_threads = 1 powersave = 2 gpu_device = 0 cooling_down = 0 squeezenet min = 9.73 max = 11.72 avg = 10.55 squeezenet_int8 min = 7.21 max = 7.34 avg = 7.27 mobilenet min = 10.87 max = 13.09 avg = 12.01 mobilenet_int8 min = 8.82 max = 9.23 avg = 9.11 mobilenet_v2 min = 15.77 max = 16.21 avg = 15.96 mobilenet_v3 min = 18.04 max = 18.68 avg = 18.40 shufflenet min = 9.82 max = 11.92 avg = 10.79 shufflenet_v2 min = 14.41 max = 15.41 avg = 14.96 mnasnet min = 16.01 max = 16.43 avg = 16.27 proxylessnasnet min = 14.18 max = 16.28 avg = 15.51 efficientnet_b0 min = 36.38 max = 37.06 avg = 36.83 efficientnetv2_b0 min = 55.98 max = 66.59 avg = 59.54 regnety_400m min = 21.94 max = 22.46 avg = 22.30 blazeface min = 3.92 max = 4.47 avg = 4.08 googlenet min = 31.79 max = 35.63 avg = 33.04 googlenet_int8 min = 23.21 max = 29.38 avg = 26.60 resnet18 min = 22.61 max = 24.05 avg = 23.09 resnet18_int8 min = 24.56 max = 24.78 avg = 24.62 alexnet min = 25.98 max = 27.05 avg = 26.49 vgg16 min = 39.00 max = 39.82 avg = 39.29 vgg16_int8 min = 207.47 max = 208.56 avg = 207.90 resnet50 min = 44.07 max = 44.43 avg = 44.29 resnet50_int8 min = 44.77 max = 47.04 avg = 45.44 squeezenet_ssd min = 33.71 max = 34.27 avg = 34.09 squeezenet_ssd_int8 min = 22.53 max = 30.33 avg = 25.07 mobilenet_ssd min = 26.91 max = 28.35 avg = 27.42 mobilenet_ssd_int8 min = 19.43 max = 19.82 avg = 19.69 mobilenet_yolo min = 28.03 max = 29.19 avg = 28.65 mobilenetv2_yolov3 min = 33.54 max = 34.65 avg = 34.31 yolov4-tiny min = 49.77 max = 51.21 avg = 50.55 nanodet_m min = 17.35 max = 18.83 avg = 18.06 yolo-fastest-1.1 min = 9.45 max = 9.59 avg = 9.51 yolo-fastestv2 min = 13.13 max = 13.63 avg = 13.36 ``` -------------------------------- ### PNNX weight binary file example Source: https://github.com/tencent/ncnn/blob/master/tools/pnnx/README.md Example demonstrating how operator and weight names are composed in a pnnx.bin file. ```text nn.Conv2d conv_0 1 1 0 1 bias=1 dilation=(1,1) groups=1 in_channels=12 kernel_size=(3,3) out_channels=16 padding=(0,0) stride=(1,1) @bias=(16) @weight=(16,12,3,3) ``` -------------------------------- ### Splitting q register into d registers with %e and %f Source: https://github.com/tencent/ncnn/wiki/developer-guide/armv7-mix-assembly-and-intrinsic Example of using 'vmla.f32' where a q register is split into two d registers using %e and %f. This performs operations on pairs of float32 elements. ```c // q寄存器拆分d寄存器使用 %e %f // use %e %f to split q reg into two d regs // a += b * c[0]c[1] // a += b * c[2]c[3] float32x2_t _a = vldq_f32(a); float32x2_t _b = vldq_f32(b); float32x4_t _c = vld1q_f32(c); asm volatile( "vmla.f32 %P0, %P2, %e3" "vmla.f32 %P0, %P2, %f3" : "=w"(_a) // %0 : "0"(_a), "w"(_b), // %2 "w"(_c) // %3 : ); ``` -------------------------------- ### q register usage with %q Source: https://github.com/tencent/ncnn/wiki/developer-guide/armv7-mix-assembly-and-intrinsic Example of using the 'vmla.f32' instruction with q registers, where %q is used to match q registers. This performs the operation a += b * c for 4 float32 elements. ```c // q寄存器全部使用 %q // q reg matches %q // a += b * c float32x4_t _a = vld1q_f32(a); float32x4_t _b = vld1q_f32(b); float32x4_t _c = vld1q_f32(c); asm volatile( "vmla.f32 %q0, %q2, %q3" : "=w"(_a) // %0 : "0"(_a), "w"(_b), // %2 "w"(_c) // %3 : ); ``` -------------------------------- ### Benchmark Command Example 2 Source: https://github.com/tencent/ncnn/blob/master/benchmark/README.md Another example command to run the benchmark with different parameters. ```bash i@ubuntu:~/projects/ncnn/benchmark$ ./benchncnn 128 1 0 0 0 [0 NVIDIA Tegra Xavier (nvgpu)] queueC=2[8] queueG=0[16] queueT=1[1] [0 NVIDIA Tegra Xavier (nvgpu)] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0 [0 NVIDIA Tegra Xavier (nvgpu)] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 NVIDIA Tegra Xavier (nvgpu)] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1 loop_count = 128 num_threads = 1 powersave = 0 gpu_device = 0 cooling_down = 0 ``` -------------------------------- ### d register single lane usage with %P[index] Source: https://github.com/tencent/ncnn/wiki/developer-guide/armv7-mix-assembly-and-intrinsic Example of using 'vmla.f32' with d registers, specifying individual lanes using %P[index]. This performs a += b * c[0] and a += b * c[1] for two separate float32 elements. ```c // d寄存器单路使用 %P[0] %P[1] // 32bit d reg matches %P[0] // a += b * c[0] // a += b * c[1] float32x2_t _a = vld1_f32(a); float32x2_t _b = vld1_f32(b); float32x2_t _c = vld1_f32(c); asm volatile( "vmla.f32 %P0, %P2, %P3[0]" "vmla.f32 %P0, %P2, %P3[1]" : "=w"(_a) // %0 : "0"(_a), "w"(_b), // %2 "w"(_c) // %3 : ); ``` -------------------------------- ### Benchmark Command Example 1 Source: https://github.com/tencent/ncnn/blob/master/benchmark/README.md Example command to run the benchmark with specific parameters. ```bash i@ubuntu:~/projects/ncnn/benchmark$ ./benchncnn 32 8 0 -1 0 loop_count = 32 num_threads = 8 powersave = 0 gpu_device = -1 cooling_down = 0 ``` -------------------------------- ### q register single lane usage with %e[index] and %f[index] Source: https://github.com/tencent/ncnn/wiki/developer-guide/armv7-mix-assembly-and-intrinsic Example of using 'vmla.f32' with q registers, specifying individual lanes using %e[index] and %f[index]. This performs operations on four float32 elements, accessing lanes within the q register. ```c // q寄存器单路使用 %e[0] %e[1] %f[0] %f[1] // 32-bit q reg matches %e[0] // a += b * c[0] // a += b * c[1] // a += b * c[2] // a += b * c[3] float32x4_t _a = vld1q_f32(a); float32x4_t _b = vld1q_f32(b); float32x4_t _c = vld1q_f32(c); asm volatile( "vmla.f32 %q0, %q2, %e3[0]" "vmla.f32 %q0, %q2, %e3[1]" "vmla.f32 %q0, %q2, %f3[0]" "vmla.f32 %q0, %q2, %f3[1]" : "=w"(_a) // %0 : "0"(_a), "w"(_b), // %2 "w"(_c) // %3 : ); ``` -------------------------------- ### Benchmark Command Example 1 Source: https://github.com/tencent/ncnn/blob/master/benchmark/README.md Example command to run benchmarks on MediaTek Dimensity 9300 with specific parameters. ```bash ../build-android/benchmark/benchncnn 8 4 1 -1 1 ``` -------------------------------- ### Benchmark Execution Example 1 Source: https://github.com/tencent/ncnn/blob/master/benchmark/README.md Example command to run benchmarks with specific settings. ```bash ./benchncnn 4 1 2 0 0 [0 Adreno (TM) 740] queueC=0[3] queueG=0[3] queueT=0[3] [0 Adreno (TM) 740] bugsbn1=1 bugbilz=0 bugcopc=0 bugihfa=0 [0 Adreno (TM) 740] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 Adreno (TM) 740] subgroup=64 basic=1 vote=1 ballot=1 shuffle=1 ```