### Example: Get Thresholds

Source: https://github.com/alibaba/mnn/blob/master/transformers/llm/collect/README.md

An example command to execute the get_thresholds.py script with specified model path, length, target sparsity, and output file. This calculates sparsity thresholds.

```bash
python get_thredsholds.py -m /path/to/MNN/transformers/llm/export/model/config.json -l 1024 -t 0.5 -o ./thresholds_0.5.json
```

--------------------------------

### Install Protobuf with nmake

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/protobuf/cmake/README.md

Install Protocol Buffers to the specified 'install' folder using nmake. This command should be run from the build directory.

```bash
C:\Path\to\protobuf\cmake\build\release>nmake install
```

```bash
C:\Path\to\protobuf\cmake\build\debug>nmake install
```

--------------------------------

### ADAM Optimizer Setup and Usage

Source: https://github.com/alibaba/mnn/wiki/train/optim

Illustrates the initialization of an ADAM optimizer, including setting model parameters, configuring both momentum values, weight decay, regularization, and learning rate. The example concludes with updating parameters using the step function.

```cpp
// 新建ADAM优化器
std::shared_ptr<SGD> solver(new ADAM);
// 设置模型中需要优化的参数
solver->append(model->parameters());
// 设置ADAM的两个momentum，设置weight decay
solver->setMomentum(0.9f);
solver->setMomentum2(0.99f);
solver->setWeightDecay(0.0005f);
// 设置正则化方法，默认L2
solver->setRegularizationMethod(RegularizationMethod::L2);
// 设置学习率
solver->setLearningRate(0.001);

// 根据loss计算梯度，并更新参数
solver->step(loss);
```

--------------------------------

### Example Usage

Source: https://github.com/alibaba/mnn/blob/master/docs/pymnn/OpInfo.md

Example demonstrating how to use OpInfo within the runSessionWithCallBackInfo callback functions.

```APIDOC
## Example

```python
import MNN

interpreter = MNN.Interpreter("mobilenet_v1.mnn")
session = interpreter.createSession()
# set input
# ...
def begin_callback(tensors, opinfo):
    print('layer name = ', opinfo.getName())
    print('layer op = ', opinfo.getType())
    # print('layer flops = ', opinfo.getFlops())
    for tensor in tensors:  
        print(tensor.getShape())
    return True
def end_callback(tensors, opinfo):
    print('layer name = ', opinfo.getName())
    print('layer op = ', opinfo.getType())
    # print('layer flops = ', opinfo.getFlops())
    for tensor in tensors:  
        print(tensor.getShape())
    return True
interpreter.runSessionWithCallBackInfo(session, begin_callback, end_callback)
```
```

--------------------------------

### Copy Example Configuration

Source: https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/scripts/README.md

Create a local configuration file by copying the example. You will need to edit this file with your specific credentials and paths.

```bash
cp scripts/release.config.example scripts/release.config
```

--------------------------------

### Build and install Protocol Buffers C++ runtime and protoc

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/protobuf/src/README.md

Compiles and installs the Protocol Buffer runtime library and the protocol compiler (protoc). Ensure you have the necessary build tools installed.

```bash
./configure
make
make check
sudo make install
sudo ldconfig
```

--------------------------------

### Compile and Install ZLib

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/protobuf/cmake/README.md

Compile and install the zlib library from source. This involves creating a build directory and navigating into it.

```bash
C:\Path\to\zlib>mkdir build & cd build
```

--------------------------------

### Example Usage

Source: https://github.com/alibaba/mnn/wiki/pymnn/_Module

Example demonstrating how to define and use a custom model by inheriting from nn.Module.

```APIDOC
## Example

```python
import MNN
import MNN.nn as nn
import MNN.expr as expr

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.conv(1, 20, [5, 5])
        self.conv2 = nn.conv(20, 50, [5, 5])
        
        self.fc1 = nn.linear(800, 500)
        self.fc2 = nn.linear(500, 10)
    def forward(self, x):
        x = expr.relu(self.conv1(x))
        x = expr.max_pool(x, [2, 2], [2, 2])
        x = expr.relu(self.conv2(x))
        x = expr.max_pool(x, [2, 2], [2, 2])
        # some ops like conv, pool , resize using special data format `NC4HW4`
        # so we need to change their data format when fed into reshape
        # we can get the data format of a variable by its `data_format` attribute
        x = expr.convert(x, expr.NCHW)
        x = expr.reshape(x, [0, -1])
        x = expr.relu(self.fc1(x))
        x = self.fc2(x)
        x = expr.softmax(x, 1)
        return x
model = Net()
```
```

--------------------------------

### Example: Get Max Values

Source: https://github.com/alibaba/mnn/blob/master/transformers/llm/collect/README.md

An example command to execute the get_max_values.py script with specified model path and output file. This collects maximum activation values.

```bash
python get_maxval.py --m /path/to/MNN/transformers/llm/export/model/config.json -o ./max_val_test.json
```

--------------------------------

### Benchmark Expression Models Examples

Source: https://github.com/alibaba/mnn/wiki/tools/benchmark

Examples of how to run the benchmark tool for models constructed from expressions. Each command specifies the model name, loop count, warm-up count, and number of threads.

```bash
./benchmarkExprModels.out MobileNetV1_100_1.0_224 10 0 4
```

```bash
./benchmarkExprModels.out MobileNetV2_100 10 0 4
```

```bash
./benchmarkExprModels.out ResNet_100_18 10 0 4
```

```bash
./benchmarkExprModels.out GoogLeNet_100 10 0 4
```

```bash
./benchmarkExprModels.out SqueezeNet_100 10 0 4
```

```bash
./benchmarkExprModels.out ShuffleNet_100_4 10 0 4
```

--------------------------------

### Install Sherpa-MNN Binaries

Source: https://github.com/alibaba/mnn/blob/master/apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/CMakeLists.txt

Installs the built Sherpa-MNN server and client executables to the 'bin' directory.

```cmake
install(
  TARGETS
    sherpa-mnn-online-websocket-server
    sherpa-mnn-online-websocket-client
    sherpa-mnn-offline-websocket-server
  DESTINATION
    bin
)
```

--------------------------------

### Install build tools on Ubuntu/Debian

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/protobuf/src/README.md

Installs the required tools for building Protocol Buffers from source on Ubuntu or Debian-based systems.

```bash
sudo apt-get install autoconf automake libtool curl make g++ unzip
```

--------------------------------

### GEMM Benchmark Examples

Source: https://github.com/alibaba/mnn/wiki/perf/gemm_speed_benchmark

Examples demonstrating how to run the GEMM benchmark with different configurations. These include CPU with default or FP16 precision, OpenCL with specific tuning, Vulkan, and specialized tests for Int4 quantization and floating-point operations.

```bash
# CPU 后端，默认精度
./run_test.out speed/GemmSpeedAll
```

```bash
# CPU 后端，FP16 精度，4 线程
./run_test.out speed/GemmSpeedAll 0 2 4
```

```bash
# OpenCL Buffer 模式，FP16 精度
./run_test.out speed/GemmSpeedAll 3 2 68
```

```bash
# Vulkan 后端，FP16 精度
./run_test.out speed/GemmSpeedAll 7 2
```

```bash
# 仅测试 Int4 量化
./run_test.out speed/GemmSpeedInt4 3 2 68
```

```bash
# 仅测试浮点
./run_test.out speed/GemmSpeedFloat 7 2
```

--------------------------------

### Install executables

Source: https://github.com/alibaba/mnn/blob/master/apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/CMakeLists.txt

Installs the specified executables to the 'bin' directory.

```cmake
install(
    TARGETS ${exes}
    DESTINATION
      bin
  )
```

--------------------------------

### MNN Model Inference Example

Source: https://github.com/alibaba/mnn/wiki/pymnn/Interpreter

Demonstrates a full inference pipeline using MNN, including interpreter creation, session setup, input preprocessing (image loading, resizing, normalization), running the session, and processing the output.

```python
import MNN
import MNN.cv as cv
import MNN.numpy as np

# 创建interpreter
interpreter = MNN.Interpreter("mobilenet_v1.mnn")
# 创建session
session = interpreter.createSession()
# 获取会话的输入输出张量
input_tensor = interpreter.getSessionInput(session)
output_tensor = interpreter.getSessionOutput(session)
# 将输入resize到[1, 3, 224, 224]
interpreter.resizeTensor(input_tensor, (1, 3, 224, 224))

# 读取图片，转换为size=(224, 224), dtype=float32，并赋值给input_tensor
image = cv.imread('cat.jpg')
image = cv.resize(image, (224, 224), mean=[103.94, 116.78, 123.68], norm=[0.017, 0.017, 0.017])
# HWC to NHWC
image = np.expand_dims(image, 0)
tmp_input = MNN.Tensor(image)
input_tensor.copyFrom(tmp_input)

# 执行会话推理
interpreter.runSession(session)

# 将输出结果拷贝到tmp_output中
tmp_output = MNN.Tensor((1, 1001), MNN.Halide_Type_Float, MNN.Tensor_DimensionType_Caffe)
output_tensor.copyToHostTensor(tmp_output)

# 打印出分类结果, 282为猫
print("output belong to class: {}".format(np.argmax(tmp_output.getData())))

```

--------------------------------

### Example dumpapp Commands

Source: https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/tools/dumpapp.md

Illustrative examples of common dumpapp commands for interacting with models and LLM status. These commands help in managing and querying the application's state.

```bash
./tools/dumpapp models list
```

```bash
./tools/dumpapp llm status
```

```bash
./tools/dumpapp openai start --model ModelScope/MNN/Qwen3___5-0___8B-MNN
```

--------------------------------

### Example Usage

Source: https://github.com/alibaba/mnn/blob/master/docs/pymnn/Var.md

A comprehensive example demonstrating the creation of Var variables, attribute access, shape manipulation, operator overloading, mathematical functions, element access, and type conversions.

```APIDOC
```python
import MNN.expr as expr
import MNN.numpy as np

# Create Var variables
x = expr.const([1., 2., 3., 4.], [2, 2], F.NCHW, F.float)  # array([[1., 2.], [3., 4.]], dtype=float32)
y = np.array([2, 0])  # array([2, 0], dtype=int32)
z = np.ones([4]) # array([1, 1, 1, 1], dtype=int32)

# Var attributes
print(x.shape) # [2, 2]
print(x.size) # 4
print(x.ndim) # 2
print(x.dtype) # dtype.float
print(x.data_format) # data_format.NCHW
print(x.op_type) # 'Const'

# Shape manipulation
x1 = x.reshape(1, 4) # array([[1., 2., 3., 4.]], dtype=float32)
x2 = x.transpose()  # array([[1., 3.], [2., 4.]], dtype=float32)
x3 = x.flatten() # array([1., 2., 3., 4.], dtype=float32)

# Operator overloading
x4 = x3 + z # array([3., 5., 7., 9.], dtype=float32)
x5 = x3 > z # array([0, 1, 1, 1], dtype=int32)

# Mathematical functions
print(x.max()) # 4.0
print(x.std()) # 1.1180340051651
print(x.sum()) # 10.0

# Element operations
print(x[0]) # array([1., 2.], dtype=float32)
print(x[0, 1]) # 2.0
print(x[:, 0]) # array([1., 3.], dtype=float32)
print(x.item(1)) # 2.0
print(len(x)) # 2
for item in x:
    print(item) # array([1., 2.], dtype=float32); array([3., 4.], dtype=float32)

# Type conversions
np_x = x.read() # array([[1., 2.], [3., 4.]], dtype=float32)
print(type(np_x)) # <class 'numpy.ndarray'>
tuple_x = x.read_as_tuple() # (1.0, 2.0, 3.0, 4.0)
print(type(tuple_x)) # <class 'tuple'>
scalar_x = x[0, 1] # 1.0
print(type(scalar_x)) # <class 'float'>
```
```

--------------------------------

### Start OpenAI Service (Bash)

Source: https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/tests/smoke/README.md

Starts the OpenAI service. This command now auto-enables the API service via a plugin if it's not already enabled, simplifying service bootstrapping.

```bash
dumpapp openai start
```

--------------------------------

### Install Dependencies for LLM Export

Source: https://github.com/alibaba/mnn/wiki/transformers/llm

Navigate to the export directory and install the required Python packages.

```default
cd ./transformers/llm/export
pip install -r requirements.txt
```

--------------------------------

### SD3.5 Diffusion Demo Usage Example

Source: https://github.com/alibaba/mnn/blob/master/transformers/diffusion/README.md

Example of running the SD3.5 diffusion demo. Specify the MNN model path, memory mode, backend type, iterations, seed, output image name, and prompt text.

```bash
./diffusion_sd35_demo /path/to/stable-diffusion-3.5-medium-MNN 0 3 20 1 demo.jpg "a cute cat"
```

--------------------------------

### Configure Protocol Buffers installation prefix

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/protobuf/src/README.md

Specifies a custom installation directory for Protocol Buffers, useful when /usr/local/lib is not in LD_LIBRARY_PATH. Use 'make clean' before rebuilding if the prefix changes.

```bash
./configure --prefix=/usr
```

--------------------------------

### Training Configuration Example

Source: https://github.com/alibaba/mnn/wiki/tools/test

An example JSON configuration file for the testTrain.out tool, specifying model, loss function, learning rate, input, decay, target, and data files.

```json
{
    "Model": "mnist.train.mnn",
    "Loss": "Loss",
    "LR": "LearningRate",
    "LearningRate":0.005,
    "Input":"Input3",
    "Decay":0.3,
    "Target":"Reshape38_Compare",
    "Data":[
        "data.mnn"
    ]
}

```

--------------------------------

### Install Dependencies

Source: https://github.com/alibaba/mnn/blob/master/transformers/llm/collect/README.md

Installs necessary Python packages and builds the pymnn library for MNN model analysis.

```bash
pip install datasets torch tqdm
cd /path/to/MNN/pymnn/pip_package
python build_deps.py llm
python setup.py install
```

--------------------------------

### Start Android RPC Tracker and Tuning

Source: https://github.com/alibaba/mnn/blob/master/benchmark/scripts/tvm/README.md

Specify RPC tracker host and port, and the Android NDK compiler. Then, start the RPC tracker and the tuning/inference script.

```bash
# Specify the RPC tracker
export TVM_TRACKER_HOST=0.0.0.0
export TVM_TRACKER_PORT=[PORT]
# Specify the standalone Android C++ compiler
export TVM_NDK_CC=/opt/android-toolchain-arm64/bin/aarch64-linux-android-g++

# start RPC tracker
python -m tvm.exec.rpc_tracker --port 9090 --host 0.0.0.0
# start tuning and inference
python android_tuning.py <test_model_file>
```

--------------------------------

### Install Android SDK and NDK (Linux)

Source: https://github.com/alibaba/mnn/blob/master/benchmark/scripts/READMD.md

Installs the Android SDK and NDK on Linux systems. Ensure to switch mirrors if downloads are slow. Sets ANDROID_SDK and ANDROID_NDK environment variables.

```bash
apt install -y android-sdk # switch mirror when slow, link: https://blog.csdn.net/qq_31456593/article/details/89638163
ANDROID_SDK=/usr/lib/android-sdk
curl -LO https://dl.google.com/android/repository/android-ndk-r21-linux-x86_64.zip
unzip android-ndk-r21-linux-x86_64.zip -d $ANDROID_SDK 1>/dev/null
echo -e "\nexport ANDROID_SDK=$ANDROID_SDK\nexport ANDROID_NDK=$ANDROID_SDK/android-ndk-r21" >> ~/.bashrc
```

--------------------------------

### Complete MNN Inference Example

Source: https://github.com/alibaba/mnn/wiki/start/python

A full example demonstrating image preprocessing, MNN inference, and post-processing. It takes the model file and image path as command-line arguments.

```python
from __future__ import print_function
import MNN.numpy as np
import MNN
import MNN.cv as cv2
import sys

def inference(net, imgpath):
    """ inference mobilenet_v1 using a specific picture """
    # 预处理
    image = cv2.imread(imgpath)
    #cv2 read as bgr format
    image = image[..., ::-1]
    #change to rgb format
    image = cv2.resize(image, (224, 224))
    #resize to mobile_net tensor size
    image = image - (103.94, 116.78, 123.68)
    image = image * (0.017, 0.017, 0.017)
    #change numpy data type as np.float32 to match tensor's format
    image = image.astype(np.float32)
    #Make var to save numpy; [h, w, c] -> [n, h, w, c]
    input_var = np.expand_dims(image, [0])
    #cv2 read shape is NHWC, Module's need is NC4HW4, convert it
    input_var = MNN.expr.convert(input_var, MNN.expr.NC4HW4)
    #inference
    output_var = net.forward([input_var])
    # 后处理及使用
    predict = np.argmax(output_var[0])
    print("expect 983")
    print("output belong to class: {}".format(predict))

# 模型加载
net = MNN.nn.load_module_from_file(sys.argv[1], ["image"], ["prob"])
inference(net, sys.argv[2])
 
```

--------------------------------

### Operator Implementation Step Selection Guide

Source: https://github.com/alibaba/mnn/blob/master/skills/add-new-op/SKILL.md

A guide to determine which steps are necessary for implementing a new operator based on its characteristics, such as shape consistency and the need for backend-specific implementation.

```markdown
| 情况 | 需要执行的步骤 |
|--------------|--------------|
| 输出形状与输入完全一致，可几何拆解 | 1 → 3(几何) → 4 |
| 输出形状与输入不同，可几何拆解 | 1 → 2 → 3(几何) → 4 |
| 输出形状与输入不同，需要后端实现 | 1 → 2 → 3(CPU) → 4 → 5 |
| 输出形状与输入完全一致，需要后端实现 | 1 → 3(CPU) → 4 → 5 |
```

--------------------------------

### Start iOS RPC Tracker, Proxy, and Tuning

Source: https://github.com/alibaba/mnn/blob/master/benchmark/scripts/tvm/README.md

Set environment variables for iOS RPC, including proxy host, root path, codesign identity, and destination. Then, start the RPC tracker, RPC proxy, and the tuning script.

```bash
export TVM_IOS_RPC_PROXY_HOST=0.0.0.0
export TVM_IOS_RPC_ROOT=${TVM_HOME}/your/testscript/path
export TVM_IOS_CODESIGN='Apple Development: xxxxx(xxxxx)'
export TVM_IOS_RPC_DESTINATION='platform=iOS,id=xxxxx-xxxxxxx'

# start RPC tracker
python -m tvm.exec.rpc_tracker --host=192.168.101.49  --port=9190 --no-fork
# start RPC proxy
python -m tvm.exec.rpc_proxy --no-fork --host 0.0.0.0 --port 9090 --tracker 0.0.0.0:9190
# tuning and inference
# change the `network` in ios_tuning.py to specify test model
python ios_tuning.py
```

--------------------------------

### SGD Optimizer Setup and Usage

Source: https://github.com/alibaba/mnn/wiki/train/optim

Demonstrates how to initialize an SGD optimizer, append model parameters, set momentum, weight decay, regularization method, and learning rate. It also shows the step to update parameters based on loss.

```cpp
// 新建SGD优化器
std::shared_ptr<SGD> solver(new SGD);
// 设置模型中需要优化的参数
solver->append(model->parameters());
// 设置momentum和weight decay
solver->setMomentum(0.9f);
solver->setWeightDecay(0.0005f);
// 设置正则化方法，默认L2
solver->setRegularizationMethod(RegularizationMethod::L2);
// 设置学习率
solver->setLearningRate(0.001);

// 根据loss计算梯度，并更新参数
solver->step(loss);
```

--------------------------------

### Buffer Head Offset

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/flatbuffers/docs/source/GoApi_generated.txt

Gets the starting offset of the useful data in the underlying byte buffer.

```APIDOC
## Head

### Description
Gives the start of useful data in the underlying byte buffer. Note: unlike other functions, this value is interpreted as from the left.

### Signature
func (b *Builder) Head() UOffsetT
```

--------------------------------

### Setup Release Environment

Source: https://github.com/alibaba/mnn/blob/master/apps/Android/MnnLlmChat/scripts/README.md

Execute this script to prepare your environment for the release process. It helps configure necessary settings.

```bash
./scripts/setup_release.sh
```

--------------------------------

### Full MNN Inference Example

Source: https://github.com/alibaba/mnn/blob/master/docs/start/python.md

A complete example demonstrating MNN inference, including image pre-processing (reading, resizing, normalization), model loading, inference, and post-processing to get the predicted class.

```python
from __future__ import print_function
import MNN.numpy as np
import MNN
import MNN.cv as cv2
import sys

def inference(net, imgpath):
    """ inference mobilenet_v1 using a specific picture """
    # Pre-processing
    image = cv2.imread(imgpath)
    #cv2 read as bgr format
    image = image[..., ::-1]
    #change to rgb format
    image = cv2.resize(image, (224, 224))
    #resize to mobile_net tensor size
    image = image - (103.94, 116.78, 123.68)
    image = image * (0.017, 0.017, 0.017)
    #change numpy data type as np.float32 to match tensor's format
    image = image.astype(np.float32)
    #Make var to save numpy; [h, w, c] -> [n, h, w, c]
    input_var = np.expand_dims(image, [0])
    #cv2 read shape is NHWC, Module's need is NC4HW4, convert it
    input_var = MNN.expr.convert(input_var, MNN.expr.NC4HW4)
    #inference
    output_var = net.forward([input_var])
    # Post-processing and usage
    predict = np.argmax(output_var[0])
    print("expect 983")
    print("output belong to class: {}".format(predict))

# Model loading
net = MNN.nn.load_module_from_file(sys.argv[1], ["image"], ["prob"])
inference(net, sys.argv[2])
```

--------------------------------

### Create and Configure RuntimeManager Example

Source: https://github.com/alibaba/mnn/wiki/pymnn/RuntimeManager

Demonstrates how to create a RuntimeManager with specific configurations for GPU backend, precision, and number of threads. It also shows setting cache, mode, and hints before loading a model and performing inference.

```python
import MNN.nn as nn
import MNN.cv as cv
import MNN.numpy as np
import MNN.expr as expr

config = {}
config['precision'] = 'low'
# 使用GPU后端
config['backend'] = 3
config['numThread'] = 4

# 创建RuntimeManager
rt = nn.create_runtime_manager((config,))
rt.set_cache(".cachefile")
# mode = auto_backend
rt.set_mode(9)
# tune_num = 20 
rt.set_hint(0, 20)
# 加载模型并使用RuntimeManager
net = nn.load_module_from_file('mobilenet_v1.mnn', ['data'], ['prob'], runtime_manager=rt)
# cv读取bgr图片
image = cv.imread('cat.jpg')
# 转换为float32, 形状为[224,224,3]        
image = cv.resize(image, (224, 224), mean=[103.94, 116.78, 123.68], norm=[0.017, 0.017, 0.017])
# 增加batch HWC to NHWC
input_var = np.expand_dims(image, 0)
# NHWC to NC4HW4
input_var = expr.convert(input_var, expr.NC4HW4)
# 执行推理
output_var = net.forward(input_var)
# NC4HW4 to NHWC 
output_var = expr.convert(output_var, expr.NHWC)
# 打印出分类结果, 282为猫
print("output belong to class: {}".format(np.argmax(output_var)))
```

--------------------------------

### Implement Custom MnistDataset in C++

Source: https://github.com/alibaba/mnn/blob/master/docs/train/data.md

Implement the `get` function to return a single data example and its label, and the `size` function to return the total number of examples in the dataset. Ensure correct data type and format for MNN.

```cpp
//  MnistDataset.cpp

// 返回MNIST数据集中一张图片数据，及其对应的label
Example MnistDataset::get(size_t index) {
    auto data  = _Input({1, kImageRows, kImageColumns}, NCHW, halide_type_of<uint8_t>());
    auto label = _Input({}, NCHW, halide_type_of<uint8_t>());

    auto dataPtr = mImagePtr + index * kImageRows * kImageColumns;
    ::memcpy(data->writeMap<uint8_t>(), dataPtr, kImageRows * kImageColumns);

    auto labelPtr = mLabelsPtr + index;
    ::memcpy(label->writeMap<uint8_t>(), labelPtr, 1);

    auto returnIndex = _Const(index);
    // return the index for test
    return {{data, returnIndex}, {label}};
}
// 返回数据集大小，对于MNIST训练集是60000，测试集是10000
size_t MnistDataset::size() {
    return mImages->getInfo()->dim[0];
}
```

--------------------------------

### Context Configuration Example (context.json)

Source: https://github.com/alibaba/mnn/wiki/transformers/llm

This JSON file defines tools available to the LLM and whether thinking steps are enabled. It includes tool definitions with names and descriptions.

```json
{
  "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_current_time",
                "description": "获取当前时间"
            }
        }
    ],
    "enable_thinking": false
}
```

--------------------------------

### Dataset Abstract Base Class Methods

Source: https://github.com/alibaba/mnn/blob/master/docs/train/data.md

Abstract methods that must be implemented by custom datasets. These define how to get the size of the dataset and retrieve individual examples or batches.

```cpp
// 返回数据集的大小，例如1000张图片的数据集，其大小为1000
virtual size_t size() = 0;
// 返回数据集中指定index的数据，如给定123，返回第123张图片数据
virtual Example get(size_t index) = 0;
// 返回数据集中指定index的一批数据，为一个batch
std::vector<Example> getBatch(std::vector<size_t> indices);
```

--------------------------------

### Running LLM Benchmark with Multiple Configurations

Source: https://github.com/alibaba/mnn/wiki/transformers/llm

Example command to run the llm_bench tool with multiple models, backends, precisions, thread counts, prompt/generation lengths, and output file settings.

```bash
./llm_bench -m ./Qwen2.5-1.5B-Instruct/config.json,./Qwen2.5-0.5B-Instruct/config.json -a cpu,opencl,metal -c 1,2 -t 8,12 -p 16,32 -n 10,20 -pg 8,16 -mmp 0 -rep 4 -kv true -fp ./test_result
```

--------------------------------

### ADB Commands for MNN TTS Demo

Source: https://github.com/alibaba/mnn/blob/master/apps/frameworks/mnn_tts/demo/android/QUICKREF.md

Utilize ADB commands for installing, uninstalling, starting, stopping, viewing logs, and clearing data for the MNN TTS demo application.

```bash
# 安装
adb install -r build/outputs/apk/debug/MNNTTSDemo-arm64-v8a-debug.apk

# 卸载
adb uninstall com.alibaba.mnn.tts.demo

# 启动
adb shell am start -n com.alibaba.mnn.tts.demo/.MainActivity

# 停止
adb shell am force-stop com.alibaba.mnn.tts.demo

# 查看日志
adb logcat -s MNN_TTS:* AndroidRuntime:E

# 清除数据
adb shell pm clear com.alibaba.mnn.tts.demo
```

--------------------------------

### MnistDataset Implementation

Source: https://github.com/alibaba/mnn/wiki/pymnn/Dataset

Example implementation of a custom dataset for MNIST. Requires the 'mnist' library to be installed. The `__getitem__` method returns input data as a constant tensor and labels as a constant uint8 tensor.

```python
try:
    import mnist
except ImportError:
    print("please 'pip install mnist' before run this demo")
import MNN
import MNN.expr as expr
class MnistDataset(MNN.data.Dataset):
    def __init__(self, training_dataset=True):
        super(MnistDataset, self).__init__()
        self.is_training_dataset = training_dataset
        if self.is_training_dataset:
            self.data = mnist.train_images() / 255.0
            self.labels = mnist.train_labels()
        else:
            self.data = mnist.test_images() / 255.0
            self.labels = mnist.test_labels()
    def __getitem__(self, index):
        dv = expr.const(self.data[index].flatten().tolist(), [1, 28, 28], expr.NCHW)
        dl = expr.const([self.labels[index]], [], expr.NCHW, expr.uint8)
        # first for inputs, and may have many inputs, so it's a list
        # second for targets, also, there may be more than one targets
        return [dv], [dl]
    def __len__(self):
        # size of the dataset
        if self.is_training_dataset:
            return 60000
        else:
            return 10000
```

--------------------------------

### Top Down Example: Creating a Monster Buffer in C

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/flatbuffers/docs/source/CUsage.md

Illustrates a top-down approach to building a FlatBuffers object in C, starting and ending nested objects. This method allows for more control over table elements within vectors.

```c
  uint8_t treasure[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
  size_t treasure_count = c_vec_len(treasure);
  ns(Weapon_ref_t) axe;

  // NOTE: if we use end_as_root, we MUST also start as root.
  ns(Monster_start_as_root(B));
  ns(Monster_pos_create(B, 1.0f, 2.0f, 3.0f));
  ns(Monster_hp_add(B, 300));
  ns(Monster_mana_add(B, 150));
  // We use create_str instead of add because we have no existing string reference.
  ns(Monster_name_create_str(B, "Orc"));
  // Again we use create because we no existing vector object, only a C-array.
  ns(Monster_inventory_create(B, treasure, treasure_count));
  ns(Monster_color_add(B, ns(Color_Red)));
  if (1) {
      ns(Monster_weapons_start(B));
      ns(Monster_weapons_push_create(B, flatbuffers_string_create_str(B, "Sword"), 3));
      // We reuse the axe object later. Note that we dereference a pointer
      // because push always returns a short-term pointer to the stored element.
      // We could also have created the axe object first and simply pushed it.
      axe = *ns(Monster_weapons_push_create(B, flatbuffers_string_create_str(B, "Axe"), 5));
      ns(Monster_weapons_end(B));
  } else {
      // We can have more control with the table elements added to a vector:
      //
      ns(Monster_weapons_start(B));
      ns(Monster_weapons_push_start(B));
      ns(Weapon_name_create_str(B, "Sword"));
      ns(Weapon_damage_add(B, 3));
      ns(Monster_weapons_push_end(B));
      ns(Monster_weapons_push_start(B));
      ns(Monster_weapons_push_start(B));
      ns(Weapon_name_create_str(B, "Axe"));
      ns(Weapon_damage_add(B, 5));
      axe = *ns(Monster_weapons_push_end(B));
      ns(Monster_weapons_end(B));
  }
  // Unions can get their type by using a type-specific add/create/start method.
  ns(Monster_equipped_Weapon_add(B, axe));

  ns(Monster_end_as_root(B));
```

--------------------------------

### MNN Session API Inference Example

Source: https://github.com/alibaba/mnn/wiki/inference/python

Perform inference using the MNN Session API (deprecated). This involves creating an interpreter, a session, getting input/output tensors, processing images, copying data, and running the session.

```python
import MNN
import MNN.cv as cv
import MNN.numpy as np
import MNN.expr as expr

# 创建interpreter
interpreter = MNN.Interpreter("mobilenet_v1.mnn")
# 创建session
config = {}
config['precision'] = 'low'
config['backend'] = 'CPU'
config['thread'] = 4
session = interpreter.createSession(config)
# 获取会话的输入输出
input_tensor = interpreter.getSessionInput(session)
output_tensor = interpreter.getSessionOutput(session)

# 读取图片
image = cv.imread('cat.jpg')

dst_height = dst_width = 224
# 使用ImageProcess处理第一张图片，将图片转换为转换为size=(224, 224), dtype=float32，并赋值给input_data1
image_processer = MNN.CVImageProcess({'sourceFormat': MNN.CV_ImageFormat_BGR,
                                      'destFormat': MNN.CV_ImageFormat_BGR,
                                      'mean': (103.94, 116.78, 123.68, 0.0),
                                      'filterType': MNN.CV_Filter_BILINEAL,
                                      'normal': (0.017, 0.017, 0.017, 0.0)})
image_data = image.ptr
src_height, src_width, channel = image.shape
input_data1 = MNN.Tensor((1, dst_height, dst_width, channel), MNN.Halide_Type_Float, MNN.Tensor_DimensionType_Tensorflow)
#设置图像变换矩阵
matrix = MNN.CVMatrix()
x_scale = src_width / dst_width
y_scale = src_height / dst_height
matrix.setScale(x_scale, y_scale)
image_processer.setMatrix(matrix)
image_processer.convert(image_data, src_width, src_height, 0, input_data1)

# 使用cv模块处理第二张图片，将图片转换为转换为size=(224, 224), dtype=float32，并赋值给input_data2
image = cv.imread('TestMe.jpg')
image = cv.resize(image, (224, 224), mean=[103.94, 116.78, 123.68], norm=[0.017, 0.017, 0.017])
input_data2 = np.expand_dims(image, 0) # [224, 224, 3] -> [1, 224, 224, 3]
```

--------------------------------

### Build and Run Metal LLM Demo

Source: https://github.com/alibaba/mnn/blob/master/skills/metal-optimize/SKILL.md

Build the MNN library with LLM demo support, switch the model configuration to use the Metal backend, and then run the demo. Ensure DYLD_LIBRARY_PATH is set correctly.

```bash
# build
cd build && make -j8 llm_demo MNN

# 切后端
sed 's/"backend_type": "cpu"/"backend_type": "metal"/' transformers/llm/export/<model>/config.json > <model>/config_mtl.json

# 跑
DYLD_LIBRARY_PATH=build:build/express build/llm_demo transformers/llm/export/<model>/config_mtl.json /tmp/prompt.txt
```

--------------------------------

### Vulkan Benchmark Example

Source: https://github.com/alibaba/mnn/blob/master/docs/perf/gemm_speed_benchmark.md

Run the GemmSpeedAll benchmark on the Vulkan backend with FP16 precision.

```bash
# Vulkan 后端，FP16 精度
./run_test.out speed/GemmSpeedAll 7 2
```

--------------------------------

### C++ Inference with Module API

Source: https://github.com/alibaba/mnn/wiki/start/quickstart_cpp

Example C++ code for performing image classification inference using MNN's Module API. It includes runtime setup, model loading, input preparation, forward pass, and output processing.

```cpp
#include <stdio.h>
#include <MNN/expr/Module.hpp>
#include <MNN/expr/Executor.hpp>
#include <MNN/expr/ExprCreator.hpp>
#include <MNN/ImageProcess.hpp>

using namespace MNN;
using namespace MNN::Express;

int main(int argc, const char* argv[]) {
    if (argc < 3) {
        printf("Usage: ./classify model.mnn input.jpg\n");
        return 1;
    }

    // 1. 创建 RuntimeManager 配置后端
    ScheduleConfig sConfig;
    sConfig.type = MNN_FORWARD_CPU;
    sConfig.numThread = 4;
    auto rtmgr = std::shared_ptr<Executor::RuntimeManager>(
        Executor::RuntimeManager::createRuntimeManager(sConfig));

    // 2. 加载模型（空 vector 表示自动检测输入输出名）
    std::shared_ptr<Module> net(
        Module::load({}, {}, argv[1], rtmgr));
    if (!net) {
        printf("Failed to load model\n");
        return 1;
    }

    // 3. 创建输入 (NC4HW4 是 MNN 内部优化格式)
    int width = 224, height = 224;
    auto input = _Input({1, 3, height, width}, NC4HW4);

    // 4. 图像预处理（使用 MNN ImageProcess）
    // 实际项目中用 stb_image/opencv 读图后用 ImageProcess 转换
    // 此处简化为填充测试数据
    auto inputPtr = input->writeMap<float>();
    // ... 填充预处理后的图像数据到 inputPtr ...
    input->unMap();

    // 5. 推理
    auto outputs = net->onForward({input});

    // 6. 读取输出
    auto output = _Convert(outputs[0], NHWC);
    output = _Reshape(output, {0, -1});
    auto outputPtr = output->readMap<float>();
    // 找到概率最大的类别
    int classId = 0;
    float maxVal = outputPtr[0];
    int outputSize = output->getInfo()->size;
    for (int i = 1; i < outputSize; i++) {
        if (outputPtr[i] > maxVal) {
            maxVal = outputPtr[i];
            classId = i;
        }
    }
    printf("预测类别编号: %d, 置信度: %f\n", classId, maxVal);
    return 0;
}
```

--------------------------------

### Install MNN with pip

Source: https://github.com/alibaba/mnn/wiki/start/quickstart_python

Install the MNN Python package using pip. If installation fails, consider compiling from source.

```bash
pip install MNN
```

--------------------------------

### Run Diffusion Demo with Stable Diffusion v1.5

Source: https://github.com/alibaba/mnn/wiki/transformers/diffusion

Example command to run the diffusion demo using a stable-diffusion-v1-5 model. Sets memory mode to 'sufficient', backend to '3' (likely OpenCL/Metal), 20 iterations, random seed, and a prompt for a cute cat.

```default
./diffusion_demo mnn_sd1.5_path 0 1 3 20 -1 demo.jpg "a cute cat"
```

--------------------------------

### Install Unix tools on macOS

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/protobuf/src/README.md

Installs essential Unix command-line tools on macOS, which are required for building Protocol Buffers from source. This command requires Xcode to be installed.

```bash
sudo xcode-select --install
```

--------------------------------

### View Benchmark Help for Expression Models

Source: https://github.com/alibaba/mnn/wiki/tools/benchmark

Display help information for benchmarking models built from expressions. This command shows available options and usage instructions.

```bash
./benchmarkExprModels.out help
```

--------------------------------

### Install CMake on macOS

Source: https://github.com/alibaba/mnn/blob/master/apps/mnncli/test/unit_tests/README.md

Command to install CMake on macOS using Homebrew if it's not found.

```bash
brew install cmake
```

--------------------------------

### Include Examples Module

Source: https://github.com/alibaba/mnn/blob/master/3rd_party/protobuf/cmake/CMakeLists.txt

Conditionally includes the 'examples.cmake' module if the 'protobuf_BUILD_EXAMPLES' flag is enabled.

```cmake
if (protobuf_BUILD_EXAMPLES)
  include(examples.cmake)
endif (protobuf_BUILD_EXAMPLES)
```

--------------------------------

### Run Winograd Example

Source: https://github.com/alibaba/mnn/wiki/tools/test

Executes the winogradExample tool with specified dimensions. This is likely used for testing convolution algorithms.

```bash
$ ./winogradExample.out 3 3                        
A=
Matrix:
1.0000000	0.0000000	0.0000000	
1.0000000	0.5000000	0.2500000	
1.0000000	-0.5000000	0.2500000	
1.0000000	1.0000000	1.0000000	
0.0000000	0.0000000	1.0000000	
B=
Matrix:
1.0000000	0.0000000	0.0000000	0.0000000	0.0000000	
-1.0000000	2.0000000	-0.6666667	-0.3333333	0.2500000	
-4.0000000	2.0000000	2.0000000	0.0000000	-0.2500000	
4.0000000	-4.0000000	-1.3333334	1.3333334	-1.0000000	
0.0000000	0.0000000	0.0000000	0.0000000	1.0000000	
G=
Matrix:
1.0000000	0.0000000	0.0000000	
1.0000000	0.5000000	0.2500000	
1.0000000	-0.5000000	0.2500000	
1.0000000	1.0000000	1.0000000	
0.0000000	0.0000000	1.0000000
```

--------------------------------

### Install Sherpa-MNN Executables

Source: https://github.com/alibaba/mnn/blob/master/apps/frameworks/sherpa-mnn/sherpa-mnn/csrc/CMakeLists.txt

Installs the defined Sherpa-MNN executables to the 'bin' directory when binary support is enabled.

```cmake
if(SHERPA_MNN_ENABLE_BINARY)
  install(
    TARGETS
      ${main_exes}
    DESTINATION
      bin
  )
endif()
```

--------------------------------

### MNN CVMatrix Initialization and Transformations Example

Source: https://github.com/alibaba/mnn/wiki/pymnn/CVMatrix

Demonstrates the creation of a CVMatrix and applying various transformations like scale, pre-scale, rotate, translate, setPolyToPoly, and invert.

```python
import MNN
import MNN.cv as cv
import MNN.expr as expr
# CVMatrix创建
matrix = MNN.CVMatrix() # [[1., 0., 0.], [0., 1., 0.], [0., 0., 0.]]

# CVMatrix设置
matrix.setScale(2, 3, 4, 5) # [[2., 0., -4.], [0., 3., -10.], [-10., 0., 0.]]
matrix.preScale(-1, -3) # [[-2., 0., -4.], [0., -9., -10.], [-10., 0., 0.]]
matrix.setRotate(5) # [[0.996195, -0.087156, 0.0], [0.087156, 0.996195, 0.0], [0.0, 0.0, 1.0]]
matrix.setTranslate(5, 6) #[[1.0, 0.0, 5.0], [0.0, 1.0, 6.0], [0.0, 0.0, 1.0]]
matrix.setPolyToPoly([0, 0, 1, 1], [0, 1, 1, 0]) # [[0.0, 1.0, 0.0], [-1.0, 0.0, 1.0], [0.0, 0.0, 1.0]]
matrix.invert() # [[0.0, -1.0, 1.0], [1.0, 0.0, -0.0], [0.0, 0.0, 1.0]]
```

--------------------------------

### Get Input Tensor

Source: https://github.com/alibaba/mnn/blob/master/docs/inference/session.md

Retrieve input tensors for a given session by name, or get all input tensors.

```APIDOC
## Get Input Tensor

### Description
Two methods are provided on the `Interpreter` to retrieve input `Tensor` objects: `getSessionInput` for a single input tensor and `getSessionInputAll` for a map of all input tensors.

### Method Signatures
```cpp
/**
 * @brief get input tensor for given name.
 * @param session   given session.
 * @param name      given name. if NULL, return first input.
 * @return tensor if found, NULL otherwise.
 */
Tensor* Interpreter::getSessionInput(const Session* session, const char* name);

/**
 * @brief get all input tensors.
 * @param session   given session.
 * @return all output tensors mapped with name.
 */
const std::map<std::string, Tensor*>& Interpreter::getSessionInputAll(const Session* session) const;
```

### Usage
If there is only one input tensor, you can pass `NULL` to `getSessionInput` to retrieve it.
```