### Start TVM RPC Tracker

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

Command to launch the TVM RPC tracker. This tool manages RPC servers, providing features like queueing and multiple server management. It requires specifying the host IP and a port range.

```shell
python3 -m tvm.exec.rpc_tracker --host RPC_TRACKER_IP --port 9190 --port-end 9191
```

--------------------------------

### Pack and Deploy TVM Runtime

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

This section details how to package the Python version of the TVM RPC server and its runtime for deployment to a device. It involves cleaning the Python directory, copying the compiled runtime library, creating a tarball, and then on the device machine, extracting it and setting the PYTHONPATH environment variable.

```bash
# On the build machine:
$ git clean -dxf python
$ cp cross_build/libtvm_runtime.so python/tvm/
$ tar -czf tvm_runtime.tar.gz python

# On the device machine:
$ tar -xzf tvm_runtime.tar.gz
$ export PYTHONPATH=`pwd`/python:${PYTHONPATH}

```

--------------------------------

### Start TVM RPC Proxy

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

Command to launch the TVM RPC proxy. The proxy is used when direct access to the RPC server is not possible. It connects to the RPC tracker to manage server requests and requires host IP, port range, and tracker address.

```shell
python3 -m tvm.exec.rpc_proxy --host RPC_PROXY_IP --port 9090 --port-end 9091 --tracker RPC_TRACKER_IP:RPC_TRACKER_PORT
```

--------------------------------

### TVM Profiler C++ Example Usage

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1profiling_1_1Profiler

An example demonstrating the usage of the TVM `Profiler` class. It shows how to initialize the profiler with CPU and GPU devices, start and stop profiling sessions, record individual function calls with their respective devices, and finally print the generated profiling report. This snippet highlights the practical application of the profiler API.

```cpp
Device cpu, gpu;
Profiler prof({cpu, gpu});
my_gpu_kernel(); // do a warmup iteration
prof.Start();
prof.StartCall("my_gpu_kernel", gpu);
my_gpu_kernel();
prof.StopCall();
prof.StartCall("my_cpu_function", cpu);
my_cpu_function();
prof.StopCall();
prof.Stop();
std::cout << prof.Report() << std::endl; // print profiling report

```

--------------------------------

### Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/globals_func

Illustrative examples demonstrating the usage of TVM components.

```APIDOC
## Examples

### Description
Provides practical examples showcasing how to use various features and functionalities of TVM.

### Endpoint
`/examples`
```

--------------------------------

### Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/structtvm_1_1relax_1_1StridedSliceAttrs-members

Links to examples demonstrating the usage of TVM components.

```APIDOC
## Examples

### Description
Find practical examples showcasing how to use various features and components of TVM.

### Endpoint
`/websites/tvm_apache/examples`
```

--------------------------------

### Example List

Source: https://tvm.apache.org/docs/reference/api/doxygen/structtvm_1_1relax_1_1ArgmaxArgminAttrs-members

Provides a list of available examples in the TVM project.

```APIDOC
## GET /websites/tvm_apache/examples

### Description
Retrieves a list of example use cases and tutorials for the TVM project.

### Method
GET

### Endpoint
/websites/tvm_apache/examples

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **examples** (array) - A list of example titles or paths.

#### Response Example
```json
{
  "examples": [
    "basic_inference.py",
    "model_optimization.ipynb"
  ]
}
```
```

--------------------------------

### Launch TVM RPC Server

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

Commands to launch the TVM RPC server on the device machine. Two options are provided: one using an RPC proxy and another directly connecting to an RPC tracker. Ensure that RPC_PROXY_IP, RPC_PROXY_PORT, RPC_TRACKER_IP, RPC_TRACKER_PORT, and RPC_KEY are configured according to your specific environment.

```bash
# Use this if you use RPC proxy.
$ python3 -m tvm.exec.rpc_server --host RPC_PROXY_IP --port RPC_PROXY_PORT --through-proxy --key RPC_KEY
# Use this if you needn't use RPC proxy.
$ python3 -m tvm.exec.rpc_server --tracker RPC_TRACKER_IP:RPC_TRACKER_PORT --key RPC_KEY

```

--------------------------------

### TVM Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1script_1_1ir__builder_1_1tir_1_1ThenFrame

Provides access to TVM examples.

```APIDOC
## TVM Examples API

### Description
This section provides access to various examples demonstrating TVM functionality.

### Endpoints
- **/api/examples**: List all available examples.
- **/api/examples/{example_name}**: Get details or code for a specific example.

### Example Usage
```bash
GET /api/examples/tir_basic
```
```

--------------------------------

### Install TVM FFI Package

Source: https://tvm.apache.org/docs/install/from_source

This snippet installs the `tvm-ffi` package, which is required for TVM's Python bindings. It navigates to the package directory and uses pip to perform a local installation.

```bash
cd 3rdparty/tvm-ffi; pip install .; cd ..

```

--------------------------------

### Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1relax_1_1DataTypePatternNode-members

Access to example usage and demonstrations within the TVM project.

```APIDOC
## Examples

### Description
Provides access to example code and usage scenarios for the TVM project.

### Endpoint
- `/websites/tvm_apache/tvm/examples`
```

--------------------------------

### TVM Examples Documentation

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1relax_1_1ShapeType-members

Documentation and links to examples provided within the TVM project.

```APIDOC
## TVM Examples

### Description
This section provides access to various examples demonstrating the usage and capabilities of the TVM project.

### Endpoint
`/websites/tvm_apache/examples`

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **example_list** (array) - A list of available examples or links to them.

#### Response Example
```json
{
  "example_list": [
    "/examples/resnet.py",
    "/examples/image_classification.cc"
  ]
}
```
```

--------------------------------

### Install TVM via Environment Variable

Source: https://tvm.apache.org/docs/install/from_source

This method installs TVM by setting environment variables. It requires specifying the `TVM_HOME` directory and then adding the relevant TVM Python paths to the `PYTHONPATH`.

```bash
export TVM_HOME=/path-to-tvm
export PYTHONPATH=$TVM_HOME/python:$TVM_HOME/ffi/python:$PYTHONPATH

```

--------------------------------

### Query TVM RPC Tracker Status

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

This command is used to query the status of all RPC servers connected to the RPC tracker. It helps in verifying that the RPC servers are running and accessible. The output shows the tracker address, server list with their keys, and queue status.

```bash
$ python3 -m tvm.exec.query_rpc_tracker --host RPC_TRACKER_IP --port RPC_TRACKER_PORT

```

--------------------------------

### TVM Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1te_1_1Operation-members

Links and information related to examples demonstrating TVM's capabilities.

```APIDOC
## TVM Examples

### Description
This section provides access to example code and use cases for the TVM library.
```

--------------------------------

### TVM Examples Documentation

Source: https://tvm.apache.org/docs/reference/api/doxygen/structtvm_1_1relax_1_1QuantizeAttrs-members

Access to examples demonstrating TVM functionality.

```APIDOC
## TVM Examples

### Description
Collection of examples showcasing the usage and capabilities of TVM.

### Endpoint
- `/websites/tvm_apache/tvm/Examples`
```

--------------------------------

### Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1relax_1_1CallNode-members

Provides access to example code and usage scenarios within the TVM project.

```APIDOC
## Examples

### Description
This section provides access to example code demonstrating the usage of TVM features.

### Endpoint
`/websites/tvm_apache/examples`

### Response
#### Success Response (200)
- **examples** (array) - A list of available examples.
  - **name** (string) - The name or title of the example.
  - **path** (string) - The path to the example code or documentation.

### Response Example
```json
{
  "examples": [
    {
      "name": "Basic Tensor Computation",
      "path": "/websites/tvm_apache/examples/basic_tensor"
    }
  ]
}
```
```

--------------------------------

### Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/structtvm_1_1relax_1_1EinsumAttrs-members

This section provides access to example code demonstrating the usage of TVM features.

```APIDOC
## Examples

- (List of available examples)
```

--------------------------------

### Install Google Test from Source (C++)

Source: https://tvm.apache.org/docs/install/from_source

Installs Google Test (GTest), a framework for C++ unit testing, from its source repository. This involves cloning the repository, configuring the build with CMake, compiling the libraries, and installing them system-wide. Ensure you have CMake and a C++ compiler installed.

```bash
git clone https://github.com/google/googletest
cd googletest
mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=ON ..
make
sudo make install
```

--------------------------------

### Build TVM Runtime and Configure Build Options

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

Commands to cross-compile the TVM runtime after setting up the CMake toolchain file. It involves creating a build directory, copying the config.cmake, modifying build options using sed (e.g., disabling LLVM and libbacktrace), and then invoking CMake build for the runtime. Adjust paths and options as needed.

```bash
$ mkdir cross_build
$ cd cross_build
$ cp ../cmake/config.cmake ./

# You maybe need to enable other options, e.g., USE_OPENCL, USE_xPU.
$ sed -i "s|USE_LLVM.*)|USE_LLVM OFF)|" config.cmake
$ sed -i "s|USE_LIBBACKTRACE.*)|USE_LIBBACKTRACE OFF)|" config.cmake

$ cmake -DCMAKE_TOOLCHAIN_FILE=/YYY/aarch64-linux-gnu.cmake -DCMAKE_BUILD_TYPE=Release ..
$ cmake --build . -j -- runtime
$ cd ..

```

--------------------------------

### TVM Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1te_1_1Tensor_1_1Slice-members

Lists available examples within the TVM project.

```APIDOC
## TVM Examples API

### Description
This endpoint lists the available examples provided within the TVM project.

### Method
GET

### Endpoint
/examples

### Parameters
None

### Response
#### Success Response (200)
- **examples** (array) - A list of example paths or identifiers.

#### Response Example
```json
{
  "examples": [
    "example1.py",
    "example2.cc",
    "tutorials/intro.md"
  ]
}
```
```

--------------------------------

### Create Dummy NumPy for TVM RPC Server (Python)

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

This snippet provides a dummy 'numpy.py' file to resolve issues where the TVM RPC server fails to launch due to a missing 'numpy' dependency. This workaround is useful when cross-compiling 'numpy' is difficult or not feasible on the target device. It creates placeholder classes and functions that satisfy the import requirements without needing the full 'numpy' library.

```python
class bool_:
  pass
class int8:
    pass
class int16:
    pass
class int32:
    pass
class int64:
    pass
class uint8:
    pass
class uint16:
    pass
class uint32:
    pass
class uint64:
    pass
class float16:
    pass
class float32:
    pass
class float64:
    pass
class float_:
    pass

class dtype:
    def __init__(self, *args, **kwargs):
        pass

class ndarray:
    pass

def sqrt(*args, **kwargs):
    pass

def log(*args, **kwargs):
    pass

def tanh(*args, **kwargs):
    pass

def power(*args, **kwargs):
    pass

def exp(*args, **kwargs):
    pass

```

--------------------------------

### Cross Compile TVM Runtime with CMake Toolchain

Source: https://tvm.apache.org/docs/how_to/dev/setup_rpc_system

This snippet shows how to configure CMake for cross-compilation of the TVM runtime on a 64-bit ARM Linux device. It specifies the system name, C/C++ compilers, and sysroot, along with find root path modes. Ensure the toolchain file path and build type are adjusted for your environment.

```cmake
set(CMAKE_SYSTEM_NAME Linux)
set(root_dir "/XXX/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu")

set(CMAKE_C_COMPILER "${root_dir}/bin/aarch64-linux-gnu-gcc")
set(CMAKE_CXX_COMPILER "${root_dir}/bin/aarch64-linux-gnu-g++")
set(CMAKE_SYSROOT "${root_dir}/aarch64-linux-gnu/libc")

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

```

--------------------------------

### Build and Run TVM C++ Unit Tests (Bash)

Source: https://tvm.apache.org/docs/install/from_source

Commands to build and execute C++ unit tests for the TVM project after Google Test has been installed. These scripts facilitate the testing process, ensuring that C++ components function as expected.

```bash
# Build and run C++ unit tests
./tests/scripts/task_cpp_unittest.sh

# Or just build C++ tests
make cpptest
```

--------------------------------

### TVM Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1tir_1_1RampNode-members

Access to example usage and code snippets for various TVM functionalities.

```APIDOC
## TVM Examples

### Description
Provides illustrative examples of how to use different components and features of the TVM library.

### Endpoint

- `/websites/tvm_apache/tvm/examples` (GET)
```

--------------------------------

### Instance Methods - Benchmarking and Initialization

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Includes methods for benchmarking function execution and initializing WebGPU pipelines.

```APIDOC
## Instance Methods - Benchmarking and Initialization

### Description
Includes methods for benchmarking function execution and initializing WebGPU pipelines.

### Methods
#### `benchmark(run, dev, number?, repeat?)`
- **Description**: Benchmarks the stable execution of a given function.
- **Parameters**:
  - `run` (Function): The function to benchmark.
  - `dev` (Device): The device to run the benchmark on.
  - `number` (number, Optional): The number of iterations per run.
  - `repeat` (number, Optional): The number of times to repeat the benchmark.
- **Returns**: `Promise<number[]>` An array of average execution times.

#### `initWebGPU()`
- **Description**: Initializes the WebGPU backend.
- **Returns**: `Promise<void>`
```

--------------------------------

### Start Jupyter Notebook

Source: https://tvm.apache.org/docs/install/docker

Starts a Jupyter Notebook server within the running Docker container. This command is typically run after launching the Docker image and entering the environment. Assumes Jupyter is installed in the image.

```bash
jupyter notebook

```

--------------------------------

### createVirtualMachine API

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Sets up a virtual machine module with a given device.

```APIDOC
## POST /createVirtualMachine

### Description
Sets up a virtual machine module on the specified device.

### Method
POST

### Endpoint
/createVirtualMachine

### Parameters
#### Request Body
- **dev** (DLDevice) - Required - The DLDevice on which to set up the virtual machine.

### Returns
VirtualMachine

### Response
#### Success Response (200)
- **virtualMachine** (VirtualMachine) - The created virtual machine instance.

#### Response Example
{
  "virtualMachine": {
    "id": "vm-123"
  }
}
```

--------------------------------

### TVM Runtime: Timer Start Function

Source: https://tvm.apache.org/docs/reference/api/doxygen/profiling_8h_source

Provides a static method to get a device-specific timer. This is crucial for timing operations on a particular device within the TVM runtime. It returns a Timer object that can be used to start and stop measurements.

```cpp
static Timer Start(Device dev)
```

--------------------------------

### initWebGPU API

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Initializes the WebGPU environment within the runtime.

```APIDOC
## POST /initWebGPU

### Description
Initializes the WebGPU capabilities within the TVM runtime, using the provided GPUDevice.

### Method
POST

### Endpoint
/initWebGPU

### Parameters
#### Request Body
- **device** (GPUDevice) - Required - The GPUDevice object to use for initialization.

### Response
#### Success Response (200)
- **message** (string) - Confirmation message indicating successful WebGPU initialization.

#### Response Example
{
  "message": "WebGPU initialized successfully."
}
```

--------------------------------

### C++: Start a device-specific timer with tvm::runtime::Timer::Start

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1Timer

This C++ code demonstrates how to use the `tvm::runtime::Timer::Start` function to get a timer for a specific device, like the CPU. The timer is started immediately upon retrieval and can be used to measure elapsed time after calling `Stop()` and `SyncAndGetElapsedNanos()`.

```cpp
Timer t = Timer::Start(Device::cpu());
my_long_running_function();
t->Stop();
// ... some more computation
int64_t nanosecs = t->SyncAndGetElapsedNanos(); // elapsed time in nanoseconds
```

--------------------------------

### Profiler Constructor

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1profiling_1_1Profiler

Constructs a Profiler object. It's recommended to create the profiler before starting any warmup iterations, as this process resets the TVM threadpool to install necessary thread handlers.

```APIDOC
## Profiler()

### Description
Constructor for the `Profiler` class. It initializes the profiler and should be called before any warmup iterations.

Note: Calling this constructor resets the TVM threadpool to install required thread handlers.

### Method
Constructor

### Parameters
- **_devs_** (std::vector<Device>) - The list of devices the profiler will operate on. This should encompass all devices utilized by the profiled operators.
- **_metric_collectors_** (std::vector<MetricCollector>) - Additional `MetricCollector`s to be used with this profiler.
- **_configuration_** (std::unordered_map<ffi::String, ffi::Any>) - Optional. Additional configuration data to be included in the profiling report. Defaults to an empty map.

### Request Example
```json
{
  "_devs_": ["cpu", "gpu"],
  "_metric_collectors_": [],
  "_configuration_": {"log_level": "info"}
}
```

### Response
#### Success Response (Constructor)
Initializes the Profiler object.

#### Response Example
(No explicit response object for constructor, object is created in scope)
```

--------------------------------

### TVM Runtime Execution in C++

Source: https://tvm.apache.org/docs/_downloads/eceb05a9badb601d2def02240aa869e9/quick_start

Provides an example of how to load and execute a compiled TVM module in C++. It demonstrates initializing the virtual machine, invoking functions like 'prefill', and obtaining results as a Tensor.

```cpp
// C++ snippet
runtime::Module vm = ex.GetFunction("load_executable")();
vm.GetFunction("init")(...);
Tensor out = vm.GetFunction("prefill")(data, weight, kv_cache);
```

--------------------------------

### Get CUDA Version

Source: https://tvm.apache.org/docs/reference/api/python/contrib

Retrieves the installed CUDA version. It can optionally take a path to the CUDA root directory; otherwise, it uses the path found by find_cuda_path(). It returns the version as a float.

```python
tvm.contrib.nvcc.get_cuda_version(_cuda_path =None_)
```

--------------------------------

### Get Target Triple from Compiler (Python)

Source: https://tvm.apache.org/docs/reference/api/python/contrib

Provides a functor to obtain the target triple (e.g., architecture, OS) using a compiler's dump machine information. This is useful for cross-compilation setups.

```python
tvm.contrib.cc.get_target_by_dump_machine(_compiler_)
```

--------------------------------

### Handle Specific Errors in Python from C++

Source: https://tvm.apache.org/docs/contribute/error_handling

Shows how to call a C++ function registered as a PackedFunc in Python and catch the specific errors raised. The example demonstrates catching ValueError and InternalError.

```python
>>> import tvm
>>> tvm.testing.ErrorTest(0, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/tvm/python/tvm/_ffi/_ctypes/function.py", line 190, in __call__
    raise get_last_ffi_error()
ValueError: Traceback (most recent call last):
  [bt] (3) /path/to/tvm/build/libtvm.so(TVMFuncCall+0x48) [0x7fab500b8ca8]
  [bt] (2) /path/to/tvm/build/libtvm.so(+0x1c4126) [0x7fab4f7f5126]
  [bt] (1) /path/to/tvm/build/libtvm.so(+0x1ba2f8) [0x7fab4f7eb2f8]
  [bt] (0) /path/to/tvm/build/libtvm.so(+0x177d12) [0x7fab4f7a8d12]
  File "/path/to/tvm/src/api/api_test.cc", line 80
ValueError: Check failed: x == y (0 vs. 1) : expect x and y to be equal.
>>>
>>> tvm.testing.ErrorTest(1, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/tvm/python/tvm/_ffi/_ctypes/function.py", line 190, in __call__
    raise get_last_ffi_error()
tvm.error.InternalError: Traceback (most recent call last):
  [bt] (3) /path/to/tvm/build/libtvm.so(TVMFuncCall+0x48) [0x7fab500b8ca8]
  [bt] (2) /path/to/tvm/build/libtvm.so(+0x1c4126) [0x7fab4f7f5126]
  [bt] (1) /path/to/tvm/build/libtvm.so(+0x1ba35c) [0x7fab4f7eb35c]
  [bt] (0) /path/to/tvm/build/libtvm.so(+0x177d12) [0x7fab4f7a8d12]
  File "/path/to/tvm/src/api/api_test.cc", line 83
InternalError: cannot reach here
TVM hint: You hit an internal error. Please open a thread on https://discuss.tvm.ai/ to report it.


```

--------------------------------

### webgpu

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Creates a new WebGPU DLDevice.

```APIDOC
## webgpu

### Description
Create a new webgpu DLDevice.

### Method
Not specified.

### Endpoint
Not specified.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **deviceId** (number) - Optional - Defaults to `0`. The device index.

### Request Example
```javascript
const webGpuDevice = tvm.webgpu(0);
```

### Response
#### Success Response (DLDevice)
* **DLDevice** - The created WebGPU DLDevice.

#### Response Example
```json
{
  "device_type": "webgpu",
  "device_id": 0,
  "//": "Represents a DLDevice object"
}
```
```

--------------------------------

### TVM Instance Device and Runtime Info

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Provides methods for interacting with devices and retrieving runtime information. `device` returns the current device context, `empty` creates an empty tensor, `initWebGPU` initializes the WebGPU backend, and `runtimeStatsText` returns a string of runtime statistics.

```javascript
device(device_type: string, device_id: number): Device
empty(shape: ShapeTuple, dtype: string, device?: Device): Tensor
initWebGPU(): Promise<void>
runtimeStatsText(): string
```

--------------------------------

### LinearCongruentialEngine: DeviceRandom

Source: https://tvm.apache.org/docs/reference/api/doxygen/random__engine_8h_source

Gets a device random state. This function is intended to retrieve a random seed from the underlying system or hardware, providing a non-deterministic starting point for the random number generator. Its implementation may vary based on the operating system or hardware capabilities.

```cpp
static TRandState DeviceRandom() {
  // TODO(tvm-team): Implement DeviceRandom
  return 1;
}
```

--------------------------------

### Install TVM via Pip Local Project

Source: https://tvm.apache.org/docs/install/from_source

This installation method uses pip to install TVM from a local project source. It involves activating a Conda environment, ensuring Python is installed, setting the library path, and then performing an editable installation of the Python package.

```bash
conda activate your-own-env
conda install python # make sure python is installed
export TVM_LIBRARY_PATH=/path-to-tvm/build
pip install -e /path-to-tvm/python

```

--------------------------------

### Synchronize and Get Elapsed Nanoseconds for TimerNode

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1TimerNode

The SyncAndGetElapsedNanos function synchronizes the timer's state and returns the elapsed time in nanoseconds between its Start and Stop calls. It's recommended to stop all timers before calling this function on any of them, and it should only be called once per TimerNode object due to potential synchronization overhead.

```cpp
virtual int64_t tvm::runtime::TimerNode::SyncAndGetElapsedNanos() const = 0
```

--------------------------------

### C++ MetricCollectorNode Example Usage

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1profiling_1_1MetricCollectorNode

This C++ code snippet demonstrates the typical workflow for using a `MetricCollectorNode` in TVM. It shows how to create an instance of a collector, start collecting metrics before an operation, execute the operation, and then stop collecting metrics to obtain profiling data. The collected metrics are then added to the profiling report.

```cpp
MetricCollectorNode mc;
for (auto op : model) {
  auto o = mc.Start();
  op();
  auto metrics = mc.Stop(o); // metrics are added the profiling report
}
```

--------------------------------

### Build TVM on Windows with MSVC (CMake)

Source: https://tvm.apache.org/docs/install/from_source

Provides instructions for building Apache TVM on Windows using MSVC and CMake. It outlines the steps to generate the solution file and then execute the build command.

```bash
mkdir build
cd build
cmake ..
cd ..
cmake --build build --config Release -- /m
```

--------------------------------

### Compile and Deploy on CPU

Source: https://tvm.apache.org/docs/_downloads/a6d7947451d373bc811080cffa18dc7c/ir_module

Compiles an IRModule for the CPU backend using 'llvm' target and deploys it using TVM's virtual machine. It demonstrates running inference on a CPU device with sample data and parameters, printing the raw output.

```python
exec = tvm.compile(mod, target="llvm")
dev = tvm.cpu()
vm = relax.VirtualMachine(exec, dev)

raw_data = np.random.rand(1, 784).astype("float32")
data = tvm.runtime.tensor(raw_data, dev)
cpu_out = vm["main"](data, *params_from_torch["main"]).numpy()
print(cpu_out)
```

--------------------------------

### Virtual Machine Configuration

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1vm_1_1VirtualMachine

APIs for initializing and configuring the TVM Virtual Machine.

```APIDOC
## POST /vm/{vm_instance_id}/init

### Description
Initializes the virtual machine for a set of devices and allocator types.

### Method
POST

### Endpoint
/vm/{vm_instance_id}/init

### Parameters
#### Path Parameters
- **vm_instance_id** (string) - Required - The identifier of the VM instance to initialize.

#### Request Body
- **devices** (array[object]) - Required - The set of TVM devices.
  - **device_type** (string) - The type of the device (e.g., "cpu", "cuda").
  - **device_id** (integer) - The ID of the device.
- **alloc_types** (array[string]) - Required - The allocator types for each device (e.g., "default").

### Request Example
```json
{
  "devices": [
    {"device_type": "cpu", "device_id": 0}
  ],
  "alloc_types": ["default"]
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message indicating successful initialization.

#### Response Example
```json
{
  "message": "VM initialized successfully."
}
```
```

--------------------------------

### Build TVM with CMake

Source: https://tvm.apache.org/docs/install/from_source

This command initiates the build process for TVM after configuration. It uses CMake to compile the source code, leveraging multiple cores for faster compilation via the `--parallel` option.

```bash
cmake .. && cmake --build . --parallel $(nproc)

```

--------------------------------

### Locate TVM Python Package

Source: https://tvm.apache.org/docs/install/from_source

Confirms that TVM is properly installed as a Python package and displays its file location. This is a crucial first step in validating the installation.

```python
>>> python -c "import tvm; print(tvm.__file__)"
/some-path/lib/python3.11/site-packages/tvm/__init__.py
```

--------------------------------

### Create Virtual Machine Instance

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1vm_1_1VirtualMachine

Provides static methods to create instances of the VirtualMachine. `Create()` instantiates a standard VM, while `CreateProfiler()` instantiates a VM with profiling enabled. Both return a smart pointer to a `VirtualMachine` object.

```cpp
static ObjectPtr<VirtualMachine> tvm::runtime::vm::VirtualMachine::Create ();
```

```cpp
static ObjectPtr<VirtualMachine> tvm::runtime::vm::VirtualMachine::CreateProfiler ();
```

--------------------------------

### Target Class Constructors and Static Methods - C++

Source: https://tvm.apache.org/docs/reference/api/doxygen/target_8h_source

Demonstrates various constructors and static methods for the Target class, including creating a Target from a null pointer, a string, a configuration map, and combining targets with hosts. It also shows how to get the current target.

```cpp
TVM_DLL explicit Target(std::nullptr_t) { data_ = nullptr; }
TVM_DLL explicit Target(const ffi::String& tag_or_config_or_target_str);
TVM_DLL explicit Target(const ffi::Map<ffi::String, ffi::Any>& config);
TVM_DLL static tvm::Target Current(bool allow_not_defined = true);
TVM_DLL explicit Target(Target target, Target host);
```

--------------------------------

### Initialize Virtual Machine

Source: https://tvm.apache.org/docs/reference/api/doxygen/vm_8h_source

Initializes the virtual machine with a specified set of devices and allocator types.

```APIDOC
## Init

### Description
Initialize the virtual machine for a set of devices.

### Method
`virtual void Init(const std::vector< Device > &devices, const std::vector< AllocatorType > &alloc_types)`

### Endpoint
`tvm::runtime::vm::VirtualMachine::Init`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **devices** (std::vector< Device >) - Required - The list of devices to initialize.
- **alloc_types** (std::vector< AllocatorType >) - Required - The types of allocators to use.

### Request Example
```json
{
  "devices": ["cpu"], 
  "alloc_types": ["default"]
}
```

### Response
#### Success Response (200)
Void

#### Response Example
None
```

--------------------------------

### PyDatabase Constructor

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1meta__schedule_1_1PyDatabaseNode

Initializes a new instance of the PyDatabase.

```APIDOC
## PyDatabase Constructor

### Description
Initializes a new instance of the PyDatabase.

### Method
CONSTRUCTOR

### Endpoint
N/A

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
{
  "mod_eq_name": "structural"
}
```

### Response
#### Success Response (200)
Instance of PyDatabase.

#### Response Example
```json
{
  "message": "PyDatabase instance created successfully"
}
```
```

--------------------------------

### MetricCollectorNode Start

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1profiling_1_1MetricCollectorNode

Starts collecting metrics for a specific function call on a given device.

```APIDOC
## POST /websites/tvm_apache/MetricCollectorNode/Start

### Description
Starts collecting metrics for a function call on the specified device.

### Method
POST

### Endpoint
/websites/tvm_apache/MetricCollectorNode/Start

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **_dev_** (Device) - Required - The device the call will be run on.

### Request Example
```json
{
  "_dev_": {
    // Device object details
  }
}
```

### Response
#### Success Response (200)
- **return_value** (ObjectRef) - An object used to maintain state of the metric collection. This object will be passed to the corresponding `Stop` call. If the device is not supported, this function will return a nullptr ObjectRef.

#### Response Example
```json
{
  "return_value": {
    // ObjectRef details or null if device not supported
  }
}
```
```

--------------------------------

### Database Constructor

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1meta__schedule_1_1DatabaseNode

Initializes a new Database instance. It allows specifying a module equality method for comparing IRModules.

```APIDOC
## POST /database

### Description
Initializes a new Database instance. It allows specifying a module equality method for comparing IRModules.

### Method
POST

### Endpoint
/database

### Parameters
#### Request Body
- **mod_eq_name** (string) - Optional - A string to specify the module equality testing and hashing method. It must be one of the followings:
  * "structural": Use StructuralEqual/Hash
  * "ignore-tensor": Same as "structural", but ignore tensor raw data during equality testing and hashing.
  * "anchor-block": Apply equality testing and hashing on the anchor block extracted from a given module. The "ignore-tensor" varint is used for the extracted blocks or in case no anchor block is found. For the definition of the anchor block, see tvm/tir/analysis.h.

### Request Example
```json
{
  "mod_eq_name": "structural"
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message of database creation.

#### Response Example
```json
{
  "message": "Database created successfully."
}
```
```

--------------------------------

### Build and VM Execution

Source: https://tvm.apache.org/docs/reference/api/python/relax/relax

APIs for building executables and inspecting their properties.

```APIDOC
## Build and VM Execution

### Description
This section details the `build` function for creating TVM executables and the `VMExecutable` class for inspecting and analyzing these executables.

### Functions and Classes

*   `build(mod, target, params=None)`: Builds a TVM module for a given target.
*   `VMExecutable`:
    *   `stats()`: Returns statistics about the VM executable.
    *   `as_text()`: Returns the executable in text format.
    *   `as_python()`: Returns the executable as Python code.
```

--------------------------------

### Install TVM via pip (Linux/MacOS CPU)

Source: https://tvm.apache.org/docs/install/index

This command installs the Apache TVM package using pip. It is suitable for CPU builds on Linux and MacOS operating systems. For other build targets, including CUDA, refer to tlcpack.ai.

```bash
# Linux/MacOS CPU build only!
# See tlcpack.ai for other pre-built binaries including CUDA
pip install apache-tvm

```

--------------------------------

### cpu API

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Creates a new CPU DLDevice.

```APIDOC
## GET /cpu

### Description
Creates and returns a new CPU DLDevice.

### Method
GET

### Endpoint
/cpu

### Parameters
#### Query Parameters
- **deviceId** (number) - Optional. The index of the device. Defaults to 0.

### Returns
DLDevice

### Response
#### Success Response (200)
- **device** (DLDevice) - The created CPU DLDevice.

#### Response Example
{
  "device": {
    "type": "cpu",
    "id": 0
  }
}
```

--------------------------------

### tvm.topi.one_hot

Source: https://tvm.apache.org/docs/reference/api/python/topi

Generates a one-hot encoded tensor. Locations specified by indices get `on_value`, others get `off_value`.

```APIDOC
## tvm.topi.one_hot

### Description
Returns a one-hot tensor where the locations repsented by indices take value on_value, other locations take value off_value. Final dimension is <indices outer dimensions> x depth x <indices inner dimensions>.

### Method
N/A (Function Call)

### Endpoint
N/A (Function Call)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
* **indices** (tvm.te.Tensor) - Locations to set to on_value.
* **on_value** (tvm.te.Tensor) - Value to fill at indices.
* **off_value** (tvm.te.Tensor) - Value to fill at all other positions besides indices.
* **depth** (int) - Depth of the one-hot dimension.
* **axis** (int) - Axis to fill.
* **dtype** (str) - Data type of the output tensor.

### Request Example
```python
# Example usage (conceptual)
indices = tvm.te.placeholder((3,), name='indices')
on_value = 1.0
off_value = 0.0
depth = 3
axis = 0
dtype = "float32"

result = tvm.topi.one_hot(indices, on_value, off_value, depth, axis, dtype)
```

### Response
#### Success Response (200)
* **ret** (tvm.te.Tensor) - The one-hot tensor.

#### Response Example
```json
{
  "type": "tvm.te.Tensor"
}
```
```

--------------------------------

### Instance Methods - WebGPU and Async Operations

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Contains methods for managing WebGPU pipelines and asynchronous operations within the TVM runtime.

```APIDOC
## Instance Methods - WebGPU and Async Operations

### Description
Contains methods for managing WebGPU pipelines and asynchronous operations within the TVM runtime.

### Methods
#### `asyncLoadWebGPUPipelines(mod)`
- **Description**: Asynchronously loads WebGPU pipelines.
- **Parameters**:
  - `mod` (Module): The input module.
- **Returns**: `Promise<void>`

#### `asyncifyEnabled()`
- **Description**: Checks if the asyncify mode is enabled.
- **Returns**: `boolean` The asyncify mode toggle.
```

--------------------------------

### Install Extra Python Dependencies (pip)

Source: https://tvm.apache.org/docs/install/from_source

Provides commands to install necessary Python dependencies for Apache TVM. This includes core dependencies like NumPy, and optional ones for RPC Tracker (Tornado) and the auto-tuning module (Tornado, psutil, XGBoost, cloudpickle).

```bash
pip3 install numpy
```

```bash
pip3 install tornado
```

```bash
pip3 install tornado psutil 'xgboost>=1.1.0' cloudpickle
```

--------------------------------

### Install Python Test Dependencies for TVM

Source: https://tvm.apache.org/docs/contribute/pull_request

Installs necessary Python packages for running TVM tests, including pytest and Cython. This command should be run before executing Python tests. It uses pip with the '--user' flag to install packages locally.

```bash
pip install --user pytest Cython

```

--------------------------------

### Initialization API

Source: https://tvm.apache.org/docs/reference/api/doxygen/c__backend__api_8h_source

Simple static initialization function.

```APIDOC
## TVMBackendRunOnce

### Description
Simple static initialization function. Run f once and set handle to be not null. This function is mainly used for the backend initialization.

### Method
N/A (C function signature)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
N/A

### Response
#### Success Response (0 on success)
- **int** - 0 on success.

#### Response Example
N/A
```

--------------------------------

### VirtualMachine Creation

Source: https://tvm.apache.org/docs/reference/api/doxygen/vm_8h_source

Provides methods to create instances of the VirtualMachine.

```APIDOC
## VirtualMachine Creation

### Description
Static methods to create and initialize the TVM Virtual Machine.

### Methods

#### `Create()`

- **Description**: Creates a standard instance of the VirtualMachine.
- **Method**: `POST`
- **Endpoint**: `/vm/create`

#### `CreateProfiler()`

- **Description**: Creates a VirtualMachine instance configured for profiling.
- **Method**: `POST`
- **Endpoint**: `/vm/createProfiler`

#### `GetContextPtr(arg)`

- **Description**: A helper function for VM closure functions to retrieve the VirtualMachine context pointer.
- **Method**: `GET`
- **Endpoint**: `/vm/contextPtr`
- **Parameters**:
  - **Path Parameters**:
    - None
  - **Query Parameters**:
    - `arg` (ffi::AnyView) - The argument to cast to a VirtualMachine pointer.
- **Response**:
  - **Success Response (200)**:
    - `VirtualMachine*` - A pointer to the VirtualMachine instance.
```

--------------------------------

### Examples

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1script_1_1printer_1_1TupleDocNode

This section provides access to example usage and demonstrations of the TVM project's features.

```APIDOC
## Examples

This section contains examples demonstrating how to use various components and features of the TVM project.
```

--------------------------------

### Virtual Machine Creation

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1runtime_1_1vm_1_1VirtualMachine

APIs for creating instances of the TVM Virtual Machine.

```APIDOC
## POST /vm/create

### Description
Creates a specific instance of the TVM Virtual Machine.

### Method
POST

### Endpoint
/vm/create

### Parameters
None

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **vm_instance_id** (string) - A unique identifier for the created VM instance.

#### Response Example
```json
{
  "vm_instance_id": "vm_12345"
}
```

## GET /vm/create_profiler

### Description
Creates an instance of the TVM Virtual Machine with the profiling feature enabled.

### Method
GET

### Endpoint
/vm/create_profiler

### Parameters
None

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **vm_instance_id** (string) - A unique identifier for the created VM instance with profiling enabled.

#### Response Example
```json
{
  "vm_instance_id": "vm_profiler_67890"
}
```
```

--------------------------------

### TVM Relax Block Builder API Example

Source: https://tvm.apache.org/docs/deep_dive/relax/tutorials/relax_creation

Demonstrates how to construct a Relax function using the Block Builder API. This example builds a 'forward' function that performs matrix multiplications and ReLU activation, mirroring the functionality of the script-based example but using a more imperative style.

```python
import tvm
from tvm import relax
from tvm.script import tir as T
from tvm.script import relax as R

bb = relax.BlockBuilder()
n = T.int64()
x = relax.Var("x", R.Tensor((n, 784), "float32"))
fc1_weight = relax.Var("fc1_weight", R.Tensor((128, 784), "float32"))
fc1_bias = relax.Var("fc1_bias", R.Tensor((128,), "float32"))
fc2_weight = relax.Var("fc2_weight", R.Tensor((10, 128), "float32"))
fc2_bias = relax.Var("fc2_bias", R.Tensor((10,), "float32"))
with bb.function("forward", [x, fc1_weight, fc1_bias, fc2_weight, fc2_bias]):
    with bb.dataflow():
        lv0 = bb.emit(relax.op.matmul(x, relax.op.permute_dims(fc1_weight)) + fc1_bias)
        lv1 = bb.emit(relax.op.nn.relu(lv0))
        gv = bb.emit(relax.op.matmul(lv1, relax.op.permute_dims(fc2_weight)) + fc2_bias)
        bb.emit_output(gv)
    bb.emit_func_output(gv)

mod = bb.get()
mod.show()
```

--------------------------------

### Create Tuple Get Item Pattern - IsTupleGetItem

Source: https://tvm.apache.org/docs/reference/api/doxygen/namespacetvm_1_1relax

Creates a TupleGetItemPattern. It requires a DFPattern representing the tuple and an optional integer for the index of the item to get.

```cpp
TupleGetItemPattern tvm::relax::IsTupleGetItem(const DFPattern _tuple_, int _index_ = -1);
```

--------------------------------

### TVM Runtime Execution in Java

Source: https://tvm.apache.org/docs/_downloads/eceb05a9badb601d2def02240aa869e9/quick_start

Shows how to interact with a compiled TVM module using Java. This includes loading the executable, initializing functions, and invoking methods like 'prefill' by pushing arguments and executing the function.

```java
// Java snippet
Module vm = ex.getFunction("load_executable").invoke();
vm.getFunction("init").pushArg(...).invoke;
Tensor out = vm.getFunction("prefill").pushArg(data).pushArg(weight).pushArg(kv_cache).invoke();
```

--------------------------------

### sampleTopPFromLogits API

Source: https://tvm.apache.org/docs/reference/api/typedoc/classes/Instance

Samples an index from a distribution using top-p sampling based on input logits.

```APIDOC
## sampleTopPFromLogits

### Description
Sample index via top-p sampling.

### Method
sampleTopPFromLogits(logits: Tensor, temperature: number, top_p: number): number

### Parameters
#### Path Parameters
* **logits** (Tensor) - Required - The input logits before normalization.
* **temperature** (number) - Required - The temperature factor. If temperature is 0.0, argmax is taken.
* **top_p** (number) - Required - The top-p value for sampling.

### Returns
* number - The sampled index.
```

--------------------------------

### Find CUDA Installation Path

Source: https://tvm.apache.org/docs/reference/api/python/contrib

A utility function to locate the root directory of the CUDA installation on the system. It returns the path as a string.

```python
tvm.contrib.nvcc.find_cuda_path()
```

--------------------------------

### TVM TIR BlockNode Usage Example

Source: https://tvm.apache.org/docs/reference/api/doxygen/classtvm_1_1tir_1_1BlockNode

Illustrative example of how a TVM TIR block can be constructed and used within a schedule.

```APIDOC
## Example Usage

A block is a basic schedule unit in TIR. Its body is parameterized by iteration variables.

```python
# Conceptual Python-like representation within TVM's schedule DSL
with T.block("my_block_name"):
    # Define iteration variables
    v0 = T.axis.S(domain, value0)
    v1 = T.axis.R(domain, value1)

    # Specify buffer regions read and written by the block
    T.reads([buffer0[start:end, ...]])
    T.writes([buffer1[start:end, ...]])

    # Optional: Define a predicate for the block
    T.where(predicate)

    # Allocate buffers within the block
    buffer2 = T.alloc_buffer(shape, dtype)

    # Match existing buffers
    buffer3 = T.match_buffer(source_buffer[start:end, ...])

    # Apply attributes or annotations
    T.attr({"attr_key": "attr_value"})

    # Optional: Define an initialization statement for reduction blocks
    with T.init():
        # init body
        pass

    # The main body of the block's computation
    # ... computation using v0, v1, buffer2, etc. ...
    pass
```
```

--------------------------------

### Device Management

Source: https://tvm.apache.org/docs/reference/api/javadoc/index-all

APIs for managing and querying devices, including CPU, CUDA, and generic device creation.

```APIDOC
## C

### `cpu()`

**Description**: Static method to construct CPU device.

**Method**: Static Method

**Endpoint**: N/A

### `cpu(int)`

**Description**: Static method to construct a CPU device.

**Method**: Static Method

**Endpoint**: N/A

### `cuda()`

**Description**: Static method to construct CUDA GPU device.

**Method**: Static Method

**Endpoint**: N/A

### `cuda(int)`

**Description**: Static method to construct a CUDA GPU device.

**Method**: Static Method

**Endpoint**: N/A

## D

### `Device`

**Description**: Represents a computing device.

### `Device(int, int)`

**Description**: Constructor for `Device`.

**Method**: Constructor

**Endpoint**: N/A

### `Device(String, int)`

**Description**: Constructor for `Device`.

**Method**: Constructor

**Endpoint**: N/A

### `deviceId`

**Description**: Variable for device ID.

### `deviceType`

**Description**: Variable for device type.

### `exist()`

**Description**: Whether this device exists.

**Method**: N/A (Method Signature)

**Endpoint**: N/A
```