Inference Engine

https://docs.unity3d.com/Packages/com.unity.ai.inference@2.3/
Admin
Inference Engine is a neural network inference library for Unity that allows importing and running...
Tokens:147,555
Snippets:1,464
Trust Score:10
Update:5 days ago
Show doc for...
Tokens:
Raw
### Async readback with ReadbackAndCloneAsync

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/read-output-async.html

Use the awaitable ReadbackAndCloneAsync method to get a CPU copy of the output tensor without blocking the main thread. This example shows a MonoBehaviour that schedules a model on the GPUCompute backend and awaits the readback.

```csharp
using Unity.InferenceEngine;
using UnityEngine;

public class AsyncReadbackCompute : MonoBehaviour
{
    [SerializeField]
    ModelAsset modelAsset;

    Tensor<float> m_Input;
    Worker m_Worker;

    async void OnEnable()
    {
        var model = ModelLoader.Load(modelAsset);
        m_Input = new Tensor<float>(new TensorShape(1, 1), new[] { 43.0f });
        m_Worker = new Worker(model, BackendType.GPUCompute);
        m_Worker.Schedule(m_Input);

        // Peek the value from Inference Engine, without taking ownership of the tensor
        var outputTensor = m_Worker.PeekOutput() as Tensor<float>;
        var cpuCopyTensor = await outputTensor.ReadbackAndCloneAsync();

        Debug.Assert(cpuCopyTensor[0] == 42);
        Debug.Log($"Output tensor value {cpuCopyTensor[0]}");
        cpuCopyTensor.Dispose();
    }

    void OnDisable()
    {
        m_Input.Dispose();
        m_Worker.Dispose();
    }
}
```

--------------------------------

### Example implementation of IONNXMetadataImportCallbackReceiver

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.IONNXMetadataImportCallbackReceiver.html

Example class that implements IONNXMetadataImportCallbackReceiver to read and store ONNX metadata. The OnMetadataImported callback stores the metadata in a public property.

```csharp
// Class to read and store the metadata from the ONNX file
public class MyMetadataCallbackHandler : IONNXMetadataImportCallbackReceiver
{
    public ONNXModelMetadata Metadata { get; private set; }

    // Callback for metadata import. Member Metadata is of type ONNXModelMetadata.
    public void OnMetadataImported(AssetImportContext ctx, ONNXModelMetadata metadata)
    {
        Metadata = metadata;
    }
}
```

--------------------------------

### Get model outputs with runtimeModel.outputs

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/inspect-a-model.html

Use the `outputs` property of the runtime model to get the names of the output layers. The example loops through each output and logs its name.

```C#
List<Model.Output> outputs = runtimeModel.outputs;

// Loop through each output
foreach (var output in outputs)
{
    // Log the name of the output
    Debug.Log(output.name);
}
```

--------------------------------

### Get model inputs with runtimeModel.inputs

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/inspect-a-model.html

Use the `inputs` property of the runtime model to retrieve a list of input tensors. The example loops through each input and logs its name and tensor shape. Requires a `ModelAsset` and `ModelLoader.Load`.

```C#
using UnityEngine;
using System.Collections.Generic;
using Unity.InferenceEngine;

public class GetModelInputs : MonoBehaviour
{
    public ModelAsset modelAsset;

    void Start()
    {
        Model runtimeModel = ModelLoader.Load(modelAsset);

        List<Model.Input> inputs = runtimeModel.inputs;

        // Loop through each input
        foreach (var input in inputs)
        {
            // Log the name of the input, for example Input3
            Debug.Log(input.name);

            // Log the tensor shape of the input, for example (1, 1, 28, 28)
            Debug.Log(input.shape);
        }
    }
}
```

--------------------------------

### Get and set values of a tensor

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/do-basic-tensor-operations.html

Demonstrates how to set and get tensor values directly when the backend type is CPU and the readback is complete.

```APIDOC
## Get and set values of a tensor

### Description
Set and get values of a tensor directly when the tensor's backend type is `BackendType.CPU` and the computation has finished (`IsReadbackRequestDone`).

### Method
Direct indexer access

### Syntax
```csharp
var tensor = new Tensor<float>(new TensorShape(1, 2, 3));
tensor[0, 1, 2] = 5.2f; // set value at index 0 of dim0 = 1, index 1 of dim1 = 2 and index 2 of dim2 = 3

float value = tensor[0, 1, 2];
Assert.AreEqual(5.2f, value);
```

### Parameters
- **index** (int[]) - Required - Multi-dimensional indices to access the tensor element.

### Return Value
- **float** - The value at the specified index when getting.

### Remarks
- Direct access is only allowed when `backendType` is `BackendType.CPU` and `IsReadbackRequestDone` is true.
```

--------------------------------

### Awaitable with callback for async readback

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/read-output-async.html

Use an awaitable with a callback to handle the readback completion. This example shows an Update loop that schedules a worker and uses the awaiter's OnCompleted callback to get the result and dispose the tensor.

```csharp
bool inferencePending = false;

void Update()
{
    if (!inferencePending)
    {
        m_Worker.Schedule(m_Input);
        var outputTensor = m_Worker.PeekOutput() as Tensor<float>;
        inferencePending = true;

        var awaiter = outputTensor.ReadbackAndCloneAsync().GetAwaiter();
        awaiter.OnCompleted(() =>
        {
            var tensorOut = awaiter.GetResult();
            inferencePending = false;
            tensorOut.Dispose();
        });
    }
}
```

--------------------------------

### Set and get tensor values

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/do-basic-tensor-operations.html

Use this snippet to directly set and get tensor values when the tensor's backendType is BackendType.CPU and IsReadbackRequestDone is true. The example sets a value at index (0,1,2) and then reads it back with an assertion.

```csharp
var tensor = new Tensor<float>(new TensorShape(1, 2, 3));
tensor[0, 1, 2] = 5.2f; // set value at index 0 of dim0 = 1, index 1 of dim1 = 2 and index 2 of dim2 = 3

float value = tensor[0, 1, 2];
Assert.AreEqual(5.2f, value);
```

--------------------------------

### buffer

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ComputeTensorData.html

Gets the data storage as a compute buffer.

```APIDOC
## buffer

### Description
The data storage as a compute buffer.

### Declaration
```csharp
public ComputeBuffer buffer { get; }
```

### Property Value
- **ComputeBuffer** - The compute buffer.

```

--------------------------------

### Avoid Tensor and Texture creation for performance

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/convert-texture-to-tensor.html

Allocates a tensor once in Start and reuses it in Update to copy webcam texture data without memory allocations.

```csharp
Tensor<float> inputTensor;
Texture webcamTexture;

// Allocate resources on startup
void Start()
{
    inputTensor = new Tensor<float>(new TensorShape(1, 3, webcamTexture.height, webcamTexture.width));
}
void Update()
{
    // Copy webcamTexture into inputTensor : no memory allocations!
    TextureConverter.ToTensor(webcamTexture, inputTensor, new TextureTransform());

    // Run inference
}
```

--------------------------------

### Request and clone output tensor asynchronously with ReadbackRequest and ReadbackAndClone

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/get-the-output.html

Requests an asynchronous readback of the output tensor with ReadbackRequest, then later calls ReadbackAndClone to get the data without blocking.

```csharp
Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;
outputTensor.ReadbackRequest();

// when done
outputTensor.ReadbackAndClone(); // not blocking
```

--------------------------------

### Asynchronously read back output tensor with ReadbackAndCloneAsync

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/get-the-output.html

Uses PeekOutput to get the output tensor and then awaits ReadbackAndCloneAsync to asynchronously retrieve a copy of the data, avoiding blocking the model.

```csharp
Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;
var result = await outputTensor.ReadbackAndCloneAsync();
```

--------------------------------

### Create and run a command buffer queue

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/use-command-buffer.html

Creates a worker with the GPUCompute backend and schedules it on a command buffer to run the model. The example shows the worker creation and the ScheduleWorker call, but the command buffer creation and execution are implied by the surrounding steps.

```csharp
// Create a worker that uses the GPUCompute backend type
worker = new Worker(ModelLoader.Load(modelAsset), BackendType.GPUCompute);

//...

// Add model execution command buffer that runs the model using the input tensor
commandBuffer.ScheduleWorker(worker, inputs);

```

--------------------------------

### StyleTransfer script: Render model output to screen

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/use-model-output.html

Attach this script to the camera and set modelAsset to a style transfer model and inputImage to a texture. It loads the model, rescales the output by dividing by 255f, compiles a runtime model, and in OnRenderImage converts the input texture to a tensor, runs the worker, and calls TextureConverter.RenderToScreen to copy the output tensor to the screen. Note: if the image is too bright, the output tensor may use 0-255 values; remap them before calling RenderToScreen. For URP/HDRP, call RenderToScreen in RenderPipelineManager.endFrameRendering or endContextRendering callbacks.

```csharp
using UnityEngine;
using Unity.InferenceEngine;

public class StyleTransfer : MonoBehaviour
{
    public ModelAsset modelAsset;
    public Model runtimeModel;
    public Texture2D inputImage;
    public RenderTexture outputTexture;

    Worker worker;
    Tensor<float> inputTensor;

    void Start()
    {
        var sourceModel = ModelLoader.Load(modelAsset);
        var graph = new FunctionalGraph();
        var input = graph.AddInput(sourceModel, 0);
        var output = Functional.Forward(sourceModel, input)[0];
        // rescale output of source model
        output /= 255f;
        graph.AddOutput(output);
        var runtimeModel = graph.Compile();

        worker = new Worker(runtimeModel, BackendType.GPUCompute);
        inputTensor = new Tensor<float>(new TensorShape(1, 3, 256, 256));
    }

    void OnRenderImage(RenderTexture source, RenderTexture destination)
    {
        // Create the input tensor from the texture
        TextureConverter.ToTensor(inputImage, inputTensor, new TextureTransform());

        // Run the model and get the output as a tensor
        worker.Schedule(inputTensor);
        Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;

        // Copy the rescaled tensor to the screen as a texture
        TextureConverter.RenderToScreen(outputTensor);
    }

    void OnDisable()
    {
        worker.Dispose();
    }
}
```

--------------------------------

### Compile()

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.FunctionalGraph.html

Compile and return an optimized runtime model.

```APIDOC
## Compile()

### Description
Compile and return an optimized runtime model.

### Method
Compile

### Returns
- **Model** - The compiled runtime model.
```

--------------------------------

### Quantize and serialize a model with ModelQuantizer and ModelWriter

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/quantize-a-model.html

Use ModelQuantizer.QuantizeWeights with QuantizationType.Float16 to quantize a model in memory, then ModelWriter.Save to serialize it to disk. Note that quantization destructively edits the source model in memory.

```csharp
using Unity.InferenceEngine;

void QuantizeAndSerializeModel(Model model, string path)
{
    // Inference Engine destructively edits the source model in memory when quantizing.
    ModelQuantizer.QuantizeWeights(QuantizationType.Float16, ref model);

    // Serialize the quantized model to a file.
    ModelWriter.Save(path, model);
}
```

--------------------------------

### ARange(float, float, float)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Functional.html

Returns a 1D tensor of size ⌈(end − start) / step⌉ with values from the interval [start, end) with a step beginning from start.

```APIDOC
## ARange(float, float, float)

### Description
Returns a 1D tensor of size ⌈(end − start) / step⌉ with values from the interval [start, end) with a step beginning from start.

### Method
ARange(float, float, float)

### Parameters
- **start** (float) - Required - The value of the first element.
- **end** (float) - Required - The upper end of the interval.
- **step** (float) - Optional - The delta between each element.

### Returns
- **FunctionalTensor** - The output tensor.
```

--------------------------------

### Include the Unity.InferenceEngine namespace

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/understand-inference-engine-workflow.html

Add this using directive at the top of your script to access the Inference Engine API.

```csharp
using Unity.InferenceEngine;
```

--------------------------------

### ARange(int, int, int)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Functional.html

Returns a 1D tensor of size ⌈(end − start) / step⌉ with values from the interval [start, end) with a step beginning from start.

```APIDOC
## ARange(int, int, int)

### Description
Returns a 1D tensor of size ⌈(end − start) / step⌉ with values from the interval [start, end) with a step beginning from start.

### Method
ARange(int, int, int)

### Parameters
- **start** (int) - Required - The value of the first element.
- **end** (int) - Required - The upper end of the interval.
- **step** (int) - Optional - The delta between each element.

### Returns
- **FunctionalTensor** - The output tensor.
```

--------------------------------

### Build command buffer once and update per frame

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/use-command-buffer.html

Builds a CommandBuffer once in Start, then updates the input tensor and executes the buffer each frame in Update. This reduces scheduling time by half compared to rebuilding the buffer every frame.

```csharp
Tensor<float> input0 = new Tensor<float>(new TensorShape(1));
Model model;
CommandBuffer cb;
Worker worker;

void Start()
{
    worker = new Worker(model, BackendType.GPUCompute);

    worker.SetInput("input0", input0);

    cb = new CommandBuffer();
    cb.ScheduleWorker(worker);
}

void Update()
{
    // modify input0
    input0.Upload(new float[] { Time.deltaTime });
    Graphics.ExecuteCommandBuffer(cb);
}

```

--------------------------------

### Get model outputs

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/inspect-a-model.html

Use the outputs property of the runtime model to get the names of the output layers of the model.

```APIDOC
## Get model outputs

### Description
Use the `outputs` property of the runtime model to get the names of the output layers of the model.

### Property
`Model.outputs`

### Returns
- **List<Model.Output>** - A list of output layers. Each `Model.Output` has:
  - **name** (string) - The name of the output layer.

### Example
```csharp
List<Model.Output> outputs = runtimeModel.outputs;

// Loop through each output
foreach (var output in outputs)
{
    // Log the name of the output
    Debug.Log(output.name);
}
```
```

--------------------------------

### Create a runtime model with ModelLoader.Load

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/import-a-model-file.html

Use this C# script in a MonoBehaviour to load an imported ONNX model asset into a runtime Model object. Assign the ModelAsset in the Inspector before running. After loading, you can create an engine to run the model.

```csharp
using UnityEngine;
using Unity.InferenceEngine;

public class CreateRuntimeModel : MonoBehaviour
{
    public ModelAsset modelAsset;
    Model runtimeModel;

    void Start()
    {
        runtimeModel = ModelLoader.Load(modelAsset);
    }
}
```

--------------------------------

### Create a Worker with BackendType.GPUCompute

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/create-an-engine.html

Creates a worker that runs on the GPU with Inference Engine compute shaders. Requires a ModelAsset and uses ModelLoader.Load to obtain a runtime model before constructing the Worker.

```csharp
using UnityEngine;
using Unity.InferenceEngine;

public class CreateWorker : MonoBehaviour
{
    ModelAsset modelAsset;
    Model runtimeModel;
    Worker worker;

    void Start()
    {
        runtimeModel = ModelLoader.Load(modelAsset);
        worker = new Worker(runtimeModel, BackendType.GPUCompute);
    }
}
```

--------------------------------

### Layer(int[] outputs, int[] inputs)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Layer.html

Initializes and returns a Layer from given arrays of input and output indices.

```APIDOC
## Constructor: Layer(int[] outputs, int[] inputs)

### Description
Initializes and returns a `Layer` from given arrays of input and output indices.

### Declaration
```csharp
protected Layer(int[] outputs, int[] inputs)
```

### Parameters
- **outputs** (int[]) - The indices array representing the outputs of this layer.
- **inputs** (int[]) - The indices array representing the inputs of this layer.

```

--------------------------------

### Get layers and layer properties

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/inspect-a-model.html

Use the layers property of the runtime model to get the neural network layers in the model. Each layer includes input and output identifiers, and other relevant properties.

```APIDOC
## Get layers and layer properties

### Description
Use the `layers` property of the runtime model to get the neural network layers in the model. Each layer includes input and output identifiers, and other relevant properties.

### Property
`Model.layers`

### Returns
- **List<Model.Layer>** - A list of neural network layers. Each layer includes input and output identifiers and other relevant properties.
```

--------------------------------

### RawPtr

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.NativeTensorArrayFromManagedArray.html

Gets the raw pointer of the backing data.

```APIDOC
## Property: RawPtr

### Description
The raw pointer of the backing data.

### Declaration
```csharp
public override void* RawPtr { get; }
```

### Property Value
- **void*** - The raw pointer to the backing data.
```

--------------------------------

### Use PeekOutput to get a reference to the output tensor

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/get-the-output.html

Calls worker.Schedule and then PeekOutput to get a reference to the output tensor, casting it to Tensor<float>. The worker owns the returned reference, so you don't need to dispose it, but it changes if you modify the output or rerun the worker.

```csharp
worker.Schedule(inputTensor);
Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;
```

--------------------------------

### Create a dot product model with the functional API

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/create-a-new-model.html

Creates a simple model that calculates the dot product of two vectors. It uses FunctionalGraph, AddInput, functional operators, AddOutput, and Compile. The example includes two inputs of shape (6) and outputs both the reduced sum and the elementwise product.

```csharp
using System;
using Unity.InferenceEngine;
using UnityEngine;

public class CreateNewModel : MonoBehaviour
{
    Model model;

    void Start()
    {
        // Create the functional graph.
        FunctionalGraph graph = new FunctionalGraph();

        // Add two inputs to the graph with data types and shapes.
        // Our dot product operates on two vector tensors of the same size `6`.
        FunctionalTensor x = graph.AddInput<float>(new TensorShape(6), "input_x");
        FunctionalTensor y = graph.AddInput<float>(new TensorShape(6), "input_y");

        // Calculate the elementwise product of the input `FunctionalTensor`s using an operator.
        FunctionalTensor prod = x * y;

        // Sum the product along the first axis flattening the summed dimension.
        FunctionalTensor reduce = Functional.ReduceSum(prod, dim: 0, keepdim: false);

        graph.AddOutput(reduce, "output_reduce");
        graph.AddOutput(prod, "output_prod");

        // Build the model using the `Compile` method.
        model = graph.Compile();
    }
}
```

--------------------------------

### ReadbackAndCloneAsync()

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Tensor-1.html

Schedules asynchronous download task of the internal data.

```APIDOC
## ReadbackAndCloneAsync()

### Description
Schedules asynchronous download task of the internal data.

### Declaration
```csharp
public Awaitable<Tensor<T>> ReadbackAndCloneAsync()
```

### Returns
| Type | Description |
|------|-------------|
| Awaitable<Tensor<T>> | Awaitable tensor on the CPU. |
```

--------------------------------

### ITensorData.maxCapacity

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ITensorData.html

Gets the maximum count of the stored data elements.

```APIDOC
## Property: maxCapacity

### Description
Gets the maximum count of the stored data elements.

### Declaration
```csharp
int maxCapacity { get; }
```

### Property Value
| Type | Description |
|------|-------------|
| int | The maximum capacity. |
```

--------------------------------

### Serialization layout diagram

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/serialize-a-model

ASCII diagram showing the FlatBuffers serialization layout of a .sentis file, including the model description and weight chunk data.

```text
             ┌───────────────────────────────────┐
             │Flatbuffer-serialized              |
             | model desription                  │
             │                                   │
          ┌─ ├───────────────────────────────────┤
          │  │ Weight chunk data                 |
          │  │                                   │
          │  │                                   │
Weights  ─┤  ├───────────────────────────────────┤
          │  │ Weight chunk data                 │
          │  │                                   │
          │  │                                   │
          │  ├───────────────────────────────────┤
          │  │...                                │
          └─ └───────────────────────────────────┘
```

--------------------------------

### ITensorData.backendType

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ITensorData.html

Gets the backend type on which the data elements are stored.

```APIDOC
## Property: backendType

### Description
Gets the backend type on which the data elements are stored.

### Declaration
```csharp
BackendType backendType { get; }
```

### Property Value
| Type | Description |
|------|-------------|
| BackendType | The backend type. |
```

--------------------------------

### Tensor Indexer Access

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/access-tensor-data-directly.html

Demonstrates how to use indexers to read and write tensor data directly when the tensor is on the CPU and readback is complete.

```APIDOC
## Tensor Indexer Access

### Description
Use indexers to directly read and write tensor data when the tensor is on the CPU and all dependent operations are complete.

### Method
Indexer access

### Syntax
```csharp
tensor[index0, index1, ...]
```

### Parameters
- **index0, index1, ...** (int) - Required - Indices for each dimension of the tensor.

### Usage Example
```csharp
var tensor = new Tensor<float>(new TensorShape(1, 2, 3));
//...
if (tensor.backendType == BackendType.CPU && tensor.IsReadbackRequestDone()) {
    // tensor is read-writable directly
    tensor[0, 1, 0] = 1f;
    tensor[0, 1, 1] = 2f;
    tensor[0, 1, 2] = 3f;
    float val = tensor[0, 0, 2];
}
```

### Notes
- The tensor must be on the CPU and all operations depending on it must be complete.
- Use `tensor.backendType == BackendType.CPU` and `tensor.IsReadbackRequestDone()` to check readiness.
```

--------------------------------

### maxCapacity

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ComputeTensorData.html

Gets the maximum count of the stored data elements.

```APIDOC
## maxCapacity

### Description
The maximum count of the stored data elements.

### Declaration
```csharp
public int maxCapacity { get; }
```

### Property Value
- **int** - The maximum capacity.

```

--------------------------------

### backendType

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ComputeTensorData.html

Gets the backend type on which the data elements are stored.

```APIDOC
## backendType

### Description
On what backend are the data elements stored.

### Declaration
```csharp
public BackendType backendType { get; }
```

### Property Value
- **BackendType** - The backend type.

```

--------------------------------

### Check tensor capacity and reshape with constraints

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/do-basic-tensor-operations.html

Shows how to check the number of elements using tensor.count and dataOnBackend.maxCapacity before reshaping. The example reshapes to a smaller shape and then to match maxCapacity, noting that the underlying dataOnBackend still contains 10 elements.

```csharp
var tensor = new Tensor<float>(new TensorShape(10));
Assert.AreEqual(10, tensor.count);
Assert.AreEqual(10, tensor.dataOnBackend.maxCapacity);

// Reshaping the tensor with a smaller shape

tensor.Reshape(new TensorShape(2, 3));
Assert.AreEqual(6, tensor.count);
Assert.AreEqual(10, tensor.dataOnBackend.maxCapacity);
// The underlying dataOnBackend still contains 10 elements

// reshape to match dataOnBackend.maxCapacity
tensor.Reshape(new TensorShape(1, 10));
```

--------------------------------

### Get<T>(int)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.NativeTensorArray.html

Returns the value of the backing data at a given index.

```APIDOC
## Get<T>(int)

### Description
Returns the value of the backing data at a given index.

### Method
Get (instance)

### Signature
```csharp
public T Get<T>(int index) where T : unmanaged
```

### Parameters
- **index** (int) - The index of the element.

### Returns
- **T** - The value of the element in the data.

### Type Parameters
- **T** - The type of the element.
```

--------------------------------

### rank

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.DynamicTensorShape.html

Gets the rank of the DynamicTensorShape. Cannot be called if the shape has a dynamic rank.

```APIDOC
## rank

### Description
The rank of a `DynamicTensorShape`. For example, a tensor of shape (5) has a rank of 1. A tensor of shape (7, 3, 5) has a rank of 3. This cannot be called if the shape has a dynamic rank. Call `isRankDynamic` first.

### Declaration
```csharp
public int rank { get; }
```

### Property Value
- **int** - The rank of the shape.

```

--------------------------------

### ReadbackAndClone: Download GPU tensor to CPU

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/use-model-output.html

Use ReadbackAndClone to move a tensor from the GPU to the CPU for reading. The returned tensor is a CPU-based, read-writable copy. For efficiency, consider using ReadbackAndCloneAsync and refer to the asynchronous read guidance.

```csharp
var outputTensor = worker.Schedule(inputTensor).PeekOutput() as Tensor<float>;
var cpuTensor = outputTensor.ReadbackAndClone();
```

--------------------------------

### isRankDynamic

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.DynamicTensorShape.html

Gets a value indicating whether the shape has a dynamic rank.

```APIDOC
## isRankDynamic

### Description
Whether the shape has a dynamic rank.

### Declaration
```csharp
public bool isRankDynamic { get; }
```

### Property Value
- **bool** - Whether the rank is dynamic.

```

--------------------------------

### this[int d0]

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Tensor-1.html

Gets or sets the tensor element at the specified axis 0 index.

```APIDOC
## this[int d0]

### Description
Returns the tensor element at offset `d0`.

### Declaration
```csharp
public T this[int d0] { get; set; }
```

### Parameters
- **d0** (int) - Axis 0.

### Property Value
- **T** - The tensor element at the specified index.
```

--------------------------------

### this[int d1, int d0]

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Tensor-1.html

Gets or sets the tensor element at the specified 2D indices.

```APIDOC
## this[int d1, int d0]

### Description
Returns the tensor element at offset `(d1, d0)`, which is position `d1 * stride0 + d0`.

### Declaration
```csharp
public T this[int d1, int d0] { get; set; }
```

### Parameters
- **d1** (int) - Axis 1.
- **d0** (int) - Axis 0.

### Property Value
- **T** - The tensor element at the specified indices.
```

--------------------------------

### ModelAsset

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.html

A binary representation of an Inference Engine asset.

```APIDOC
## ModelAsset

### Description
A binary representation of an Inference Engine asset.
```

--------------------------------

### Get(int axis)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.DynamicTensorShape.html

Returns the dimension at a given axis as an integer. Dynamic dimensions are represented as -1.

```APIDOC
## Get(int axis)

### Description
Returns the dimension at a given axis as an integer. If a dimension is dynamic it is represented with a -1.

### Declaration
```csharp
public int Get(int axis)
```

### Parameters
- **axis** (int) - The axis to get the dimension of.

### Returns
- **int** - The integer dimension of an axis.

```

--------------------------------

### NativeTensorArray Constructor (int, bool, Allocator)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.NativeTensorArray.html

Initializes and returns an instance of NativeTensorArray with a given length.

```APIDOC
## NativeTensorArray(int length, bool clearOnInit = false, Allocator allocator = Allocator.Persistent)

### Description
Initializes and returns an instance of `NativeTensorArray` with a given length.

### Declaration
```csharp
public NativeTensorArray(int length, bool clearOnInit = false, Allocator allocator = Allocator.Persistent)
```

### Parameters
- **length** (int) - Required - The integer number of elements to allocate.
- **clearOnInit** (bool) - Optional - Whether to zero the data after allocating.
- **allocator** (Allocator) - Optional - The allocation type to use as an `Allocator`.
```

--------------------------------

### Worker(Model, DeviceType)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Worker.html

Creates and initializes a Worker with a model on a backend type. The fastest backend type for the device type will be chosen.

```APIDOC
## Worker(Model, DeviceType)

### Description
Creates and initializes a Worker with a model on a backend type. The fastest backend type for the device type will be chosen.

### Declaration
public Worker(Model model, DeviceType deviceType)

### Parameters
- **model** (Model) - Required - The model for the worker.
- **deviceType** (DeviceType) - Required - The device type for the worker.

```

--------------------------------

### SetTensorLayout(TensorLayout)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.TextureTransform.html

Sets the layout of the input tensor with a TensorLayout object, for example TensorLayout.NHWC. The default is TensorLayout.NCHW.

```APIDOC
## SetTensorLayout(TensorLayout)

### Description
Sets the layout of the input tensor with a `TensorLayout` object, for example `TensorLayout.NHWC`. The default is `TensorLayout.NCHW`.

### Method
SetTensorLayout

### Signature
```
public TextureTransform SetTensorLayout(TensorLayout tensorLayout)
```

### Parameters
- **tensorLayout** (TensorLayout) - Required - Selected tensor layout type.

### Returns
- **TextureTransform** - `TextureTransform` that you can use to chain other methods.

### Exceptions
- **ArgumentOutOfRangeException** - Thrown if unknown `TensorLayout` is used.

```

--------------------------------

### this[int]

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.TensorShape.html

Gets or sets the tensor shape at a given axis. A negative axis counts backwards from the inner dimension.

```APIDOC
## this[int]

### Description
Gets or sets the tensor shape at a given axis. A negative axis counts backwards from the inner dimension.

### Declaration
```csharp
public int this[int axis] { get; set; }
```

### Parameters
- **axis** (int) - The axis to get or set.

### Property Value
- **int** - The value at the specified axis.
```

--------------------------------

### DownloadAsync<T>(int)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ComputeTensorData.html

Asynchronously downloads a contiguous block of data from internal storage.

```APIDOC
## DownloadAsync<T>(int)

### Description
Awaitable contiguous block of data from internal storage.

### Declaration
```csharp
public Awaitable<NativeArray<T>> DownloadAsync<T>(int dstCount) where T : unmanaged
```

### Parameters
- **dstCount** (int) - The number of elements to download.

### Returns
- **Awaitable<NativeArray<T>>** - A awaitable native array of downloaded elements.

### Type Parameters
- **T** - The data type of the elements.

```

--------------------------------

### this[int d2, int d1, int d0]

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Tensor-1.html

Gets or sets the tensor element at the specified 3D indices.

```APIDOC
## this[int d2, int d1, int d0]

### Description
Returns the tensor element at offset `(d2, d1, d0)`, which is position `d2 * stride1 + d1 * stride0 + d0`.

### Declaration
```csharp
public T this[int d2, int d1, int d0] { get; set; }
```

### Parameters
- **d2** (int) - Axis 2.
- **d1** (int) - Axis 1.
- **d0** (int) - Axis 0.

### Property Value
- **T** - The tensor element at the specified indices.
```

--------------------------------

### SetInput and Schedule with no arguments

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/create-an-input-tensor.html

Use this snippet when a model requires multiple input tensors. Call SetInput for each input, then call Schedule with no arguments.

```csharp
worker.SetInput("x", xTensor);
worker.SetInput("y", yTensor);
worker.Schedule();
```

--------------------------------

### PeekOutput

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/get-the-output.html

Gets a reference to the output tensor of the worker. Returns a Tensor object that usually needs to be cast to a specific type.

```APIDOC
## PeekOutput

### Description
Gets a reference to the output tensor of the worker. Returns a `Tensor` object that usually needs to be cast to a specific type.

### Method
PeekOutput

### Parameters
None

### Returns
- `Tensor` - A reference to the output tensor. Cast to `Tensor<float>` or `Tensor<int>` as needed.

### Remarks
- The worker's memory allocator owns the returned reference. You do not need to call `Dispose` on it.
- If you change the output or rerun the worker, both the worker output and the `PeekOutput` copy change.
- Calling `Dispose` on the worker disposes the `PeekOutput` copy.
- If you call `Schedule` again, the tensor is overwritten.

### Example
```csharp
worker.Schedule(inputTensor);
Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;
```
```

--------------------------------

### Create Worker with DeviceType.GPU

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.DeviceType.html

Shows how to specify the device when creating a Worker for model inference. Requires a ModelAsset field named model.

```csharp
public ModelAsset model;
Worker worker = new Worker(ModelLoader.Load(model), DeviceType.GPU);
```

--------------------------------

### Narrow(FunctionalTensor, int, FunctionalTensor, FunctionalTensor)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Functional.html

Narrows the input tensor along a dimension using functional tensors for start and length.

```APIDOC
## Narrow(FunctionalTensor, int, FunctionalTensor, FunctionalTensor)

### Description
Returns the input tensor narrowed along a dimension.

### Method
Narrow

### Parameters
- **input** (FunctionalTensor) - Required - The input tensor.
- **dim** (int) - Required - The dimension along which to narrow.
- **start** (FunctionalTensor) - Required - The functional start index along the dimension.
- **length** (FunctionalTensor) - Required - The functional number of elements along the dimension.

### Returns
- **FunctionalTensor** - The output tensor.

### Declaration
```csharp
public static FunctionalTensor Narrow(this FunctionalTensor input, int dim, FunctionalTensor start, FunctionalTensor length)
```
```

--------------------------------

### LinSpace(float, float, int)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Functional.html

Returns a 1D tensor of size steps with values evenly spaced from the interval [start, end].

```APIDOC
## LinSpace(float, float, int)

### Description
Returns a 1D tensor of size steps with values evenly spaced from the interval [start, end].

### Declaration
```csharp
public static FunctionalTensor LinSpace(float start, float end, int steps)
```

### Parameters
- **start** (float) - The value of the first element.
- **end** (float) - The value of the last element.
- **steps** (int) - The number of elements.

### Returns
- **FunctionalTensor** - The output tensor.
```

--------------------------------

### Save(string, Model)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ModelWriter.html

Serializes and saves the model description and weights to a file.

```APIDOC
## Save(string, Model)

### Description
Serializes and saves the model description and weights to a file.

### Method
Save

### Parameters
#### Parameters
- **fileName** (string) - Required - The path to save the model file to.
- **model** (Model) - Required - The model to save to file.

### Declaration
```
public static void Save(string fileName, Model model)
```
```

--------------------------------

### Get output tensor with PeekOutput

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/understand-inference-engine-workflow.html

Retrieves the output data from the model using the PeekOutput method, casting the result to Tensor<float>.

```csharp
Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;
```

--------------------------------

### Get model inputs

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/inspect-a-model.html

Use the inputs property of the runtime model to retrieve a list of all input tensors, including their names and shapes.

```APIDOC
## Get model inputs

### Description
Use the `inputs` property of the runtime model to retrieve a list of all input tensors, including their names and shapes.

### Property
`Model.inputs`

### Returns
- **List<Model.Input>** - A list of input tensors. Each `Model.Input` has:
  - **name** (string) - The name of the input tensor (e.g., "Input3").
  - **shape** (array) - The tensor shape (e.g., (1, 1, 28, 28)).

### Example
```csharp
using UnityEngine;
using System.Collections.Generic;
using Unity.InferenceEngine;

public class GetModelInputs : MonoBehaviour
{
    public ModelAsset modelAsset;

    void Start()
    {
        Model runtimeModel = ModelLoader.Load(modelAsset);

        List<Model.Input> inputs = runtimeModel.inputs;

        // Loop through each input
        foreach (var input in inputs)
        {
            // Log the name of the input, for example Input3
            Debug.Log(input.name);

            // Log the tensor shape of the input, for example (1, 1, 28, 28)
            Debug.Log(input.shape);
        }
    }
}
```
```

--------------------------------

### Load(string)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.ModelLoader.html

Loads a model that has been serialized to disk.

```APIDOC
## Load(string)

### Description
Loads a model that has been serialized to disk.

### Method
Load(string)

### Parameters
- **path** (string) - Required - The path of the binary serialized model.

### Returns
- **Model** - The loaded `Model`.
```

--------------------------------

### this[int d3, int d2, int d1, int d0]

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.Tensor-1.html

Gets or sets the tensor element at the specified 4D indices.

```APIDOC
## this[int d3, int d2, int d1, int d0]

### Description
Returns the tensor element at offset `(d3, d2, d1, d0)`, which is position `d3 * stride2 + d2 * stride1 + d1 * stride0 + d0` in this tensor.

### Declaration
```csharp
public T this[int d3, int d2, int d1, int d0] { get; set; }
```

### Parameters
- **d3** (int) - Axis 3.
- **d2** (int) - Axis 2.
- **d1** (int) - Axis 1.
- **d0** (int) - Axis 0.

### Property Value
- **T** - The tensor element at the specified indices.
```

--------------------------------

### Create input tensors from texture and array

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/manual/understand-inference-engine-workflow.html

Shows two ways to create tensors: converting a Texture2D to a Tensor<float> using TextureConverter.ToTensor, and creating a Tensor<int> directly from an int array with a TensorShape.

```csharp
// Convert a texture to a tensor
Texture2D inputTexture = Resources.Load("image-file") as Texture2D;
Tensor<float> inputTensor = new Tensor<float>(new TensorShape(1, 4, inputTexture.height, inputTexture.width));
TextureConverter.ToTensor(inputTexture, inputTensor);
// Convert an array to a tensor
int[] array = new int[] {1,2,3,4};
Tensor<int> inputTensor = new Tensor<int>(new TensorShape(4), array);
```

--------------------------------

### Length(int)

Source: https://docs.unity3d.com/Packages/com.unity.ai.inference%402.3/api/Unity.InferenceEngine.TensorShape.html

Returns the number of elements represented by the TensorShape, starting from a given axis. A negative axis counts backwards from the inner dimension.

```APIDOC
## Length(int)

### Description
Returns the number of elements represented by the TensorShape, starting from a given axis. A negative axis counts backwards from the inner dimension.

### Declaration
```csharp
public int Length(int start)
```

### Parameters
- **start** (int) - The first axis to count length from.

### Returns
- **int** - The number of elements in the shape.
```
Tokens:
Raw