### Example Command

Source: https://ai.google.dev/edge/litert-lm/android

Command to run the tool use example from the cloned repository.

```bash
bazel run -c opt //kotlin/java/com/google/ai/edge/litertlm/example:tool -- <abs_model_path>
```

--------------------------------

### Install LiteRT for Python

Source: https://ai.google.dev/edge/litert/microcontrollers/python

Install the tflite_runtime package using pip.

```bash
python3 -m pip install tflite-runtime

```

--------------------------------

### Initialize the Engine

Source: https://ai.google.dev/edge/litert-lm/android

Example of initializing the LiteRT-LM Engine with configuration, including model path and backend.

```kotlin
import com.google.ai.edge.litertlm.Backend
import com.google.ai.edge.litertlm.Engine
import com.google.ai.edge.litertlm.EngineConfig

val engineConfig = EngineConfig(
    modelPath = "/path/to/your/model.litertlm", // Replace with your model path
    backend = Backend.GPU(), // Or Backend.NPU(nativeLibraryDir = "...")
    // Optional: Pick a writable dir. This can improve 2nd load time.
    // cacheDir = "/tmp/" or context.cacheDir.path (for Android)
)

val engine = Engine(engineConfig)
engine.initialize()
// ... Use the engine to create a conversation ...

// Close the engine when done
engine.close()

```

--------------------------------

### Install necessary packages

Source: https://ai.google.dev/edge/litert/conversion/pytorch/overview.md

Install the litert-torch-nightly and torchvision packages.

```bash
pip install litert-torch-nightly torchvision
```

--------------------------------

### Install Model Explorer

Source: https://ai.google.dev/edge/litert/conversion/pytorch/overview.md

Installs the necessary package for visualizing TFLite models.

```bash
pip install ai-edge-model-explorer
```

--------------------------------

### Registering Tools

Source: https://ai.google.dev/edge/litert-lm/android

Example of registering tool instances in the ConversationConfig.

```kotlin
val conversation = engine.createConversation(
    ConversationConfig(
        tools = listOf(
            tool(SampleToolSet()),
            tool(SampleOpenApiTool()),
        ),
        // ... other configs
    )
)

// Send messages that might trigger the tool
conversation.sendMessageAsync("What's the weather like in London?", callback)

```

--------------------------------

### Install and import packages

Source: https://ai.google.dev/edge/litert/conversion/tensorflow/build/ondevice_training?hl=bn

This example code installs and imports necessary packages for the tutorial, including TensorFlow and Matplotlib.

```python
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf

print("TensorFlow version:", tf.__version__)

```

--------------------------------

### Setup

Source: https://ai.google.dev/edge/litert/conversion/tensorflow/quantization/quantization_debugger

Installs the necessary TensorFlow version and datasets library.

```bash
# Quantization debugger is available from TensorFlow 2.7.0
pip uninstall -y tensorflow
pip install tf-nightly
pip install tensorflow_datasets --upgrade  # imagenet_v2 needs latest checksum

```

--------------------------------

### NPU Backend Configuration

Source: https://ai.google.dev/edge/litert-lm/android

Example of configuring the NPU backend, specifying the directory for native libraries.

```kotlin
val engineConfig = EngineConfig(
    modelPath = modelPath,
    backend = Backend.NPU(nativeLibraryDir = context.applicationInfo.nativeLibraryDir)
)

```

--------------------------------

### Running the Sample

Source: https://ai.google.dev/edge/litert-lm/android

Command to run the sample terminal chat app using Bazel.

```bash
bazel run -c opt //kotlin/java/com/google/ai/edge/litertlm/example:main -- <abs_model_path>

```

--------------------------------

### Manual Model Testing with Fake Inputs

Source: https://ai.google.dev/edge/litert/web/get_started

Example code for generating fake inputs and running a model using `runWithTfjsTensors` for manual testing.

```javascript
// Imports, initialization, and model loading...
// Create fake inputs for the model
const fakeInputs = model.getInputDetails().map(
    ({shape, dtype}) => tf.ones(shape, dtype));

// Run the model
const outputs = await runWithTfjsTensors(model, fakeInputs);
console.log(outputs);

```

--------------------------------

### TensorFlow Version Output

Source: https://ai.google.dev/edge/litert/conversion/tensorflow/build/ondevice_training

Example output showing the installed TensorFlow version.

```text
TensorFlow version: 2.8.0

```

--------------------------------

### Defining Tools with Kotlin Functions

Source: https://ai.google.dev/edge/litert-lm/android

Example of defining custom Kotlin functions as tools with annotations for description and parameters.

```kotlin
import com.google.ai.edge.litertlm.Tool
import com.google.ai.edge.litertlm.ToolParam

class SampleToolSet: ToolSet {
    @Tool(description = "Get the current weather for a city")
    fun getCurrentWeather(
        @ToolParam(description = "The city name, e.g., San Francisco") city: String,
        @ToolParam(description = "Optional country code, e.g., US") country: String? = null,
        @ToolParam(description = "Temperature unit (celsius or fahrenheit). Default: celsius") unit: String = "celsius"
    ): Map<String, Any> {
        // In a real application, you would call a weather API here
        return mapOf("temperature" to 25, "unit" to  unit, "condition" to "Sunny")
    }

    @Tool(description = "Get the sum of a list of numbers.")
    fun sum(
        @ToolParam(description = "The numbers, could be floating point.") numbers: List<Double>,
    ): Double {
        return numbers.sum()
    }
}

```

--------------------------------

### Install Xcode

Source: https://ai.google.dev/edge/litert/build/ios.md

Installs Xcode and its command-line tools.

```bash
xcode-select --install

```

```bash
sudo xcodebuild -license accept

```

--------------------------------

### Install CMake tool

Source: https://ai.google.dev/edge/litert/build/cmake

Command to install CMake on Ubuntu.

```bash
sudo apt-get install cmake

```

--------------------------------

### Install the @litertjs/core package

Source: https://ai.google.dev/edge/litert/web

Install the @litertjs/core package from npm.

```bash
npm install @litertjs/core

```

--------------------------------

### Include the unit test framework header

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Includes the LiteRT for Microcontrollers unit test framework.

```c++
#include "tensorflow/lite/micro/testing/micro_test.h"

```

--------------------------------

### Import and Initialize LiteRT.js and TFJS Utilities

Source: https://ai.google.dev/edge/litert/web/get_started

This snippet shows how to import LiteRT.js, TFJS conversion utilities, and TensorFlow.js itself, along with initializing the WebGPU backend and LiteRT.js Wasm files. It also demonstrates how to make TFJS use the same GPU device as LiteRT.js for tensor conversion.

```javascript
import {loadLiteRt, getWebGpuDevice} from 'https://cdn.jsdelivr.net/npm/@litertjs/core@2.5.0/+esm';
import {runWithTfjsTensors} from 'https://cdn.jsdelivr.net/npm/@litertjs/tfjs-interop@2.5.0/+esm';

// TensorFlow.js imports
import * as tf from 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs/+esm';
import {WebGPUBackend} from 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgpu/+esm';

async function main() {
  // Initialize TensorFlow.js WebGPU backend
  await tf.setBackend('webgpu');

  // Initialize LiteRT.js's Wasm files
  await loadLiteRt('https://cdn.jsdelivr.net/npm/@litertjs/core/wasm/');

  // Make TFJS use the same GPU device as LiteRT.js (for tensor conversion)
  const device = getWebGpuDevice();
  tf.removeBackend('webgpu');
  tf.registerBackend('webgpu', () => new WebGPUBackend(device, device.adapterInfo));
  await tf.setBackend('webgpu');
  // ...
}

main();

```

--------------------------------

### Install litert-lm with uv

Source: https://ai.google.dev/edge/litert-lm/cli

Installs `litert-lm` as a system-wide binary.

```bash
uv tool install litert-lm

```

--------------------------------

### Set up logging

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Creates a tflite::ErrorReporter pointer using a pointer to a tflite::MicroErrorReporter instance.

```c++
tflite::MicroErrorReporter micro_error_reporter;
tflite::ErrorReporter* error_reporter = &micro_error_reporter;

```

--------------------------------

### Load Model with WebGPU Acceleration

Source: https://ai.google.dev/edge/litert/web/get_started

This snippet demonstrates how to load a converted LiteRT model targeting Generic WebGPU graphics evaluators for fast inference.

```javascript
import {loadLiteRt, loadAndCompile} from 'https://cdn.jsdelivr.net/npm/@litertjs/core/+esm';

await loadLiteRt('https://cdn.jsdelivr.net/npm/@litertjs/core/wasm/');

const model = await loadAndCompile('path_to_model.tflite', {
  accelerator: 'webgpu',
});

```

--------------------------------

### Example config.toml

Source: https://ai.google.dev/edge/litert-lm/file_builder

Example TOML configuration file for building a .litertlm file.

```toml
[system_metadata]
entries = [
  { key = "author", value_type = "String", value = "Authors" }
]

[[section]]
section_type = "LlmMetadata"
data_path = "path/to/llm_metadata.pb"

[[section]]
section_type = "SP_Tokenizer"
data_path = "path/to/sp.model"

[[section]]
section_type = "TFLiteModel"
model_type = "PREFILL_DECODE"
data_path = "path/to/model.tflite"
additional_metadata = [
  { key = "model_version", value_type = "String", value = "1.0.1" }
]


```

--------------------------------

### Train the Hello World model

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Builds and trains the Hello World model for sinwave recognition, saving the TensorFlow model to a specified directory.

```bash
bazel build tensorflow/lite/micro/examples/hello_world:train
bazel-bin/tensorflow/lite/micro/examples/hello_world/train --save_tf_model --save_dir=/tmp/model_created/
```

--------------------------------

### Java Example

Source: https://ai.google.dev/edge/litert/android/gpu

Example of how to initialize the TensorFlow Lite interpreter with the GpuDelegate in Java.

```java
import org.tensorflow.lite.Interpreter;
import org.tensorflow.lite.gpu.CompatibilityList;
import org.tensorflow.lite.gpu.GpuDelegate;

// Initialize interpreter with GPU delegate
Interpreter.Options options = new Interpreter.Options();
CompatibilityList compatList = CompatibilityList();

if(compatList.isDelegateSupportedOnThisDevice){
    // if the device has a supported GPU, add the GPU delegate
    GpuDelegate.Options delegateOptions = compatList.getBestOptionsForThisDevice();
    GpuDelegate gpuDelegate = new GpuDelegate(delegateOptions);
    options.addDelegate(gpuDelegate);
} else {
    // if the GPU is not supported, run on 4 threads
    options.setNumThreads(4);
}

Interpreter interpreter = new Interpreter(model, options);

// Run inference
writeToInput(input);
interpreter.run(input, output);
readFromOutput(output);

```

--------------------------------

### Install LiteRT-LM

Source: https://ai.google.dev/edge/litert-lm/js

Instructions for installing the LiteRT-LM core package from npm or importing it from a CDN.

```bash
# From npm
npm i --save @litert-lm/core

# From a CDN (in your JavaScript file)
import * as litertlm from 'https://cdn.jsdelivr.net/npm/@litert-lm/core/+esm';


```

--------------------------------

### Multi-Modality Example

Source: https://ai.google.dev/edge/litert-lm/python

Example demonstrating multi-modal input with audio and vision backends.

```python
# Initialize with vision and/or audio backends if needed
with litert_lm.Engine(
    "path/to/multimodal_model.litertlm",
    audio_backend=litert_lm.Backend.CPU(),
    vision_backend=litert_lm.Backend.GPU(),
) as engine:
    with engine.create_conversation() as conversation:
        response = conversation.send_message(
            litert_lm.Contents.of(
                "Describe this audio.",
                litert_lm.Content.AudioFile(absolute_path="/path/to/audio.wav"),
            )
        )
        print(response["content"][0]["text"])

```

--------------------------------

### Build the Hello World application binary

Source: https://ai.google.dev/edge/litert/microcontrollers/library

Example command to build the binary for the Hello World application.

```bash
make -f tensorflow/lite/micro/tools/make/Makefile hello_world_bin

```

--------------------------------

### End-to-End Pipeline Example for ResNet18

Source: https://ai.google.dev/edge/litert/web/get_started

An example of a pre- and post-processing pipeline for ResNet18 using TensorFlow.js, including image normalization and model execution.

```javascript
// Wrap in a tf.tidy call to automatically clean up intermediate TensorFlow.js tensors.
// (Note: tidy only supports synchronous functions).
const imageData = tf.tidy(() => {
  // Get RGB data values from an image element and convert it to range [0, 1).
  const image = tf.browser.fromPixels(dogs, 3).div(255);

  // These preprocessing steps come from https://github.com/pytorch/vision/blob/main/torchvision/models/resnet.py#L315
  // The mean and standard deviation for the image normalization come from https://github.com/pytorch/vision/blob/main/torchvision/transforms/_presets.py#L38
  return image.resizeBilinear([224, 224])
    .sub([0.485, 0.456, 0.406])
    .div([0.229, 0.224, 0.225])
    .reshape([1, 224, 224, 3])
    .transpose([0, 3, 1, 2]);
});

// Run the model
const outputs = await runWithTfjsTensors(model, [imageData]);
const probabilities = outputs[0];

// Get the top five classes.
const top5 = tf.topk(probabilities, 5);

const values = await top5.values.data();
const indices = await top5.indices.data();

// Clean up TFJS tensors
tf.dispose(outputs);
tf.dispose(top5);
tf.dispose(imageData);

// Print the top five classes.
const classes = ... // Class names are loaded from a JSON file in the demo.
for (let i = 0; i < 5; ++i) {
  const text = `${classes[indices[i]]}: ${values[i]}`;
  console.log(text);
}

```

--------------------------------

### Install LiteRT.js NPM packages

Source: https://ai.google.dev/edge/litert/web

Install the necessary LiteRT.js NPM packages for integration with TensorFlow.js.

```bash
npm install @litertjs/core @litertjs/tfjs-interop
```

--------------------------------

### Instantiate Interpreter using tf

Source: https://ai.google.dev/edge/litert/microcontrollers/python

Original instantiation of the Interpreter using the tensorflow module.

```python
interpreter = tf.lite.Interpreter(model_path=args.model_file)

```

--------------------------------

### Defining Tools with OpenAPI Specification

Source: https://ai.google.dev/edge/litert-lm/android

Example of defining a tool by implementing the OpenApiTool class and providing a JSON string conforming to the OpenAPI specification.

```kotlin
import com.google.ai.edge.litertlm.OpenApiTool

class SampleOpenApiTool : OpenApiTool {

    override fun getToolDescriptionJsonString(): String {
        return """
        {
          "name": "addition",
          "description": "Add all numbers.",
          "parameters": {
            "type": "object",
            "properties": {
              "numbers": {
                "type": "array",
                "items": {
                  "type": "number"
                }
              },
              "description": "The list of numbers to sum."
            },
            "required": [
              "numbers"
            ]
          }
        }
        """.trimIndent() // Tip: trim to save tokens
    }

    override fun execute(paramsJsonString: String): String {
        // Parse paramsJsonString with your choice of parser or deserializer and
        // execute the tool.

        // Return the result as a JSON string
        return """{"result": 1.4142}"""
    }
}

```

--------------------------------

### Import Interpreter from tensorflow

Source: https://ai.google.dev/edge/litert/microcontrollers/python

Original import statement in label_image.py.

```python
import tensorflow as tf

```

--------------------------------

### Build Hello World for a generic cortex-m0 target

Source: https://ai.google.dev/edge/litert/microcontrollers/library

Command to build the Hello World example for a generic cortex-m0 target, specifying TARGET and TARGET_ARCH.

```bash
make -f tensorflow/lite/micro/tools/make/Makefile TARGET=cortex_m_generic TARGET_ARCH=cortex-m0 hello_world_bin

```

--------------------------------

### Install litert-lm with pip

Source: https://ai.google.dev/edge/litert-lm/cli

Standard installation within a virtual environment. Using `--upgrade` ensures you get the latest version even if a previous version was already installed.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade litert-lm

```

--------------------------------

### Install required packages

Source: https://ai.google.dev/edge/litert/libraries/modify/image_classification

Installs the necessary packages for the example, including the tflite-model-maker library.

```bash
sudo apt -y install libportaudio2
pip install -q tflite-model-maker
```

--------------------------------

### Import packages

Source: https://ai.google.dev/edge/litert/conversion/pytorch/overview.md

Import the required libraries for PyTorch conversion.

```python
import litert_torch
import numpy
import torch
import torchvision

```

--------------------------------

### Setup imports

Source: https://ai.google.dev/edge/litert/conversion/tensorflow/quantization/post_training_integer_quant_16x8

Imports necessary libraries for TensorFlow and model building.

```python
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

```

```python
import logging
logging.getLogger("tensorflow").setLevel(logging.DEBUG)

import tensorflow as tf
from tensorflow import keras
import numpy as np
import pathlib

```

--------------------------------

### Install the required packages

Source: https://ai.google.dev/edge/litert/libraries/modify/text_classification

To run this example, install the required packages, including the Model Maker package from the GitHub repo.

```bash
sudo apt -y install libportaudio2
pip install -q tflite-model-maker
pip uninstall -y tflite_support_nightly
pip install tflite_support_nightly
```

--------------------------------

### Instantiate the PyTorch model

Source: https://ai.google.dev/edge/litert/conversion/pytorch/overview.md

Instantiate a ResNet18 model from torchvision and prepare sample inputs.

```python
resnet18 = torchvision.models.resnet18(torchvision.models.ResNet18_Weights.IMAGENET1K_V1).eval()
sample_inputs = (torch.randn(1, 3, 224, 224),)
torch_output = resnet18(*sample_inputs)

```

--------------------------------

### Instantiate Interpreter using tflite

Source: https://ai.google.dev/edge/litert/microcontrollers/python

Modified instantiation of the Interpreter using the tflite_runtime module.

```python
interpreter = tflite.Interpreter(model_path=args.model_file)

```

--------------------------------

### Create a Conversation

Source: https://ai.google.dev/edge/litert-lm/android

Creates a Conversation instance, optionally with custom configuration.

```kotlin
import com.google.ai.edge.litertlm.ConversationConfig
import com.google.ai.edge.litertlm.Message
import com.google.ai.edge.litertlm.SamplerConfig

// Optional: Configure the system instruction, initial messages, sampling
// parameters, etc.
val conversationConfig = ConversationConfig(
    systemInstruction = Contents.of("You are a helpful assistant."),
    initialMessages = listOf(
        Message.user("What is the capital city of the United States?"),
        Message.model("Washington, D.C."),
    ),
    samplerConfig = SamplerConfig(topK = 10, topP = 0.95, temperature = 0.8),
)

val conversation = engine.createConversation(conversationConfig)
// Or with default config:
// val conversation = engine.createConversation()

// ... Use the conversation ...

// Close the conversation when done
conversation.close()
```

```kotlin
engine.createConversation(conversationConfig).use { conversation ->
    // Interact with the conversation
}
```

--------------------------------

### Build a binary for a project

Source: https://ai.google.dev/edge/litert/microcontrollers/library

Command to build a runnable binary for a given project. Replace `<project_name>` with the desired project.

```bash
make -f tensorflow/lite/micro/tools/make/Makefile <project_name>_bin

```

--------------------------------

### Synchronous Send Message

Source: https://ai.google.dev/edge/litert-lm/android

Sends a message synchronously and waits for a complete response.

```kotlin
import com.google.ai.edge.litertlm.Content
import com.google.ai.edge.litertlm.Message

print(conversation.sendMessage("What is the capital of France?"))
```

--------------------------------

### Run all unit tests

Source: https://ai.google.dev/edge/litert/microcontrollers/library

Command to build the library and run all of its unit tests.

```bash
make -f tensorflow/lite/micro/tools/make/Makefile test

```

--------------------------------

### Android Manifest for GPU Backend

Source: https://ai.google.dev/edge/litert-lm/android

Required additions to AndroidManifest.xml to use the GPU backend.

```xml
  <application>
    <uses-native-library android:name="libvndksupport.so" android:required="false"/>
    <uses-native-library android:name="libOpenCL.so" android:required="false"/>
  </application>

```

--------------------------------

### Visualize TFLite Model

Source: https://ai.google.dev/edge/litert/conversion/pytorch/overview.md

Uses the Model Explorer to visualize a TFLite model file.

```python
import model_explorer
model_explorer.visualize('resnet.tflite')
```

--------------------------------

### Gradle Dependencies

Source: https://ai.google.dev/edge/litert-lm/android

Dependencies for integrating LiteRT-LM into Android or JVM projects using Gradle.

```gradle
dependencies {
    // For Android
    implementation("com.google.ai.edge.litertlm:litertlm-android:latest.release")

    // For JVM (Linux, macOS, Windows)
    implementation("com.google.ai.edge.litertlm:litertlm-jvm:latest.release")
}

```

--------------------------------

### Instantiate interpreter

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Creates a tflite::MicroInterpreter instance with the necessary components.

```c++
tflite::MicroInterpreter interpreter(model, resolver, tensor_arena,
                                     tensor_arena_size, error_reporter);

```

--------------------------------

### Instantiate operations resolver

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Declares a MicroMutableOpResolver instance and registers operations.

```c++
using HelloWorldOpResolver = tflite::MicroMutableOpResolver<1>;

TfLiteStatus RegisterOps(HelloWorldOpResolver& op_resolver) {
  TF_LITE_ENSURE_STATUS(op_resolver.AddFullyConnected());
  return kTfLiteOk;
}

HelloWorldOpResolver op_resolver;
TF_LITE_ENSURE_STATUS(RegisterOps(op_resolver));

```

--------------------------------

### Quantization Configuration Example

Source: https://ai.google.dev/edge/litert/conversion/pytorch/genai

Example command to export a model with a custom quantization recipe JSON file.

```bash
litert-torch export_hf \
  --model=google/gemma-3-270m-it \
  --output_dir=/tmp/gemma3-270m-it-litertlm \
  --quantization_recipe=/path/to/my/quantization_recipe.json

```

--------------------------------

### Disable Automatic Tool Calling

Source: https://ai.google.dev/edge/litert-lm/android

To manually execute tools, set `automaticToolCalling` in `ConversationConfig` to `false`.

```kotlin
val conversation = engine.createConversation(
    ConversationConfig(
        tools = listOf(
            tool(SampleOpenApiTool()),
        ),
        automaticToolCalling = false,
    )
)
```

--------------------------------

### Enable Multi-Token Prediction (MTP)

Source: https://ai.google.dev/edge/litert-lm/android

Enables speculative decoding for MTP optimization before initializing the engine.

```kotlin
import com.google.ai.edge.litertlm.ExperimentalApi
import com.google.ai.edge.litertlm.ExperimentalFlags
import com.google.ai.edge.litertlm.Backend
import com.google.ai.edge.litertlm.Engine
import com.google.ai.edge.litertlm.EngineConfig

// Enable MTP via speculative decoding
@OptIn(ExperimentalApi::class)
ExperimentalFlags.enableSpeculativeDecoding = true

val engineConfig = EngineConfig(
    modelPath = "/path/to/your/model.litertlm",
    backend = Backend.GPU(),
)

val engine = Engine(engineConfig)
engine.initialize()

// The same steps to create Conversation and send messages as below...
```

--------------------------------

### Asynchronous Send Message with Callback

Source: https://ai.google.dev/edge/litert-lm/android

Sends a message asynchronously and handles responses via a callback.

```kotlin
import com.google.ai.edge.litertlm.Content
import com.google.ai.edge.litertlm.Message
import com.google.ai.edge.litertlm.MessageCallback
import java.util.concurrent.CountDownLatch
import java.util.concurrent.TimeUnit

val callback = object : MessageCallback {
    override fun onMessage(message: Message) {
        print(message)
    }

    override fun onDone() {
        // Streaming completed
    }

    override fun onError(throwable: Throwable) {
        // Error during streaming
    }
}

conversation.sendMessageAsync("What is the capital of France?", callback)
```

--------------------------------

### Configure WORKSPACE and .bazelrc

Source: https://ai.google.dev/edge/litert/android/lite_build

Example entries in the .tf_configure.bazelrc file after successful configuration.

```bash
build --action_env ANDROID_NDK_HOME="/usr/local/android/android-ndk-r25b"
build --action_env ANDROID_NDK_API_LEVEL="21"
build --action_env ANDROID_BUILD_TOOLS_VERSION="30.0.3"
build --action_env ANDROID_SDK_API_LEVEL="30"
build --action_env ANDROID_SDK_HOME="/usr/local/android/android-sdk-linux"

```

--------------------------------

### List model file sizes

Source: https://ai.google.dev/edge/litert/conversion/tensorflow/quantization/post_training_integer_quant_16x8

Check the file sizes of the converted models.

```bash
ls -lh {tflite_models_dir}
```

--------------------------------

### Define the test structure

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Defines the structure for a unit test using macros.

```c++
TF_LITE_MICRO_TESTS_BEGIN

TF_LITE_MICRO_TEST(LoadModelAndPerformInference) {
  . // add code here
  .
}

TF_LITE_MICRO_TESTS_END

```

--------------------------------

### Import Interpreter from tflite_runtime

Source: https://ai.google.dev/edge/litert/microcontrollers/python

Modified import statement to use tflite_runtime.

```python
import tflite_runtime.interpreter as tflite

```

--------------------------------

### Multi-Modality Message

Source: https://ai.google.dev/edge/litert-lm/android

Sends a message containing multiple content types like text, images, and audio.

```kotlin
// Initialize the `visionBackend`, `audioBackend`, or both
val engineConfig = EngineConfig(
    modelPath = "/path/to/your/model.litertlm", // Replace with your model path
    backend = Backend.CPU(), // Or Backend.GPU() or Backend.NPU(...)
    visionBackend = Backend.GPU(), // Or Backend.NPU(...)
    audioBackend = Backend.CPU(), // Or Backend.NPU(...)
)

// Sends a message with multi-modality.
// See the Content class for other variants.
conversation.sendMessage(Contents.of(
    Content.ImageFile("/path/to/image"),
    Content.AudioBytes(audioBytes), // ByteArray of the audio
    Content.Text("Describe this image and audio."),
))
```

--------------------------------

### Asynchronous Send Message with Flow

Source: https://ai.google.dev/edge/litert-lm/android

Sends a message asynchronously and returns a Kotlin Flow for streaming responses.

```kotlin
import com.google.ai.edge.litertlm.Content
import com.google.ai.edge.litertlm.Message
import kotlinx.coroutines.flow.catch
import kotlinx.coroutines.launch

// Within a coroutine scope
conversation.sendMessageAsync("What is the capital of France?")
    .catch { ... } // Error during streaming
    .collect { print(it.toString()) }
```

--------------------------------

### Minimal TFLite CMakeLists.txt example

Source: https://ai.google.dev/edge/litert/build/cmake

Example CMakeLists.txt for a minimal project that uses LiteRT, demonstrating how to add the subdirectory and link the library.

```cmake
cmake_minimum_required(VERSION 3.16)
project(minimal C CXX)

set(TENSORFLOW_SOURCE_DIR "" CACHE PATH
  "Directory that contains the TensorFlow project"
 )
if(NOT TENSORFLOW_SOURCE_DIR)
  get_filename_component(TENSORFLOW_SOURCE_DIR
    "${CMAKE_CURRENT_LIST_DIR}/../../../..//" ABSOLUTE)
endif()

add_subdirectory(
  "${TENSORFLOW_SOURCE_DIR}/tensorflow/lite"
  "${CMAKE_CURRENT_BINARY_DIR}/tensorflow-lite" EXCLUDE_FROM_ALL)

add_executable(minimal minimal.cc)
target_link_libraries(minimal tensorflow-lite)

```

--------------------------------

### Sample Terminal Chat App

Source: https://ai.google.dev/edge/litert-lm/android

A sample terminal chat app built with the Kotlin API for LiteRT-LM.

```kotlin
import com.google.ai.edge.litertlm.*

suspend fun main() {
  Engine.setNativeMinLogSeverity(LogSeverity.ERROR) // Hide log for TUI app

  val engineConfig = EngineConfig(modelPath = "/path/to/model.litertlm")
  Engine(engineConfig).use { engine ->
    engine.initialize()

    engine.createConversation().use { conversation ->
      while (true) {
        print("\n>>> ")
        conversation.sendMessageAsync(readln()).collect { print(it) }
      }
    }
  }
}

```

--------------------------------

### Inference with LiteRT model

Source: https://ai.google.dev/edge/litert/conversion/pytorch/overview.md

Perform inference using the converted LiteRT model by calling it directly with the sample inputs.

```python
edge_output = edge_model(*sample_inputs)

```

--------------------------------

### Assert successful invocation

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

This code asserts that the invoke_status is kTfLiteOk, indicating successful inference.

```cpp
TF_LITE_MICRO_EXPECT_EQ(kTfLiteOk, invoke_status);
```

--------------------------------

### Launch unit tests using delegates

Source: https://ai.google.dev/edge/litert/build/cmake

Examples of launching unit tests with delegates using ctest, requiring CTestTestfile.cmake and run-tests.cmake.

```bash
cmake -E env TESTS_ARGUMENTS=--use_xnnpack=true ctest -L delegate
```

```bash
cmake -E env TESTS_ARGUMENTS=--external_delegate_path=<PATH> ctest -L delegate
```

--------------------------------

### Run CMake tool with configurations - Build installable package

Source: https://ai.google.dev/edge/litert/build/cmake

Command to run CMake to build an installable package.

```bash
cmake ../tensorflow_src/tensorflow/lite -DTFLITE_ENABLE_INSTALL=ON \
  -DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON \
  -DSYSTEM_FARMHASH=ON \
  -DSYSTEM_PTHREADPOOL=ON \
  -Dabsl_DIR=<install path>/lib/cmake/absl \
  -DEigen3_DIR=<install path>/share/eigen3/cmake \
  -DFlatBuffers_DIR=<install path>/lib/cmake/flatbuffers \
  -Dgemmlowp_DIR=<install path>/lib/cmake/gemmlowp \
  -DNEON_2_SSE_DIR=<install path>/lib/cmake/NEON_2_SSE \
  -Dcpuinfo_DIR=<install path>/share/cpuinfo \
  -Druy_DIR=<install path>/lib/cmake/ruy

```

--------------------------------

### Provide an input value

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Sets the contents of the input tensor with a floating point value.

```c++
input->data.f[0] = 0.;

```

--------------------------------

### Allocate tensors

Source: https://ai.google.dev/edge/litert/microcontrollers/get_started

Tells the interpreter to allocate memory from the tensor_arena for the model's tensors.

```c++
interpreter.AllocateTensors();

```

--------------------------------

### Get the data path

Source: https://ai.google.dev/edge/litert/libraries/modify/image_classification

Downloads and extracts a dataset of flower images for the image classification example.

```python
image_path = tf.keras.utils.get_file(
      'flower_photos.tgz',
      'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
      extract=True)
image_path = os.path.join(os.path.dirname(image_path), 'flower_photos')
```

--------------------------------

### Whole model verify mode setup

Source: https://ai.google.dev/edge/litert/conversion/tensorflow/quantization/quantization_debugger

Setting up the converter for whole model verification.

```Python
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.representative_dataset = representative_dataset(ds)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter._experimental_calibrate_only = True
calibrated_model = converter.convert()

```

--------------------------------

### CocoaPods - Specifying versions (Swift)

Source: https://ai.google.dev/edge/litert/ios/quickstart

Specify a version constraint for the TensorFlowLiteSwift pod, for example, version 2.10.0.

```ruby
pod 'TensorFlowLiteSwift', '~> 2.10.0'

```