### Install iOS Pods Source: https://docs.zetic.ai/platform-integration/flutter/setup After configuring the `ios/Podfile`, navigate to the `ios` directory and run `pod install`. ```bash cd ios pod install ``` -------------------------------- ### Initialize ZeticMLangeLLMModel Example Source: https://docs.zetic.ai/api-reference/android/ZeticMLangeLLMModel Example of initializing ZeticMLangeLLMModel with a specific context, personal key, model name, and initialization options. ```kotlin val model = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = "account_name/project_name", initOption = LLMInitOption(nCtx = 4096), ) ``` -------------------------------- ### Install and Use onnxsim Source: https://docs.zetic.ai/how-to-guides/performance-optimization Install the onnxsim tool and use it to simplify ONNX models before uploading to reduce redundant operations. ```bash pip install onnxsim onnxsim input_model.onnx output_model.onnx ``` -------------------------------- ### Verify Zetic Melange Setup Source: https://docs.zetic.ai/platform-integration/flutter/setup Import the package and create a `ZeticMLangeModel` instance to verify the setup. Model creation downloads the optimized model artifact on first use. ```dart import 'package:zetic_mlange/zetic_mlange.dart'; final model = await ZeticMLangeModel.create( personalKey: personalKey, name: 'account_name/project_name', onProgress: (progress) { print('Loading ${(progress * 100).round()}%'); }, ); ``` -------------------------------- ### Install skl2onnx Source: https://docs.zetic.ai/model-preparation/onnx Install the skl2onnx package to enable conversion of scikit-learn models to ONNX. ```bash pip install skl2onnx ``` -------------------------------- ### ZeticMLangeLLMModel.run Source: https://docs.zetic.ai/api-reference/flutter/ZeticMLangeLLMModel Starts text generation for a given prompt using the loaded LLM. ```APIDOC ## ZeticMLangeLLMModel.run ### Description Starts text generation for a given prompt using the loaded LLM. ### Method Signature ```dart LLMRunResult run(String text) ``` ### Parameters #### Path Parameters * `text` (String) - Required - The prompt to generate text from. ### Request Example ```dart final result = model.run('Explain on-device AI in one paragraph.'); ``` ``` -------------------------------- ### Full Working Example: iOS ViewController for Inference Source: https://docs.zetic.ai/platform-integration/ios/basic-inference A complete example integrated within an iOS ViewController's viewDidLoad method. It includes error handling for the inference process. Remember to prepare your input tensors correctly. ```swift import ZeticMLange class ViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() do { // Load model let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "Steve/YOLOv11_comparison") // Prepare inputs let inputs: [Tensor] = [] // Prepare your inputs // Run inference let outputs = try model.run(inputs: inputs) // Process outputs for output in outputs { // Process each output tensor } } catch { print("Melange error: \(error)") } } } ``` -------------------------------- ### Repository Structure Example Source: https://docs.zetic.ai/model-deployment/understanding-model-keys Illustrates the hierarchical structure of a model repository, including versions and default/latest tags. ```text username/my-model-repository ├── v1 (uploaded 2024-01-15) ├── v2 (uploaded 2024-02-20) └── v3 (uploaded 2024-03-10) ``` -------------------------------- ### Full Working Example: Running a Model with Tensors Source: https://docs.zetic.ai/api-reference/android/Tensor Demonstrates how to create a ZeticMLangeModel, run inference with input tensors, and read typed output arrays. ```kotlin import com.zeticai.mlange.core.model.ZeticMLangeModel import com.zeticai.mlange.core.tensor.Tensor class MainActivity : AppCompatActivity() { private lateinit var model: ZeticMLangeModel override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) model = ZeticMLangeModel(this, PERSONAL_KEY, "Steve/YOLOv11_comparison") } private fun runFrame(input: Tensor) { // (1) Run inference with tensors matching the model input order val outputs: Array = model.run(arrayOf(input)) // (2) Read outputs as typed arrays val logits: FloatArray = outputs[0].data() } } ``` -------------------------------- ### Fetch Packages Source: https://docs.zetic.ai/platform-integration/flutter/setup After adding the dependency, run `flutter pub get` to fetch the packages. ```bash flutter pub get ``` -------------------------------- ### Install tf2onnx Source: https://docs.zetic.ai/model-preparation/onnx Install the tf2onnx package to enable conversion of TensorFlow and Keras models to ONNX. ```bash pip install tf2onnx ``` -------------------------------- ### Text Generation - run(_:) Source: https://docs.zetic.ai/api-reference/ios/ZeticMLangeLLMModel Starts text generation for a given prompt. This method is synchronous and can throw errors. ```APIDOC ## Text Generation - `run(_:)` ### Description Starts generation for a prompt. ### Method Signature ```swift public func run(_ text: String) throws -> LLMRunResult ``` ### Parameters - **text** (`String`) - Required - The prompt to generate text from. ### Response - **LLMRunResult** - The result of the text generation. ### Request Example ```swift let result = try model.run("Explain on-device AI in one paragraph.") ``` ``` -------------------------------- ### Run Text Generation Source: https://docs.zetic.ai/api-reference/ios/ZeticMLangeLLMModel Starts text generation for a given prompt using the initialized LLM model. ```swift let result = try model.run("Explain on-device AI in one paragraph.") ``` -------------------------------- ### Face Detection Pipeline (Kotlin) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize and run the face detection model. Preprocess the image, run the model, and postprocess to get face regions. ```kotlin // (0) Initialize face detection model val faceDetectionModel = ZeticMLangeModel(this, PERSONAL_KEY, "google/MediaPipe-Face-Detection") val faceDetection = FaceDetectionWrapper() // (1) Preprocess image val faceDetectionInputs = faceDetection.preprocess(bitmap) // (2) Run face detection model val faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs) // (3) Postprocess to get face regions val faceDetectionPostprocessed = faceDetection.postprocess(faceDetectionOutputs) ``` -------------------------------- ### Full Working Example: iOS Model Inference Source: https://docs.zetic.ai/api-reference/ios/Tensor Demonstrates how to load a ZeticMLange model, prepare input data as a Tensor, run inference, and process the output Tensors in an iOS application. ```swift import ZeticMLange class ViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() do { let model = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: "Steve/YOLOv11_comparison" ) // (1) Build an input tensor from preprocessed pixel bytes let pixels: [Float] = preparePixels() let data = pixels.withUnsafeBufferPointer { Data(buffer: $0) } let input = Tensor( data: data, dataType: BuiltinDataType.float32, shape: [1, 3, 640, 640] ) // (2) Run inference let outputs = try model.run(inputs: [input]) // (3) Read outputs as a typed array of Floats let output = outputs[0] let floatCount = output.count() let floats: [Float] = output.data.withUnsafeBytes { Array($0.bindMemory(to: Float.self).prefix(floatCount)) } } catch { print("Melange error: \(error)") } } } ``` -------------------------------- ### Face Detection Pipeline (Swift) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize and run the face detection model. Preprocess the image, run the model, and postprocess to get face regions. ```swift // (0) Initialize face detection model let faceDetectionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "google/MediaPipe-Face-Detection") let faceDetection = FaceDetectionWrapper() // (1) Preprocess image let faceDetectionInputs = faceDetection.preprocess(bitmap) // (2) Run face detection model let faceDetectionOutputs = try faceDetectionModel.run(inputs: faceDetectionInputs) // (3) Postprocess to get face regions let faceDetectionPostprocessed = faceDetection.postprocess(faceDetectionOutputs) ``` -------------------------------- ### Verify ZeticMLange Initialization Source: https://docs.zetic.ai/platform-integration/ios/setup Add this test to confirm the SDK is installed and configured correctly. The initializer may perform a network call on first use to download the model binary. ```swift import ZeticMLange // Test initialization let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) ``` -------------------------------- ### Text Generation - run Source: https://docs.zetic.ai/api-reference/android/ZeticMLangeLLMModel Starts text generation for a given prompt. This method initiates the LLM inference process. ```APIDOC ## Text Generation ### `run(text)` Starts generation for a prompt. ```java fun run(text: String): LLMRunResult ``` ### Parameters - **text** (`String`) - Required - The prompt text to generate from. ### Returns - `LLMRunResult` - The result of the text generation. ### Example ```java val result = model.run("Explain on-device AI in one paragraph.") ``` ``` -------------------------------- ### iOS Swift Inference Mode Examples Source: https://docs.zetic.ai/how-to-guides/inference-mode-selection Initialize ZeticMLangeModel instances using different inference modes: .RUN_AUTO (default), .RUN_SPEED, and .RUN_ACCURACY. ```swift // Default: Auto mode // Speed first, but maintains SNR above 20dB let modelDefault = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_AUTO ) // Speed First Mode let modelFast = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_SPEED ) // Accuracy First Mode let modelAccurate = try ZeticMLangeModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_ACCURACY ) ``` -------------------------------- ### Install ZeticMLange Android SDK Source: https://docs.zetic.ai/release-notes/changelog Use this implementation for Android projects to integrate the ZeticMLange SDK version 1.9.0. ```gradle implementation("com.zeticai.mlange:mlange:1.9.0") ``` -------------------------------- ### Face Emotion Recognition Pipeline (Kotlin) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize and run the face emotion recognition model. Preprocess with detected face regions, run the model, and postprocess to get emotions. ```kotlin // (0) Initialize face emotion recognition model val faceEmotionRecognitionModel = ZeticMLangeModel(this, PERSONAL_KEY, "ElenaRyumina/FaceEmotionRecognition") val faceEmotionRecognition = FaceEmotionRecognitionWrapper() // (1) Preprocess with detected face regions val faceEmotionRecognitionInputs = faceEmotionRecognition.preprocess(bitmap, faceDetectionPostprocessed) // (2) Run face emotion recognition model val faceEmotionRecognitionOutputs = faceEmotionRecognitionModel.run(faceEmotionRecognitionInputs) // (3) Postprocess to get emotions val faceEmotionRecognitionPostprocessed = faceEmotionRecognition.postprocess(faceEmotionRecognitionOutputs) ``` -------------------------------- ### Android: Face Landmark Model Initialization and Run Source: https://docs.zetic.ai/how-to-guides/multi-model-pipeline Initializes the face landmark model, preprocesses the image using detected face regions, runs inference, and postprocesses the output to get landmarks. This step follows face detection in a multi-model pipeline. ```kotlin // Initialize face landmark model val faceLandmarkModel = ZeticMLangeModel(this, PERSONAL_KEY, "face_landmark") val faceLandmark = FaceLandmarkWrapper() // Preprocess with detected face regions val faceLandmarkInputs = faceLandmark.preprocess(bitmap, faceRegions) // Run face landmark model val faceLandmarkOutputs = faceLandmarkModel.run(faceLandmarkInputs) // Postprocess to get landmarks val landmarks = faceLandmark.postprocess(faceLandmarkOutputs) ``` -------------------------------- ### Initialize and Use Face Emotion Recognition (Android Kotlin) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize the FaceEmotionRecognitionWrapper, preprocess a bitmap, run the model, and postprocess the outputs to get a result bitmap. This snippet is for Android development using Kotlin. ```kotlin // (0) Initialize Face Emotion Recognition wrapper val feature = FaceEmotionRecognitionWrapper() // (1) Preprocess bitmap and get processed float array val inputs = feature.preprocess(bitmap) // ... run model ... // (2) Postprocess to bitmap val resultBitmap = feature.postprocess(outputs) ``` -------------------------------- ### Initialize and Use Face Emotion Recognition (iOS Swift) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize the FaceEmotionRecognitionWrapper, preprocess a UIImage, run the model, and postprocess the outputs to get a result bitmap. This snippet is for iOS development using Swift. ```swift import ZeticMLange // (0) Initialize Face Emotion Recognition wrapper let feature = FaceEmotionRecognitionWrapper() // (1) Preprocess UIImage and get processed float array let inputs = feature.preprocess(image) // ... run model ... // (2) Postprocess to UIImage let resultBitmap = feature.postprocess(&outputs) ``` -------------------------------- ### Example Usage of Tensor Source: https://docs.zetic.ai/api-reference/flutter/Tensor Demonstrates creating a Tensor using a typed factory, printing the element count, and accessing its data as a Float32List. ```dart final input = Tensor.float32View( Float32List.fromList([1.0, 2.0, 3.0, 4.0]), shape: const [1, 4], ); print(input.count()); // 4 print(input.asFloat32List()); ``` -------------------------------- ### Android: Face Detection Model Initialization and Run Source: https://docs.zetic.ai/how-to-guides/multi-model-pipeline Initializes the face detection model, preprocesses the image, runs inference, and postprocesses the output to get face regions. This is the first step in a face detection and landmark extraction pipeline. ```kotlin // Initialize face detection model val faceDetectionModel = ZeticMLangeModel(this, PERSONAL_KEY, "face_detection") val faceDetection = FaceDetectionWrapper() // Preprocess image val faceDetectionInputs = faceDetection.preprocess(bitmap) // Run face detection val faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs) // Postprocess to get face regions val faceRegions = faceDetection.postprocess(faceDetectionOutputs) ``` -------------------------------- ### Face Emotion Recognition Pipeline (Swift) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize and run the face emotion recognition model. Preprocess with detected face regions, run the model, and postprocess to get emotions. ```swift // (0) Initialize face emotion recognition model let faceEmotionRecognitionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "ElenaRyumina/FaceEmotionRecognition") let faceEmotionRecognition = FaceEmotionRecognitionWrapper() // (1) Preprocess with detected face regions let faceEmotionRecognitionInputs = faceEmotionRecognition.preprocess(bitmap, faceDetectionPostprocessed) // (2) Run face emotion recognition model let faceEmotionRecognitionOutputs = try faceEmotionRecognitionModel.run(inputs: faceEmotionRecognitionInputs) // (3) Postprocess to get emotions let faceEmotionRecognitionPostprocessed = faceEmotionRecognition.postprocess(faceEmotionRecognitionOutputs) ``` -------------------------------- ### Initialize and Use Face Landmark Wrapper (Kotlin) Source: https://docs.zetic.ai/tutorials/face-landmark Initialize the Face Landmark wrapper, preprocess a bitmap, run the model, and postprocess the outputs to a bitmap. ```kotlin // (0) Initialize Face Landmark wrapper val feature = FaceLandmarkWrapper() // (1) Preprocess bitmap and get processed float array val inputs = feature.preprocess(bitmap) // ... run model ... // (2) Postprocess to bitmap val resultBitmap = feature.postprocess(outputs) ``` -------------------------------- ### Install Python Packages for Whisper Source: https://docs.zetic.ai/tutorials/speech-recognition-whisper Installs the necessary Python packages for running OpenAI Whisper, including PyTorch, Transformers, and NumPy. ```bash pip install torch transformers numpy ``` -------------------------------- ### Initialize and Use Face Landmark Wrapper (Swift) Source: https://docs.zetic.ai/tutorials/face-landmark Initialize the Face Landmark wrapper, preprocess a UIImage, run the model, and postprocess the outputs to a UIImage. ```swift import ZeticMLange // (0) Initialize Face Landmark wrapper let feature = FaceLandmarkWrapper() // (1) Preprocess UIImage and get processed float array let inputs = feature.preprocess(image) // ... run model ... // (2) Postprocess to UIImage let resultBitmap = feature.postprocess(&outputs) ``` -------------------------------- ### Get Tensor Element Count Source: https://docs.zetic.ai/api-reference/android/Tensor Returns the number of elements in the tensor. ```kotlin fun count(): Int ``` -------------------------------- ### Lifecycle - cleanUp() Source: https://docs.zetic.ai/api-reference/ios/ZeticMLangeLLMModel Cleans up resources. It is recommended to call this before starting a fresh conversation. ```APIDOC ## Lifecycle - `cleanUp()` ### Method Signature ```swift public func cleanUp() throws ``` ### Description Cleans up resources. Call `cleanUp()` before starting a fresh conversation. ``` -------------------------------- ### Initialize and Use Face Detection Wrapper (Kotlin) Source: https://docs.zetic.ai/tutorials/face-detection Initialize the Face Detection wrapper, preprocess a bitmap, and postprocess the model outputs to a bitmap in Kotlin. ```kotlin // (0) Initialize Face Detection wrapper val feature = FaceDetectionWrapper() // (1) Preprocess bitmap and get processed float array val inputs = feature.preprocess(bitmap) // ... run model ... // (2) Postprocess to bitmap val resultBitmap = feature.postprocess(outputs) ``` -------------------------------- ### Initialize Model Once, Run Many Source: https://docs.zetic.ai/how-to-guides/performance-optimization Initialize the ZeticMLangeModel once to download and create the NPU context, then reuse the instance for multiple inferences. ```kotlin // Do this once val model = ZeticMLangeModel(context, PERSONAL_KEY, MODEL_NAME) // Reuse for multiple inferences for (frame in videoFrames) { val outputs = model.run(preprocessFrame(frame)) } ``` -------------------------------- ### LLMInitOption Source: https://docs.zetic.ai/api-reference/android/enums-and-constants Configuration options for initializing an LLM model, including KV cache cleanup policy and context size. ```APIDOC ## `LLMInitOption` Configuration class for LLM initialization. ```kotlin data class LLMInitOption( val kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy = LLMKVCacheCleanupPolicy.CLEAN_UP_ON_FULL, val nCtx: Int = 2048, ) ``` ### Fields * **`kvCacheCleanupPolicy`** (`LLMKVCacheCleanupPolicy`) - Default: `CLEAN_UP_ON_FULL`. In-memory KV-cache behavior when context storage is full. * **`nCtx`** (`Int`) - Default: `2048`. Requested context size. The runtime may normalize it. ``` -------------------------------- ### Get Tensor Size in Bytes Source: https://docs.zetic.ai/api-reference/android/Tensor Returns the size of the tensor in bytes, equivalent to data.capacity(). ```kotlin fun size(): Int ``` -------------------------------- ### Prepare Audio Input for Upload (Python) Source: https://docs.zetic.ai/how-to-guides/custom-preprocessing Loads an audio file, resamples it to 16kHz, and saves it as a float32 numpy array. ```python # Example: preparing audio input for upload import numpy as np import librosa # Load and resample audio to 16kHz audio, sr = librosa.load("audio.wav", sr=16000) # Convert to numpy and save np_audio = audio.astype(np.float32) np.save("audio_input.npy", np_audio) ``` -------------------------------- ### Lifecycle - resetSession Source: https://docs.zetic.ai/api-reference/android/ZeticMLangeLLMModel Resets the current session of the LLM. Recommended before starting a fresh conversation. ```APIDOC ```java fun resetSession() ``` ### Example ```java model.resetSession() ``` ``` -------------------------------- ### Initialize and Use Face Detection Wrapper (Swift) Source: https://docs.zetic.ai/tutorials/face-detection Initialize the Face Detection wrapper, preprocess a UIImage, and postprocess the model outputs to a UIImage in Swift. ```swift import ZeticMLange import ext // (0) Initialize Face Detection wrapper let feature = FaceDetectionWrapper() // (1) Preprocess UIImage and get processed float array let inputs = feature.preprocess(image) // ... run model ... // (2) Postprocess to UIImage let resultBitmap = feature.postprocess(&outputs) ``` -------------------------------- ### get(index) Source: https://docs.zetic.ai/api-reference/android/Tensor Reads a single element at the specified index and returns it as the requested numeric type. ```APIDOC ## `get(index)` Reads a single element at `index` as the requested numeric type. ```kotlin inline fun get(index: Int): T ``` **Supported `T`:** `Int`, `Long`, `Float`, `Double`, `Char`, `Short`, `Byte`. **Example:** ```kotlin val first: Float = tensor.get(0) ``` ``` -------------------------------- ### Initialize and Run LLM Models Source: https://docs.zetic.ai/platform-integration/flutter/advanced-configuration Utilize ZeticMLangeLLMModel for text generation, configuring context size and cache cleanup policy. Use waitForNextToken for token-by-token output. ```dart final llm = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: llmModelName, initOption: const LLMInitOption( nCtx: 4096, kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy.cleanUpOnFull, ), onDownload: (progress) { print('Loading LLM ${(progress * 100).round()}%'); }, ); llm.run('Explain on-device AI in one paragraph.'); while (true) { final next = llm.waitForNextToken(); if (next.isFinished) { break; } print(next.token); } llm.cleanUp(); llm.close(); ``` -------------------------------- ### LLMInitOption Struct Source: https://docs.zetic.ai/api-reference/ios/enums-and-constants Configuration options for initializing an LLM, including KV cache cleanup policy, context size, and run configuration ID. ```APIDOC ## `LLMInitOption` ```swift public struct LLMInitOption { public var kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy public var nCtx: Int public var runConfigId: Int64 } ``` ### Fields - **`kvCacheCleanupPolicy`** (`LLMKVCacheCleanupPolicy`) - Default: `.CLEAN_UP_ON_FULL` - In-memory KV-cache behavior when context storage is full. - **`nCtx`** (`Int`) - Default: `2048` - Requested context size. The runtime may normalize it. - **`runConfigId`** (`Int64`) - Default: `0` - Run configuration id used by compatible LFM-VL packages. ``` -------------------------------- ### DataType Protocol Source: https://docs.zetic.ai/api-reference/ios/Tensor The DataType protocol defines a way to get the byte size of a tensor element. ```APIDOC ## DataType `DataType` is a protocol that exposes the byte size of a tensor element. ```swift public protocol DataType { var size: Int { get } } ``` ``` -------------------------------- ### Initialize and Run LLM with Token Streaming (iOS Swift) Source: https://docs.zetic.ai/introduction/key-concepts Initialize ZeticMLangeLLMModel for large language models and handle token streaming. Requires personal key and model name. ```swift let model = try ZeticMLangeLLMModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) try model.run("prompt") while true { let result = model.waitForNextToken() if result.generatedTokens == 0 { break } print(result.token) } ``` -------------------------------- ### Lifecycle - cleanUp Source: https://docs.zetic.ai/api-reference/android/ZeticMLangeLLMModel Cleans up resources associated with the LLM model. Recommended before starting a fresh conversation. ```APIDOC ## Lifecycle ```java fun cleanUp() ``` ### Example ```java model.cleanUp() ``` ``` -------------------------------- ### Initialize and Run LLM with Token Streaming (Android Kotlin) Source: https://docs.zetic.ai/introduction/key-concepts Instantiate ZeticMLangeLLMModel for large language models and process token streams. Requires context, personal key, and model name. ```kotlin val model = ZeticMLangeLLMModel(context, PERSONAL_KEY, MODEL_NAME) model.run("prompt") while (true) { val result = model.waitForNextToken() if (result.generatedTokens == 0) break print(result.token) } ``` -------------------------------- ### Clean Up LLM Resources Source: https://docs.zetic.ai/api-reference/ios/ZeticMLangeLLMModel Cleans up LLM resources, typically called before starting a new conversation. ```swift try model.cleanUp() ``` -------------------------------- ### Instantiate Random Tensor Source: https://docs.zetic.ai/api-reference/android/Tensor Example of creating a random Tensor with Float32 data type and a specified shape. ```kotlin val fakeInput = Tensor.random(DataType.Float32, intArrayOf(1, 3, 640, 640)) ``` -------------------------------- ### Get Tensor Element by Index Source: https://docs.zetic.ai/api-reference/android/Tensor Reads a single element at the specified index as the requested numeric type. ```kotlin inline fun get(index: Int): T ``` ```kotlin val first: Float = tensor.get(0) ``` -------------------------------- ### Initialize Model with Personal Key (Android Kotlin) Source: https://docs.zetic.ai/model-deployment/understanding-model-keys Demonstrates initializing a ZeticMLangeModel in Android (Kotlin) using a personal key stored in build configuration. ```kotlin val model = ZeticMLangeModel( context, BuildConfig.PERSONAL_KEY, // From build configuration "USERID/REPOSITORY_NAME" ) ``` -------------------------------- ### Reset KV State Source: https://docs.zetic.ai/api-reference/ios/ZeticMLangeLLMModel Resets the current KV state of the LLM, effectively starting a fresh conversation. ```swift try model.resetKVState() ``` -------------------------------- ### Load Model, Prepare Inputs, and Run Inference Source: https://docs.zetic.ai/platform-integration/ios/basic-inference This snippet demonstrates the core steps for running inference: loading the Melange model, preparing input tensors, and executing the inference. Ensure input shapes match the model's requirements. ```swift import ZeticMLange // (1) Load model // This handles model download (if needed) and Neural Engine context creation let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) // (2) Prepare model inputs // Ensure input shapes match your model's requirement (e.g., Float32 arrays) let inputs: [Tensor] = [] // Prepare your inputs // (3) Run Inference // Executes the fully automated hardware graph. // No manual delegate configuration or memory syncing required. let outputs = try model.run(inputs: inputs) ``` -------------------------------- ### LLMInitOption Structure Source: https://docs.zetic.ai/api-reference/ios/enums-and-constants Defines initialization options for LLM models, including KV cache cleanup policy, context size, and run configuration ID. ```swift public struct LLMInitOption { public var kvCacheCleanupPolicy: LLMKVCacheCleanupPolicy public var nCtx: Int public var runConfigId: Int64 } ``` -------------------------------- ### Install ZeticMLange Flutter SDK Source: https://docs.zetic.ai/release-notes/changelog Use this dependency for Flutter projects to integrate the ZeticMLange SDK version 1.9.1. ```yaml zetic_mlange: ^1.9.1 ``` -------------------------------- ### Initialize and Process Audio with Whisper (Swift) Source: https://docs.zetic.ai/tutorials/speech-recognition-whisper Initializes Whisper components, processes audio, runs the encoder, generates tokens, and decodes them into text using Swift. ```Swift // Initialize components let wrapper = WhisperFeatureWrapper() let encoder = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "OpenAI/whisper-tiny-encoder") let decoder = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "OpenAI/whisper-tiny-decoder") // Process audio to features let features = wrapper.process(input.audio) // Run encoder let outputs = encoder.process(features) // Generate tokens using decoder let generatedIds = decoder.process(outputs) // Convert tokens to text let text = wrapper.decodeToken(generatedIds, true) return WhisperOutput(text: text) ``` -------------------------------- ### Instantiate Tensor with Specific Shape Source: https://docs.zetic.ai/api-reference/android/Tensor Example of creating a Tensor from a float array with a specified data type and shape. ```kotlin val input = Tensor.of( data = floatArrayOf(/* ... */), dataType = DataType.Float32, shape = intArrayOf(1, 3, 640, 640), ) ``` -------------------------------- ### Conversation Reset - Android (Kotlin) Source: https://docs.zetic.ai/llm-inference/streaming-generation Clears the current conversation context and starts a new one with a fresh prompt. ```kotlin model.cleanUp() model.run("Start a new conversation") ``` -------------------------------- ### Initialize LLM Model with Processor and Quantization Filters (Android) Source: https://docs.zetic.ai/llm-inference/inference-modes Initialize the LLM model with `LLMModelMode.RUN_AUTO`, specifying `APType.GPU` and `LLMQuantType.GGUF_QUANT_Q4_K_M` to filter candidates. Unsupported combinations fail during initialization. ```kotlin val model = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = LLMModelMode.RUN_AUTO, apType = APType.GPU, quantType = LLMQuantType.GGUF_QUANT_Q4_K_M, ) ``` -------------------------------- ### Initialize and Run General AI Model (Android Kotlin) Source: https://docs.zetic.ai/introduction/key-concepts Instantiate ZeticMLangeModel for general AI tasks and execute inference. Requires context, personal key, and model name. ```kotlin val model = ZeticMLangeModel(context, PERSONAL_KEY, MODEL_NAME) val outputs = model.run(inputs) ``` -------------------------------- ### Install ZeticMLange Flutter SDK (1.9.0) Source: https://docs.zetic.ai/release-notes/changelog Use this dependency for Flutter projects to integrate the ZeticMLange SDK version 1.9.0. ```yaml zetic_mlange: ^1.9.0 ``` -------------------------------- ### Android Kotlin Inference Mode Examples Source: https://docs.zetic.ai/how-to-guides/inference-mode-selection Instantiate ZeticMLangeModel with different inference modes: RUN_AUTO (default), RUN_SPEED, and RUN_ACCURACY. ```kotlin // Default: Auto mode // Speed first, but maintains SNR above 20dB val modelDefault = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_AUTO ) // Speed First Mode val modelFast = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_SPEED ) // Accuracy First Mode val modelAccurate = ZeticMLangeModel( context = this, personalKey = PERSONAL_KEY, name = MODEL_NAME, modelMode = ModelMode.RUN_ACCURACY ) ``` -------------------------------- ### Initialize and Run LLM with Token Streaming (Flutter Dart) Source: https://docs.zetic.ai/introduction/key-concepts Create an instance of ZeticMLangeLLMModel for large language models and stream tokens. Requires personal key and model name. ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: modelName, ); model.run('prompt'); while (true) { final result = model.waitForNextToken(); if (result.isFinished) break; print(result.token); } ``` -------------------------------- ### Initialize Model with Personal Key (iOS Swift) Source: https://docs.zetic.ai/model-deployment/understanding-model-keys Shows how to initialize a ZeticMLangeModel in iOS (Swift) using a personal key loaded from a configuration file. ```swift let model = try ZeticMLangeModel( personalKey: Configuration.personalKey, // From config name: "USERID/REPOSITORY_NAME" ) ``` -------------------------------- ### Tensor Example Source: https://docs.zetic.ai/api-reference/flutter/Tensor Demonstrates how to create and use a Tensor object in Flutter, including printing its element count and accessing its data as a Float32List. ```APIDOC ## Example ```dart final input = Tensor.float32View( Float32List.fromList([1.0, 2.0, 3.0, 4.0]), shape: const [1, 4], ); print(input.count()); // 4 print(input.asFloat32List()); ``` ``` -------------------------------- ### Initialize and Process Audio with Whisper (Kotlin) Source: https://docs.zetic.ai/tutorials/speech-recognition-whisper Initializes Whisper components, processes audio data, runs the encoder, generates tokens, and decodes them into text using Kotlin. ```Kotlin // Initialize components val whisper = WhisperFeatureWrapper() val encoder = ZeticMLangeModel(this, PERSONAL_KEY, "OpenAI/whisper-tiny-encoder") val decoder = ZeticMLangeModel(this, PERSONAL_KEY, "OpenAI/whisper-tiny-decoder") // Process audio val features = whisper.process(audioData) // Run encoder encoder.process(features) // Generate tokens using decoder val generatedIds = decoder.generateTokens(outputs) // Convert tokens to text val text = whisper.decodeToken(generatedIds.toIntArray(), true) ``` -------------------------------- ### Flutter Dart Inference Mode Examples Source: https://docs.zetic.ai/how-to-guides/inference-mode-selection Create ZeticMLangeModel instances with explicit inference modes: ModelMode.runAuto (default), ModelMode.runSpeed, and ModelMode.runAccuracy. ```dart // Default: Auto mode // Speed first, but maintains SNR above 20dB final modelDefault = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runAuto, ); // Speed First Mode final modelFast = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runSpeed, ); // Accuracy First Mode final modelAccurate = await ZeticMLangeModel.create( personalKey: personalKey, name: modelName, modelMode: ModelMode.runAccuracy, ); ``` -------------------------------- ### Basic Streaming - iOS (Swift) Source: https://docs.zetic.ai/llm-inference/streaming-generation Initializes the LLM model, executes a prompt, and accumulates generated tokens. Ensure proper error handling for model operations. ```swift let model = try ZeticMLangeLLMModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) try model.run(userPrompt) var buffer = "" while true { let result = model.waitForNextToken() if result.generatedTokens == 0 { break } buffer.append(result.token) } let output = buffer ``` -------------------------------- ### Initialize LLM Model for Auto Selection with GPU (iOS) Source: https://docs.zetic.ai/llm-inference/inference-modes Use `.RUN_AUTO` and specify `.GPU` to let the SDK select the best runtime and quantization for the device's GPU. ```swift let modelForGpu = try await ZeticMLangeLLMModel( personalKey: PERSONAL_KEY, name: MODEL_NAME, modelMode: .RUN_AUTO, apType: .GPU ) ``` -------------------------------- ### Tensor Methods Source: https://docs.zetic.ai/api-reference/flutter/Tensor Provides methods to count elements, get the total size in bytes, and access tensor data as various typed lists. ```dart int count() int size() Uint8List asUint8List() Float32List asFloat32List() Float64List asFloat64List() Int32List asInt32List() Int64List asInt64List() ``` -------------------------------- ### Face Detection Pipeline (Dart) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize and run the face detection model. Preprocess the image, run the model, and postprocess to get face regions. ```dart // (0) Initialize face detection model final faceDetectionModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'google/MediaPipe-Face-Detection', ); // (1) Preprocess image final faceDetectionInputs = preprocessFaceDetectionImage(image); // (2) Run face detection model final faceDetectionOutputs = faceDetectionModel.run(faceDetectionInputs); // (3) Postprocess to get face regions final faceDetectionPostprocessed = postprocessFaceDetection(faceDetectionOutputs); ``` -------------------------------- ### Initialize and Run General AI Model (iOS Swift) Source: https://docs.zetic.ai/introduction/key-concepts Initialize ZeticMLangeModel for general AI tasks and execute inference. Requires personal key and model name. ```swift let model = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: MODEL_NAME) let outputs = try model.run(inputs: inputs) ``` -------------------------------- ### Basic Streaming - Android (Kotlin) Source: https://docs.zetic.ai/llm-inference/streaming-generation Initiates LLM model, runs a prompt, and collects generated tokens incrementally in a loop. Use this for basic token collection. ```kotlin val model = ZeticMLangeLLMModel(context, PERSONAL_KEY, MODEL_NAME) model.run(userPrompt) val sb = StringBuilder() while (true) { val result = model.waitForNextToken() if (result.generatedTokens == 0) break if (result.token.isNotEmpty()) sb.append(result.token) } val output = sb.toString() ``` -------------------------------- ### Basic Streaming - Flutter (Dart) Source: https://docs.zetic.ai/llm-inference/streaming-generation Creates an LLM model instance, starts generation with a prompt, and collects tokens. The `isFinished` property indicates completion. ```dart final model = await ZeticMLangeLLMModel.create( personalKey: personalKey, name: modelName, ); model.run(userPrompt); final buffer = StringBuffer(); while (true) { final result = model.waitForNextToken(); if (result.isFinished) { break; } buffer.write(result.token); } final output = buffer.toString(); ``` -------------------------------- ### Convert Face Landmark Model to ONNX Source: https://docs.zetic.ai/tutorials/face-landmark Use tf2onnx to convert a TensorFlow Lite face landmark model to ONNX format. Ensure you have installed the tf2onnx package. ```bash python -m tf2onnx.convert --tflite face_landmark.tflite --output face_landmark.onnx --opset 13 ``` -------------------------------- ### Convert TFLite to ONNX Model Source: https://docs.zetic.ai/tutorials/face-detection Convert the MediaPipe Face Detection TFLite model to ONNX format using tf2onnx. Ensure Python 3.8+ and tf2onnx are installed. ```bash pip install tf2onnx python -m tf2onnx.convert --tflite face_detection_short_range.tflite --output face_detection_short_range.onnx --opset 13 ``` -------------------------------- ### Load Model, Prepare Inputs, and Run Inference in Java Source: https://docs.zetic.ai/platform-integration/android/basic-inference This Java snippet demonstrates loading a Melange model, preparing input tensors, and running inference. It's crucial that input shapes align with the model's specifications. ```java // (1) Load model // This handles model download (if needed) and NPU context creation ZeticMLangeModel model = new ZeticMLangeModel(CONTEXT, PERSONAL_KEY, MODEL_NAME); // (2) Prepare model inputs // Ensure input shapes match your model's requirement (e.g., Float32 arrays) Tensor[] inputs = // Prepare your inputs; // (3) Run Inference // Executes the hardware-accelerated graph. This is a blocking call. Tensor[] outputs = model.run(inputs); ``` -------------------------------- ### Flutter RAG Pipeline with Custom Retriever Source: https://docs.zetic.ai/llm-inference/rag Integrate a custom RagRetriever with the RagPipeline in Flutter. This example shows how to instantiate a retriever and use it within the pipeline to respond to queries. ```dart final retriever = MyRetriever(); final rag = RagPipeline( retriever: retriever, llm: model, profile: const RagProfile.qwen25(), ); await for (final token in rag.respond(query: 'What does ZeticMLange do?')) { append(token); } ``` -------------------------------- ### Define and Register Weather Tool (Flutter) Source: https://docs.zetic.ai/llm-inference/function-calling Defines a tool specification for getting weather and registers it with the LLM model on Flutter. The executor returns a hardcoded weather result. ```dart final weatherTool = LLMToolSpec( name: 'get_weather', description: 'Get the weather for a city.', parametersJson: ''' { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } ''', ); model.registerTool(weatherTool, (call) { return const LLMToolResult(content: '{"summary":"Sunny, 24C"}'); }); ``` -------------------------------- ### Text Generation with run() Source: https://docs.zetic.ai/api-reference/android/ZeticMLangeLLMModel Initiate text generation for a given prompt using the run() method. ```kotlin val result = model.run("Explain on-device AI in one paragraph.") ``` -------------------------------- ### Define and Register Weather Tool (iOS) Source: https://docs.zetic.ai/llm-inference/function-calling Defines a tool specification for getting weather and registers it with the LLM model on iOS. The executor returns a hardcoded weather result. ```swift let weatherTool = LLMToolSpec( name: "get_weather", description: "Get the weather for a city.", parametersJson: """ { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } """ ) try model.registerTool(weatherTool) { call in LLMToolResult(content: #"{"summary":"Sunny, 24C"}"#) } ``` -------------------------------- ### iOS: Face Detection Model Initialization and Run Source: https://docs.zetic.ai/how-to-guides/multi-model-pipeline Initializes the face detection model, preprocesses the image, runs inference, and postprocesses the output to obtain face regions. This is the initial stage for face analysis in an iOS application. ```swift // Initialize face detection model let faceDetectionModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "face_detection") let faceDetection = FaceDetectionWrapper() // Preprocess image let faceDetectionInputs = faceDetection.preprocess(image) // Run face detection let faceDetectionOutputs = try faceDetectionModel.run(inputs: faceDetectionInputs) // Postprocess to get face regions let faceRegions = faceDetection.postprocess(faceDetectionOutputs) ``` -------------------------------- ### Define and Register Weather Tool (Android) Source: https://docs.zetic.ai/llm-inference/function-calling Defines a tool specification for getting weather and registers it with the LLM model on Android. The executor returns a hardcoded weather result. ```kotlin val weatherTool = LLMToolSpec( name = "get_weather", description = "Get the weather for a city.", parametersJson = """ { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } """.trimIndent() ) model.registerTool(weatherTool) { call -> LLMToolResult(content = "{"summary":"Sunny, 24C"}") } ``` -------------------------------- ### iOS: Face Landmark Model Initialization and Run Source: https://docs.zetic.ai/how-to-guides/multi-model-pipeline Initializes the face landmark model, preprocesses the image using detected face regions, runs inference, and postprocesses the output to extract landmarks. This follows the face detection step in a pipeline. ```swift // Initialize face landmark model let faceLandmarkModel = try ZeticMLangeModel(personalKey: PERSONAL_KEY, name: "face_landmark") let faceLandmark = FaceLandmarkWrapper() // Preprocess with detected face regions let faceLandmarkInputs = faceLandmark.preprocess(image, faceRegions) // Run face landmark model let faceLandmarkOutputs = try faceLandmarkModel.run(inputs: faceLandmarkInputs) // Postprocess to get landmarks let landmarks = faceLandmark.postprocess(faceLandmarkOutputs) ``` -------------------------------- ### Export YOLOv11 Model to ONNX Source: https://docs.zetic.ai/tutorials/object-detection-yolo Export the YOLOv11 model to ONNX format using the ultralytics library. This script requires Python 3.8+ with ultralytics, opencv-python, and numpy installed. ```python from ultralytics import YOLO import torch model = YOLO("yolo11n.pt") model.export(format="onnx", opset=12, simplify=True, dynamic=False, imgsz=640) ``` -------------------------------- ### Configure Android Build Settings Source: https://docs.zetic.ai/platform-integration/flutter/setup Set Android `minSdk` to 24 or later, build for `arm64-v8a`, and enable legacy JNI packaging in `android/app/build.gradle.kts`. ```kotlin android { defaultConfig { minSdk = maxOf(flutter.minSdkVersion, 24) ndk { abiFilters += "arm64-v8a" } } packaging { jniLibs { useLegacyPackaging = true } } } ``` -------------------------------- ### Flutter Custom RagRetriever Implementation Source: https://docs.zetic.ai/llm-inference/rag Implement the RagRetriever interface in Flutter to provide custom retrieval logic. This example defines a class that fulfills the interface requirements for retrieving chunks. ```dart final class MyRetriever implements RagRetriever { @override Future> retrieve(String query, {required int topK}) async { return const [ RetrievedChunk( text: 'ZeticMLange runs optimized models on-device.', score: 0.94, source: 'docs', ), ]; } } ``` -------------------------------- ### ZeticMLangeLLMModel Constructor Source: https://docs.zetic.ai/api-reference/android/ZeticMLangeLLMModel Initializes the ZeticMLangeLLMModel with specified parameters. It loads an on-device LLM from the Melange registry and configures its behavior for inference. ```APIDOC ## Constructor ```java ZeticMLangeLLMModel( context: Context, personalKey: String, name: String, version: Int? = null, modelMode: LLMModelMode = LLMModelMode.RUN_AUTO, apType: APType? = null, quantType: LLMQuantType? = null, cacheHandlingPolicy: ModelCacheHandlingPolicy = ModelCacheHandlingPolicy.REMOVE_OVERLAPPING, initOption: LLMInitOption = LLMInitOption(), onDownload: ((Float) -> Unit)? = null, ) ``` ### Parameters - **context** (`Context`) - Required - Android context used for cache and file access. - **personalKey** (`String`) - Required - Personal key for accessing the model. - **name** (`String`) - Required - Model name in `account_name/project_name` format. - **version** (`Int?`) - Optional - Model version. `null` loads the latest version. - **modelMode** (`LLMModelMode`) - Optional - Backend selection strategy. Defaults to `RUN_AUTO`. - **apType** (`APType?`) - Optional - Optional processor filter. - **quantType** (`LLMQuantType?`) - Optional - Optional quantization filter. - **cacheHandlingPolicy** (`ModelCacheHandlingPolicy`) - Optional - Managed artifact cache cleanup policy. Defaults to `REMOVE_OVERLAPPING`. - **initOption** (`LLMInitOption`) - Optional - LLM initialization options. Defaults to `LLMInitOption()`. - **onDownload** (`((Float) -> Unit)?`) - Optional - Download progress callback from `0.0` to `1.0`. ### Example ```java val model = ZeticMLangeLLMModel( context = context, personalKey = PERSONAL_KEY, name = "account_name/project_name", initOption = LLMInitOption(nCtx = 4096), ) ``` ``` -------------------------------- ### Import ZeticMLangeLLMModel Source: https://docs.zetic.ai/api-reference/android/ZeticMLangeLLMModel Import the necessary class for using ZeticMLangeLLMModel. ```kotlin import com.zeticai.mlange.core.model.llm.ZeticMLangeLLMModel ``` -------------------------------- ### Face Emotion Recognition Pipeline (Dart) Source: https://docs.zetic.ai/tutorials/face-emotion-recognition Initialize and run the face emotion recognition model. Preprocess with detected face regions, run the model, and postprocess to get emotions. ```dart // (0) Initialize face emotion recognition model final faceEmotionRecognitionModel = await ZeticMLangeModel.create( personalKey: personalKey, name: 'ElenaRyumina/FaceEmotionRecognition', ); // (1) Preprocess with detected face regions final faceEmotionRecognitionInputs = preprocessFaceEmotionImage(image, faceDetectionPostprocessed); // (2) Run face emotion recognition model final faceEmotionRecognitionOutputs = faceEmotionRecognitionModel.run(faceEmotionRecognitionInputs); // (3) Postprocess to get emotions final faceEmotionRecognitionPostprocessed = postprocessFaceEmotion(faceEmotionRecognitionOutputs); ```