### Start Local Development for Docusaurus Site

Source: https://github.com/leehack/llamadart/blob/main/website/README.md

Navigate to the website directory, install dependencies, and start the Docusaurus development server.

```bash
cd website
npm ci
npm run start
```

--------------------------------

### Get Dependencies and View Help

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_cli/README.md

Installs project dependencies and displays the command-line help for the llamadart CLI.

```bash
dart pub get
dart run bin/llamadart_cli.dart --help
```

--------------------------------

### Run TUI Coding Agent

Source: https://github.com/leehack/llamadart/blob/main/example/tui_coding_agent/README.md

Standard command to get started with the TUI coding agent. Ensure you are in the example directory.

```bash
cd example/tui_coding_agent
dart pub get
dart run bin/tui_coding_agent.dart
```

--------------------------------

### Run Basic App Example

Source: https://github.com/leehack/llamadart/blob/main/CONTRIBUTING.md

Navigate to the basic app example directory and run it. This command will automatically download the correct pre-compiled native binaries for your platform.

```bash
cd example/basic_app
dart run
```

--------------------------------

### Installation Bundle Configuration

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/linux/CMakeLists.txt

Configures the installation prefix to create a relocatable bundle. It also ensures the build bundle directory is clean before installation.

```cmake
# === Installation ===
# By default, "installing" just makes a relocatable bundle in the build
# directory.
set(BUILD_BUNDLE_DIR "${PROJECT_BINARY_DIR}/bundle")
if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
  set(CMAKE_INSTALL_PREFIX "${BUILD_BUNDLE_DIR}" CACHE PATH "..." FORCE)
endif()

# Start with a clean build bundle directory every time.
install(CODE "
  file(REMOVE_RECURSE \"${BUILD_BUNDLE_DIR}/\")
  " COMPONENT Runtime)

set(INSTALL_BUNDLE_DATA_DIR "${CMAKE_INSTALL_PREFIX}/data")
set(INSTALL_BUNDLE_LIB_DIR "${CMAKE_INSTALL_PREFIX}/lib")
```

--------------------------------

### Install Dependencies

Source: https://github.com/leehack/llamadart/blob/main/example/training_notebook/lora_training.ipynb

Installs necessary libraries for LoRA training, including transformers, peft, bitsandbytes, datasets, accelerate, and trl.

```python
%pip install -q -U transformers peft bitsandbytes datasets accelerate trl
```

--------------------------------

### Run Chat App Example

Source: https://github.com/leehack/llamadart/blob/main/example/README.md

Commands to set up and run the Flutter chat application.

```bash
cd chat_app
flutter pub get
flutter run
```

--------------------------------

### Run llama.cpp-style CLI Example

Source: https://github.com/leehack/llamadart/blob/main/example/README.md

Commands to set up and run the llama.cpp-style CLI tool, including displaying help information.

```bash
cd llamadart_cli
dart pub get
dart run bin/llamadart_cli.dart --help
```

--------------------------------

### Install APK to Device

Source: https://github.com/leehack/llamadart/blob/main/doc/android_runtime_smoke_test_plan.md

Install the built release APK onto a connected Android device.

```bash
adb install -r build/app/outputs/flutter-apk/app-release.apk
```

--------------------------------

### Install Application Target and Core Files

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/linux/CMakeLists.txt

Installs the application executable, ICU data, and the Flutter library to the bundle. These are essential components for the application to run.

```cmake
install(TARGETS ${BINARY_NAME} RUNTIME DESTINATION "${CMAKE_INSTALL_PREFIX}"
  COMPONENT Runtime)

install(FILES "${FLUTTER_ICU_DATA_FILE}" DESTINATION "${INSTALL_BUNDLE_DATA_DIR}"
  COMPONENT Runtime)

install(FILES "${FLUTTER_LIBRARY}" DESTINATION "${INSTALL_BUNDLE_LIB_DIR}"
  COMPONENT Runtime)
```

--------------------------------

### Run API Server Example

Source: https://github.com/leehack/llamadart/blob/main/example/README.md

Commands to set up and run the Relic-based HTTP API server, specifying a local model path.

```bash
cd llamadart_server
dart pub get
dart run llamadart_server --model /path/to/model.gguf
```

--------------------------------

### Run Basic App Examples

Source: https://github.com/leehack/llamadart/blob/main/example/README.md

Commands to run the basic console application, including embedding and SQLite vector retrieval demos.

```bash
cd basic_app
dart pub get
dart run
```

```bash
# Embedding demo
dart run bin/llamadart_embedding_example.dart -i "hello world"
```

```bash
# SQLite vector retrieval demo
dart run bin/llamadart_sqlite_vector_example.dart \
  -q "How do I improve embedding throughput?" \
  -d "Increase maxParallelSequences for wider embedding batches." \
  -d "Tune batchSize and ubatchSize together."
```

--------------------------------

### Run Chat App Example on macOS

Source: https://github.com/leehack/llamadart/blob/main/CONTRIBUTING.md

Navigate to the chat app example directory and run it using Flutter on macOS. Other platforms like Linux, Windows, Android, or iOS can also be targeted.

```bash
cd example/chat_app
flutter run -d macos  # or linux, windows, android, ios
```

--------------------------------

### Install Dependencies with Dart

Source: https://github.com/leehack/llamadart/blob/main/example/basic_app/README.md

Run this command to install the necessary project dependencies using Dart's package manager.

```bash
dart pub get
```

--------------------------------

### Install Linux Runtime Dependencies (Fedora/RHEL/CentOS)

Source: https://github.com/leehack/llamadart/blob/main/README.md

Installs necessary host runtime dependencies for Vulkan, BLAS, and general tools on Fedora/RHEL/CentOS-based systems.

```bash
sudo dnf install -y vulkan-loader vulkan-tools openblas
```

--------------------------------

### Install Linux Runtime Dependencies (Ubuntu/Debian)

Source: https://github.com/leehack/llamadart/blob/main/README.md

Installs necessary host runtime dependencies for Vulkan, BLAS, and general tools on Ubuntu/Debian-based systems.

```bash
sudo apt-get update
sudo apt-get install -y libvulkan1 vulkan-tools libopenblas0
```

--------------------------------

### Install Linux Runtime Dependencies (Arch Linux)

Source: https://github.com/leehack/llamadart/blob/main/README.md

Installs necessary host runtime dependencies for Vulkan, BLAS, and general tools on Arch Linux.

```bash
sudo pacman -S --needed vulkan-icd-loader vulkan-tools openblas
```

--------------------------------

### Run with a Specific Model

Source: https://github.com/leehack/llamadart/blob/main/example/basic_app/README.md

Start the application using a specified local model file or a Hugging Face GGUF URL.

```bash
dart run -- -m "path/to/model.gguf"
```

--------------------------------

### Run TUI Coding Agent Example

Source: https://github.com/leehack/llamadart/blob/main/example/README.md

Commands to set up and run the terminal UI coding agent.

```bash
cd tui_coding_agent
dart pub get
dart run bin/tui_coding_agent.dart
```

--------------------------------

### Setup Environment and Configuration

Source: https://github.com/leehack/llamadart/blob/main/example/training_notebook/lora_training.ipynb

Imports required libraries and sets up configuration variables for model and dataset IDs, and output directory. Detects CUDA and Apple Silicon availability for optimized loading.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TrainingArguments, Trainer, DataCollatorForLanguageModeling
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from datasets import load_dataset
import os

# Configuration
model_id = "Qwen/Qwen2.5-0.5B-Instruct" # Small model for demonstration (Downloaded from Hugging Face)
dataset_id = "timdettmers/openassistant-guanaco" # Instruction following dataset
output_dir = "./lora-adapter-output"

# Note: Models and datasets are cached by default in ~/.cache/huggingface/

# Detect environment
has_cuda = torch.cuda.is_available()
is_mac = torch.backends.mps.is_available()
print(f"CUDA available: {has_cuda}")
print(f"Apple Silicon (MPS) available: {is_mac}")
```

--------------------------------

### Run SQLite Vector Search Example

Source: https://github.com/leehack/llamadart/blob/main/example/basic_app/README.md

Execute the SQLite vector search example script with a query and documents. This command initiates the retrieval flow, generating embeddings and searching within the SQLite database.

```bash
dart run bin/llamadart_sqlite_vector_example.dart \
  -q "How do I improve embedding throughput?" \
  -d "Increase maxParallelSequences for wider embedding batches." \
  -d "Tune batchSize and ubatchSize together." \
  -d "Use benchmark sweeps to compare sequential and batch throughput."
```

--------------------------------

### Run the Chat App

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/README.md

Navigate to the chat_app directory, install dependencies, and run the Flutter application.

```bash
cd example/chat_app
flutter pub get
flutter run
```

--------------------------------

### Embedding Example with CPU and Runtime Knobs

Source: https://github.com/leehack/llamadart/blob/main/example/basic_app/README.md

Configure the embedding CLI to force CPU usage and align runtime knobs for closer parity with `llama.cpp`.

```bash
dart run bin/llamadart_embedding_example.dart \
  --cpu --ctx-size 2048 --threads 12 --threads-batch 12 \
  --batch-size 2048 --ubatch-size 2048 --max-seq 2 \
  -i "hello world" -i "semantic search"
```

--------------------------------

### Clean and Get Dependencies

Source: https://github.com/leehack/llamadart/blob/main/doc/android_runtime_smoke_test_plan.md

Run these commands to clean the Flutter project and fetch dependencies before building.

```bash
flutter clean
flutter pub get
```

--------------------------------

### Install Bundled Plugin Libraries

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/linux/CMakeLists.txt

Installs bundled libraries provided by plugins into the application's library directory. This ensures all necessary plugin code is available at runtime.

```cmake
foreach(bundled_library ${PLUGIN_BUNDLED_LIBRARIES})
  install(FILES "${bundled_library}"
    DESTINATION "${INSTALL_BUNDLE_LIB_DIR}"
    COMPONENT Runtime)
endforeach(bundled_library)
```

--------------------------------

### Embedding Example for Retrieval Experiments

Source: https://github.com/leehack/llamadart/blob/main/example/basic_app/README.md

Use this command for retrieval-style experiments by providing a query followed by candidate strings.

```bash
dart run bin/llamadart_embedding_example.dart \
  -i "how do I improve embedding throughput?" \
  -i "Increase maxParallelSequences for wider embedding batches." \
  -i "Tune batchSize and ubatchSize together."
```

--------------------------------

### Clone llama.cpp and Install Requirements

Source: https://github.com/leehack/llamadart/blob/main/example/training_notebook/lora_training.ipynb

Clones the llama.cpp repository from GitHub and installs its Python dependencies using the provided requirements file. This is a prerequisite for converting the LoRA adapter to GGUF format.

```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
pip install -r requirements.txt
```

--------------------------------

### Create a Docs Snapshot for Versioning

Source: https://github.com/leehack/llamadart/blob/main/website/README.md

Manually create a documentation snapshot for a specific version using Docusaurus commands. Ensure you are in the website directory and have installed dependencies.

```bash
cd website
npm ci
npm run docusaurus docs:version <version>
```

--------------------------------

### Install Flutter Assets

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/linux/CMakeLists.txt

Installs the Flutter assets directory into the application's data directory. It first removes any existing assets to ensure a clean copy.

```cmake
# Fully re-copy the assets directory on each build to avoid having stale files
# from a previous install.
set(FLUTTER_ASSET_DIR_NAME "flutter_assets")
install(CODE "
  file(REMOVE_RECURSE \"${INSTALL_BUNDLE_DATA_DIR}/${FLUTTER_ASSET_DIR_NAME}\")
  " COMPONENT Runtime)
install(DIRECTORY "${PROJECT_BUILD_DIR}/${FLUTTER_ASSET_DIR_NAME}"
  DESTINATION "${INSTALL_BUNDLE_DATA_DIR}" COMPONENT Runtime)
```

--------------------------------

### Docker Validation: Prepare Native Modules and Run CLI

Source: https://github.com/leehack/llamadart/blob/main/README.md

Prepares Linux native modules within a Docker container and runs the llamadart_cli example, useful for development and validation.

```bash
# 1) Prepare linux-x64 native modules in .dart_tool/lib
docker run --rm --platform linux/amd64 \
  -v "$PWD:/workspace" \
  -v "/absolute/path/to/model.gguf:/models/your.gguf:ro" \
  -w /workspace/example/llamadart_cli \
  ghcr.io/cirruslabs/flutter:stable \
  bash -lc '
    rm -rf .dart_tool /workspace/.dart_tool/lib &&
    dart pub get &&
    dart run bin/llamadart_cli.dart --model /models/your.gguf --no-interactive --predict 1 --gpu-layers 0
  '
```

--------------------------------

### Run LlamaDart Server

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_server/README.md

Basic command to get dependencies and run the server with a specified model. Use a remote GGUF URL for automatic download.

```bash
dart pub get
dart run llamadart_server --model /path/to/model.gguf
```

--------------------------------

### Persist Settings using SharedPreferences

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/README.md

Example of saving application settings, such as model path and backend preference, using `SharedPreferences`. This requires the `shared_preferences` package.

```dart
final prefs = await SharedPreferences.getInstance();
await prefs.setString('model_path', modelPath);
await prefs.setInt('preferred_backend', backendIndex);
```

--------------------------------

### Build and Verify Documentation Site Links

Source: https://github.com/leehack/llamadart/blob/main/website/README.md

Execute scripts from the repository root to build the documentation site and validate internal and external links.

```bash
./tool/docs/build_site.sh
./tool/docs/validate_links.sh
```

--------------------------------

### Install AOT Library Conditionally

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/linux/CMakeLists.txt

Installs the Ahead-Of-Time (AOT) compiled library to the bundle, but only for non-Debug builds. This optimizes performance for release and profile builds.

```cmake
# Install the AOT library on non-Debug builds only.
if(NOT CMAKE_BUILD_TYPE MATCHES "Debug")
  install(FILES "${AOT_LIBRARY}" DESTINATION "${INSTALL_BUNDLE_LIB_DIR}"
    COMPONENT Runtime)
endif()
```

--------------------------------

### Configure and Run Trainer

Source: https://github.com/leehack/llamadart/blob/main/example/training_notebook/lora_training.ipynb

Sets up Hugging Face TrainingArguments and initializes a Trainer for the LoRA model. The training is configured for a maximum of 50 steps with specific batch size and gradient accumulation.

```python
training_args = TrainingArguments(
    output_dir=output_dir,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    logging_steps=10,
    max_steps=50,
    fp16=not is_mac and not torch.cuda.is_bf16_supported(),
    bf16=torch.cuda.is_bf16_supported() or is_mac,
    save_strategy="no",
    report_to="none",
    remove_unused_columns=False
)

trainer = Trainer(
    model=model,
    train_dataset=tokenized_dataset,
    args=training_args,
    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
)

trainer.train()

# Save only the adapter
model.save_pretrained(output_dir)
```

--------------------------------

### Generate Embeddings with CLI Example

Source: https://github.com/leehack/llamadart/blob/main/example/basic_app/README.md

Generate embedding vectors from input text using the dedicated embedding CLI example. The default model is `ggml-org/embeddinggemma-300M-GGUF`.

```bash
dart run bin/llamadart_embedding_example.dart -i "hello world" -i "rag"
```

--------------------------------

### Test LoRA Adapter with basic_app CLI

Source: https://github.com/leehack/llamadart/blob/main/example/training_notebook/lora_training.ipynb

Use the basic_app CLI to test your trained LoRA adapter. This command assumes the default Qwen2.5-0.5B base model, ensuring compatibility.

```bash
dart run bin/llamadart_basic_example.dart --lora path/to/my_adapter.gguf
```

--------------------------------

### Clear Device Logs

Source: https://github.com/leehack/llamadart/blob/main/doc/android_runtime_smoke_test_plan.md

Clear the log buffer on the Android device before starting a test.

```bash
adb logcat -c
```

--------------------------------

### Initialize and Generate Text with LlamaEngine

Source: https://github.com/leehack/llamadart/blob/main/README.md

Demonstrates basic text generation using the default LlamaBackend. Ensure the model path is correct and always dispose of the engine to free resources.

```dart
import 'package:llamadart/llamadart.dart';

void main() async {
  // Automatically selects Native or Web backend
  final engine = LlamaEngine(LlamaBackend());

  try {
    // Initialize with a local GGUF model
    await engine.loadModel('path/to/model.gguf');

    // Generate text (streaming)
    await for (final token in engine.generate('The capital of France is')) {
      print(token);
    }
  } finally {
    // CRITICAL: Always dispose the engine to release native resources
    await engine.dispose();
  }
}
```

--------------------------------

### Generate Embeddings

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_server/README.md

Create embeddings for a list of input texts. The `encoding_format` can be specified, for example, as `float`.

```bash
curl http://127.0.0.1:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llamadart-local",
    "input": [
      "llamadart supports local inference.",
      "Embeddings are useful for semantic search."
    ],
    "encoding_format": "float"
  }'
```

--------------------------------

### Build llama.cpp with Server Enabled

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_cli/README.md

Compiles the llama.cpp project, enabling the llama-server target for tool-call parity testing. Metal GPU support is included.

```bash
mkdir -p .parity_tools
git clone https://github.com/ggml-org/llama.cpp .parity_tools/llama.cpp
cmake -S .parity_tools/llama.cpp -B .parity_tools/llama.cpp/build \
  -DBUILD_SHARED_LIBS=OFF -DGGML_METAL=ON -DLLAMA_BUILD_SERVER=ON
cmake --build .parity_tools/llama.cpp/build --config Release -j --target llama-server
```

--------------------------------

### Get OpenAPI JSON

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_server/README.md

Retrieve the OpenAPI specification for the API. Useful for understanding the available endpoints and request/response formats.

```bash
curl http://127.0.0.1:8080/openapi.json
```

--------------------------------

### Run Local-Only E2E Tests

Source: https://github.com/leehack/llamadart/blob/main/CONTRIBUTING.md

Execute end-to-end tests that are tagged as 'local-only'. Ensure you have the necessary setup for these tests.

```bash
dart test --run-skipped -t local-only
```

--------------------------------

### Initialize LlamaBackend and LlamaEngine

Source: https://context7.com/leehack/llamadart/llms.txt

Use `LlamaBackend()` to automatically select the appropriate backend (native or web). This backend is then passed to `LlamaEngine` for model loading and inference. Ensure `engine.dispose()` is called to release native resources.

```dart
import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  // LlamaBackend() automatically resolves to native or web backend
  final backend = LlamaBackend();

  final engine = LlamaEngine(backend);
  try {
    await engine.loadModel('path/to/model.gguf');
    print('Backend: ${await engine.getBackendName()}');
    // => "Metal", "Vulkan", "CPU", "WebGPU", etc.
  } finally {
    await engine.dispose();
  }
}
```

--------------------------------

### API Request with Bearer Token

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_server/README.md

Authenticate API requests using a Bearer token. This is required if the server is started with the `--api-key` option.

```bash
curl http://127.0.0.1:8080/v1/models \
  -H "Authorization: Bearer YOUR_KEY"
```

--------------------------------

### Run with Local Model Path

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_cli/README.md

Launches the llamadart CLI using a model file located on the local filesystem, with specified context size and fit option.

```bash
dart run bin/llamadart_cli.dart \
  --model /path/to/GLM-4.7-Flash-UD-Q4_K_XL.gguf \
  --ctx-size 16384 --fit on --jinja
```

--------------------------------

### Run with Prompt from File

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_cli/README.md

Executes the llamadart CLI using a local model and a prompt specified in a text file, with simple I/O mode enabled.

```bash
dart run bin/llamadart_cli.dart \
  --model ./models/GLM-4.7-Flash-UD-Q4_K_XL.gguf \
  --file ./tool/parity_prompts/flappy_bird.txt \
  --simple-io
```

--------------------------------

### Add Dependency Libraries and Include Directories

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/windows/runner/CMakeLists.txt

Links necessary libraries (like flutter, flutter_wrapper_app, and dwmapi.lib) and specifies include directories for the application. Add any custom application-specific dependencies here.

```cmake
# Add dependency libraries and include directories. Add any application-specific
# dependencies here.
target_link_libraries(${BINARY_NAME} PRIVATE flutter flutter_wrapper_app)
target_link_libraries(${BINARY_NAME} PRIVATE "dwmapi.lib")
target_include_directories(${BINARY_NAME} PRIVATE "${CMAKE_SOURCE_DIR}")
```

--------------------------------

### Initialize LlamaDart Web Bridge Runtime

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/web/index.html

Logs the initialization of the LlamaDart web bridge runtime with various configuration parameters. This is used when the bridge is not a mock scenario.

```javascript
bridgeBootstrapLog( 'info', 
`llamadart: Web bridge runtime loaded (${source}), runtimeAssets=${source === 'cdn' ? 'local' : source}, worker=${workerModuleUrl ?? 'auto'}, mem64=${coreModuleUrlMem64 ?? 'none'}, preferMem64=${preferMemory64}, threadPool=${bridgeThreadPoolSize}, utf8Hotfix=on
`, );
```

--------------------------------

### Clone Native Build Repository

Source: https://github.com/leehack/llamadart/blob/main/CONTRIBUTING.md

Clone the native build repository to prepare for building binaries. Ensure submodules are initialized.

```bash
git clone https://github.com/leehack/llamadart-native.git
cd llamadart-native
git submodule update --init --recursive
```

--------------------------------

### Run Android Runtime Smoke Test Script

Source: https://github.com/leehack/llamadart/blob/main/doc/android_runtime_smoke_test_plan.md

Execute the helper script to automate the build, install, and log capture process for runtime smoke tests.

```bash
./scripts/android_runtime_smoke.sh --app-id com.example.chat_app
```

--------------------------------

### Dart Function for Summation

Source: https://github.com/leehack/llamadart/blob/main/tool/testing/prompts/native_prompt_reuse_parity_prompts.txt

A simple Dart function named 'add' that takes two integers and returns their sum. This is a basic example of function definition in Dart.

```dart
int add(int a, int b) {
  return a + b;
}
```

--------------------------------

### Install Native Assets

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/linux/CMakeLists.txt

Copies native assets provided by the build.dart script from all packages into the application's library directory. This ensures native resources are correctly bundled.

```cmake
# Copy the native assets provided by the build.dart from all packages.
set(NATIVE_ASSETS_DIR "${PROJECT_BUILD_DIR}native_assets/linux/")
install(DIRECTORY "${NATIVE_ASSETS_DIR}"
   DESTINATION "${INSTALL_BUNDLE_LIB_DIR}"
   COMPONENT Runtime)
```

--------------------------------

### Build llama.cpp for Parity Testing

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_cli/README.md

Compiles the llama.cpp project, including the llama-cli target, for use in parity testing. Metal GPU support is enabled.

```bash
mkdir -p .parity_tools
git clone https://github.com/ggml-org/llama.cpp .parity_tools/llama.cpp
cmake -S .parity_tools/llama.cpp -B .parity_tools/llama.cpp/build \
  -DBUILD_SHARED_LIBS=OFF -DGGML_METAL=ON
cmake --build .parity_tools/llama.cpp/build --config Release -j --target llama-cli
```

--------------------------------

### Minimal Model Load and Generation

Source: https://github.com/leehack/llamadart/blob/main/README.md

Load a GGUF model and generate text using the LlamaEngine. Ensure to dispose of the engine after use. This example demonstrates basic text generation.

```dart
import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  final engine = LlamaEngine(LlamaBackend());
  try {
    await engine.loadModel('path/to/model.gguf');
    await for (final token in engine.generate('Hello')) {
      print(token);
    }
  } finally {
    await engine.dispose();
  }
}
```

--------------------------------

### Load Llama Model with GPU Offloading

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/README.md

Demonstrates loading a Llama model with specified parameters, including GPU layer offloading and context size. Ensure the model path and optional multimodal projector path are correctly set.

```dart
final engine = LlamaEngine(LlamaBackend());
await engine.loadModel(
  modelPath,
  modelParams: ModelParams(
    gpuLayers: 99, // Offload all layers for best performance on GPU
    contextSize: 2048,
    preferredBackend: GpuBackend.vulkan,
  ),
);

// Optional: Load multimodal projector
if (mmprojPath != null) {
  await engine.loadMultimodalProjector(mmprojPath);
}
```

--------------------------------

### Catching LlamaException Subtypes

Source: https://context7.com/leehack/llamadart/llms.txt

This example shows how to use try-catch blocks to handle different LlamaException subtypes, such as LlamaModelException, LlamaContextException, LlamaStateException, and LlamaUnsupportedException. Ensure the LlamaEngine is disposed in a finally block.

```dart
import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  final engine = LlamaEngine(LlamaBackend());

  try {
    await engine.loadModel('nonexistent.gguf');
  } on LlamaModelException catch (e) {
    print('Model load failed: ${e.message}'); // file not found, bad format, etc.
    print('Details: ${e.details}');
  }

  try {
    // Using engine before loadModel
    await engine.getContextSize();
  } on LlamaContextException catch (e) {
    print('Context error: ${e.message}'); // "Engine not ready"
  }

  try {
    await engine.loadModel('models/model.gguf');
    await engine.loadModel('models/other.gguf'); // second load without unload
  } on LlamaStateException catch (e) {
    print('State error: ${e.message}'); // "Model is already loaded"
  }

  try {
    await engine.loadModel('models/text-model.gguf');
    await engine.embed('hello'); // text model doesn't support embeddings
  } on LlamaUnsupportedException catch (e) {
    print('Unsupported: ${e.message}'); // "Embeddings are not supported"
  } finally {
    await engine.dispose();
  }
}
```

--------------------------------

### Fetch WebGPU Bridge Assets

Source: https://github.com/leehack/llamadart/blob/main/doc/webgpu_bridge.md

Vendors pinned assets into local app web files. Requires setting the WEBGPU_BRIDGE_ASSETS_TAG environment variable.

```bash
WEBGPU_BRIDGE_ASSETS_TAG=v0.1.10 ./scripts/fetch_webgpu_bridge_assets.sh
```

--------------------------------

### Build Bundled llamadart_cli Binary

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_cli/README.md

Creates a standalone, bundled executable of the llamadart CLI, which can be used to avoid 'dart run' overhead.

```bash
dart build cli --output .parity_tools/build_cli
```

--------------------------------

### Multimodal Model Inference with LlamaDart

Source: https://github.com/leehack/llamadart/blob/main/README.md

Example of loading a vision model and its projector, then performing a multimodal inference request using `LlamaChatMessage.withContent`. Handles potential exceptions and ensures engine disposal.

```dart
import 'package:llamadart/llamadart.dart';

void main() async {
  final engine = LlamaEngine(LlamaBackend());
  
  try {
    await engine.loadModel('vision-model.gguf');
    await engine.loadMultimodalProjector('mmproj.gguf');

    final session = ChatSession(engine);

    // Create a multimodal message
    final messages = [
      LlamaChatMessage.withContent(
        role: LlamaChatRole.user,
        content: [
          LlamaImageContent(path: 'image.jpg'),
          LlamaTextContent('What is in this image?'),
        ],
      ),
    ];

    // Use stateless engine.create for one-off multimodal requests
    final response = engine.create(messages);
    await for (final chunk in response) {
      stdout.write(chunk.choices.first.delta.content ?? '');
    }
  } finally {
    await engine.dispose();
  }
}
```

--------------------------------

### Configure Compact Baseline Packaging for Android arm64

Source: https://github.com/leehack/llamadart/blob/main/MIGRATION.md

Use this YAML configuration to preserve compact baseline-style packaging for Android arm64. This is useful when you need smaller packaging sizes.

```yaml
hooks:
  user_defines:
    llamadart:
      llamadart_native_backends:
        platforms:
          android-arm64:
            backends: [vulkan]
            cpu_profile: compact
```

--------------------------------

### Chat with Tool Calling using ChatSession

Source: https://context7.com/leehack/llamadart/llms.txt

Shows how ChatSession handles tool calls. After a tool call chunk is detected, invoke the tool, add the result as a tool message, and call session.create([]) again to get the final synthesized response.

```dart
import 'dart:io';
import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  final engine = LlamaEngine(LlamaBackend());
  await engine.loadModel('models/hermes-qwen2.gguf');

  final session = ChatSession(engine);

  final tools = [
    ToolDefinition(
      name: 'get_weather',
      description: 'Get current weather for a city',
      parameters: [
        ToolParam.string('city', description: 'City name', required: true),
        ToolParam.enumType('unit', values: ['celsius', 'fahrenheit']),
      ],
      handler: (params) async {
        final city = params.getRequiredString('city');
        final unit = params.getString('unit') ?? 'celsius';
        return {'temperature': 22, 'unit': unit, 'condition': 'sunny', 'city': city};
      },
    ),
  ];

  // First turn: model may call a tool
  LlamaToolCallContent? toolCall;
  await for (final chunk in session.create(
    [LlamaTextContent("What's the weather in Tokyo?")],
    tools: tools,
    toolChoice: ToolChoice.auto,
  )) {
    final delta = chunk.choices.first.delta;
    if (delta.content != null) stdout.write(delta.content);
    if (delta.toolCalls != null && toolCall == null) {
      // tool call detected; session already appended assistant msg with tool call
      final tc = session.history.last.parts.whereType<LlamaToolCallContent>().firstOrNull;
      toolCall = tc;
    }
  }

  if (toolCall != null) {
    // Execute the tool
    final result = await toolCall.arguments['city'] != null
        ? tools.first.invoke(toolCall.arguments)
        : null;

    // Add tool result to history
    session.addMessage(LlamaChatMessage.withContent(
      role: LlamaChatRole.tool,
      content: [
        LlamaToolResultContent(
          id: toolCall.id,
          name: toolCall.name,
          result: result,
        ),
      ],
    ));

    // Second turn: model synthesizes final answer
    await for (final chunk in session.create([])) {
      stdout.write(chunk.choices.first.delta.content ?? '');
    }
  }

  await engine.dispose();
}
```

--------------------------------

### LlamaBackend() - Platform-Agnostic Backend Factory

Source: https://context7.com/leehack/llamadart/llms.txt

The `LlamaBackend()` factory constructor automatically selects the appropriate backend implementation for the current platform, whether it's a native llama.cpp isolate-based backend or a WebGPU bridge backend for web applications. This backend is then passed to the `LlamaEngine`.

```APIDOC
## LlamaBackend()

### Description
A factory constructor that automatically selects the correct backend implementation for the current platform.

### Usage
```dart
import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  final backend = LlamaBackend();
  final engine = LlamaEngine(backend);
  try {
    await engine.loadModel('path/to/model.gguf');
    print('Backend: ${await engine.getBackendName()}');
  } finally {
    await engine.dispose();
  }
}
```
```

--------------------------------

### Correct Import Order in Dart

Source: https://github.com/leehack/llamadart/blob/main/AGENTS.md

Demonstrates the correct order for importing Dart libraries, packages, and local files. Follow this order for consistency and maintainability.

```dart
// Correct import order:
import 'dart:async';
import 'dart:io';

import 'package:flutter/material.dart';

import '../core/engine/engine.dart';
import '../backends/backend.dart';

```

--------------------------------

### Apply Standard Build Settings

Source: https://github.com/leehack/llamadart/blob/main/example/chat_app/windows/runner/CMakeLists.txt

Applies a predefined set of standard build settings to the application target. This can be removed if custom build configurations are required.

```cmake
# Apply the standard set of build settings. This can be removed for applications
# that need different build settings.
apply_standard_settings(${BINARY_NAME})
```

--------------------------------

### List Available Models

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_server/README.md

Fetch a list of models available on the server. This endpoint is useful for verifying server status and model availability.

```bash
curl http://127.0.0.1:8080/v1/models
```

--------------------------------

### Run Tool-Call Parity Report

Source: https://github.com/leehack/llamadart/blob/main/example/llamadart_cli/README.md

Generates a tool-call parity report by comparing llama.cpp's llama-server with the llamadart-backed API server. Requires paths to the llama-server binary and the API server entry point.

```bash
dart run tool/tool_call_parity.dart \
  --model ./models/GLM-4.7-Flash-UD-Q4_K_XL.gguf \
  --llama-server-path ./.parity_tools/llama.cpp/build/bin/llama-server \
  --api-server-entry ../llamadart_server/bin/llamadart_server.dart
```

--------------------------------

### Configure Hugging Face Static Spaces Headers

Source: https://github.com/leehack/llamadart/blob/main/doc/webgpu_bridge.md

Set custom headers in Space README frontmatter for cross-origin isolation. Header keys and values must be lowercase.

```yaml
custom_headers:
  cross-origin-embedder-policy: require-corp
  cross-origin-opener-policy: same-origin
  cross-origin-resource-policy: cross-origin
```