### Run Example from Examples Directory

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Navigates into the 'examples/basic' directory and runs the 'generate-text' example using pnpm. This is an alternative way to run examples.

```bash
cd examples/basic
pnpm generate-text
```

--------------------------------

### Minimal TypeScript Setup

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Basic setup for using the llama-cpp-provider with a model path. Imports the necessary function.

```typescript
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./model.gguf",
});
```

--------------------------------

### Example LlamaCppModelMemoryInfo Configuration

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/types.md

An example demonstrating how to configure LlamaCppModelMemoryInfo, including max context size and KV cache layer details.

```typescript
const memory: LlamaCppModelMemoryInfo = {
  maxContextSize: 262144,
  kvCache: {
    bytesPerValue: 2,
    layers: [
      { count: 32, keyValueHeads: 8, headDim: 128 },
      { count: 8, keyHeads: 16, valueHeads: 8, keyHeadDim: 64, valueHeadDim: 64 },
    ],
  },
};
```

--------------------------------

### Run Examples with pnpm Workspace Filter

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/AGENTS.md

Execute specific examples within the 'examples/basic' workspace package using pnpm's filter command. This is useful for running individual example scripts.

```bash
pnpm --filter @examples/basic generate-text
```

```bash
pnpm --filter @examples/basic stream-text
```

```bash
pnpm --filter @examples/basic generate-text-output
```

```bash
pnpm --filter @examples/basic chatbot
```

```bash
pnpm --filter @examples/basic embed-many
```

--------------------------------

### Run Example: Generate Text

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Executes the 'generate-text' example using pnpm, filtering for the '@examples/basic' package. This demonstrates basic text generation capabilities.

```bash
pnpm --filter @examples/basic generate-text
```

--------------------------------

### Type-Safe TypeScript Setup

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Setup using type safety by importing configuration types. Ensures correct parameter usage.

```typescript
import {
  llamaCpp,
  type LlamaCppProviderConfig,
  type LlamaCppModelConfig,
} from "@lgrammel/llama-cpp-provider";

const config: LlamaCppProviderConfig = {
  modelPath: "./model.gguf",
  contextSize: 4096,
};

const model = llamaCpp(config);
```

--------------------------------

### Run Example: Generate Text with Tool Call

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Executes the 'generate-text-tool-call' example using pnpm, filtering for the '@examples/basic' package. This demonstrates text generation with tool call capabilities.

```bash
pnpm --filter @examples/basic generate-text-tool-call
```

--------------------------------

### Run Example: Stream Text

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Executes the 'stream-text' example using pnpm, filtering for the '@examples/basic' package. This demonstrates streaming text generation.

```bash
pnpm --filter @examples/basic stream-text
```

--------------------------------

### Clone Repository and Install Dependencies

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Clones the project repository and installs all project dependencies, including building the native addon. This is the primary step for setting up the development environment.

```bash
git clone https://github.com/lgrammel/ai-sdk-llama-cpp.git
cd ai-sdk-llama-cpp
pnpm install
```

--------------------------------

### Install clang-format

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Installs clang-format, a tool for C++ code formatting, via Homebrew. This is optional but recommended for maintaining code style.

```bash
brew install clang-format
```

--------------------------------

### Complete Llama.cpp Provider Configuration Example

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

A comprehensive example demonstrating the configuration of Llama.cpp provider, including model path, optional loading parameters, model-specific settings, memory safety, caching, and debugging options.

```typescript
import {
  llamaCpp,
  gemma4_31b_it,
  thinkTagsReasoning,
} from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  // Required
  modelPath: "/home/user/models/gemma-4-31b-it.gguf",

  // Optional model loading
  mmprojPath: "/home/user/models/mmproj.gguf",
  contextSize: 50000,
  gpuLayers: 99,
  threads: 8,

  // Optional model-specific config
  model: {
    ...gemma4_31b_it,
    chatTemplate: "gemma",
    reasoning: thinkTagsReasoning,
  },

  // Optional memory safety
  memorySafety: {
    mode: "clamp",
    memoryUtilization: 0.8,
    reserveMemoryBytes: 4 * 1024 ** 3,
  },

  // Optional caching
  cache: {
    mode: "prefix",
  },

  // Optional debugging
  debug: false,
  logPrompts: false,
});

// Use model...

await model.dispose();
```

--------------------------------

### Install llama-cpp-provider

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/README.md

Install the package using npm. This command downloads and builds the llama.cpp library with Metal support and compiles the native Node.js addon.

```bash
npm install @lgrammel/llama-cpp-provider
```

--------------------------------

### Install Xcode Command Line Tools

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Installs the necessary command-line developer tools for macOS. This is a prerequisite for building native addons.

```bash
xcode-select --install
```

--------------------------------

### Example Template with Streaming and Schema

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/AGENTS.md

A template for creating new examples, including options for streaming text and defining output schemas. It emphasizes using try/finally for resource management and setting the correct model path.

```typescript
import { generateText, streamText, Output } from "ai";
import { z } from "zod";
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./models/your-model.gguf",
 contextSize: 4096, // optional, tune for your machine memory
  model: {
  },
});

try {
  // Your example code here
  const { text } = await generateText({
    model,
    prompt: "Hello, world!",
    maxTokens: 100,
  });
  console.log(text);
} finally {
  await model.dispose();
}
```

--------------------------------

### Install Xcode Command Line Tools and CMake

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/README.md

Install necessary development tools for building the native addon. Xcode Command Line Tools are required for compilation, and CMake is needed for the build system.

```bash
xcode-select --install
brew install cmake
```

--------------------------------

### Build Native Addon

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/AGENTS.md

Builds the native addon for the llama-cpp-provider package. This command is used when dependencies are already installed and only the native component needs to be rebuilt, for example, after updating llama.cpp.

```bash
pnpm build:native
```

--------------------------------

### Basic Text Generation Example

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/AGENTS.md

Demonstrates how to use the `llamaCpp` provider with the AI SDK's `generateText` function. Ensure the model path is correctly set and always dispose of the model after use.

```typescript
import { generateText } from "ai";
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

// Create model instance with config
const model = llamaCpp({
  modelPath: "./models/your-model.gguf",
 // Optional for image inputs with multimodal models:
 // mmprojPath: "./models/your-mmproj.gguf",
 // Optional load config: contextSize, gpuLayers, threads, debug
 // Optional model info: model.chatTemplate, model.reasoning
});

try {
  // Use with AI SDK functions
  const result = await generateText({
    model,
    prompt: "Your prompt here",
  });

  console.log(result.text);
} finally {
  // Always dispose to free resources
  await model.dispose();
}
```

--------------------------------

### Quick Start: Initialize Model and Generate Text

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/README.md

Initializes the LlamaCpp provider with model path, context size, and GPU layers, then performs text generation. Ensure to import necessary components and dispose of the model when done.

```typescript
import { llamaCpp, gemma4_31b_it } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./models/gemma-4-31b-it.gguf",
  contextSize: 50000,
  gpuLayers: 99,
  model: gemma4_31b_it,
});

const result = await model.doGenerate({
  prompt: [
    { role: "user", content: [{ type: "text", text: "Hello!" }] },
  ],
  maxOutputTokens: 100,
});

console.log(result.content);

await model.dispose();
```

--------------------------------

### TypeScript Setup with Presets

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Utilizes pre-defined configuration presets for Gemma 4 models, including reasoning settings. Imports necessary presets and types.

```typescript
import {
  llamaCpp,
  gemma4_31b_it,
  gemma4Reasoning,
  type LlamaCppProviderConfig,
} from "@lgrammel/llama-cpp-provider";

const config: LlamaCppProviderConfig = {
  modelPath: "./gemma-4-31b.gguf",
  model: {
    ...gemma4_31b_it,
    reasoning: gemma4Reasoning,
  },
};

const model = llamaCpp(config);
```

--------------------------------

### Example GBNF Grammar Output

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/schema-converter.md

Illustrates the GBNF grammar generated for a simple person schema.

```plaintext
root ::= object
object ::= "{" space "name" space ":" space string space "," space "age" space ":" space integer space "}"
string ::= "\"" [^"\\]* "\""
integer ::= [0-9]+
space ::= | " " | "\n" | "\t"
```

--------------------------------

### Install pnpm

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Installs pnpm, a performant Node.js package manager, globally using npm. pnpm is used for managing project dependencies.

```bash
npm install -g pnpm
```

--------------------------------

### Install CMake via Homebrew

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Installs CMake, a build system generator, using the Homebrew package manager. CMake is required for compiling native code.

```bash
brew install cmake
```

--------------------------------

### Llama.cpp Build Options

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Configures build options for llama.cpp, such as building it as a static library and disabling tests, examples, and the server.

```cmake
# Build llama.cpp as a static library
set(LLAMA_STATIC ON CACHE BOOL "Build llama.cpp as static library" FORCE)
set(LLAMA_BUILD_TESTS OFF CACHE BOOL "Disable llama.cpp tests" FORCE)
set(LLAMA_BUILD_EXAMPLES OFF CACHE BOOL "Disable llama.cpp examples" FORCE)
set(LLAMA_BUILD_SERVER OFF CACHE BOOL "Disable llama.cpp server" FORCE)
```

--------------------------------

### TypeScript Memory Utilities Example

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Demonstrates memory utilities for estimating memory usage and checking memory safety with Gemma 4 presets. Imports relevant functions and types.

```typescript
import {
  estimateMemoryUsage,
  checkMemorySafety,
  gemma4_31b_it,
  type MemoryUsageEstimate,
} from "@lgrammel/llama-cpp-provider";

// Estimate memory for configuration
const estimate: MemoryUsageEstimate = estimateMemoryUsage({
  model: gemma4_31b_it.memory!,
  contextSize: 50000,
  modelFileSizeBytes: 7 * 1024 ** 3,
});

console.log(`Total: ${estimate.totalBytes / (1024 ** 3)} GiB`);

// Check if configuration is safe
const check = checkMemorySafety({
  model: gemma4_31b_it.memory,
  contextSize: 50000,
  memorySafety: { mode: "clamp" },
});

console.log(`Safe context: ${check.contextSize}`);
```

--------------------------------

### High-Level API Usage Example

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/native-binding.md

Demonstrates how to use the high-level API for text generation, which handles native binding details internally. Ensure the model path is correctly specified.

```typescript
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

// High-level API - handles binding internally
const model = llamaCpp({ modelPath: "./model.gguf" });
const result = await model.doGenerate({
  prompt: [...],
});
// model.dispose() calls unloadModel() internally
await model.dispose();
```

--------------------------------

### Configure Compute Overhead Bytes

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

Set extra memory for compute buffers and Metal allocations on GPU. Example sets 2 GiB overhead.

```typescript
const model = llamaCpp({
  modelPath: "./model.gguf",
  memorySafety: {
    computeOverheadBytes: 2 * 1024 ** 3,  // 2 GiB overhead
  },
});
```

--------------------------------

### Configure Memory Safety Options during Model Creation

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/memory-estimation.md

Example of configuring memory safety options, including mode, memory utilization, and reserved memory, when creating a Llama.cpp model instance.

```typescript
import { llamaCpp, gemma4_31b_it } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./models/gemma-4-31B.gguf",
  contextSize: 50000,
  model: gemma4_31b_it,
  memorySafety: {
    mode: "clamp",  // Automatically reduce context size if needed
    memoryUtilization: 0.8,  // Use at most 80% of available memory
    reserveMemoryBytes: 2 * 1024 ** 3,  // Keep 2 GiB free
  },
});
```

--------------------------------

### buildToolSystemPrompt

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/language-model.md

Generates a system prompt that instructs the language model on how to utilize provided tools. This is essential for enabling the model to interact with external functionalities.

```APIDOC
## buildToolSystemPrompt(tools)

### Description
Generates a system prompt instructing the model how to call tools.

### Method
`buildToolSystemPrompt`

### Parameters
#### Path Parameters
- **tools** (LanguageModelV4FunctionTool[]) - Required - Available tools

### Returns
`string` — System prompt text.
```

--------------------------------

### Build Tool System Prompt

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Constructs the system prompt instructions for enabling tool usage by the model.

```typescript
// Build tool instructions
buildToolSystemPrompt()
```

--------------------------------

### Build TypeScript

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Compiles the TypeScript code into JavaScript. This command is part of the development setup and can be run independently.

```bash
pnpm build:ts
```

--------------------------------

### Import Package Entry Point

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Import all necessary modules and types from the main package entry point. This includes the provider factory, configuration types, model presets, language and embedding model classes, memory utilities, schema conversion tools, and re-exports from the AI SDK.

```typescript
import {
  llamaCpp,
  // Embedding factory
  llamaCpp.embedding,
  
  // Configuration types
  type LlamaCppProviderConfig,
  type LlamaCppModelInfo,
  type LlamaCppReasoningConfig,
  type LlamaCppMemorySafetyConfig,
  type LlamaCppCacheConfig,
  type LlamaCppModelMemoryInfo,
  type LlamaCppKvCacheLayerMemoryInfo,
  
  // Preset configurations
  gemma4_31b_it,
  gemma4_26b_a4b,
  gemma4Reasoning,
  qwen3_6_dense,
  qwen3_6_moe,
  thinkTagsReasoning,
  
  // Language model class and types
  LlamaCppLanguageModel,
  type LlamaCppModelConfig,
  type LlamaCppGenerationConfig,
  convertMessages,
  convertFinishReason,
  convertUsage,
  resolveReasoningConfig,
  splitReasoningContent,
  generateToolCallGrammar,
  parseToolCalls,
  buildToolSystemPrompt,
  type ParsedToolCall,
  type ParsedReasoningPart,
  
  // Embedding model class
  LlamaCppEmbeddingModel,
  
  // Memory utilities
  checkMemorySafety,
  estimateMemoryUsage,
  type EstimateMemoryUsageOptions,
  type MemorySafetyCheckOptions,
  type MemorySafetyCheckResult,
  type MemoryUsageEstimate,
  
  // Schema conversion
  convertJsonSchemaToGrammar,
  SchemaConverter,
  type SchemaConverterOptions,
  
  // Re-export from AI SDK
  type JSONSchema7,
} from "@lgrammel/llama-cpp-provider";
```

--------------------------------

### Initialize and Use Llama.cpp Model

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/language-model.md

Demonstrates how to initialize a Llama.cpp model and properly dispose of it to free resources. Always call `dispose()` when finished.

```typescript
const model = llamaCpp({ modelPath: "./model.gguf" });
// ... use model ...
await model.dispose();
```

--------------------------------

### Default Export Usage

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/provider.md

The `llamaCpp` provider can be imported as the default export from the package. This example shows a basic instantiation with a model path.

```typescript
import llamaCpp from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({ modelPath: "./model.gguf" });
```

--------------------------------

### Configure llamaCpp Language Model

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/README.md

Create a language model instance using `llamaCpp`, specifying various configuration options for model loading and behavior. Key options include model and projector paths, context size, and GPU layer offloading.

```typescript
import { ToolLoopAgent } from "ai";
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./models/your-model.gguf",
  mmprojPath: "./models/mmproj.gguf",
  contextSize: 4096,
  gpuLayers: 99,
  threads: 8,
  debug: false,
  logPrompts: false,
  model: {
    chatTemplate: "auto",
    reasoning: {},
  },
});

const agent = new ToolLoopAgent({ model });
```

--------------------------------

### llamaCpp(config)

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/README.md

Creates an AI SDK language model for ToolLoopAgent and other AI SDK consumers. It runs local GGUF models through native C++ bindings.

```APIDOC
## llamaCpp(config)

### Description
Creates an AI SDK language model for `ToolLoopAgent` and other AI SDK consumers. It runs local GGUF models through native C++ bindings.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **modelPath** (string) - Required - Path to a local GGUF model file.
- **mmprojPath** (string) - Optional - Path to a multimodal projector GGUF file, required for image inputs.
- **contextSize** (number) - Optional - Defaults to `2048`. Higher values can use significant memory.
- **gpuLayers** (number) - Optional - Defaults to `99`, offloads all available layers to GPU. Use `0` to disable GPU offload.
- **threads** (number) - Optional - Defaults to `4`.
- **debug** (boolean) - Optional - Enables verbose llama.cpp output.
- **logPrompts** (boolean) - Optional - Prints the final chat-template-rendered prompt sent to llama.cpp to stderr. Intended for local debugging only.
- **model** (object) - Optional - Configuration for the model.
  - **chatTemplate** (string) - Optional - Defaults to `"auto"`. Can be a llama.cpp template name like `"llama3"`, `"chatml"`, or `"gemma"`.
  - **reasoning** (object) - Optional - Extracts thinking text into AI SDK reasoning parts.
- **memorySafety** (boolean) - Optional - Can reject or clamp context sizes that are estimated to exceed available memory when model memory metadata is provided.

Standard AI SDK generation settings are supported, including `maxOutputTokens`, `temperature`, `topP`, `topK`, and `stopSequences`.

### Request Example
```typescript
import { ToolLoopAgent } from "ai";
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./models/your-model.gguf",
  mmprojPath: "./models/mmproj.gguf",
  contextSize: 4096,
  gpuLayers: 99,
  threads: 8,
  debug: false,
  logPrompts: false,
  model: {
    chatTemplate: "auto",
    reasoning: {},
  },
});

const agent = new ToolLoopAgent({ model });
```

### Response
#### Success Response (200)
An AI SDK language model instance.

#### Response Example
(No specific response example provided in source, but it returns a model object compatible with AI SDK)
```

--------------------------------

### Login to npm

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Before publishing, ensure you are logged into your npm account using this command.

```bash
npm login
```

--------------------------------

### Create and Push Git Tag

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

After publishing to npm, create a Git tag for the current version and push it to the remote repository.

```bash
git tag v$(node -p "require('./packages/llama-cpp-provider/package.json').version")
git push --tags
```

--------------------------------

### Use Preset Reasoning Configuration

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

Utilize predefined reasoning configurations, such as thinkTagsReasoning, for convenience.

```typescript
// Use preset configuration:
import { thinkTagsReasoning } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./model.gguf",
  model: {
    reasoning: thinkTagsReasoning,
  },
});
```

--------------------------------

### Distinguish Between Different Error Types

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/errors.md

Differentiate between various error messages during model loading to provide specific user feedback. This example distinguishes between file-not-found errors and memory-related issues.

```typescript
try {
  const model = llamaCpp({ modelPath, contextSize: 100000 });
} catch (error) {
  const message = error instanceof Error ? error.message : String(error);

  if (message.includes("does not exist")) {
    // File not found - user error
    console.error("Download the model file first");
  } else if (message.includes("memory")) {
    // Memory safety - suggest solutions
    console.error("Reduce contextSize or set memorySafety: { mode: 'clamp' } ");
  } else {
    // Unknown error
    console.error("Unexpected error:", error);
  }
}
```

--------------------------------

### LlamaCppEmbeddingModel Constructor

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/embedding-model.md

Initializes a new instance of the LlamaCppEmbeddingModel class with the specified configuration.

```APIDOC
## Constructor LlamaCppEmbeddingModel

### Description
Initializes a new instance of the LlamaCppEmbeddingModel class.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
#### `config`
- **config** (`LlamaCppProviderConfig`) - Required - Configuration object for the embedding model.

### Request Example
```typescript
import { LlamaCppEmbeddingModel } from "@lgrammel/llama-cpp-provider";

const model = new LlamaCppEmbeddingModel({
  modelPath: "./models/embedding-model.gguf",
  contextSize: 2048,
  gpuLayers: 99,
});
```

### Response
None

### Error Handling
None
```

--------------------------------

### Handle Streaming Generation Errors

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/errors.md

Implement try-catch blocks around streaming generation calls to manage errors that may occur after the stream has started, ensuring robust handling of stream interruptions.

```typescript
try {
  const { stream } = await model.doStream({
    prompt: [...],
  });

  for await (const event of stream) {
    if (event.type === "text-delta") {
      process.stdout.write(event.delta);
    }
  }
} catch (error) {
  console.error("Stream generation failed:", error);
}
```

--------------------------------

### Instantiate LlamaCppEmbeddingModel

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/embedding-model.md

Create an instance of LlamaCppEmbeddingModel with the specified configuration. Ensure the model path points to a valid GGUF file.

```typescript
import { LlamaCppEmbeddingModel } from "@lgrammel/llama-cpp-provider";

const model = new LlamaCppEmbeddingModel({
  modelPath: "./models/embedding-model.gguf",
  contextSize: 2048,
  gpuLayers: 99,
});
```

--------------------------------

### Run All Tests

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/AGENTS.md

Executes all tests in the project, including unit, integration, and end-to-end tests. This is a comprehensive check to ensure the project is functioning correctly.

```bash
pnpm test:run
```

--------------------------------

### Download Model for E2E Tests

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/AGENTS.md

Download a GGUF model file from Hugging Face for use in end-to-end tests. Ensure the 'models' directory exists.

```bash
mkdir -p models
wget -P models/ https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf
```

--------------------------------

### LlamaCppGenerationConfig Interface

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/types.md

Options passed to the native generation layer for controlling output. Includes parameters like max tokens, temperature, topP, topK, and stop sequences.

```typescript
interface LlamaCppGenerationConfig {
  maxTokens?: number;
  temperature?: number;
  topP?: number;
  topK?: number;
  stopSequences?: string[];
}
```

--------------------------------

### Find Node.js and Node-Addon-API

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Includes directories for Node.js and finds the node-addon-api include path using execute_process.

```cmake
# Find Node.js and node-addon-api
include_directories(${CMAKE_JS_INC})

# Find node-addon-api
execute_process(
    COMMAND node -p "require('node-addon-api').include"
    WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
    OUTPUT_VARIABLE NODE_ADDON_API_DIR
    OUTPUT_STRIP_TRAILING_WHITESPACE
)
string(REPLACE "\"" "" NODE_ADDON_API_DIR ${NODE_ADDON_API_DIR})
include_directories(${NODE_ADDON_API_DIR})
```

--------------------------------

### Basic CMake Configuration

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Sets the minimum CMake version, project name, C++ standard, and required standard compliance. Also configures independent code positioning.

```cmake
cmake_minimum_required(VERSION 3.15)
project(llama_binding)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
```

--------------------------------

### Instantiate LlamaCppLanguageModel

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/language-model.md

Create a new instance of the LlamaCppLanguageModel with the specified configuration. Ensure the model path points to a valid GGUF file.

```typescript
import { LlamaCppLanguageModel } from "@lgrammel/llama-cpp-provider";

const model = new LlamaCppLanguageModel({
  modelPath: "./models/llama-3.2-1b.gguf",
  contextSize: 2048,
  gpuLayers: 99,
});
```

--------------------------------

### Classes

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Instantiate these classes directly for more control over model creation and schema conversion.

```APIDOC
## Classes

### `LlamaCppLanguageModel(config)`

**Description**: Represents a language model created via the Llama.cpp provider. Typically instantiated via the `llamaCpp` factory.

### `LlamaCppEmbeddingModel(config)`

**Description**: Represents an embedding model created via the Llama.cpp provider. Typically instantiated via the `llamaCpp.embedding` factory.

### `SchemaConverter(options)`

**Description**: Handles manual schema conversion with specified options.

```

--------------------------------

### Build Tool System Prompt

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/language-model.md

Generates a system prompt for instructing the model on how to call tools. This function takes an array of available tools as input.

```typescript
export function buildToolSystemPrompt(
  tools: LanguageModelV4FunctionTool[]
): string

```

--------------------------------

### Set Up Embedding Model

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Configure and initialize an embedding model using `llamaCpp.embedding`. Ensure the model path is correct and dispose of the model when no longer needed.

```typescript
import {
  llamaCpp,
  type LlamaCppProviderConfig,
} from "@lgrammel/llama-cpp-provider";
import { embedMany } from "ai";

const embeddingConfig: LlamaCppProviderConfig = {
  modelPath: "./embedding-model.gguf",
  gpuLayers: 99,
};

const embeddingModel = llamaCpp.embedding(embeddingConfig);

const { embeddings } = await embedMany({
  model: embeddingModel,
  values: ["text 1", "text 2"],
});

await embeddingModel.dispose();
```

--------------------------------

### llamaCpp(config)

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/README.md

Creates an AI SDK language model for ToolLoopAgent and other AI SDK consumers. This function allows you to configure and load a local GGUF model for inference.

```APIDOC
## llamaCpp(config)

### Description
Creates an AI SDK language model for `ToolLoopAgent` and other AI SDK consumers. This function allows you to configure and load a local GGUF model for inference.

### Method
`llamaCpp(config)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **modelPath** (string) - Required - Path to a local GGUF model file.
- **mmprojPath** (string) - Optional - Path to a multimodal projector GGUF file, required for image inputs.
- **contextSize** (number) - Optional - Defaults to `2048`. Higher values can use significant memory.
- **gpuLayers** (number) - Optional - Defaults to `99`, which offloads all available layers to GPU. Use `0` to disable GPU offload.
- **threads** (number) - Optional - Defaults to `4`.
- **debug** (boolean) - Optional - Enables verbose llama.cpp output.
- **logPrompts** (boolean) - Optional - Prints the final chat-template-rendered prompt sent to llama.cpp to stderr. Intended for local debugging only.
- **model** (object) - Optional - Configuration for the model itself.
  - **chatTemplate** (string) - Optional - Defaults to `"auto"`. Can be set to a llama.cpp template name (e.g., `"llama3"`, `"chatml"`, `"gemma"`).
  - **reasoning** (object) - Optional - Extracts thinking text into AI SDK reasoning parts.
- **memorySafety** (boolean) - Optional - Can reject or clamp context sizes that are estimated to exceed available memory when model memory metadata is provided.

### Request Example
```typescript
import { ToolLoopAgent } from "ai";
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./models/your-model.gguf",
  mmprojPath: "./models/mmproj.gguf",
  contextSize: 4096,
  gpuLayers: 99,
  threads: 8,
  debug: false,
  logPrompts: false,
  model: {
    chatTemplate: "auto",
    reasoning: {},
  },
});

const agent = new ToolLoopAgent({ model });
```

### Response
#### Success Response (200)
- **model** (object) - An AI SDK language model instance.

#### Response Example
(No specific response example provided in source, but the created model object is used by `ToolLoopAgent`)

### Important options:
- `modelPath` is required and must point to a local GGUF model file.
- `mmprojPath` points to a multimodal projector GGUF file and is required for image inputs.
- `contextSize` defaults to `2048`. Higher values can use significant memory.
- `gpuLayers` defaults to `99`, which offloads all available layers to GPU. Use `0` to disable GPU offload.
- `threads` defaults to `4`.
- `debug` enables verbose llama.cpp output.
- `logPrompts` prints the final chat-template-rendered prompt sent to llama.cpp to stderr. It can include private user data and is intended for local debugging only.
- `model.chatTemplate` defaults to `"auto"`, which uses the template embedded in the GGUF file. You can also pass a llama.cpp template name such as `"llama3"`, `"chatml"`, or `"gemma"`.
- `model.reasoning` extracts thinking text into AI SDK reasoning parts.
- `memorySafety` can reject or clamp context sizes that are estimated to exceed available memory when model memory metadata is provided.

Standard AI SDK generation settings are supported by the language model, including `maxOutputTokens`, `temperature`, `topP`, `topK`, and `stopSequences`.
```

--------------------------------

### llamaCpp.languageModel(config)

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/provider.md

Explicitly creates a language model. This is an alias for the `llamaCpp()` function itself, providing a direct way to instantiate a language model.

```APIDOC
## `llamaCpp.languageModel(config)` Method

### Description
Explicitly creates a language model. This is an alias for the `llamaCpp()` function itself.

### Parameters
#### Parameters
- **config** (`LlamaCppProviderConfig`) - Required - Configuration object for the model

### Returns
`LlamaCppLanguageModel` — A language model implementing the AI SDK LanguageModelV4 interface.

### Example
```typescript
const model = llamaCpp.languageModel({
  modelPath: "./models/model.gguf",
});
```
```

--------------------------------

### Generation-Time Options for doGenerate

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

Set generation-time options like max output tokens, temperature, topP, topK, stop sequences, tools, tool choice, response format, and abort signal for the doGenerate function.

```typescript
const result = await model.doGenerate({
  prompt: messages,
  maxOutputTokens: 500,      // Max generation length
  temperature: 0.8,          // Sampling temperature (0–2)
  topP: 0.95,                // Nucleus sampling
  topK: 40,                  // Top-K filtering
  stopSequences: ["\n\nUser:"],  // Stop strings
  tools: toolDefinitions,    // Available tools
  toolChoice: { type: "auto" },   // Tool selection
  responseFormat: {          // Structured output
    type: "json",
    schema: myJsonSchema,
  },
  abortSignal: abortController.signal,  // Cancellation
});
```

--------------------------------

### LlamaCppLanguageModel Constructor

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/language-model.md

Initializes a new instance of the LlamaCppLanguageModel class with the specified configuration.

```APIDOC
## Constructor LlamaCppLanguageModel

### Description
Initializes a new instance of the LlamaCppLanguageModel class with the specified configuration.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
* **config** (`LlamaCppModelConfig`) - Required - Configuration object for the language model.

### Request Example
```typescript
import { LlamaCppLanguageModel } from "@lgrammel/llama-cpp-provider";

const model = new LlamaCppLanguageModel({
  modelPath: "./models/llama-3.2-1b.gguf",
  contextSize: 2048,
  gpuLayers: 99,
});
```

### Response
None

### Error Handling
* `Error` - If model fails to load
* `Error` - If context size exceeds memory safety limits
```

--------------------------------

### Configuration Presets

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Pre-configured settings for various models and reasoning strategies.

```APIDOC
## Configuration Presets

### Gemma 4 Presets

- `gemma4_31b_it`: Model info, reasoning, and memory configuration for Gemma 4 31B IT.
- `gemma4_26b_a4b`: Model info, reasoning, and memory configuration for Gemma 4 26B A4B.
- `gemma4Reasoning`: Reasoning configuration for Gemma 4 models.

### Qwen 3.6 Presets

- `qwen3_6_dense`: Model info and memory configuration for the dense variant of Qwen 3.6.
- `qwen3_6_moe`: Model info and memory configuration for the MoE variant of Qwen 3.6.

### Reasoning Presets

- `thinkTagsReasoning`: Default reasoning configuration using `<think>...</think>` tags.

```

--------------------------------

### Configure Llama CPP Provider with Environment Variables

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

Set model path, context size, and GPU layers using environment variables. Ensure these variables are correctly set before application startup. The `parseInt` function is used to convert string environment variables to numbers.

```typescript
const modelPath = process.env.MODEL_PATH || "./models/default.gguf";
const contextSize = parseInt(process.env.CONTEXT_SIZE || "2048", 10);
const gpuLayers = parseInt(process.env.GPU_LAYERS || "99", 10);

const model = llamaCpp({
  modelPath,
  contextSize,
  gpuLayers,
});
```

--------------------------------

### Create LlamaCpp Language Model

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/provider.md

Creates an AI SDK language model for use with ToolLoopAgent and other AI SDK consumers. Ensure the model path is valid and memory safety checks are considered if enabled.

```typescript
import { llamaCpp } from "@lgrammel/llama-cpp-provider";
import { ToolLoopAgent } from "ai";

const model = llamaCpp({
  modelPath: "./models/llama-3.2-1b-instruct.gguf",
  contextSize: 4096,
  gpuLayers: 99,
  threads: 8,
});

const agent = new ToolLoopAgent({
  model,
  instructions: "You are a concise local assistant.",
});

// Use the model...

await model.dispose();
```

--------------------------------

### Reasoning Presets

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/types.md

Standard preset for reasoning using XML-style thinking tags.

```typescript
export const thinkTagsReasoning: LlamaCppReasoningConfig
```

--------------------------------

### Add a Changeset

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Initiates the process of adding a changeset for versioning and changelog management. This command prompts the user to select a package, choose a semver bump type, and write a summary.

```bash
pnpm changeset
```

--------------------------------

### Version Packages

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Run this command to consume all changesets and update package versions. It updates package.json, CHANGELOG.md, and removes changeset files.

```bash
pnpm changeset:version
```

--------------------------------

### Build Prompt Cache Test Executable

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Defines and builds the 'prompt_cache_test' executable if the LLAMA_PROVIDER_BUILD_TESTS option is enabled.

```cmake
if(LLAMA_PROVIDER_BUILD_TESTS)
    add_executable(prompt_cache_test
        prompt-cache.test.cpp
        prompt-cache.cpp
    )
    target_compile_features(prompt_cache_test PRIVATE cxx_std_17)

    add_test(NAME prompt_cache_test COMMAND prompt_cache_test)
endif()
```

--------------------------------

### Load Model with Preset Memory and Reasoning

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

Load a model using a preset configuration that includes memory and reasoning details, like gemma4_31b_it.

```typescript
import { gemma4_31b_it } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./gemma4_31b.gguf",
  model: gemma4_31b_it,  // Includes memory, reasoning, chatTemplate
});
```

--------------------------------

### Create Language Model

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Use this factory to create a language model instance. Requires a configuration object.

```typescript
// Create language model
const model = llamaCpp(config);
```

--------------------------------

### Set Addon Output Properties

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Configures the output name and extension for the native addon to be 'llama_binding.node'.

```cmake
# Set output name and extension
set_target_properties(${PROJECT_NAME} PROPERTIES
    PREFIX ""
    SUFFIX ".node"
    OUTPUT_NAME "llama_binding"
)
```

--------------------------------

### Publish to npm

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/CONTRIBUTING.md

Build the TypeScript code and publish the package to npm using this command. Ensure you are logged in to npm first.

```bash
pnpm changeset:publish
```

--------------------------------

### Load Qwen 3.6 Model Preset

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

Load a Qwen 3.6 model preset. Ensure the modelPath points to the correct GGUF file for either the dense or MoE variant.

```typescript
import { qwen3_6_dense, qwen3_6_moe } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./qwen-3.6-dense.gguf",
  model: qwen3_6_dense,
});
```

--------------------------------

### LlamaCppReasoningConfig Interface

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/types.md

Configuration for extracting model thinking or reasoning from its output. Allows customization of markers and prompt prefixes.

```typescript
interface LlamaCppReasoningConfig {
  openingMarker?: string;
  closingMarker?: string;
  promptPrefix?: string | false;
}
```

--------------------------------

### convertFinishReason

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/language-model.md

Converts native finish reason strings to AI SDK format. This helps in standardizing the reasons for model completion across different formats.

```APIDOC
## convertFinishReason(reason)

### Description
Converts native finish reason strings to AI SDK format.

### Parameters
#### Path Parameters
- **reason** (string) - Required - Native finish reason ("stop", "length", "error")

### Returns
`{ unified: "stop" | "length" | "other", raw: string }`
```

--------------------------------

### Add Llama.cpp Subdirectory

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Includes the llama.cpp source directory as a subdirectory for building.

```cmake
# Add llama.cpp subdirectory
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp ${CMAKE_CURRENT_BINARY_DIR}/llama.cpp)
```

--------------------------------

### LlamaCppCacheConfig Interface

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/types.md

Defines configuration for prompt caching. Use 'prefix' mode to reuse matching prompt prefixes across requests. Note that prefix caching makes the model stateful and is intended for single-threaded chat loops.

```typescript
interface LlamaCppCacheConfig {
  mode?: "prefix";
}
```

--------------------------------

### Run Interactive Chat with Local Model

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/README.md

Initialize a local llama.cpp model and use it with ToolLoopAgent for interactive chat. Ensure to call `dispose()` on the model when done to release resources.

```typescript
import { runAgentTUI } from "@lgrammel/agent-tui";
import { ToolLoopAgent } from "ai";
import { llamaCpp } from "@lgrammel/llama-cpp-provider";

const model = llamaCpp({
  modelPath: "./models/llama-3.2-1b-instruct.Q4_K_M.gguf",
});

const agent = new ToolLoopAgent({
  model,
  instructions: "You are a concise local assistant.",
});

try {
  await runAgentTUI({ name: "Local assistant", agent });
} finally {
  await model.dispose();
}
```

--------------------------------

### Include Llama.cpp Headers

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Adds include directories for llama.cpp and its ggml submodule.

```cmake
# Include llama.cpp headers
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/include)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/ggml/include)
```

--------------------------------

### Instantiate SchemaConverter Class

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/MODULES.md

Instantiate the SchemaConverter class for manual schema conversions. Options can be provided.

```typescript
// Manual schema conversion
new SchemaConverter(options)
```

--------------------------------

### loadModel(options): Promise<number>

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/api-reference/native-binding.md

Asynchronously loads a GGUF model into memory and returns a handle. It includes parameters for model path, GPU layers, context size, threads, and more. Path validation is performed before passing to the native binding.

```APIDOC
## loadModel(options): Promise<number>

### Description
Asynchronously loads a GGUF model into memory and returns a handle. This function is used to prepare a model for subsequent operations.

### Method
`loadModel`

### Parameters
#### Path Parameters
- **options** (`LoadModelOptions`) - Required - Configuration for loading the model
  - **modelPath** (`string`) - Required - Path to GGUF model file (absolute path)
  - **mmprojPath** (`string`) - Optional - Path to multimodal projector GGUF file
  - **gpuLayers** (`number`) - Optional - Layers to offload to GPU (default: 99)
  - **contextSize** (`number`) - Optional - Context window size in tokens
  - **threads** (`number`) - Optional - Number of CPU threads (default: 4)
  - **debug** (`boolean`) - Optional - Enable verbose output (default: false)
  - **logPrompts** (`boolean`) - Optional - Log rendered prompts to stderr (default: false)
  - **chatTemplate** (`string`) - Optional - Chat template name (default: "auto")
  - **embedding** (`boolean`) - Optional - Load in embedding mode (default: false)

### Returns
`Promise<number>` - A model handle (opaque integer) used in subsequent operations.

### Throws
- `Error` - If file validation fails (not found, permission denied, is directory)
- `Error` - If native binding fails to load model

### Example
```typescript
import { loadModel } from "@lgrammel/llama-cpp-provider";

const handle = await loadModel({
  modelPath: "/path/to/model.gguf",
  gpuLayers: 99,
  contextSize: 4096,
  threads: 8,
});

console.log(handle);  // e.g., 1 (opaque identifier)
```
```

--------------------------------

### Expand Tilde Path for Model Files

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/errors.md

Avoid errors caused by unexpanded tilde paths in modelPath or mmprojPath. Always expand '~' to the user's home directory.

```typescript
import { homedir } from "node:os";
import { join } from "node:path";

// ❌ This will fail
const model = llamaCpp({
  modelPath: "~/models/model.gguf",
});

// ✅ Correct: expand to absolute path
const model = llamaCpp({
  modelPath: join(homedir(), "models/model.gguf"),
});
```

--------------------------------

### Configure Model Reasoning with Custom Markers

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/_autodocs/configuration.md

Customize the opening and closing markers for reasoning extraction, and optionally disable prompt injection.

```typescript
const model = llamaCpp({
  modelPath: "./models/model.gguf",
  model: {
    reasoning: {
      openingMarker: "<think>",
      closingMarker: "</think>",
      promptPrefix: "Please think step-by-step:\n",
    },
  },
});
```

--------------------------------

### Build mtmd Library

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Defines and builds the 'mtmd' static library, including its source files and linking against ggml and llama.

```cmake
# Build libmtmd without enabling the llama.cpp tools bundle.
file(GLOB MTMD_MODEL_SOURCES
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/tools/mtmd/models/*.cpp
)
add_library(mtmd STATIC
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/tools/mtmd/mtmd.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/tools/mtmd/mtmd-audio.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/tools/mtmd/mtmd-image.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/tools/mtmd/mtmd-helper.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/tools/mtmd/clip.cpp
    ${MTMD_MODEL_SOURCES}
)
target_link_libraries(mtmd PUBLIC ggml llama)
target_include_directories(mtmd PUBLIC
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/tools/mtmd
)
target_include_directories(mtmd PRIVATE
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/../llama.cpp/vendor
)
target_compile_features(mtmd PRIVATE cxx_std_17)
set_target_properties(mtmd PROPERTIES POSITION_INDEPENDENT_CODE ON)
```

--------------------------------

### Run Tests with Coverage

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/AGENTS.md

Generate a code coverage report for all tests. Helps identify areas of the code that are not adequately tested.

```bash
pnpm test:coverage
```

--------------------------------

### Enable Metal Support on macOS

Source: https://github.com/lgrammel/llama-cpp-provider/blob/main/packages/llama-cpp-provider/native/CMakeLists.txt

Enables Metal support and Metal library embedding for macOS builds.

```cmake
# Enable Metal on macOS
if(APPLE)
    set(GGML_METAL ON CACHE BOOL "Enable Metal support" FORCE)
    set(GGML_METAL_EMBED_LIBRARY ON CACHE BOOL "Embed Metal library" FORCE)
endif()
```