### Run Basic Text Generation Example

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Launches the basic text generation example. Requires a GGUF file path to be provided as a command-line argument.

```bash
dart examples/basic_example.dart -m <GGUF file path>
```

--------------------------------

### Summarization Output Example

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

This is an example of the output generated by the RAG summarization script for the provided URL.

```text
 William Anthony "The Refrigerator" Perry, born December 16, 1962 in Aiken, South Carolina, 
 was a renowned NFL defensive tackle who played for the Chicago Bears and Clemson Tigers. 
 He earned the nickname "Refrigerator" due to his massive size during college football at 
 Clemson University, where he also received the ACC Player of the Year award. Drafted in 
 1985 by the Bears, Perry gained fame as part of their first Super Bowl-winning team, 
 setting a record for the heaviest player to score a touchdown in the Super Bowl and 
 possessing the largest Super Bowl ring. Despite facing challenges with his weight 
 throughout his professional career, he played 10 seasons and became an iconic figure among
 Bears fans. Perry's off-field ventures included music collaborations with Walter Payton 
 and media appearances, including a notable boxing match against Bob Sapp. After retiring 
 from football in 1994, he faced personal struggles, losing over one hundred pounds before 
 regaining much of his weight. His health issues continued to affect him later on, leading 
 to hospitalization for diabetes treatment and hearing loss. Perry's life post-football has 
 been marked by various ups and downs, including a brief comeback attempt in the World 
 League of American Football and an induction into the WWE Hall of Fame. He also faced 
 legal disputes over his Super Bowl ring, which was later auctioned for $200,000. Perry's 
 legacy is remembered not just for his athletic achievements but also for his 
 larger-than-life personality off the field.

Performance data: 358 tokens (1529 characters) total in 371565.63 ms (0.96 T/s) ; 
26962 prompt tokens in 289057.38 ms (93.28 T/s)
```

--------------------------------

### Tokenize Text and Get Token Count

Source: https://context7.com/tbogdala/woolydart/llms.txt

Use `tokenizeText` to convert strings into token lists and `getTokenCount` to get the number of tokens. The `addSpecial` and `parseSpecial` parameters control the inclusion and parsing of special tokens.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load model ...

final testText = "<|start_header_id|>assistant<|end_header_id|>

Hello, world!";

// Get just the token count
final tokenCount = llamaModel.getTokenCount(
    testText,
    false,  // addSpecial - don't add BOS/EOS tokens
    true    // parseSpecial - parse model-specific special tokens
);
print('Token count: $tokenCount');

// Get the actual token list
final TokenList tokens = llamaModel.tokenizeText(
    testText,
    true,   // addSpecial - add BOS token
    true    // parseSpecial - parse special tokens
);
print('Tokens: $tokens');
print('Token list length: ${tokens.length}');

// Convert tokens back to text
final String? decodedText = llamaModel.detokenizeToText(
    tokens,
    true  // renderSpecials - show special tokens in output
);
print('Decoded text: $decodedText');

llamaModel.freeModel();
```

--------------------------------

### DRY Sampler Configuration in Dart

Source: https://context7.com/tbogdala/woolydart/llms.txt

Configure the DRY sampler to prevent repetitive text by specifying sequence breaking strings. This example sets up common breakers like newlines and punctuation.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load model ...

final params = llamaModel.getTextGenParams()
  ..n_predict = 200
  ..temp = 0.8
  ..penalty_repeat = 1.1;

// Configure DRY sampler sequence breakers
// These strings break repetition detection
params.setDrySequenceBreakers([
  "\n",      // Newlines break sequences
  ".",       // Sentence endings
  "!",
  "?",
  ",",
  ";",
]);

params.setPrompt("List 10 creative uses for a paperclip:");
params.setAntiprompts(["11."]);

final (result, output) = llamaModel.predictText(params, nullptr);
print(output);

// Clean up - dispose() frees all allocated native strings
params.dispose();  // Calls freePrompt(), freeAntiprompts(), freeGrammar(), freeDrySequenceBreakers()
lamaModel.freeModel();
```

--------------------------------

### Run RAG Summarization with Meta-Llama-3.1-8B-Instruct

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Execute the RAG summarization script with a different LLM model, demonstrating performance variations on a Windows machine. This command assumes the specified model is available.

```bash
dart example/rag_summarize.dart -m ~/.cache/lm-studio/models/bartowski/Phi-3.1-mini-128k-instruct-GGUF/Phi-3.1-mini-128k-instruct-Q4_K_M.gguf --url "https://en.wikipedia.org/wiki/William_Perry_(American_football)" -c 28000 -q | say
```

--------------------------------

### Run RAG Summarization with Phi-3.1-mini

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Execute the RAG summarization script with a specified LLM model, URL, and context size. Ensure the LLM model path is correct and the URL is accessible.

```bash
dart example/rag_summarize.dart -m ~/.cache/lm-studio/models/bartowski/Phi-3.1-mini-128k-instruct-GGUF/Phi-3.1-mini-128k-instruct-Q4_K_M.gguf --url "https://en.wikipedia.org/wiki/William_Perry_(American_football)" -c 28000
```

--------------------------------

### Build woolycore Library (Windows CUDA)

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Compiles the woolycore library on Windows with CUDA support and specific flags for shared library building. Remove -DGGML_CUDA=On for CPU-only builds.

```bash
cd src
cmake -B build -DWOOLY_TESTS=Off -DGGML_CUDA=On -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE -DBUILD_SHARED_LIBS=Off woolycore
cmake --build build --config Release
cd ..
cp src/build/Release/woolycore.dll .
```

--------------------------------

### Initialize LlamaModel Constructor in Dart

Source: https://context7.com/tbogdala/woolydart/llms.txt

Demonstrates how to create a LlamaModel instance by specifying the filepath to the compiled native library. Platform-specific paths are handled for desktop and mobile.

```dart
import 'dart:io';
import 'package:woolydart/woolydart.dart';

String getPlatformLibraryFilepath() {
  if (Platform.isMacOS) {
    return "src/build/libwoolycore.dylib";
  } else if (Platform.isWindows) {
    return "woolycore.dll";
  } else {
    return "src/build/libwoolycore.so";
  }
}

// Create a new LlamaModel instance
final libFilepath = getPlatformLibraryFilepath();
var llamaModel = LlamaModel(libFilepath);

// For iOS, use an empty string to load from the active process
var iosModel = LlamaModel('');

// For Android, use the bundled library name
var androidModel = LlamaModel('libwoolydart.so');

```

--------------------------------

### Build woolycore Library (CUDA)

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Compiles the woolycore library with CUDA support enabled, disabling unit tests. This is for GPU acceleration on compatible systems.

```bash
cd src
cmake -B build -DWOOLY_TESTS=Off -DGGML_CUDA=On woolycore
cmake --build build --config Release
```

--------------------------------

### LlamaModel Constructor

Source: https://context7.com/tbogdala/woolydart/llms.txt

Initializes a LlamaModel instance by specifying the filepath to the compiled native library. Platform-specific paths are handled.

```APIDOC
## LlamaModel Constructor

### Description
Creates a new LlamaModel wrapper for llama.cpp by specifying the filepath to the compiled native library. On desktop platforms, pass the full path to the compiled binary; on mobile platforms, use the bundled library name.

### Method
Constructor

### Endpoint
N/A

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```dart
import 'dart:io';
import 'package:woolydart/woolydart.dart';

String getPlatformLibraryFilepath() {
  if (Platform.isMacOS) {
    return "src/build/libwoolycore.dylib";
  } else if (Platform.isWindows) {
    return "woolycore.dll";
  } else {
    return "src/build/libwoolycore.so";
  }
}

// Create a new LlamaModel instance
final libFilepath = getPlatformLibraryFilepath();
var llamaModel = LlamaModel(libFilepath);

// For iOS, use an empty string to load from the active process
var iosModel = LlamaModel('');

// For Android, use the bundled library name
var androidModel = LlamaModel('libwoolydart.so');
```

### Response
N/A
```

--------------------------------

### makePromptFromMessages - Build Chat Prompt

Source: https://context7.com/tbogdala/woolydart/llms.txt

Constructs a formatted prompt string from a list of chat messages, using either the model's default template or a custom override.

```APIDOC
## makePromptFromMessages - Build Chat Prompt

### Description
Constructs a formatted prompt string from a list of chat messages using the model's embedded chat template or a custom template override. Returns a tuple with the formatted prompt and the number of messages processed.

### Method
(Implicitly POST or similar, as it involves sending data and receiving a formatted string)

### Endpoint
(Not explicitly defined, operates on a loaded LlamaModel instance)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body (Implicitly via arguments)
- **messages** (List<ChatMessage>) - Required - A list of chat messages, where each `ChatMessage` has a `role` (e.g., 'system', 'user', 'assistant') and `content` (string).
- **addAssistant** (bool) - Required - If true, appends the start of an assistant response block to the prompt, preparing for generation.
- **templateOverride** (string?) - Optional - The name of a custom template to use (e.g., 'chatml'). If null, the model's default template is used.

### Request Example
```dart
final messages = [
  ChatMessage("system", "You are a helpful AI assistant."),
  ChatMessage("user", "What is the capital of France?"),
  ChatMessage("assistant", "The capital of France is Paris."),
  ChatMessage("user", "What's the population?"),
];

// Using default template
final (prompt, numProcessed) = llamaModel.makePromptFromMessages(
    messages,
    true,   // addAssistant
    null    // templateOverride
);

// Using custom template
final (customPrompt, _) = llamaModel.makePromptFromMessages(
    messages,
    true,
    "chatml"  // templateOverride
);
```

### Response
#### Success Response (Tuple)
- **prompt** (string) - The formatted prompt string.
- **numProcessed** (int) - The number of messages successfully processed.

#### Response Example
```json
{
  "prompt": "<|system|>\nYou are a helpful AI assistant.<|end|>\n<|user|>\nWhat is the capital of France?<|end|>\n<|assistant|>\nThe capital of France is Paris.<|end|>\n<|user|>\nWhat's the population?<|end|>\n<|assistant|>\n",
  "numProcessed": 4
}
```

#### Error Handling
(Specific error handling details not provided in the source text, but potential errors could include invalid message formats or unsupported template names.)
```

--------------------------------

### Step-by-Step Inference API in Dart

Source: https://context7.com/tbogdala/woolydart/llms.txt

Use this for fine-grained control over inference, especially for real-time token streaming in chat interfaces. It requires loading the model and setting generation parameters.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load model ...

final params = llamaModel.getTextGenParams()
  ..seed = 42
  ..n_threads = 4
  ..n_predict = 50
  ..temp = 0.8
  ..top_k = 40
  ..min_p = 0.05;

params.setPrompt("Once upon a time");
params.setAntiprompts(["The End", "\n\n"]);

// Step 1: Process the prompt and get a sampler
final (promptTokens, gptSampler) = llamaModel.processPrompt(params);
if (promptTokens < 0) {
  print('Error processing prompt');
  exit(1);
}
print('Processed $promptTokens prompt tokens');

// Step 2: Generate tokens one at a time
List<Token> generatedTokens = [];
for (int i = 0; i < params.n_predict; i++) {
  // Sample the next token
  final Token nextToken = llamaModel.sampleNextToken(gptSampler);
  generatedTokens.add(nextToken);

  // Convert token to text and print
  final tokenText = llamaModel.detokenizeToText([nextToken], false);
  stdout.write(tokenText);

  // Check for end-of-generation or antiprompt
  if (llamaModel.checkEogAndAntiprompt(params, gptSampler)) {
    print('\n[End of generation detected]');
    break;
  }

  // Process the token for the next iteration
  if (!llamaModel.processNextToken(nextToken)) {
    print('\nError processing token');
    break;
  }
}

// Step 3: Add more prompt text dynamically (optional)
final additionalTokens = llamaModel.processAdditionalPrompt(
    gptSampler,
    " and they lived happily"
);
print('
Added $additionalTokens tokens to context');

// Continue generation...

// Clean up
llamaModel.freeGptSampler(gptSampler);
params.dispose();
llamaModel.freeModel();
```

--------------------------------

### loadModel - Load GGUF Model File

Source: https://context7.com/tbogdala/woolydart/llms.txt

Loads a GGUF model file with specified model and context parameters. This method configures GPU offloading, context size, and other llama.cpp settings.

```APIDOC
## loadModel - Load GGUF Model File

### Description
Loads a GGUF model file with specified model and context parameters. Returns true if successful, false otherwise. The model and context parameters control GPU offloading, context size, and other llama.cpp settings.

### Method
POST (or similar, depending on implementation)

### Endpoint
`/model/load` (hypothetical endpoint for documentation structure)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **modelFilepath** (String) - Required - The path to the GGUF model file.
- **modelParams** (ModelParams) - Required - Parameters for model loading, including GPU offloading.
- **contextParams** (ContextParams) - Required - Parameters for the model's context, including context size.
- **silenceLlamaCpp** (bool) - Optional - If true, suppresses llama.cpp output.

### Request Example
```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());

// Get default parameters from llama.cpp
final modelParams = llamaModel.getDefaultModelParams();
modelParams.n_gpu_layers = 100;  // Offload up to 100 layers to GPU

final contextParams = llamaModel.getDefaultContextParams();
contextParams.n_ctx = 4096;  // Set context size to 4096 tokens

// Load the model (silenceLlamaCpp = true to suppress llama.cpp output)
final String modelFilepath = "models/llama-3-8b.gguf";
final bool loadedResult = llamaModel.loadModel(
    modelFilepath,
    modelParams,
    contextParams,
    true  // silenceLlamaCpp
);

if (!loadedResult) {
  print('Failed to load the model');
  exit(1);
}

// Check if model is loaded
print('Model loaded: ${llamaModel.isModelLoaded()}');
print('Loaded filepath: ${llamaModel.loadedModelFilepath}');

// When finished, free the model resources
llamaModel.freeModel();
```

### Response
#### Success Response (200)
- **loadedResult** (bool) - True if the model was loaded successfully, false otherwise.

#### Response Example
```json
{
  "loadedResult": true
}
```
```

--------------------------------

### Load GGUF Model File with Woolydart in Dart

Source: https://context7.com/tbogdala/woolydart/llms.txt

Shows how to load a GGUF model file using LlamaModel. It includes setting model and context parameters for GPU offloading and context size, and checks the loading status. Remember to free model resources when done.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());

// Get default parameters from llama.cpp
final modelParams = llamaModel.getDefaultModelParams();
modelParams.n_gpu_layers = 100;  // Offload up to 100 layers to GPU

final contextParams = llamaModel.getDefaultContextParams();
contextParams.n_ctx = 4096;  // Set context size to 4096 tokens

// Load the model (silenceLlamaCpp = true to suppress llama.cpp output)
final String modelFilepath = "models/llama-3-8b.gguf";
final bool loadedResult = llamaModel.loadModel(
    modelFilepath,
    modelParams,
    contextParams,
    true  // silenceLlamaCpp
);

if (!loadedResult) {
  print('Failed to load the model');
  exit(1);
}

// Check if model is loaded
print('Model loaded: ${llamaModel.isModelLoaded()}');
print('Loaded filepath: ${llamaModel.loadedModelFilepath}');

// When finished, free the model resources
llamaModel.freeModel();

```

--------------------------------

### Build woolycore Library (CPU)

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Compiles the woolycore library for CPU usage, disabling unit tests. This generates the necessary library files for the Dart wrapper.

```bash
cd src
cmake -B build -DWOOLY_TESTS=Off woolycore
cmake --build build --config Release
```

--------------------------------

### Build Chat Prompt with WoolyDart

Source: https://context7.com/tbogdala/woolydart/llms.txt

Use makePromptFromMessages to construct a formatted prompt string from a list of chat messages. Supports using the model's embedded chat template or a custom template override.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load model ...

// Create a conversation with ChatMessage objects
final messages = [
  ChatMessage("system", "You are a helpful AI assistant."),
  ChatMessage("user", "What is the capital of France?"),
  ChatMessage("assistant", "The capital of France is Paris."),
  ChatMessage("user", "What's the population?"),
];

// Build the prompt using the model's embedded chat template
// addAssistant: true adds the start of an assistant response block
final (prompt, numProcessed) = llamaModel.makePromptFromMessages(
    messages,
    true,   // addAssistant - add assistant prefix for generation
    null    // templateOverride - null uses model's default template
);

print('Processed $numProcessed messages');
print('Generated prompt:\n$prompt');

// Use with a custom template override (e.g., for ChatML format)
final (customPrompt, _) = llamaModel.makePromptFromMessages(
    messages,
    true,
    "chatml"  // Use ChatML template
);

// Now use the prompt for text generation
final params = llamaModel.getTextGenParams()
  ..n_predict = 200;
params.setPrompt(prompt);
params.setAntiprompts(["<|end|>"]);

final (result, output) = llamaModel.predictText(params, nullptr);
print('Response: $output');

params.dispose();

```

--------------------------------

### Practical RAG with Embeddings and Cosine Similarity

Source: https://context7.com/tbogdala/woolydart/llms.txt

Demonstrates a practical application of `similarityCos` within a Retrieval-Augmented Generation (RAG) system. This involves generating embeddings for a query and documents, then finding the most similar document using cosine similarity.

```dart
final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load embedding model ...

final documents = [
  "How to cook pasta",
  "Italian pasta recipes",
  "Machine learning basics",
];

final query = "making spaghetti at home";

// Generate embeddings for query and documents
final queryTokens = llamaModel.tokenizeText(query, true, false);
final docTokens = documents.map((d) => llamaModel.tokenizeText(d, true, false)).toList();

final allTokens = [queryTokens, ...docTokens];
final embeddings = llamaModel.makeEmbeddings(EmbeddingNormalization.euclidean, allTokens);

// Find most similar document
final queryEmb = embeddings[0];
var bestScore = -1.0;
var bestDoc = '';

for (int i = 1; i < embeddings.length; i++) {
  final score = similarityCos(queryEmb, embeddings[i]);
  print('Score for "${documents[i-1]}": $score');
  if (score > bestScore) {
    bestScore = score;
    bestDoc = documents[i-1];
  }
}
print('Most relevant document: $bestDoc (score: $bestScore)');

lamaModel.freeModel();
```

--------------------------------

### Run Dart Tests

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Executes the Dart unit tests for Woolydart. Requires the WOOLY_TEST_MODEL_FILE environment variable to be set and limits concurrency to avoid performance issues.

```bash
export WOOLY_TEST_MODEL_FILE=models/example-llama-3-8b.gguf
dart test --concurrency=1
```

--------------------------------

### Set Grammar for Constrained Output

Source: https://context7.com/tbogdala/woolydart/llms.txt

Use `setGrammar` to enforce a BNF-like grammar on the model's output, ensuring structured data generation. Requires defining the grammar as a raw string.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load model ...

final params = llamaModel.getTextGenParams()
  ..n_predict = 500
  ..temp = 0.7;

// Define a grammar for generating JSON objects
final jsonGrammar = r'''
root   ::= object
value  ::= object | array | string | number | ("true" | "false" | "null") ws

object ::=
  "{" ws (
            string ":" ws value
    ("," ws string ":" ws value)*
  )? "}" ws

array  ::=
  "[" ws (
            value
    ("," ws value)*
  )? "]" ws

string ::=
  "\"" (
    [^"\\] | 
    "\\" (["\\/bfnrt"] | "u" [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F])
  )* "\"" ws

number ::= ("-"? ([0-9] | [1-9] [0-9]*)) ("." [0-9]+)? ([eE] [-+]? [0-9]+)? ws

ws ::= ([ \t\n] ws)?
''';

params.setGrammar(jsonGrammar);
params.setPrompt("Generate a JSON object representing a person with name, age, and city:");
params.setAntiprompts(["\n\n"]);

final (result, output) = llamaModel.predictText(params, nullptr);
print('Generated JSON: $output');
// Output will be valid JSON like: {"name": "John", "age": 30, "city": "New York"}

params.dispose();
llamaModel.freeModel();
```

--------------------------------

### Generate Vector Embeddings

Source: https://context7.com/tbogdala/woolydart/llms.txt

Use `makeEmbeddings` to generate vector embeddings for tokenized prompts, suitable for semantic search and RAG. Requires configuring the model for embedding mode and specifying normalization.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());

// Configure for embedding model
final params = llamaModel.getDefaultModelParams()
  ..n_gpu_layers = 100;

final contextParams = llamaModel.getDefaultContextParams()
  ..n_ctx = 2048
  ..n_batch = 2048
  ..embeddings = true  // Enable embedding mode
  ..n_ubatch = 2048
  ..pooling_type = LlamaPoolingType.mean.value;

// Load an embedding model (e.g., nomic-ai/nomic-embed-text-v1.5-GGUF)
llamaModel.loadModel(
    "models/nomic-embed-text-v1.5.Q8_0.gguf",
    params,
    contextParams,
    true
);

// Prepare prompts for embedding
final prompts = [
  "That is a happy person.",
  "That's a very happy person.",
  "The weather is beautiful today.",
  "Attack ships on fire off the shoulder of Orion.",
];

// Tokenize all prompts
List<TokenList> tokenizedPrompts = [];
for (final prompt in prompts) {
  final tokens = llamaModel.tokenizeText(prompt, true, false);
  tokenizedPrompts.add(tokens);
}

// Generate embeddings with Euclidean normalization
final List<Embedding> embeddings = llamaModel.makeEmbeddings(
    EmbeddingNormalization.euclidean,
    tokenizedPrompts
);

print('Generated ${embeddings.length} embeddings');
print('Embedding dimension: ${embeddings.first.length}');

// Calculate similarity between first prompt and all others
print('\nSimilarity to "${prompts.first}":');
for (int i = 0; i < prompts.length; i++) {
  final score = similarityCos(embeddings[0], embeddings[i]);
  print('  $score: ${prompts[i]}');
}
// Output shows semantic similarity scores (1.0 = identical, 0.0 = unrelated)

llamaModel.freeModel();
```

--------------------------------

### Prompt Caching with Freeze and Defrost in Dart

Source: https://context7.com/tbogdala/woolydart/llms.txt

Cache processed prompt states for fast regeneration. Useful for chatbots or applications that frequently reuse prompts. Ensure to free frozen states and samplers when done.

```dart
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load model ...

final params = llamaModel.getTextGenParams()
  ..seed = 42
  ..n_predict = 50;

params.setPrompt("Write a story about a robot:");
params.setAntiprompts(["The End"]);

// Process prompt and generate first response
final (tokenCount, sampler1) = llamaModel.processPrompt(params);
List<Token> firstResponse = [];
for (int i = 0; i < 50; i++) {
  final token = llamaModel.sampleNextToken(sampler1);
  firstResponse.add(token);
  if (llamaModel.checkEogAndAntiprompt(params, sampler1)) break;
  llamaModel.processNextToken(token);
}
print('First response: ${llamaModel.detokenizeToText(firstResponse, false)}');

// Freeze the prompt state (without predictions)
final FrozenState frozenPrompt = llamaModel.freezePrompt(params);

// Or freeze with predictions included for resuming later
final FrozenState frozenWithPrediction = llamaModel.freezePromptWithPrediction(
    params,
    firstResponse
);

lamaModel.freeGptSampler(sampler1);

// Later: Defrost and generate again with different seed (much faster!)
params.seed = 1337;
final (restoredTokens, sampler2) = llamaModel.defrostFrozenState(params, frozenPrompt);
print('Restored $restoredTokens tokens from frozen state');

List<Token> secondResponse = [];
for (int i = 0; i < 50; i++) {
  final token = llamaModel.sampleNextToken(sampler2);
  secondResponse.add(token);
  if (llamaModel.checkEogAndAntiprompt(params, sampler2)) break;
  llamaModel.processNextToken(token);
}
print('Second response: ${llamaModel.detokenizeToText(secondResponse, false)}');

// Clean up frozen states when done
lamaModel.freeFrozenState(frozenPrompt);
lamaModel.freeFrozenState(frozenWithPrediction);
lamaModel.freeGptSampler(sampler2);
params.dispose();
lamaModel.freeModel();
```

--------------------------------

### predictText - Generate Text

Source: https://context7.com/tbogdala/woolydart/llms.txt

Generates text based on loaded model and provided parameters. Supports streaming tokens via a callback.

```APIDOC
## predictText - Generate Text

### Description
Runs text inference on the loaded model to predict text based on generation parameters. Returns a tuple containing the prediction result (with timing data) and the generated output string. An optional callback can be provided to receive each token as it's generated.

### Method
(Implicitly POST or similar, as it involves sending parameters and receiving generated text)

### Endpoint
(Not explicitly defined, operates on a loaded LlamaModel instance)

### Parameters
#### Request Body (Implicitly via `TextGenParams` object)
- **seed** (int) - Optional - Seed for random number generation.
- **n_threads** (int) - Optional - Number of threads to use for generation.
- **n_predict** (int) - Required - Maximum number of tokens to generate.
- **top_k** (int) - Optional - Top-K sampling parameter.
- **top_p** (double) - Optional - Top-P (nucleus) sampling parameter.
- **min_p** (double) - Optional - Minimum probability for sampling.
- **temp** (double) - Optional - Temperature for sampling.
- **penalty_repeat** (double) - Optional - Repetition penalty.
- **penalty_last_n** (int) - Optional - Window size for repetition penalty.
- **ignore_eos** (bool) - Optional - Whether to ignore the end-of-sequence token.
- **flash_attn** (bool) - Optional - Whether to use flash attention if available.
- **n_batch** (int) - Optional - Batch size for prompt processing.
- **prompt_cache_all** (bool) - Optional - Whether to cache the entire prompt.

#### Prompt and Stop Phrases (Set via `params.setPrompt` and `params.setAntiprompts`)
- **prompt** (string) - The input prompt for text generation.
- **antiprompts** (List<string>) - Phrases that will stop generation.

#### Callback Function (Optional)
- **onNewToken** (Function) - A callback function that receives each generated token as a string. It should return `true` to continue generation or `false` to stop.

### Request Example
```dart
final params = llamaModel.getTextGenParams()
  ..seed = 42
  ..n_threads = 4
  ..n_predict = 100
  ..top_k = 1
  ..top_p = 1.0
  ..min_p = 0.1
  ..temp = 0.7
  ..penalty_repeat = 1.1
  ..penalty_last_n = 512
  ..ignore_eos = false
  ..flash_attn = true
  ..n_batch = 128
  ..prompt_cache_all = true;

params.setPrompt(
    "<|user|>
Write a haiku about programming.<|end|>
<|assistant|>
"
);
params.setAntiprompts(["<|end|>", "<|user|>"]);

// Optional callback
bool onNewToken(Pointer<Char> tokenString) {
  var dartToken = (tokenString as Pointer<Utf8>).toDartString();
  stdout.write(dartToken);
  return true;
}

final (predictResult, outputString) = llamaModel.predictText(
    params,
    Pointer.fromFunction(onNewToken, false) // or nullptr
);
```

### Response
#### Success Response (Tuple)
- **predictResult** (dynamic) - An object containing prediction metadata:
  - **result** (int) - Result code (0 for success).
  - **t_start_ms** (int) - Start time in milliseconds.
  - **t_end_ms** (int) - End time in milliseconds.
  - **t_p_eval_ms** (int) - Prompt evaluation time in milliseconds.
  - **n_eval** (int) - Number of tokens evaluated.
  - **n_p_eval** (int) - Number of prompt tokens evaluated.
- **outputString** (string) - The complete generated text.

#### Response Example
```json
{
  "predictResult": {
    "result": 0,
    "t_start_ms": 1678886400000,
    "t_end_ms": 1678886410000,
    "t_p_eval_ms": 50,
    "n_eval": 100,
    "n_p_eval": 20
  },
  "outputString": "Green grass, summer breeze,\nNature's beauty all around,\nPeaceful, calm, serene."
}
```

#### Error Response
- **predictResult.result** (int) - Non-zero value indicating an error.

#### Error Example
```
Error: predictText returned 1
```
```

--------------------------------

### Generate Text with WoolyDart

Source: https://context7.com/tbogdala/woolydart/llms.txt

Use predictText to generate text based on loaded models and configurable parameters. Supports streaming tokens via a callback and provides detailed timing information.

```dart
import 'dart:ffi';
import 'package:ffi/ffi.dart';
import 'package:woolydart/woolydart.dart';

final llamaModel = LlamaModel(getPlatformLibraryFilepath());
// ... load model ...

// Configure text generation parameters
final params = llamaModel.getTextGenParams()
  ..seed = 42
  ..n_threads = 4
  ..n_predict = 100          // Generate up to 100 tokens
  ..top_k = 1
  ..top_p = 1.0
  ..min_p = 0.1
  ..temp = 0.7               // Temperature for sampling
  ..penalty_repeat = 1.1     // Repetition penalty
  ..penalty_last_n = 512     // Window for repetition penalty
  ..ignore_eos = false       // Stop at end-of-sequence token
  ..flash_attn = true        // Use flash attention if available
  ..n_batch = 128            // Batch size for prompt processing
  ..prompt_cache_all = true; // Enable prompt caching

// Set the prompt and stop phrases
params.setPrompt(
    "<|user|>
Write a haiku about programming.<|end|>
<|assistant|>
"
);
params.setAntiprompts(["<|end|>"]);

// Optional: Define a callback for streaming tokens
bool onNewToken(Pointer<Char> tokenString) {
  var dartToken = (tokenString as Pointer<Utf8>).toDartString();
  stdout.write(dartToken);  // Print each token as generated
  return true;  // Return false to stop generation
}

// Run text prediction
final (predictResult, outputString) = llamaModel.predictText(
    params,
    Pointer.fromFunction(onNewToken, false)  // or nullptr for no callback
);

if (predictResult.result != 0) {
  print('Error: predictText returned ${predictResult.result}');
} else {
  print('\nGenerated: $outputString');

  // Access timing information
  final totalMs = predictResult.t_end_ms - predictResult.t_start_ms;
  final tokensPerSec = 1000 / totalMs * predictResult.n_eval;
  print('Generated ${predictResult.n_eval} tokens in ${totalMs}ms (${tokensPerSec} T/s)');
  print('Prompt processed: ${predictResult.n_p_eval} tokens in ${predictResult.t_p_eval_ms}ms');
}

// Clean up
params.dispose();
llamaModel.freeModel();

```

--------------------------------

### Update Git Submodules

Source: https://github.com/tbogdala/woolydart/blob/main/README.md

Updates all submodules for upstream projects. This command ensures that all necessary external dependencies are up-to-date.

```bash
git pull --recurse-submodules
```

--------------------------------

### Calculate Cosine Similarity with Embeddings

Source: https://context7.com/tbogdala/woolydart/llms.txt

Use `similarityCos` to compute the cosine similarity between two embedding vectors. It returns a value between -1 and 1. This function is useful for comparing vector similarity in RAG systems.

```dart
import 'package:woolydart/woolydart.dart';

// Example with pre-computed embeddings
final Embedding embd1 = [0.1, 0.2, 0.3, 0.4, 0.5];
final Embedding embd2 = [0.1, 0.2, 0.3, 0.4, 0.5];
final Embedding embd3 = [0.5, 0.4, 0.3, 0.2, 0.1];

// Identical vectors = 1.0
print('Same vectors: ${similarityCos(embd1, embd2)}');  // Output: 1.0

// Different vectors
print('Different vectors: ${similarityCos(embd1, embd3)}');  // Output: ~0.64
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.