### Install rllama with Bundler

Source: https://github.com/docusealco/rllama/blob/master/README.md

Run this command to install the gem and its dependencies after adding it to your Gemfile.

```bash
bundle install
```

--------------------------------

### Basic generation example

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/model.md

Example of basic text generation with `max_tokens` and `temperature` parameters. Prints the generated text and tokens per second from the result stats.

```ruby
result = model.generate('What is the capital of France?', max_tokens: 1024, temperature: 0.7)
puts result.text
puts result.stats[:tps]          # tokens per second
```

--------------------------------

### Direct low-level access example

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cpp.md

Example of direct low-level access from application code: sets log to stderr, creates a model, and prints model size and GPU offload capability.

```ruby
Rllama::Cpp.set_log($stderr)

model_ptr = Rllama::Model.new('model.gguf').pointer
puts Rllama::Cpp.llama_model_size(model_ptr)     # bytes on disk
puts Rllama::Cpp.llama_supports_gpu_offload      # capability probe
```

--------------------------------

### Direct invocation example (rllama executable)

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cli.md

Example of how the rllama executable invokes the CLI. Shows the shebang, require, and call to Rllama::Cli.start with ARGV, including comments on behavior for different argument scenarios.

```ruby
#!/usr/bin/env ruby
require 'rllama'

Rllama::Cli.start(ARGV)
# ARGV.empty?                          -> interactive model picker
# ARGV == ['models/model.gguf']        -> chat with that model
# ARGV == ['https://host/path.gguf']   -> download & chat
```

--------------------------------

### Use sampler_init, sampler_print, and sampler_free

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/common.md

Example of creating a sampler with a temperature of 0.8, top_k 40, top_p 0.95, min_p 0.05, and seed 1, printing its human-readable chain description, then freeing it. Used internally per generation.

```ruby
handle = Rllama::Common.sampler_init(model.pointer, { temp: 0.8, top_k: 40, top_p: 0.95, min_p: 0.05, seed: 1 })
puts Rllama::Common.sampler_print(handle)   # human-readable chain description
Rllama::Common.sampler_free(handle)
```

--------------------------------

### Start rllama CLI chat

Source: https://github.com/docusealco/rllama/blob/master/README.md

Start the interactive chat interface. Without arguments, it displays downloaded models and a curated list of popular models. Enter the number of a model to use it; if not downloaded, it will be automatically downloaded from Hugging Face.

```bash
rllama
```

--------------------------------

### format_bytes example

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/loader.md

Example usage of format_bytes, showing the output for 5,335,285,280 bytes as "4.97 GB".

```ruby
puts Rllama::Loader.format_bytes(5_335_285_280)  # => "4.97 GB"
```

--------------------------------

### Run rllama with a local model path

Source: https://github.com/docusealco/rllama/blob/master/README.md

Start chatting with a local model file by specifying its path.

```bash
rllama path/to/your/model.gguf
```

--------------------------------

### Install rllama gem directly

Source: https://github.com/docusealco/rllama/blob/master/README.md

Install the rllama gem directly without using Bundler.

```bash
gem install rllama
```

--------------------------------

### Create a new model instance

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/model.md

Example of constructing a model from a local .gguf file. The path is resolved by Rllama::Loader, and the model is loaded with default parameters.

```ruby
model = Rllama::Model.new('models/phi-4-Q4_K_S.gguf')
```

--------------------------------

### Tool calls with tool output

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/context.md

Tool call example: initializes a context with tools, generates a request, sends tool output back as role: 'tool' messages, and prints the final answer. Tools are remembered across turns.

```ruby
ctx = model.init_context(tools: [tool_schema])   # tools remembered across turns
r = ctx.generate('Weather in Paris?')
r.tool_calls.each do |call|
  r = ctx.generate({ role: 'tool', tool_call_id: call[:id], name: call[:name], content: '21C' })
end
puts r.text   # final natural-language answer

```

--------------------------------

### start class method signature

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cli.md

Class method that creates a new instance with the given arguments and calls run. Returns whatever run returns, typically nil after the chat loop exits.

```ruby
def self.start(args)
  new(args).run
end
```

--------------------------------

### run method signature

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cli.md

Main entry point that selects or loads a model, initializes context, and starts the interactive chat loop. Handles interrupts and errors, exiting with status 0 or 1 respectively. Ensures context and model are closed.

```ruby
def run
  model_path = select_or_load_model

  puts "\n#{colorize('Loading model...', :yellow)}"

  model = Rllama.load_model(model_path)
  context = model.init_context

  puts colorize('Model loaded successfully!', :green)
  puts "\n#{colorize('Chat started. Type your message and press Enter. Type "exit" or "quit" to end the chat.',
                     :cyan)}\n"

  chat_loop(context)
rescue Interrupt
  puts "\n\n#{colorize('Chat interrupted. Goodbye!', :yellow)}"
  exit(0)
rescue StandardError => e
  puts "\n#{colorize("Error: #{e.message}", :red)}"
  exit(1)
ensure
  context&.close
  model&.close
end
```

--------------------------------

### Multi-turn chat with system prompt

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/context.md

Multi-turn chat example: initializes a context with a system prompt, generates two turns, prints the final text, and closes the context. Demonstrates that history is remembered across generate calls.

```ruby
ctx = model.init_context(system: 'You are a concise assistant.')
r1 = ctx.generate('Capital of France?')
r2 = ctx.generate('And its population?')   # history is remembered
puts r2.text
ctx.close

```

--------------------------------

### Run rllama with a model URL

Source: https://github.com/docusealco/rllama/blob/master/README.md

Start chatting with a model by providing a direct URL to a GGUF file, such as a Hugging Face download link.

```bash
rllama https://huggingface.co/microsoft/phi-4-gguf/resolve/main/phi-4-Q3_K_S.gguf
```

--------------------------------

### generate

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/model.md

Documentation for the `generate` method, including its signature, parameters, return type, errors, and examples.

```APIDOC
## `generate`

### Description
Generates text from a prompt using the model. Creates a temporary single-turn context, runs generation, and returns the result. Supports streaming via a block.

### Method
`generate` (Ruby method)

### Signature
```ruby
def generate(prompt, max_tokens: DEFAULT_CONTEXT_LENGTH, temperature: nil, top_k: nil, top_p: nil, min_p: nil,
             seed: nil, system: nil, tools: nil, reasoning: nil, &block)
```

### Parameters
- `prompt` (String / Array<Hash>) - Required - Message text or OpenAI-style messages array.
- `max_tokens` (Integer) - Optional - Default: `DEFAULT_CONTEXT_LENGTH` (8192). Used as the context size `n_ctx` for the temporary context.
- `temperature` (Float) - Optional - Default: `nil` (model default). Sampling temperature.
- `top_k` (Integer) - Optional - Default: `nil` (model default). Top-k sampling.
- `top_p` (Float) - Optional - Default: `nil` (model default). Top-p sampling.
- `min_p` (Float) - Optional - Default: `nil` (model default). Min-p sampling.
- `seed` (Integer) - Optional - Default: `nil` (random per call). RNG seed (masked to 32 bits).
- `system` (String) - Optional - Default: `nil`. System prompt injected as the first system message.
- `tools` (Array<Hash>) - Optional - Default: `nil`. OpenAI tool definitions; enables tool-call parsing.
- `reasoning` (Boolean) - Optional - Default: `nil` (off). Enable the model's reasoning phase and parse `reasoning_content`.
- `&block` (Proc) - Optional - Streaming callback; each argument is a decoded UTF-8 token piece.

### Returns
`Rllama::Result` — `text`, `reasoning`, `tool_calls`, `stats`. A fresh single-turn `Context` is created with `init_context(n_ctx: max_tokens)` and closed after the call, so no conversation state survives the call.

### Throws
`Rllama::Error` — context-length exhaustion ("context length has been exceeded"), decode failure ("llama_decode failed."), overly long prompt ("Prompt is too long."), chat-template parse failure.

### Example
```ruby
result = model.generate('What is the capital of France?', max_tokens: 1024, temperature: 0.7)
puts result.text
puts result.stats[:tps]          # tokens per second
```

### Streaming Example
```ruby
begin
  model.generate(
    'Write a Ruby haiku',
    temperature: 0.8, top_k: 40, top_p: 0.95, min_p: 0.05,
    seed: 42, system: 'You are a poet.',
    reasoning: true
  ) do |piece|
    print piece
  end
  puts
rescue Rllama::Error => e
  warn "generation failed: #{e.message}"
end
```
```

--------------------------------

### Set system prompt with model.generate system:

Source: https://github.com/docusealco/rllama/blob/master/README.md

Include a system prompt to guide model behavior by passing the system: keyword argument to model.generate.

```ruby
result = model.generate(
  'What are best practices for Ruby development?',
  system: 'You are an expert Ruby developer with 10 years of experience.'
)
```

--------------------------------

### Rllama::Context#initialize

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/context.md

Documentation for the Rllama::Context constructor, including its full signature, parameters, behavior, and error conditions.

```APIDOC
## Rllama::Context#initialize

### Description
Creates a new chat context that wraps a native llama.cpp `llama_context` and manages conversation history, KV cache, token decoding, sampling, streaming, tool-call parsing, and embedding batches. The constructor builds `LlamaContextParams` from native defaults, overrides context length, batch size, and thread counts, and optionally prefills the system prompt and tool definitions into the KV cache.

### Method
`initialize`

### Signature
```ruby
def initialize(model, embeddings: false, n_ctx: nil, n_batch: nil, n_threads: Etc.nprocessors,
               system: nil, tools: nil, reasoning: false)
```

### Parameters
- `model` (Rllama::Model) - Required - Loaded model
- `embeddings` (Boolean) - Optional, default `false` - Embedding mode: sets `n_seq_max`, `embeddings: true`, `kv_unified: true`, `n_ubatch`
- `n_ctx` (Integer) - Optional, default `nil` (native default) - Context length in tokens
- `n_batch` (Integer) - Optional, default `nil` (native default) - Decode batch size
- `n_threads` (Integer) - Optional, default `Etc.nprocessors` - CPU threads for compute and batch phases
- `system` (String) - Optional, default `nil` - Prepended as `{ role: 'system', content: system }`
- `tools` (Array<Hash>) - Optional, default `nil` - Tool definitions stored for all turns
- `reasoning` (Boolean) - Optional, default `false` - Reasoning parsing enabled for all turns

### Behavior
- Builds `LlamaContextParams` from `Cpp.llama_context_default_params` (copied by value)
- Overrides `n_ctx`, `n_batch`, `n_threads`, `n_threads_batch`
- In embeddings mode also sets `n_seq_max = min(n_batch, model.n_seq_max)`, `embeddings`, `kv_unified`, `n_ubatch`
- Creates the native context via `Cpp.llama_init_from_model`
- If `system || tools` (non-embedding), runs `prefill` to bake tools/system into the cache

### Throws
- `Rllama::Error` - "Failed to create the llama_context" if the native pointer is null
```

--------------------------------

### Model.new

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/configuration.md

Initializes a new Model instance.

```APIDOC
## Model.new(path_or_name, dir: nil)

### Description
Initializes a new Model instance. Accepts the same `path_or_name` and `dir` parameters as `Rllama.load_model`. Native loading uses unmodified `llama_model_default_params` defaults.

### Method
N/A (Ruby constructor)

### Endpoint
Model.new

### Parameters
#### Path Parameters
- **path_or_name** (String) - Required - Local `.gguf` path, `https://` URL, or Hugging Face `org/repo/file.gguf`

#### Query Parameters
- **dir** (String) - Optional - Download cache root (default `Loader::DEFAULT_DIR`)

### Request Example
```ruby
model = Model.new("path/to/model.gguf")
```

### Response
#### Success Response (200)
- **model** (Model) - The new Model instance

#### Response Example
```ruby
model = Model.new("path/to/model.gguf")
```
```

--------------------------------

### silence_log! - install empty log callback

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cpp.md

Installs SILENCE_LOG_CALLBACK (an empty FFI::Function.new(:void, %i[int string pointer], proc {})) as the ggml log callback. Called during module load.

```ruby
def silence_log!
  llama_log_set(SILENCE_LOG_CALLBACK, nil)
end
```

--------------------------------

### Backend init/free

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cpp.md

Bindings for initializing and freeing the llama.cpp backend, NUMA, and loading backend modules. `llama_numa_init` takes an integer mode.

```ruby
attach_function :llama_backend_init, [], :void
attach_function :llama_backend_free, [], :void
attach_function :llama_numa_init, [:int], :void
attach_function :ggml_backend_load_all, [], :void
attach_function :ggml_backend_load_all_from_path, [:string], :void
```

--------------------------------

### silence_log!

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cpp.md

Installs an empty log callback to silence all llama library log output.

```APIDOC
## silence_log!

### Description
Installs an empty log callback to silence all llama library log output.

### Method
FUNCTION

### Endpoint
`Rllama::Cpp.silence_log!`

### Parameters
None

### Return Value
- **:void** - No return value

### Example
```ruby
Rllama::Cpp.silence_log!
```
```

--------------------------------

### Implement Rllama.silence_log!

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/rllama.md

Silences all native llama.cpp logging by installing an empty log callback.

```ruby
def silence_log!
  Cpp.silence_log!
end
```

--------------------------------

### initialize method signature

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cli.md

Initializes a new CLI instance, storing the arguments array and extracting the first argument as the model path.

```ruby
def initialize(args)
  @args = args

  @model_path = args.first
end
```

--------------------------------

### embed

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/model.md

Documentation for the `embed` method of the Rllama model class, including parameters, return types, and example usage.

```APIDOC
## `embed`

### Description
Embeds one or more text prompts into vector representations using the model's embedding context. Handles tokenization, batching, and optional L2 normalization.

### Method
`embed`

### Signature
```ruby
def embed(prompt, normalize: true, batch_size: 512, &block)
```

### Parameters
- **prompt** (`String` / `Array<String>`) - Required - Text(s) to embed.
- **normalize** (`Boolean`) - Optional, default `true` - L2-normalize resulting vectors.
- **batch_size** (`Integer`) - Optional, default `512` - Tokens per decode batch; raised to the longest prompt's token count automatically.
- **&block** (`Proc`) - Optional - Forwarded to the embedding context (no per-token streaming in the current implementation).

### Returns
- `Array<Float>` for single string input.
- `Array<Array<Float>>` for array input (one vector per input, each of length `n_embd`).

### Throws
- `Rllama::Error` — see context.md#embed for batch-size violations; "llama_decode failed"; "Failed to get embedding".

### Example
```ruby
vec = model.embed('The quick brown fox')                    # Array<Float> of n_embd floats
vecs = model.embed(['roses are red', 'violets are blue'])   # Array<Array<Float>>
raw = model.embed('sample', normalize: false)
```
```

--------------------------------

### verify_download method

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/loader.md

Verifies that the downloaded file at local_path matches the expected_size from the server. Returns true immediately if expected_size is <= 0, skipping verification. Otherwise compares the actual file size with the expected size and returns true only if they match.

```ruby
def verify_download(local_path, expected_size)
  return true if expected_size <= 0

  actual_size = File.size(local_path)
  actual_size == expected_size
end
```

--------------------------------

### Rllama.silence_log!

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/rllama.md

Installs an empty llama.cpp log callback to discard all native logging. Called implicitly during module evaluation; a later `Rllama.set_log` re-registers a real callback.

```APIDOC
## Rllama.silence_log!

### Description
Installs an empty llama.cpp log callback to discard all native logging. Called implicitly during module evaluation; a later `Rllama.set_log` re-registers a real callback.

### Method
N/A (Ruby module function)

### Endpoint
`Rllama.silence_log!`

### Parameters
None.

### Request Example
```ruby
Rllama.silence_log!
```

### Response
#### Success Response (200)
- **result** (nil) - Always returns `nil` (FFI void call).

#### Response Example
```ruby
Rllama.silence_log!  # => nil
```
```

--------------------------------

### sampler_init: Initialize sampler handle

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/common.md

Initializes a sampler using the model handle and sampling parameters. Raises Rllama::Error on failure with message 'Failed to initialize sampler: …'.

```ruby
def sampler_init(model, params)
  with_err do |err|
    handle = rlc_sampler_init(model, params.to_json, err)

    raise Error, "Failed to initialize sampler: #{read_err(err)}" if handle.null?

    handle
  end
end
```

--------------------------------

### embed usage examples

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/model.md

Shows how to call `embed` with a single string, an array of strings, and with `normalize: false`. The first returns an `Array<Float>`, the second an `Array<Array<Float>>`, and the third returns unnormalized vectors.

```ruby
vec = model.embed('The quick brown fox')                    # Array<Float> of n_embd floats
vecs = model.embed(['roses are red', 'violets are blue'])   # Array<Array<Float>>
raw = model.embed('sample', normalize: false)
```

--------------------------------

### System info and support checks

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cpp.md

Query runtime timing, sequence limits, and hardware/feature support (mmap, mlock, GPU offload, RPC, flash attention).

```ruby
attach_function :llama_time_us, [], :int64
attach_function :llama_max_parallel_sequences, [], :size_t
attach_function :llama_supports_mmap, [], :bool
attach_function :llama_supports_mlock, [], :bool
attach_function :llama_supports_gpu_offload, [], :bool
attach_function :llama_supports_rpc, [], :bool
attach_function :llama_flash_attn_type_name, [:int], :string
```

--------------------------------

### Streaming generation with error handling

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/model.md

Example of streaming generation with explicit sampling parameters, a system prompt, and reasoning enabled. Uses a block to print each token piece and rescues `Rllama::Error` to handle failures.

```ruby
begin
  model.generate(
    'Write a Ruby haiku',
    temperature: 0.8, top_k: 40, top_p: 0.95, min_p: 0.05,
    seed: 42, system: 'You are a poet.',
    reasoning: true
  ) do |piece|
    print piece
  end
  puts
rescue Rllama::Error => e
  warn "generation failed: #{e.message}"
end
```

--------------------------------

### download_from_huggingface method

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/loader.md

Downloads a HuggingFace model file to a local cache directory. Raises Rllama::Error for paths with fewer than 3 parts, returns the local path if the file already exists, and otherwise prints the destination and calls download_file.

```ruby
def download_from_huggingface(hf_path, dir)
  parts = hf_path.split('/')

  raise Error, "Invalid HuggingFace path: #{hf_path}" if parts.length < 3

  org = parts[0]
  repo = parts[1]
  file_path = parts[2..].join('/')

  url = "#{HUGGINGFACE_BASE_URL}/#{org}/#{repo}/resolve/main/#{file_path}"

  local_path = File.join(dir, org, repo, file_path)

  return local_path if File.exist?(local_path)

  puts "Destination: #{local_path}"

  download_file(url, local_path, "HuggingFace model: #{hf_path}")
end
```

--------------------------------

### Apply chat template with rlc_chat_templates_init and rlc_chat_apply

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/llama-common-c.md

Applies a chat template through the raw C function, as Rllama::Common does. Initializes a chat template with the model and BOS/EOS tokens, applies a user message with add_generation_prompt, parses the response, then frees resources. Requires including llama_common.h and a valid model handle.

```c
#include "llama_common.h"

char *err = NULL;
rlc_chat_templates *t = rlc_chat_templates_init(model, "", bos, eos, &err);
char *json = rlc_chat_apply(t, "{\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}],"
                               "\"add_generation_prompt\":true}", &err);
char *out  = rlc_chat_parse("hello", false,
                            "{\"format\":0,\"parser\":\"\",\"parse_tool_calls\":true}", &err);
rlc_free(json); rlc_free(out);
rlc_chat_templates_free(t);
```

--------------------------------

### llama_print_system_info

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cpp.md

Returns a string containing system information about the llama library build and host environment.

```APIDOC
## llama_print_system_info

### Description
Returns a string containing system information about the llama library build and host environment.

### Method
FUNCTION

### Endpoint
`llama_print_system_info`

### Parameters
None

### Return Value
- **:string** - System information string

### Example
```ruby
info = Rllama::Cpp.llama_print_system_info
```
```

--------------------------------

### llama_opt_init

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/cpp.md

Initializes the optimizer with a context, model, and parameters.

```APIDOC
## llama_opt_init

### Description
Initializes the optimizer with a context, model, and parameters.

### Method
FUNCTION

### Endpoint
`llama_opt_init`

### Parameters
- **ctx** (llama_context_p) - Required - Pointer to the llama context
- **model** (llama_model_p) - Required - Pointer to the llama model
- **params** (LlamaOptParams) - Required - Optimization parameters (by value)

### Return Value
- **:void** - No return value

### Example
```ruby
Rllama::Cpp.llama_opt_init(ctx, model, params)
```
```

--------------------------------

### Generate embeddings with model.embed

Source: https://github.com/docusealco/rllama/blob/master/README.md

Generate vector embeddings for text using embedding models. Load an embedding model with `Rllama.load_model`, then call `model.embed` with a single string or an array of strings. The output length depends on the model; the example shows 724 for a single text and 768 for each of three sentences. Close the model when done.

```ruby
require 'rllama'

# Load an embedding model
model = Rllama.load_model('lmstudio-community/embeddinggemma-300m-qat-GGUF/embeddinggemma-300m-qat-Q4_0.gguf')

# Generate embedding for a single text
embedding = model.embed('Hello, world!')
puts embedding.length
# => 724 (depending on your model)

# Generate embeddings for multiple sentences
embeddings = model.embed([
  'roses are red',
  'violets are blue',
  'sugar is sweet'
])

puts embeddings.length
# => 3
puts embeddings[0].length
# => 768

model.close
```

--------------------------------

### Initialize Rllama::Model with path or name

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/model.md

Constructor that resolves the model path or name using Rllama::Loader, loads the model via llama.cpp, and raises an error if loading fails. The `dir` parameter sets the download cache root for remote models.

```ruby
def initialize(path_or_name, dir: nil)
  resolved_path = Loader.resolve(path_or_name, dir:)

  model_params = Cpp.llama_model_default_params

  @pointer = Cpp.llama_model_load_from_file(resolved_path, model_params)

  raise Error, "Unable to load model from #{resolved_path}" if @pointer.null?
end
```

--------------------------------

### Resolve local, Hugging Face, and URL models

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/loader.md

Shows the resolution order and caching behavior: local paths are returned as-is, while Hugging Face and URL inputs are downloaded to the cache directory.

```ruby
Rllama::Loader.resolve('models/local.gguf', dir: '/data/models')          # already local
p = Rllama::Loader.resolve('microsoft/phi-4-gguf/phi-4-Q4_K_S.gguf')     # downloads to ~/.rllama/models
Rllama::Loader.resolve('https://example.org/model.gguf')                 # downloads to ~/.rllama/models/model.gguf
```

--------------------------------

### chat_templates_init

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/common.md

Initializes a chat templates handle from a model, source, and BOS/EOS tokens. Raises Rllama::Error with the message "Failed to parse chat template: <msg>" if the shim returns NULL.

```ruby
def chat_templates_init(model, source, bos_token, eos_token)
  with_err do |err|
    handle = rlc_chat_templates_init(model, source, bos_token, eos_token, err)

    raise Error, "Failed to parse chat template: #{read_err(err)}" if handle.null?

    handle
  end
end
```

--------------------------------

### rlc_sampler_init params JSON contract

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/llama-common-c.md

Documentation for the JSON parameters accepted by the `rlc_sampler_init` function.

```APIDOC
## `rlc_sampler_init` — params JSON contract

### Description
Defines the JSON object passed to `rlc_sampler_init` to configure the sampler. All fields are optional; llama.cpp defaults apply when omitted.

### Parameters

#### JSON Object Fields
- **`seed`** (int) - Optional - Sampler RNG seed.
- **`top_k`** (int) - Optional - Top-k sampling parameter.
- **`top_p`** (float) - Optional - Nucleus sampling parameter.
- **`min_p`** (float) - Optional - Min-p sampling parameter.
- **`typ_p`** (float) - Optional - Typical sampling parameter.
- **`temp`** (float) - Optional - Temperature sampling parameter.
- **`penalty_last_n`** (int) - Optional - Repetition penalty last-n.
- **`penalty_repeat`** (float) - Optional - Repetition penalty repeat.
- **`penalty_freq`** (float) - Optional - Repetition penalty frequency.
- **`penalty_present`** (float) - Optional - Repetition penalty presence.
- **`dry_multiplier`** (float) - Optional - DRY sampling multiplier.
- **`dry_base`** (float) - Optional - DRY sampling base.
- **`dry_allowed_length`** (int) - Optional - DRY sampling allowed length.
- **`dry_penalty_last_n`** (int) - Optional - DRY sampling penalty last-n.
- **`xtc_probability`** (float) - Optional - XTC sampling probability.
- **`xtc_threshold`** (float) - Optional - XTC sampling threshold.
- **`mirostat`** (int) - Optional - Mirostat mode (v1/v2).
- **`mirostat_tau`** (float) - Optional - Mirostat tau.
- **`mirostat_eta`** (float) - Optional - Mirostat eta.
- **`min_keep`** (int) - Optional - Minimum keep.
- **`grammar`** (str) - Optional - GBNF grammar string; empty disables grammar.
- **`grammar_type`** (int) - Optional - Grammar type: 1 = user GBNF, 3 = tool calls.
- **`grammar_lazy`** (bool) - Optional - Lazy grammar pattern matching.
- **`grammar_triggers`** (array) - Optional - Trigger descriptors, each with `type` (int), `value` (str), `token` (int).
- **`generation_prompt`** (str) - Optional - Prefilled into non-lazy tool-call grammars.

### Example
```json
{
  "seed": 42,
  "top_k": 40,
  "top_p": 0.95,
  "min_p": 0.05,
  "typ_p": 1.0,
  "temp": 0.8,
  "penalty_last_n": 64,
  "penalty_repeat": 1.1,
  "penalty_freq": 0.0,
  "penalty_present": 0.0,
  "dry_multiplier": 0.0,
  "dry_base": 1.75,
  "dry_allowed_length": 2,
  "dry_penalty_last_n": -1,
  "xtc_probability": 0.0,
  "xtc_threshold": 0.1,
  "mirostat": 0,
  "mirostat_tau": 5.0,
  "mirostat_eta": 0.1,
  "min_keep": 0,
  "grammar": "",
  "grammar_type": 1,
  "grammar_lazy": false,
  "grammar_triggers": [],
  "generation_prompt": ""
}
```
```

--------------------------------

### chat_templates_init

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/api-reference/common.md

Initializes a chat templates handle from a model, source, BOS token, and EOS token. Returns a shim handle that can be used with `chat_apply` and must be freed with `chat_templates_free`.

```APIDOC
## `chat_templates_init`

### Description
Initializes a chat templates handle from a model, source, BOS token, and EOS token. Returns a shim handle that can be used with `chat_apply` and must be freed with `chat_templates_free`.

### Method
`chat_templates_init`

### Signature
```ruby
def chat_templates_init(model, source, bos_token, eos_token)
```

### Parameters
- `model` (`:pointer`, native `llama_model`) - Required - Model handle for vocab BOS/EOS handling (may be `nil`)
- `source` (`String`) - Required - Chat template source; overrides the model template when non-empty
- `bos_token` (`String`) - Required - BOS token text
- `eos_token` (`String`) - Required - EOS token text

### Return
Shim handle `:pointer` (non-null `rlc_chat_templates*`).

### Throws
`Rllama::Error` — "Failed to parse chat template: <msg>" when the shim returns NULL (the shim message is freed via `read_err`).
```

--------------------------------

### Handle tool call results with model.init_context and context.generate

Source: https://github.com/docusealco/rllama/blob/master/README.md

Create a context with model.init_context(tools:) to enable multi-turn tool calling. The context remembers tools and system prompts for all subsequent turns. When the model emits a tool call, it is parsed into result.tool_calls and stored in history. Execute the calls and send each result back as a role: 'tool' message. The context automatically rolls back the KV cache to the common prefix when history is edited.

```ruby
context = model.init_context(tools:)

result = context.generate('What is the weather in Paris?')
result.tool_calls
# => [{ name: 'get_weather', arguments: { 'location' => 'Paris' }, id: 'call_1' }]

result.tool_calls.each do |call|
  weather = get_weather(call[:arguments]['location'])

  result = context.generate(
    { role: 'tool', tool_call_id: call[:id], name: call[:name], content: weather.to_json }
  )
end

puts result.text
# => "The weather in Paris right now is 21°C."
```

--------------------------------

### Model.init_context

Source: https://github.com/docusealco/rllama/blob/master/_autodocs/configuration.md

Initializes a context for generation or embedding.

```APIDOC
## Model.init_context(...)

### Description
Initializes a context for generation or embedding. Allows setting embedding mode, context window size, batch size, system prompt, tools, and reasoning parsing.

### Method
N/A (Ruby instance method)

### Endpoint
Model#init_context

### Parameters
#### Path Parameters
- **embeddings** (Boolean) - Optional - Embedding-mode context; default `false`
- **n_ctx** (Integer) - Optional - Context window (tokens); default `DEFAULT_CONTEXT_LENGTH` = 8192
- **n_batch** (Integer) - Optional - Decode batch size (tokens); default `512`
- **system** (String) - Optional - System prompt; prefilled into the cache when set (or when `tools` set)
- **tools** (Array<Hash>) - Optional - Tool definitions remembered for all turns
- **reasoning** (Boolean) - Optional - Reasoning parsing for all turns; default `false`

### Request Example
```ruby
model.init_context(n_ctx: 4096, system: "You are a helpful assistant")
```

### Response
#### Success Response (200)
- **context** (Context) - The initialized context object

#### Response Example
```ruby
context = model.init_context(n_ctx: 4096)
```
```