### Install Chat Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/chat/README.md
Install the chat example using Go.
```shell
go install ./examples/chat
```
--------------------------------
### Install Embeddings Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/embeddings/README.md
Use this command to install the embeddings example. Ensure you have Go installed and configured.
```shell
go install ./examples/embeddings
```
--------------------------------
### Run Multi-step Tool Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/multitool/README.md
Execute the multitool example with specified model, library, and question.
```shell
go run ./examples/multitool -model /path/to/model.gguf -lib /path/to/llama/lib -question "the question"
```
--------------------------------
### Run Hello World Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/hello/README.md
Execute the Go example program after downloading the model. This demonstrates a basic YZMA application.
```go
go run ./examples/hello/
"Yes, I'm ready to go."
```
--------------------------------
### Run Chat Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/chat/README.md
Execute the chat example using Go, specifying the model file.
```shell
go run ./examples/chat/ -model ./models/qwen2.5-0.5b-instruct-fp16.gguf
```
--------------------------------
### Run Installer with CUDA Processor
Source: https://github.com/hybridgroup/yzma/blob/main/examples/installer/README.md
Execute the installer using 'go run' specifying the CUDA processor and the upgrade flag to install the latest llama.cpp version.
```shell
$ go run ./examples/installer/ -processor cuda -upgrade
installing llama.cpp version b6924 to /home/ron/Development/yzma/lib
done.
```
--------------------------------
### Run SpaceQwen2.5 VLA Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Execute a VLA example with the SpaceQwen2.5-VL-3B-Instruct model. This command requires the model, projector, an image, and a prompt.
```bash
$ go run ./examples/vlm/ -model ~/models/SpaceQwen2.5-VL-3B-Instruct.i1-Q4_K_M.gguf --mmproj ~/models/spaceqwen2.5-vl-3b-instruct-vision.gguf -p "What is in this picture? Provide a description, bounding box, and estimated distance for the llama in json format." -sys "You are a helpful robotic drone camera currently in flight." -image ./images/domestic_llama.jpg
```
--------------------------------
### Install the Describe Command-Line Tool
Source: https://github.com/hybridgroup/yzma/blob/main/examples/describe/README.md
Install the 'describe' tool using the Go build command. Ensure you are in the root of the hybridgroup/yzma project.
```shell
go install ./examples/describe
```
--------------------------------
### Install yzma CLI Tool
Source: https://github.com/hybridgroup/yzma/blob/main/cmd/README.md
Installs the yzma command-line tool using go install. Ensure you have Go installed and configured.
```shell
go install github.com/hybridgroup/yzma@latest
```
--------------------------------
### Run Qwen3-VL-2B-Instruct VLM Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
This command runs an example for the Qwen3-VL-2B-Instruct Vision Language Model. Ensure you have the model, projector, and an image file specified.
```bash
go run ./examples/vlm/ -model ~/models/Qwen_Qwen3-VL-2B-Instruct-Q4_K_M.gguf -mmproj ~/models/mmproj-Qwen_Qwen3-VL-2B-Instruct-f16.gguf -image ./images/domestic_llama.jpg -p "What is in this picture?"
```
--------------------------------
### Install llama.cpp Libraries (macOS/Linux CPU)
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs the llama.cpp libraries to a specified local path. This command is used for macOS and Linux CPU installations.
```bash
yzma install --lib /path/to/lib
```
--------------------------------
### Install llama.cpp with Short Flags
Source: https://github.com/hybridgroup/yzma/blob/main/cmd/README.md
Demonstrates installing llama.cpp using short flag equivalents for common options like library path, version, processor, and upgrade.
```shell
# Using short flags
yzma install -l /path/to/lib -v b1234 -p cuda -u
```
--------------------------------
### Run Chat Example Model
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Execute a chat example with a specified model. Ensure the model file exists at the provided path.
```bash
go run ./examples/chat/ -model ~/models/Qwen_Qwen3-0.6B-Q4_K_M.gguf -temp=0.6 -n=512
```
--------------------------------
### Run Tool Use Example with Default Model
Source: https://github.com/hybridgroup/yzma/blob/main/examples/tooluse/README.md
Execute the tooluse example with a specified model. This demonstrates a basic tool-calling scenario.
```shell
$ go run ./examples/tooluse/ -model ~/models/qwen2.5-0.5b-instruct-fp16.gguf
=== Tool Calling Example ===
User: What is 15 + 27?
Assistant: 15 + 27 = 42
The result of adding 15 and 27 is 42.
```
--------------------------------
### Run Tool Use Example with Specific Question
Source: https://github.com/hybridgroup/yzma/blob/main/examples/tooluse/README.md
Execute the tooluse example with a specified model and a direct question. This shows how to pass a question as an argument.
```shell
$ go run ./examples/tooluse/ -model ~/models/qwen2.5-0.5b-instruct-fp16.gguf -question="what is 9 times 9?"
=== Tool Calling Example ===
User: what is 9 times 9?
Assistant: 9 times 9 is 81.
9 times 9 = 81
The final answer is 81.
```
--------------------------------
### Install llama.cpp Libraries (Linux CUDA)
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs the llama.cpp libraries with CUDA support for GPU acceleration on Linux. Ensure CUDA drivers are installed first.
```bash
yzma install --lib /path/to/lib --processor cuda
```
--------------------------------
### Install llama.cpp to a Specific Path
Source: https://github.com/hybridgroup/yzma/blob/main/cmd/README.md
Installs llama.cpp pre-built binaries to a specified directory using the --lib flag.
```shell
# Install to specific path
yzma install --lib /path/to/lib
```
--------------------------------
### Run InternVLA Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Execute a VLA example using the InternVLA-M1 model. This command requires the model, projector, an image, and a prompt.
```bash
go run ./examples/vlm/ -model ~/models/InternVLA-M1.Q8_0.gguf --mmproj ~/models/InternVLA-M1.mmproj-Q8_0.gguf -p "What is in this picture? Provide a description, bounding box, and estimated distance for the llama in json format." -sys "You are a helpful robotic drone camera currently in flight." -image ./images/domestic_llama.jpg
```
--------------------------------
### Run Qwen2.5 Chat Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Run a chat example using the downloaded Qwen2.5 model. Specify the model path and other parameters like temperature and token count.
```bash
go run ./examples/chat/ -model ~/models/qwen2.5-0.5b-instruct-q4_k_m.gguf -temp=0.6 -n=512
```
--------------------------------
### Run Model Info Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/modelinfo/README.md
Execute the modelinfo example with a specified GGUF model file. This command will output detailed information about the model.
```shell
go run ./examples/modelnfo/ -model /path/to/model.gguf
```
```shell
$ go run ./examples/modelinfo/ -model ~/models/gemma-3-1b-it-q4_0.gguf
```
--------------------------------
### Run LFM2.5-VL-1.6B VLM Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Execute an example for the LFM2.5-VL-1.6B Vision Language Model. Ensure the model, projector, and image path are correctly specified.
```bash
go run ./examples/vlm/ -model ~/models/LFM2.5-VL-1.6B-Q4_0.gguf -mmproj ~/models/mmproj-LFM2.5-VL-1.6b-Q8_0.gguf -image ./images/domestic_llama.jpg -p "What is in this picture?"
```
--------------------------------
### Run Gemma Chat Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Run a chat example using the Gemma-3-1B-IT model. Specify the model path for execution.
```bash
go run ./examples/chat/ -model ~/models/gemma-3-1b-it-Q4_K_M.gguf
```
--------------------------------
### Run System Information Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/systeminfo/README.md
Execute the systeminfo example to display device and llama.cpp system information. This command will output details about available CUDA devices and the CPU's supported instruction sets.
```shell
$ go run ./examples/systeminfo/
-- Devices --
Device 0: CUDA0
Device 1: CPU
-- llama.cpp System Information --
CUDA : ARCHS = 860,890 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
```
--------------------------------
### Install yzma with Vulkan on Linux
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs yzma with Vulkan processor support on Linux. Ensure Vulkan drivers are installed first.
```shell
yzma install --lib /path/to/lib --processor vulkan
```
--------------------------------
### Run Qwen3-4B-GGUF Text Generation Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
This command runs a text generation example using the Qwen3-4B model. Adjust temperature and token count as needed.
```bash
go run ./examples/chat/ -model ~/models/Qwen3-4B-Q4_K_M.gguf -temp=0.6 -n=512
```
--------------------------------
### Run VLM Example with YZMA
Source: https://github.com/hybridgroup/yzma/blob/main/examples/vlm/README.md
Execute the VLM example using the Go command. Ensure you have the model, mmproj, and image files in the specified paths.
```shell
go run ./examples/vlm/ -model ~/models/Qwen2.5-VL-3B-Instruct-Q8_0.gguf -mmproj ~/models/mmproj-Qwen2.5-VL-3B-Instruct-Q8_0.gguf -image ./images/domestic_llama.jpg -p "What is in this picture?"
```
--------------------------------
### Multi-step Tool Calling Example Output
Source: https://github.com/hybridgroup/yzma/blob/main/examples/multitool/README.md
Illustrates a multi-step tool calling process, including user input, assistant's tool calls, detected tool calls with arguments and results, and the final answer.
```text
$ go run ./examples/multitool -model ~/models/Qwen3-VL-2B-Instruct-Q8_0.gguf -question "Tell me what is (15 + 27) * 3, then tell me what is (50 + 33) * 4, then calculate the flying speed of a swallow. After that then tell me 100 * 43 times that flying speed."
=== Multi-Step Tool Calling Example ===
User: Tell me what is (15 + 27) * 3, then tell me what is (50 + 33) * 4, then calculate the flying speed of a swallow. After that then tell me 100 * 43 times that flying speed.
=== Iteration 1 ===
Assistant: First, let's calculate (15 + 27) * 3:
{"name": "add", "arguments": {"a": 15, "b": 27}}
{"name": "multiply", "arguments": {"a": 3, "b": 45}}
Next, let's calculate (50 + 33) * 4:
{"name": "add", "arguments": {"a": 50, "b": 33}}
{"name": "multiply", "arguments": {"a": 4, "b": 83}}
Now, let's calculate the flying speed of a swallow. The average flying speed of a swallow is approximately 10 meters per second.
Then, we'll calculate 100 * 43 times that flying speed:
{"name": "multiply", "arguments": {"a": 100, "b": 43}}
{"name": "multiply", "arguments": {"a": 10, "b": 4300}}
The final answer is the result of the last calculation. Let's do that.
{"name": "multiply", "arguments": {"a": 10, "b": 4300}}
=== Detected 7 Tool Call(s) ===
[1] Function: add
Arguments: {"a":"15","b":"27"}
Result: 42.00
[2] Function: multiply
Arguments: {"a":"3","b":"45"}
Result: 135.00
[3] Function: add
Arguments: {"a":"50","b":"33"}
Result: 83.00
[4] Function: multiply
Arguments: {"a":"4","b":"83"}
Result: 332.00
[5] Function: multiply
Arguments: {"a":"100","b":"43"}
Result: 4300.00
[6] Function: multiply
Arguments: {"a":"10","b":"4300"}
Result: 43000.00
[7] Function: multiply
Arguments: {"a":"10","b":"4300"}
Result: 43000.00
=== Iteration 2 ===
Assistant: The result of (15 + 27) * 3 is 135.
The result of (50 + 33) * 4 is 332.
The flying speed of a swallow is not provided in the query, so I cannot calculate the exact flying speed. However, based on typical data, a swallow can fly at speeds of around 10 to 15 meters per second.
The result of 100 * 43 times the flying speed of a swallow is 43000.00.
Therefore, the final answer is 43000.00.
=== Final Answer (no more tool calls) ===
The result of (15 + 27) * 3 is 135.
The result of (50 + 33) * 4 is 332.
The flying speed of a swallow is not provided in the query, so I cannot calculate the exact flying speed. However, based on typical data, a swallow can fly at speeds of around 10 to 15 meters per second.
The result of 100 * 43 times the flying speed of a swallow is 43000.00.
Therefore, the final answer is 43000.00.
```
--------------------------------
### Run SpaceQwen2.5 VLA Example Output
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Example JSON output from the SpaceQwen2.5-VL-3B-Instruct VLA model, providing bounding box, label, and estimated distance.
```json
{
"bbox_2d": [40, 20, 67, 35],
"label": "llama",
"estimated_distance": "1.5 meters"
}
```
--------------------------------
### Run InternVLA Example Output
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Example JSON output from the InternVLA-M1 VLA model, providing label, bounding box, and distance.
```json
{"label": "llama", "bbox_2d": [43, 352, 647, 822], "distance": 10.0}
```
--------------------------------
### Installer Command-Line Flags
Source: https://github.com/hybridgroup/yzma/blob/main/examples/installer/README.md
Reference for the available command-line flags for the installer program, including options for help, library path, processor type, upgrade, and version.
```shell
-help string
show help
-lib string
path to llama.cpp compiled library files (leave empty to use YZMA_LIB env var)
-processor string
processor to use (cpu, cuda, metal, vulkan) (default "cpu")
-upgrade
upgrade existing installation
-version string
version of llama.cpp to install (leave empty for latest)
```
--------------------------------
### Run Embeddings Example
Source: https://github.com/hybridgroup/yzma/blob/main/examples/embeddings/README.md
Execute the embeddings example with a local model and a text prompt. The output will be a numerical vector representing the embedding.
```shell
$ go run ./examples/embeddings/ -model ~/models/SmolLM-135M.Q2_K.gguf -p "Hello World"
-0.009294 -0.003438 0.004629 -0.005551 0.003048 0.004245 -0.026342 -0.013455 0.024450 -0.016283 -0.007215 -0.002221 0.028461 -0.009824 -0.074387 0.002456 0.000385 0.005366 0.005864 -0.008317 0.009290 0.029637 -0.022828 0.031284 -0.027327 -0.004377 0.015330 -0.004048 -0.008763 -0.028619 0.019352 -0.086852 0.005373 0.008012 -0.001056 0.020512 -0.026946 0.006843 -0.350056 0.010412 0.008697 -0.032326 0.005726 0.029471 -0.005899 -0.037877 0.004732 0.004339 0.004682 0.000675 0.066087 0.019378 -0.018498 -0.005532 -0.040203 0.020117 0.026317 -0.000038 0.011160 0.366920 0.003074 0.003564 -0.003546 -0.008394 0.001933 0.011310 -0.013500 0.019717 -0.011379 -0.006733 0.008523 0.005653 -0.006199 -0.003720 -0.017128 0.022056 -0.000022 -0.018016 0.005120 -0.019274 -0.023972 0.007885 0.057404 -0.022332 -0.000257 0.010177 -0.006600 0.004635 -0.008806 -0.006519 0.019043 -0.002400 0.022735 -0.017498 0.030291 0.006090 0.002094 0.007653 -0.013877 -0.003526 0.023533 -0.002207 -0.010918 0.009498 0.016913 0.008177 0.008351 0.008877 -0.007721 0.001599 0.000942 0.013530 0.000382 0.000549 0.032687 -0.034191 0.004187 0.027761 -0.000883 -0.008853 0.015638 0.018709 0.034082 -0.001951 -0.043994 0.048507 0.031798 -0.003628 -0.006571 -0.007348 -0.009426 0.009931 -0.018174 0.006690 0.005256 -0.014743 -0.016806 0.076989 0.008792 -0.023598 -0.204146 0.009500 -0.003226 0.011616 -0.017259 -0.007851 -0.007528 -0.020459 0.021579 -0.005841 -0.008600 0.007185 0.037630 0.017205 0.014291 -0.003039 -0.032115 0.004234 0.004249 -0.013451 0.005640 0.049510 0.001347 0.008653 0.005214 -0.015064 0.007493 0.004493 0.003971 -0.027876 -0.007108 0.013640 0.007069 0.014737 0.015190 0.001510 -0.022362 0.011587 -0.001232 -0.002467 0.035419 -0.014710 -0.000285 -0.022228 -0.049875 0.008767 0.006536 0.002699 0.040382 -0.016606 0.007632 0.006721 0.012126 0.021254 -0.011685 -0.002364 0.013657 -0.004036 0.005235 -0.080048 -0.014253 0.026099 -0.018668 0.002287 -0.028133 -0.004854 0.001343 0.011049 -0.000906 0.006002 0.005035 -0.010946 0.024173 0.018683 -0.007696 0.000728 0.006861 -0.022654 0.008060 0.011827 0.014805 0.001385 0.000469 0.001041 0.117387 -0.010227 0.018633 0.011632 -0.010318 -0.010616 -0.011712 -0.000238 -0.001990 0.015564 0.016649 -0.004155 0.015080 -0.013632 0.021798 -0.004125 0.006376 -0.002246 -0.014517 -0.015706 -0.015362 0.019608 -0.012896 -0.003242 -0.028408 -0.002223 0.011485 -0.023718 -0.009686 0.025559 -0.007473 0.002376 0.025898 0.010104 -0.014540 0.008452 -0.000097 0.008545 -0.015669 0.002078 -0.005053 -0.009616 0.008982 0.182888 0.008555 -0.014526 0.000596 -0.008744 -0.021412 -0.001803 0.012503 -0.005819 0.004841 0.011593 -0.009761 -0.002378 -0.026656 0.033827 -0.009942 -0.004817 -0.006367 0.017203 -0.026291 0.052526 -0.012147 -0.023652 -0.007877 -0.014496 0.003353 0.017669 0.014576 -0.004669 0.003518 0.000599 0.004280 0.034702 0.000528 -0.009289 -0.003675 -0.475297 -0.022867 0.006588 0.003851 -0.000066 0.033669 0.020303 0.011807 0.021626 -0.007720 -0.008870 0.010188 -0.020345 -0.012348 0.014122 -0.008394 0.011429 0.004497 -0.012894 -0.008270 0.035153 0.008964 0.007033 -0.013388 -0.000615 0.012533 -0.004421 -0.019222 -0.014865 -0.019599 0.015851 -0.006950 -0.010854 0.056598 0.015705 -0.027555 0.055638 0.030018 -0.009473 -0.018070 0.009129 -0.015566 -0.002056 -0.025975 -0.025816 -0.000039 -0.012531 -0.001222 -0.024580 -0.007266 -0.007296 -0.006999 -0.022426 -0.017072 0.007941 -0.013217 0.002429 0.031590 0.034501 0.012757 -0.000692 0.139424 -0.004850 0.022750 0.003531 0.009165 0.016407 -0.007635 -0.279694 -0.001265 -0.022567 -0.015918 -0.022029 -0.013511 0.001776 0.024392 0.008396 -0.015653 -0.016821 -0.008474 -0.004853 -0.001240 0.011519 0.011669 0.002298 -0.002394 0.007571 -0.024275 -0.035274 0.004212 0.016985 -0.006570 0.009654 0.217346 0.124502 -0.009721 0.020639 -0.024440 -0.000880 0.018449 0.277163 0.020574 0.004435 0.004617 0.004508 -0.015539 0.026073 -0.007535 -0.008762 0.011087 -0.011268 0.010718 0.005335 -0.010370 0.009961 -0.005889 -0.010854 0.001639 0.001766 -0.012997 -0.001267 -0.003391 -0.023273 -0.012704 0.010400 0.009928 -0.028458 0.005621 0.005785 -0.012331 -0.012173 0.010067 -0.000992 0.012858 -0.022465 0.000942 -0.001604 0.010308 -0.019494 0.021680 -0.026196 0.001912 0.005955 0.002391 -0.011373 -0.004782 -0.007032 -0.030814 -0.015098 0.004048 0.008324 0.003898 -0.006457 -0.012564 0.041164 -0.023170 0.028478 0.003008 0.008305 -0.011099 -0.008642 -0.004916 0.011275 0.012587 0.015166 -0.012230 -0.011522 0.008072 0.007406 0.000232 0.004559 -0.019521 -0.010930 0.011808 -0.016565 0.002163 -0.014520 -0.014930 -0.025087 0.017157 -0.014805 0.004862 -0.008538 -0.019253 0.000401 0.006187 -0.004919 0.009315 0.001966 -0.014516 -0.001594 0.008164 0.018165 0.008666 0.015600 0.024113 -0.014162 0.008052 -0.003477 -0.026149 0.005396 -0.004517 -0.004010 0.008297 0.010977 -0.023384 0.023408 -0.004210 0.004346 0.027236 -0.006821 -0.017775 0.00
```
--------------------------------
### Run moondream2 VLM Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
This command runs an example using the moondream2 VLM. Ensure the model, projector, and image paths are correctly set.
```bash
go run ./examples/vlm/ -model ~/models/moondream2-text-model-f16_ct-vicuna.gguf -mmproj ~/models/moondream2-mmproj-f16-20250414.gguf -image ./images/domestic_llama.jpg -p "What is in this picture?"
```
--------------------------------
### Download Qwen2.5-VL-3B-Instruct Model and Projector
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download both the Qwen2.5-VL-3B-Instruct model and its corresponding projector file in Q8_0 quantization. These are needed to run VLM examples.
```bash
yzma model get -u https://huggingface.co/ggml-org/Qwen2.5-VL-3B-Instruct-GGUF/resolve/main/Qwen2.5-VL-3B-Instruct-Q8_0.gguf
yzma model get -u https://huggingface.co/ggml-org/Qwen2.5-VL-3B-Instruct-GGUF/resolve/main/mmproj-Qwen2.5-VL-3B-Instruct-Q8_0.gguf
```
--------------------------------
### Run a Local LLM with yzma in Go
Source: https://github.com/hybridgroup/yzma/blob/main/README.md
This example demonstrates how to load a language model, tokenize a prompt, and generate a response using yzma and llama.cpp. Ensure the llama.cpp library is accessible via the YZMA_LIB environment variable and the model file is downloaded.
```go
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/hybridgroup/yzma/pkg/download"
"github.com/hybridgroup/yzma/pkg/llama"
)
var (
modelFile = "SmolLM2-135M.Q4_K_M.gguf"
prompt = "Are you ready to go?"
libPath = os.Getenv("YZMA_LIB")
responseLength int32 = 12
)
func main() {
llama.Load(libPath)
llama.LogSet(llama.LogSilent())
llama.Init()
model, _ := llama.ModelLoadFromFile(filepath.Join(download.DefaultModelsDir(), modelFile), llama.ModelDefaultParams())
ctx, _ := llama.InitFromModel(model, llama.ContextDefaultParams())
vocab := llama.ModelGetVocab(model)
tokens := llama.Tokenize(vocab, prompt, true, false)
batch := llama.BatchGetOne(tokens)
sampler := llama.SamplerChainInit(llama.SamplerChainDefaultParams())
llama.SamplerChainAdd(sampler, llama.SamplerInitGreedy())
for pos := int32(0); pos < responseLength; pos += batch.NTokens {
llama.Decode(ctx, batch)
token := llama.SamplerSample(sampler, ctx, -1)
if llama.VocabIsEOG(vocab, token) {
fmt.Println()
break
}
buf := make([]byte, 36)
len := llama.TokenToPiece(vocab, token, buf, 0, true)
fmt.Print(string(buf[:len]))
batch = llama.BatchGetOne([]llama.Token{token})
}
fmt.Println()
}
```
--------------------------------
### Run SmolLM2 Chat Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Execute a chat example with the SmolLM2-135M-Instruct model. This includes parameters for context size, temperature, token count, and a system prompt.
```bash
go run ./examples/chat/ -model ~/models/SmolLM2-135M-Instruct-Q4_K_M.gguf -c 2048 -temp 0.8 -n 48 -sys "You are a helpful robot companion."
```
--------------------------------
### Install llama.cpp with Default Settings
Source: https://github.com/hybridgroup/yzma/blob/main/cmd/README.md
Installs llama.cpp pre-built binaries using default settings, typically utilizing the YZMA_LIB environment variable for the library path.
```shell
# Install with default settings (uses YZMA_LIB env var)
yzma install
```
--------------------------------
### Run TinyLlama Chat Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Execute a chat example with the TinyLlama model. This command includes parameters for context size, temperature, token count, and a system prompt.
```bash
go run ./examples/chat/ -model ~/models/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf -c 2048 -temp 0.7 -n 512 -sys "You are a helpful robot companion."
```
--------------------------------
### Run Pelican VLA Example
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Execute a vision-language action (VLA) example with the Pelican1.0-VL-3B model. This command requires both the model and projector files, along with an image and a prompt.
```bash
$ go run ./examples/vlm/ -model ~/models/Pelican1.0-VL-3B.i1-Q4_K_M.gguf --mmproj ~/models/Pelican1.0-VL-3B.mmproj-Q8_0.gguf -p "What is in this picture? Provide a description, bounding box, and estimated distance for the llama in json format." -sys "You are a helpful robotic drone camera currently in flight." -image ./images/domestic_llama.jpg
```
--------------------------------
### Run Pelican VLA Example Output
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Example JSON output from the Pelican1.0-VL-3B VLA model, providing description, bounding box, and estimated distance.
```json
{
"description": "The image shows a fluffy white llama standing on a green grassy area with a dirt path nearby. The llama has a curly coat and appears to be in a fenced-in area with trees and some buildings in the background.",
"bounding_box_2d": [40, 155, 635, 811],
"estimated_distance": "The llama is approximately 1 meter away from the camera."
}
```
--------------------------------
### Download Qwen2.5 Model
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download the Qwen2.5-0.5B-Instruct model using the yzma model get command. This command fetches the model from a Hugging Face repository.
```bash
yzma model get -u https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-q4_k_m.gguf
```
--------------------------------
### Download SpaceQwen2.5 VLA Model
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download the SpaceQwen2.5-VL-3B-Instruct model and its associated vision projector file. Use the yzma model get command for both.
```bash
yzma model get -u https://huggingface.co/mradermacher/SpaceQwen2.5-VL-3B-Instruct-i1-GGUF/resolve/main/SpaceQwen2.5-VL-3B-Instruct.i1-Q4_K_M.gguf
yzma model get -u https://huggingface.co/remyxai/SpaceQwen2.5-VL-3B-Instruct/resolve/main/spaceqwen2.5-vl-3b-instruct-vision.gguf
```
--------------------------------
### Download YZMA Model
Source: https://github.com/hybridgroup/yzma/blob/main/examples/hello/README.md
Use this command to download a model to the default location. Ensure YZMA is installed and the YZMA_LIB environment variable is set.
```shell
yzma model get -u https://huggingface.co/QuantFactory/SmolLM2-135M-GGUF/resolve/main/SmolLM2-135M.Q4_K_M.gguf
```
--------------------------------
### Install llama.cpp Libraries with ROCm
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs the llama.cpp libraries with ROCm support for AMD GPU acceleration on Linux. yzma can auto-detect an existing ROCm installation.
```bash
yzma install --lib /path/to/lib --processor rocm
```
--------------------------------
### Install ROCm 7.2 on Ubuntu 22.04
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs ROCm 7.2 for AMD GPU support on Ubuntu 22.04. This involves downloading a package, updating apt, and installing ROCm.
```shell
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/jammy/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo apt install rocm
```
--------------------------------
### Install ROCm 7.2 on Ubuntu 24.04
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs ROCm 7.2 for AMD GPU support on Ubuntu 24.04. This involves downloading a package, updating apt, and installing ROCm.
```shell
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/noble/amdgpu-install_7.2.70200-1_all.deb
sudo apt install ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo apt install rocm
```
--------------------------------
### Verify ROCm Installation
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Verifies the ROCm installation by checking the ROCm management interface information.
```shell
rocminfo
```
--------------------------------
### Install Linux Vulkan Drivers
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs necessary Vulkan drivers and tools on Debian-based Linux systems.
```shell
sudo apt install -y mesa-vulkan-drivers vulkan-tools
```
--------------------------------
### Upgrade Existing llama.cpp Installation
Source: https://github.com/hybridgroup/yzma/blob/main/cmd/README.md
Upgrades an existing llama.cpp installation to the latest version in the specified library path.
```shell
# Upgrade existing installation
yzma install --lib /path/to/lib --upgrade
```
--------------------------------
### Set YZMA_LIB Environment Variable
Source: https://github.com/hybridgroup/yzma/blob/main/examples/installer/README.md
Before running the installer, set the YZMA_LIB environment variable to specify the target installation directory.
```shell
export YZMA_LIB="/home/ron/Development/yzma/lib"
```
--------------------------------
### Download Qwen3-VL-2B-Instruct Model (Q4_K_M)
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Use this command to download the Qwen3-VL-2B-Instruct model in Q4_K_M quantization format. This is a good quality, default size option for most use cases.
```bash
yzma model get -u https://huggingface.co/bartowski/Qwen_Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen_Qwen3-VL-2B-Instruct-Q4_K_M.gguf
```
--------------------------------
### Download Qwen3-VL-2B-Instruct Model (IQ4_XS)
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Use this command to download a smaller version of the Qwen3-VL-2B-Instruct model in IQ4_XS quantization format. This option offers decent quality and is smaller than Q4_K_S.
```bash
yzma model get -u https://huggingface.co/bartowski/Qwen_Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen_Qwen3-VL-2B-Instruct-IQ4_XS.gguf
```
--------------------------------
### Download SmolVLM2-500M-Video-Instruct Model (Q4_K_S)
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download the SmolVLM2-500M-Video-Instruct model in Q4_K_S quantization. This option offers optimal size, speed, and quality.
```bash
yzma model get -u https://huggingface.co/mradermacher/SmolVLM2-500M-Video-Instruct-i1-GGUF/resolve/main/SmolVLM2-500M-Video-Instruct.i1-Q4_K_S.gguf
```
--------------------------------
### Install Specific llama.cpp Version with CUDA
Source: https://github.com/hybridgroup/yzma/blob/main/cmd/README.md
Installs a specific version of llama.cpp pre-built binaries to a specified directory, utilizing CUDA as the processor.
```shell
# Install specific version with CUDA
yzma install --lib /path/to/lib --version b1234 --processor cuda
```
--------------------------------
### Download SmolVLM2-500M-Video-Instruct Model (Q4_K_M)
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download the SmolVLM2-500M-Video-Instruct model in Q4_K_M quantization. This is a fast and recommended option for this VLM.
```bash
yzma model get -u https://huggingface.co/mradermacher/SmolVLM2-500M-Video-Instruct-i1-GGUF/resolve/main/SmolVLM2-500M-Video-Instruct.i1-Q4_K_M.gguf
```
--------------------------------
### Install yzma with CPU on Raspberry Pi OS (Trixie)
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs yzma with CPU processor support on Raspberry Pi OS (Trixie, 64-bit).
```shell
yzma install --lib /path/to/lib --processor cpu --os trixie
```
--------------------------------
### Download Qwen3-VL-2B-Instruct Projector
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
This command downloads the necessary projector file for the Qwen3-VL-2B-Instruct model. It is required for vision language model functionalities.
```bash
yzma model get -u https://huggingface.co/bartowski/Qwen_Qwen3-VL-2B-Instruct-GGUF/resolve/main/mmproj-Qwen_Qwen3-VL-2B-Instruct-f16.gguf
```
--------------------------------
### Install yzma with CPU on Raspberry Pi OS (Bookworm)
Source: https://github.com/hybridgroup/yzma/blob/main/INSTALL.md
Installs yzma with CPU processor support on Raspberry Pi OS (Bookworm, 64-bit legacy).
```shell
yzma install --lib /path/to/lib --processor cpu --os bookworm
```
--------------------------------
### Run CPU Benchmarks on Linux amd64
Source: https://github.com/hybridgroup/yzma/blob/main/BENCHMARKS.md
Executes Go benchmarks for text inference on a Linux amd64 system using the CPU. The benchmark runs for 10 seconds per iteration, repeated 5 times.
```shell
$ go test -benchtime=10s -count=5 -run=nada -bench .
goos: linux
goarch: amd64
pkg: github.com/hybridgroup/yzma/pkg/llama
cpu: 13th Gen Intel(R) Core(TM) i9-13900HX
BenchmarkInference-32 99 110913774 ns/op 270.5 tokens/s
BenchmarkInference-32 100 111035054 ns/op 270.2 tokens/s
BenchmarkInference-32 100 110369390 ns/op 271.8 tokens/s
BenchmarkInference-32 100 112705133 ns/op 266.2 tokens/s
BenchmarkInference-32 100 111892770 ns/op 268.1 tokens/s
PASS
ok github.com/hybridgroup/yzma/pkg/llama 61.199s
```
--------------------------------
### Download Qwen3-0.6B-GGUF Model
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download the Qwen3-0.6B model in Q4_K_M quantization format. This is a smaller text generation model.
```bash
yzma model get -u https://huggingface.co/bartowski/Qwen_Qwen3-0.6B-GGUF/resolve/main/Qwen_Qwen3-0.6B-Q4_K_M.gguf
```
--------------------------------
### Download LFM2.5-VL-1.6B Model (Q8_0)
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download the LFM2.5-VL-1.6B model in Q8_0 quantization. This is a high-quality option.
```bash
yzma model get -u https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-GGUF/resolve/main/LFM2.5-VL-1.6B-Q8_0.gguf
```
--------------------------------
### CUDA Device Information
Source: https://github.com/hybridgroup/yzma/blob/main/BENCHMARKS.md
Displays information about the NVIDIA GPU and its driver/CUDA versions. This is useful for verifying hardware compatibility and setup.
```bash
C:\Users\ron>nvidia-smi
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 581.57 Driver Version: 581.57 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Driver-Model | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3070 WDDM | 00000000:01:00.0 Off | N/A |
| 0% 42C P8 6W / 240W | 22MiB / 8192MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
```
--------------------------------
### amd64 Benchmark with Vulkan Backend
Source: https://github.com/hybridgroup/yzma/blob/main/BENCHMARKS.md
Runs Go benchmarks for YZMA inference with a specified context length and Vulkan device.
```bash
go test -benchtime=10s -count=5 -run=nada -bench . -nctx=32000 -device="vulkan0"
```
```text
goos: linux
goarch: amd64
pkg: github.com/hybridgroup/yzma/pkg/llama
cpu: AMD EPYC 7443P 24-Core Processor
BenchmarkInference-48 328 36234037 ns/op 828.0 tokens/s
BenchmarkInference-48 339 35194859 ns/op 852.4 tokens/s
BenchmarkInference-48 333 35395438 ns/op 847.6 tokens/s
BenchmarkInference-48 338 35334138 ns/op 849.0 tokens/s
BenchmarkInference-48 339 35255138 ns/op 850.9 tokens/s
PASS
ok github.com/hybridgroup/yzma/pkg/llama 61.232s
```
--------------------------------
### Download Gemma Model
Source: https://github.com/hybridgroup/yzma/blob/main/MODELS.md
Download the Gemma-3-1B-IT model. Use the yzma model get command to fetch the GGUF file.
```bash
yzma model get -u https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF/resolve/main/gemma-3-1b-it-Q4_K_M.gguf
```