### Serve model with 1bit serve on booted system

Source: https://github.com/1bit-monster/engine/blob/main/os/README.md

Start the 1bit engine server on the booted system, serving a model from the 1BIT-DATA partition. The --device auto flag selects HRX0.

```sh
1bit serve -m /data/models/<file.gguf> --host 0.0.0.0     # --device auto: HRX0
```

--------------------------------

### Serve Ternary-Bonsai-2-27B with 1bit serve

Source: https://github.com/1bit-monster/engine/blob/main/docs/hrx.md

Starts the 1bit server with the Ternary-Bonsai-2-27B-PTQ1_0.gguf model. The --device auto flag selects hrx for this file, and files in PrismML's types run on --device hrx only.

```sh
1bit serve -m Ternary-Bonsai-2-27B-PTQ1_0.gguf        # --device auto picks hrx for this file
```

--------------------------------

### Setup Hyperloom on Strix Halo (gfx1151)

Source: https://github.com/1bit-monster/engine/blob/main/tools/hyperloom/README.md

Sets up the Hyperloom workspace for Strix Halo: creates a venv, installs Hyperloom 1.0.0 and Ray 2.55.1, patches the package, starts a Ray head with a GPU and serving_slot resource, copies the custom benchmark script and reference JSON, then launches the detached optimization run. Requires a worktree of the fork built with GGML_HRX=ON and the ZAYA1-8B Q4_K_M GGUF at the specified paths.

```sh
python3 -m venv ~/hyperloom-ws/.venv && . ~/hyperloom-ws/.venv/bin/activate
pip install "hyperloom-inference-optimizer[runtime]==1.0.0" "ray[default]==2.55.1"
PKG=~/hyperloom-ws/.venv/lib/python3.14/site-packages/hyperloom
python tools/hyperloom/hl_patch_strixhalo.py $PKG
python tools/hyperloom/hl_cli_login.py $PKG
# Hyperloom needs a Ray head with a GPU and a serving_slot resource
ray start --head --num-gpus=1 --resources='{"serving_slot":1}' --disable-usage-stats
mkdir -p ~/hyperloom-zaya/scripts && cp tools/hyperloom/custom_strixhalo.sh ~/hyperloom-zaya/scripts/
cp tools/hyperloom/zaya1-8b-q4_k_m.reference.json ~/hyperloom-zaya/reference.json
# ~/hyperloom-zaya/llama.cpp: a worktree of the fork built with GGML_HRX=ON
# ~/hyperloom-zaya/model -> the ZAYA1-8B Q4_K_M GGUF
cd ~/hyperloom-zaya && (setsid nohup bash /path/to/tools/hyperloom/launch.sh > hyperloom.log 2>&1 < /dev/null &)
```

--------------------------------

### Recipe JSON structure for 1bit serve

Source: https://github.com/1bit-monster/engine/blob/main/docs/recipes.md

Example recipe entry for 1bit serve's config/recipes.json. Each recipe defines what it matches (device, drafter), the flags to add, and requires measured and source fields. A recipe without a measurement does not load.

```json
{
  "id": "example",
  "match": {"device": ["hrx"], "drafter": ["dflash"]},
  "args": ["--spec-draft-p-min", "0.4"],
  "why": "what the setting does for this match",
  "measured": "the model, file, settings and numbers, measured with tools/bench.py",
  "source": "the PR that added it"
}
```

--------------------------------

### Fetch and run Laya routing with 1bit CLI

Source: https://github.com/1bit-monster/engine/blob/main/docs/laya.md

Shows the commands to fetch the Laya model and run routing with the 1bit CLI. The first command fetches the Laya model, the second starts a server with Laya routing enabled, and the third classifies a sample request, outputting the class, confidence, and device.

```shell
scripts/fetch-laya.sh
1bit serve -m model.gguf --device auto --laya            # or --laya-model DIR
1bit route --laya-model DIR --classify --state "Write a Rust function that sums a slice."
# code 0.44 hrx
```

--------------------------------

### Build pinned XDNA stack and configure engine

Source: https://github.com/1bit-monster/engine/blob/main/docs/npu.md

Builds the pinned XRT and XDNA shim, then configures the engine to use that prefix. Nothing is installed system-wide; both installs use DESTDIR because XRT hard-codes /etc/OpenCL/vendors.

```bash
scripts/build-xdna.sh ~/.cache/xdna-pin/prefix
```

```bash
cmake -B build -G Ninja -DONEBIT_NPU=ON -DONEBIT_XRT_ROOT=$HOME/.cache/xdna-pin/prefix/root/opt/xilinx/xrt
```

--------------------------------

### 1bit serve command-line usage

Source: https://github.com/1bit-monster/engine/blob/main/docs/serve.md

Start the 1bit serve backend with a single model. The command accepts many optional flags for device selection, context size, model aliasing, and RAG support. The server exposes only an OpenAI-compatible API and can run standalone or as a Lemonade backend.

```sh
1bit serve -m <model> [--port 8000] [--host 127.0.0.1]
           [--device auto|npu|hrx|cpu|ds4|mlx|onnx] [--ctx-size N] [--alias NAME]
           [--llama-server PATH] [--ds4 PATH] [--ssd-streaming] [--hrx-libhsa PATH] [--mlx-server PATH]
           [--mtp HEAD.gguf | --dflash DRAFT.gguf] [--mtp-max N] [--mtp-p-min P] [--mmproj MMPROJ.gguf]
           [--laya | --laya-model DIR] [--route-policy FILE]
           [--moe-slots N|auto] [--moe-subst R] [--moe-prefetch N]
           [--parallel N]
           [--embed MODEL.gguf] [--rerank MODEL.gguf]
           [--npu-opt KEY=VALUE ...]
```

--------------------------------

### Check 1bit OS image in QEMU with 1bit.check

Source: https://github.com/1bit-monster/engine/blob/main/os/README.md

Boot the 1bit OS initramfs in QEMU/KVM to check the image without the hardware. The 1bit.check init parameter makes init verify that the engine, llama-server, the HSA runtime, the XRT plugin and the NPU firmware are in place and start, then print 1BIT-OS CHECK PASS or FAIL and power off.

```sh
qemu-system-x86_64 -enable-kvm -cpu host -m 4G -kernel /boot/vmlinuz-$(uname -r) \
  -initrd out/initramfs.img -append "console=ttyS0 rdinit=/init 1bit.check" -nographic -no-reboot
```

--------------------------------

### Build and run the GGUF scorer with 1bit serve

Source: https://github.com/1bit-monster/engine/blob/main/docs/laya.md

Builds ggmlc's laya on HRX with TheRock's amdclang, fetches the GGUF, and starts the server with the GGUF scorer. Requires the binary and GGUF present; otherwise the C++ scorer on safetensors is used.

```sh
cmake -B build -DONEBIT_LAYA_GGML=ON ...   # builds ggmlc's laya on HRX with TheRock's amdclang
scripts/fetch-laya.sh
1bit serve -m MODEL.gguf --laya            # uses the GGUF scorer when both are there
```

--------------------------------

### Run 1bit serve standalone and query with curl

Source: https://github.com/1bit-monster/engine/blob/main/docs/lemonade.md

Starts a standalone OpenAI-compatible server with 1bit serve, then sends a chat completion request to it. The --device auto flag selects HRX0 if available, otherwise the CPU. The curl command posts a simple user message to the /v1/chat/completions endpoint.

```sh
1bit serve -m ~/models/Qwen3-0.6B-Q4_K_M.gguf --port 8000      # --device auto: HRX0, else the CPU
curl http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \
     -d '{"messages": [{"role": "user", "content": "Hello"}]}'
```

--------------------------------

### Build 1bit OS image with cmake and mkimage.sh

Source: https://github.com/1bit-monster/engine/blob/main/os/README.md

Build the 1bit OS image on a machine that has built the engine with the NPU lane and HRX (TheRock in /opt/rocm-therock). The mkimage.sh script copies the HRX llama-server and the HSA runtime the build was configured with (ONEBIT_HRX_LIBHSA in build/CMakeCache.txt) at their build paths, with every library they link; 1bit has both paths compiled in.

```sh
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DONEBIT_NPU=ON -DONEBIT_HRX=ON
cmake --build build
os/mini/mkimage.sh out build
```

--------------------------------

### Serve a model with 1bit serve on ds4

Source: https://github.com/1bit-monster/engine/blob/main/docs/dwarfstar.md

Starts the 1bit serve command with the ds4 device, specifying a model file and context size. The server waits for ds4-server's /v1/models endpoint before opening its port.

```shell
1bit serve --device ds4 -m ~/models/ds4/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx-size 8192
```

--------------------------------

### Initialize submodules and build with HRX

Source: https://github.com/1bit-monster/engine/wiki/GPU-HRX-and-Vulkan

Initializes the HRX and llama.cpp submodules, configures the build with ONEBIT_HRX=ON (requires TheRock at /opt/rocm-therock), and builds the onebit target.

```bash
git submodule update --init --depth 1 third_party/hrx-system third_party/llama.cpp
cmake -B build -G Ninja -DONEBIT_HRX=ON     # needs TheRock at /opt/rocm-therock
cmake --build build --target onebit
```

--------------------------------

### Serve MLX model with 1bit serve

Source: https://github.com/1bit-monster/engine/blob/main/docs/apple.md

Starts the MLX server on macOS. The -m flag takes an mlx-community Hugging Face id, which the server downloads on first load. The --mlx-server option overrides the server binary path; otherwise it falls back to $LEMONADE_MLX_SERVER and then lemon-mlx-server on PATH.

```bash
1bit serve -m mlx-community/Qwen3-0.6B-4bit --device mlx --port 8000 [--mlx-server PATH]
```

--------------------------------

### Run onebit serve via Lemonade

Source: https://github.com/1bit-monster/engine/blob/main/docs/lemonade.md

Runs the onebit engine through Lemonade's recipe, mirroring the llama-server recipe. The backend selector picks the device (hrx or npu; unset sends --device auto), and replies carry Lemonade's model name via --alias.

```bash
1bit serve -m <model> --port <p> [--device ...] [--ctx-size N] [--alias <name>]
```