### Embed Server Startup Output

Source: https://github.com/chrishayuk/larql/blob/main/docs/adr/0008-embed-server.md

Example output displayed after successfully starting the larql embed server, showing configuration details.

```text
LARQL Embed Server v0.4.1
  Model:      google/gemma-4-31B-it
  Vocab:      262,208 tokens
  Hidden:     5,376
  Embeddings: 2.7 GB  (mmap)
  lm_head:    2.7 GB  (tied, mmap)
  Tokenizer:  loaded
  Mode:       embed-service
  Listening:  http://0.0.0.0:8082
  Ready.
```

--------------------------------

### Grid Mode Quickstart for larql-router

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/router-spec.md

Start larql-router with grid mode enabled, then launch larql-server instances that join the grid. Servers announce their capabilities at runtime.

```bash
# Router — grid gRPC on 50052, HTTP on 9090
larql-router --grid-port 50052 --port 9090

# Servers connect to the router on startup, no router restart needed
larql-server output/gemma3-4b-q4k.vindex \
  --ffn-only --layers 0-16 \
  --join "http://127.0.0.1:50052" \
  --public-url "http://127.0.0.1:8080" \
  --port 8080

larql-server output/gemma3-4b-q4k.vindex \
  --ffn-only --layers 17-33 \
  --join "http://127.0.0.1:50052" \
  --public-url "http://127.0.0.1:8081" \
  --port 8081
```

--------------------------------

### Static Mode Quickstart for larql-router

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/router-spec.md

Configure and run larql-server instances for specific layer ranges, then start larql-router with a static shard map.

```bash
larql-server output/gemma3-4b-q4k.vindex --ffn-only --layers 0-16  --port 8080
larql-server output/gemma3-4b-q4k.vindex --ffn-only --layers 17-33 --port 8081

larql-router \
  --shards "0-16=http://127.0.0.1:8080,17-33=http://127.0.0.1:8081" \
  --port 9090
```

--------------------------------

### Run Server Demo Example

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Executes the server_demo example, which does not require a real vindex.

```bash
cargo run -p larql-server --example server_demo
```

--------------------------------

### gRPC Server Start Command

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/server-spec.md

Example command to start the Larql server with both HTTP and gRPC ports enabled. Requires a vindex file and specifies ports for HTTP and gRPC.

```bash
larql serve gemma3-4b.vindex --port 8080 --grpc-port 50051
```

--------------------------------

### Run Embed Demo Example

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Executes the embed_demo example, which does not require a real vindex.

```bash
cargo run -p larql-server --example embed_demo
```

--------------------------------

### Run Larql Server Demo Examples

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Execute synthetic demonstration examples for the larql-server. These examples showcase server functionality without requiring a real vector index.

```bash
cargo run -p larql-server --example server_demo
```

```bash
cargo run -p larql-server --example embed_demo
```

--------------------------------

### Larql Server Startup Log Example

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/server-spec.md

Example of the server's startup log when running in FFN-only mode with layer sharding.

```log
larql-server v0.1.0
Loading: output/gemma3-4b-q4k.vindex
  Layers: 0–16 (of 34)
  Model: google/gemma-3-4b-it (34 layers, 348160 features)
  Warmup: skipped (--ffn-only, lazy gate decode on first request)
  Mode: ffn-service (--ffn-only)
Listening: http://0.0.0.0:8080
```

--------------------------------

### Run larql-compute Examples

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-compute/examples/README.md

Command to run any example from the larql-compute crate. Requires release mode and the 'metal' feature.

```bash
cargo run --release --features metal -p larql-compute --example <name>
```

--------------------------------

### Run model-compute example

Source: https://github.com/chrishayuk/larql/blob/main/crates/model-compute/README.md

Execute the 'gauss' example provided with the model-compute crate to see native kernels in action.

```bash
cargo run --example gauss -p model-compute
```

--------------------------------

### Check Larql Server Examples

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Compiles all examples for the larql-server crate to ensure they are syntactically correct.

```bash
cargo check -p larql-server --examples
```

--------------------------------

### Serve Command with API Key

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Example of starting the Larql server with an API key for authentication. All endpoints except /v1/health will require a Bearer token.

```bash
larql serve output/gemma3-4b-v2.vindex --api-key "sk-abc123"
```

--------------------------------

### Serve Command with Rate Limiting

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Example of starting the Larql server with a specified rate limit. Supports N/sec, N/min, N/hour formats.

```bash
larql serve output/gemma3-4b-v2.vindex --rate-limit "100/min"
```

--------------------------------

### Grid with 3x redundancy deployment example

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/router-spec.md

Configures a LARQL grid with 3x redundancy across 10 servers for different layer ranges, including router setup and server startup commands.

```bash
larql-router --grid-port 50052 --grid-key "$KEY" --port 9090

# Range A: layers 0-10, 4 replicas
for port in 8080 8081 8082 8083; do
  larql-server model.vindex --ffn-only --layers 0-10 --port $port \
    --join "http://router:50052" --grid-key "$KEY" \
    --public-url "http://$(hostname):$port" &
done

# Range B: layers 11-22, 3 replicas
# Range C: layers 23-33, 3 replicas
```

--------------------------------

### Start larql-server

Source: https://github.com/chrishayuk/larql/blob/main/docs/inference-engine.md

Command to start the `larql-server` with a specified vindex path and port.

```APIDOC
## Start larql-server

```bash
cargo run --release -p larql-server -- path/to/vindex --port 8080
```
```

--------------------------------

### Demo: Start Router

Source: https://github.com/chrishayuk/larql/blob/main/docs/adr/0004-ffn-grid.md

Starts the LARQL Grid Router. It listens for incoming server connections and administrative commands. Initially, the grid is empty.

```bash
# Terminal 1: Start the router (nothing configured — just listening)
$ larql-router start --port 50051 --admin-port 9090

  LARQL Grid Router v0.4.1
  Grid:    empty
  Listening: grpc://0.0.0.0:50051
  Admin:     http://0.0.0.0:9090
  Ready. Waiting for servers to join.
```

--------------------------------

### Run Inference Engine Examples

Source: https://github.com/chrishayuk/larql/blob/main/README.md

Execute various examples for the inference engine, including demos for attention, mechanistic interpretation, and benchmarks. Some examples require specific features like the Metal GPU backend.

```bash
cargo run --release -p larql-inference --example attention_demo    # fused attention demo
```

```bash
cargo run --release -p larql-inference --example mech_interp_demo  # capture / lens / ablate / steer / patch (synthetic — no vindex)
```

```bash
cargo run --release -p larql-inference --example bench_attention   # attention benchmarks
```

```bash
cargo run --release -p larql-inference --example backend_demo --features metal   # backend demo
```

```bash
cargo run --release -p larql-inference --example bench_backend --features metal  # backend benchmarks
```

```bash
cargo run --release -p larql-inference --example bench_inference   # full inference benchmarks
```

--------------------------------

### Run Larql Server Examples

Source: https://github.com/chrishayuk/larql/blob/main/README.md

Commands to run the Larql server, which allows walking inference over HTTP. Examples include a synthetic HTTP surface demo, an embed/logits/token demo, and server operation benchmarks. Some require a path to vindex data.

```bash
cargo run --release -p larql-server -- path/to/vindex --port 8080
```

```bash
cargo run -p larql-server --example server_demo             # synthetic HTTP surface demo
```

```bash
cargo run -p larql-server --example embed_demo              # synthetic embed/logits/token demo
```

```bash
cargo run --release -p larql-server --example server_bench  # synthetic server operation benchmark
```

```bash
cargo run --release -p larql-server --example bench_embed_server -- path/to/vindex
```

--------------------------------

### Install and Run Larql Server (Bare Metal/VPS)

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/server-spec.md

Installs the Larql server using Cargo and provides commands to run it directly or configure it as a systemd service.

```bash
# Install
cargo install larql-server

# Run
larql-server /path/to/model.vindex --port 8080

# Systemd service
[Unit]
Description=LARQL Vindex Server
After=network.target

[Service]
ExecStart=/usr/local/bin/larql-server /data/gemma3-4b.vindex --port 8080
Restart=always
MemoryMax=8G

[Install]
WantedBy=multi-user.target
```

--------------------------------

### Two-Shard Local Deployment Example

Source: https://github.com/chrishayuk/larql/blob/main/docs/ffn/distributed.md

Set up a two-shard FFN model locally using larql-server for each shard and larql-router to manage requests. This example uses Gemma 3 4B with 34 layers.

```bash
# Terminal A
larql-server output/gemma3-4b-q4k.vindex --ffn-only --layers 0-16 --port 8080

# Terminal B
larql-server output/gemma3-4b-q4k.vindex --ffn-only --layers 17-33 --port 8081

# Terminal C
larql-router --shards "0-16=http://127.0.0.1:8080,17-33=http://127.0.0.1:8081" --port 9090

# Client — unchanged
larql walk --ffn-remote http://127.0.0.1:9090 --predict --prompt "The capital of France is"
```

--------------------------------

### Start Larql Server with gRPC

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Enable gRPC alongside HTTP by specifying a gRPC port when starting the server. The proto definition for services is available in `proto/vindex.proto`.

```bash
larql serve output/gemma3-4b-v2.vindex --port 8080 --grpc-port 50051
```

--------------------------------

### Load Vindex and Describe Entity

Source: https://github.com/chrishayuk/larql/blob/main/docs/larql-python.md

Load a vindex from a local file and describe knowledge edges for a given entity. This is a basic example to get started with querying.

```python
import larql

vindex = larql.load("gemma3-4b.vindex")
edges = vindex.describe("France")
# [Edge(relation="capital", target="Paris", confidence=0.97, source="probe"), ...]
```

--------------------------------

### Demonstrate Model Loading

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-models/README.md

Execute an example to load a real model and inspect its structure. Provide the path to the model as an argument.

```bash
cargo run -p larql-models --example demo_loading -- /path/to/model
```

--------------------------------

### Setup LARQL Python Environment

Source: https://github.com/chrishayuk/larql/blob/main/experiments/README.md

Installs the LARQL Python package using maturin. Ensure you are in the 'crates/larql-python' directory before running.

```bash
cd crates/larql-python
maturin develop --release
```

--------------------------------

### Run OpenAI-Compat Live Demo

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Boots an in-process server and exercises OpenAI-compatible API endpoints against a real vindex. This command requires the release profile and a specified vindex file.

```bash
cargo run --release -p larql-server --example openai_demo -- \
  output/gemma3-4b-q4k-streaming.vindex
```

--------------------------------

### Quickstart: Load and Query Vindex

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-python/README.md

Basic usage of the larql library to load a vindex, perform knowledge queries, and insert data.

```python
import larql

# Load a vindex
vindex = larql.load("output/gemma3-4b-v2.vindex")

# Knowledge queries — instant, no inference
edges = vindex.describe("France")
for e in edges[:5]:
    print(f"  {e.relation} → {e.target}  score={e.gate_score:.0f}")

# Full inference — Rust attention + walk FFN
result = vindex.infer("The capital of France is")
# [("Paris", 0.805), ...]

# Insert knowledge — no training
vindex.insert("Colchester", "country", "England")

# Bulk gate vectors for research (SVD, PCA)
gates = vindex.gate_vectors(layer=26)     # numpy (10240, 2560)
```

--------------------------------

### Live OpenAI Demo Server Bootstrapping

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/ROADMAP.md

Example of booting the Larql server in-process using `tower::ServiceExt::oneshot` to exercise OpenAI API endpoints.

```rust
examples/openai_demo.rs
```

--------------------------------

### Demonstrate Architecture Detection

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-models/README.md

Run an example to showcase architecture detection, including all 12 architectures, tensor keys, sliding window, MoE, and quantization.

```bash
cargo run -p larql-models --example architecture_demo
```

--------------------------------

### Start Larql Server for Sharded Decode

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-vindex/README.md

This command starts a larql-server instance for a specific shard of a multi-shard setup. It specifies the vindex file, port, layer range, and cache configuration. The `--no-infer` flag is used when the server is only responsible for routing and not inference.

```bash
larql-server <vindex> --port 9181 --layers 0-14 --no-infer \
    --max-q4k-cache-layers 1 --warmup-walk-ffn
```

```bash
larql-server <vindex> --port 9182 --layers 15-29 --no-infer \
    --max-q4k-cache-layers 1 --warmup-walk-ffn
```

--------------------------------

### Python Bindings Setup

Source: https://github.com/chrishayuk/larql/blob/main/AGENTS.md

Sets up the development environment for Python bindings using `uv`. This command creates a virtual environment and installs development dependencies.

```bash
cd crates/larql-python
uv sync --no-install-project --group dev
```

--------------------------------

### Larql Compute Quick Start

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-compute/README.md

Demonstrates basic usage of the larql-compute crate, including backend initialization, matrix multiplication, and quantized matrix-vector operations. It also shows how to probe for hardware capabilities and perform decode/prefill operations.

```rust
use larql_compute::prelude::*;
use larql_compute::{default_backend, QuantFormat};

let backend = default_backend();
println!("Using: {} ({})", backend.name(), backend.device_info());

// f32 matmul
let c = backend.matmul_transb(a.view(), b.view());

// Unified quant matvec — dispatches on format. Q4_K / Q4_KF / Q6_K
// take f32 input directly; Q4_0 / Q8_0 internally re-quantise.
let scores = backend.quant_matvec(QuantFormat::Q4_K, &q4k_data, &x, rows, hidden);

// Pre-quantised fast path for hot decode loops (avoid re-quantising
// the layer's input on every gate/up matvec):
let scores = backend.q4_matvec(&q4_0_data, &q8_x, &q8_scales, rows, hidden);

// Capability probe — branch on what the backend accelerates instead
// of pattern-matching on `Option<…> = None`.
if backend.supports(Capability::F32Gemv) {
    let logits = backend.f32_gemv_force(lm_head.view(), &h_last);
}

// KV-cached decode (one token through all layers).
let h = backend.decode_token(&layers, &x, hidden, inter, q_dim, kv_dim,
    num_q_heads, num_kv_heads, head_dim, rope_base);

// GPU prefill (seq>1, populates KV cache).
let h = backend.prefill_q4(&layers, &x, hidden, inter, q_dim, kv_dim,
    seq_len, num_q_heads, num_kv_heads, head_dim, rope_base, qk_norm, softcap);
```

--------------------------------

### Inference Request (Compare Mode)

Source: https://github.com/chrishayuk/larql/blob/main/docs/inference-engine.md

Example of an HTTP POST request to the `/v1/infer` endpoint using 'compare' mode to get side-by-side predictions from walk and dense modes.

```APIDOC
## Inference Request (Compare Mode)

### Description
Performs inference using the 'compare' mode, which provides side-by-side predictions from both 'walk' and 'dense' modes for comparison.

### Method
POST

### Endpoint
/v1/infer

### Request Body
- **prompt** (string) - Required - The input text prompt for inference.
- **top** (integer) - Required - The number of top predictions to return.
- **mode** (string) - Required - The inference mode, should be "compare".

### Request Example
```json
{
  "prompt": "The capital of France is",
  "top": 3,
  "mode": "compare"
}
```
```

--------------------------------

### Demonstrate Tensor Key Comparison

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-models/README.md

Run an example to compare tensor key patterns across different architectures.

```bash
cargo run -p larql-models --example demo_tensor_keys
```

--------------------------------

### Multi-Model Serving

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Details on serving multiple models when using the `--dir` flag, where each vindex resides in its own namespace. Examples show how to construct GET requests for specific models and entities.

```APIDOC
## Multi-Model Serving

When using `--dir`, each vindex gets its own namespace:

```bash
larql serve --dir ./vindexes/ --port 8080
# /v1/gemma-3-4b-it/describe, /v1/llama-3-8b/describe, ...
```

```
GET /v1/gemma-3-4b-it/describe?entity=France
GET /v1/llama-3-8b/describe?entity=France
```
```

--------------------------------

### Query Vindex via HTTP API

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/README.md

Example of how to query the served vindex using a simple HTTP GET request to the /v1/describe endpoint. This demonstrates accessing knowledge graph information.

```bash
curl "http://localhost:8080/v1/describe?entity=France"
# {"entity":"France","edges":[{"relation":"capital","target":"Paris","gate_score":1436.9,"layer":27,"source":"probe"}, ...]}
```

--------------------------------

### Insert Edges into Vindex (KNN Mode)

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-lql/README.md

This SQL example shows the default KNN mode for inserting edges, where the residual at the install layer is stored as a key in the KnnStore for retrieval-based overrides.

```sql
-- Default: KNN mode, residual captured at knowledge.hi − 1.
INSERT INTO EDGES (entity, relation, target)
    VALUES ("Atlantis", "capital-of", "Poseidon");
```

--------------------------------

### LQL Session Example

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-python/README.md

Demonstrates how to create an LQL session for querying and interacting with the vindex.

```python
session = larql.session("model.vindex")
session.query("DESCRIBE 'France'")
session.query("WALK 'The capital of France is' TOP 10")
session.vindex.gate_vectors(layer=26)  # numpy access on same session
```

--------------------------------

### Get Model and Index Statistics with GET /v1/stats

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/server-spec.md

The GET /v1/stats endpoint provides statistics about the loaded model and index, including layer information and memory usage.

```http
GET /v1/stats
```

--------------------------------

### Serve Single Server with All Experts

Source: https://github.com/chrishayuk/larql/blob/main/README.md

The simplest setup where a single server hosts all experts. Use this for testing or when all experts can fit on one machine.

```bash
larql serve gemma4-26b-a4b.vindex --port 8080
```

--------------------------------

### Rebalance Knowledge Graph

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-lql/docs/spec.md

REBALANCE is used for global fixed-point rebalancing of compose-mode installs. It iterates through installed edges, infers their canonical prompt, and scales their down_col. KNN installs do not require rebalancing.

```sql
REBALANCE
    [UNTIL CONVERGED]
    [MAX <n>]
    [FLOOR <float>]
    [CEILING <float>];
```

```sql
REBALANCE;
```

```sql
REBALANCE UNTIL CONVERGED MAX 16 FLOOR 0.30 CEILING 0.90;
```

--------------------------------

### Run Larql Demos

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-vindex/README.md

Execute various demonstration examples for larql-vindex, showcasing features, memory mapping, and Q4K streaming.

```bash
cargo run -p larql-vindex --example demo_features
```

```bash
cargo run --release -p larql-vindex --example mmap_demo
```

```bash
cargo run --release -p larql-vindex --example q4k_demo
```

```bash
cargo run --release -p larql-vindex --example demo_memit_solve
```

--------------------------------

### LARQL Quick Start Commands

Source: https://github.com/chrishayuk/larql/blob/main/README.md

Essential commands for building LARQL, pulling pre-built vindex files, listing cached models, running inference, and extracting local vindex files.

```bash
# Build
carqo build --release
```

```bash
# Pull a pre-built vindex from HuggingFace
larql pull hf://chrishayuk/gemma-3-4b-it-vindex
```

```bash
# List what's cached
larql list
```

```bash
# Run it — one-shot or chat
larql run gemma-3-4b-it-vindex "The capital of France is"
larql run gemma-3-4b-it-vindex          # drops into chat mode
```

```bash
# Or extract locally — inference-ready at f16 by default
larql extract google/gemma-3-4b-it -o gemma3-4b.vindex
larql run gemma3-4b.vindex "Einstein is known for"
```

--------------------------------

### Install and Verify Larql Python Package

Source: https://github.com/chrishayuk/larql/blob/main/docs/larql-python.md

Provides commands for installing the Larql Python package via pip or from source using uv and maturin. Includes a verification step to ensure the installation is successful.

```bash
# From PyPI
pip install larql

# From source — managed via uv
cd crates/larql-python
uv sync --no-install-project --group dev
uv run --no-sync maturin develop --release

# Verify
uv run --no-sync python -c "import larql; print(larql.load('test.vindex').stats())"
```

--------------------------------

### Start larql REPL

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-cli/README.md

Launch the interactive LQL REPL to experiment with queries and commands.

```bash
cargo run --release -p larql-cli -- repl
```

--------------------------------

### LARQL Metadata Example

Source: https://github.com/chrishayuk/larql/blob/main/docs/format.md

Provides an example of the metadata object, used for storing extraction provenance information.

```json
{
  "model": "google/gemma-3-4b-it",
  "method": "weight-extract",
  "extraction_date": "2026-03-27"
}
```

--------------------------------

### Demo: First Server Joins

Source: https://github.com/chrishayuk/larql/blob/main/docs/adr/0004-ffn-grid.md

Starts a LARQL server that announces a specific range of layers for a model. The router registers this server and updates its coverage status.

```bash
# Terminal 2: First server joins — announces layers 0-19
$ larql-server output/gemma4-31b-q4k.vindex \
    --ffn-only --layers 0-19 \
    --join grpc://localhost:50051

  [server-a] Connected to router
  [server-a] Announcing: gemma4-31b-q4k layers=0-19
  [server-a] Registered. Serving.

# Router output:
  [router] server-a joined: gemma4-31b-q4k layers=0-19  ✓
  [router] Coverage: 33%  Gaps: 20-39, 40-59
```

--------------------------------

### Install Larql Dependencies

Source: https://github.com/chrishayuk/larql/blob/main/knowledge/README.md

Command to install all necessary dependencies for the Larql project, including optional extras.

```bash
# 1. Install everything
pip install -e ".[all]"
```

--------------------------------

### Run Backend Demo

Source: https://github.com/chrishayuk/larql/blob/main/docs/inference-engine.md

Executes the backend demo, showcasing routing, cache, and calibration. Requires the 'metal' feature.

```bash
cargo run --release -p larql-inference --example backend_demo --features metal
```

--------------------------------

### LARQL Interactive Query Examples

Source: https://github.com/chrishayuk/larql/blob/main/README.md

Demonstrates basic LQL commands for using a vindex, describing entities, inserting edges, and performing inference.

```sql
larql> USE "gemma3-4b.vindex";
Using: gemma3-4b.vindex (34 layers, 348.2K features, relations: 512 types)
```

```sql
larql> DESCRIBE "France";
France
  Edges (L14-27):
    capital     → Paris              1436.9  L27  (probe)
    language    → French               35.2  L24  (probe)
    continent   → Europe               14.4  L25  (probe)
    borders     → Spain                13.3  L18  (probe)
```

```sql
larql> INSERT INTO EDGES (entity, relation, target)
   ...   VALUES ("John Coyle", "lives-in", "Colchester");
Inserted 1 edge. Feature F8821@L26 allocated.
```

```sql
larql> INFER "The capital of France is" TOP 3;
  1. Paris                (97.91%)
  2. the                  (0.42%)
  3. a                    (0.31%)
```

--------------------------------

### Format Decision Flow Example

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-vindex/docs/compute-integration.md

Outlines the logic for selecting the appropriate quantization format and compute shaders during vindex build and inference.

```text
Vindex build: user chooses quantization level
  → build_q4k_weights: creates attn_weights_q4k.bin
  → build_attn_q8: creates attn_weights_q8.bin

Inference: auto-selects best available (attention)
  if has_q4k → Q4_K path (q4kf_qkv_proj shader, skip Q8 quantize)
  elif has_q8 → Q8 path (q8_qkv_proj shader, fused Q8 input)
  else → f32 path (CPU BLAS matmul_transb)

Inference: auto-selects best available (FFN)
  if has_interleaved_q4k → Q4_K FFN (QuantFormat::Q4_K on FullPipelineLayer)
  elif has_interleaved_q4 → Q4_0 FFN (QuantFormat::Q4_0, fallback)
  else → CPU walk FFN (f32 sparse)
```

--------------------------------

### List Active Patches with GET /v1/patches

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/server-spec.md

The GET /v1/patches endpoint lists all currently active patches on the server.

```json
{
  "patches": [
    {"name": "drug-interactions@2.1.0", "operations": 5000, "source": "hf://medical-ai/drug-interactions@2.1.0"}
  ]
}
```

--------------------------------

### Authenticated Request Example

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/server-spec.md

Example of making a request to a protected endpoint, including the necessary Authorization header with the API key.

```http
GET /v1/describe?entity=France
Authorization: Bearer sk-abc123
```

--------------------------------

### Initialize Compute Backend

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-inference/README.md

Demonstrates how to initialize the default compute backend, which auto-selects between CPU and Metal (GPU). It prints the name and device information of the selected backend.

```rust
use larql_compute::{default_backend, ComputeBackend};

let backend = default_backend();  // Auto-selects CPU or Metal, calibrates
println!("Using: {} ({})", backend.name(), backend.device_info());
```

--------------------------------

### Health Check with GET /v1/health

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-server/docs/server-spec.md

The GET /v1/health endpoint returns the current status, uptime, and requests served by the server.

```json
{"status": "ok", "uptime_seconds": 3600, "requests_served": 12450}
```

--------------------------------

### Start Larql Server

Source: https://github.com/chrishayuk/larql/blob/main/crates/larql-inference/README.md

Command to start the larql inference server. It requires a path to vindex files and can be configured with a port.

```bash
cargo run --release -p larql-server -- path/to/vindex --port 8080
```

--------------------------------

### Larql CLI Command Example

Source: https://github.com/chrishayuk/larql/blob/main/docs/virtual-experts-dispatch.md

Illustrates the command-line interface for running larql, including options for specifying experts, expert directories, and constrained operations.

```bash
larql run <model> --experts [--experts-dir <DIR>] [--ops <CSV>] [--constrained]
```

--------------------------------

### Serve model with authentication and HTTPS

Source: https://github.com/chrishayuk/larql/blob/main/docs/cli.md

Starts a server with API key authentication and HTTPS enabled. Requires certificate and key files.

```bash
larql serve output/gemma3-4b.vindex --api-key "sk-abc123" --tls-cert cert.pem --tls-key key.pem
```

--------------------------------

### Install larql-knowledge in Dev Mode

Source: https://github.com/chrishayuk/larql/blob/main/knowledge/README.md

Installs the project in editable mode, including optional WordNet dependencies. Use this for development.

```bash
pip install -e ".[wordnet]"
```