### Run System Setup Script

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Executes a setup script to install all necessary system packages and tools, including build tools, pipx, and the CUDA Toolkit. Run this script as root.

```bash
sudo bash dflash/scripts/setup_system.sh
```

--------------------------------

### DGX Spark / GB10 Quick Start

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Quick start guide for DGX Spark / GB10, requiring CUDA 12.9+. CMake automatically adds the necessary architecture. Ensure your nvcc version meets the requirement.

```bash
# CUDA 12.9+ required for sm_121
nvcc --version  # must show >= 12.9
git clone --recurse-submodules https://github.com/Luce-Org/lucebox-hub && cd lucebox-hub/dflash
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release   # CMake auto-adds sm_121
cmake --build build --target test_dflash -j
```

--------------------------------

### Set Up OpenAI-Compatible HTTP Server

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Create a virtual environment, install dependencies, and run a Python script to start an HTTP server compatible with OpenAI API standards.

```bash
python3 -m venv .venv
.venv/bin/pip install fastapi uvicorn transformers jinja2
.venv/bin/python scripts/server.py --port 8000 --daemon
```

--------------------------------

### Jetson AGX Thor Quick Start

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Quick start guide for Jetson AGX Thor, requiring CUDA 13.0+. The build process is similar to other platforms.

```bash
# CUDA 13.0+ required for sm_110
nvcc --version  # must show >= 13.0
git clone --recurse-submodules https://github.com/Luce-Org/lucebox-hub && cd lucebox-hub/dflash
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release   # CMake auto-adds sm_110
cmake --build build --target test_dflash -j
```

--------------------------------

### Install and Run Megakernel Benchmark

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Clone the repository, set up a virtual environment, and install dependencies to run the Megakernel benchmark. Weights are streamed from HuggingFace on the first run. Ensure Python 3.10+, CUDA 12+, and PyTorch 2.0+ are installed.

```bash
# 1. clone + enter
git clone https://github.com/Luce-Org/lucebox-hub && cd lucebox-hub/megakernel

# 2. install (Python 3.10+, CUDA 12+, PyTorch 2.0+). Weights stream from HF on first run.
python -m venv .venv && source .venv/bin/activate   # required on Ubuntu 24+ system Python (PEP 668) 
pip install --upgrade pip
pip install torch                          # install BEFORE the next step; setup.py imports torch at build time
pip install -e . --no-build-isolation      # --no-build-isolation lets the build see the torch you just installed

# 3. run the benchmark (prefill pp520 + decode tg128 vs llama.cpp BF16 + PyTorch HF)
python final_bench.py
```

--------------------------------

### Start DFlash Server

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Use this command to start the DFlash server. Ensure you have the target and draft models specified.

```bash
python dflash/scripts/server.py \
  --target models/Qwen3.5-27B-Q4_K_M.gguf \
  --draft models/Qwen3.5-3B-f16.safetensors \
  --budget 22 --port 8080
```

--------------------------------

### Install and Run Luce Megakernel

Source: https://github.com/luce-org/lucebox-hub/blob/main/megakernel/README.md

Clone the repository, set up a virtual environment, install dependencies, and run the performance benchmark. Ensure PyTorch is installed before the editable install of the package.

```bash
git clone https://github.com/Luce-Org/luce-megakernel
cd luce-megakernel
python3 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install torch                          # install BEFORE the next step; setup.py imports torch at build time
pip install -e . --no-build-isolation      # --no-build-isolation lets the build see the torch you just installed
python final_bench.py    # runs pp520 tg128 (properly warmed), prints tok/s
```

--------------------------------

### Start DFlash Server

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/CODEX.md

Start the DFlash server using Python, specifying the draft model and port. Ensure the server is running by checking its output.

```bash
cd dflash
python scripts/server.py \
  --draft models/draft/draft-Qwen3.6-27B.gguf \
  --port 8080
```

--------------------------------

### Start the DFlash server

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Command to start the DFlash server with specified target and draft models, and budget.

```APIDOC
## Start the DFlash server

### Description
Starts the DFlash server, making it available for requests.

### Command
```bash
python dflash/scripts/server.py \
  --target models/Qwen3.5-27B-Q4_K_M.gguf \
  --draft models/Qwen3.5-3B-f16.safetensors \
  --budget 22 --port 8080
```

### Parameters
- `--target` (string): Path to the target model file.
- `--draft` (string): Path to the draft model file.
- `--budget` (integer): The budget for inference.
- `--port` (integer): The port to run the server on.
```

--------------------------------

### Start DFlash Server and Use Codex CLI

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Start the DFlash server locally and then use the Codex CLI to interact with it. This demonstrates a typical local development workflow.

```bash
# Start the server
python dflash/scripts/server.py --port 8080

# In another terminal
codex --provider dflash "Explain this codebase"
```

--------------------------------

### Install System Packages

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Installs essential system packages for building and running DFlash. This includes build tools, git, and the CUDA Toolkit.

```bash
build-essential  cmake  git  git-lfs  nvcc (CUDA Toolkit)
```

--------------------------------

### Install Python Packages

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Installs the required Python packages for the DFlash server and API. Use this command within an activated virtual environment.

```bash
pip install fastapi uvicorn transformers pydantic starlette
```

--------------------------------

### Run DFlash Server

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Starts the DFlash server. This command assumes you are in the dflash directory and have the necessary models and daemon binary built.

```bash
cd dflash
python scripts/server.py
```

--------------------------------

### Install PFlash Bench Harness

Source: https://github.com/luce-org/lucebox-hub/blob/main/pflash/README.md

Installs the Python-based benchmarking harness for PFlash. This is used for benchmarking purposes only and does not affect the runtime inference path.

```bash
cd ../pflash
python -m venv .venv && source .venv/bin/activate
pip install -e .
```

--------------------------------

### Start OpenAI Server with Prefill Compression

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Launches the DFlash server with automatic prefill compression enabled for prompts exceeding a specified threshold. Configure the drafter model and keep ratio for compression.

```bash
python3 dflash/scripts/server.py \
  --target dflash/models/Qwen3.6-27B-Q4_K_M.gguf \
  --draft  dflash/models/draft/model.safetensors \
  --max-ctx 8192 --budget 16 --fa-window 0 \
  --prefill-compression auto \
  --prefill-threshold 4096 \
  --prefill-keep-ratio 0.05 \
  --prefill-drafter dflash/models/Qwen3-0.6B-BF16.gguf \
  --port 8080
```

--------------------------------

### Install Megakernel Python Extension

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Installs the megakernel as a PyTorch C++ extension. Ensure PyTorch is installed before this step. The build process auto-detects GPU architecture and SM count.

```bash
# Prerequisites: Python 3.10+, CUDA 12+, PyTorch 2.0+
git clone https://github.com/Luce-Org/lucebox-hub
cd lucebox-hub/megakernel

python -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install torch                        # MUST come before the next line
pip install -e . --no-build-isolation    # --no-build-isolation lets the build see torch

# Verify CUDA arch detection (printed during build, also available afterwards):
python -c "import torch; p=torch.cuda.get_device_properties(0); print(p.name, 'sm_%d%d'%(p.major,p.minor))"
```

--------------------------------

### Install PFlash Python Harness

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Sets up the Python environment for the PFlash harness, including creating a virtual environment and installing the package in editable mode. This enables the use of Python tooling for benchmarks.

```bash
cd lucebox-hub/pflash
python -m venv .venv && source .venv/bin/activate
pip install -e .
```

--------------------------------

### Install Codex CLI

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/CODEX.md

Install the Codex CLI globally using npm. Verify the installation by checking the version.

```bash
npm install -g @openai/codex
codex --version          # should print 0.1xx.x
```

```bash
mkdir -p ~/.npm-global && npm config set prefix ~/.npm-global
export PATH=~/.npm-global/bin:$PATH   # add to ~/.bashrc
npm install -g @openai/codex
```

--------------------------------

### Run OpenAI-Compatible Server for Laguna

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Starts an OpenAI-compatible server for the Laguna model. Does not require a --draft argument. Configurable with max context length and port.

```bash
python3 scripts/server.py \
    --target models/laguna-xs2-Q4_K_M.gguf \
    --max-ctx 16384 --port 8000
```

--------------------------------

### Install System Dependencies for dflash

Source: https://github.com/luce-org/lucebox-hub/blob/main/CONTRIBUTING.md

Run this script to install all necessary system dependencies for dflash on Ubuntu 22.04 or 24.04. It includes build tools and the CUDA Toolkit.

```bash
sudo dflash/scripts/setup_system.sh
```

--------------------------------

### Start DFlash Server for OpenAI Codex CLI

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Starts the DFlash server configured to serve responses via the OpenAI Codex wire API. Ensure the target and draft models are correctly specified.

```bash
python3 dflash/scripts/server.py \
  --target models/Qwen3.5-27B-Q4_K_M.gguf \
  --draft  models/Qwen3.5-3B-f16.safetensors \
  --budget 22 --port 8080
```

--------------------------------

### Create and Activate Python Virtual Environment

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Sets up a Python virtual environment to manage project dependencies. Activate the environment before installing Python packages.

```bash
python3 -m venv venv
source venv/bin/activate
```

--------------------------------

### Build and Test dflash

Source: https://github.com/luce-org/lucebox-hub/blob/main/CONTRIBUTING.md

After system setup and initializing git submodules, use CMake to build the dflash project in release mode and run its tests.

```bash
git submodule update --init --recursive
cmake -B dflash/build -S dflash -DCMAKE_BUILD_TYPE=Release
cmake --build dflash/build --target test_dflash -j
```

--------------------------------

### Start Multi-Turn Chat REPL

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Launch an interactive chat interface for multi-turn conversations with the model.

```python
python3 examples/chat.py
```

--------------------------------

### Install Python Test Packages

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Installs the pytest package for running Python unit tests. Ensure this is done within the activated virtual environment.

```bash
pip install pytest
```

--------------------------------

### Run Script with Asymmetric KV Quantization (Short Flags)

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Execute the run.py script with asymmetric KV quantization settings using short CLI flags. This example sets K to q8_0 and V to q4_0.

```bash
python3 scripts/run.py --ctk q8_0 --ctv q4_0 --prompt "hello"
```

--------------------------------

### Generate NIAH Cases and Benchmark

Source: https://github.com/luce-org/lucebox-hub/blob/main/pflash/README.md

Use these commands to generate NIAH test cases and then benchmark the C++ daemon against them. Ensure the paths to binaries and models are correct for your setup.

```bash
python tests/niah_gen.py --n 1 --ctx 131072 --out /tmp/niah_128k.jsonl
```

```bash
python tests/bench_niah_cpp.py \
  --bin    ../dflash/build/test_dflash \
  --target ../dflash/models/Qwen3.6-27B-Q4_K_M.gguf \
  --draft-spec ../dflash/models/draft/model.safetensors \
  --drafter-gguf ../dflash/models/Qwen3-0.6B-BF16.gguf \
  --cases  /tmp/niah_128k.jsonl --keep-ratio 0.05 --n-gen 256
```

--------------------------------

### Target Layer-Split with DFlash Decode Loop

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Configure the target layer-split harness to run a chain DFlash decode loop and report acceptance length. This example uses GPUs 0, 1 for targets, splitting layers 1 and 1, and loads the draft on GPU 0.

```bash
./test_dflash target.gguf draft.gguf prompt.bin 128 out.bin \
  --draft-gpu=0 --target-gpus=0,1 --target-layer-split=1,1 \
  --target-split-dflash
```

--------------------------------

### Target Layer-Split with Draft Loading

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Configure the target layer-split harness to load the DFlash draft on a specific GPU and capture target features into a draft-side mirror for capacity checks. This example uses GPU 0 for the draft and GPUs 0, 1 for targets, splitting layers 1 and 3.

```bash
./test_dflash target.gguf draft.safetensors prompt.bin 128 out.bin \
  --draft-gpu=0 --target-gpus=0,1 --target-layer-split=1,3 \
  --target-split-load-draft
```

--------------------------------

### Configure Asymmetric KV Quantization via CLI Flags (Short)

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Use short CLI flags to specify the quantization types for keys and values when running test_dflash or test_generate. This example uses q8_0 for keys and q4_0 for values.

```bash
./test_dflash … -ctk q8_0 -ctv q4_0
```

--------------------------------

### Run Laguna-XS.2 with OpenAI-compatible HTTP server

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Start the dflash server with the Laguna-XS.2 target and test its functionality using a curl request. The server automatically dispatches to the correct daemon based on the target architecture.

```bash
# OpenAI-compatible HTTP server (same scripts/server.py used for qwen35).
# server.py drops --draft and the DFlash/DDTree flags when arch=laguna; 
# test_dflash itself routes to run_laguna_daemon().
python3 scripts/server.py \
    --target models/laguna-xs2-Q4_K_M.gguf \
    --max-ctx 16384 --port 8000
```

```bash
curl -sN http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"luce-dflash","messages":[{"role":"user","content":"Hi"}],"max_tokens":32}'
```

--------------------------------

### OpenAI Client Request to DFlash Server

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Example of how to send a request to the DFlash server using curl, mimicking an OpenAI-compatible client. This demonstrates sending a long prompt for processing.

```bash
curl -sN http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "luce-dflash",
    "messages": [{"role": "user", "content": "<128K token document here>"}],
    "max_tokens": 512,
    "stream": true
  }'
```

--------------------------------

### OpenAI-compatible HTTP server with server.py

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Deploy `server.py` to expose an OpenAI-compatible API for chat completions, messages, and model information. Requires installation of several Python packages. Serves on port 8080 by default.

```bash
pip install fastapi uvicorn transformers jinja2
```

```bash
python3 scripts/server.py \
  --target models/Qwen3.6-27B-Q4_K_M.gguf \
  --draft  models/draft/model.safetensors \
  --budget 22 \
  --max-ctx 16384 \
  --port 8080
```

```bash
curl -s http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"luce-dflash","messages":[{"role":"user","content":"hi"}]}'
```

```bash
curl -sN http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"luce-dflash","messages":[{"role":"user","content":"hi"}],"stream":true}'
```

```bash
curl -s http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "luce-dflash",
    "messages": [{"role":"user","content":"What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather for a city",
        "parameters": {"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}
      }
    }]
  }'
```

```bash
curl -s http://localhost:8080/v1/messages \
  -H 'Content-Type: application/json' \
  -d '{"model":"luce-dflash","max_tokens":256,"messages":[{"role":"user","content":"Hello"}]}'
```

```bash
curl http://localhost:8080/health
# {"status": "ok"}
```

```bash
OPENAI_API_BASE=http://localhost:8080/v1
OPENAI_API_KEY=sk-any
```

--------------------------------

### Run Script with Asymmetric KV Quantization (Long Flags)

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Execute the run.py script with asymmetric KV quantization settings using long CLI flags. This example sets the key cache type to q8_0 and the value cache type to q4_0.

```bash
python3 scripts/run.py --cache-type-k q8_0 --cache-type-v q4_0 --prompt "hello"
```

--------------------------------

### Configure Asymmetric KV Quantization via Environment Variables

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Set independent quantization types for keys (K) and values (V) using environment variables. Supported types include f16, bf16, q4_0, q4_1, q5_0, q5_1, q8_0, tq3_0. This example sets K to q8_0 and V to q4_0.

```bash
DFLASH27B_KV_K=q8_0 DFLASH27B_KV_V=q4_0 ./test_dflash …
```

--------------------------------

### Download and Benchmark Qwen3.6-27B with dflash

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Steps to download the Qwen3.6-27B target and draft models, and then benchmark performance using the provided script. Ensure Hugging Face CLI is set up and any necessary tokens are configured.

```bash
# 1. target
huggingface-cli download unsloth/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf --local-dir models/

# 2. matched 3.6 draft (gated: accept terms + set HF_TOKEN first)
huggingface-cli download z-lab/Qwen3.6-27B-DFlash --local-dir models/draft/

# 3. bench
DFLASH_TARGET=models/Qwen3.6-27B-Q4_K_M.gguf python3 scripts/bench_he.py --n-gen 128
```

--------------------------------

### Target Layer-Split Harness Configuration

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Configure the target layer-split harness on test_dflash for multi-GPU setups. This example splits layers across GPUs 1, 2, and 3 with equal distribution.

```bash
./test_dflash target.gguf draft.safetensors prompt.bin 128 out.bin \
  --target-gpus=1,2,3,4 --target-layer-split=1,1,1,1
```

--------------------------------

### Configure OpenAI Server with Compression

Source: https://github.com/luce-org/lucebox-hub/blob/main/pflash/README.md

Run the OpenAI-compatible server with transparent compression for long prompts. Adjust `--prefill-threshold` and `--prefill-keep-ratio` based on your model and hardware constraints.

```bash
python dflash/scripts/server.py \
  --target dflash/models/Qwen3.6-27B-Q4_K_M.gguf \
  --draft  dflash/models/draft/model.safetensors \
  --max-ctx 8192 --budget 16 --fa-window 0 \
  --prefill-compression auto \
  --prefill-threshold 4096 \
  --prefill-keep-ratio 0.02 \
  --prefill-drafter dflash/models/Qwen3-0.6B-BF16.gguf
```

--------------------------------

### Run Simple A/B Quality Test

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/eval/README.md

Execute a series of conversational prompts against multiple server configurations to compare their outputs against a baseline. Ensure PFLASH environment variables are set correctly.

```bash
PFLASH_TARGET=/path/to/target.gguf \
PFLASH_DRAFT=/path/to/draft-dir-or-safetensors \
PFLASH_BIN=dflash/build/test_dflash \
PFLASH_DRAFTER=/path/to/Qwen3-0.6B-BF16.gguf \
python3 dflash/scripts/quality_ab_simple.py
```

--------------------------------

### Conventional Commit Message Examples

Source: https://github.com/luce-org/lucebox-hub/blob/main/CONTRIBUTING.md

Examples of commit messages following the conventional commits format. Use allowed types like feat, fix, docs, etc., with a scope.

```git
feat(megakernel): fused QKV+RoPE path cuts per-token launch by 1 kernel
```

```git
fix(dflash): clamp int8 DeltaNet state update before dequant
```

```git
docs(hub): add DVFS methodology link
```

--------------------------------

### Install Lucebox Megakernel

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Install the Lucebox megakernel package. Use --no-build-isolation for proper dependency handling. The build process automatically detects GPU architecture and selects the appropriate CUDA code path.

```bash
pip install -e . --no-build-isolation
```

--------------------------------

### Spawn DFlash Daemon and Load Weights

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Initializes the DFlash client, loading target weights into VRAM. This process blocks until weights are fully loaded.

```python
client = DflashClient(
    bin_path="dflash/build/test_dflash",
    target_path="dflash/models/Qwen3.6-27B-Q4_K_M.gguf",
    draft_path="dflash/models/draft/model.safetensors",
    max_ctx=16384,
    ddtree_budget=22,
    fa_window=0,     # full attention (required for PFlash)
    kv_tq3=True,     # TQ3_0 KV cache
)
```

--------------------------------

### Fetch Model Weights

Source: https://github.com/luce-org/lucebox-hub/blob/main/pflash/README.md

Downloads necessary model weights for the target, spec-decode draft, and drafter scorer. Ensure you have huggingface-cli installed.

```bash
huggingface-cli download unsloth/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf --local-dir models/
huggingface-cli download Qwen/Qwen3-0.6B model.safetensors tokenizer.json --local-dir models/drafter/
huggingface-cli download z-lab/Qwen3.6-27B-DFlash model.safetensors --local-dir models/draft/
```

--------------------------------

### Run Python Integration Tests

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Execute Python integration tests that start their own server subprocess. These require the daemon binary and model files.

```bash
cd dflash/scripts
python test_server_prefix_cache.py
python test_multi_turn_prefix_cache.py
python test_full_compress_cache.py
```

--------------------------------

### Quick Megakernel Benchmark (bench_pp_tg.py)

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Runs a single warm pass of prefill and decode with an end-to-end correctness assertion. Useful for quick sanity checks, though it may underreport prefill throughput.

```bash
python bench_pp_tg.py
# Runs pp+tg, checks that the first generated token on "The capital of France is"
# matches the reference decode.  Prints a PASS/FAIL line.
```

--------------------------------

### Build CUDA and HIP Binaries

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/docs/MIXED_BACKEND.md

Configure and build separate CUDA and HIP binaries for PFlash and DFlash. Ensure to replace `<your-gfx-arch>` with your specific HIP architecture.

```bash
cmake -S . -B build-cuda -DCMAKE_BUILD_TYPE=Release \
  -DDFLASH27B_GPU_BACKEND=cuda
cmake --build build-cuda --target pflash_daemon test_dflash -j

cmake -S . -B build-hip -DCMAKE_BUILD_TYPE=Release \
  -DDFLASH27B_GPU_BACKEND=hip \
  -DDFLASH27B_HIP_ARCHITECTURES=<your-gfx-arch>
cmake --build build-hip --target pflash_daemon test_dflash -j
```

--------------------------------

### Build dflash with CMake

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/docs/SPEC_PREFILL.md

Use CMake to build the dflash project, including test executables and FlashPrefill kernels. Ensure CUDA Toolkit 12.0+ is installed and submodules are updated.

```bash
git submodule update --init --recursive
mkdir build && cd build
cmake -DCMAKE_CUDA_ARCHITECTURES=86 -DDFLASH27B_ENABLE_BSA=ON ..
cmake --build . --target test_dflash test_flashprefill_kernels -- -j8
```

--------------------------------

### Download Laguna-XS.2 model and dependencies

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Download the Laguna-XS.2 GGUF model, a smaller Qwen3-0.6B drafter model, and necessary tokenizer files. These are required for running the Laguna-XS.2 target.

```bash
# 19 GB Q4_K_M target + 1.2 GB Qwen3-0.6B BF16 drafter + tokenizers
huggingface-cli download Lucebox/Laguna-XS.2-GGUF laguna-xs2-Q4_K_M.gguf --local-dir models/
huggingface-cli download unsloth/Qwen3-0.6B-GGUF Qwen3-0.6B-BF16.gguf --local-dir models/
huggingface-cli download poolside/Laguna-XS.2 --local-dir models/Laguna-XS-2 \
    --include 'tokenizer*' '*.json'
```

--------------------------------

### Download and Build Laguna-XS.2 (MoE Target)

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Downloads Laguna-XS.2 model weights and tokenizer files, then builds the necessary targets for running the model. Assumes a CMake build environment.

```bash
huggingface-cli download Lucebox/Laguna-XS.2-GGUF laguna-xs2-Q4_K_M.gguf \
    --local-dir models/
```

```bash
huggingface-cli download poolside/Laguna-XS.2 --local-dir models/Laguna-XS-2 \
    --include 'tokenizer*' '*.json'
```

```bash
cmake --build build --target test_dflash pflash_daemon -j
```

--------------------------------

### Verify GPU Properties and CUDA Version

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Verifies the GPU's name, compute capability, number of SMs, and total memory, along with the installed CUDA toolkit version. This is useful for ensuring compatibility.

```python
python -c "import torch; p=torch.cuda.get_device_properties(0); print(p.name, 'sm_%d%d'%(p.major,p.minor), p.multi_processor_count,'SMs', round(p.total_memory/1e9,1),'GB')"
```

```bash
nvcc --version
```

--------------------------------

### Run Megakernel Benchmark (final_bench.py)

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Executes the canonical benchmark for Megakernel, including warmup and timed runs for prefill and decode. Weights are downloaded from HuggingFace on the first run. Optional power limit setting can optimize for tok/J.

```bash
# Optional: set power limit for best tok/J sweet spot on RTX 3090
sudo nvidia-smi -pl 220

python final_bench.py
# Expected output (RTX 3090 @220W):
#   Prefill pp520:  21,347 tok/s
#   Decode  tg128:     413 tok/s
#   Energy:           1.87 tok/J

# RTX 2080 Ti (sm_75, FP16 path — auto-detected):
#   Prefill pp520:  13,919 tok/s
#   Decode  tg128:     250 tok/s

# Blackwell / DGX Spark GB10 — NVFP4 path:
python final_bench.py --backend nvfp4
# GB10 result: ~194 tok/s tg128 (NVFP4)
```

--------------------------------

### HIP PFlash Drafter with CUDA Target Layer Split

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/docs/MIXED_BACKEND.md

Example of running a HIP PFlash drafter followed by a CUDA target layer split. This command utilizes the `phase_split_dual_gpu.py` script to manage the hybrid backend execution.

```bash
python scripts/phase_split_dual_gpu.py bench-niah \
  --build-dir build-hip \
  --pflash-backend hip \
  --pflash-visible-devices 0 \
  --run-target \
  --target-bin build-cuda/test_dflash \
  --target-backend cuda \
  --target-visible-devices 0,1 \
  --target-gpus 0,1 \
  --target-layer-split 1,1 \
  --target-gen-tokens 8 \
  --contexts 4096 \
  --local-files-only \
  --report-dir reports/pflash_hybrid_hip_drafter_cuda_target
```

--------------------------------

### Download model weights

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Downloads necessary model weights for the Qwen3.6-27B target, Qwen3-0.6B drafter, and the DFlash speculative decode draft using the huggingface-cli. Ensure models are placed in the 'models/' directory.

```bash
huggingface-cli download unsloth/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf --local-dir models/
```

```bash
huggingface-cli download unsloth/Qwen3-0.6B-GGUF Qwen3-0.6B-BF16.gguf --local-dir models/
```

```bash
huggingface-cli download z-lab/Qwen3.6-27B-DFlash model.safetensors --local-dir models/draft/
```

--------------------------------

### Fetch Model Weights

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Downloads necessary model weights using the huggingface-cli. Ensure you have sufficient disk space (~16 GB for Q4_K_M and ~3.46 GB for the bf16 draft).

```bash
huggingface-cli download unsloth/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf --local-dir models/
```

```bash
huggingface-cli download z-lab/Qwen3.6-27B-DFlash model.safetensors --local-dir models/draft/
```

--------------------------------

### Start DFlash Daemon for Long Contexts

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Launches the dflash daemon with specific environment variables for PFlash, including BSA kernel usage and alpha value. It loads a large context model and configures it for fast rollback and DDTree operations.

```bash
DFLASH_FP_USE_BSA=1 DFLASH_FP_ALPHA=0.85 \
./build/test_dflash models/Qwen3.6-27B-Q4_K_M.gguf \
  models/draft/model.safetensors \
  --daemon --fast-rollback --ddtree --ddtree-budget=22 \
  --max-ctx=131072 --stream-fd=3
```

--------------------------------

### Configure Asymmetric KV Quantization via CLI Flags (Long)

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/README.md

Use long CLI flags to specify the quantization types for keys and values. This example sets the key cache type to q8_0 and the value cache type to q4_0.

```bash
./test_dflash … --cache-type-k=q8_0 --cache-type-v=q4_0
```

--------------------------------

### Smoke Qwen3 0.6B Forward Test

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/CMakeLists.txt

Builds the 'smoke_qwen3_0p6b_forward' executable if the source file exists. It includes the project's source directory and links against dflash27b, ggml, and the specified GGML backend target.

```cmake
if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/test/smoke_qwen3_0p6b_forward.cpp")
        add_executable(smoke_qwen3_0p6b_forward test/smoke_qwen3_0p6b_forward.cpp)
        target_include_directories(smoke_qwen3_0p6b_forward PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src)
        target_link_libraries(smoke_qwen3_0p6b_forward PRIVATE dflash27b ggml ${DFLASH27B_GGML_BACKEND_TARGET})
    endif()
```

--------------------------------

### Download DFlash Model Weights

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Downloads the necessary model weights for DFlash, including the target GGUF model and the DFlash draft model weights. Supports both Qwen3.6-27B and Qwen3.5-27B.

```bash
# Qwen3.6-27B Q4_K_M target (~16 GB) + z-lab DFlash BF16 draft (~3.46 GB)
huggingface-cli download unsloth/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf \
    --local-dir models/
huggingface-cli download z-lab/Qwen3.6-27B-DFlash model.safetensors \
    --local-dir models/draft/

# Or for Qwen3.5-27B (same architecture, swappable):
# huggingface-cli download unsloth/Qwen3.5-27B-GGUF Qwen3.5-27B-Q4_K_M.gguf --local-dir models/
# huggingface-cli download z-lab/Qwen3.5-27B-DFlash model.safetensors --local-dir models/draft/
```

--------------------------------

### Run Head-to-Head NIAH Benchmark

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Executes a head-to-head benchmark comparing the dflash daemon against llama-bench for NIAH test cases. Requires specifying paths to the dflash binary, target and draft models, and the generated test cases.

```bash
python tests/bench_niah_cpp.py \
  --bin    ../dflash/build/test_dflash \
  --target ../dflash/models/Qwen3.6-27B-Q4_K_M.gguf \
  --draft-spec ../dflash/models/draft/model.safetensors \
  --drafter-gguf ../dflash/models/Qwen3-0.6B-BF16.gguf \
  --cases  /tmp/niah_128k.jsonl \
  --keep-ratio 0.05 \
  --n-gen 256
```

--------------------------------

### Build dflash with BSA kernel

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Builds the dflash component along with the Block-Sparse-Attention (BSA) kernel, which requires sm_80+ architecture. This process can take around 10 minutes for a cold compile. It also runs tests for dflash and the flashprefill kernels.

```bash
git clone --recurse-submodules https://github.com/Luce-Org/lucebox-hub && cd lucebox-hub/dflash
```

```bash
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release \
                    -DCMAKE_CUDA_ARCHITECTURES=86 \
                    -DDFLASH27B_ENABLE_BSA=ON
```

```bash
cmake --build build --target test_dflash test_flashprefill_kernels -j
```

--------------------------------

### Build dflash and test

Source: https://github.com/luce-org/lucebox-hub/blob/main/README.md

Builds the dflash component and runs tests. Requires CUDA 13.0+ and git. CMake automatically detects the supported GPU architecture.

```bash
nvcc --version
```

```bash
git clone --recurse-submodules https://github.com/Luce-Org/lucebox-hub && cd lucebox-hub/dflash
```

```bash
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release   # CMake auto-adds the Thor arch your nvcc supports
```

```bash
cmake --build build --target test_dflash -j
```

--------------------------------

### Reproduce Paper Benchmarks with scripts/bench_llm.py

Source: https://context7.com/luce-org/lucebox-hub/llms.txt

Runs HumanEval, GSM8K, and Math500 benchmarks using autoregressive and DFlash+DDTree methods. Allows selection of specific benchmarks and sweeping the DDTree budget.

```bash
python3 scripts/bench_llm.py
```

```bash
python3 scripts/bench_llm.py --bench HumanEval GSM8K
```

```bash
python3 scripts/bench_llm.py --budget 40
```

--------------------------------

### Download Target Model

Source: https://github.com/luce-org/lucebox-hub/blob/main/dflash/DEVELOPER.md

Downloads the specified target model (e.g., Qwen3.6-27B Q4_K_M) using the huggingface-cli. The model will be saved to the dflash/models/ directory.

```bash
# Target model (Q4_K_M quantized Qwen3.6-27B)
huggingface-cli download <repo-id> --local-dir dflash/models/
```