### Install Minikube

Source: https://github.com/defilantech/llmkube/blob/main/examples/metal-quickstart/README.md

Install Minikube using Homebrew and start it with the Docker driver. Ensure your system meets the hardware prerequisites.

```bash
brew install minikube
minikube start --driver=docker
```

--------------------------------

### Build and Install Metal Agent using Makefile

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Recommended method to build the Metal agent binary, install it to /usr/local/bin, set up the launchd service, and start it automatically.

```bash
make install-metal-agent
```

--------------------------------

### Install and Start Metal Agent

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/guides/macos-metal.md

Builds and installs the Metal Agent binary, configures it as a launchd service, and starts the agent on macOS.

```bash
git clone https://github.com/defilantech/LLMKube.git
cd LLMKube
make install-metal-agent
```

--------------------------------

### Configure Single-GPU Terraform Setup

Source: https://github.com/defilantech/llmkube/blob/main/terraform/gke/README.md

Copies the example Terraform variables file and sets essential parameters like project ID and region.

```bash
cp terraform.tfvars.example terraform.tfvars
```

```hcl
project_id = "your-gcp-project-id"  # REQUIRED: Your GCP project
region     = "us-central1"           # Change if needed
use_spot   = true                    # true = cheap but interruptible
```

--------------------------------

### Install kubectl on Windows

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Install the Kubernetes CLI using Chocolatey.

```powershell
choco install kubernetes-cli
```

--------------------------------

### Install LLMKube CLI via Homebrew

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.0.md

Use Homebrew to add the defilantech tap and install the LLMKube CLI. Verify the installation with the version command.

```bash
brew tap defilantech/tap

brew install llmkube

llmkube version
```

--------------------------------

### Install Prerequisites

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/TEST.md

Install necessary tools including Helm, kubeconform, kind, and yq for local testing.

```bash
# Install Helm
brew install helm

# Install kubeconform (for manifest validation)
brew install kubeconform

# Install kind (for local Kubernetes testing)
brew install kind

# Install yq (for YAML processing)
brew install yq
```

--------------------------------

### Install LLMKube with Helm

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Basic installation command for the LLMKube Helm chart.

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace
```

--------------------------------

### Check Example Values Files

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/TEST.md

Confirms the existence of example values files for different deployment scenarios (basic, production, gpu-cluster).

```bash
for example in basic production gpu-cluster; do
  test -f charts/llmkube/examples/values-${example}.yaml && echo "✅ Example: values-${example}.yaml"
done
```

--------------------------------

### Install LLMKube from Local Repository

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Clone the source repository and install the chart directly from the local filesystem.

```bash
# Clone the repository
git clone https://github.com/defilantech/LLMKube.git
cd LLMKube

# Install the chart
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace
```

--------------------------------

### Install Minikube on Windows

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Install Minikube using Chocolatey or the official installer.

```powershell
# Using Chocolatey
choco install minikube

# Or download from https://minikube.sigs.k8s.io/docs/start/
```

--------------------------------

### Install Minikube on Linux

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Download and install the Minikube binary for Linux systems.

```bash
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
```

--------------------------------

### Multi-GPU Quick Start Script

Source: https://github.com/defilantech/llmkube/blob/main/terraform/eks/README.md

Executes an automated script for a multi-GPU EKS cluster quick start. This script handles prerequisites, AWS credential configuration, GPU type selection, Terraform variable setup, cluster deployment, and GPU allocation testing.

```bash
# Use the automated script
./multi-gpu-quick-start.sh
```

--------------------------------

### Install LLMKube CLI on Linux (amd64)

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.2.md

Downloads and installs the binary for amd64 Linux systems.

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.3.2/LLMKube_0.3.2_linux_amd64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

--------------------------------

### Install kubectl on Linux

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Download and install the latest stable version of kubectl.

```bash
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install kubectl /usr/local/bin/kubectl
```

--------------------------------

### Install LLMKube from Helm Repo (Untar)

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/guides/openshift-install.md

Install LLMKube by first pulling and untarring the chart from the Helm repository, then installing it locally. This approach also requires specifying the `values-openshift.yaml` for OpenShift compatibility.

```bash
helm repo add llmkube https://defilantech.github.io/LLMKube
helm pull llmkube/llmkube --untar
helm install llmkube ./llmkube \
  -f ./llmkube/values-openshift.yaml \
  -n llmkube-system
```

--------------------------------

### Install oMLX via Homebrew

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Installs the oMLX runtime and its dependencies using Homebrew. Ensure Homebrew is installed before running these commands.

```bash
brew tap jundot/omlx https://github.com/jundot/omlx
brew install omlx
```

--------------------------------

### Manual Installation of Metal Agent

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Manual steps to build the agent, copy it to the system path, install the launchd plist, and load the service.

```bash
make build-metal-agent
sudo cp bin/llmkube-metal-agent /usr/local/bin/
mkdir -p ~/Library/LaunchAgents
cp deployment/macos/com.llmkube.metal-agent.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/com.llmkube.metal-agent.plist
```

--------------------------------

### Install LLMKube CLI on Linux (arm64)

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.2.md

Downloads and installs the binary for arm64 Linux systems.

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.3.2/LLMKube_0.3.2_linux_arm64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

--------------------------------

### Install kubectl on macOS

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Install the Kubernetes command-line tool via Homebrew.

```bash
brew install kubectl
```

--------------------------------

### Install LLMKube via Helm

Source: https://github.com/defilantech/llmkube/blob/main/charts/HELM_CHART_SUMMARY.md

Commands for installing the LLMKube chart with various configurations.

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace
```

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace \
  -f charts/llmkube/examples/values-production.yaml
```

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace \
  -f charts/llmkube/examples/values-gpu-cluster.yaml
```

--------------------------------

### Install llama.cpp with Metal Support

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Installs the llama.cpp library with Metal support using Homebrew. Ensure you have Homebrew installed.

```bash
brew install llama.cpp
```

--------------------------------

### Install LLMKube with Monitoring

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Installation command including custom values for Prometheus ServiceMonitor and PrometheusRule.

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace \
  --values - <<EOF
controllerManager:
  replicaCount: 1
  resources:
    limits:
      cpu: 1
      memory: 4Gi
    requests:
      cpu: 100m
      memory: 1Gi

prometheus:
  serviceMonitor:
    enabled: true
    interval: 30s
  prometheusRule:
    enabled: true
    namespace: monitoring
    rules:
      gpu:
        enabled: true
        highUtilizationThreshold: 85
      inference:
        enabled: true
EOF
```

--------------------------------

### Install LLMKube with Custom Image Registry

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Installation command for air-gapped or private registry environments.

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace \
  --set controllerManager.image.registry=my-registry.company.com \
  --set controllerManager.image.repository=llmkube/llmkube-controller \
  --set controllerManager.initContainer.registry=my-registry.company.com \
  --set controllerManager.initContainer.repository=curlimages/curl
```

--------------------------------

### Verify llama-server Installation

Source: https://github.com/defilantech/llmkube/blob/main/examples/metal-quickstart/README.md

Confirm that 'llama-server' is installed and accessible in your PATH.

```bash
# 2. Check llama-server is installed
which llama-server
# Should show: /usr/local/bin/llama-server
```

--------------------------------

### Verify Installations

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Check the versions of Minikube and kubectl to ensure successful installation.

```bash
minikube version
# Expected: minikube version: v1.32.0 (or newer)

kubectl version --client
# Expected: Client Version: v1.28.0 (or newer)
```

--------------------------------

### Install LLMKube CLI on macOS (Intel)

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.2.md

Downloads and installs the binary for Intel-based macOS systems.

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.3.2/LLMKube_0.3.2_darwin_amd64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

--------------------------------

### Install LLMKube CLI

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Install the LLMKube CLI tool on macOS or Linux.

```bash
# macOS (Homebrew)
brew install defilantech/tap/llmkube

# Linux/macOS (install script)
curl -sSL https://raw.githubusercontent.com/defilantech/LLMKube/main/install.sh | bash
```

--------------------------------

### Install Metal Agent from Source

Source: https://github.com/defilantech/llmkube/blob/main/examples/metal-quickstart/README.md

Build and install the Metal Agent from the LLMKube repository source. Alternatively, download a pre-built binary from the releases page.

```bash
# From llmkube repository
make install-metal-agent
```

--------------------------------

### Go Code Style Example

Source: https://github.com/defilantech/llmkube/blob/main/CONTRIBUTING.md

Example of a Go function with comments explaining its purpose and implementation details.

```go
// ReconcileModel downloads and validates the model specified in the Model CRD.
// It updates the Model status with download progress and any errors encountered.
func (r *ModelReconciler) ReconcileModel(ctx context.Context, model *inferencev1alpha1.Model) error {
    // Implementation
}
```

--------------------------------

### Troubleshooting Agent Startup

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Check agent logs and verify the installation of llama-server and Metal support. Ensure the agent binary is correctly installed and executable.

```bash
# Check logs
cat /tmp/llmkube-metal-agent.log

# Verify llama-server is installed
which llama-server

# Verify Metal support
llmkube-metal-agent --version
```

--------------------------------

### Install LLMKube with Custom Values

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Override default chart parameters during installation using the --set flag.

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace \
  --set controllerManager.image.tag=0.2.1 \
  --set prometheus.serviceMonitor.enabled=true
```

--------------------------------

### Copy Example Terraform Configuration

Source: https://github.com/defilantech/llmkube/blob/main/terraform/eks/README.md

Copies the example Terraform variables file to be edited for manual configuration.

```bash
cp terraform.tfvars.example terraform.tfvars
```

--------------------------------

### Install LLMKube via Helm Repository

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Add the official LLMKube repository and install the chart into a specific namespace.

```bash
helm repo add llmkube https://defilantech.github.io/LLMKube
helm repo update

# Install the chart
helm install llmkube llmkube/llmkube \
  --namespace llmkube-system \
  --create-namespace
```

--------------------------------

### Install LLMKube CLI on macOS (Apple Silicon)

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.2.md

Downloads and installs the binary for Apple Silicon macOS systems.

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.3.2/LLMKube_0.3.2_darwin_arm64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

--------------------------------

### Install LLMKube Operator Locally

Source: https://github.com/defilantech/llmkube/blob/main/examples/quickstart/README.md

Clone the repository, install CRDs, and run the controller locally for development. Requires Go 1.25+.

```bash
git clone https://github.com/defilantech/LLMKube.git
cd LLMKube

make install

make run
```

--------------------------------

### Manual Installation of LLMKube CLI on Linux (ARM64)

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.0.md

Manually install the LLMKube CLI on Linux for ARM64 architecture by downloading the tarball and moving the binary to the PATH.

```bash
# ARM64
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.3.0/LLMKube_0.3.0_linux_arm64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

--------------------------------

### LLMKube Deploy Help Integration Example

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.2.2.md

Shows how the `llmkube deploy --help` command now includes examples demonstrating catalog usage and how to list available catalog models, improving developer experience.

```bash
# Deploy from catalog (simplest - recommended!)
llmkube deploy llama-3.1-8b --gpu
llmkube deploy qwen-2.5-coder-7b --gpu

# List available catalog models
llmkube catalog list
```

--------------------------------

### Install LLMKube CRDs and Run Controller from Source

Source: https://github.com/defilantech/llmkube/blob/main/README.md

Clone the LLMKube repository, install the Custom Resource Definitions (CRDs), and run the controller locally for development.

```bash
git clone https://github.com/defilantech/LLMKube.git && cd LLMKube
make install  # Install CRDs
make run      # Run controller locally
```

--------------------------------

### Manual Installation of LLMKube CLI on Linux (AMD64)

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.0.md

Manually install the LLMKube CLI on Linux for AMD64 architecture by downloading the tarball and moving the binary to the PATH.

```bash
# AMD64
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.3.0/LLMKube_0.3.0_linux_amd64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

--------------------------------

### Install LLMKube Operator from GitHub

Source: https://github.com/defilantech/llmkube/blob/main/examples/metal-quickstart/README.md

Install the LLMKube Operator by applying the default Kubernetes configuration from GitHub. Wait for the operator to become ready.

```bash
kubectl apply -k https://github.com/defilantech/LLMKube/config/default

# Wait for the operator to be ready:
kubectl wait --for=condition=ready pod -l control-plane=controller-manager -n llmkube-system --timeout=60s
```

--------------------------------

### Install and Run Metal Agent on macOS

Source: https://github.com/defilantech/llmkube/blob/main/README.md

Install the Metal Agent using Homebrew and start it, specifying your Mac's IP address. This allows your Mac to participate in the Kubernetes cluster for Metal GPU inference.

```bash
# On your Mac
brew install llama.cpp
llmkube-metal-agent --host-ip <your-mac-ip>
```

--------------------------------

### Watch Foreman Operator Logs

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/foreman/install-verifier-node.md

Monitor the Foreman operator logs to ensure all reconcilers are starting, indicating a healthy installation.

```sh
kubectl -n foreman-system \
  logs -l app.kubernetes.io/component=operator -f
```

--------------------------------

### Deploy Model Before v0.2.1 (Manual Configuration)

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.2.2.md

This example shows the manual deployment process before LLMKube v0.2.2, requiring explicit specification of the model source URL, quantization, CPU, memory, and GPU layers.

```bash
llmkube deploy llama-3.1-8b --gpu \
  --source https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf \
  --quantization Q5_K_M \
  --cpu 4 \
  --memory 8Gi \
  --gpu-layers 33
```

--------------------------------

### Install LLMKube from Source

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.1.0.md

Build and deploy the controller directly from the repository source code.

```bash
git clone https://github.com/Defilan/LLMKube.git
cd LLMKube
git checkout v0.1.0

make install  # Install CRDs
make deploy IMG=ghcr.io/defilan/llmkube-controller:v0.1.0
```

--------------------------------

### LLMKube Error Message Example

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.2.2.md

Illustrates an improved error message when attempting to deploy a custom model not found in the catalog without providing a source URL. It guides the user to check the catalog.

```bash
$ llmkube deploy my-custom-model
Error: model 'my-custom-model' not found in catalog and no --source provided.
Use 'llmkube catalog list' to see available models
```

--------------------------------

### Manual CLI version check examples

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.2.md

Use the --check flag to manually query for updates. These examples show output for both outdated and up-to-date states.

```bash
$ llmkube version --check
llmkube version 0.3.1
  git commit: unknown
  build date: unknown

Checking for updates...
  ⚠️  New version available: v0.3.2
     Update with: brew upgrade llmkube
     Or download from: https://github.com/defilantech/LLMKube/releases/latest
```

```bash
$ llmkube version --check
llmkube version 0.3.2
  git commit: fb3adf5
  build date: 2025-11-24

Checking for updates...
  ✅ You're running the latest version!
```

--------------------------------

### Setup Simple HTTP Model Server

Source: https://github.com/defilantech/llmkube/blob/main/docs/air-gapped-quickstart.md

Start a basic HTTP server using Python's http.server module to host model files within an air-gapped network. This is used for the 'Private HTTP Server' deployment option.

```bash
# On your internal server
cd /path/to/models
python3 -m http.server 8080
```

--------------------------------

### LLMKube Metal Agent Configuration Example

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Example XML configuration for the launchd plist, showing arguments like namespace, model store path, llama-server path, and metrics port.

```xml
<key>ProgramArguments</key>
<array>
    <string>/usr/local/bin/llmkube-metal-agent</string>
    <string>--namespace</string>
    <string>default</string>              <!-- Kubernetes namespace to watch -->
    <string>--model-store</string>
    <string>/tmp/llmkube-models</string>  <!-- Where to store downloaded models -->
    <string>--llama-server</string>
    <string>/usr/local/bin/llama-server</string>  <!-- Path to llama-server binary -->
    <string>--port</string>
    <string>9090</string>                 <!-- Agent metrics port -->
</array>
```

--------------------------------

### Apply Demo Manifest and Start Pipeline

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/foreman/runbook-m4.md

Apply the M4 two-step demo manifest to initiate the pipeline. Ensure to update issue, task names, and branch slugs in the manifest before applying.

```bash
# Edit examples/foreman/m4-two-step-demo.yaml: bump the `issue:`
# field on both tasks, the task `name:` (code-N + gate-N), and the
# `payload.branch:` to a fresh `foreman/issue-N` slug.

kubectl apply -f examples/foreman/m4-two-step-demo.yaml
```

--------------------------------

### Typical Workflow

Source: https://github.com/defilantech/llmkube/blob/main/terraform/gke/QUICK-REFERENCE.md

A summary of the end-to-end process from cluster creation to cleanup.

```bash
# 1. Create cluster
cd terraform/gke && ./quick-start.sh

# 2. Test GPU
kubectl apply -f <gpu-test-pod>

# 3. Deploy llmkube
cd ../.. && kubectl apply -f config/crd/bases/

# 4. Test model
kubectl apply -f config/samples/inference_v1alpha1_model.yaml
kubectl logs -f <controller-pod>

# 5. CLEANUP!
cd terraform/gke && ./teardown.sh
```

--------------------------------

### Install LLMKube CLI and llama.cpp via Homebrew

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.0.md

Install the LLMKube CLI and llama.cpp for Metal support on macOS using Homebrew. Verify the installation afterwards.

```bash
# Add the tap
brew tap defilantech/tap

# Install CLI
brew install llmkube

# Install llama.cpp for Metal support
brew install llama.cpp

# Verify installation
llmkube version
```

--------------------------------

### Start Minikube Cluster

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Initialize a Minikube cluster with resources configured for LLM workloads.

```bash
# Recommended configuration for LLMKube
minikube start \
  --cpus=4 \
  --memory=6144 \
  --disk-size=20g \
  --driver=docker

# Wait for cluster to be ready
minikube status
```

--------------------------------

### Install Foreman Helm Chart

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/foreman/README.md

Commands to add the Foreman Helm repository and install or upgrade the Foreman Helm chart, ensuring LLMKube core is installed first.

```bash
# Make sure LLMKube core is installed first (0.8.0+)
helm repo add llmkube https://defilantech.github.io/LLMKube
helm repo update
helm upgrade --install llmkube llmkube/llmkube \
  --namespace llmkube-system --create-namespace
```

--------------------------------

### Install Terraform

Source: https://github.com/defilantech/llmkube/blob/main/terraform/gke/README.md

Installs the Terraform CLI on macOS using Homebrew.

```bash
brew install terraform
```

--------------------------------

### Verify CLI Installation

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Check the installed version of the LLMKube CLI.

```bash
llmkube version
```

--------------------------------

### LLMKube Development Workflow Example

Source: https://github.com/defilantech/llmkube/blob/main/CONTRIBUTING.md

A comprehensive bash script demonstrating the typical steps involved in adding a new feature to the LLMKube project, from branching and coding to testing, linting, documentation, and creating a pull request.

```bash
# 1. Create feature branch
git checkout -b feat/multi-gpu-single-node

# 2. Make changes
vim api/v1alpha1/model_types.go
vim internal/controller/inferenceservice_controller.go

# 3. Generate manifests
make manifests

# 4. Write tests
vim internal/controller/inferenceservice_controller_test.go

# 5. Run tests
make test

# 6. Test manually (CPU)
make install
make run
kubectl apply -f config/samples/inference_v1alpha1_model.yaml

# 7. Test with GPU (if GPU feature)
kubectl apply -f config/samples/gpu-model-example.yaml
./test/e2e/gpu_test.sh

# 8. Lint
make lint

# 9. Update documentation
vim README.md
vim docs/gpu-setup-guide.md  # If GPU-related

# 10. Commit
git add .
git commit -m "feat(gpu): Add multi-GPU single-node support

Enables layer offloading across multiple GPUs on single node.
Adds spec.hardware.gpu.count field to Model CRD.
Verified with 13B model on 2x L4 GPUs.

Closes #123"

# 11. Push and create PR
git push origin feat/multi-gpu-single-node
# Open PR on GitHub
```

--------------------------------

### Apply M4 Two-Step Demo

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/foreman/runbook-m4.md

Applies the M4 two-step demo configuration. This is the initial step to start the pipeline.

```bash
kubectl apply -f examples/foreman/m4-two-step-demo.yaml
```

--------------------------------

### Install Foreman Chart from Public Repository

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/foreman/install-verifier-node.md

Installs the Foreman chart with the agent in native mode and assigns 'worker' and 'verifier' roles. This is the recommended method after LLMKube core is installed.

```bash
helm upgrade --install foreman \
  defilantech/foreman \
  --namespace foreman-system --create-namespace \
  --set agent.mode=native \
  --set 'agent.roles={worker,verifier}'
```

--------------------------------

### Troubleshoot Model Download

Source: https://github.com/defilantech/llmkube/blob/main/examples/gpu-quickstart/README.md

Inspect the model-downloader init container logs if the deployment is stuck.

```bash
POD_NAME=$(kubectl get pods -l app=llama-3b-gpu-service -o jsonpath='{.items[0].metadata.name}')
kubectl logs $POD_NAME -c model-downloader
```

--------------------------------

### Verify LLMKube CLI Installation

Source: https://github.com/defilantech/llmkube/blob/main/examples/quickstart/README.md

Check the installed version of the LLMKube CLI.

```bash
llmkube version
```

--------------------------------

### Clone and Build LLMKube

Source: https://github.com/defilantech/llmkube/blob/main/CONTRIBUTING.md

Clone the repository, install Go dependencies, generate manifests, and build the operator and CLI. Run tests to ensure code integrity.

```bash
# Fork the repository on GitHub first, then:
git clone git@github.com:YOUR_USERNAME/llmkube.git
cd llmkube

# Install dependencies
go mod download

# Generate CRD manifests
make manifests

# Build the operator binary
make build

# Build the CLI
make build-cli

# Run tests
make test
```

--------------------------------

### Install and Run Act

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/TEST.md

Instructions for installing the 'act' tool and running GitHub Actions jobs locally, including specific jobs, all jobs, or jobs triggered by a pull request event.

```bash
# Install act
brew install act
```

```bash
# Run specific job
act -j lint-and-validate
```

```bash
# Run all jobs
act
```

```bash
# Run on pull request event
act pull_request
```

--------------------------------

### Run Full Local Test Suite

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/TEST.md

Execute the complete sequence of linting, template rendering, validation, and packaging.

```bash
# 1. Lint
echo "Running helm lint..."
helm lint charts/llmkube

# 2. Template rendering (default)
echo "Testing default template rendering..."
helm template llmkube charts/llmkube --namespace llmkube-system > /tmp/llmkube-default.yaml

# 3. Template rendering (all examples)
for values in basic production gpu-cluster; do
  echo "Testing values-${values}.yaml..."
  helm template llmkube charts/llmkube \
    --namespace llmkube-system \
    -f charts/llmkube/examples/values-${values}.yaml \
    > /tmp/llmkube-${values}.yaml
done

# 4. Validate manifests with kubeconform
echo "Validating Kubernetes manifests..."
kubeconform -summary -ignore-missing-schemas -skip CustomResourceDefinition /tmp/llmkube-default.yaml

# 5. Test Prometheus resources
echo "Testing ServiceMonitor..."
helm template llmkube charts/llmkube \
  --namespace llmkube-system \
  --set prometheus.serviceMonitor.enabled=true | \
  grep -q "kind: ServiceMonitor" && echo "✅ ServiceMonitor OK"

echo "Testing PrometheusRule..."
helm template llmkube charts/llmkube \
  --namespace llmkube-system \
  --set prometheus.prometheusRule.enabled=true | \
  grep -q "kind: PrometheusRule" && echo "✅ PrometheusRule OK"

# 6. Test CRD installation
echo "Testing CRD installation..."
helm template llmkube charts/llmkube \
  --namespace llmkube-system \
  --set crds.install=true | \
  grep -q "kind: CustomResourceDefinition" && echo "✅ CRDs enabled OK"

helm template llmkube charts/llmkube \
  --namespace llmkube-system \
  --set crds.install=false | \
  grep -q "kind: CustomResourceDefinition" && echo "❌ CRDs should be disabled" || echo "✅ CRDs disabled OK"

# 7. Package
echo "Packaging chart..."
helm package charts/llmkube -d /tmp

echo ""
echo "✅ All local tests passed!"
```

--------------------------------

### Deployment Verification and Testing

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.3.3.md

Example output showing the sanitized service name and commands for testing the inference endpoint.

```bash
✅ Deployment ready!
═══════════════════════════════════════════════
Model:       llama-3.1-8b
Endpoint:    http://llama-3-1-8b.default.svc.cluster.local:8080/v1/chat/completions
Replicas:    1/1
═══════════════════════════════════════════════

🧪 To test the inference endpoint:

  # Port forward the service
  kubectl port-forward -n default svc/llama-3-1-8b 8080:8080

  # Send a test request
  curl http://localhost:8080/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"messages":[{"role":"user","content":"What is 2+2?"}]}'
```

--------------------------------

### Install LLMKube without CRDs

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Installation command for environments where CRDs are managed externally.

```bash
helm install llmkube charts/llmkube \
  --namespace llmkube-system \
  --create-namespace \
  --set crds.install=false
```

--------------------------------

### Deploy 13B Model with 2 GPUs

Source: https://github.com/defilantech/llmkube/blob/main/test/e2e/multi-gpu-test-plan.md

Apply the sample configuration to deploy a 13B model requiring 2 GPUs. Ensure to wait for the model and service to become ready.

```bash
# 1. Deploy 13B model with 2 GPUs
kubectl apply -f config/samples/multi-gpu-llama-13b-model.yaml

# 2. Wait for model to download
kubectl wait --for=jsonpath='{.status.phase}'=Ready model/llama-13b-multi-gpu --timeout=600s

# 3. Wait for service to be ready
kubectl wait --for=jsonpath='{.status.phase}'=Ready \
  inferenceservice/llama-13b-multi-gpu-service --timeout=900s

# 4. Get pod name
export POD_NAME=$(kubectl get pod -l app=llama-13b-multi-gpu-service \
  -o jsonpath='{.items[0].metadata.name}')
```

--------------------------------

### Conventional Commits Type Examples

Source: https://github.com/defilantech/llmkube/blob/main/CONTRIBUTING.md

Examples of commit types used in Conventional Commits.

```markdown
feat: New feature
fix: Bug fix
docs: Documentation only
refactor: Code refactoring (no functional change)
test: Adding tests
chore: Build process, tooling changes
```

--------------------------------

### Inspect Pod Configuration and GPU Resources

Source: https://github.com/defilantech/llmkube/blob/main/test/e2e/multi-gpu-test-plan.md

Use these commands to verify container images, arguments, and resource allocations for troubleshooting deployment issues.

```bash
kubectl get pod $POD_NAME -o jsonpath='{.spec.containers[0].image}'
```

```bash
kubectl get pod $POD_NAME -o jsonpath='{.spec.containers[*].args}'
```

```bash
kubectl get pod $POD_NAME -o yaml | grep -A 5 "resources:"
```

--------------------------------

### Install LLMKube Operator with Helm

Source: https://github.com/defilantech/llmkube/blob/main/README.md

Installs the LLMKube operator on a Kubernetes cluster using Helm.

```bash
# Install the operator on any K8s cluster
helm repo add llmkube https://defilantech.github.io/LLMKube
helm install llmkube llmkube/llmkube --namespace llmkube-system --create-namespace
```

--------------------------------

### Install and Authenticate gcloud CLI

Source: https://github.com/defilantech/llmkube/blob/main/terraform/gke/README.md

Installs the gcloud CLI on macOS and authenticates with Google Cloud.

```bash
# macOS
brew install --cask google-cloud-sdk

# Authenticate
gcloud auth login
gcloud auth application-default login
```

--------------------------------

### Install AWS CLI

Source: https://github.com/defilantech/llmkube/blob/main/terraform/eks/README.md

Installs the AWS CLI on macOS using Homebrew and configures credentials.

```bash
# macOS
brew install awscli

# Configure credentials
aws configure
```

--------------------------------

### Install Ollama using Homebrew

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Install Ollama on macOS using the Homebrew package manager.

```bash
brew install ollama
```

--------------------------------

### Install Foreman

Source: https://github.com/defilantech/llmkube/blob/main/docs/site/foreman/README.md

Installs the Foreman operator and a foreman-agent Deployment. This registers a FleetNode for the gate-runner role.

```bash
helm install foreman llmkube/foreman \
  --namespace foreman-system --create-namespace
```

--------------------------------

### List All Available Models in Catalog

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.2.2.md

Use this command to list all models available in the LLMKube Model Catalog. The output includes model ID, name, size, quantization, use case, and estimated VRAM.

```bash
llmkube catalog list
```

--------------------------------

### Create and Manage Test GPU Pod

Source: https://github.com/defilantech/llmkube/blob/main/terraform/azure/README.md

Deploys a test pod requesting 2 GPUs, waits for completion, checks logs, and cleans up.

```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: gpu-test
spec:
  containers:
  - name: cuda-test
    image: nvidia/cuda:12.2.0-base-ubuntu22.04
    command: ["nvidia-smi"]
    resources:
      limits:
        nvidia.com/gpu: 2
  tolerations:
  - key: nvidia.com/gpu
    operator: Exists
    effect: NoSchedule
  restartPolicy: Never
EOF

# Wait for pod to complete (node will auto-scale up if needed)
kubectl wait --for=condition=complete pod/gpu-test --timeout=600s

# Check logs (should show 2 GPUs)
kubectl logs gpu-test

# Cleanup
kubectl delete pod gpu-test
```

--------------------------------

### Install LLMKube CLI via cURL

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.4.0.md

Download and install the LLMKube binary for various architectures.

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.4.0/LLMKube_0.4.0_darwin_arm64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.4.0/LLMKube_0.4.0_darwin_amd64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.4.0/LLMKube_0.4.0_linux_amd64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

```bash
curl -L https://github.com/defilantech/LLMKube/releases/download/v0.4.0/LLMKube_0.4.0_linux_arm64.tar.gz | tar xz
sudo mv llmkube /usr/local/bin/
```

--------------------------------

### Start Metal Agent with Ollama Runtime

Source: https://github.com/defilantech/llmkube/blob/main/deployment/macos/README.md

Launch the LLMKube Metal Agent, specifying Ollama as the runtime backend.

```bash
llmkube-metal-agent --runtime ollama
```

--------------------------------

### Install Minikube on macOS

Source: https://github.com/defilantech/llmkube/blob/main/docs/minikube-quickstart.md

Use Homebrew or direct binary download to install Minikube on macOS.

```bash
# Using Homebrew
brew install minikube

# Or download directly
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64
sudo install minikube-darwin-amd64 /usr/local/bin/minikube
```

--------------------------------

### Check Prometheus Operator Installation

Source: https://github.com/defilantech/llmkube/blob/main/charts/llmkube/README.md

Verify if the 'prometheus-operator' is installed and running by checking its pods in the 'monitoring' namespace.

```bash
# Check if prometheus-operator is installed
kubectl get pods -n monitoring | grep prometheus-operator
```

--------------------------------

### Install LLMKube

Source: https://github.com/defilantech/llmkube/blob/main/docs/releases/RELEASE_NOTES_v0.2.0.md

Clone the repository and deploy the controller to your Kubernetes cluster.

```bash
git clone https://github.com/Defilan/LLMKube.git
cd llmkube
make install
make deploy IMG=ghcr.io/defilan/llmkube-controller:v0.2.0
```

--------------------------------

### Deploy 7B Model with CUDA Backend

Source: https://github.com/defilantech/llmkube/blob/main/docs/gpu-setup-guide.md

Create a `Model` resource for a 7B parameter model, specifying the CUDA backend and offloading all layers to the GPU. This prepares the model for GPU inference.

```bash
kubectl apply -f - <<EOF
apiVersion: inference.llmkube.dev/v1alpha1
kind: Model
metadata:
  name: phi-3-7b-gpu
  namespace: default
spec:
  source: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf
  format: gguf
  quantization: Q4
  hardware:
    accelerator: cuda
    gpu:
      enabled: true
      count: 1
      vendor: nvidia
      layers: -1  # Offload all layers to GPU
  resources:
    cpu: "4"
    memory: "8Gi"
EOF
```

--------------------------------

### Install Node Exporter as Systemd Service

Source: https://github.com/defilantech/llmkube/blob/main/config/grafana/SETUP.md

Install and configure Node Exporter as a persistent systemd service.

```bash
# Download latest release
wget https://github.com/prometheus/node_exporter/releases/download/v1.8.2/node_exporter-1.8.2.linux-amd64.tar.gz
tar xvfz node_exporter-1.8.2.linux-amd64.tar.gz
sudo mv node_exporter-1.8.2.linux-amd64/node_exporter /usr/local/bin/

# Create systemd service
sudo tee /etc/systemd/system/node_exporter.service << 'EOF'
[Unit]
Description=Node Exporter
After=network.target

[Service]
User=node_exporter
ExecStart=/usr/local/bin/node_exporter
Restart=always

[Install]
WantedBy=multi-user.target
EOF

sudo useradd -rs /bin/false node_exporter
sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter
```

--------------------------------

### Automated Deployment Script

Source: https://github.com/defilantech/llmkube/blob/main/terraform/azure/README.md

Execute the quick-start script to automate the deployment of the AKS cluster.

```bash
cd terraform/azure
./multi-gpu-quick-start.sh
```

--------------------------------

### Troubleshooting: WaitingForGPU or Insufficient GPU Events

Source: https://github.com/defilantech/llmkube/blob/main/docs/intel-gpu-quickstart.md

Steps to troubleshoot 'WaitingForGPU' or 'Insufficient GPU' events. This involves verifying the node allocatable resource key, ensuring the controller configuration matches, and checking pod events for detailed error information.

```bash
kubectl describe pod -l app=intel-phi4
```