Cluster API Provider Hetzner

https://syself.com/docs/caph/
Admin
Cluster API Provider Hetzner enables the management of Kubernetes clusters on Hetzner Cloud and bare...
Tokens:20,819
Snippets:267
Trust Score:7.1
Update:4 months ago
Show doc for...
Tokens:
Raw
### Example pre-provision-command usage

Source: https://syself.com/docs/caph/developers/pre-provision-command

This example shows how to specify the --pre-provision-command flag with a path to a custom script.

```bash
--pre-provision-command=/shared/my-pre-provision-command.sh
```

--------------------------------

### Install All Tools

Source: https://syself.com/docs/caph/developers/development-guide

Use this command to check and install missing or outdated development tools required for Tilt.

```shell
make all-tools
```

--------------------------------

### Example Cluster Manifest Variables Output

Source: https://syself.com/docs/caph/getting-started/quickstart/management-cluster-setup

An example output showing the variables needed for generating a cluster manifest. This output guides the user on what environment variables to set.

```shell
export SSH_KEY_NAME="my-ssh-key"
export HCLOUD_REGION="hel1"
export HCLOUD_X_MACHINE_TYPE="cx21"
```

--------------------------------

### HCloud Custom Command Example

Source: https://syself.com/docs/caph/developers/image-url-command

Example configuration for using a custom image URL command with HCloud machines.

```yaml
apiVersion: "pool.cluster.x-k8s.io/v1beta1"
kind: HCloudMachineTemplate
spec:
  template:
    spec:
      imageURL: "http://example.com/my-custom-image.tar.gz"
      imageURLCommand: "image-url-command-custom"

```

--------------------------------

### Create Management Cluster and Install CAPH

Source: https://syself.com/docs/caph/developers/development-guide

Configures the kind cluster as a management cluster and installs Cluster API (CAP) and CAPH.

```shell
make create-mgt-cluster
```

--------------------------------

### Describe HetznerBareMetalHost

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-host

Run this command on your management cluster to find the hardware details of your bare metal hosts after provisioning has started.

```shell
kubectl describe hetznerbaremetalhost
```

--------------------------------

### Start Tilt and Build Management Cluster

Source: https://syself.com/docs/caph/developers/development-guide

Builds a kind cluster and initiates Tilt for development. Access the Tilt UI at http://localhost:10351.

```shell
kind create cluster
tilt up
```

--------------------------------

### Install Cluster API Controllers

Source: https://syself.com/docs/caph/getting-started/quickstart/next-steps

Install the latest version of Cluster API controllers to move Cluster API objects. For a specific version, use the `--infrastructure <provider>:<version>` flag.

```shell
clusterctl init --infrastructure hetzner:v1.0.0
```

--------------------------------

### Image URL Command Execution Example

Source: https://syself.com/docs/caph/developers/image-url-command

Demonstrates how the custom command receives arguments including image URL, bootstrap data, machine name, and root devices.

```bash
image-url-command-custom http://example.com/my-custom-image.tar.gz bootstrap-data machine-name /dev/sda /dev/sdb
IMAGE_URL_DONE

```

--------------------------------

### HetznerBareMetalHost Provisioning Status

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Example output showing a HetznerBareMetalHost object in a 'Provisioning' state with a 'ValidationFailed' error due to missing root device hints.

```console
$ kubectl get hetznerbaremetalhost -A
default     my-cluster-md-1-tgvl5   my-cluster   default/test-bm-gpu    my-cluster-md-1-t9znj-694hs   Provisioning   23m   ValidationFailed   no root device hints specified


```

--------------------------------

### Initialize Hetzner Infrastructure Provider

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Installs the Cluster API core components and the Hetzner infrastructure provider onto the management cluster. This command ensures that CAPH is ready to manage Hetzner resources.

```shell
clusterctl init --infrastructure hetzner

```

--------------------------------

### HetznerBareMetalHost without WWN

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Example of a HetznerBareMetalHost manifest where the WWN for root device hints is not specified.

```yaml
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HetznerBareMetalHost
metadata:
  name: "caph-baremetal-server"
  namespace: default
spec:
  description: CAPH BareMetal Server
  serverID: <ID-of-your-server> # please check robot console
  maintenanceMode: false


```

--------------------------------

### HetznerBareMetalHost Object for RAID Setup

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-host

This snippet demonstrates how to configure a HetznerBareMetalHost object for a RAID setup. Update the serverID and the WWNs within the raid configuration.

```yaml
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HetznerBareMetalHost
metadata:
  name: "bm-0" #example
spec:
  serverID: 1682566 #change
  rootDeviceHints:
    raid:
      wwn:
        - "eui.0068475201b4egh2" #change
        - "eui.0068475201b4egh3" #change
  maintenanceMode: false
  description: Test Machine 0 #example

```

--------------------------------

### Basic HetznerBareMetalHost Object

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-host

Use this snippet to create a HetznerBareMetalHost object for a single bare metal server. Ensure to replace example values for serverID, rootDeviceHints, and description.

```yaml
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HetznerBareMetalHost
metadata:
  name: "bm-0" #example
spec:
  serverID: 1682566 #change
  rootDeviceHints:
    wwn: "eui.0068475201b4egh2" #change
  maintenanceMode: false
  description: Test Machine 0 #example

```

--------------------------------

### Describe Workload Cluster

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Get a detailed overview of the workload cluster and its resources.

```shell
clusterctl describe cluster my-cluster
```

--------------------------------

### Install Syself CCM

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Install the Syself CCM for Hetzner cloud, overriding the default secret name to 'hcloud'. Ensure the KUBECONFIG environment variable is set.

```shell
helm repo add syself https://charts.syself.com
helm repo update syself

KUBECONFIG=$CAPH_WORKER_CLUSTER_KUBECONFIG helm upgrade --install ccm syself/ccm-hetzner \
  --version 2.0.6 \
  --namespace kube-system \
  --set secret.name=hcloud

```

--------------------------------

### Initialize Management Cluster with Clusterctl

Source: https://syself.com/docs/caph/getting-started/quickstart/management-cluster-setup

Initializes a management cluster using Clusterctl. This command installs the necessary Cluster API provider components. For a specific version, use the `--infrastructure hetzner:vX.X.X` flag.

```shell
clusterctl init --infrastructure hetzner
```

--------------------------------

### Deploy Hetzner Cloud Controller Manager

Source: https://syself.com/docs/caph/topics/baremetal/creating-workload-cluster

Install the Syself CCM for bare metal using Helm. This requires a secret with Hetzner Robot and HCloud credentials. Ensure the HetznerCluster has the correct annotation for bare metal.

```shell
helm repo add syself https://charts.syself.com
helm repo update syself

helm upgrade --install ccm syself/ccm-hetzner --version 2.0.6 \
             --namespace kube-system \
             --set-json 'extraEnvVars=[{"name":"HCLOUD_USE_HROBOT_PROVIDER_ID_FOR_BAREMETAL","value":"true"}]' \
             --kubeconfig workload-kubeconfig
```

--------------------------------

### Install Hetzner CCM (Upstream)

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Apply the upstream Hetzner CCM static manifest. This works out-of-the-box as CAPH ensures the required secrets are present.

```shell
KUBECONFIG=$CAPH_WORKER_CLUSTER_KUBECONFIG kubectl apply \
  -f https://raw.githubusercontent.com/hetznercloud/hcloud-cloud-controller-manager/main/deploy/ccm.yaml

```

--------------------------------

### Check clusterctl version

Source: https://syself.com/docs/caph/topics/upgrading-caph

Verify the installed version of clusterctl to determine if an upgrade is necessary. The output shows build details and the current version.

```console
$ clusterctl version -oyaml
clusterctl:
  buildDate: "2024-04-09T17:23:12Z"
  compiler: gc
  gitCommit: c9136af030eaba5deed7be5ca9f6c3f5e6d69334
  gitTreeState: clean
  gitVersion: v1.7.0-rc.1
  goVersion: go1.21.9
  major: "1"
  minor: "7"
  platform: linux/amd64

```

--------------------------------

### OCI Registry Authentication Setup

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-machine-template

To pull images from a private OCI registry, provide the OCI_REGISTRY_AUTH_TOKEN environment variable to the controller via a Kubernetes secret.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  # ...
spec:
  # ...
  template:
    spec:
      containers:
        - command:
            - /manager
          image: ghcr.io/syself/caph:vXXX
          env:
            - name: OCI_REGISTRY_AUTH_TOKEN
              valueFrom:
                secretKeyRef:
                  name: my-oci-registry-secret # The name of the secret
                  key: OCI_REGISTRY_AUTH_TOKEN # The key in the secret. Format: "user:pwd" or just "token"
      # ... other container specs

```

--------------------------------

### Initialize Management Cluster with Cluster API Provider Hetzner

Source: https://syself.com/docs/caph/getting-started/preparation

Use this command to initialize a Kubernetes cluster as a management cluster with the Cluster API provider for Hetzner. Ensure kubectl is configured and Clusterctl is installed.

```shell
clusterctl init --core cluster-api --bootstrap kubeadm --control-plane kubeadm --infrastructure hetzner
```

--------------------------------

### Deploy Cilium CNI with Helm

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Install Cilium as the CNI solution using Helm. This command adds the Cilium Helm repository, upgrades/installs Cilium, and applies a specific values file for configuration.

```shell
helm repo add cilium https://helm.cilium.io/ 
KUBECONFIG=$CAPH_WORKER_CLUSTER_KUBECONFIG helm upgrade --install cilium cilium/cilium \
--namespace kube-system \
-f https://raw.githubusercontent.com/syself/cluster-api-provider-hetzner/main/templates/cilium/values.yaml
```

--------------------------------

### Generate Workload Cluster Manifest

Source: https://syself.com/docs/caph/topics/baremetal/creating-workload-cluster

Use `clusterctl generate` to create the YAML manifest for your workload cluster. Ensure the flavor matches your Hetzner setup.

```shell
clusterctl generate cluster my-cluster --flavor hetzner-hcloud-control-planes > my-cluster.yaml
```

--------------------------------

### Test Workload Cluster Reachability

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Perform a quick reachability check to the workload cluster's API server before installing the CNI.

```shell
KUBECONFIG=$CAPH_WORKER_CLUSTER_KUBECONFIG kubectl get --raw='/readyz'
```

--------------------------------

### Packer Configuration for Hcloud Node Image

Source: https://syself.com/docs/caph/topics/node-image

This JSON configuration defines how HashiCorp Packer can build a custom node image for Hetzner Cloud. It specifies variables, builder settings for Hcloud, and provisioners to install necessary components.

```json
{
  "variables": {
    "hcloud_token": "{{env `HCLOUD_TOKEN`}}",
    "scripts": "{{template_dir}}/scripts",
    "os": "ubuntu-YY.MM",
    "arch": "amd64",
    "image-name": "1.XX.Y-ubuntu-YY.MM-containerd",
    "version": "{{isotime \"2006-01-02-1504\"}}"
  },
  "sensitive-variables": [
    "hcloud_token"
  ],
  "builders": [
    {
      "type": "hcloud",
      "token": "{{user `hcloud_token`}}",
      "image": "{{user `os`}}",
      "location": "fsn1",
      "server_type": "cx23",
      "ssh_username": "root",
      "snapshot_name": "{{user `image-name`}}-{{user `version`}}",
      "snapshot_labels": {
        "caph-image-name": "{{user `image-name`}}-{{user `version`}}"
      }
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "scripts": [
        "{{user `scripts`}}/install-kubelet-and-kubectl.sh"
      ]
    }
  ],
  "post-processors": [
    [
      {
        "output": "manifest.json",
        "strip_path": false,
        "type": "manifest",
        "custom_data": {
          "snapshot_label": "{{user `image-name`}}-{{user `version`}}"
        }
      }
    ]
  ]
}

```

--------------------------------

### Install Cilium CNI

Source: https://syself.com/docs/caph/topics/baremetal/creating-workload-cluster

Deploy Cilium as the CNI for the workload cluster using Helm to manage networking. This command installs Cilium with Hubble enabled.

```shell
$ helm install cilium cilium/cilium --kubeconfig workload-kubeconfig
```

--------------------------------

### Show All Make Targets

Source: https://syself.com/docs/caph/developers/development-guide

Displays a list of all available make targets and their descriptions.

```shell
make help
```

--------------------------------

### Custom Image URL Command Configuration

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-machine-template

Provision a bare metal machine using a custom script to fetch the image. The 'name' and 'path' fields must be empty when using this method.

```yaml
imageURLCommand: image-url-command-install-foo.sh
image:
  url: oci://ghcr.io/myorg/images/Ubuntu-2404-noble-amd64-custom:1.0.1

```

--------------------------------

### Get Workload Cluster Kubeconfig

Source: https://syself.com/docs/caph/topics/baremetal/creating-workload-cluster

Obtain the kubeconfig for the workload cluster and set appropriate permissions. This is necessary for subsequent cluster management operations.

```shell
clusterctl get kubeconfig my-cluster > workload-kubeconfig
chmod go-r workload-kubeconfig # required to avoid helm warning
```

--------------------------------

### HetznerBareMetalHost Object Manifest

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

A sample manifest for creating a HetznerBareMetalHost object, including server ID and optional root device hints.

```yaml
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HetznerBareMetalHost
metadata:
  name: "caph-baremetal-server"
  namespace: default
spec:
  description: CAPH BareMetal Server
  serverID: <ID-of-your-server> # please check robot console
  rootDeviceHints:
    wwn: <wwn> 
  maintenanceMode: false


```

--------------------------------

### Create Kind Bootstrap Cluster

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Creates a local management cluster using 'kind'. This cluster serves as the bootstrap environment for deploying other clusters.

```shell
kind create cluster

```

--------------------------------

### List Required Variables for Hetzner Cluster

Source: https://syself.com/docs/caph/getting-started/preparation

Use `clusterctl` to list all variables needed for generating a Hetzner cluster manifest. This helps identify all required and optional configuration parameters.

```console
$ clusterctl generate cluster --infrastructure hetzner:<caph-version> --list-variables hetzner-cluster
Required Variables:
  - HCLOUD_CONTROL_PLANE_MACHINE_TYPE
  - HCLOUD_REGION
  - SSH_KEY_NAME
  - HCLOUD_WORKER_MACHINE_TYPE

Optional Variables:
  - CLUSTER_NAME                 (defaults to hetzner-cluster)
  - CONTROL_PLANE_MACHINE_COUNT  (defaults to 3)
  - KUBERNETES_VERSION           (defaults to 1.36.0)
  - WORKER_MACHINE_COUNT         (defaults to 3)


```

--------------------------------

### Generate Cluster Configuration and List Variables

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Generates a cluster configuration using 'clusterctl' and lists the required and optional variables for Hetzner infrastructure. This helps in understanding the parameters needed for cluster deployment.

```console
export CAPH_VERSION="v1.0.7"
clusterctl generate cluster my-cluster \
--infrastructure hetzner:${CAPH_VERSION} \
--list-variables \
--flavor hetzner-hcloud-control-planes

Required Variables:
  - HCLOUD_CONTROL_PLANE_MACHINE_TYPE
  - HCLOUD_REGION
  - SSH_KEY_NAME
  - HCLOUD_WORKER_MACHINE_TYPE

Optional Variables:
  - CLUSTER_NAME                 (defaults to my-cluster)
  - CONTROL_PLANE_MACHINE_COUNT  (defaults to 3)
  - KUBERNETES_VERSION           (defaults to v1.36.0)
  - WORKER_MACHINE_COUNT         (defaults to 3)

```

--------------------------------

### List Block Devices with WWN and Size

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Shell command to list block devices, their WWNs, and sizes, useful when multiple disks are present.

```shell
lsblk -d -o name,type,wwn,size
NAME TYPE WWN                  SIZE
sda  disk <wwn>                238.5G
sdb  disk <wwn>                238.5G
sdc  disk <wwn>                  1.8T
sdd  disk <wwn>                  1.8T


```

--------------------------------

### Update HetznerBareMetalHost with Root Device Hints

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

YAML snippet showing the updated rootDeviceHints section of a HetznerBareMetalHost object with a specific WWN.

```yaml
rootDeviceHints:
  wwn: "0x500a07511bb48992"


```

--------------------------------

### Patch HCloud Secret for Auto-Move

Source: https://syself.com/docs/caph/getting-started/quickstart/management-cluster-setup

Patch the created HCloud secret to enable automatic movement to the target cluster. This ensures the secret is available for CCM and CSI installations.

```shell
kubectl patch secret hcloud -n hcloud-system -p '{"data":{"HCLOUD_TOKEN":"'$HCLOUD_TOKEN'"}}'
```

--------------------------------

### Apply Workload Cluster Configuration

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Apply the generated `my-cluster.yaml` file to create the workload cluster resources.

```shell
kubectl apply -f my-cluster.yaml
```

--------------------------------

### Configuring CAPH deployment with an init-container

Source: https://syself.com/docs/caph/developers/pre-provision-command

This YAML configuration demonstrates how to use an init-container to provide a custom pre-provision command script to the CAPH controller. It includes mounting a shared volume and adding the command to the CAPH container's arguments.

```yaml
      # Init container, which makes the command available to caph.
      initContainers:
      - command:
        - /bin/sh
        - -c
        - cp /my-pre-provision-command.sh /shared/
        image: ghcr.io/syself/caph-staging:pre-provision-command
        imagePullPolicy: Always
        name: init-container
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /shared
          name: shared


      # Add this to the args of the caph container:
        args:
        - --pre-provision-command=/shared/my-pre-provision-command.sh

        # Add this to the caph container
        volumeMounts:
        - mountPath: /shared
          name: shared

      # Add this after "container"
      volumes:
      - emptyDir: {}
        name: shared
```

--------------------------------

### List Variables for Cluster Manifest

Source: https://syself.com/docs/caph/getting-started/quickstart/management-cluster-setup

Lists all required variables for generating a cluster manifest from the cluster-template.yaml. This helps in preparing the necessary environment variables.

```shell
clusterctl config cluster --kubernetes-version v1.28.0 --control-plane-machine-count 1 --worker-machine-count 0 --infrastructure hetzner:v0.70.0
```

--------------------------------

### List Block Devices with WWN

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Shell command to list block devices and their WWNs, useful for identifying storage devices on a bare metal server.

```shell
# lsblk --nodeps --output name,type,wwn
NAME TYPE WWN
sda  disk 0x500a07511bb48992
sdb  disk 0x500a07511bb48b25


```

--------------------------------

### Bare Metal Custom Command Configuration

Source: https://syself.com/docs/caph/developers/image-url-command

When using bare metal custom-command mode, ensure that `image.name` and `image.path` remain empty.

```yaml
spec:
  installImage:
    imageURLCommand: "image-url-command-custom"
    # image.name and image.path must stay empty
    # image:
    #   name: ""
    #   path: ""

```

--------------------------------

### Deploy Hetzner CSI

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Deploy the Hetzner CSI using its Helm chart. A custom values file is used to define storage classes. Ensure KUBECONFIG is set.

```shell
helm repo add hcloud https://charts.hetzner.cloud
helm repo update hcloud

cat << EOF > csi-values.yaml
storageClasses:
- name: hcloud-volumes
  defaultStorageClass: true
  reclaimPolicy: Retain
EOF

KUBECONFIG=$CAPH_WORKER_CLUSTER_KUBECONFIG helm upgrade --install csi hcloud/hcloud-csi \
--namespace kube-system -f csi-values.yaml

```

--------------------------------

### Create Git Tag

Source: https://syself.com/docs/caph/developers/releasing

Steps to create an annotated Git tag for a release. Ensure you are on the main branch and have pulled the latest changes before tagging.

```bash
git switch main
git pull
export RELEASE_TAG=<the tag of the release to be cut> (eg. export RELEASE_TAG=v1.0.7 )
git tag -a ${RELEASE_TAG} -m ${RELEASE_TAG}
```

--------------------------------

### Switch to Workload Cluster

Source: https://syself.com/docs/caph/getting-started/quickstart/next-steps

Use this command to switch your current context to the workload cluster.

```shell
clusterctl cluster switch --cluster <workload-cluster-name>
```

--------------------------------

### Switch Back to Management Cluster

Source: https://syself.com/docs/caph/getting-started/quickstart/next-steps

Command to switch back to the management cluster. Assumes the management cluster's kubeconfig is at the default location.

```shell
export KUBECONFIG=~/.kube/config
```

--------------------------------

### HTTPS Image URL Configuration

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-machine-template

Configure this to use a custom image hosted via HTTPS. Hetzner's installImage script uses the image name to detect the version, so follow their naming conventions.

```yaml
image:
  name: Ubuntu-2404-noble-amd64-custom
  url: https://user:pwd@example.com/images/Ubuntu-2404-noble-amd64-custom.tar.gz

```

--------------------------------

### Run Unit Tests

Source: https://syself.com/docs/caph/developers/development-guide

Triggers all unit tests. If tests fail unexpectedly, re-run them as they can sometimes be unstable due to Kubernetes envtest issues.

```shell
make test
```

--------------------------------

### Run Linting and Verification Checks

Source: https://syself.com/docs/caph/developers/development-guide

Executes all linting and verification checks. Ensure all checks pass before submitting a Pull Request. All changes must be committed for these checks to succeed.

```shell
make verify
```

--------------------------------

### Plan Cluster API upgrade

Source: https://syself.com/docs/caph/topics/upgrading-caph

Use `clusterctl upgrade plan` to preview available upgrades for Cluster API components and providers. Note that pre-release versions are not shown by default.

```console
$ clusterctl upgrade plan
Checking cert-manager version...
Cert-Manager is already up to date

Checking new release availability...

Latest release available for the v1beta2 API Version of Cluster API (contract):

NAME                     NAMESPACE                           TYPE                     CURRENT VERSION   NEXT VERSION
bootstrap-kubeadm        capi-kubeadm-bootstrap-system       BootstrapProvider        v1.12.7            v1.13.1
control-plane-kubeadm    capi-kubeadm-control-plane-system   ControlPlaneProvider     v1.12.7            v1.13.1
cluster-api              capi-system                         CoreProvider             v1.12.7            v1.13.1
infrastructure-hetzner   caph-system                         InfrastructureProvider   v1.0.7           Already up to date

You can now apply the upgrade by executing the following command:

clusterctl upgrade apply --contract v1beta2

```

--------------------------------

### Create Release Image

Source: https://syself.com/docs/caph/developers/releasing

Command to manually create release images after logging into ghcr. This is an alternative to the automated GitHub Actions.

```bash
make release-image
```

--------------------------------

### Create Workload Cluster with HCloud Control Planes

Source: https://syself.com/docs/caph/developers/development-guide

Deploys a workload cluster utilizing HCloud control planes and HCloud/Baremetal machine deployments.

```shell
make create-workload-cluster-hetzner-hcloud-control-plane
```

--------------------------------

### Scale CAPBK Controller Manager

Source: https://syself.com/docs/caph/topics/HA-setup

Scale the cluster-api-bootstrap-provider-kubeadm controller manager deployment to 2 replicas. This targets the 'capi-kubeadm-bootstrap-system' namespace.

```shell
kubectl -n capi-kubeadm-bootstrap-system scale deployment capi-kubeadm-bootstrap-controller-manager --replicas=2
```

--------------------------------

### Create SSH Key in Hetzner Cloud CLI

Source: https://syself.com/docs/caph/topics/managing-ssh-keys

Use the Hetzner Cloud CLI to create a new SSH key. This key will be used for accessing provisioned machines.

```shell
hcloud ssh-key create --name caph --public-key-from-file ~/.ssh/hetzner-cluster.pub

```

--------------------------------

### Run a Specific Unit Test Locally

Source: https://syself.com/docs/caph/developers/development-guide

Executes a single unit test. This is faster than running tests that require a cluster. The command filters tests by focusing on those containing 'foo' in their description and filters logs for easier debugging.

```shell
reset
DEBUG=1 ginkgo run --focus "foo" ./controllers/ | ./hack/filter-caph-controller-manager-logs.py -
```

--------------------------------

### Check Kubeadm Control Plane Status

Source: https://syself.com/docs/caph/getting-started/quickstart/creating-a-workload-cluster

Verify that the control plane has been initialized. Wait until `INITIALIZED` is `True`.

```shell
kubectl get kubeadmcontrolplane
```

--------------------------------

### Apply Cluster API upgrade

Source: https://syself.com/docs/caph/topics/upgrading-caph

Execute the `clusterctl upgrade apply` command with the specified contract to upgrade the Cluster API core components to the latest available version.

```console
$ clusterctl upgrade apply --contract v1beta2
Checking cert-manager version...
Cert-manager is already up to date
Performing upgrade...
Scaling down Provider="cluster-api" Version="v1.12.7" Namespace="capi-system"
Scaling down Provider="bootstrap-kubeadm" Version="v1.12.7" Namespace="capi-kubeadm-bootstrap-system"
Scaling down Provider="control-plane-kubeadm" Version="v1.12.7" Namespace="capi-kubeadm-control-plane-system"
Deleting Provider="cluster-api" Version="v1.12.7" Namespace="capi-system"
Installing Provider="cluster-api" Version="v1.13.1" TargetNamespace="capi-system"
Deleting Provider="bootstrap-kubeadm" Version="v1.12.7" Namespace="capi-kubeadm-bootstrap-system"
Installing Provider="bootstrap-kubeadm" Version="v1.13.1" TargetNamespace="capi-kubeadm-bootstrap-system"
Deleting Provider="control-plane-kubeadm" Version="v1.12.7" Namespace="capi-kubeadm-control-plane-system"
Installing Provider="control-plane-kubeadm" Version="v1.13.1" TargetNamespace="capi-kubeadm-control-plane-system"

```

--------------------------------

### Push Git Tag to GitHub

Source: https://syself.com/docs/caph/developers/releasing

Push the newly created Git tag to the GitHub repository. This action triggers a GitHub Action to create a draft release.

```bash
git push origin ${RELEASE_TAG}
```

--------------------------------

### Pushing Image to OCI Registry with Oras

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-machine-template

Use the 'oras' tool to push an image to an OCI registry. Ensure the artifact type is correctly specified.

```shell
oras push ghcr.io/myorg/images/Ubuntu-2404-noble-amd64-custom:1.0.1 \
    --artifact-type application/vnd.myorg.machine-image.v1 Ubuntu-2404-noble-amd64-custom.tar.gz

```

--------------------------------

### NFS Image Path Configuration

Source: https://syself.com/docs/caph/reference/hetzner-bare-metal-machine-template

Use this configuration to specify an image located on an NFS share provided by Hetzner.

```yaml
image:
  path: /root/.oldroot/nfs//images/Ubuntu-2404-noble-amd64-base.tar.gz

```

--------------------------------

### Scale KCP Controller Manager

Source: https://syself.com/docs/caph/topics/HA-setup

Scale the cluster-api-control-plane-kubeadm controller manager deployment to 2 replicas. This operation is performed within the 'capi-kubeadm-control-plane-system' namespace.

```shell
kubectl -n capi-kubeadm-control-plane-system scale deployment capi-kubeadm-control-plane-controller-manager --replicas=2
```

--------------------------------

### HetznerBareMetalHost Storage Device Information

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Status output of a HetznerBareMetalHost object detailing available storage devices and their WWNs.

```yaml
storage:
  - hctl: "2:0:0:0"
    model: Micron_1100_MTFDDAK512TBN
    name: sda
    serialNumber: 18081BB48B25
    sizeBytes: 512110190592
    sizeGB: 512
    vendor: "ATA     "
    wwn: "0x500a07511bb48b25"
  - hctl: "1:0:0:0"
    model: Micron_1100_MTFDDAK512TBN
    name: sdb
    serialNumber: 18081BB48992
    sizeBytes: 512110190592
    sizeGB: 512
    vendor: "ATA     "
    wwn: "0x500a07511bb48992"


```

--------------------------------

### Output of Applying Workload Cluster Manifest

Source: https://syself.com/docs/caph/topics/baremetal/creating-workload-cluster

This output shows the Kubernetes resources created when applying the workload cluster manifest, including Cluster API objects and Hetzner-specific infrastructure templates.

```console
$ kubectl apply -f my-cluster.yaml
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/my-cluster-md-0 created
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/my-cluster-md-1 created
cluster.cluster.x-k8s.io/my-cluster created
machinedeployment.cluster.x-k8s.io/my-cluster-md-0 created
machinedeployment.cluster.x-k8s.io/my-cluster-md-1 created
machinehealthcheck.cluster.x-k8s.io/my-cluster-control-plane-unhealthy-5m created
machinehealthcheck.cluster.x-k8s.io/my-cluster-md-0-unhealthy-5m created
machinehealthcheck.cluster.x-k8s.io/my-cluster-md-1-unhealthy-5m created
kubeadmcontrolplane.controlplane.cluster.x-k8s.io/my-cluster-control-plane created
hcloudmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-control-plane created
hcloudmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-md-0 created
hcloudremediationtemplate.infrastructure.cluster.x-k8s.io/control-plane-remediation-request created
hcloudremediationtemplate.infrastructure.cluster.x-k8s.io/worker-remediation-request created
hetznerbaremetalmachinetemplate.infrastructure.cluster.x-k8s.io/my-cluster-md-1 created
hetznercluster.infrastructure.cluster.x-k8s.io/my-cluster created

```

--------------------------------

### Export Hetzner API Credentials and SSH Key Paths

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Sets environment variables for HCloud token, Robot API credentials, and paths to your SSH public and private keys. These are required for creating Kubernetes secrets.

```shell
export HCLOUD_TOKEN="<YOUR-TOKEN>"
export HETZNER_ROBOT_USER="<YOUR-ROBOT-USER>"
export HETZNER_ROBOT_PASSWORD="<YOUR-ROBOT-PASSWORD>"
export SSH_KEY_NAME="<YOUR-SSH-KEY-NAME>"
export HETZNER_SSH_PUB_PATH="<YOUR-SSH-PUBLIC-PATH>"
export HETZNER_SSH_PRIV_PATH="<YOUR-SSH-PRIVATE-PATH>"

```

--------------------------------

### Configure Machine Health Check and HetznerBareMetalRemediationTemplate

Source: https://syself.com/docs/caph/topics/advanced/custom-templates-mhc

This snippet shows how to configure a Machine Health Check to use a HetznerBareMetalRemediationTemplate for custom remediation. The template specifies a 'Reboot' strategy with a retry limit and timeout.

```yaml
apiVersion: cluster.x-k8s.io/v1beta2
kind: MachineHealthCheck
metadata:
  name: "cluster123-control-plane-unhealthy-5m"
spec:
  clusterName: "cluster123"
  selector:
    matchLabels:
      cluster.x-k8s.io/control-plane: ""
  checks:
    nodeStartupTimeoutSeconds: 1200
    unhealthyNodeConditions:
      - type: Ready
        status: Unknown
        timeoutSeconds: 300
      - type: Ready
        status: "False"
        timeoutSeconds: 300
  remediation:
    triggerIf:
      unhealthyLessThanOrEqualTo: 100%
    templateRef: # added infrastructure reference
      kind: HetznerBareMetalRemediationTemplate
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      name: control-plane-remediation-request
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HetznerBareMetalRemediationTemplate
metadata:
  name: control-plane-remediation-request
spec:
  template:
    spec:
      strategy:
        type: "Reboot"
        retryLimit: 2
        timeout: 300s

```

--------------------------------

### Verify CAPH Pod Status

Source: https://syself.com/docs/caph/topics/upgrading-caph

After the upgrade, check the status of the CAPH controller manager pod in the `caph-system` namespace to ensure it is running.

```bash
$ kubectl get pods -n caph-system
NAME                                       READY   STATUS    RESTARTS   AGE
caph-controller-manager-85fcb6ffcb-4sj6d   1/1     Running   0          79s

```

--------------------------------

### Set kubectl context to management cluster

Source: https://syself.com/docs/caph/topics/upgrading-caph

Connect kubectl to your management cluster and verify the current context to ensure you are operating on the correct cluster.

```console
❯ k config current-context
mgm-cluster-admin@mgm-cluster

```

--------------------------------

### Tilt Configuration

Source: https://syself.com/docs/caph/developers/tilt

This JSON configuration is used by Tilt to manage the development environment for Cluster API Provider Hetzner. It specifies cluster contexts, component versions, and customizable parameters for cluster creation.

```json
"allowed_contexts": [
    "kind-caph",
],
"deploy_cert_manager": True,
"deploy_observability": False,
"preload_images_for_kind": True,
"kind_cluster_name": "caph",
"capi_version": "v1.13.1",
"cabpt_version": "v0.5.6",
"cacppt_version": "v0.4.11",
"cert_manager_version": "v1.20.2",
"kustomize_substitutions": {
    "HCLOUD_REGION": "fsn1",
    "CONTROL_PLANE_MACHINE_COUNT": "3",
    "WORKER_MACHINE_COUNT": "3",
    "KUBERNETES_VERSION": "v1.36.0",
    "HCLOUD_CONTROL_PLANE_MACHINE_TYPE": "cpx32",
    "HCLOUD_WORKER_MACHINE_TYPE": "cpx32",
    "CLUSTER_NAME": "test",
    "HETZNER_SSH_PUB_PATH": "~/.ssh/test",
    "HETZNER_SSH_PRIV_PATH": "~/.ssh/test",
    "HETZNER_ROBOT_USER": "test",
    "HETZNER_ROBOT_PASSWORD": "pw"
}


```

--------------------------------

### Scale CAPI Controller Manager

Source: https://syself.com/docs/caph/topics/HA-setup

Scale the CAPI controller manager deployment to 2 replicas for increased availability. Ensure the 'capi-system' namespace is correctly targeted.

```shell
kubectl -n capi-system scale deployment capi-controller-manager --replicas=2
```

--------------------------------

### Generate SSH Key Pair

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Creates an Ed25519 SSH key pair locally. The public key can then be uploaded to Hetzner Cloud.

```shell
ssh-keygen -t ed25519 -f ~/.ssh/caph

```

--------------------------------

### Create Hetzner and Robot API Secrets

Source: https://syself.com/docs/caph/topics/baremetal/management-cluster

Creates Kubernetes secrets for HCloud API token and Robot API user/password, and for SSH key details. These secrets enable communication with Hetzner services.

```shell
kubectl create secret generic hetzner --from-literal=hcloud=$HCLOUD_TOKEN --from-literal=robot-user=$HETZNER_ROBOT_USER --from-literal=robot-password=$HETZNER_ROBOT_PASSWORD

kubectl create secret generic robot-ssh --from-literal=sshkey-name=$SSH_KEY_NAME \
        --from-file=ssh-privatekey=$HETZNER_SSH_PRIV_PATH \
        --from-file=ssh-publickey=$HETZNER_SSH_PUB_PATH

```

--------------------------------

### Upgrade CAPH Infrastructure

Source: https://syself.com/docs/caph/topics/upgrading-caph

Use this command to upgrade the CAPH infrastructure to a specific version. Ensure you specify the provider and version correctly.

```bash
$ clusterctl upgrade apply --infrastructure=hetzner:v1.0.7
Checking cert-manager version...
Cert-manager is already up to date
Performing upgrade...
Scaling down Provider="infrastructure-hetzner" Version="" Namespace="caph-system"
Deleting Provider="infrastructure-hetzner" Version="" Namespace="caph-system"
Installing Provider="infrastructure-hetzner" Version="v1.0.7" TargetNamespace="caph-system"

```

--------------------------------

### Set Environment Variables for HCloud and Robot Integration

Source: https://syself.com/docs/caph/getting-started/preparation

Define environment variables for both HCloud and Robot API access, including tokens, user credentials, and SSH key paths. These are needed for advanced Hetzner cluster provisioning.

```shell
export HCLOUD_TOKEN="<YOUR-TOKEN>"
export HETZNER_ROBOT_USER="<YOUR-ROBOT-USER>"
export HETZNER_ROBOT_PASSWORD="<YOUR-ROBOT-PASSWORD>"
export HETZNER_SSH_PUB_PATH="<YOUR-SSH-PUBLIC-PATH>"
export HETZNER_SSH_PRIV_PATH="<YOUR-SSH-PRIVATE-PATH>"


```

--------------------------------

### Move Cluster API Objects

Source: https://syself.com/docs/caph/getting-started/quickstart/next-steps

Use this command to move Cluster API objects into the new cluster. Refer to Clusterctl Flags for detailed options.

```shell
clusterctl move --cluster <workload-cluster-name> --namespace <namespace>
```

--------------------------------

### Create HCloud and Robot Secrets for API Communication

Source: https://syself.com/docs/caph/getting-started/preparation

Create Kubernetes secrets for both HCloud and Robot API access. This involves creating a 'hetzner' secret with HCloud token and Robot credentials, and a 'robot-ssh' secret with SSH keys. Patch both secrets to allow automatic movement to the target cluster.

```shell
kubectl create secret generic hetzner --from-literal=hcloud=$HCLOUD_TOKEN --from-literal=robot-user=$HETZNER_ROBOT_USER --from-literal=robot-password=$HETZNER_ROBOT_PASSWORD

kubectl create secret generic robot-ssh --from-literal=sshkey-name=cluster --from-file=ssh-privatekey=$HETZNER_SSH_PRIV_PATH --from-file=ssh-publickey=$HETZNER_SSH_PUB_PATH

# Patch the created secrets so that they get automatically moved to the target cluster later.
kubectl patch secret hetzner -p '{"metadata":{"labels":{"clusterctl.cluster.x-k8s.io/move":""}}}'
kubectl patch secret robot-ssh -p '{"metadata":{"labels":{"clusterctl.cluster.x-k8s.io/move":""}}}'


```
Tokens:
Raw