### Install Nginx on VM

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/networking/kubeovn-vpcnatgateway.md

Install and start the Nginx web server on a VM within the overlay network to test inbound access via DNAT.

```bash
sudo apt update
sudo apt install nginx -y
sudo systemctl status nginx

```

--------------------------------

### Configure DNS and Install Helm in Chroot

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/harvester-configuration.md

This example demonstrates setting up a temporary DNS resolver and installing Helm using `curl` within the `after_install_chroot_commands` stage. It includes cleaning up the temporary resolv.conf afterwards.

```yaml
os:
  after_install_chroot_commands:
    - "echo 'nameserver 8.8.8.8' | sudo tee /etc/resolv.conf"
    - "mkdir /usr/local/bin"
    - "curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 700 get_helm.sh && ./get_helm.sh"
    - "rm -f /etc/resolv.conf"
```

--------------------------------

### Example IP Route Configuration

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/iso-install.md

This example shows the output of the `ip route` command, illustrating how the default route is configured via DHCP. This is crucial for the node to start correctly.

```bash
default via 192.168.122.1 dev mgmt-br proto dhcp
```

--------------------------------

### Example Bundle Status Output

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/upgrade/v1-1-2-to-v1-2-0.md

An example output of the `kubectl get bundle -A` command, illustrating the status of different bundles, including those that are OutOfSync or WaitApplied.

```bash
# kubectl get bundle -A
NAMESPACE     NAME                                          BUNDLEDEPLOYMENTS-READY   STATUS
...
fleet-local   mcc-local-managed-system-upgrade-controller   1/1
fleet-local   mcc-rancher-logging                           0/1                       OutOfSync(1) [Cluster fleet-local/local]
fleet-local   mcc-rancher-logging-crd                       0/1                       OutOfSync(1) [Cluster fleet-local/local]
fleet-local   mcc-rancher-monitoring                        0/1                       OutOfSync(1) [Cluster fleet-local/local]
fleet-local   mcc-rancher-monitoring-crd                    0/1                       WaitApplied(1) [Cluster fleet-local/local]
```

--------------------------------

### Configure multipathd via Cloud-init File

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/csidriver.md

Example of a cloud-init file to enable and start the multipathd service during system initialization. This automates the setup for nodes.

```yaml
stages:
   initramfs:
   - name: "start multipathd"
     systemctl:
       enable:
         - multipathd
       start:
         - multipathd
```

--------------------------------

### Example Upgrade Pods Output

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/upgrade/v1-1-2-to-v1-2-0.md

An example output of the `kubectl get pods -A | grep upgrade` command, showing typical pods related to system upgrades and their status.

```bash
# kubectl get pods -A | grep upgrade
cattle-system               system-upgrade-controller-5685d568ff-tkvxb                 1/1     Running     0              85m
harvester-system            hvst-upgrade-vq4hl-apply-manifests-65vv8                   1/1     Running     0              87m  // waiting for managedchart to be ready
..
```

--------------------------------

### Start Local Development Server

Source: https://github.com/harvester/docs/blob/main/README.md

Starts a local development server with live updates for most file changes.

```bash
yarn start
```

--------------------------------

### RKE2 Installation and Configuration

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/rancher/import-existing-vm.md

Installs RKE2 server on the VM, configures basic settings, and enables the service. Ensure the VM has internet access to download the installation script.

```shell
sudo mkdir -p /etc/rancher/rke2
echo "cni: calico
disable-kube-proxy: false
etcd-expose-metrics: false" | sudo tee /etc/rancher/rke2/config.yaml
curl -sfL https://get.rke2.io | sudo sh -
sudo systemctl enable rke2-server.service
sudo systemctl start rke2-server.service
```

--------------------------------

### Install qemu-guest-agent

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/rancher/node/node-driver.md

If your image does not contain the qemu-guest-agent, install it using this cloud-config to ensure successful cluster provisioning.

```yaml
#cloud-config
package_update: true
packages:
- qemu-guest-agent
runcmd:
- - systemctl
  - enable
  - '--now'
  - qemu-guest-agent.service

```

--------------------------------

### Storage Network Setting Configuration Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/storagenetwork.md

This is an example output of the storage network setting, showing its status as 'configured'.

```yaml
apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
  annotations:
    storage-network.settings.harvesterhci.io/hash: da39a3ee5e6b4b0d3255bfef95601890afd80709
    storage-network.settings.harvesterhci.io/net-attach-def: ""
    storage-network.settings.harvesterhci.io/old-net-attach-def: ""
  creationTimestamp: "2022-10-13T06:36:39Z"
  generation: 51
  name: storage-network
  resourceVersion: "154638"
  uid: 2233ad63-ee52-45f6-a79c-147e48fc88db
status:
  conditions:
  - lastUpdateTime: "2022-10-13T13:05:17Z"
    reason: Completed
    status: "True"
    type: configured
```

--------------------------------

### Example Cloud-Init User Data Output

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/rancher/cloud-provider.md

This example shows the complete cloud-init user data output, including the cloud config content written to `/etc/kubernetes/cloud-config`.

```yaml
########## cloud config ############
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <CACERT>
    server: https://HARVESTER-ENDPOINT/k8s/clusters/local
  name: local
contexts:
- context:
    cluster: local
    namespace: default
    user: harvester-cloud-provider-default-local
  name: harvester-cloud-provider-default-local
current-context: harvester-cloud-provider-default-local
kind: Config
preferences: {}
users:
- name: harvester-cloud-provider-default-local
  user:
    token: <TOKEN>

########## cloud-init user data ############
write_files:
- encoding: b64
  content: <CONTENT>
  owner: root:root
  path: /etc/kubernetes/cloud-config
  permissions: '0644'
```

--------------------------------

### Example RWX Network Setting Status Output

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/rwxnetwork.md

This is an example output of the `kubectl get settings.harvesterhci.io rwx-network -o yaml` command. It shows the status, conditions, and applied value of the RWX network configuration.

```yaml
apiVersion: harvesterhci.io/v1beta1
default: '{"share-storage-network":false}'
kind: Setting
metadata:
  annotations:
    rwx-network.settings.harvesterhci.io/hash: cc5102f8a0442e4e6d7e13923482f6594333b498
    rwx-network.settings.harvesterhci.io/initialized: "true"
    rwx-network.settings.harvesterhci.io/net-attach-def: harvester-system/rwx-network-976r9
    rwx-network.settings.harvesterhci.io/old-net-attach-def: ""
  creationTimestamp: "2026-03-20T01:48:26Z"
  generation: 6
  name: rwx-network
  resourceVersion: "1280504"
  uid: 5f934df5-b483-403c-af43-c1f1ec146db4
status:
  conditions:
  - lastUpdateTime: "2026-03-20T01:58:25Z"
    reason: Completed
    status: "True"
    type: configured
value: '{"share-storage-network":false,"network":{"clusterNetwork":"rwx","range":"192.168.201.0/24","exclude":["192.168.201.1/32"]}}'
```

--------------------------------

### Install Upgrade Manager Add-on

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/addons/upgrade-manager.md

Apply the add-on manifest to install the Upgrade Manager.

```bash
kubectl apply -f     https://raw.githubusercontent.com/harvester/experimental-addons/v1.8/harvester-upgrade-manager/harvester-upgrade-manager.yaml
```

--------------------------------

### Kubernetes Event Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/logging/harvester-logging.md

This is an example of a standard Kubernetes event object.

```json
{
            "apiVersion": "v1",
            "count": 1,
            "eventTime": null,
            "firstTimestamp": "2022-08-24T11:17:35Z",
            "involvedObject": {
                "apiVersion": "kubevirt.io/v1",
                "kind": "VirtualMachineInstance",
                "name": "vm-ide-1",
                "namespace": "default",
                "resourceVersion": "604601",
                "uid": "1bd4133f-5aa3-4eda-bd26-3193b255b480"
            },
            "kind": "Event",
            "lastTimestamp": "2022-08-24T11:17:35Z",
            "message": "VirtualMachineInstance defined.",
            "metadata": {
                "creationTimestamp": "2022-08-24T11:17:35Z",
                "name": "vm-ide-1.170e43cbdd833b62",
                "namespace": "default",
                "resourceVersion": "604626",
                "uid": "0114f4e7-1d4a-4201-b0e5-8cc8ede202f4"
            },
            "reason": "Created",
            "reportingComponent": "",
            "reportingInstance": "",
            "source": {
                "component": "virt-handler",
                "host": "harv1"
            },
            "type": "Normal"
        }
```

--------------------------------

### VM Migration Network Setting YAML Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/vm-migration-network.md

An example of the YAML output for the VM migration network setting, showing its status and associated network definitions.

```yaml
apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
  annotations:
    vm-migration-network.settings.harvesterhci.io/hash: ec8322fb6b741f94739cbb904fc73c3fda864d6d
    vm-migration-network.settings.harvesterhci.io/net-attach-def: harvester-system/vm-migration-network-6flk7
  creationTimestamp: "2025-06-13T06:36:39Z"
  generation: 51
  name: vm-migration--network
  resourceVersion: "154638"
  uid: 2233ad63-ee52-45f6-a79c-147e48fc88db
status:
  conditions:
  - lastUpdateTime: "2025-06-13T13:05:17Z"
    reason: Completed
    status: "True"
    type: configured
```

--------------------------------

### PXE Boot Kernel Parameter for Net Install

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/net-install.md

When using the net install ISO as a PXE installation source, add this parameter to the kernel boot configuration.

```bash
harvester.install.with_net_images=true
```

--------------------------------

### Identify Node with VIP (SSH Method) Example Output

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/management-address.md

Example output from 'ip address show mgmt-br' command, showing both node and VIP addresses.

```shell
4: mgmt-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8c:dc:d4:b5:f0:fd brd ff:ff:ff:ff:ff:ff
    inet 172.19.108.45/21 brd 172.19.111.255 scope global mgmt-br
       valid_lft forever preferred_lft forever
    inet 172.19.108.34/32 scope global mgmt-br
       valid_lft forever preferred_lft forever
```

--------------------------------

### Go: Check VM Starting Status

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/vm.md

This Go code snippet checks if a VM is currently starting by examining its state change requests. It's used to prevent concurrent start or restart operations.

```go
func (vf *vmformatter) canStart(vm *kubevirtv1.VirtualMachine, vmi *kubevirtv1.VirtualMachineInstance) bool {
  if vf.isVMStarting(vm) {
    return false
  }
..
}

func (vf *vmformatter) canRestart(vm *kubevirtv1.VirtualMachine, vmi *kubevirtv1.VirtualMachineInstance) bool {
  if vf.isVMStarting(vm) {
    return false
  }
...
}

func (vf *vmformatter) isVMStarting(vm *kubevirtv1.VirtualMachine) bool {
  for _, req := range vm.Status.StateChangeRequests {
    if req.Action == kubevirtv1.StartRequest {
      return true
    }
  }
  return false
}

```

--------------------------------

### Load Kernel Modules on Start

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/harvester-configuration.md

Specify a list of kernel modules to be loaded automatically when the system starts using the `os.modules` option.

```yaml
os:
  modules:
    - kvm
    - nvme
```

--------------------------------

### Install Rancher Helm Chart

Source: https://github.com/harvester/docs/blob/main/src/pages/v1.0/dev/dev-mode.md

Install the Rancher Helm chart with specific configurations for developer mode.

```bash
helm install rancher rancher-latest/rancher \
--namespace cattle-system \
--set tls=external \
--set rancherImagePullPolicy=IfNotPresent \
--set rancherImage=rancher/rancher \
--set rancherImageTag=v2.6.3-harvester1 \
--set noDefaultAdmin=false \
--set features="multi-cluster-management=false\,multi-cluster-management-agent=false" \
--set useBundledSystemChart=true \
--set bootstrapPassword=admin
```

--------------------------------

### Start RKE2 Server Service

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/host/host.md

Start the RKE2 server service on management nodes after manually rotating certificates.

```bash
systemctl start rke2-server
```

--------------------------------

### Check VirtualMachineInstance Interface Info

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/vm.md

Check the status of the VirtualMachineInstance object to determine if the qemu-guest-agent is installed. The output should contain 'guest-agent' if installed.

```bash
$ kubectl get vmi -n <NAMESPACE> <NAME> -ojsonpath='{.status.interfaces[0].infoSource}'
```

--------------------------------

### Install NFS Client on SUSE/OpenSUSE

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/rancher/csi-driver.md

Install the NFS client package on SUSE or OpenSUSE-based systems. This is required for guest cluster nodes to interact with NFS storage.

```bash
zypper install -y nfs-client

```

--------------------------------

### Get VIP MAC Address Output Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/management-address.md

Example JSON output when retrieving the VIP MAC address.

```json
{"kube-vip.io/hwaddr":"02:00:00:09:7f:3f","kube-vip.io/requestedIP":"10.84.102.31"}
```

--------------------------------

### Construct Support Bundle Download URL

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/harvester.md

This is an example URL structure for downloading a support bundle. Ensure you replace `{vip/dns-name}` with your cluster's VIP or DNS name and include `?retain=true` to keep the resources after download.

```text
https://{vip/dns-name}/v1/harvester/supportbundles/bundle-htl5f/download?retain=true
```

--------------------------------

### Install qemu-guest-agent using cloud-init

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/faq.md

After adding cloud-init configuration for the qemu-guest-agent, reboot the VM using this command to ensure the configuration is applied. This command cleans cloud-init logs and initiates a reboot.

```shell
# cloud-init will only be executed once, reboot it after add the cloud-init config with the following command.
$ cloud-init clean --logs --reboot
```

--------------------------------

### Harvester Installation Configuration Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/harvester-configuration.md

This YAML configuration file can be used during Harvester installation to define various operational parameters. It covers OS settings, network interfaces, storage devices, and cluster-wide configurations.

```yaml
scheme_version: 1
server_url: "" # omitted or empty in CREATE mode; set to https://cluster-VIP:443 in JOIN mode
token: TOKEN_VALUE
os:
  ssh_authorized_keys:
    - ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...
    - github:username
  write_files:
  - encoding: ""
    content: test content
    owner: root
    path: /etc/test.txt
    permissions: '0755'
  hostname: myhost
  modules:
    - kvm
    - nvme
  sysctls:
    kernel.printk: "4 4 1 7"
    kernel.kptr_restrict: "1"
  dns_nameservers:
    - 8.8.8.8
    - 1.1.1.1
  ntp_servers:
    - 0.suse.pool.ntp.org
    - 1.suse.pool.ntp.org
  password: rancher
  environment:
    http_proxy: http://myserver
    https_proxy: http://myserver
  labels:
    topology.kubernetes.io/zone: zone1
    foo: bar
    mylabel: myvalue
  externalStorageConfig:
    enabled: true
    multiPathConfig:
      blacklist:
      - vendor: "!QEMU"
        product: "!QEMU HARDDISK"
      blacklistWwids:
      - ".*"
      blacklistExceptions:
      - vendor: "DELL"
        product: "POWERVAULT"
      blacklistExceptionWwids:
      - "^0QEMU_QEMU_HARDDISK_disk[0-9]+"
  additionalKernelArguments: "multipath=on"
install:
  mode: create
  management_interface:
    interfaces:
    - name: ens5
      hwAddr: "B8:CA:3A:6A:64:7C"
    method: dhcp
  force_efi: true
  device: /dev/sda
  data_disk: /dev/sdb
  silent: true
  iso_url: http://myserver/test.iso
  poweroff: true
  no_format: true
  debug: true
  tty: ttyS0
  vip: 10.10.0.19
  vip_hw_addr: 52:54:00:ec:0e:0b
  vip_mode: dhcp
  cluster_pod_cidr: 10.52.0.0/16
  cluster_service_cidr: 10.53.0.0/16
  cluster_dns: 10.53.0.10
  addons:
    harvester_vm_import_controller:
      enabled: false
      values_content: ""
    harvester_pcidevices_controller:
      enabled: false
      values_content: ""
    rancher_monitoring:
      enabled: true
      values_content: ""
    rancher_logging:
      enabled: false
      values_content: ""
    harvester_seeder:
      enabled: false
      values_content: ""
system_settings:
  auto-disk-provision-paths: ""

```

--------------------------------

### Example Auto-Provisioning Configuration

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/disk-filter-and-auto-provision.md

This YAML snippet shows how to configure disks for automatic provisioning as VM storage. It specifies devices to be provisioned for different hostnames.

```yaml
autoprovision.yaml: |
  - hostname: "*"
    devices:
      - "/dev/sdc"
      - "/dev/sdd"
  - hostname: "harvester1"
    devices:
      - "/dev/sde"
```

--------------------------------

### Configure Webhooks for Installation Events

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/harvester-configuration.md

Define webhooks to receive notifications for installation events such as STARTED, SUCCEEDED, or FAILED. Supports GET/POST methods, insecure connections, basic authentication, custom headers, and JSON payloads.

```yaml
install:
  webhooks:
    - event: SUCCEEDED
      method: GET
      url: http://10.100.0.100/cblr/svc/op/nopxe/system/{{.Hostname}}
    - event: STARTED
      method: GET
      url: https://10.100.0.100/started/{{.Hostname}}
      insecure: true
      basicAuth:
        user: admin
        password: p@assword
    - event: FAILED
      method: POST
      url: http://10.100.0.100/record
      headers:
        Content-Type:
           - 'application/json; charset=utf-8'
      payload: |
        {
          "host": "{{.Hostname}}",
          "device": "hd"
        }
```

--------------------------------

### Get Pod Storage Network IP

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/storagenetwork.md

Example output showing how to identify the storage network IP address for an instance manager pod.

```text
instance-manager-43f1624d14076e1d95cd72371f0316e2
storage network IP: 10.0.16.8

instance-manager-ba38771e483008ce61249acf9948322f
storage network IP: 10.0.16.14
```

--------------------------------

### Label local-kubeconfig Secret

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/upgrade/v1-2-2-to-v1-3-1.md

Label the `local-kubeconfig` secret to prevent the cluster upgrade from getting stuck after the first node is upgraded. This should be done before starting the upgrade process.

```bash
kubectl label secret local-kubeconfig -n fleet-local cluster.x-k8s.io/cluster-name=local
```

--------------------------------

### Prepare Environment for Manual Support Bundle Data Collection

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/harvester.md

Prepare the environment by creating a directory for support bundle data and setting up necessary environment variables and configurations.

```sh
mkdir -p /tmp/support-bundle # ensure /tmp/support-bundle exists
echo 'JOURNALCTL="/usr/bin/journalctl -o short-precise"' > /tmp/common
export SUPPORT_BUNDLE_NODE_NAME=$(hostname)
```

--------------------------------

### Elemental Upgrade Error Log Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/upgrade/v1-1-2-to-v1-2-0.md

This log snippet shows an error indicating an invalid upgrade command setup due to an undefined state partition, often caused by an incomplete state.yaml.

```bash
Flag --directory has been deprecated, 'directory' is deprecated please use 'system' instead
INFO[2023-09-13T12:02:42Z] Starting elemental version 0.3.1
INFO[2023-09-13T12:02:42Z] reading configuration form '/tmp/tmp.N6rn4F6mKM'
ERRO[2023-09-13T12:02:42Z] Invalid upgrade command setup undefined state partition
elemental upgrade failed with return code: 33
+ ret=33
+ '[' 33 '!=' 0 ']'
+ echo 'elemental upgrade failed with return code: 33'
+ cat /host/usr/local/upgrade_tmp/elemental-upgrade-20230913120242.log
```

--------------------------------

### Configure Installation to Skip Checks

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/harvester-configuration.md

Set `install.skipchecks` to `true` to allow installation to proceed even if minimum hardware requirements for production use are not met. Warning messages will still be logged.

```yaml
install:
  skipchecks: true
```

--------------------------------

### BundleDeployment Status Update Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/installation.md

This YAML snippet shows a partial status of a BundleDeployment resource, indicating the `appliedDeploymentID` and the `Installed` condition. This status is updated by the Fleet agent when changes to the deployment image are detected.

```yaml
status:
  appliedDeploymentID: s-89f9ce3f33c069befb4ebdceaa103af7b71db0e70a39760cb6653366964e5:1cd9188211e318033f89b77acf7b996
e5bb3d9a25319528c47dc052528056f78
  conditions:
  - lastUpdateTime: "2025-08-28T04:44:18Z"
    status: "True"
    type: Installed

```

--------------------------------

### Host VM Image with HTTP Server

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/vm/backup-restore.md

Start a simple HTTP server to host a VM image for download by a new cluster.

```bash
$ python -m http.server
```

--------------------------------

### Configure Calico to Skip VIP Interfaces

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/rancher.md

Modify the Calico installation parameters to prevent the Calico controller from taking over Load Balancer IPs. Add the `skipInterface` configuration to exclude interfaces starting with 'vip.*' from IP autodetection.

```yaml
installation:
  backend: VXLAN
  calicoNetwork:
    bgp: Disabled
    nodeAddressAutodetectionV4:
      skipInterface: vip.*

```

--------------------------------

### Customizing Node Affinity for NVIDIA Driver Toolkit

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/addons/nvidiadrivertoolkit.md

Define custom node affinity settings to target specific GPU models for driver installation. This example shows how to target nodes with 'A100' or 'A40' GPU models that also have the 'sriovgpu.harvesterhci.io/driver-needed' label set to 'true'.

```yaml
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: sriovgpu.harvesterhci.io/driver-needed
          operator: In
          values:
          - "true"
        - key: gpu.model
          operator: In
          values:
          - "A100"
          - "A40"
```

--------------------------------

### Set Harvester Installation Mode

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/harvester-configuration.md

Specify the mode for installing Harvester. Use 'create' for a new installation or 'join' to add to an existing cluster.

```yaml
install:
  mode: create
```

--------------------------------

### Compare with Expected Device File

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/vm.md

This example shows the expected output when listing the contents of a CSI volume device directory for a running VM, where a block device file (e.g., `brw-rw----`) is present instead of an empty file.

```bash
$ ls  /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/publish/pvc-732f8496-103b-4a08-83af-8325e1c314b7/ -alth
total 8.0K
drwxr-x--- 2 root root  4.0K Feb 21 10:53 .
drwxr-x--- 4 root root  4.0K Feb 21 10:53 ..
brw-rw---- 1 root root 8, 16 Feb 21 10:53 4883af80-c202-4529-a2c6-4e7f15fe5a9b
```

--------------------------------

### List Installed Charts

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/troubleshooting/installation.md

Use this command to list all installed Helm charts across all namespaces. This is useful for verifying chart installations and their statuses.

```shell
helm list -A
```

--------------------------------

### Install NFS CSI Driver with Helm

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/csidriver.md

Installs the csi-driver-nfs Helm chart into the kube-system namespace. Ensure the NFS server is running and squashing is disabled before installation.

```bash
$ helm repo add csi-driver-nfs https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts
$ helm install csi-driver-nfs csi-driver-nfs/csi-driver-nfs --namespace kube-system --version v4.10.0
```

--------------------------------

### Example VM and POD Object Memory Configuration

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/settings.md

Illustrates the memory configuration for a VM and its corresponding POD when `additional-guest-memory-overhead-ratio` is set to "1.5". This shows how guest OS memory, VM configured memory, and POD container memory limit are determined.

```yaml
When `additional-guest-memory-overhead-ratio` is set as "1.5".

The VM object:
...
        memory:
          guest: 2Gi    // Guest OS Memory
        resources:
          limits:
            cpu: "1"
            memory: 2Gi // VM Configured Memory

The POD object:
...
    resources:
      limits:
        cpu: "1"
        devices.kubevirt.io/kvm: "1"
        devices.kubevirt.io/tun: "1"
        devices.kubevirt.io/vhost-net: "1"
        memory: "2532309561"                // POD Container Memory Limit

```

--------------------------------

### Harvester Advanced Settings Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/settings.md

This example demonstrates a comprehensive configuration for Harvester, including image preloading strategy, node upgrade options, VM restore settings, and log readiness timeout.

```json
{
  "imagePreloadOption": {
    "strategy": {
      "type": "parallel",
      "concurrency": 2
    }
  },
  "nodeUpgradeOption": {
    "strategy": {
      "mode": "manual",
      "pauseNodes": [
        "node-3"
      ]
    }
  },
  "restoreVM": true,
  "logReadyTimeout": "5"
}
```

--------------------------------

### Storage Network Configuration Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/settings.md

Configures a segregated storage network for Longhorn traffic, including VLAN, cluster network, and IP range.

```json
{
  "vlan": 100,
  "clusterNetwork": "storage",
  "range": "192.168.0.0/24"
}
```

--------------------------------

### Install RPM Package using Chroot Commands

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/harvester-configuration.md

Utilize `os.after_install_chroot_commands` to execute commands in a chroot environment after installation, such as installing RPM packages. Note that DNS resolution is unavailable in this stage.

```yaml
os:
  after_install_chroot_commands:
    - rpm -ivh <the url of rpm package>
```

--------------------------------

### Version YAML Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/settings.md

Example structure for version.yaml used by Harvester for upgrades.

```yaml
apiVersion: harvesterhci.io/v1beta1
kind: Version
metadata:
  name: ${VERSION}
  namespace: harvester-system
spec:
  isoChecksum: ${ISO_CHECKSUM}
  isoURL: ${ISO_URL}
```

--------------------------------

### Create UpgradePlan (Version-based)

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/addons/upgrade-manager.md

Create an UpgradePlan that references a Version resource by name to start an upgrade.

```yaml
apiVersion: management.harvesterhci.io/v1beta1
kind: UpgradePlan
metadata:
  generateName: hvst-upgrade- 
spec:
  version: v1.8.0
```

--------------------------------

### Install SELinux Policy Module

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/vm/backup-restore.md

Install the generated SELinux policy module using the 'semodule' command. This applies the custom policy to allow required QEMU Guest Agent operations.

```bash
semodule -i my_qemu_ga.pp
```

--------------------------------

### Install Docusaurus Dependencies

Source: https://github.com/harvester/docs/blob/main/README.md

Run this command to install Docusaurus and its necessary development dependencies.

```bash
yarn install
```

--------------------------------

### Download iPXE UEFI Program

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/install/pxe-boot-install.md

Download the appropriate iPXE UEFI program for your system's architecture and place it in the web server's directory.

```bash
cd /usr/share/nginx/html/harvester/
wget http://boot.ipxe.org/x86_64-efi/ipxe.efi
```

--------------------------------

### Preload Container Images for Air-gapped Upgrade

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/upgrade/v1-4-2-to-v1-5-0.md

In an air-gapped environment, use these commands to download and import the required container images before starting the upgrade. This ensures that Fluentd and Fluent Bit pods can start correctly.

```bash
ghcr.io/kube-logging/config-reloader:v0.0.5 \
    fluent/fluent-bit:2.1.8 > upgradelog-images.tar

# After transferring the tar file to the cluster nodes, import the images (need to be run on each node)
ctr -n k8s.io images import upgradelog-images.tar
```

--------------------------------

### Harvester Server Version Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/settings.md

Example format for the Harvester server version string.

```text
v1.0.0-abcdef-head
```

--------------------------------

### Configure Kubelet Bind Mount

Source: https://github.com/harvester/docs/blob/main/src/pages/v1.0/dev/dev-mode.md

If the Kubelet's RootDir is not the default, create a bind mount to ensure proper functionality.

```bash
KUBELET_ROOT_DIR="path to your kubelet root dir"
echo "${KUBELET_ROOT_DIR} /var/lib/kubelet none bind 0 0" >> /etc/fstab
mkdir -p /var/lib/kubelet && mount -a
```

--------------------------------

### Credentials Secret Example

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/addons/vmimport.md

Example of a Kubernetes Secret containing credentials for source clusters.

```APIDOC
## Secret for Credentials

### Description
Kubernetes Secret containing credentials for authenticating with source providers.

### Kind
Secret

### API Version
v1

### StringData
- **username** (string) - Required for most sources - The username for authentication.
- **password** (string) - Required for most sources - The password for authentication.
- **project_name** (string) - Optional for OpenStack - The OpenStack project name.
- **domain_name** (string) - Optional for OpenStack - The OpenStack domain name.
- **ca_cert** (string) - Optional for OpenStack/OVA - PEM-encoded CA certificate for HTTPS endpoints.
- **ca.crt** (string) - Optional for OVA - PEM-encoded CA certificate for HTTPS endpoints.

### Example (vSphere)
```yaml
apiVersion: v1
kind: Secret
metadata: 
  name: vsphere-credentials
  namespace: default
stringData:
  "username": "user"
  "password": "password"
```

### Example (OpenStack)
```yaml
apiVersion: v1
kind: Secret
metadata: 
  name: devstack-credentials
  namespace: default
stringData:
  "username": "user"
  "password": "password"
  "project_name": "admin"
  "domain_name": "default"
  "ca_cert": "pem-encoded-ca-cert"
```

### Example (OVA)
```yaml
apiVersion: v1
kind: Secret
metadata: 
  name: example-ova-credentials
  namespace: default
stringData:
  "username": "user"
  "password": "password"
  "ca.crt": "pem-encoded-ca-cert"
```
```

--------------------------------

### Example VM Migration Network Configuration

Source: https://github.com/harvester/docs/blob/main/versioned_docs/version-v1.8/advanced/settings.md

Sets up a segregated network for virtual machine migration traffic. Requires specifying a VLAN ID, cluster network name, and an IP range in IPv4 CIDR format.

```json
{
  "vlan": 100,
  "clusterNetwork": "vm-migration",
  "range": "192.168.1.0/24"
}
```

--------------------------------

### Install Harvester CRD Chart

Source: https://github.com/harvester/docs/blob/main/src/pages/v1.0/dev/dev-mode.md

Install the Custom Resource Definition (CRD) chart for Harvester.

```bash
helm install harvester-crd ./harvester-crd --namespace harvester-system
```