### Quick Start Installation and Service Management

Source: https://github.com/rancher/rke2/blob/master/README.md

Installs RKE2 using a curl script, enables and starts the RKE2 server service, and sets up the KUBECONFIG environment variable for kubectl access. Ensure the system has necessary prerequisites for RKE2.

```sh
curl -sfL https://get.rke2.io | sh -
systemctl enable rke2-server.service
systemctl start rke2-server.service
# Wait a bit
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml PATH=$PATH:/var/lib/rancher/rke2/bin
kubectl get nodes
```

--------------------------------

### RKE2 Release Tag Example

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Example of a release candidate tag format for RKE2.

```bash
v1.23.5-rc1+rke2r1.testing.0
```

--------------------------------

### Running RKE2 Install Tests with Vagrant

Source: https://github.com/rancher/rke2/blob/master/tests/install/README.md

Commands to run RKE2 installation tests using Vagrant. Includes starting the VM and optional provisioners for specific RKE2 components.

```shell
cd tests/install/rocky-8
vagrant up
# The following provisioners are optional. In GitHub Actions CI they are invoked
# explicitly to avoid certain timeout issues on slow runners
vagrant provision --provision-with=rke2-wait-for-node
vagrant provision --provision-with=rke2-wait-for-coredns
vagrant provision --provision-with=rke2-wait-for-local-storage
vagrant provision --provision-with=rke2-wait-for-metrics-server
vagrant provision --provision-with=rke2-wait-for-traefik
vagrant provision --provision-with=rke2-status
vagrant provision --provision-with=rke2-procps
```

--------------------------------

### Setting up Build Environment with xx-apk

Source: https://github.com/rancher/rke2/blob/master/developer-docs/docker_multiarch.md

Prepare the build stage by copying `xx` scripts and installing necessary packages. Use `xx-apk` to install Alpine packages (like `musl-dev`, `gcc`, `lld`) for the target architecture, enabling cross-compilation.

```dockerfile
FROM --platform=$BUILDPLATFORM ${GO_IMAGE} as base-builder
# copy xx scripts to your build stage
COPY --from=xx / /
RUN apk add file make git clang lld
ARG TARGETPLATFORM
# setup required packages
RUN set -x && \
    xx-apk --no-cache add musl-dev gcc lld 
```

--------------------------------

### Start RKE2 Server for Integration Tests

Source: https://github.com/rancher/rke2/blob/master/developer-docs/testing.md

Command to start an RKE2 server within a development shell, a prerequisite for running integration tests.

```bash
./bin/rke2 server
```

--------------------------------

### RKE2 Latest RPM Tag Example

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Example of a tag format for RKE2 latest RPMs.

```bash
v1.23.5+rke2r1.latest.0
```

--------------------------------

### RKE2 Primary Release Tag Example

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Example of a primary release tag format for RKE2.

```bash
v1.23.5+rke2r1
```

--------------------------------

### Example Output of Configuration Generation

Source: https://github.com/rancher/rke2/blob/master/contrib/custom-image-kubelet/README.md

Illustrates the expected output when the genconfig.py script runs, showing the creation of configuration files and extraction of binaries.

```text
I Got Release: kubernetes-1-18-9
I Writing HelmChartConfig to /var/lib/rancher/rke2/server/manifests/rke2-kube-proxy-config.yaml
I Writing HelmChartConfig to /var/lib/rancher/rke2/server/manifests/rke2-coredns-config.yaml
I Writing HelmChartConfig to /var/lib/rancher/rke2/server/manifests/rke2-metrics-server-config.yaml
I Extracting files from https://X/kubernetes-1-18/releases/1/artifacts/kubernetes/v1.18.9/kubernetes-node-linux-amd64.tar.gz
I Extracting /var/lib/rancher/rke2/opt/bin/kube-proxy
I Extracting /var/lib/rancher/rke2/opt/bin/kubelet
I Extracting /var/lib/rancher/rke2/opt/bin/kubeadm
I Getting auth tokens for ['X'] in us-east-1
I Writing credentials to /etc/rancher/rke2/registries.yaml
I Writing config to /etc/rancher/rke2/config.yaml
```

--------------------------------

### Install vagrant-libvirt Plugin

Source: https://github.com/rancher/rke2/blob/master/tests/install/README.md

Installs the necessary vagrant-libvirt plugin for using the libvirt provider.

```shell
vagrant plugin install vagrant-libvirt
```

--------------------------------

### Basic RKE2 Server Configuration File

Source: https://github.com/rancher/rke2/blob/master/README.md

An example YAML configuration file for an RKE2 server. This sets file permissions for the kubeconfig, adds a TLS SAN, and defines node labels. Ensure the file path is correct for your RKE2 installation.

```yaml
# /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
tls-san:
  - "foo.local"
node-label:
  - "foo=bar"
  - "something=amazing"
```

--------------------------------

### RKE2 Server Configuration using CLI Arguments

Source: https://github.com/rancher/rke2/blob/master/README.md

An equivalent configuration to the YAML example, but using command-line arguments for an RKE2 server. This demonstrates how repeatable arguments like --node-label are handled.

```bash
rke2 server \
  --write-kubeconfig-mode "0644" \
  --tls-san "foo.local" \
  --node-label "foo=bar" \
  --node-label "something=amazing"
```

--------------------------------

### RKE2 Testing RPM Tag Example

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Example of a tag format for RKE2 testing RPMs.

```bash
v1.23.5+rke2r1.testing.0
```

--------------------------------

### RKE2 Stable RPM Tag Example

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Example of a tag format for RKE2 stable RPMs.

```bash
v1.23.5+rke2r1.stable.0
```

--------------------------------

### Install RKE2 and Dependencies on Ubuntu

Source: https://github.com/rancher/rke2/blob/master/contrib/custom-image-kubelet/README.md

Installs a specific RKE2 version, Python 3 virtual environment, and project dependencies. Ensure you activate the virtual environment before proceeding.

```bash
curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=v1.18.13+rke2r1 sh -

sudo apt update
sudo apt install -y python3-venv python3-wheel python3-pip
python3 -m venv ~/python3
. ~/python3/bin/activate

git clone --shallow git@github.com:rancher/rke2.git
cd rke2/contrib/custom-image-kubelet

pip install -r requirements.txt
```

--------------------------------

### Run RKE2 Server

Source: https://github.com/rancher/rke2/blob/master/BUILDING.md

Command to run the RKE2 server locally or on a remote host after installing prerequisites and copying necessary artifacts.

```shell
rke2 server --token=test
```

--------------------------------

### Run Integration Tests

Source: https://github.com/rancher/rke2/blob/master/developer-docs/testing.md

Execute integration tests for RKE2. This command is run after entering a development shell and starting an RKE2 server.

```bash
go test ./tests/ -run Integration
```

--------------------------------

### Cross-Compiling Go Binaries with xx-go

Source: https://github.com/rancher/rke2/blob/master/developer-docs/docker_multiarch.md

Compile Go applications for the target architecture using `go-build-static.sh`. Precede the command with `xx-go --wrap && \` to ensure proper cross-compilation setup, including setting `GOOS` and `GOARCH`.

```dockerfile
RUN xx-go --wrap && \
    GO_LDFLAGS="-linkmode=external -X ${PKG}/coremain.GitCommit=$(git rev-parse --short HEAD)" \
    go-build-static.sh -gcflags=-trimpath=${GOPATH}/src -o bin/coredns .
```

--------------------------------

### Format Date for Hardened Kubernetes Build Tag

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Use this command to get the correct UTC date format for the build tag when creating a new release in the image-build-kubernetes repository.

```bash
TZ=utc date '+-build%Y%m%d'
```

--------------------------------

### Disabling Security Responder via Config

Source: https://github.com/rancher/rke2/blob/master/docs/adrs/010-security-responder.md

Example configuration to disable the security-responder client by adding it to the 'disable' list in the RKE2 configuration file.

```yaml
# /etc/rancher/rke2/config.yaml
disable:
- rke2-security-responder
```

--------------------------------

### Security Responder Data Payload Example

Source: https://github.com/rancher/rke2/blob/master/docs/adrs/010-security-responder.md

This JSON structure represents the data payload collected by the security-responder client, including cluster metadata and version information.

```json
{
  "appVersion": "v1.31.6+rke2r1",
  "extraTagInfo": {
    "kubernetesVersion": "v1.31.6",
    "clusteruuid": "53741f60-f208-48fc-ae81-8a969510a598"
  },
  "extraFieldInfo": {
    "serverNodeCount": 3,
    "agentNodeCount": 2,
    "cni-plugin": "flannel",
    "ingress-controller": "rke2-ingress-nginx",
    "os": "ubuntu",
    "selinux": "enabled"
  }
}
```

--------------------------------

### Enter Development Shell

Source: https://github.com/rancher/rke2/blob/master/BUILDING.md

This make target spins up a privileged container with the environment set up for invoking `./bin/rke2`.

```shell
make dev-shell
```

--------------------------------

### Compare Server and Agent Arguments

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Download RKE2 binaries for different versions, verify their versions, and then diff their server and agent arguments. This helps identify changes in command-line options.

```bash
curl https://github.com/rancher/rke2/releases/download/v1.23.5+rke2r2/rke2.linux-amd64 -L -o rke2-r1
curl https://github.com/rancher/rke2/releases/download/v1.21.5+rke2r1/rke2.linux-amd64 -L -o rke2-r2
chmod u+x rke2-r*

# verify versions
./rke2-r1 --version
./rke2-r2 --version

# output args to a file
./rke2-r1 server --help >r1 && ./rke2-r1 agent --help >>r1args
./rke2-r2 server --help >r2 && ./rke2-r2 agent --help >>r2args

# diff arguments
diff -y --suppress-common-lines r1args r2args
```

--------------------------------

### Default FROM Instruction Behavior

Source: https://github.com/rancher/rke2/blob/master/developer-docs/docker_multiarch.md

Contrast with using a standard `FROM` instruction without `--platform=$BUILDPLATFORM`. This approach leads to emulation for each target platform, resulting in slower build times.

```dockerfile
FROM ${GO_IMAGE} as base-builder
```

--------------------------------

### Run All E2E Tests

Source: https://github.com/rancher/rke2/blob/master/developer-docs/testing.md

Execute all E2E tests using the Go test command. The timeout flag is recommended as the default is 10 minutes.

```bash
go test -timeout=15m ./tests/e2e/...

```

--------------------------------

### Build RKE2 using Make

Source: https://github.com/rancher/rke2/blob/master/BUILDING.md

This command builds RKE2 inside a Docker container. Use `make build` to use host-local tooling instead.

```shell
make
```

--------------------------------

### Using BUILDPLATFORM for Native Execution

Source: https://github.com/rancher/rke2/blob/master/developer-docs/docker_multiarch.md

Force Dockerfile instructions to run on the native architecture of the runner by specifying `--platform=$BUILDPLATFORM` in the `FROM` instruction. This is useful for stages where emulation is not required.

```dockerfile
FROM --platform=$BUILDPLATFORM ${GO_IMAGE} as base-builder
```

--------------------------------

### Configure Kubectl for RKE2

Source: https://github.com/rancher/rke2/blob/master/BUILDING.md

Export KUBECONFIG and PATH environment variables to use kubectl with a running RKE2 instance.

```shell
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
export PATH=/var/lib/rancher/rke2/bin:$PATH
```

--------------------------------

### Generate Gotests Templates

Source: https://github.com/rancher/rke2/blob/master/developer-docs/testing.md

Command to generate RKE2 unit tests using custom templates. Specify the path to the RKE2 repository's contrib directory.

```bash
gotests --template_dir=<PATH_TO_RKE2>/contrib/gotests_templates
```

--------------------------------

### Generate RKE2 Configuration Locally

Source: https://github.com/rancher/rke2/blob/master/contrib/custom-image-kubelet/README.md

Generates RKE2 configuration files and extracts binaries to a specified local directory. This is useful for embedding into deployment pipelines or copying to multiple hosts.

```bash
./genconfig.py --prefix ./kubernetes-1-18/ --release-url https://X/kubernetes-1-18/kubernetes-1-18.yaml
```

--------------------------------

### Run All E2E Tests and Generate JUnit Report

Source: https://github.com/rancher/rke2/blob/master/developer-docs/testing.md

Execute all E2E tests using the Ginkgo CLI and generate a JUnit XML report. Ginkgo has a default timeout of 1 hour, so no timeout flag is needed.

```bash
ginkgo --junit-report=result.xml ./tests/e2e/...

```

--------------------------------

### Generate RKE2 Configuration and Override Binaries

Source: https://github.com/rancher/rke2/blob/master/contrib/custom-image-kubelet/README.md

Generates RKE2 configuration files and extracts necessary binaries from a specified Kubernetes release URL. This command is intended to be run on the target RKE2 host.

```bash
sudo ~/python3/bin/python genconfig.py --release-url https://X/kubernetes-1-18/kubernetes-1-18.yaml

systemctl start rke2-server
```

--------------------------------

### Create RKE2 Release Candidate Tag

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Create a new release candidate tag in the RKE2 repository. This tag should follow the format 'vX.Y.Z-rcN+rke2rM' and be created against the correct release branch (e.g., 'release-X.Y'). Ensure 'Set as a pre-release' is selected and 'Set as the latest release' is NOT selected.

```git
v1.23.5-rc1+rke2r1
```

--------------------------------

### Run Unit Tests

Source: https://github.com/rancher/rke2/blob/master/developer-docs/testing.md

Execute unit tests for RKE2 packages. This command filters tests to include only those marked with 'Unit'.

```bash
go test ./pkg/... -run Unit
```

--------------------------------

### Prime Flag Logic in Helm Templates

Source: https://github.com/rancher/rke2/blob/master/docs/adrs/012-prime-flag-and-ingress-nginx.md

This code demonstrates the logic within the Helm templates for setting the ingress-nginx image tag when the 'prime' flag is enabled. It shows how the 'primeTag' is determined.

```go-template
{{- if .Values.prime }}
  {{ .Values.primeTag | default .Values.tag }}
{{- else }}
  {{ .Values.tag }}
{{- end -}}
```

--------------------------------

### YAML Anchors for Chart Definitions

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Demonstrates using YAML anchors to reference and extend chart definitions in `channels-rke2.yaml`. This is useful for managing chart versions across different RKE2 releases.

```yaml
- version: v1.23.5+rke2r1
minChannelServerVersion: v2.6.0-alpha1
maxChannelServerVersion: v2.6.99
...
charts: &charts-v1
rke2-cilium:
    repo: rancher-rke2-charts
    version: 1.9.808
```

```yaml
- version: v1.21.5+rke2r1
minChannelServerVersion: v2.6.0-alpha1
maxChannelServerVersion: v2.6.99
...
charts: *charts-v1
```

```yaml
- version: v1.21.5+rke2r1
minChannelServerVersion: v2.6.0-alpha1
maxChannelServerVersion: v2.6.99
...
charts: &charts-v2
<<: *charts-v1
harvester-cloud-provider:
    repo: rancher-rke2-charts
    version: 0.1.200
```

--------------------------------

### Run Specific E2E Test Suite

Source: https://github.com/rancher/rke2/blob/master/developer-docs/testing.md

Execute a specific E2E test suite, such as validatecluster. The timeout flag is recommended.

```bash
go test -timeout=15m ./tests/e2e/validatecluster/...

```

--------------------------------

### Strip Binaries in Dockerfile

Source: https://github.com/rancher/rke2/blob/master/developer-docs/docker_multiarch.md

Use this command to strip binaries and reduce final image size. Ensure it runs on the target platform, not the build platform, to avoid issues with cross-compiled binaries.

```Dockerfile
RUN install -s bin/* /usr/local/bin
```

```Dockerfile
FROM ${GO_IMAGE} as strip_binary
#strip needs to run on TARGETPLATFORM, not BUILDPLATFORM
COPY --from=coredns-builder /usr/local/bin/coredns /coredns
RUN strip /coredns
```

--------------------------------

### RKE2 Release Notification Template

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Template for notifying QA about a new RKE2 release candidate, including links to KDM, RC, and RPMs.

```bash
RKE2 <tag> is now available for testing! @k3s-rke2-qa
KDM PRs:
<links to relevant KDM PRs>
RC:
<links to relevant rcs>
RPM:
<links to relevant rpms>
```

--------------------------------

### Update Air-Gap Image Versions for Core RKE2 Components

Source: https://github.com/rancher/rke2/blob/master/developer-docs/updating_rke2_charts.md

Update image versions for core RKE2 components like coredns, cluster-autoscaler, etc., in the scripts/build-images file for air-gap scenarios.

```shell
xargs -n1 -t docker image pull --quiet << EOF >> build/images-core.txt
    ${REGISTRY}/rancher/hardened-kubernetes:${KUBERNETES_IMAGE_TAG}
    ${REGISTRY}/rancher/hardened-coredns:v1.8.3-build20210720
    ${REGISTRY}/rancher/hardened-cluster-autoscaler:v1.8.3-build20210729
    ${REGISTRY}/rancher/hardened-dns-node-cache:1.20.0-build20210803
    ${REGISTRY}/rancher/hardened-etcd:${ETCD_VERSION}-build20220413
    ${REGISTRY}/rancher/hardened-k8s-metrics-server:v0.5.0-build20210915
    ${REGISTRY}/rancher/klipper-helm:v0.6.1-build20210616
    ${REGISTRY}/rancher/mirrored-pause:${PAUSE_VERSION}
    ${REGISTRY}/rancher/mirrored-jettech-kube-webhook-certgen:v1.5.1
    ${REGISTRY}/rancher/nginx-ingress-controller:nginx-0.47.0-hardened1
    ${REGISTRY}/rancher/rke2-cloud-provider:${CCM_VERSION}
EOF
```

--------------------------------

### Compare Chart Differences

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Use `git diff` to compare chart changes between two RKE2 versions. Fetch tags from upstream before running the diff.

```bash
git fetch upstream --tags
git diff v1.23.5+rke2r2..v1.21.5+rke2r1 -- Dockerfile
```

--------------------------------

### Update Air-Gap Image Versions for Calico

Source: https://github.com/rancher/rke2/blob/master/developer-docs/updating_rke2_charts.md

Update image versions for Calico components in the scripts/build-images file for air-gap scenarios.

```shell
xargs -n1 -t docker image pull --quiet << EOF > build/images-calico.txt
    ${REGISTRY}/rancher/mirrored-calico-operator:v1.17.6
    ${REGISTRY}/rancher/mirrored-calico-ctl:v3.19.2
    ${REGISTRY}/rancher/mirrored-calico-kube-controllers:v3.19.2
    ${REGISTRY}/rancher/mirrored-calico-typha:v3.19.2
    ${REGISTRY}/rancher/mirrored-calico-node:v3.19.2
    ${REGISTRY}/rancher/mirrored-calico-pod2daemon-flexvol:v3.19.2
    ${REGISTRY}/rancher/mirrored-calico-cni:v3.19.2
EOF
```

--------------------------------

### Switch CNI to Flannel in RKE2 E2E Tests

Source: https://github.com/rancher/rke2/blob/master/tests/e2e/mixedos/README.md

Set the E2E_CNI environment variable to 'flannel' before running the go test command to use Flannel as the CNI plugin instead of the default Calico.

```bash
E2E_CNI=flannel go test -v -timeout=30m tests/e2e/mixedos/mixedos_test.go
```

--------------------------------

### Setting Target Platform Argument

Source: https://github.com/rancher/rke2/blob/master/developer-docs/docker_multiarch.md

Declare the `TARGETPLATFORM` argument in a Dockerfile. Docker automatically sets this variable when building multi-arch images, creating separate build processes for each target platform.

```dockerfile
ARG TARGETPLATFORM
```

--------------------------------

### Ingress-Nginx Helm Chart Image Tag Logic

Source: https://github.com/rancher/rke2/blob/master/docs/adrs/012-prime-flag-and-ingress-nginx.md

This snippet illustrates how the ingress-nginx Helm chart selects image tags based on the 'prime' flag. The 'tag' field is for community, while 'primeTag' is for customers.

```yaml
tag: "v1.14.5-hardenedN"
primeTag: "v1.14.5-primeN"
```

--------------------------------

### Update Chart and CRD Versions in Rancher KDM for Calico

Source: https://github.com/rancher/rke2/blob/master/developer-docs/updating_rke2_charts.md

Update both the main chart and its CRD chart versions in Rancher's kontainer-driver-metadata (KDM) for charts like Calico.

```yaml
      rke2-calico:
        repo: rancher-rke2-charts
        version: v3.19.2-201
      rke2-calico-crd:
        repo: rancher-rke2-charts
        version: v1.0.101
```

--------------------------------

### Update Kubernetes Version in version.sh

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Update the KUBERNETES_VERSION variable in the version.sh script to reflect the new Kubernetes version. This is crucial for RKE2 to track the correct Kubernetes release.

```bash
KUBERNETES_VERSION=${KUBERNETES_VERSION:-v1.23.5}
```

--------------------------------

### Update Kubernetes Image Tag in version.sh

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

Update the KUBERNETES_IMAGE_TAG variable in the version.sh script to the new image tag. This ensures RKE2 uses the correct hardened Kubernetes image.

```bash
KUBERNETES_IMAGE_TAG=${KUBERNETES_IMAGE_TAG:-v1.21.13-rke2r1-build20220525}
```

--------------------------------

### Update Chart Version in Rancher KDM for CoreDNS

Source: https://github.com/rancher/rke2/blob/master/developer-docs/updating_rke2_charts.md

Update the chart version in Rancher's kontainer-driver-metadata (KDM) for charts like CoreDNS.

```yaml
      rke2-coredns:
        repo: rancher-rke2-charts
        version: 1.10.101-build2021022304
```

--------------------------------

### Update kube-proxy Chart Version in Dockerfile

Source: https://github.com/rancher/rke2/blob/master/developer-docs/upgrading_kubernetes.md

For Kubernetes v1.21, update the CHART_FILE variable in the Dockerfile to reference the correct RKE2 Kubernetes build version for the kube-proxy chart.

```dockerfile
RUN CHART_VERSION="v1.21.10-build2021041301"     CHART_FILE=/charts/rke2-kube-proxy.yaml
```

--------------------------------

### Update CNI Plugin Version in Dockerfile.windows

Source: https://github.com/rancher/rke2/blob/master/developer-docs/updating_rke2_charts.md

Update the CNI plugin version variable (e.g., CALICO_VERSION) in Dockerfile.windows for CNI plugins.

```shell
ENV CALICO_VERSION="v3.19.2"
```

--------------------------------

### Update RKE2 Chart Version in Dockerfile

Source: https://github.com/rancher/rke2/blob/master/developer-docs/updating_rke2_charts.md

Modify the CHART_VERSION variable in the RKE2 Dockerfile to specify the new chart version.

```shell
RUN CHART_VERSION="v3.19.1-105"               CHART_FILE=/charts/rke2-calico.yaml         CHART_BOOTSTRAP=true   /charts/build-chart.sh
```

```shell
RUN CHART_VERSION="v3.19.2-201"               CHART_FILE=/charts/rke2-calico.yaml         CHART_BOOTSTRAP=true   /charts/build-chart.sh
```