### K3s Installation with Config File and Flags

Source: https://docs.k3s.io/installation/configuration

Install K3s using a configuration file and the installation script, combining environment variables and CLI arguments for a complete setup.

```yaml
token: "secret"  
debug: true  
```

```bash
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC="server" sh -s - --flannel-backend none  
```

--------------------------------

### Install K3s, Skipping Auto-Start and Enablement

Source: https://docs.k3s.io/advanced

Use this command to download and install K3s without automatically starting the service or enabling it to start on boot. This is useful for custom startup configurations.

```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START=true INSTALL_K3S_SKIP_ENABLE=true sh -
```

--------------------------------

### Install and Start Nginx System Service

Source: https://docs.k3s.io/datastore/cluster-loadbalancer

This command copies the Nginx configuration to the system directory and starts the Nginx service. This method is used when Nginx is installed directly on the host.

```bash
cp nginx.conf /etc/nginx/nginx.conf
systemctl start nginx
```

--------------------------------

### Install NVIDIA Container Runtime Packages

Source: https://docs.k3s.io/advanced

Example command to install NVIDIA container runtime packages on a Debian-based system. Ensure the NVIDIA repository is added first.

```bash
apt install -y nvidia-container-runtime cuda-drivers-fabricmanager-515 nvidia-headless-515-server

```

--------------------------------

### Install K3s Server via Curl with Arguments

Source: https://docs.k3s.io/installation/airgap

Install K3s server using the curl command, passing arguments for token and datastore endpoint. This is an alternative to using the local install script.

```bash
curl -sfL https://get.k3s.io | sh -s - server \
  --token=SECRET \
  --datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name"
```

--------------------------------

### Install K3s on Single Server

Source: https://docs.k3s.io/installation/airgap?_highlight=.cache.json&airgap-load-images=Manually+Deploy+Images

Run the install script locally on the server node, skipping the download of the K3s binary as it has already been placed in /usr/local/bin.

```bash
INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh
```

--------------------------------

### Install K3s Server

Source: https://docs.k3s.io/quick-start

Run this command to install K3s as a service on a server node. The service will automatically restart on reboots or crashes. Additional utilities like kubectl are also installed.

```bash
curl -sfL https://get.k3s.io | sh -
```

--------------------------------

### Download New K3s Version Without Starting

Source: https://docs.k3s.io/upgrades/manual

Use the `INSTALL_K3S_SKIP_START=true` environment variable with the installation script to download the new K3s binary without automatically starting the service.

```shell
curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START=true <EXISTING_K3S_ENV> sh -s - <EXISTING_K3S_ARGS>
```

--------------------------------

### Download K3s Install Script

Source: https://docs.k3s.io/installation/airgap?_highlight=.cache.json&airgap-load-images=Manually+Deploy+Images

Download the K3s install script to install.sh and make it executable.

```bash
curl -Lo install.sh https://get.k3s.io
```

```bash
chmod +x install.sh
```

--------------------------------

### Install Previous K3s Version (Server)

Source: https://docs.k3s.io/upgrades/roll-back

Roll back the K3s binary to a specific previous version for server nodes. Use INSTALL_K3S_SKIP_START="true" to prevent K3s from starting immediately after installation.

```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Zk3s1 INSTALL_K3S_EXEC="server" INSTALL_K3S_SKIP_START="true" sh -
```

--------------------------------

### Install K3s Server with External Datastore

Source: https://docs.k3s.io/installation/airgap?_highlight=.cache.json&airgap-load-images=Manually+Deploy+Images

Modify the installation command to use local script execution and specify arguments for the server, token, and datastore endpoint.

```bash
INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC='server --token=SECRET' \
K3S_DATASTORE_ENDPOINT='mysql://username:password@tcp(hostname:3306)/database-name' \
./install.sh
```

--------------------------------

### Download and Install K3s Binary

Source: https://docs.k3s.io/installation/configuration

Download the K3s binary from GitHub releases and make it executable. This is useful for quick tests and not recommended for permanent installations.

```shell
curl -Lo /usr/local/bin/k3s https://github.com/k3s-io/k3s/releases/download/v1.26.5+k3s1/k3s; chmod a+x /usr/local/bin/k3s
```

--------------------------------

### Start K3s Server with Tailscale

Source: https://docs.k3s.io/networking/distributed-multicloud

Command to start a K3s server with Tailscale integration enabled. Requires a token, join key, and the server's Tailscale IP.

```bash
k3s server --token <token> --vpn-auth="name=tailscale,joinKey=<joinKey>" --node-external-ip=<TailscaleIPOfServerNode>
```

--------------------------------

### Example Kubelet Arguments from Logs

Source: https://docs.k3s.io/security/self-assessment-1.7

This is an example of the output from the audit command, showing the arguments passed to the Kubelet. It is used to verify the presence or absence of specific arguments like RotateKubeletServerCertificate.

```text
Aug 09 19:01:30 server-0 k3s[2354]: time="2024-08-09T19:01:30Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --event-qps=0 --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=server-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --make-iptables-util-chains=true --node-ip=10.10.10.100 --node-labels= --pod-infra-container-image=rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --protect-kernel-defaults=true --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --streaming-connection-idle-timeout=5m --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
```

--------------------------------

### Start K3s Service

Source: https://docs.k3s.io/cli/etcd-snapshot

Use this command to start the K3s service after a snapshot restore.

```bash
systemctl start k3s
```

--------------------------------

### Kubelet Configuration Example (config.yaml)

Source: https://docs.k3s.io/security/self-assessment-1.7

Example of how anonymous-auth might be incorrectly configured in the K3s config.yaml file. This should be removed or set to false for security.

```yaml
kubelet-arg:
  - "anonymous-auth=true"
```

--------------------------------

### K3s kube-apiserver Log Output Example

Source: https://docs.k3s.io/security/self-assessment-1.7

An example of the kube-apiserver command line arguments as logged by K3s, showing the --client-ca-file configuration.

```log
Aug 09 19:01:28 server-0 k3s[2354]: time="2024-08-09T19:01:28Z" level=info msg="Running kube-apiserver --admission-control-config-file=/var/lib/rancher/k3s/server/psa.yaml --advertise-address=10.10.10.100 --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --audit-log-maxage=30 --audit-log-maxbackup=10 --audit-log-maxsize=100 --audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log --audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --encryption-provider-config=/var/lib/rancher/k3s/server/cred/encryption-config.json --etcd-cafile=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/k3s/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/k3s/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --feature-gates=JobTrackingWithFinalizers=true --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"  
```

--------------------------------

### Install Previous K3s Version (Agent)

Source: https://docs.k3s.io/upgrades/roll-back

Roll back the K3s binary to a specific previous version for agent nodes. Use INSTALL_K3S_SKIP_START="true" to prevent K3s from starting immediately after installation.

```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Zk3s1 INSTALL_K3S_EXEC="agent" INSTALL_K3S_SKIP_START="true" sh -
```

--------------------------------

### Install Tailscale

Source: https://docs.k3s.io/networking/distributed-multicloud

Install Tailscale on your nodes using this curl command. This is a prerequisite for integrating K3s with Tailscale.

```bash
curl -fsSL https://tailscale.com/install.sh | sh
```

--------------------------------

### Install K3s Server with Custom Configuration

Source: https://docs.k3s.io/installation/configuration

Use the installation script with environment variables and command flags to configure K3s server. These options are persisted to the service configuration.

```shell
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - --flannel-backend none --token 12345
```

```shell
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --flannel-backend none" K3S_TOKEN=12345 sh -s -
```

```shell
curl -sfL https://get.k3s.io | K3S_TOKEN=12345 sh -s - server --flannel-backend none
```

```shell
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--flannel-backend none --token 12345" sh -s -
```

```shell
curl -sfL https://get.k3s.io | sh -s - --flannel-backend none --token 12345
```

--------------------------------

### Install K3s with Docker Runtime

Source: https://docs.k3s.io/advanced

Install K3s using the provided script with the --docker option to utilize Docker as the container runtime instead of containerd.

```bash
curl -sfL https://get.k3s.io | sh -s - --docker
```

--------------------------------

### Install HAProxy and KeepAlived

Source: https://docs.k3s.io/datastore/cluster-loadbalancer

Installs HAProxy and KeepAlived packages on load balancer nodes using apt-get.

```bash
sudo apt-get install haproxy keepalived  

```

--------------------------------

### Install Docker on K3s Node

Source: https://docs.k3s.io/advanced

Use this script to install Docker on a K3s node. Ensure Docker is installed before proceeding with K3s installation using the --docker flag.

```bash
curl https://releases.rancher.com/install-docker/20.10.sh | sh
```

--------------------------------

### Install Sysbox Binaries

Source: https://docs.k3s.io/blog/2025/09/27/k3s-sysbox

Install the built sysbox binaries to the system's executable path. This command is used when building on a different machine than where containerd will run.

```bash
make install
```

--------------------------------

### Install K3s Agent Node

Source: https://docs.k3s.io/installation/airgap?_highlight=.cache.json&airgap-load-images=Manually+Deploy+Images

On each agent node, run the install script locally, skipping the download. Specify the server URL and token for connection.

```bash
INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://<SERVER_IP>:6443 K3S_TOKEN=<YOUR_TOKEN> ./install.sh
```

--------------------------------

### Install HAProxy on Load Balancer

Source: https://docs.k3s.io/blog/2025/03/10/simple-ha

Installs haproxy on the load balancer node. This is a prerequisite for distributing traffic to K3s servers.

```bash
sudo apt install haproxy
```

--------------------------------

### Install K3s Server with External MySQL Datastore

Source: https://docs.k3s.io/datastore/ha

Use this command to install the K3s server with a MySQL database as the external datastore. Ensure the datastore endpoint is correctly formatted for your database type. The `--token` parameter sets a deterministic token, and `--tls-san` is optional for fixed registration addresses.

```bash
curl -sfL https://get.k3s.io | sh -s - server \
  --token=SECRET \
  --datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name" \
  --tls-san=<FIXED_IP> # Optional, needed if using a fixed registration address

```

--------------------------------

### Run Sysbox Binary

Source: https://docs.k3s.io/blog/2025/09/27/k3s-sysbox

Execute the sysbox binary to start the sysbox runtime.

```bash
sysbox
```

--------------------------------

### Download and Install SELinux RPM

Source: https://docs.k3s.io/installation/airgap?_highlight=.cache.json&airgap-load-images=Manually+Deploy+Images

Manually install the k3s-selinux RPM on air-gapped nodes with SELinux enabled. Ensure dependencies like container-selinux, policycoreutils, and selinux-policy are available.

```bash
# On internet accessible machine:
curl -LO https://github.com/k3s-io/k3s-selinux/releases/download/v1.6.stable.1/k3s-selinux-1.6-1.el8.noarch.rpm
```

```bash
# Transfer RPM to air-gapped machine
sudo yum install ./k3s-selinux-1.6-1.el8.noarch.rpm
```

--------------------------------

### Example K3s API Server Configuration Output

Source: https://docs.k3s.io/security/self-assessment-1.7

This is an example of the output from the audit command, showing the full arguments passed to the kube-apiserver. It is used to verify the presence or absence of specific admission plugins.

```text
Aug 09 19:01:28 server-0 k3s[2354]: time="2024-08-09T19:01:28Z" level=info msg="Running kube-apiserver --admission-control-config-file=/var/lib/rancher/k3s/server/psa.yaml --advertise-address=10.10.10.100 --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --audit-log-maxage=30 --audit-log-maxbackup=10 --audit-log-maxsize=100 --audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log --audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --encryption-provider-config=/var/lib/rancher/k3s/server/cred/encryption-config.json --etcd-cafile=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/k3s/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/k3s/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --feature-gates=JobTrackingWithFinalizers=true --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key
```

--------------------------------

### Example Pod Security Policy (PSP) Configuration

Source: https://docs.k3s.io/security/hardening-guide

This is an example of a compliant PSP configuration for K3s v1.24 and older, adhering to CIS Benchmark section 5.2. It restricts privileged operations, escalations, and host access, while defining allowed volume types and user/group ID rules.

```yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted-psp
spec:
  privileged: false                # CIS - 5.2.1
  allowPrivilegeEscalation: false  # CIS - 5.2.5
  requiredDropCapabilities:        # CIS - 5.2.7/8/9
    - ALL
  volumes:
    - 'configMap'
    - 'emptyDir'
    - 'projected'
    - 'secret'
    - 'downwardAPI'
    - 'csi'
    - 'persistentVolumeClaim'
    - 'ephemeral'
  hostNetwork: false               # CIS - 5.2.4
  hostIPC: false                   # CIS - 5.2.3
  hostPID: false                   # CIS - 5.2.2
  runAsUser:
    rule: 'MustRunAsNonRoot'       # CIS - 5.2.6
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
  readOnlyRootFilesystem: false
```

--------------------------------

### Install K3s with Latest Channel

Source: https://docs.k3s.io/reference/env-variables

Use this command to install the latest available K3s version by setting the INSTALL_K3S_CHANNEL environment variable. This overrides the default 'stable' channel.

```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=latest sh -
```

--------------------------------

### Install etcdctl CLI

Source: https://docs.k3s.io/advanced

Download and install the etcdctl command-line interface to interact with K3s's embedded etcd. This script downloads a specific version and extracts it to /usr/local/bin.

```bash
ETCD_VERSION="v3.5.5"
ETCD_URL="https://github.com/etcd-io/etcd/releases/download/${ETCD_VERSION}/etcd-${ETCD_VERSION}-linux-amd64.tar.gz"
curl -sL ${ETCD_URL} | sudo tar -zxv --strip-components=1 -C /usr/local/bin
```

--------------------------------

### K3s API Server Configuration Example

Source: https://docs.k3s.io/security/self-assessment-1.7

This snippet shows an example of how the '--token-auth-file' parameter might be configured in the K3s config file. If this check fails, it indicates that token-based authentication is enabled, which is not recommended for enhanced security.

```yaml
kube-apiserver-arg:
  - "token-auth-file=<path>"
```

--------------------------------

### Install K3s Agent with Custom Configuration

Source: https://docs.k3s.io/installation/configuration

Configure K3s agent during installation using environment variables and command flags. Ensure server URL and token are correctly provided.

```shell
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent --server https://k3s.example.com --token mypassword" sh -s -
```

```shell
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent" K3S_TOKEN="mypassword" sh -s - --server https://k3s.example.com
```

```shell
curl -sfL https://get.k3s.io | K3S_URL=https://k3s.example.com sh -s - agent --token mypassword
```

```shell
curl -sfL https://get.k3s.io | K3S_URL=https://k3s.example.com K3S_TOKEN=mypassword sh -s - # agent is assumed because of K3S_URL
```

--------------------------------

### Example K3s API Server Configuration (to be removed)

Source: https://docs.k3s.io/security/self-assessment-1.7

This snippet shows an example of how the '--disable-admission-plugins' argument might be configured in the K3s config file. If this check fails, such lines should be removed or modified.

```yaml
kube-apiserver-arg:
  - "disable-admission-plugins=...,NamespaceLifecycle,..."
```

--------------------------------

### K3s API Server Configuration Example

Source: https://docs.k3s.io/security/self-assessment-1.7

This snippet shows an example of how the profiling argument might be incorrectly set to true in the K3s configuration file. This should be removed or set to false for security.

```yaml
kube-apiserver-arg:
  - "profiling=true"
```

--------------------------------

### Example Pod Security Admission (PSA) Configuration

Source: https://docs.k3s.io/security/hardening-guide

This is an example of a compliant PSA configuration file (`psa.yaml`) for K3s v1.25 and newer. It sets default policies to 'restricted' and exempts specific namespaces.

```yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
  configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1beta1
    kind: PodSecurityConfiguration
    defaults:
      enforce: "restricted"
      enforce-version: "latest"
      audit: "restricted"
      audit-version: "latest"
      warn: "restricted"
      warn-version: "latest"
    exemptions:
      usernames: []
      runtimeClasses: []
      namespaces: [kube-system, compliance-operator-system]
```

--------------------------------

### Start K3s Server with systemd

Source: https://docs.k3s.io/upgrades/killall

Use this command to restart the K3s server service when using systemd.

```bash
sudo systemctl start k3s
```

--------------------------------

### K3s Server Execution with Config and Flags

Source: https://docs.k3s.io/installation/configuration

Run the K3s server binary with specific environment variables and CLI flags, demonstrating how to combine different configuration methods.

```bash
K3S_KUBECONFIG_MODE="644" k3s server --flannel-backend none  
```

--------------------------------

### Disable Traefik AddOn in K3s

Source: https://docs.k3s.io/installation/packaged-components

Use the `--disable` flag when starting K3s to prevent Traefik from being installed or to uninstall it from an existing cluster. This flag also removes the manifest from the directory.

```bash
k3s --disable=traefik
```

--------------------------------

### Configure K3s Server with Binary

Source: https://docs.k3s.io/installation/configuration

Run the K3s server binary and pass configuration using environment variables or command flags. Ensure critical flags match across server nodes.

```shell
K3S_KUBECONFIG_MODE="644" k3s server
```

```shell
k3s server --write-kubeconfig-mode=644
```

--------------------------------

### Kubelet Configuration via Config Directory Flag

Source: https://docs.k3s.io/installation/configuration

Configure Kubelet by pointing to a directory containing configuration files using the `--kubelet-arg=config-dir` flag. Drop-in files in this directory will be loaded.

```bash
k3s server --kubelet-arg=config-dir=/etc/rancher/k3s/kubelet.conf.d
```

--------------------------------

### Install Kernel Modules Extra on RHEL 10

Source: https://docs.k3s.io/installation/requirements?_highlight=nm&_highlight=cloud&_highlight=setup.service&os=rhel

Installs the required 'kernel-modules-extra' package on RHEL 10 systems for K3s compatibility.

```bash
sudo dnf install -y kernel-modules-extra
```

--------------------------------

### Start Rootless K3s Server in a Systemd Scope

Source: https://docs.k3s.io/advanced

Use this command to wrap the rootless K3s server startup in a systemd scope, which allows for cgroup v2 delegation required for rootless mode when running from a terminal. This is an alternative to running directly in a terminal.

```bash
systemd-run --user -p Delegate=yes --tty k3s server --rootless
```

--------------------------------

### Install SELinux RPMs for K3s

Source: https://docs.k3s.io/advanced

Installs necessary SELinux policies for K3s on compatible systems. Ensure you have the correct RPMs for your CentOS version.

```bash
yum install -y container-selinux selinux-policy-base
```

```bash
yum install -y https://rpm.rancher.io/k3s/latest/common/centos/9/noarch/k3s-selinux-1.6-1.el9.noarch.rpm
```

--------------------------------

### Install Longhorn

Source: https://docs.k3s.io/add-ons/storage

Apply the Longhorn installation YAML to deploy Longhorn in the 'longhorn-system' namespace. Ensure Longhorn is compatible with your architecture as it does not support ARM32.

```bash
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.8.1/deploy/longhorn.yaml

```

--------------------------------

### Create Image Directory and Copy Example

Source: https://docs.k3s.io/add-ons/import-images

Create the directory for image import and copy a text file containing image names. This is used for online image importing.

```bash
mkdir /var/lib/rancher/k3s/agent/images  

cp example.txt /var/lib/rancher/k3s/agent/images  
```

--------------------------------

### Download K3s Binary

Source: https://docs.k3s.io/installation/airgap?_highlight=.cache.json&airgap-load-images=Manually+Deploy+Images

Download the K3s binary and place it in /usr/local/bin. Ensure the binary is executable.

```bash
sudo curl -Lo /usr/local/bin/k3s https://github.com/k3s-io/k3s/releases/download/v1.33.3%2Bk3s1/k3s
```

```bash
sudo chmod +x /usr/local/bin/k3s
```

--------------------------------

### Install System Upgrade Controller

Source: https://docs.k3s.io/blog/2025/03/10/simple-ha

Installs the system-upgrade-controller and its Custom Resource Definitions (CRDs) using kubectl. This controller manages automated K3s upgrades.

```bash
kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/system-upgrade-controller.yaml
```

```bash
kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/crd.yaml
```

--------------------------------

### Kubelet Command Line Argument Example

Source: https://docs.k3s.io/security/self-assessment-1.7

Example of an incorrect --kubelet-arg for anonymous-auth when using the command line. This argument should be removed or set to false.

```bash
--kubelet-arg="anonymous-auth=true"
```

--------------------------------

### Upgrade K3s to Specific Channel using Install Script

Source: https://docs.k3s.io/upgrades/manual

Use the `INSTALL_K3S_CHANNEL` environment variable to specify a release channel, such as 'latest', when upgrading with the installation script.

```shell
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=latest <EXISTING_K3S_ENV> sh -s - <EXISTING_K3S_ARGS>
```

--------------------------------

### Configure K3s Agent with Binary

Source: https://docs.k3s.io/installation/configuration

Run the K3s agent binary and pass configuration using command flags. This requires specifying the server URL and token.

```shell
k3s agent --server https://k3s.example.com --token mypassword
```

--------------------------------

### Install System Upgrade Controller

Source: https://docs.k3s.io/upgrades/automated

Installs the system-upgrade-controller components, including CRDs, deployment, and configuration. Ensure you are not managing versioning through Rancher if you are not using the Rancher UI.

```bash
kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/crd.yaml -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/system-upgrade-controller.yaml  
```

--------------------------------

### Kubelet Configuration via Config File Flag

Source: https://docs.k3s.io/installation/configuration

Configure Kubelet using a specified configuration file path with the `--kubelet-arg=config` flag. This allows for advanced Kubelet settings.

```bash
k3s server --kubelet-arg=config=/etc/rancher/k3s/kubelet.conf
```

--------------------------------

### Initialize K3s Server with Cluster Init

Source: https://docs.k3s.io/datastore/ha-embedded

Use this command to launch the first server node in an HA K3s cluster. The `--cluster-init` flag enables clustering, and a `K3S_TOKEN` is required for other nodes to join. The `--tls-san` flag is optional but necessary if a fixed registration address is used.

```bash
curl -sfL https://get.k3s.io | K3S_TOKEN=SECRET sh -s - server \
    --cluster-init \
    --tls-san=<FIXED_IP> # Optional, needed if using a fixed registration address  
```

--------------------------------

### Start Dedicated control-plane Node

Source: https://docs.k3s.io/installation/server-roles

Use this command to start a K3s server with only the control-plane role, disabling etcd. This requires an existing etcd node to join.

```bash
curl -fL https://get.k3s.io | sh -s - server --token <token> --disable-etcd --server https://<etcd-only-node>:6443   
```

--------------------------------

### Equivalent K3s Server CLI Arguments

Source: https://docs.k3s.io/installation/configuration

These CLI arguments achieve the same configuration as the example YAML file. Repeatable arguments like `--node-label` can be specified multiple times.

```bash
k3s server \ 
  --write-kubeconfig-mode "0644"    \ 
  --tls-san "foo.local"             \ 
  --node-label "foo=bar"            \ 
  --node-label "something=amazing"  \ 
  --cluster-init  
```

--------------------------------

### Verify Embedded Registry Mirror Startup Logs

Source: https://docs.k3s.io/blog/2025/11/11/strategies-for-large-images

Check the K3s logs for these informational messages to confirm that the distributed registry mirror and P2P node have started successfully.

```log
level=info msg="Starting distributed registry mirror at https://10.11.0.11:6443/v2 for registries [docker.io registry.k8s.io]"
level=info msg="Starting distributed registry P2P node at 10.11.0.11:5001"
```

--------------------------------

### Start Dedicated etcd Node

Source: https://docs.k3s.io/installation/server-roles

Use this command to start a K3s server with only the etcd role, disabling all control-plane components. This node will initialize the cluster and wait for other nodes.

```bash
curl -fL https://get.k3s.io | sh -s - server --cluster-init --disable-apiserver --disable-controller-manager --disable-scheduler  
```

--------------------------------

### K3s Kubernetes Components Startup

Source: https://docs.k3s.io/blog/2025/03/25/K3s-initialization

Logs the startup of core Kubernetes components like the API server, scheduler, and controller manager. These run as goroutines within the K3s process to optimize boot time and resource usage.

```log
Running kube-apiserver  

Running kube-scheduler  

Running kube-controller-manager  

```