### Run K3s Install Script for High Availability Source: https://github.com/k3s-io/docs/blob/main/docs/installation/airgap.md Examples showing how to adapt standard K3s installation commands for air-gapped high availability setups using local scripts. ```bash curl -sfL https://get.k3s.io | sh -s - server \ --token=SECRET \ --datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name" ``` ```bash INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC='server --token=SECRET' \ K3S_DATASTORE_ENDPOINT='mysql://username:password@tcp(hostname:3306)/database-name' \ ./install.sh ``` -------------------------------- ### K3s Installation with Config File and Environment Variables Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md Example of installing K3s using a curl script, setting environment variables for kubeconfig mode, and passing server arguments. ```bash curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC="server" sh -s - --flannel-backend none ``` -------------------------------- ### Kubelet Configuration Example Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.12.md This log snippet shows a typical Kubelet configuration, including the paths for 'tlsCertFile' and 'tlsPrivateKeyFile'. This is useful for verifying the current TLS setup. ```log address: 0.0.0.0 allowedUnsafeSysctls: - net.ipv4.ip_forward - net.ipv6.conf.all.forwarding apiVersion: kubelet.config.k8s.io/v1beta1 authentication: anonymous: enabled: false webhook: cacheTTL: 2m0s enabled: true x509: clientCAFile: /var/lib/rancher/k3s/agent/client-ca.crt authorization: mode: Webhook webhook: cacheAuthorizedTTL: 5m0s cacheUnauthorizedTTL: 30s cgroupDriver: systemd clusterDNS: - 10.43.0.10 clusterDomain: cluster.local containerRuntimeEndpoint: unix:///run/k3s/containerd/containerd.sock cpuManagerReconcilePeriod: 10s crashLoopBackOff: {} evictionHard: imagefs.available: 5% nodefs.available: 5% evictionMinimumReclaim: imagefs.available: 10% nodefs.available: 10% evictionPressureTransitionPeriod: 5m0s failSwapOn: false fileCheckFrequency: 20s healthzBindAddress: 127.0.0.1 httpCheckFrequency: 20s imageMaximumGCAge: 0s imageMinimumGCAge: 2m0s kind: KubeletConfiguration logging: flushFrequency: 5s format: text options: json: infoBufferSize: "0" text: infoBufferSize: "0" verbosity: 0 memorySwap: {} nodeStatusReportFrequency: 5m0s nodeStatusUpdateFrequency: 10s protectKernelDefaults: true resolvConf: /run/systemd/resolve/resolv.conf runtimeRequestTimeout: 2m0s serializeImagePulls: false shutdownGracePeriod: 0s shutdownGracePeriodCriticalPods: 0s staticPodPath: /var/lib/rancher/k3s/agent/pod-manifests streamingConnectionIdleTimeout: 4h0m0s syncFrequency: 1m0s tlsCertFile: /var/lib/rancher/k3s/agent/serving-kubelet.crt tlsPrivateKeyFile: /var/lib/rancher/k3s/agent/serving-kubelet.key volumeStatsAggPeriod: 1m0s ``` -------------------------------- ### Install Docker Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md Use the Rancher installation script to set up Docker on the K3s node. ```bash curl https://releases.rancher.com/install-docker/20.10.sh | sh ``` -------------------------------- ### Install sysbox binaries Source: https://github.com/k3s-io/docs/blob/main/blog/2025-09-27-k3s-sysbox.md Install the built binaries to the system path. ```bash make install ``` -------------------------------- ### Start Local Development Server with Docker Source: https://github.com/k3s-io/docs/blob/main/README.md Launches the K3s documentation website locally using Docker, avoiding the need for yarn installation. It maps the current directory and exposes port 3000. ```bash $ docker run --rm -d --name k3s-docs -e HOME=$PWD -v $PWD:$PWD -w $PWD -u $(id -u) -p 3000:3000 node:22 sh -c 'corepack yarn install --immutable && corepack yarn start -h 0.0.0.0' ``` -------------------------------- ### Install etcdctl CLI Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md Downloads and installs the etcdctl binary to /usr/local/bin. ```bash ETCD_VERSION="v3.5.5" ETCD_URL="https://github.com/etcd-io/etcd/releases/download/${ETCD_VERSION}/etcd-${ETCD_VERSION}-linux-amd64.tar.gz" curl -sL ${ETCD_URL} | sudo tar -zxv --strip-components=1 -C /usr/local/bin ``` -------------------------------- ### Install K3s with Default Settings Source: https://github.com/k3s-io/docs/blob/main/docs/reference/env-variables.md Executes the K3s installation script using the default configuration. ```bash curl -sfL https://get.k3s.io | sh - ``` -------------------------------- ### Install SELinux RPM Source: https://github.com/k3s-io/docs/blob/main/docs/installation/airgap.md Manual installation steps for the k3s-selinux RPM on air-gapped nodes with SELinux enabled. ```bash # On internet accessible machine: curl -LO https://github.com/k3s-io/k3s-selinux/releases/download/v1.6.stable.1/k3s-selinux-1.6-1.el8.noarch.rpm # Transfer RPM to air-gapped machine sudo yum install ./k3s-selinux-1.6-1.el8.noarch.rpm ``` -------------------------------- ### Install K3s with Environment Variables Source: https://github.com/k3s-io/docs/blob/main/docs/reference/env-variables.md Demonstrates passing an environment variable to the installation script to specify a release channel. ```bash curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=latest sh - ``` -------------------------------- ### Download New K3s Version Without Starting Source: https://github.com/k3s-io/docs/blob/main/docs/upgrades/manual.md Use the INSTALL_K3S_SKIP_START=true environment variable with the installation script to download the new K3s binary without automatically starting the service. This allows for manual control over the service restart. ```sh curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START=true sh -s - ``` -------------------------------- ### Install HAProxy Source: https://github.com/k3s-io/docs/blob/main/blog/2025-03-10-simple-ha.md Install the HAProxy load balancer on the designated load balancer node. ```bash sudo apt install haproxy ``` -------------------------------- ### Enable and Start Rootless K3s Service Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md After installing the k3s-rootless.service file, reload the systemd user daemon and enable/start the service. Verify the pods are running using kubectl. ```bash systemctl --user daemon-reload systemctl --user enable --now k3s-rootless KUBECONFIG=~/.kube/k3s.yaml kubectl get pods -A ``` -------------------------------- ### Install Tailscale on nodes Source: https://github.com/k3s-io/docs/blob/main/docs/networking/distributed-multicloud.md This command installs the Tailscale client on your nodes. Ensure you have generated an auth key from your Tailscale account. ```bash curl -fsSL https://tailscale.com/install.sh | sh ``` -------------------------------- ### Download K3s Binary and Install Script Source: https://github.com/k3s-io/docs/blob/main/docs/installation/airgap.md Commands to download the K3s binary and installation script for manual deployment on air-gapped nodes. ```bash sudo curl -Lo /usr/local/bin/k3s https://github.com/k3s-io/k3s/releases/download/v1.33.3%2Bk3s1/k3s sudo chmod +x /usr/local/bin/k3s ``` ```bash curl -Lo install.sh https://get.k3s.io chmod +x install.sh ``` -------------------------------- ### Configure K3s Server via Install Script Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md Demonstrates various ways to pass server configuration, such as disabling flannel and setting a token, using the K3s installation script. ```bash curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - --flannel-backend none --token 12345 curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --flannel-backend none" K3S_TOKEN=12345 sh -s - curl -sfL https://get.k3s.io | K3S_TOKEN=12345 sh -s - server --flannel-backend none # server is assumed below because there is no K3S_URL curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--flannel-backend none --token 12345" sh -s - curl -sfL https://get.k3s.io | sh -s - --flannel-backend none --token 12345 ``` -------------------------------- ### Run K3s Install Script for Single Server Source: https://github.com/k3s-io/docs/blob/main/docs/installation/airgap.md Commands to install K3s on a single server or add agent nodes in an air-gapped environment using the local install script. ```bash INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh ``` ```bash INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://:6443 K3S_TOKEN= ./install.sh ``` -------------------------------- ### Install Dependencies Source: https://github.com/k3s-io/docs/blob/main/README.md Installs project dependencies using Yarn. Run this command in the root of the project. ```bash $ yarn ``` -------------------------------- ### Install NVIDIA Container Runtime Packages Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md Use this command to install the necessary NVIDIA runtime and driver packages on the host node. ```bash apt install -y nvidia-container-runtime cuda-drivers-fabricmanager-515 nvidia-headless-515-server ``` -------------------------------- ### Kubeconfig File Example Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.8.md This is an example of a Kubeconfig file. While not directly showing the rotate-certificates argument, it illustrates the typical structure and paths for certificates used by Kubelet. ```yaml apiVersion: v1 clusters: - cluster: server: https://127.0.0.1:6443 certificate-authority: /var/lib/rancher/k3s/agent/server-ca.crt name: local contexts: - context: cluster: local namespace: default user: user name: Default current-context: Default kind: Config preferences: {} users: - name: user user: client-certificate: /var/lib/rancher/k3s/agent/client-kubelet.crt client-key: /var/lib/rancher/k3s/agent/client-kubelet.key ``` -------------------------------- ### Start Local Development Server Source: https://github.com/k3s-io/docs/blob/main/README.md Starts a local development server for the K3s documentation website. Changes are reflected live without a server restart. This command uses Yarn. ```bash $ yarn start ``` -------------------------------- ### Configure K3s Agent via Install Script Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md Shows equivalent methods for registering a K3s agent using the installation script with different combinations of environment variables and flags. ```bash curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent --server https://k3s.example.com --token mypassword" sh -s - curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent" K3S_TOKEN="mypassword" sh -s - --server https://k3s.example.com curl -sfL https://get.k3s.io | K3S_URL=https://k3s.example.com sh -s - agent --token mypassword curl -sfL https://get.k3s.io | K3S_URL=https://k3s.example.com K3S_TOKEN=mypassword sh -s - # agent is assumed because of K3S_URL ``` -------------------------------- ### Start K3s Service Source: https://github.com/k3s-io/docs/blob/main/docs/cli/etcd-snapshot.md Start the K3s service after a snapshot restore or cluster reset. This command is used for both single-server and multi-server setups after the initial restore. ```bash systemctl start k3s ``` -------------------------------- ### Configure K3s Agent via Binary Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md Shows how to configure and start a K3s agent using command-line flags. ```bash k3s agent --server https://k3s.example.com --token mypassword ``` -------------------------------- ### Disable Auto-start and Enablement Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md Prevents the installation script from automatically starting or enabling the K3s service. ```bash curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START=true INSTALL_K3S_SKIP_ENABLE=true sh - ``` -------------------------------- ### Equivalent CLI Arguments for Server Configuration Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md This demonstrates the command-line arguments that are equivalent to the basic server configuration file example. ```bash k3s server \ --write-kubeconfig-mode "0644" \ --tls-san "foo.local" \ --node-label "foo=bar" \ --node-label "something=amazing" \ --cluster-init ``` -------------------------------- ### Failed Upgrade Pods and Cordoned Nodes Source: https://github.com/k3s-io/docs/blob/main/docs/upgrades/automated.md This example shows the output of `kubectl get pods` in the `system-upgrade` namespace, indicating failed upgrade pods, and `kubectl get nodes`, showing nodes that have been cordoned due to the upgrade failure. ```shell $ kubectl get pods -n system-upgrade NAME READY STATUS RESTARTS AGE apply-k3s-server-on-ip-172-31-0-16-with-7af95590a5af8e8c3-2cdc6 0/1 Error 0 9m25s apply-k3s-server-on-ip-172-31-10-23-with-7af95590a5af8e8c-9xvwg 0/1 Error 0 14m apply-k3s-server-on-ip-172-31-13-213-with-7af95590a5af8e8-8j72v 0/1 Error 0 18m system-upgrade-controller-7c4b84d5d9-kkzr6 1/1 Running 0 20m $ kubectl get nodes NAME STATUS ROLES AGE VERSION ip-172-31-0-16 Ready,SchedulingDisabled control-plane,etcd,master 19h v1.27.4+k3s1 ip-172-31-10-23 Ready,SchedulingDisabled control-plane,etcd,master 19h v1.27.4+k3s1 ip-172-31-13-213 Ready,SchedulingDisabled control-plane,etcd,master 19h v1.27.4+k3s1 ip-172-31-2-13 Ready 19h v1.27.4+k3s1 ``` -------------------------------- ### Configure K3s Server via Binary Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md Demonstrates setting configuration for the K3s server using environment variables or command-line flags. ```bash K3S_KUBECONFIG_MODE="644" k3s server ``` ```bash k3s server --write-kubeconfig-mode=644 ``` -------------------------------- ### Upgrade K3s to a Specific Version using Install Script Source: https://github.com/k3s-io/docs/blob/main/docs/upgrades/manual.md Upgrade K3s to an exact version by specifying the desired version with the INSTALL_K3S_VERSION environment variable when re-running the installation script. Pass any existing environment variables and arguments to maintain your setup. ```sh curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z+k3s1 sh -s - ``` -------------------------------- ### Example K3s Encryption Configuration File Source: https://github.com/k3s-io/docs/blob/main/docs/security/secrets-encryption.md This JSON file is generated at `/var/lib/rancher/k3s/server/cred/encryption-config.json` when K3s is started with `--secrets-encryption`. It specifies the encryption resources and providers. ```json { "kind": "EncryptionConfiguration", "apiVersion": "apiserver.config.k8s.io/v1", "resources": [ { "resources": [ "secrets" ], "providers": [ { "aescbc": { "keys": [ { "name": "aescbckey", "secret": "xxxxxxxxxxxxxxxxxxx" } ] } }, { "identity": {} } ] } ] } ``` -------------------------------- ### Example Kubelet Log Output Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.9.md Sample log output showing the active kubelet configuration parameters. ```log Jul 29 19:36:16 server-0 k3s[2235]: time="2025-07-29T19:36:16Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --event-qps=0 --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=server-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --make-iptables-util-chains=true --node-ip=10.10.10.100 --node-labels= --pod-infra-container-image=rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --protect-kernel-defaults=true --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --streaming-connection-idle-timeout=5m --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key" ``` -------------------------------- ### Kube-Proxy Configuration Log Output Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.9.md Example log output showing the current kube-proxy startup arguments. ```log Jul 29 19:36:16 server-0 k3s[2235]: time="2025-07-29T19:36:16Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=server-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables" ``` -------------------------------- ### K3s API Server Log Entry Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.12.md An example log entry from the K3s API server, showing the various arguments it was started with. This is used to verify the authorization-mode setting. ```log May 26 22:08:48 server-0 k3s[2226]: time="2026-05-26T22:08:48Z" level=info msg="Running kube-apiserver --admission-control-config-file=/var/lib/rancher/k3s/server/psa.yaml --advertise-address=10.10.10.100 --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --audit-log-maxage=30 --audit-log-maxbackup=10 --audit-log-maxsize=100 --audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log --audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --encryption-provider-config=/var/lib/rancher/k3s/server/cred/encryption-config.json --encryption-provider-config-automatic-reload=true --etcd-cafile=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/k3s/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/k3s/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-extend-token-expiration=false --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" ``` -------------------------------- ### Running K3s Server with Environment Variables and Flags Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md Demonstrates how to run the K3s server binary directly, using environment variables for configuration and command-line flags for specific settings. ```bash K3S_KUBECONFIG_MODE="644" k3s server --flannel-backend none ``` -------------------------------- ### Install Rancher Logging via Helm Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md Commands to install the Rancher logging stack on K3s without a full Rancher management installation. ```bash helm repo add rancher-charts https://charts.rancher.io helm repo update helm install --create-namespace -n cattle-logging-system rancher-logging-crd rancher-charts/rancher-logging-crd helm install --create-namespace -n cattle-logging-system rancher-logging --set additionalLoggingSources.k3s.enabled=true rancher-charts/rancher-logging ``` -------------------------------- ### Install Longhorn Source: https://github.com/k3s-io/docs/blob/main/docs/add-ons/storage.md Apply this command to install Longhorn, an open-source distributed block storage system for Kubernetes, into your K3s cluster. Longhorn will be installed in the 'longhorn-system' namespace. ```bash kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.8.1/deploy/longhorn.yaml ``` -------------------------------- ### Install K3s server with external MySQL datastore Source: https://github.com/k3s-io/docs/blob/main/docs/datastore/ha.md Use this command to initialize the first K3s server node with a MySQL database connection string and a cluster token. ```bash curl -sfL https://get.k3s.io | sh -s - server \ --token=SECRET \ --datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name" \ --tls-san= # Optional, needed if using a fixed registration address ``` -------------------------------- ### K3s Kubelet Command Line Configuration Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.7.md This demonstrates how to pass Kubelet arguments, such as 'event-qps' and TLS certificate paths, directly via the command line when starting K3s. This is an alternative to using the config.yaml file. ```bash k3s --kubelet-arg="event-qps=" ``` ```bash k3s --kubelet-arg="tls-cert-file=" --kubelet-arg="tls-private-key-file=" ``` -------------------------------- ### Starting Kubernetes Components Source: https://github.com/k3s-io/docs/blob/main/blog/2025-03-25-K3s-initialization.md Logs showing the startup of core Kubernetes control-plane components within the K3s process. These are run as goroutines for efficiency. ```text Running kube-apiserver Running kube-scheduler Running kube-controller-manager ``` -------------------------------- ### Returned Audit Value Example Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.23.md An example of the log output showing the kube-apiserver configuration. ```log Sep 13 13:26:40 k3s-123-cis-pool3-b403f678-bzdg5 k3s[1600]: time="2022-09-13T13:26:40Z" level=info msg="Running kube-apiserver --advertise-address=172.31.0.140 --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --etcd-cafile=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/k3s/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/k3s/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --feature-gates=JobTrackingWithFinalizers=true --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" ``` -------------------------------- ### Kubelet Log Output Example Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.10.md This log snippet shows the 'Running kubelet' command with various arguments, including '--streaming-connection-idle-timeout' and '--make-iptables-util-chains'. ```log Sep 11 17:22:10 server-0 k3s[2234]: time="2025-09-11T17:22:10Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --event-qps=0 --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=server-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --make-iptables-util-chains=true --node-ip=10.10.10.100 --node-labels= --pod-infra-container-image=rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --protect-kernel-defaults=true --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --streaming-connection-idle-timeout=5m --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key ``` -------------------------------- ### Returned Audit Value Example Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.23.md An example of the full log output returned by the audit command. ```log Sep 13 13:26:40 k3s-123-cis-pool3-b403f678-bzdg5 k3s[1600]: time="2022-09-13T13:26:40Z" level=info msg="Running kube-apiserver --advertise-address=172.31.0.140 --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --etcd-cafile=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/k3s/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/k3s/server/tls/client.key --etcd-servers=https://127.0.0.1:2379 --feature-gates=JobTrackingWithFinalizers=true --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" ``` -------------------------------- ### Manually Install SELinux Policies Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md Installs required SELinux policy packages for K3s on compatible systems. ```bash yum install -y container-selinux selinux-policy-base yum install -y https://rpm.rancher.io/k3s/latest/common/centos/9/noarch/k3s-selinux-1.6-1.el9.noarch.rpm ``` -------------------------------- ### Configure Kubelet Event QPS via Command Line Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.11.md This shows how to set the Kubelet's event QPS argument directly on the command line when starting K3s. Replace with your desired QPS setting. ```bash k3s --kubelet-arg="event-qps=" ``` -------------------------------- ### Upgrade K3s to Stable Release using Install Script Source: https://github.com/k3s-io/docs/blob/main/docs/upgrades/manual.md Re-run the installation script to upgrade K3s to the current stable release. Ensure you pass any existing environment variables and arguments used during the initial installation to maintain configuration. ```sh curl -sfL https://get.k3s.io | sh -s - ``` -------------------------------- ### K3s kubelet log output with --make-iptables-util-chains=true Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.7.md Example log output from K3s showing the kubelet running with the --make-iptables-util-chains argument set to true. ```log Aug 09 19:01:30 server-0 k3s[2354]: time="2024-08-09T19:01:30Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --event-qps=0 --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=server-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --make-iptables-util-chains=true --node-ip=10.10.10.100 --node-labels= --pod-infra-container-image=rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --protect-kernel-defaults=true --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --streaming-connection-idle-timeout=5m --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key ``` -------------------------------- ### K3s API Server Configuration Example Source: https://github.com/k3s-io/docs/blob/main/docs/security/self-assessment-1.12.md This log snippet shows the arguments used when running the K3s API server. It is used to verify that '--token-auth-file' is not present in the configuration. ```log May 26 22:08:48 server-0 k3s[2226]: time="2026-05-26T22:08:48Z" level=info msg="Running kube-apiserver --admission-control-config-file=/var/lib/rancher/k3s/server/psa.yaml --advertise-address=10.10.10.100 --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --audit-log-maxage=30 --audit-log-maxbackup=10 --audit-log-maxsize=100 --audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log --audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --encryption-provider-config=/var/lib/rancher/k3s/server/cred/encryption-config.json --encryption-provider-config-automatic-reload=true --etcd-cafile=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/k3s/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/k3s/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-extend-token-expiration=false --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key ``` -------------------------------- ### Install HAProxy and KeepAlived Source: https://github.com/k3s-io/docs/blob/main/docs/datastore/cluster-loadbalancer.md Installs HAProxy and KeepAlived on load balancer nodes. Ensure these packages are available on your system. ```bash sudo apt-get install haproxy keepalived ``` -------------------------------- ### Install System Upgrade Controller Source: https://github.com/k3s-io/docs/blob/main/docs/upgrades/automated.md Installs the system-upgrade-controller components, including CRDs, deployment, and service account, into your Kubernetes cluster. ```bash kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/crd.yaml -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/system-upgrade-controller.yaml ``` -------------------------------- ### Install K3s with Docker Source: https://github.com/k3s-io/docs/blob/main/docs/advanced.md Deploy K3s using the --docker flag to enable the Docker runtime. ```bash curl -sfL https://get.k3s.io | sh -s - --docker ``` -------------------------------- ### Download and Install K3s Binary Source: https://github.com/k3s-io/docs/blob/main/docs/installation/configuration.md Downloads the K3s binary from GitHub and makes it executable for manual execution. ```bash curl -Lo /usr/local/bin/k3s https://github.com/k3s-io/k3s/releases/download/v1.26.5+k3s1/k3s; chmod a+x /usr/local/bin/k3s ``` -------------------------------- ### Install kernel-modules-extra on RHEL 10 Source: https://github.com/k3s-io/docs/blob/main/docs/installation/requirements.md On RHEL 10, an additional package is required for K3s networking. Install it using dnf. ```bash sudo dnf install -y kernel-modules-extra ```