### Install Latest Kured with Manifests Source: https://kured.dev/docs/installation Applies the latest Kured release using combined manifests. Ensure you have kubectl installed and configured. ```bash latest=$(curl -s https://api.github.com/repos/kubereboot/kured/releases | jq -r '.[0].tag_name') kubectl apply -f "https://github.com/kubereboot/kured/releases/download/$latest/kured-$latest-combined.yaml" ``` -------------------------------- ### Minikube Setup and Testing Source: https://kured.dev/docs/development Steps to set up a minikube cluster, build and deploy kured, and trigger a reboot test. Ensure to adjust kured configuration for immediate testing. ```bash # start minikube minikube start --driver=kvm2 --kubernetes-version # build kured image and publish to registry accessible by minikube make image minikube-publish # edit kured-ds.yaml to # - point to new image # - change e.g. period and reboot-days option for immediate results minikube kubectl -- apply -f kured-rbac.yaml minikube kubectl -- apply -f kured-ds.yaml minikube kubectl -- logs daemonset.apps/kured -n kube-system -f # In separate terminal minikube ssh sudo touch /var/run/reboot-required minikube logs -f ``` -------------------------------- ### Kured Control-Plane Taint Example (JSON) Source: https://kured.dev/docs/faq This JSON snippet shows an example of a taint that might be present on control-plane nodes, preventing pods from being scheduled. ```json [ { "effect": "NoExecute", "key": "CriticalAddonsOnly", "value": "true" } ] ``` -------------------------------- ### Kured DaemonSet Tolerations Example Source: https://kured.dev/docs/faq This YAML snippet shows the default tolerations for the kured daemonset, which might need modification to run on tainted control-plane nodes. ```yaml tolerations: - key: node-role.kubernetes.io/control-plane effect: NoSchedule - key: node-role.kubernetes.io/master effect: NoSchedule ``` -------------------------------- ### Configure Email Notification URL for Kured Source: https://kured.dev/docs/configuration Configure email notifications using an SMTP URL. This example includes username, password, host, port, sender address, and recipient addresses. ```bash smtp://username:password@host:port/?fromAddress=fromAddress&toAddresses=recipient1[,recipient2,...] ``` -------------------------------- ### Alert for Failsafe Reboot Daemon Source: https://kured.dev/docs/configuration An example alert configuration to monitor the kured_reboot_required metric. This acts as a failsafe if the reboot daemon cannot automatically reboot nodes. ```prometheus # Alert if a reboot is required for any machines. Acts as a failsafe for the # reboot daemon, which will not reboot nodes if there are pending alerts save ``` -------------------------------- ### Configure Slack Notification URL for Kured Source: https://kured.dev/docs/configuration Configure Slack notifications by providing a URL. This example shows the format for a new Slack App integration, including bot token, channel, and bot name. ```bash slack://xoxb:123456789012-1234567890123-4mt0t4l1YL3g1T5L4cK70k3N@?botname= ``` -------------------------------- ### Set Reboot Schedule Source: https://kured.dev/docs/configuration Define specific days, start and end times, and the timezone for reboots to occur. This helps in scheduling reboots during predictable hours. ```bash --reboot-days=mon,tue,wed,thu,fri --start-time=9am --end-time=5pm --time-zone=America/Los_Angeles ``` -------------------------------- ### Kured Control-Plane Taint Example (YAML) Source: https://kured.dev/docs/faq This YAML snippet represents a common taint found on control-plane nodes, such as 'CriticalAddonsOnly'. ```yaml taints: - effect: NoExecute key: CriticalAddonsOnly value: "true" ``` -------------------------------- ### Configure Concurrent Reboots in Kured Source: https://kured.dev/docs/configuration Enable concurrent reboots by setting the `--concurrency` argument. This example allows up to three nodes to reboot simultaneously. Use with caution in production. ```bash --concurrency=3 ``` -------------------------------- ### Configure Teams Notification URL for Kured Source: https://kured.dev/docs/configuration Configure Microsoft Teams notifications by providing a URL. This example includes group, tenant, alternative ID, group owner, and optional host. ```bash teams://group@tenant/altId/groupOwner?host=organization.webhook.office.com ``` -------------------------------- ### Configure Alert Filtering for Kured Source: https://kured.dev/docs/configuration When using Kured with alerts, specify `--alert-filter-regexp` to prevent deadlocks. This example filters for the 'RebootRequired' alert. ```bash --alert-filter-regexp=^RebootRequired$ ``` -------------------------------- ### Get Node Taints with kubectl Source: https://kured.dev/docs/faq Use this command to retrieve the taints applied to a specific Kubernetes node. This helps in understanding why a pod might not be scheduled on that node. ```bash kubectl get nodes ${NODENAME} -o json | jq .spec.taints ``` -------------------------------- ### Configure Rocketchat Notification URL for Kured Source: https://kured.dev/docs/configuration Configure Rocket.Chat notifications by providing a URL. This example shows the general format including optional username, host, token, and recipient. ```bash rocketchat://[username@]rocketchat-host/token[/channel|@recipient] ``` -------------------------------- ### Customize Kured Notification Messages Source: https://kured.dev/docs/configuration Customize the text of Kured's notification messages for draining nodes using `--message-template-drain`. This example shows a template for a specific cluster and region. ```bash --message-template-drain="Draining node %s part of *my-cluster* in region *xyz*" ``` -------------------------------- ### Kind Cluster Creation and Testing Source: https://kured.dev/docs/development Instructions for creating a kind cluster and running scripts to test kured's reboot coordination. ```bash # create kind cluster kind create cluster --config .github/kind-cluster-.yaml # create reboot required files on pre-defined kind nodes ./https://github.com/kubereboot/kured/blob/main/tests/kind/create-reboot-sentinels.sh # check if reboot is working fine ./https://github.com/kubereboot/kured/blob/main/tests/kind/follow-coordinated-reboot.sh ``` -------------------------------- ### Run Basic Golang Tests with Make Source: https://kured.dev/docs/development Execute basic tests for Golang code using the Makefile. This provides a quick check of code behavior. ```bash make test ``` -------------------------------- ### Update Manifests with New Version Source: https://kured.dev/docs/development Set the desired version and use the make command to generate updated manifests for publishing a new kured release. ```bash export VERSION=1.20.0 make DH_ORG="kubereboot" VERSION="${VERSION}" manifest ``` -------------------------------- ### Describe Node Details Source: https://kured.dev/docs/faq The describe command provides a comprehensive overview of a Kubernetes node, including its taints, conditions, and events. It's a general-purpose command for node troubleshooting. ```bash kubectl describe nodes ${NODENAME} ``` -------------------------------- ### Automated Kind E2E Testing Source: https://kured.dev/docs/development Run end-to-end tests for kured on a kind cluster using the provided make target. ```bash # Build kured:dev image, build manifests, and run the "long" go https://github.com/kubereboot/kured/blob/main/tests make e2e-test ``` -------------------------------- ### Run Local Lint Checks with Make Source: https://kured.dev/docs/development Execute local lint checks using the provided Makefile. Ensure your code adheres to linting standards before submitting. ```bash make lint ``` -------------------------------- ### Add Node Labels Before and After Reboot Source: https://kured.dev/docs/configuration Apply specific node labels before initiating a reboot and after the reboot process is complete. Labels can be comma-delimited or specified multiple times. ```bash --pre-reboot-node-labels=zalando=notready --post-reboot-node-labels=zalando=ready ``` -------------------------------- ### Provoke Reboot on a Node Source: https://kured.dev/docs/operation Use this command to test your kured configuration by simulating a reboot requirement on a node. Ensure you have the necessary permissions. ```bash sudo touch /var/run/reboot-required ``` -------------------------------- ### Download Latest Kured Release Combined YAML Source: https://kured.dev/docs/faq Fetches the latest kured release tag from GitHub and downloads the combined YAML manifest. This is useful for obtaining the full deployment configuration. ```bash # while writing latest is 1.21.0 latest=$(curl -s https://api.github.com/repos/kubereboot/kured/releases | jq -r '.[0].tag_name') wget "https://github.com/kubereboot/kured/releases/download/$latest/kured-$latest-combined.yaml" ``` -------------------------------- ### View Node Taints in VS Code Source: https://kured.dev/docs/faq This command fetches node information in YAML format and opens it in VS Code for easier viewing and editing. Useful for inspecting complex node configurations. ```bash kubectl get nodes ${NODENAME} -o yaml | code - ``` -------------------------------- ### Create Combined Manifest for Kured Release Source: https://kured.dev/docs/development Generates a combined YAML manifest for a new Kured release by concatenating RBAC and DaemonSet configurations. Ensure the VERSION environment variable is set correctly. ```bash export VERSION=1.20.0 export MANIFEST="kured-$VERSION-combined.yaml" make DH_ORG="kubereboot" VERSION="${VERSION}" manifest # just to be safe cat kured-rbac.yaml > "$MANIFEST" cat kured-ds.yaml >> "$MANIFEST" ``` -------------------------------- ### Kured Daemonset Usage Source: https://kured.dev/docs/configuration This displays the available command-line flags for the kured daemonset. Use these flags to customize kured's behavior regarding alerts, node draining, reboot scheduling, and notifications. ```bash Kubernetes Reboot Daemon Usage: kured [flags] Flags: --alert-filter-regexp regexp.Regexp alert names to ignore when checking for active alerts --alert-filter-match-only Only block if the alert-filter-regexp matches active alerts --alert-firing-only only consider firing alerts when checking for active alerts --annotate-nodes if set, the annotations 'weave.works/kured-reboot-in-progress' and 'weave.works/kured-most-recent-reboot-needed' will be given to nodes undergoing kured reboots --blocking-pod-selector stringArray label selector identifying pods whose presence should prevent reboots --drain-delay duration delay drain for this duration (default: 0, disabled) --drain-grace-period int time in seconds given to each pod to terminate gracefully, if negative, the default value specified in the pod will be used (default -1) --drain-pod-selector string only drain pods with labels matching the selector (default: '', all pods) --drain-timeout duration timeout after which the drain is aborted (default: 0, infinite time) --ds-name string name of daemonset on which to place lock (default "kured") --ds-namespace string namespace containing daemonset on which to place lock (default "kube-system") --end-time string schedule reboot only before this time of day (default "23:59:59") --force-reboot force a reboot even if the drain fails or times out -h, --help help for kured --lock-annotation string annotation in which to record locking node (default "weave.works/kured-node-lock") --lock-release-delay duration delay lock release for this duration (default: 0, disabled) --lock-ttl duration expire lock annotation after this duration (default: 0, disabled) --log-format string use text or json log format (default "text") --message-template-drain string message template used to notify about a node being drained (default "Draining node %s") --message-template-reboot string message template used to notify about a node being rebooted (default "Rebooting node %s") --message-template-uncordon string message template used to notify about a node being successfully uncordoned (default "Node %s rebooted & uncordoned successfully!") --metrics-host string host where metrics will listen (default "") --metrics-port int port number where metrics will listen (default 8080) --node-id string node name kured runs on, should be passed down from spec.nodeName via KURED_NODE_ID environment variable --notify-url string notify URL for reboot notifications (cannot use with --slack-hook-url flags) --period duration sentinel check period (default 1h0m0s) --post-reboot-node-labels strings labels to add to nodes after uncordoning --pre-reboot-node-labels strings labels to add to nodes before cordoning --prefer-no-schedule-taint string Taint name applied during pending node reboot (to prevent receiving additional pods from other rebooting nodes). Disabled by default. Set e.g. to "weave.works/kured-node-reboot" to enable tainting. --prometheus-url string Prometheus instance to probe for active alerts --reboot-command string command to run when a reboot is required (default "/bin/systemctl reboot") --reboot-days strings schedule reboot on these days (default [su,mo,tu,we,th,fr,sa]) --reboot-delay duration delay reboot for this duration (default: 0, disabled) --reboot-method method to use for reboots (default command), available: command, signal --reboot-signal signal to use for reboots (default 39 = SIGRTMIN+5). --reboot-sentinel string path to file whose existence triggers the reboot command (default "/var/run/reboot-required") --reboot-sentinel-command string command for which a zero return code will trigger a reboot command --skip-wait-for-delete-timeout int when seconds is greater than zero, skip waiting for the pods whose deletion timestamp is older than N seconds while draining a node --slack-channel string slack channel for reboot notifications --slack-hook-url string slack hook URL for reboot notifications [deprecated in favor of --notify-url] --slack-username string slack username for reboot notifications (default "kured") ``` -------------------------------- ### Build Kured Inside GOPATH Source: https://kured.dev/docs/building Use this command to build Kured when your project is located inside your Go workspace, ensuring Go Modules are enabled. ```bash GO111MODULE=on make ``` -------------------------------- ### Build Kured Outside GOPATH Source: https://kured.dev/docs/building Use this command to build Kured when your project is located outside of your Go workspace. ```bash make ``` -------------------------------- ### Check Kured DaemonSet Pod Count Source: https://kured.dev/docs/faq After applying configuration changes, use this command to verify the number of kured pods managed by the daemonset in the kube-system namespace. ```bash kubectl -n kube-system get daemonsets/kured ``` -------------------------------- ### Customizing Kind E2E Test Execution Source: https://kured.dev/docs/development Control the execution of end-to-end tests on kind by passing arguments to the make e2e-test command. ```bash # Run only TestE2EWithSignal test for the kubernetes version named "current" (see kind file) make e2e-test ARGS="-run ^TestE2EWithSignal/current" # Run all https://github.com/kubereboot/kured/blob/main/tests but make sure to extend the timeout, for slower machines. make e2e-test ARGS="-timeout 1200s'" ``` -------------------------------- ### Configure Reboot Sentinel Command Source: https://kured.dev/docs/configuration Use a command to determine if a reboot is required. The command should exit with code 0 if a reboot is needed. This overrides the sentinel file check. ```yaml configuration: rebootSentinelCommand: sh -c "! needs-restarting --reboothint" ``` -------------------------------- ### Manually Disable Reboots with Annotation Source: https://kured.dev/docs/operation Temporarily stop kured from rebooting nodes by manually acquiring the cluster lock. Remember to release the lock afterwards. ```bash kubectl -n kube-system annotate ds kured weave.works/kured-node-lock='{"nodeID":"manual"}' ``` -------------------------------- ### Block Reboots with Pod Selectors Source: https://kured.dev/docs/configuration Prevent reboots on a node if specific pods are scheduled on it, based on label selectors. Multiple selectors can be used with commas for 'AND' logic or specified multiple times for 'OR' logic. ```bash --blocking-pod-selector=runtime=long,cost=expensive ``` ```bash --blocking-pod-selector=name=temperamental ``` -------------------------------- ### Modified Kured Toleration for Tainted Nodes Source: https://kured.dev/docs/faq This YAML snippet demonstrates how to modify the kured daemonset's tolerations to allow it to run on nodes with specific taints, like 'CriticalAddonsOnly'. ```yaml tolerations: - key: CriticalAddonsOnly operator: Exists ``` -------------------------------- ### Configure Kured Lock Annotation Source: https://kured.dev/docs/configuration Override the default annotation name used by Kured for locking with the `--lock-annotation` argument. The default is generally safe to use. ```bash --lock-annotation= ``` -------------------------------- ### Block Reboots with Prometheus Alerts Source: https://kured.dev/docs/configuration Prevent reboots when active alerts are present in Prometheus. Specify the Prometheus URL and optionally filter alerts by regular expression or alert status. ```bash --prometheus-url=http://prometheus.monitoring.svc.cluster.local ``` ```bash --alert-filter-regexp=^(RebootRequired|AnotherBenignAlert|...$ ``` ```bash --alert-firing-only=true ``` ```bash --alert-filter-match-only=true ``` -------------------------------- ### Manually Remove Cluster Lock Source: https://kured.dev/docs/operation In cases of permanent node failure during a reboot, use this command to remove the cluster lock entirely. The trailing hyphen is crucial for removing the annotation. ```bash kubectl -n kube-system annotate ds kured weave.works/kured-node-lock- ``` -------------------------------- ### Kured Reboot Required Prometheus Metric Source: https://kured.dev/docs/configuration Exposes a gauge metric indicating if an OS requires a reboot due to software updates. This metric can be used to create alerts for manual intervention. ```prometheus # HELP kured_reboot_required OS requires reboot due to software updates. # TYPE kured_reboot_required gauge kured_reboot_required{node="ip-xxx-xxx-xxx-xxx.ec2.internal"} 0 ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.