### Example Bottlerocket updater interface version label

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

This is an example of the label format used to indicate the updater interface version on a Bottlerocket node.

```text
bottlerocket.aws/updater-interface-version=2.0.0
```

--------------------------------

### Install Bottlerocket Shadow CRD with Helm

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-shadow/templates/NOTES.txt

Use this Helm command to install the Bottlerocket Shadow CRD. Ensure you specify the correct chart and version.

```bash
helm install brupop bottlerocket-update-operator/bottlerocket-shadow \
  --version v1.0.0
```

--------------------------------

### Install Cert-Manager using Helm

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Installs cert-manager and its CRDs using the Helm chart. This command specifies the namespace, version, and enables CRD installation.

```sh
helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.8.2 \
  --set installCRDs=true
```

--------------------------------

### Custom Resource Owner Reference Example

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/DESIGN.md

Example of ownerReferences for a custom resource, associating it with a Kubernetes Node. Kubernetes garbage collection relies on these references.

```yaml
metadata:
  name: ip-192-168-48-253.us-west-2.compute.internal
  ownerReferences:
    - apiVersion: v1
      kind: Node
      uid: c5a4b820-39a6-46e8-9a2f-a2623a17dfbf
      name: ip-192-168-48-253.us-west-2.compute.internal
```

--------------------------------

### Install Bottlerocket Shadow CRD Helm Chart Locally

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Installs the Bottlerocket Shadow CRD Helm chart locally. This CRD is a dependency for the update operator.

```sh
helm install \
  brupop-crd \
  deploy/charts/bottlerocket-shadow
```

--------------------------------

### Install Cert-Manager

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/1.0.0-release.md

Install cert-manager, a prerequisite for the Bottlerocket Update Operator, using its official release manifest.

```sh
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.2/cert-manager.yaml
```

--------------------------------

### Generate Manifest

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Run this command to re-generate the yaml manifest found at the root of the repository. Requires helm to be installed.

```shell
make manifest
```

--------------------------------

### Set up Brupop Test Environment

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/integ/src/README.md

Use this command to set up the Brupop test environment, install Brupop, and label nodes. Ensure you replace placeholders with your specific cluster details and versions.

```bash
cargo run --bin integ integration-test --cluster-name <YOUR_CLUSTER_NAME> --region <YOUR_CLUSTER_REGION> --bottlerocket-version <OLD_BOTTLEROCKET_VERSION>  --arch <ARCH> --nodegroup-name <NODEGROUP_NAME>
```

--------------------------------

### Install Cert-Manager using kubectl

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Applies the cert-manager manifest using kubectl. Ensure you are using a compatible release version.

```sh
kubectl apply -f \
  https://github.com/cert-manager/cert-manager/releases/download/v1.8.2/cert-manager.yaml
```

--------------------------------

### Install Bottlerocket Update Operator Helm Chart Locally

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Installs the Bottlerocket Update Operator Helm chart from the local repository. This command assumes you are in the root of the repository.

```sh
helm install \
  brupop \
  deploy/charts/bottlerocket-update-operator \
  --create-namespace
```

--------------------------------

### Deploy Prometheus for Update Operator Metrics

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Apply a sample Prometheus configuration to deploy Prometheus within the cluster. This setup is configured to gather metrics data from the update operator.

```sh
kubectl apply -f ./deploy/telemetry/prometheus-resources.yaml
```

--------------------------------

### Shortened Command to Get BottlerocketShadows

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

A shorter alias for retrieving BottlerocketShadow custom resources, useful for quick checks.

```sh
kubectl get brs --namespace brupop-bottlerocket-aws
```

--------------------------------

### Get all nodes using kubectl

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Use this command to list all nodes in the Kubernetes cluster. This is a prerequisite for labeling individual nodes.

```sh
kubectl get nodes
```

--------------------------------

### Example Regional Image URI

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

An example of a regional Amazon ECR image URI for the Bottlerocket update operator. This can be used when pulling images from specific AWS regions.

```text
328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-update-operator:v1.1.0
```

--------------------------------

### Install Bottlerocket Update Operator using Helm

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Installs the Bottlerocket Update Operator (Brupop) and its CRD using Helm. This includes the operator's controller, agent, and API server deployments.

```sh
# Add the bottlerocket-update-operator chart
helm repo add brupop https://bottlerocket-os.github.io/bottlerlerocket-update-operator

# Update your local chart cache with the latest updates
helm repo update

# Create a namespace
kubectl create namespace brupop-bottlerocket-aws

# Install the brupop CRD
helm install brupop-crd brupop/bottlerocket-shadow

# Install the brupop operator
helm install brupop-operator brupop/bottlerocket-update-operator
```

--------------------------------

### Clean Brupop Integration Test Resources

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/integ/src/README.md

Use this command to destroy all resources created during the Brupop integration test installation. Specify your cluster details and nodegroup name.

```bash
cargo run --bin integ clean --cluster-name <YOUR_CLUSTER_NAME> --region <YOUR_CLUSTER_REGION> --nodegroup-name <NODEGROUP_NAME>
```

--------------------------------

### Bottlerocket Shadow Configuration

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-shadow/README.md

Example configuration values for the Bottlerocket update operator, specifying the deployment namespace and the API server service port.

```yaml
# The namespace to deploy the update operator into
namespace: "brupop-bottlerocket-aws"

# API server internal address where the conversion webhook is served
apiserver_service_port: "443"

```

--------------------------------

### Get Bottlerocket Shadows

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/templates/NOTES.txt

Use this command to view the status of your Bottlerocket nodes. This command retrieves custom resources that reflect the state of Bottlerocket nodes.

```bash
kubectl get bottlerocketshadows
```

--------------------------------

### Get Prometheus Pod Name

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Retrieve the name of the Prometheus pod, which is necessary for setting up port forwarding.

```sh
kubectl get pods --namespace brupop-bottlerocket-aws
```

--------------------------------

### Get BottlerocketShadow Custom Resources

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Query the Kubernetes API to get the current state of BottlerocketShadow objects. This helps in understanding the Bottlerocket version and update status of managed nodes.

```sh
kubectl get bottlerocketshadows --namespace brupop-bottlerocket-aws
```

--------------------------------

### Add Cert-Manager Helm Repository

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Adds the Jetstack Helm repository, which hosts the cert-manager chart. This is a prerequisite for installing cert-manager via Helm.

```sh
helm repo add jetstack https://charts.jetstack.io
```

--------------------------------

### Update Helm Repositories

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Updates the local Helm chart repository cache to fetch the latest chart information. This should be run before installing any new charts.

```sh
helm repo update
```

--------------------------------

### Bottlerocket Update Operator Default Configuration

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Shows the default configuration values for the Bottlerocket Update Operator Helm chart. These values can be overridden during installation.

```yaml
# Default values for bottlerocket-update-operator.

# The namespace to deploy the update operator into
namespace: "brupop-bottlerocket-aws"

# The image to use for brupop
image: "public.ecr.aws/bottlerocket/bottlerocket-update-operator:v1.8.0"

# Provide pod level labels and annotations for all brupop workloads
podLabels: {}
podAnnotations: {}

# Brupop workload specific pod labels and annotations
controller:
  podLabels: {}
  podAnnotations: {}
apiserver:
  podLabels: {}
  podAnnotations: {}
agent:
  podLabels: {}
  podAnnotations: {}

# Placement controls
# See the Kubernetes documentation about placement controls for more details:
# * https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
# * https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector
# * https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
placement:
  agent:
    # The agent is a daemonset, so the only controls that apply to it are tolerations.
    tolerations: []

  controller:
    tolerations: []
    nodeSelector: {}
    podAffinity: {}
    podAntiAffinity: {}

  apiserver:
    tolerations: []
    nodeSelector: {}
    podAffinity: {}
    # By default, apiserver pods prefer not to be scheduled to the same node.
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 1
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: brupop.bottlerocket.aws/component
                operator: In
                values:
                - apiserver
            topologyKey: kubernetes.io/hostname

# If testing against a private image registry, you can set the pull secret to fetch images.
# This can likely remain as `brupop` so long as you run something like the following:
# kubectl create secret docker-registry brupop \
#  --docker-server 109276217309.dkr.ecr.us-west-2.amazonaws.com \
#  --docker-username=AWS \
#  --docker-password=$(aws --region us-west-2 ecr get-login-password) \
#  --namespace=brupop-bottlerocket-aws
#imagePullSecrets:
#  - name: "brupop"

# External load balancer setting.
# When `exclude_from_lb_wait_time_in_sec` is set to a positive value
# brupop will exclude the node from load balancing
# and will wait for `exclude_from_lb_wait_time_in_sec` seconds before draining nodes.
# Under the hood, this uses the `node.kubernetes.io/exclude-from-external-load-balancers` label
# to exclude those nodes from load balancing.
exclude_from_lb_wait_time_in_sec: "0"

# Concurrent update nodes setting.
# When `max_concurrent_updates` is set to a positive integer value,
# brupop will concurrently update max `max_concurrent_updates` nodes.
# When `max_concurrent_updates` is set to "unlimited",
# brupop will concurrently update all nodes with respecting `PodDisruptionBudgets`

```

--------------------------------

### Install Bottlerocket Update Operator

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/1.0.0-release.md

Apply the Bottlerocket Update Operator manifest to your Kubernetes cluster. This command creates necessary resources like namespaces, CRDs, roles, and deployments.

```sh
kubectl apply -f ./bottlerocket-update-operator.yaml
```

--------------------------------

### Build and Tag Development Image

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

This command builds and tags a development image using the local Docker daemon. The image can then be pushed to a container registry.

```shell
make brupop-image
```

--------------------------------

### Port Forward to Prometheus UI

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Set up port forwarding to access the Prometheus UI from your local machine. Replace $prometheus-pod-name with the actual pod name obtained from the previous command.

```sh
kubectl port-forward $prometheus-pod-name 9090:9090 --namespace brupop-bottlerocket-aws
```

--------------------------------

### Apply Latest Bottlerocket OS Update

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/1.0.0-release.md

Initiate the application of the latest available update. The system will automatically migrate its configuration upon the next reboot into the new version. This command is part of the automated update process.

```sh
apiclient update apply
```

--------------------------------

### Configure Update Time Window (Deprecated)

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Set UPDATE_WINDOW_START and UPDATE_WINDOW_STOP environment variables to define a time window for updates. This setting is deprecated and will be ignored if cron expressions are used.

```yaml
      containers:
        - command:
            - "./controller"
          env:
            - name: MY_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: UPDATE_WINDOW_START
              value: "09:00:00"
            - name: UPDATE_WINDOW_STOP
              value: "21:00:00"
```

--------------------------------

### Check for Available Bottlerocket OS Updates

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/1.0.0-release.md

Use this command to query the Bottlerocket API for any pending system updates. The output will indicate the chosen update and a list of all available versions, including older ones for rollback purposes.

```sh
apiclient update check
```

--------------------------------

### Fetch Controller Logs

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Retrieve logs from the Bottlerocket Update Operator's controller deployment. These logs are generally less helpful for update state issues but can be useful for other operational problems.

```sh
kubectl logs deployment/brupop-controller-deployment --namespace brupop-bottlerocket-aws
```

--------------------------------

### Fetch API Server Logs

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Retrieve logs from the Bottlerocket Update Operator's API server deployment. This is useful for debugging cluster-wide update orchestration issues.

```sh
kubectl logs deployment/brupop-apiserver --namespace brupop-bottlerocket-aws
```

--------------------------------

### Regenerate YAML Resource Definitions

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

After modifying the .env file with the desired image name and pull secret, run this command to regenerate the YAML resource definitions for deployment to Kubernetes.

```shell
cargo build -p deploy
```

--------------------------------

### Fetch Bottlerocket Agent Logs

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Retrieve logs from a specific Bottlerocket agent pod running on a node. This is crucial for troubleshooting node-specific update failures.

```sh
kubectl logs brupop-agent-podname --namespace brupop-bottlerocket-aws
```

--------------------------------

### Enable Node Exclusion from Load Balancers

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Configure the EXCLUDE_FROM_LB_WAIT_TIME_IN_SEC environment variable to enable and set the wait time for excluding nodes from load balancers before draining.

```yaml
      ...
      containers:
        - command:
            - "./agent"
          env:
            - name: MY_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: EXCLUDE_FROM_LB_WAIT_TIME_IN_SEC
              value: "180"
      ...
```

--------------------------------

### List Bottlerocket Agent Pods

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

List all Bottlerocket agent pods across the cluster, filtering by component. This helps identify the specific agent pod on a node experiencing update issues.

```sh
kubectl get pods --selector=brupop.bottlerocket.aws/component=agent -o wide --namespace brupop-bottlerocket-aws
```

--------------------------------

### Automatic node labeling via Bottlerocket user-data

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Configure Bottlerocket user-data to automatically apply Kubernetes node labels during provisioning. This ensures nodes are correctly labeled upon creation.

```toml
# Configure the node-labels Bottlerocket setting
[settings.kubernetes.node-labels]
"bottlerocket.aws/updater-interface-version" = "2.0.0"
```

--------------------------------

### Port-forwarding Prometheus

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/1.0.0-release.md

Use this command to access the Prometheus UI for visualizing update operator metrics. Replace PROMETHEUS-POD-NAME with the actual name of your Prometheus pod.

```bash
kubectl port-forward pods/PROMETHEUS-POD-NAME 9090:9090
```

--------------------------------

### Automatic nodegroup labeling via eksctl

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Use eksctl configuration to automatically add labels to Bottlerocket nodegroups in an EKS cluster. This simplifies label management for EKS environments.

```yaml
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: bottlerocket-cluster
  region: us-west-2
  version: '1.17'

nodeGroups:
  - name: ng-bottlerocket
    labels: { bottlerocket.aws/updater-interface-version: 2.0.0 }
    instanceType: m5.large
    desiredCapacity: 3
    amiFamily: Bottlerocket
```

--------------------------------

### Monitor Bottlerocket Node Updates

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/integ/src/README.md

This command verifies if nodes are being updated to the target Bottlerocket version. Provide your cluster and region details.

```bash
cargo run --bin integ monitor --cluster-name <YOUR_CLUSTER> --region <YOUR_CLUSTER_REGION>
```

--------------------------------

### Reboot Bottlerocket OS Immediately

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/1.0.0-release.md

Triggers an immediate reboot of the Bottlerocket system. The system is designed to automatically roll back to the previous version if it encounters issues booting after the update.

```sh
apiclient reboot
```

--------------------------------

### Configure API Server Service Port

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Update the 'port' entry in the webhook clientConfig.service section to match the API server's service port.

```yaml
    ...
    webhook:
      clientConfig:
        service:
          name: brupop-apiserver
          namespace: brupop-bottlerocket-aws
          path: /crdconvert
          port: 123
```

--------------------------------

### Label Bottlerocket Nodes for Update Operator

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-shadow/README.md

Labels all nodes in the cluster with the specified Bottlerocket updater interface version. This is necessary for the Brupop controller to operate on the nodes.

```bash
kubectl label node $(kubectl get nodes -o jsonpath='{.items[*].metadata.name}') bottlerocket.aws/updater-interface-version=2.0.0
```

--------------------------------

### Configure Scheduler Cron Expression

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Set the SCHEDULER_CRON_EXPRESSION environment variable to define a cron-based schedule for updates. The format includes seconds, minutes, hours, day of month, month, day of week, and year.

```yaml
      containers:
        - command:
            - "./controller"
          env:
            - name: MY_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: SCHEDULER_CRON_EXPRESSION
              value: "* * * * * * *"
```

--------------------------------

### Label a Single Node for Bottlerocket Update Operator

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/README.md

Applies the necessary label to a specific node to enable Bottlerocket update operator functionality. Ensure the node is running Bottlerocket OS.

```sh
kubectl label node NODE_NAME bottlerocket.aws/updater-interface-version=2.0.0
```

--------------------------------

### BottlerocketShadow State Machine

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/design/1.0.0-release.md

Defines the possible states for a Bottlerocket node during an update process managed by the update operator. Used to track the progress and status of updates.

```rust
pub enum BottlerocketShadowState {
    /// Nodes in this state are waiting for new updates to become available. 
    /// This is both the starting, terminal and recovery state in the update process.
    Idle,
    /// Nodes in this state have staged a new update image, have installed 
    /// the new image, and have updated the partition table
    /// to mark it as the new active image.
    StagedAndPerformedUpdate,
    /// Nodes in this state have used the kubernetes cordon and drain APIs to remove
    /// running pods, have un-cordoned the node to allow work to be scheduled, and
    /// have rebooted after performing an update.
    RebootedIntoUpdate,
    /// Nodes in this state are monitoring to ensure that the node seems healthy
    /// before marking the update as complete.
    MonitoringUpdate,
    /// Nodes in this state have crashed due to Bottlerocket Update API call failure.
    ErrorReset,
}
```

--------------------------------

### Label Bottlerocket Nodes

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/deploy/charts/bottlerocket-update-operator/templates/NOTES.txt

Apply this label to your Bottlerocket nodes to enable the update operator to manage them. Replace {MY_NODE_NAME} with the actual name of your node.

```bash
kubectl label node {MY_NODE_NAME} bottlerocket.aws/updater-interface-version=2.0.0
```

--------------------------------

### Configure Maximum Concurrent Updates

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Set the MAX_CONCURRENT_UPDATE environment variable to control the number of nodes updated simultaneously. This can be a positive integer or 'unlimited'.

```yaml
      containers:
        - command:
            - "./controller"
          env:
            - name: MY_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: MAX_CONCURRENT_UPDATE
              value: "1"
```

--------------------------------

### Label Node for Automatic Updates

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Labels a Bottlerocket node to indicate it should be automatically updated by the Bottlerocket Update Operator. Only nodes with this label will be updated.

```sh
kubectl label node MY_NODE_NAME bottlerocket.aws/updater-interface-version=2.0.0
```

--------------------------------

### Configure API Server Internal Port

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

Adjust the APISERVER_INTERNAL_PORT environment variable within the container definition to set the API server's internal port.

```yaml
      ...
      containers:
        - command:
            - "./api-server"
          env:
            - name: APISERVER_INTERNAL_PORT
              value: "999"
```

--------------------------------

### Uninstall Bottlerocket Update Operator Helm charts

Source: https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/develop/README.md

These commands uninstall the Bottlerocket Update Operator by removing its associated Helm charts. This is the final step in fully removing the operator from the cluster.

```sh
helm uninstall brupop
helm uninstall brupop-crd
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.