Rook

https://rook.github.io/docs/rook/v1.18
Admin
Rook is a cloud-native storage orchestrator that automates deployment and management of Ceph storage...
Tokens:118,461
Snippets:750
Trust Score:8.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Deploy Rook Cluster using kubectl

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/quickstart

This snippet shows the basic kubectl commands to clone the Rook repository, navigate to the examples directory, and apply the necessary CRDs and operator manifests to create a Rook Ceph cluster. It assumes you have kubectl installed and configured.

```bash
git clone --single-branch --branch v1.18.4 https://github.com/rook/rook.git
cd rook/deploy/examples
kubectl create -f crds.yaml -f common.yaml -f csi-operator.yaml -f operator.yaml
kubectl create -f cluster.yaml
```

--------------------------------

### Deploy Rook Operator using kubectl

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/quickstart

This snippet details the steps to deploy the Rook operator using kubectl. It involves navigating to the Rook examples directory and applying the CRDs, common manifests, CSI operator, and the main operator manifest. A verification step using `kubectl get pod` is included to ensure the operator is running.

```bash
cd deploy/examples
kubectl create -f crds.yaml -f common.yaml -f csi-operator.yaml -f operator.yaml

# verify the rook-ceph-operator is in the `Running` state before proceeding
kubectl -n rook-ceph get pod
```

--------------------------------

### Start Minikube Cluster for Rook Development

Source: https://rook.github.io/docs/rook/v1.18/Contributing/development-environment

Commands to start a Minikube cluster with specific configurations suitable for Rook development. It includes options for different operating systems and emphasizes the need for sufficient disk space and additional disks for Ceph OSDs.

```bash
# On Linux
minikube start --disk-size=40g --extra-disks=1 --driver qemu2
```

```bash
# On MacOS with Intel processor
minikube start --disk-size=40g --extra-disks=1 --driver hyperkit
```

```bash
# On MacOS with Apple silicon
minikube start --disk-size=40g --extra-disks 1 --driver qemu
```

--------------------------------

### Deploy Filesystem Mirror Daemon (Kubectl)

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Shared-Filesystem-CephFS/filesystem-mirroring

This command deploys the 'rook-ceph-fs-mirror' pod, which runs the cephfs-mirror daemon on the source Ceph cluster. It requires the Rook installation to include the example filesystem mirror configuration.

```bash
kubectl create -f deploy/examples/filesystem-mirror.yaml
```

--------------------------------

### Apply Custom ceph.conf Settings - YAML

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

An example of updating the `rook-config-override` ConfigMap to apply custom Ceph configuration settings. This YAML snippet demonstrates setting global options such as `osd crush update on start` and `osd pool default size`.

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: rook-config-override
  namespace: rook-ceph
data:
  config: |
    [global]
    osd crush update on start = false
    osd pool default size = 2
```

--------------------------------

### Verify Ceph Cluster Pods Running

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/quickstart

This command checks the status of pods within the `rook-ceph` namespace to verify that the Ceph cluster components are running. It lists pods such as CSI controllers, crash collectors, exporters, monitors, managers, OSDs, and preparation jobs.

```bash
$ kubectl -n rook-ceph get pod
```

--------------------------------

### Deploy Ceph Cluster with Rook

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/quickstart

This command deploys a Ceph cluster using Rook by applying the `cluster.yaml` configuration file. Ensure Rook operator is running before executing. The configuration determines the number of OSDs based on available devices per node.

```bash
kubectl create -f cluster.yaml
```

--------------------------------

### Deploy Ceph COSI Driver Example

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Object-Storage-RGW/cosi

Applies the example Ceph COSI driver configuration using kubectl from the specified example directory. This command assumes you have navigated to the correct directory.

```bash
cd deploy/examples/cosi
kubectl create -f cephcosidriver.yaml
```

--------------------------------

### Preview Documentation Locally

Source: https://rook.github.io/docs/rook/v1.18/Contributing/documentation

Command to locally preview the documentation. Ensure MkDocs and its extensions are installed by running `pip3 install -r build/release/requirements_docs.txt` if you encounter a `command not found` error. Verify your Python binary path is in your system's PATH.

```shell
make docs-preview
```

--------------------------------

### Deploy CephFilesystemMirror Daemon CRD Example

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Shared-Filesystem/ceph-fs-mirror-crd

This YAML example demonstrates the basic structure of a CephFilesystemMirror CRD for deploying a cephfs-mirror daemon within a Kubernetes environment managed by Rook. It specifies the API version, kind, metadata (name and namespace), and an empty spec, serving as a starting point for configuration.

```yaml
apiVersion: ceph.rook.io/v1
kind: CephFilesystemMirror
metadata:
  name: my-fs-mirror
  namespace: rook-ceph
spec: {}
```

--------------------------------

### Install Documentation Dependencies

Source: https://rook.github.io/docs/rook/v1.18/Contributing/documentation

Installs the necessary dependencies for MkDocs and its extensions, required for local documentation preview. This command should be run if the `make docs-preview` command fails with a 'command not found' error.

```shell
pip3 install -r build/release/requirements_docs.txt
```

--------------------------------

### Start Direct Mount Pod with Rook

Source: https://rook.github.io/docs/rook/v1.18/Troubleshooting/direct-tools

This command starts a pod configured for direct storage mounting using Rook. It's a prerequisite for testing block and file storage directly.

```bash
kubectl create -f deploy/examples/direct-mount.yaml
```

--------------------------------

### Configure Block Storage Class

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/example-configurations

Sets up a Kubernetes StorageClass for Ceph block devices using the CephBlockPool CRD. Examples include replication for production (`storageclass.yaml`), erasure coding for efficiency (`storageclass-ec.yaml`), and a non-redundant option for testing (`storageclass-test.yaml`). These examples are located in the `csi/rbd` directory for the CSI driver.

```shell
# Example for production replication (3x)
# kubectl create -f storageclass.yaml

# Example for erasure coding
# kubectl create -f storageclass-ec.yaml

# Example for test replication (1x) - NOT FOR PRODUCTION
# kubectl create -f storageclass-test.yaml

# CSI driver examples for RBD
# See examples in csi/rbd directory
```

--------------------------------

### Install Rook kubectl Plugin

Source: https://rook.github.io/docs/rook/v1.18/Troubleshooting/kubectl-plugin

Installs the Rook plugin for kubectl using krew. Ensure krew is installed prior to running this command. This command installs the plugin, enabling subsequent Rook-specific kubectl operations.

```bash
kubectl krew install rook-ceph
```

--------------------------------

### Example output after running python script

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Cluster/external-cluster/topology-for-external-mode

Shows example output after running the python script. This includes exported variables containing cluster ID, pool information, topology information, and failure domain details, all required for configuring the external Rook cluster.

```bash
export ROOK_EXTERNAL_FSID=8f01d842-d4b2-11ee-b43c-0050568fb522
....
....
....
export TOPOLOGY_POOLS=pool-a,pool-b,pool-c
export TOPOLOGY_FAILURE_DOMAIN_LABEL=zone
export TOPOLOGY_FAILURE_DOMAIN_VALUES=zone-a,zone-b,zone-c
```

--------------------------------

### Install Rook Ceph Cluster Chart (Development Build)

Source: https://rook.github.io/docs/rook/v1.18/Helm-Charts/ceph-cluster-chart

Installs the Rook Ceph cluster chart from a local development build. This requires deploying the operator chart first to ensure CRDs are present. The installation command uses Helm to install the chart, creating a namespace if it doesn't exist, and applies configurations from a specified values.yaml file.

```bash
cd deploy/charts/rook-ceph-cluster
helm install --create-namespace --namespace rook-ceph rook-ceph-cluster -f values.yaml .
```

--------------------------------

### Example Ceph Configuration File (ceph.conf)

Source: https://rook.github.io/docs/rook/v1.18/CRDs/ceph-client-crd

This is an example of a general Ceph configuration file (`ceph.conf`). It includes the global section with the `mon_host` directive, populated with retrieved monitor endpoints, and a `log file` directive for logging.

```ini
[global]
mon_host=10.107.72.122:6789,10.103.244.218:6789,10.99.33.227:6789
log file = /tmp/ceph-$pid.log
```

--------------------------------

### Example S3 Client Connection Environment Variables for Rook

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Object-Storage-RGW/object-storage

This example provides specific values for environment variables to connect to a Rook Ceph object store. It includes the RGW service DNS name, port, and example access credentials.

```bash
export AWS_HOST=rook-ceph-rgw-my-store.rook-ceph.svc
export PORT=80
export AWS_ACCESS_KEY_ID=XEZDB3UJ6X7HVBE7X7MA
export AWS_SECRET_ACCESS_KEY=7yGIZON7EhFORz0I40BFniML36D2rl8CQQ5kXU6l
```

--------------------------------

### Enable Ceph Telemetry Reporting

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/quickstart

This command enables anonymous telemetry reporting for the Ceph cluster through the Rook toolbox. Telemetry data helps Rook maintainers understand cluster usage and identify potential issues without collecting identifying information.

```bash
ceph telemetry on
```

--------------------------------

### Get OSD Node and Drive Associations (Shell)

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

This script retrieves a list of OSD pods, then iterates through each pod to identify the associated node and its drives. It uses kubectl to get pod information and execs into the pod to run commands like 'lsblk' or 'findmnt' to detail drive size, type, and readiness. This helps in understanding the storage layout within a Rook/Ceph cluster.

```shell
OSD_PODS=$(kubectl get pods --all-namespaces -l \
  app=rook-ceph-osd,rook_cluster=rook-ceph -o jsonpath='{.items[*].metadata.name}')

# Find node and drive associations from OSD pods
for pod in $(echo ${OSD_PODS})
do
 echo "Pod:  ${pod}"
 echo "Node: $(kubectl -n rook-ceph get pod ${pod} -o jsonpath='{.spec.nodeName}')"
 kubectl -n rook-ceph exec ${pod} -- sh -c '
  for i in /var/lib/ceph/osd/ceph-*; do
    [ -f ${i}/ready ] || continue
    echo -ne "-$(basename ${i}) "
    echo $(lsblk -n -o NAME,SIZE ${i}/block 2> /dev/null || \
    findmnt -n -v -o SOURCE,SIZE -T ${i}) $(cat ${i}/type)
  done | sort -V
  echo'
done
```

--------------------------------

### Create Kubernetes Namespace using kubectl

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

Creates a new Kubernetes namespace using the `kubectl create namespace` command. This is a prerequisite for deploying Rook components in a custom namespace.

```bash
kubectl create namespace $ROOK_OPERATOR_NAMESPACE
```

--------------------------------

### Rook OSD Preparation Log Analysis - Example Output

Source: https://rook.github.io/docs/rook/v1.18/Troubleshooting/ceph-common-issues

This is an example of log output from the Rook OSD preparation process. It shows messages indicating why a device might be skipped (e.g., 'skipping device sda that is in use') or messages confirming a device is being configured.

```text
# A device will be skipped if Rook sees it has partitions or a filesystem
2019-05-30 19:02:57.353171 W | cephosd: skipping device sda that is in use
2019-05-30 19:02:57.452168 W | skipping device "sdb5": ["Used by ceph-disk"]

# Other messages about a disk being unusable by ceph include:
Insufficient space (<5GB) on vgs
Insufficient space (<5GB)
LVM detected
Has BlueStore device label
locked
read-only

# A device is going to be configured
2019-05-30 19:02:57.535598 I | cephosd: device sdc to be configured by ceph-volume

# For each device configured you will see a report printed to the log
2019-05-30 19:02:59.844642 I |   Type            Path                                                    LV Size         % of device
2019-05-30 19:02:59.844651 I | ----------------------------------------------------------------------------------------------------
2019-05-30 19:02:59.844677 I |   [data]          /dev/sdc                                                7.00 GB         100%
```

--------------------------------

### Connect to Direct Mount Pod

Source: https://rook.github.io/docs/rook/v1.18/Troubleshooting/direct-tools

After starting the direct mount pod, these commands allow you to connect to it. First, find the pod's name, then execute commands inside the pod using `kubectl exec`.

```bash
kubectl -n rook-ceph get pod -l app=rook-direct-mount
kubectl -n rook-ceph exec -it <pod> -- bash
```

--------------------------------

### Example Ceph Keyring File (ceph.keyring)

Source: https://rook.github.io/docs/rook/v1.18/CRDs/ceph-client-crd

This example shows a Ceph keyring file (`ceph.keyring`) for a specific client. It defines the client section, including the `key` (to be filled with the decoded secret) and `caps` for monitor and OSD access, tailored for RBD workloads. Note that OSD caps might need modification for other workloads.

```ini
[client.example]
  key = < key, decoded from k8s secret>
  # The caps below are for a rbd workload -- you may need to edit/modify these capabilities for other workloads
  # see https://docs.ceph.com/en/latest/cephfs/capabilities
  caps mon = 'allow r'
  caps osd = 'profile rbd pool=<your pool>, profile rb pool=<another pool>'
```

--------------------------------

### Get Help for Rook Multus Validation Tool

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Cluster/network-providers

Displays the help text for the `rook multus validation run` command, outlining its options and usage. This is useful for understanding how to perform the validation.

```bash
rook multus validation run --help
```

--------------------------------

### Example Commit Message Structure

Source: https://rook.github.io/docs/rook/v1.18/Contributing/development-flow

Provides an example of a well-structured commit message, including a component prefix, a detailed explanation of the issue and fix, and the Signed-off-by tag.

```git-commit
component: commit title

This is the commit message. Here I'm explaining what the bug was along with its root cause.
Then I'm explaining how I fixed it.

Signed-off-by: FirstName LastName <email address>
```

--------------------------------

### OBC Custom Resource Example (YAML)

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Object-Storage-RGW/ceph-object-bucket-claim

An example of an ObjectBucketClaim (OBC) custom resource in YAML format. It shows the specification for requesting an object bucket, including metadata, the desired object bucket name, and the storage class to use. The status section indicates the current phase of the bucket provisioning process.

```yaml
apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  creationTimestamp: "2019-10-18T09:54:01Z"
  generation: 2
  name: ceph-bucket
  namespace: default
  resourceVersion: "559491"
spec:
  ObjectBucketName: obc-default-ceph-bucket
  additionalConfig: null
  bucketName: photo-booth-c1178d61-1517-431f-8408-ec4c9fa50bee
  storageClassName: rook-ceph-bucket
status:
  phase: Bound
```

--------------------------------

### Install Ceph Cluster Helm Chart

Source: https://rook.github.io/docs/rook/v1.18/Helm-Charts/ceph-cluster-chart

Installs the Rook Ceph cluster using the Helm package manager. This command deploys the cluster into the 'rook-ceph' namespace, sets the operator namespace, and applies a custom 'values.yaml' file for configuration. Ensure the Rook Operator is installed first and 'values.yaml' is customized.

```bash
helm repo add rook-release https://charts.rook.io/release
helm install --create-namespace --namespace rook-ceph rook-ceph-cluster \
   --set operatorNamespace=rook-ceph rook-release/rook-ceph-cluster -f values.yaml
```

--------------------------------

### Get Help for Rook Multus Validation Configuration

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Cluster/network-providers

Displays the help text for the `rook multus validation config` command, which is used to generate configuration files for advanced validation scenarios.

```bash
rook multus validation config --help
```

--------------------------------

### Installing Debugging Tools in a Ceph Pod

Source: https://rook.github.io/docs/rook/v1.18/Troubleshooting/performance-profiling

Installs necessary debugging utilities such as gdb, perf, and git within the Ceph pod to facilitate performance data collection.

```bash
# dnf install gdb git perf -y
```

--------------------------------

### Deploy Second Rook Ceph Cluster with sed

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

Deploys a second Rook Ceph cluster into a separate namespace by modifying configuration files with `sed`. It sets the operator namespace and a new cluster namespace, then applies the configuration.

```bash
cd deploy/examples
export ROOK_OPERATOR_NAMESPACE="rook-ceph"
export SECOND_ROOK_CLUSTER_NAMESPACE="rook-ceph-secondary"

sed -i.bak \
    -e "s/\(.*\):.*# namespace:operator/\1: $ROOK_OPERATOR_NAMESPACE # namespace:operator/g" \
    -e "s/\(.*\):.*# namespace:cluster/\1: $SECOND_ROOK_CLUSTER_NAMESPACE # namespace:cluster/g" \
  common-second-cluster.yaml

kubectl create -f common-second-cluster.yaml
```

--------------------------------

### Install Rook Helm Chart

Source: https://rook.github.io/docs/rook/v1.18/Helm-Charts/operator-chart

Installs the Rook Ceph Helm chart into a specified namespace. This command deploys Rook using a Helm chart, creating the necessary namespace if it doesn't exist.

```shell
cd deploy/charts/rook-ceph
helm install --create-namespace --namespace rook-ceph rook-ceph .
```

--------------------------------

### Deploy Ceph Cluster CRD

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/example-configurations

Creates the Ceph cluster configuration using the CephCluster CRD. This CR is critical for defining how the operator configures the storage. Various examples are provided for different needs, such as production (`cluster.yaml`), testing (`cluster-test.yaml`), cloud environments (`cluster-on-pvc.yaml`), external clusters (`cluster-external.yaml`, `cluster-external-management.yaml`), and stretched configurations (`cluster-stretched.yaml`).

```shell
# Example for a production storage cluster
# kubectl create -f cluster.yaml

# Example for a test cluster (non-redundant)
# kubectl create -f cluster-test.yaml

# Example for backing Mons and OSDs by PVs
# kubectl create -f cluster-on-pvc.yaml

# Example to connect to an external Ceph cluster (monitoring)
# kubectl create -f cluster-external.yaml

# Example to connect to an external Ceph cluster (full management)
# kubectl create -f cluster-external-management.yaml

# Example for a stretched cluster
# kubectl create -f cluster-stretched.yaml
```

--------------------------------

### Clone Rook Repository for Upgrade

Source: https://rook.github.io/docs/rook/v1.18/Upgrade/rook-upgrade

This command clones the specified branch of the Rook repository to obtain the necessary upgrade manifests. It uses a shallow clone for efficiency. Ensure you are in the correct directory after cloning.

```bash
git clone --single-branch --depth=1 --branch v1.18.4 https://github.com/rook/rook.git
cd rook/deploy/examples
```

--------------------------------

### View Ceph OSD Tree

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

This command retrieves a hierarchical view of the Ceph OSDs, showing their weights and relationships within the CRUSH map. This is essential for understanding data distribution and identifying potential bottlenecks.

```bash
ceph osd tree

```

--------------------------------

### Example CephCluster Status Output

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Cluster/ceph-cluster-crd

An example of the status section within a CephCluster CR, showing detailed information about the Ceph cluster's health (HEALTH_OK), capacity, current phase (Ready), and operational state.

```yaml
  status:
    ceph:
      health: HEALTH_OK
      lastChecked: "2021-03-02T21:22:11Z"
      capacity:
        bytesAvailable: 22530293760
        bytesTotal: 25757220864
        bytesUsed: 3226927104
        lastUpdated: "2021-03-02T21:22:11Z"
    message: Cluster created successfully
    phase: Ready
    state: Created
    storage:
      deviceClasses:
      - name: hdd
    version:
      image: quay.io/ceph/ceph:v19.2.3
      version: 16.2.6-0
    conditions:
    - lastHeartbeatTime: "2021-03-02T21:22:11Z"
      lastTransitionTime: "2021-03-02T21:21:09Z"
      message: Cluster created successfully
      reason: ClusterCreated
      status: "True"
      type: Ready
```

--------------------------------

### CephBlockPoolRadosNamespace Mirroring Configuration

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Block-Storage/ceph-block-pool-rados-namespace-crd

Example of configuring mirroring for a CephBlockPoolRadosNamespace.

```APIDOC
## PUT /apis/ceph.rook.io/v1/cephblockpoolradosnamespaces/{name}

### Description
Updates an existing CephBlockPoolRadosNamespace resource to enable mirroring.

### Method
PUT

### Endpoint
`/apis/ceph.rook.io/v1/cephblockpoolradosnamespaces/{name}`

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the CephBlockPoolRadosNamespace to update.

#### Request Body
- **apiVersion** (string) - Required - The version of the API.
- **kind** (string) - Required - The type of resource ('CephBlockPoolRadosNamespace').
- **metadata** (object) - Required - Contains the name and namespace of the CR.
  - **name** (string) - Required - The name for the RADOS namespace.
  - **namespace** (string) - Required - The Kubernetes namespace where the CR is created.
- **spec** (object) - Required - Specification for the RADOS namespace.
  - **blockPoolName** (string) - Required - The name of the CephBlockPool CR where the namespace is created.
  - **mirroring** (object) - Required - Mirroring configuration.
    - **mode** (string) - Required - Mirroring mode ('pool' or 'image').
    - **remoteNamespace** (string) - Optional - Name of the RADOS namespace on the peer cluster. Defaults to the local namespace name.
    - **snapshotSchedules** (array) - Optional - Schedules for snapshotting at the RADOS namespace level.
      - **interval** (string) - Required - Frequency of snapshots (e.g., '24h').
      - **startTime** (string) - Optional - ISO 8601 formatted start time for snapshots.

### Request Example
```json
{
  "apiVersion": "ceph.rook.io/v1",
  "kind": "CephBlockPoolRadosNamespace",
  "metadata": {
    "name": "namespace-a",
    "namespace": "rook-ceph"
  },
  "spec": {
    "blockPoolName": "replicapool",
    "mirroring": {
      "mode": "image",
      "remoteNamespace": "namespace-a",
      "snapshotSchedules": [
        {
          "interval": "24h",
          "startTime": "14:00:00-05:00"
        }
      ]
    }
  }
}
```

### Response
#### Success Response (200)
- **status** (object) - The status of the updated resource.
- **metadata** (object) - Metadata of the updated resource.

#### Response Example
```json
{
  "apiVersion": "ceph.rook.io/v1",
  "kind": "CephBlockPoolRadosNamespace",
  "metadata": {
    "name": "namespace-a",
    "namespace": "rook-ceph"
  },
  "spec": {
    "blockPoolName": "replicapool",
    "mirroring": {
      "mode": "image",
      "remoteNamespace": "namespace-a",
      "snapshotSchedules": [
        {
          "interval": "24h",
          "startTime": "14:00:00-05:00"
        }
      ]
    }
  },
  "status": {
    "phase": "Active"
  }
}
```
```

--------------------------------

### CephBlockPool Mirroring Configuration

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Block-Storage/ceph-block-pool-rados-namespace-crd

Example of configuring mirroring for a CephBlockPool.

```APIDOC
## PUT /apis/ceph.rook.io/v1/cephblockpools/{name}

### Description
Updates an existing CephBlockPool resource to enable mirroring.

### Method
PUT

### Endpoint
`/apis/ceph.rook.io/v1/cephblockpools/{name}`

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the CephBlockPool to update.

#### Request Body
- **apiVersion** (string) - Required - The version of the API.
- **kind** (string) - Required - The type of resource ('CephBlockPool').
- **metadata** (object) - Required - Contains the name and namespace of the CR.
  - **name** (string) - Required - The name for the Ceph Block Pool.
  - **namespace** (string) - Required - The Kubernetes namespace where the CR is created.
- **spec** (object) - Required - Specification for the Ceph Block Pool.
  - **replicated** (object) - Required - Replication settings.
    - **size** (integer) - Required - The replication size.
  - **mirroring** (object) - Required - Mirroring configuration.
    - **enabled** (boolean) - Required - Whether mirroring is enabled.
    - **mode** (string) - Required - Mirroring mode ('pool' or 'image').
    - **snapshotSchedules** (array) - Optional - Schedules for snapshotting.
      - **interval** (string) - Required - Frequency of snapshots (e.g., '24h').
      - **startTime** (string) - Optional - ISO 8601 formatted start time for snapshots.

### Request Example
```json
{
  "apiVersion": "ceph.rook.io/v1",
  "kind": "CephBlockPool",
  "metadata": {
    "name": "replicapool",
    "namespace": "rook-ceph"
  },
  "spec": {
    "replicated": {
      "size": 3
    },
    "mirroring": {
      "enabled": true,
      "mode": "image",
      "snapshotSchedules": [
        {
          "interval": "24h",
          "startTime": "14:00:00-05:00"
        }
      ]
    }
  }
}
```

### Response
#### Success Response (200)
- **status** (object) - The status of the updated resource.
- **metadata** (object) - Metadata of the updated resource.

#### Response Example
```json
{
  "apiVersion": "ceph.rook.io/v1",
  "kind": "CephBlockPool",
  "metadata": {
    "name": "replicapool",
    "namespace": "rook-ceph"
  },
  "spec": {
    "replicated": {
      "size": 3
    },
    "mirroring": {
      "enabled": true,
      "mode": "image",
      "snapshotSchedules": [
        {
          "interval": "24h",
          "startTime": "14:00:00-05:00"
        }
      ]
    }
  },
  "status": {
    "phase": "Active"
  }
}
```
```

--------------------------------

### CephBlockPoolRadosNamespace Creation

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Block-Storage/ceph-block-pool-rados-namespace-crd

Example of a CR to create a CephBlockPoolRadosNamespace on a specified CephBlockPool.

```APIDOC
## POST /apis/ceph.rook.io/v1/cephblockpoolradosnamespaces

### Description
Creates a CephBlockPoolRadosNamespace custom resource.

### Method
POST

### Endpoint
`/apis/ceph.rook.io/v1/cephblockpoolradosnamespaces`

### Parameters
#### Request Body
- **apiVersion** (string) - Required - The version of the API.
- **kind** (string) - Required - The type of resource, 'CephBlockPoolRadosNamespace'.
- **metadata** (object) - Required - Contains the name and namespace of the CR.
  - **name** (string) - Required - The name for the RADOS namespace.
  - **namespace** (string) - Required - The Kubernetes namespace where the CR is created.
- **spec** (object) - Required - Specification for the RADOS namespace.
  - **blockPoolName** (string) - Required - The name of the CephBlockPool CR where the namespace will be created.
  - **clusterID** (string) - Optional - ClusterID for CSI configuration. If not set, Rook auto-generates it.
  - **mirroring** (object) - Optional - Configuration for mirroring the RADOS namespace.
    - **mode** (string) - Required - Mirroring mode ('pool' or 'image').
    - **remoteNamespace** (string) - Optional - Name of the RADOS namespace on the peer cluster. Defaults to the local namespace name.
    - **snapshotSchedules** (array) - Optional - Schedules for snapshotting at the RADOS namespace level.
      - **interval** (string) - Required - Frequency of snapshots (e.g., '24h', '30m').
      - **startTime** (string) - Optional - ISO 8601 formatted start time for snapshots.

### Request Example
```json
{
  "apiVersion": "ceph.rook.io/v1",
  "kind": "CephBlockPoolRadosNamespace",
  "metadata": {
    "name": "namespace-a",
    "namespace": "rook-ceph"
  },
  "spec": {
    "blockPoolName": "replicapool"
  }
}
```

### Response
#### Success Response (200 or 201)
- **status** (object) - The status of the created resource.
- **metadata** (object) - Metadata of the created resource.

#### Response Example
```json
{
  "apiVersion": "ceph.rook.io/v1",
  "kind": "CephBlockPoolRadosNamespace",
  "metadata": {
    "name": "namespace-a",
    "namespace": "rook-ceph",
    "uid": "...",
    "creationTimestamp": "..."
  },
  "spec": {
    "blockPoolName": "replicapool"
  },
  "status": {
    "phase": "Created"
  }
}
```
```

--------------------------------

### Deploy Rook Operator

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/example-configurations

Deploys the Rook operator, which manages Rook clusters. The `operator.yaml` file contains common settings for production deployments. For OpenShift environments, `operator-openshift.yaml` should be used, and OpenShift prerequisites should be reviewed.

```shell
kubectl create -f operator.yaml
```

```shell
oc create -f operator-openshift.yaml
```

--------------------------------

### StorageClass Creation with ClusterID

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Block-Storage/ceph-block-pool-rados-namespace-crd

Example of creating a StorageClass that utilizes a specific RADOS namespace, using the clusterID obtained from the CephBlockPoolRadosNamespace status.

```APIDOC
## POST /apis/storage.k8s.io/v1/storageclasses

### Description
Creates a StorageClass resource that provisions PersistentVolumes using RBD on a specific RADOS namespace within a Ceph cluster.

### Method
POST

### Endpoint
`/apis/storage.k8s.io/v1/storageclasses`

### Parameters
#### Query Parameters
- **clusterID** (string) - Required - The unique identifier for the Ceph cluster, obtained from the CephBlockPoolRadosNamespace status.
- **pool** (string) - Required - The name of the Ceph Block Pool.
- **provisioner** (string) - Required - The CSI provisioner name (e.g., `rook-ceph.rbd.csi.ceph.com`).

#### Request Body
- **apiVersion** (string) - Required - The version of the API (`storage.k8s.io/v1`).
- **kind** (string) - Required - The type of resource (`StorageClass`).
- **metadata** (object) - Required - Contains the name of the StorageClass.
  - **name** (string) - Required - The name for the StorageClass.
- **parameters** (object) - Required - Parameters for the provisioner.
  - **clusterID** (string) - Required - The cluster ID obtained from the CephBlockPoolRadosNamespace status.
  - **pool** (string) - Required - The name of the Ceph Block Pool.
  - **...** (any other required parameters for the provisioner)

### Request Example
```json
{
  "apiVersion": "storage.k8s.io/v1",
  "kind": "StorageClass",
  "metadata": {
    "name": "rook-ceph-block-rados-ns"
  },
  "parameters": {
    "clusterID": "80fc4f4bacc064be641633e6ed25ba7e",
    "pool": "replicapool"
  }
}
```

### Response
#### Success Response (200 or 201)
- **status** (object) - The status of the created resource.
- **metadata** (object) - Metadata of the created resource.

#### Response Example
```json
{
  "apiVersion": "storage.k8s.io/v1",
  "kind": "StorageClass",
  "metadata": {
    "name": "rook-ceph-block-rados-ns",
    "uid": "...",
    "creationTimestamp": "..."
  },
  "parameters": {
    "clusterID": "80fc4f4bacc064be641633e6ed25ba7e",
    "pool": "replicapool"
  }
}
```
```

--------------------------------

### Build Local Rook Image

Source: https://rook.github.io/docs/rook/v1.18/Contributing/development-environment

Builds a local Rook image using the `make` command within the Rook source directory. The `BUILD_REGISTRY=local` flag ensures the image is built for local use and tagged appropriately for the Minikube environment.

```bash
cd <your_rook_src_directory>
make BUILD_REGISTRY=local
```

--------------------------------

### Configure Rook Cluster Manifests with sed

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

Modifies Rook configuration files (`common.yaml`, `operator.yaml`, `cluster.yaml`) using `sed` to set custom operator and cluster namespaces. This allows for flexible deployment of Rook components.

```bash
cd deploy/examples

export ROOK_OPERATOR_NAMESPACE="rook-ceph"
export ROOK_CLUSTER_NAMESPACE="rook-ceph"

sed -i.bak \
    -e "s/\(.*\):.*# namespace:operator/\1: $ROOK_OPERATOR_NAMESPACE # namespace:operator/g" \
    -e "s/\(.*\):.*# namespace:cluster/\1: $ROOK_CLUSTER_NAMESPACE # namespace:cluster/g" \
    -e "s/\(.*serviceaccount\):.*:\(.*\) # serviceaccount:namespace:operator/\1:$ROOK_OPERATOR_NAMESPACE:\2 # serviceaccount:namespace:operator/g" \
    -e "s/\(.*serviceaccount\):.*:\(.*\) # serviceaccount:namespace:cluster/\1:$ROOK_CLUSTER_NAMESPACE:\2 # serviceaccount:namespace:cluster/g" \
    -e "s/\(.*\): [-_A-Za-z0-9]*\.\(.*\) # driver:namespace:cluster/\1: $ROOK_CLUSTER_NAMESPACE.\2 # driver:namespace:cluster/g" \
  common.yaml operator.yaml cluster.yaml # add other files or change these as desired for your config

# You need to use `apply` for all Ceph clusters after the first if you have only one Operator
kubectl apply -f crds.yaml -f common.yaml -f operator.yaml -f cluster.yaml # add other files as desired for yourconfig
```

--------------------------------

### Get PVC Details for Mirroring Setup

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Block-Storage-RBD/rbd-mirroring

Retrieves information about a PersistentVolumeClaim (PVC) to confirm its status, storage class, and other attributes before configuring mirroring. Assumes a PVC named 'rbd-pvc' exists.

```bash
--- 
## Enable mirroring on a PVC¶
Below guide assumes that we have a PVC (rbd-pvc) in BOUND state; created using **StorageClass** with `Retain` reclaimPolicy.
```
1
2
3
```
| ```
[cluster-1]$ kubectl get pvc
NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
rbd-pvc   Bound    pvc-65dc0aac-5e15-4474-90f4-7a3532c621ec   1Gi        RWO            csi-rbd-sc   44s

```
```

--------------------------------

### Deploy Object Store, Create User, and Get Credentials (Bash)

Source: https://context7.com/context7/rook_github_io_rook_v1_18/llms.txt

Commands to deploy the Ceph object store, wait for its RGW pods, create an object store user, and retrieve the S3 access key and secret key. It also shows how to find the S3 endpoint service.

```bash
# Create object store
kubectl create -f object-store.yaml

# Wait for RGW pods
kubectl -n rook-ceph get pod -l app=rook-ceph-rgw

# Create object store user
kubectl create -f - <<EOF
apiVersion: ceph.rook.io/v1
kind: CephObjectStoreUser
metadata:
  name: my-user
  namespace: rook-ceph
spec:
  store: my-store
  displayName: "My Object Store User"
EOF

# Get S3 credentials
kubectl -n rook-ceph get secret rook-ceph-object-user-my-store-my-user -o jsonpath='{.data.AccessKey}' | base64 --decode
kubectl -n rook-ceph get secret rook-ceph-object-user-my-store-my-user -o jsonpath='{.data.SecretKey}' | base64 --decode

# Get S3 endpoint
kubectl -n rook-ceph get service rook-ceph-rgw-my-store
```

--------------------------------

### Get rook-config-override ConfigMap - YAML

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

Retrieves the current `rook-config-override` ConfigMap in YAML format from the `rook-ceph` namespace. This is useful for inspecting the default or current configuration settings applied to Ceph daemons.

```bash
kubectl -n rook-ceph get ConfigMap rook-config-override -o yaml
```

--------------------------------

### Deploy Common Rook/Ceph Resources

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/example-configurations

Deploys the Custom Resource Definitions (CRDs) and other common resources required for Rook and Ceph deployments. These resources are typically the same for most deployments. It requires the `crds.yaml` and `common.yaml` files.

```shell
kubectl create -f crds.yaml -f common.yaml
```

--------------------------------

### Collect Rook Logs from All Pods and Containers

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/ceph-configuration

Retrieves logs from all containers within all Rook pods in a specified Kubernetes namespace (`rook-ceph`). The logs are then printed to standard output, allowing for easy redirection or compression.

```bash
for p in $(kubectl -n rook-ceph get pods -o jsonpath='{.items[*].metadata.name}')
do
    for c in $(kubectl -n rook-ceph get pod ${p} -o jsonpath='{.spec.containers[*].name}')
    do
        echo "BEGIN logs from pod: ${p} ${c}"
        kubectl -n rook-ceph logs -c ${c} ${p}
        echo "END logs from pod: ${p} ${c}"
    done
done
```

--------------------------------

### Check Ceph Cluster Health Status

Source: https://rook.github.io/docs/rook/v1.18/Getting-Started/quickstart

This command connects to the Rook toolbox (or directly if configured) to display the current health status of the Ceph cluster. It reports on the state of monitors, managers, and OSDs, and indicates if the cluster is in a `HEALTH_OK` state.

```bash
$ ceph status
```

--------------------------------

### Build Rook Project and Fetch Remotes

Source: https://rook.github.io/docs/rook/v1.18/Contributing/development-flow

Fetches all remote tags to synchronize with the latest upstream changes and then builds the Rook project. This is a crucial step before starting development to ensure your local environment is up-to-date. May require setting GOPROXY in Linux environments.

```shell
# Fetch all remotes
git fetch -a
make build
```

--------------------------------

### Ephemeral Volume Support Example in Pod Spec

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Ceph-CSI/ceph-csi-drivers

This snippet demonstrates how to define an ephemeral volume using a volumeClaimTemplate within a pod specification. The volume is provisioned when the pod starts and destroyed when the pod is deleted. It requires the GenericEphemeralVolume feature gate to be enabled.

```yaml
kind: Pod
apiVersion: v1
metadata:
  name: mypod
spec:
  containers:
    - name: mycontainer
      image: nginx
      volumeMounts:
        - mountPath: "/data"
          name: mypvc
  volumes:
    - name: mypvc
      ephemeral:
        volumeClaimTemplate:
          spec:
            accessModes: ["ReadWriteOnce"]
            storageClassName: "rook-ceph-block"
            resources:
              requests:
                storage: 1Gi
```

--------------------------------

### Configure Mirroring on CephBlockPoolRadosNamespace

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Block-Storage/ceph-block-pool-rados-namespace-crd

This example configures mirroring for a CephBlockPoolRadosNamespace CR. It specifies the mirroring mode and the remote namespace on the peer cluster. Snapshot schedules can also be defined at the namespace level. This CRD setting works in conjunction with the mirroring configuration on the parent CephBlockPool.

```yaml
apiVersion: ceph.rook.io/v1
kind: CephBlockPoolRadosNamespace
metadata:
  name: namespace-a
  namespace: rook-ceph # namespace:cluster
spec:
  # The name of the CephBlockPool CR where the namespace is created.
  blockPoolName: replicapool
  mirroring:
    mode: image
    remoteNamespace: namespace-a # default is the same as the local rados namespace
    # schedule(s) of snapshot
    snapshotSchedules:
      - interval: 24h # daily snapshots
        startTime: 14:00:00-05:00

```

--------------------------------

### Create CephBlockPoolRadosNamespace CR

Source: https://rook.github.io/docs/rook/v1.18/CRDs/Block-Storage/ceph-block-pool-rados-namespace-crd

This example demonstrates the basic structure of a CephBlockPoolRadosNamespace Custom Resource (CR) in Kubernetes. It specifies the name of the namespace and the parent CephBlockPool where it will be created. This CRD allows for tenant isolation and sharing of a single Ceph cluster across multiple Kubernetes clusters.

```yaml
apiVersion: ceph.rook.io/v1
kind: CephBlockPoolRadosNamespace
metadata:
  name: namespace-a
  namespace: rook-ceph # namespace:cluster
spec:
  # The name of the CephBlockPool CR where the namespace is created.
  blockPoolName: replicapool

```

--------------------------------

### Configure CephBlockPool with Default PG/PGP Counts

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/configuration

This example demonstrates how to define default placement group (PG) and placement group for pools (PGP) counts for a CephBlockPool resource in Rook. These parameters help in pre-allocating resources for pools and should be set based on expected OSD counts.

```yaml
apiVersion: ceph.rook.io/v1
kind: CephBlockPool
metadata:
  name: ceph-block-pool-test
  namespace: rook-ceph
spec:
  deviceClass: hdd
  replicated:
    size: 3
spec:
  parameters:
    pg_num: '128' # create the pool with a pre-configured placement group number
    pgp_num: '128' # this should at least match `pg_num` so that all PGs are used
```

--------------------------------

### Vault Policy for Rook

Source: https://rook.github.io/docs/rook/v1.18/Storage-Configuration/Advanced/key-management-system

An example of a Vault policy that grants Rook the necessary capabilities to manage secrets. This policy allows create, read, update, delete, and list operations on paths starting with 'rook/'. It also grants read access to system mounts.

```hcl
path "rook/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}
path "sys/mounts" {
capabilities = ["read"]
}
```

--------------------------------

### Deploy and Use Ceph Filesystem (Bash)

Source: https://context7.com/context7/rook_github_io_rook_v1_18/llms.txt

Commands to create a Ceph filesystem, wait for its metadata server pods to become ready, and then provision a PersistentVolumeClaim that can be used with ReadWriteMany access mode in other pods.

```bash
# Create filesystem
kubectl create -f filesystem.yaml

# Wait for MDS pods
kubectl -n rook-ceph get pod -l app=rook-ceph-mds
# NAME                                    READY   STATUS    RESTARTS   AGE
# rook-ceph-mds-myfs-7d59fdfcf4-h8kw9     1/1     Running   0          12s
# rook-ceph-mds-myfs-7d59fdfcf4-kgkjp     1/1     Running   0          12s

# Use in pod with ReadWriteMany access
kubectl create -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-data
spec:
  storageClassName: rook-cephfs
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
EOF
```
Tokens:
Raw