### Install Helm Chart

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/deploy/helm/charts/README.md

Basic command to install the OpenEBS LocalPV Provisioner Helm chart.

```console
helm install openebs-localpv openebs-localpv/localpv-provisioner --namespace openebs --create-namespace
```

--------------------------------

### Install Chart with Node Affinity Labels

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/deploy/helm/charts/README.md

Sample command to install the provisioner with custom node affinity labels for the hostpath StorageClass.

```console
helm install openebs-localpv openebs-localpv/localpv-provisioner --namespace openebs --create-namespace \
	--set-string hostpathClass.nodeAffinityLabels="{openebs.io/node-affinity-key-1,openebs.io/node-affinity-key-2}"
```

--------------------------------

### Install LocalPV Provisioner Helm Chart

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/deploy/helm/charts/README.md

Use this command to install the localpv-provisioner Helm chart, specifying a custom values file and namespace.

```bash
helm install <release-name> -f values.yaml --namespace openebs openebs-localpv/localpv-provisioner
```

--------------------------------

### Install Nix Package Manager

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Use this command to install the Nix package manager. Do not use sudo. This is the recommended way to set up the development environment.

```bash
curl -L https://nixos.org/nix/install | sh
```

--------------------------------

### Install Dynamic-LocalPV-Provisioner using Operator YAMLs

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/microk8s.md

Installs the Dynamic-LocalPV-Provisioner using the openebs-operator-lite.yaml and openebs-lite-sc.yaml files. This method also creates default directories under '/var/snap/microk8s/common' by modifying the default paths.

```bash
#Apply openebs-operator-lite.yaml
curl -fSsL https://openebs.github.io/charts/openebs-operator-lite.yaml | sed 's|\(/var/openebs\)|/var/snap/microk8s/common\1|g' | kubectl apply -f -
#Apply openebs-lite-sc.yaml
curl -fSsL https://openebs.github.io/charts/openebs-lite-sc.yaml | sed 's|\(/var/openebs\)|/var/snap/microk8s/common\1|g' | kubectl apply -f -
```

--------------------------------

### Install XFS Utilities

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/use-xfs-fs-with-loop-device.md

Install the necessary tools for managing XFS filesystems. This command is for Debian/Ubuntu-based systems.

```bash
sudo apt update
sudo apt-get install -y xfsprogs
```

--------------------------------

### Install Ginkgo Binary

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/tests/README.md

Installs the latest Ginkgo binary, a testing framework used for running the BDD tests.

```bash
$ go install -mod=mod github.com/onsi/ginkgo/v2/ginkgo@v2.14.0
```

--------------------------------

### Install Ubuntu/Debian Packages

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/tests/README.md

Installs necessary packages for running tests on Ubuntu/Debian systems, including xfsprogs, quota, and the extra kernel module for quota.

```bash
$ sudo apt-get update && sudo apt-get install -y xfsprogs quota linux-image-extra-virtual
```

--------------------------------

### Install OpenEBS LocalPV using Helm

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/quickstart.md

Installs the OpenEBS LocalPV provisioner using the official Helm chart. Ensure Helm is configured with the openebs repository.

```bash
#helm repo add openebs https://openebs.github.io/openebs
#helm repo update
helm install openebs openebs/openebs -n openebs --create-namespace
```

--------------------------------

### Start Minikube with No VM Driver

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/minikube.md

Use this command to start minikube without a VM driver. This is required for the node-disk-manager DaemonSet Pods to access host directories.

```bash
minikube start --driver=none
```

--------------------------------

### Install OpenEBS LocalPV Provisioner

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/enable-xfs-quota.md

Apply the Lite operator manifest to install the OpenEBS LocalPV provisioner. This command is used to deploy the necessary components for the provisioner.

```bash
kubectl apply -f https://openebs.github.io/charts/openebs-operator-lite.yaml
```

--------------------------------

### Run Make Bootstrap for Dependencies

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

On non-nix environments, use 'make bootstrap' to install all necessary Go tools and dependencies required for building and development.

```bash
make bootstrap
```

--------------------------------

### Install CentOS/RHEL Packages

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/tests/README.md

Installs necessary packages for running tests on CentOS/RHEL systems, including xfsprogs and quota.

```bash
$ sudo yum install -y xfsprogs quota
```

--------------------------------

### Deploy Hostpath LocalPV Provisioner using Helm

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/design/hostpath_localpv_provisioner.md

Install the Hostpath LocalPV Provisioner using Helm. Ensure the 'openebs' chart is available and specify the target namespace.

```bash
helm install openebs openebs/openebs -n openebs
```

--------------------------------

### Install OpenEBS LocalPV Provisioner Chart

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/deploy/helm/charts/README.md

Install the OpenEBS LocalPV provisioner Helm chart onto your Kubernetes cluster. Replace [RELEASE_NAME] and [NAMESPACE] with your desired values.

```console
helm install [RELEASE_NAME] openebs-localpv/localpv-provisioner --namespace [NAMESPACE] --create-namespace
```

--------------------------------

### Install xfsprogs on Ubuntu/Debian

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Installs the xfsprogs package required for XFS quota management on Ubuntu and Debian-based systems.

```bash
sudo apt-get update
sudo apt-get install -y xfsprogs
```

--------------------------------

### Install MinIO Operator and Tenant with Helm

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/minio.md

Installs the MinIO Operator and a single-server MinIO tenant using the official Helm chart. Configures access keys, storage class, and volume settings for the tenant. Ensure Helm repositories are updated before running.

```bash
#helm repo add minio https://operator.min.io/
#helm repo update
helm install minio-operator minio/minio-operator \
	--namespace minio-operator \
	--create-namespace \
	--set tenants[0].name="openebs-backup-minio" \
	--set tenants[0].namespace="velero" \
	--set tenants[0].certificate.requestAutoCert=false \
	--set tenants[0].pools[0].servers=1 \
	--set tenants[0].pools[0].volumesPerServer=4 \
	--set tenants[0].pools[0].size=10Gi \
	--set tenants[0].pools[0].storageClassName="openebs-hostpath" \
	--set tenants[0].secrets.enabled=true \
	--set tenants[0].secrets.name="openebs-backup-minio-creds-secret" \
	--set tenants[0].secrets.accessKey="minio" \
	--set tenants[0].secrets.secretKey="minio123"
```

--------------------------------

### Install xfsprogs on RHEL/CentOS

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Installs the xfsprogs package required for XFS quota management on RHEL and CentOS systems. For Fedora, use 'dnf'.

```bash
sudo yum install -y xfsprogs
```

```bash
#Using Fedora?
#sudo dnf install -y xfsprogs
```

--------------------------------

### Start NixOS Shell for Kubernetes Cluster

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Launch a NixOS shell to set up a single-node K3s cluster within a headless QEMU virtual machine. This environment is suitable for running CI tests.

```bash
$ nixos-shell
Welcome to NixOS 24.11 (Vicuna)!

[  OK  ] Created slice Slice /system/getty.
[  OK  ] Created slice Slice /system/modprobe.
[  OK  ] Created slice Slice /system/serial-getty
....
[  OK  ] Reached target Network is Online.
         Starting Docker Application Container Engine...
         Starting k3s service...
[  OK  ] Started Docker Application Container Engine.

<<< Welcome to NixOS 24.11.712512.3f0a8ac25fb6 (x86_64) - hvc0 >>>
Log in as "root" with an empty password.
Run 'nixos-help' for the NixOS manual.
nixos login: root

```

--------------------------------

### Install Velero with Restic using Helm

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/velero_with_restic.md

Install Velero and Restic using the Velero Helm chart. This command configures Velero to use the MinIO instance for backups and enables Restic.

```bash
$ #helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
$ #helm repo update
$ helm install velero vmware-tanzu/velero \
 --namespace velero \
 --create-namespace \
 --set-file credentials.secretContents.cloud=$(pwd)/minio-credentials \
 --set configuration.provider="aws" \
 --set configuration.backupStorageLocation.name="default" \
 --set configuration.backupStorageLocation.bucket="velero" \
 --set configuration.backupStorageLocation.config.region="minio" \
 --set configuration.backupStorageLocation.config.s3ForcePathStyle="true" \
 --set configuration.backupStorageLocation.config.s3Url="http://minio.velero.svc:80" \
 --set backupsEnabled=true \
 --set snapshotsEnabled=false \
 --set deployRestic=true \
 --set initContainers[0].name=velero-plugin-for-aws \
 --set initContainers[0].image=velero/velero-plugin-for-aws:latest \
 --set initContainers[0].volumeMounts[0].mountPath=/target \
 --set initContainers[0].volumeMounts[0].name=plugins
```

--------------------------------

### Install OpenEBS LocalPV with Custom Hostpath

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/quickstart.md

Installs the OpenEBS LocalPV provisioner with a custom base path for hostpath volumes. This modifies the 'BasePath' setting for the 'openebs-hostpath' StorageClass.

```bash
helm install openebs openebs/openebs -n openebs --create-namespace \
 --set localpv-provisioner.hostpathClass.basePath=<custom-hostpath>
```

--------------------------------

### Verify OpenEBS Addon Installation

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/microk8s.md

Verifies the creation of Dynamic-LocalPV-Provisioner components after enabling the OpenEBS addon. Checks for pods in the 'openebs' namespace and available storage classes.

```bash
microk8s kubectl get pods -n openebs
```

```bash
microk8s kubectl get storageclass
```

--------------------------------

### Install OpenEBS Helm Chart with Custom Paths

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/microk8s.md

Installs the OpenEBS Helm chart with custom base paths for local storage, sparse files, and variable directories. This command sets the default directories under '/var/snap/microk8s/common'.

```bash
microk8s helm3 install openebs openebs/openebs -n openebs --create-namespace \
 --set localprovisioner.basePath="/var/snap/microk8s/common/var/openebs/local" \
 --set ndm.sparse.path="/var/snap/microk8s/common/var/openebs/sparse" \
 --set varDirectoryPath.baseDir="/var/snap/microk8s/common/var/openebs"
```

--------------------------------

### Setup RBAC and CRDs with kubectl

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/e2e-tests/README.md

Clone the repository and navigate to the 'e2e-tests/hack' directory to set up necessary RBAC and custom resource definitions using kubectl.

```bash
git clone https://github.com/openebs/dynamic-localpv-provisioner.git
cd dynamic-localpv-provisioner/e2e-tests
kubectl apply -f hack/rbac.yaml
kubectl apply -f hack/crds.yaml
```

--------------------------------

### Verify Provisioner Pods

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/enable-xfs-quota.md

Check the status of the OpenEBS LocalPV provisioner pods in the 'openebs' namespace. This helps confirm that the installation was successful and the pods are running.

```bash
$ kubectl get pods -n openebs

NAME                                           READY   STATUS    RESTARTS       AGE
openebs-localpv-provisioner-6ddbd95d4d-htp7g   1/1     Running       0          7m12s
```

--------------------------------

### Verify Velero and Restic Installation

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/velero_with_restic.md

Check the status of Velero and Restic components after installation. This command lists secrets, backup storage locations, and pods in the 'velero' namespace.

```bash
$ kubectl get secrets,backupstoragelocations,pods -n velero

NAME                                       TYPE                                  DATA   AGE
secret/default-token-kqlr8                 kubernetes.io/service-account-token   3      7h17m
secret/openebs-backup-minio-creds-secret   Opaque                                2      7h9m
secret/operator-tls                        Opaque                                1      7h9m
secret/operator-webhook-secret             Opaque                                3      7h9m
secret/sh.helm.release.v1.velero.v1        helm.sh/release.v1                    1      6h57m
secret/velero                              Opaque                                1      6h57m
secret/velero-restic-credentials           Opaque                                1      6h56m
secret/velero-server-token-6r4sq           kubernetes.io/service-account-token   3      6h57m

NAME                                      PHASE       LAST VALIDATED   AGE     DEFAULT
backupstoragelocation.velero.io/default   Available   26s              6h57m   true

NAME                              READY   STATUS    RESTARTS   AGE
pod/openebs-backup-minio-ss-0-0   1/1     Running   0          7h9m
pod/restic-2xwsf                  1/1     Running   0          6h57m
pod/velero-7dd57b857-2gd25        1/1     Running   0          6h57m
```

--------------------------------

### Get All Backups

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/backup.md

List all created backups and their current status. This command helps in verifying the success or failure of backup operations.

```bash
$ ./velero backup get
NAME                STATUS      ERRORS   WARNINGS   CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
my-localpv-backup   Completed   0        0          2021-09-04 01:13:36 +0000 UTC   29d       default            <none>
```

--------------------------------

### Run Integration Tests with Reset

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Start integration tests with a clean slate by using the '--reset' flag, which ensures a fresh environment before test execution.

```bash
./ci/ci-test.sh run --reset
```

--------------------------------

### Get e2e-test pod logs

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/e2e-tests/README.md

After creating the Kubernetes job, monitor the test execution by retrieving the logs of the instantiated pod in the 'e2e' namespace.

```bash
kubectl get pods -n e2e
kubectl logs -f <application-pod-failure-xxxxx-xxxxx> -n e2e
```

--------------------------------

### Create New Feature/Bugfix Branch

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Start new development by creating a branch from 'develop'. Name your branch using the format XX-descriptive, where XX is the issue number.

```bash
git checkout develop
# Make sure the develop is rebased with the latest changes as described in previous step.
git checkout -b 1234-fix-developer-docs
Switched to a new branch '1234-fix-developer-docs'
```

--------------------------------

### Enable Helm3 Addon in MicroK8s

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/microk8s.md

Enables the Helm3 addon in MicroK8s, which is required for installing the OpenEBS Helm chart.

```bash
microk8s enable helm3
```

--------------------------------

### Get Node Information

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/modify-xfs-quota.md

Retrieve details about the specific node where the hostpath volume is located, using its hostname.

```bash
kubectl get node -l 'kubernetes.io/hostname in (storage-node-2)'
```

--------------------------------

### Run MinIO Client Container

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/velero_with_restic.md

Execute the 'minio/mc' container to interact with the MinIO instance. This command starts a TTY shell inside the container for running commands.

```bash
kubectl -n velero run minio-client --image=minio/mc --rm -it --command -- /bin/sh
```

--------------------------------

### Mount Volume to Application Pod

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/enable-xfs-quota.md

Configure a Pod to mount the PersistentVolumeClaim. This example uses a BusyBox container that writes to a file within the mounted volume, demonstrating volume usage and triggering quota application.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: busybox
spec:
  volumes:
  - name: local-storage
    persistentVolumeClaim:
      claimName: local-hostpath-xfs
  containers:
  - name: busybox
    image: busybox
    command:
       - sh
       - -c
       - 'while true; do echo "`date` [`hostname`] Hello from OpenEBS Local PV." >> /mnt/store/greet.txt; sleep $(($RANDOM % 5 + 300)); done'
    volumeMounts:
    - mountPath: /mnt/store
      name: local-storage
```

--------------------------------

### Fetch and Rebase from Upstream

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Before starting development, fetch the latest changes from the upstream repository and rebase your feature branch onto the develop branch to integrate recent updates and resolve conflicts.

```bash
$ git fetch upstream
$ git rebase upstream/develop
```

--------------------------------

### Hostpath LocalPV StorageClass Example

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/design/hostpath_localpv_provisioner.md

Defines a Kubernetes StorageClass for the hostpath provisioner. Configure the base path for hostpath directories and node affinity labels using annotations. Ensure the provisioner name is unique for correct processing.

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: openebs-hostpath
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: "hostpath"
      - name: BasePath
        value: "/var/openebs/local/"
provisioner: openebs.io/local
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

```

--------------------------------

### Backup and Regenerate grub.cfg

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Creates a backup of the current grub.cfg and then generates a new configuration file to include the updated mount options.

```bash
sudo cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg.backup
```

```bash
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
```

--------------------------------

### Configure MinIO Client and Add Host

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/velero_with_restic.md

Configure the MinIO client within the container to connect to the MinIO service and add it as a host. This step is necessary before creating buckets.

```bash
$ mc config host add velero http://minio.velero.svc:80 'minio' 'minio123'

mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.
Added `velero` successfully.
```

--------------------------------

### Enter Nix Shell for Dependencies

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Use the nix-shell to automatically set up a development environment with all required packages. Ensure your zshrc configurations do not interfere with Go environment variables set by the nix-shell.

```bash
$ nix-shell
go: downloading ....
....

```

--------------------------------

### Create MinIO Credentials File

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/velero_with_restic.md

Create a file named 'minio-credentials' to store the MinIO access key and secret key. This file will be passed to Velero.

```bash
$ cat << EOF > ./minio-credentials
[default]
aws_access_key_id=minio
aws_secret_access_key=minio123
EOF
```

--------------------------------

### Run Unit Tests with Make

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Execute unit tests using the 'make test' command. This command also runs go fmt and go test, providing feedback on code formatting and test results, including coverage.

```bash
$ make test
--> Running go fmt
--> Running go test
?       github.com/openebs/dynamic-localpv-provisioner/cmd      [no test files]
ok      github.com/openebs/dynamic-localpv-provisioner/pkg/driver       0.011s  coverage: 0.5% of statements
?       github.com/openebs/dynamic-localpv-provisioner/pkg/equality     [no test files]
...
```

--------------------------------

### Create StorageClass for Hostpath Provisioner

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/design/hostpath_localpv_provisioner.md

Create a StorageClass resource for the Hostpath LocalPV Provisioner. This involves applying a YAML configuration file.

```bash
kubectl create -f <sc-yaml-file>
```

--------------------------------

### Create Application Pod consuming PersistentVolumeClaim

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/design/hostpath_localpv_provisioner.md

Deploy an application Pod that consumes the previously created PersistentVolumeClaim. The volume will be mounted into the Pod.

```bash
kubectl create -f <pod-yaml-file>
```

--------------------------------

### Get Velero Pod Name

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/backup.md

Retrieve the name of the Velero Pod running in the 'velero' namespace to facilitate exec commands.

```bash
$ kubectl -n velero get pods

NAME                          READY   STATUS    RESTARTS   AGE
openebs-backup-minio-ss-0-0   1/1     Running   0          7h23m
restic-2xwsf                  1/1     Running   0          7h12m
velero-7dd57b857-2gd25        1/1     Running   0          7h12m
```

--------------------------------

### Recreate NixOS VM for Testing

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

If encountering persistent issues, delete the 'nixos.qcow2' file and restart the nixos-shell to obtain a completely new and clean virtual machine environment for testing.

```bash
rm nixos.qcow2
nixos-shell
```

--------------------------------

### Create a LocalPV Backup with Restic

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/backup.md

Initiate a backup of specified namespaces using Restic for volumes. The `--wait` flag ensures the command waits for the backup to complete.

```bash
$ ./velero create backup my-localpv-backup --include-namespaces <app-namespace> --default-volumes-to-restic --wait

Backup request "my-localpv-backup" submitted successfully.
Waiting for backup to complete. You may safely press ctrl-c to stop waiting - your backup will continue in the background.
....
Backup completed with status: Completed. You may check for more information using the commands `velero backup describe my-localpv-backup` and `velero backup logs my-localpv-backup`.
```

--------------------------------

### Run Local PV Provisioner BDD Tests

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/tests/README.md

Executes the BDD tests for the Local PV Provisioner from the tests directory. Requires root privileges and the KUBECONFIG environment variable to be set.

```bash
$ cd <repo-directory>/tests
$ sudo -E env "PATH=$PATH" ginkgo -v
```

--------------------------------

### Get Velero Restore Status

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/restore.md

Retrieves the status of ongoing or completed Velero restore operations. This helps in verifying the success and components restored.

```bash
./velero restore get
```

--------------------------------

### Check Existing Mount Options for 'pquota'

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Verifies if the specified device's mount options already include 'pquota' or 'prjquota'.

```bash
$ sudo mount | grep "^/dev/nvme0n1p1"

/dev/nvme0n1p1 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
```

--------------------------------

### Mount XFS Loop Device with Project Quota

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/use-xfs-fs-with-loop-device.md

Mount the formatted sparse file as a loop device. The '-o pquota' option enables project quota support for the filesystem.

```bash
sudo mount -o loop,rw xfs.32M -o pquota /var/openebs/local
```

--------------------------------

### Retrieve e2e-test results

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/e2e-tests/README.md

Test results are stored in a custom resource ('e2eresult') named after the experiment. Use kubectl to get the resource and check its 'phase' and 'result' status.

```bash
kubectl get e2er
kubectl get e2er application-pod-failure -n e2e --no-headers -o custom-columns=:.spec.testStatus.phase
kubectl get e2er application-pod-failure -n e2e --no-headers -o custom-columns=:.spec.testStatus.result
```

--------------------------------

### Create MinIO Bucket

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/velero_with_restic.md

Create a dedicated bucket named 'velero' within the MinIO instance using the configured MinIO client. This bucket will store volume backups.

```bash
$ mc mb -p velero/velero

Bucket created successfully `velero/velero`.
```

--------------------------------

### Clean and Restart NixOS VM

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

If the virtual machine becomes unstable or damaged, delete the 'nixos.qcow2' file and restart the nixos-shell to create a fresh environment.

```bash
# Leave the vm terminal and run:
rm nixos.qcow2
nixos-shell
```

--------------------------------

### Add and Update OpenEBS Helm Chart Repository

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/microk8s.md

Adds the OpenEBS Helm chart repository and updates the local Helm chart repository cache. This is a prerequisite for installing the OpenEBS Helm chart.

```bash
microk8s helm3 repo add openebs https://openebs.github.io/charts
```

```bash
microk8s helm3 repo update
```

--------------------------------

### Create Velero Namespace

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/minio.md

Creates a Kubernetes namespace named 'velero' to host the MinIO tenant and Velero resources.

```bash
kubectl create namespace velero
```

--------------------------------

### StorageClass with NodeAffinityLabels

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/nodeaffinitylabels.md

Define a StorageClass for Hostpath LocalPV and specify custom node labels for unique identification. This configuration ensures that the provisioner correctly identifies nodes, particularly in complex cluster setups.

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-hostpath
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: "hostpath"
      - name: NodeAffinityLabels
        list:
          - "openebs.io/custom-node-unique-id"
provisioner: openebs.io/local
volumeBindingMode: WaitForFirstConsumer

```

--------------------------------

### Create Kubernetes Job for Experiment

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/e2e-tests/experiments/localpv-provisioner/README.md

Create the Kubernetes job to run the experiment after updating test-specific values in the run_e2e_test.yml file. Environment variable descriptions are available as comments within the file.

```bash
kubectl create -f run_e2e_test.yml
```

--------------------------------

### Create Mount Directory

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/use-xfs-fs-with-loop-device.md

Create the directory where the XFS filesystem will be mounted. Ensure you are in the parent directory for relative path operations.

```bash
sudo mkdir -p /var/openebs/local
cd /var/openebs
```

--------------------------------

### Clone Dynamic LocalPV Provisioner Fork

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Clone your forked repository to your local machine. Replace '$user' with your GitHub username. You can use SSH instead of HTTPS.

```bash
mkdir -p ~/git
cd ~/git
# $user is your github user
git clone https://github.com/$user/dynamic-localpv-provisioner.git # you may use ssh instead
cd dynamic-localpv-provisioner
git remote add upstream https://github.com/openebs/dynamic-localpv-provisioner.git
# Don't push to upstream directly
git remote set-url --push upstream no_push
```

--------------------------------

### Common Make Commands

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Utilize provided make commands for common development tasks such as bootstrapping dependencies, formatting code, cleaning artifacts, and building the project.

```bash
make bootstrap
make format
make clean
make
```

--------------------------------

### Run CI Integration Tests

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Execute integration tests using the provided script. Ensure you are within the correct development environment (e.g., nixos-shell) before running.

```bash
./ci/ci-test.sh run
```

--------------------------------

### Verify Data Disk Mount Options

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Confirms that the data disk is mounted with 'prjquota', indicating successful application of the 'pquota' mount option.

```bash
$ sudo mount | grep "^/dev/nvme1n1"

/dev/nvme1n1 on /mnt/data type xfs (rw,relatime,seclabel,attr2,inode64,prjquota)
```

--------------------------------

### Verify Root Filesystem Mount Options After Reboot

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Confirms that the root filesystem is now mounted with 'prjquota' (indicating 'pquota' is active) after a system reboot.

```bash
$ sudo mount | grep "^/dev/nvme0n1p1"

/dev/nvme0n1p1 on / type xfs (rw,relatime,seclabel,attr2,inode64,prjquota)
```

--------------------------------

### List Velero Backups

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/restore.md

Lists all available Velero backups within the current context. Use this to identify the backup name for restoration.

```bash
./velero backup get
```

--------------------------------

### Create Sparse File for XFS

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/use-xfs-fs-with-loop-device.md

Create a sparse file that will serve as the backing store for the XFS filesystem. This command creates a file named 'xfs.32M' with a maximum size of 32MiB.

```bash
sudo dd if=/dev/zero of=xfs.32M bs=1 count=0 seek=32M
```

--------------------------------

### Run an e2e-test experiment

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/e2e-tests/README.md

To run an e2e-test, create the Kubernetes job defined in the experiment's 'run_e2e_test.yml' file. Ensure application information and test-specific environment variables are updated.

```bash
kubectl create -f experiments/chaos/app_pod_failure/run_e2e_test.yml
```

--------------------------------

### Format Sparse File as XFS

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/use-xfs-fs-with-loop-device.md

Format the created sparse file with the XFS filesystem. The '-q' option enables quiet mode.

```bash
sudo mkfs -t xfs -q xfs.32M
```

--------------------------------

### Create Velero Restore

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/restore.md

Creates a new restore operation from a specified backup. Use '--restore-volumes=true' to restore associated persistent volumes. The '--namespace-mappings' flag can be used to restore to a different namespace.

```bash
./velero restore create my-localpv-restore --from-backup my-localpv-backup --restore-volumes=true
```

--------------------------------

### Mount Data Disk with 'pquota' Option

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Mounts the data disk device with the 'pquota' option enabled. Note that 'pquota' is not compatible with the 'remount' option.

```bash
sudo mount -o rw,pquota /dev/nvme1n1 /mnt/data
```

--------------------------------

### Configure Kubelet Extra Binds for Rancher/RKE

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/rancher.md

Add this configuration to your cluster.yml file or Rancher cluster settings to bind-mount the default OpenEBS LocalPV hostpath directory.

```yaml
services:
  kubelet:
    extra_binds:
    #Default hostpath directory
    - /var/openebs/local:/var/openebs/local
```

--------------------------------

### Deploy openebs-operator Lite

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/e2e-tests/experiments/localpv-provisioner/README.md

Apply the operator file to deploy openebs operator lite if not using this experiment for its deployment. Ensure the release version is correctly specified.

```bash
kubectl apply -f https://raw.githubusercontent.com/openebs/charts/gh-pages/<release-version>/openebs-operator-lite.yaml
```

--------------------------------

### List Velero Pods

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/restore.md

Executes a command to list Velero pods in the 'velero' namespace. This is a prerequisite to exec into the Velero container.

```bash
kubectl -n velero get pods
```

--------------------------------

### Verify Velero Namespace Resources

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/velero/minio.md

Confirms the status of MinIO Tenant objects, secrets, and pods within the 'velero' namespace, indicating successful MinIO tenant deployment.

```bash
kubectl get tenants,secrets,pods -n velero

NAME                                       STATE         AGE
tenant.minio.min.io/openebs-backup-minio   Initialized   6h47m

NAME                                       TYPE                                  DATA   AGE
secret/default-token-kqlr8                 kubernetes.io/service-account-token   3      6h49m
secret/openebs-backup-minio-creds-secret   Opaque                                2      6h41m
secret/operator-tls                        Opaque                                1      6h41m
secret/operator-webhook-secret             Opaque                                3      6h41m

NAME                              READY   STATUS    RESTARTS   AGE
pod/openebs-backup-minio-ss-0-0   1/1     Running   0          6h40m
```

--------------------------------

### Workaround for Non-existent Hostpath Directory

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

This script helps determine the filesystem type and device for a hostpath directory that may not yet exist by iteratively checking parent directories.

```bash
BASEPATH="/var/openebs/local"

until OUTPUT=$(df -Th $BASEPATH 2> /dev/null)
do
BASEPATH=$(echo "$BASEPATH" | sed 's|\(.*ersihan)/.*|\1|')
done

echo "PATH=${BASEPATH}"
#Final output
echo "$OUTPUT"
```

--------------------------------

### Configure Kubelet Extra Mounts for Talos

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/talos.md

Add this configuration to your Talos cluster's machineconfig to bind-mount the default OpenEBS LocalPV hostpath directory to the kubelet container. Ensure the source path matches your custom hostpath if not using the default.

```yaml
kubelet:
    extraMounts:
          #Default Hostpath directory
        - destination: /var/openebs/local
          type: bind
          source: /var/openebs/local
          options:
            - rbind
            - rshared
            - rw
```

--------------------------------

### Report Current XFS Project Quotas

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/modify-xfs-quota.md

Execute this command on the node to view the current XFS project quota usage, soft limits, and hard limits for a given path. Note the Project ID for subsequent operations.

```bash
sudo xfs_quota -x -c 'report -h' /var/openebs/local
```

--------------------------------

### Describe StorageClass to find BasePath

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/modify-xfs-quota.md

Use this command to identify the BasePath directory configured in your StorageClass, which is essential for locating hostpath volumes.

```bash
kubectl describe sc <storageclass-name>
```

--------------------------------

### Verify XFS Project Quota Application

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/enable-xfs-quota.md

Use the 'xfs_quota' command to report on the project quota applied to the hostpath directory. This command verifies that the storage limits are active and shows the current usage against soft and hard limits.

```bash
$ sudo xfs_quota -x -c 'report -h' /var/openebs/local/

Project quota on /var/openebs/local (/dev/loop16)
                        Blocks
Project ID   Used   Soft   Hard Warn/Grace
---------- ---------------------------------
#0              0      0      0  00 [------]
#1              0   5.7G   6.7G  00 [------]
```

--------------------------------

### StorageClass with Allowed Topologies

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/allowedtopologies.md

Define a StorageClass that uses the hostpath provisioner and specifies allowed topologies based on node labels. This ensures that volumes are provisioned on nodes matching the specified criteria.

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: custom-hostpath
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: "hostpath"
      - name: BasePath
        value: "/var/openebs/local"
provisioner: openebs.io/local
volumeBindingMode: WaitForFirstConsumer
allowedTopologies:
- matchLabelExpressions:
  - key: kubernetes.io/hostname
    values:
    - worker-2
    - worker-3
    - worker-5

```

--------------------------------

### Add and Update Helm Repository

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/deploy/helm/charts/README.md

Add the OpenEBS LocalPV provisioner Helm repository to your Helm client and update your local repository cache.

```console
helm repo add openebs-localpv https://openebs.github.io/dynamic-localpv-provisioner
helm repo update
```

--------------------------------

### Apply RBAC and CRDS for e2e-framework

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/e2e-tests/experiments/localpv-provisioner/README.md

Before creating the Kubernetes job, apply the necessary RBAC and CRDS for the e2e-framework. These are essential for the experiment's proper functioning.

```bash
kubectl apply -f dynamic-localpv-provisioner/e2e-tests/hack/rbac.yaml
```

```bash
kubectl apply -f dynamic-localpv-provisioner/e2e-tests/hack/crds.yaml
```

--------------------------------

### Enable OpenEBS Addon in MicroK8s

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/installation/platforms/microk8s.md

Enables the OpenEBS addon in MicroK8s, which deploys LocalPV, cStor, and Jiva storage engine control-plane components. Use this for MicroK8s v1.21 onwards.

```bash
microk8s enable openebs
```

--------------------------------

### Make Data Disk Quota Mount Persistent with fstab

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Adds an entry to '/etc/fstab' to ensure the data disk is mounted with the 'pquota' option across system reboots.

```bash
UUID=9cff3d69-3769-4ad9-8460-9c54050583f9 /mnt/data               xfs     defaults,pquota 0 0
```

--------------------------------

### Create PersistentVolumeClaim for Hostpath Volume

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/design/hostpath_localpv_provisioner.md

Create a PersistentVolumeClaim (PVC) to request a local hostpath volume. This action triggers the provisioner if the StorageClass is correctly configured.

```bash
kubectl create -f <pvc-yaml-file>
```

--------------------------------

### Create PersistentVolumeClaim

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/enable-xfs-quota.md

Define a PersistentVolumeClaim (PVC) that utilizes the 'openebs-hostpath-xfs' StorageClass. This PVC requests 5Gi of storage and will be in a 'Pending' state until a volume is mounted.

```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: local-hostpath-xfs
spec:
  storageClassName: openebs-hostpath-xfs
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
```

--------------------------------

### Describe PV to find Node and Path

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/modify-xfs-quota.md

Describe the Persistent Volume (PV) to determine the node where the volume resides and its specific path. This information is crucial for accessing the volume on the correct node.

```bash
kubectl get pvc --namespace demo

kubectl describe pv pvc-0365904e-0add-45ec-9b4e-f4080929d6cd
```

--------------------------------

### Verify Backup Location Availability

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/backup_and_restore/backup.md

Check if the 'default' backup location is available and configured correctly before proceeding with backup creation.

```bash
$ ./velero backup-location get

NAME      PROVIDER   BUCKET/PREFIX   PHASE       LAST VALIDATED                  ACCESS MODE   DEFAULT
default   aws        velero          Available   2021-09-04 01:05:06 +0000 UTC   ReadWrite     true
```

--------------------------------

### Check Hostpath Filesystem Type and Device

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Verifies if the hostpath directory's filesystem is XFS and identifies the associated device. This command may fail if the path does not exist.

```bash
$ df -Th /var/openebs/local

Filesystem     Type  Size  Used  Avail Use% Mounted on
/dev/nvme0n1p1 xfs   8.0G  959M  7.1G  12% /
```

--------------------------------

### Modify GRUB Configuration for Root Filesystem

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/tutorials/hostpath/xfs_quota/prerequisites.md

Edits the '/etc/default/grub' file to add 'rootflags=pquota' to the kernel command line for enabling XFS quota on the root filesystem.

```bash
sudo vi /etc/default/grub
```

```bash
GRUB_CMDLINE_LINUX="console=tty0 crashkernel=auto net.ifnames=0 console=ttyS0"

# Modified line:
GRUB_CMDLINE_LINUX="console=tty0 crashkernel=auto net.ifnames=0 console=ttyS0 rootflags=pquota"
```

--------------------------------

### Checkout and Sync Develop Branch

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/developer.md

Ensure your local develop branch is up-to-date with the upstream develop branch by fetching and rebasing.

```bash
cd ~/git/dynamic-localpv-provisioner
git checkout develop
Switched to branch 'develop'
Your branch is up-to-date with 'origin/develop'.
git remote -v
origin https://github.com/$user/dynamic-localpv-provisioner.git (fetch)
origin https://github.com/$user/dynamic-localpv-provisioner.git (push)
upstream https://github.com/openebs/dynamic-localpv-provisioner.git (fetch)
upstream https://github.com/openebs/dynamic-localpv-provisioner.git (no_push)
git fetch upstream develop
remote: Counting objects: 141, done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 141 (delta 52), reused 46 (delta 46), pack-reused 66
Receiving objects: 100% (141/141), 112.43 KiB | 0 bytes/s, done.
Resolving deltas: 100% (79/79), done.
From github.com:openebs/dynamic-localpv-provisioner
  * branch            develop     -> FETCH_HEAD
git rebase upstream/develop
First, rewinding head to replay your work on top of it...
Fast-forwarded develop to upstream/develop.
git status
On branch develop
Your branch is ahead of 'origin/develop' by 38 commits.
(use "git push" to publish your local commits)
nothing to commit, working directory clean
```

--------------------------------

### Verify LocalPV Provisioner Pod Status

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/docs/quickstart.md

Checks if the OpenEBS LocalPV provisioner pods are running in the 'openebs' namespace.

```bash
$ kubectl get pods -n openebs -l openebs.io/component-name=openebs-localpv-provisioner

NAME                                            READY   STATUS    RESTARTS   AGE
openebs-localpv-provisioner-6599766b76-kg5z9    1/1     Running   0          67s
```

--------------------------------

### Run BDD Tests with Custom Namespace

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/tests/README.md

Runs the BDD tests, specifying a custom Kubernetes namespace for OpenEBS LocalPV components using the '-openebs-namespace' flag.

```bash
$ sudo -E env "PATH=$PATH" ginkgo -v -openebs-namespace=<your-namespace>
```

--------------------------------

### Code Comment for Technical Debt (Small)

Source: https://github.com/openebs/dynamic-localpv-provisioner/blob/develop/CONTRIBUTING.md

Use this comment format to indicate small or similar code changes that represent technical debt and have less impact.

```go
// TD: SMALL  -- indicates few/similar code changes & hence less impact
```