### Producer interaction example Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Shows the producer command and example messages sent to the topic. ```sh $ kubectl -n kafka run kafka-producer -ti --image=quay.io/strimzi/kafka:0.28.0-kafka-3.1.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9092 --topic my-topic If you don't see a command prompt, try pressing enter. >message1 >message2 >message3 > ``` -------------------------------- ### Install example Tenant with KES encryption Source: https://github.com/minio/operator/blob/master/docs/kes.md Use this command to install the example Tenant with KES encryption from the MinIO Operator repository. It applies the kustomization for tenant-kes-encryption. ```shell kubectl apply -k github.com/minio/operator/examples/kustomization/tenant-kes-encryption ``` -------------------------------- ### Deploy preconfigured example with kustomize Source: https://github.com/minio/operator/blob/master/docs/examples.md Build and apply the preconfigured example using kustomize and kubectl. ```bash kustomize build examples/kustomization/base | kubectl apply -f - ``` -------------------------------- ### Build Go SDK STS example with Docker Source: https://github.com/minio/operator/blob/master/examples/kustomization/sts-example/sample-clients/minio-sdk/go/README.md Builds the Go SDK STS example container image using Docker. Run this command from the example directory. ```shell docker build -t minio-sts-go-example . ``` -------------------------------- ### Install Tenant (Optional) Source: https://github.com/minio/operator/blob/master/examples/kustomization/sts-example/README.md Installs the tenant example using kustomize. This is optional if you already have an existing tenant. ```shell kubectl apply -k examples/kustomization/sts-example/tenant ``` -------------------------------- ### Start Minikube with sufficient resources Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Stops and deletes any existing Minikube cluster, then starts a new one with 10 GB of memory to ensure enough space for the integration (minimum 6 vCPU and 10 GB RAM). ```sh minikube stop minikube delete minikube start --memory=10240 ``` -------------------------------- ### Install MinIO Operator and Tenant Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Applies the MinIO Operator resources and creates a tenant using the provided kustomization files. ```sh kubectl apply -k ~/operator/resources kubectl apply -k ~/operator/examples/kustomization/tenant-lite ``` -------------------------------- ### Start cluster with CodeReady Containers Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Deletes, cleans up, sets up, and starts a CodeReady Containers cluster for OpenShift. ```sh crc delete crc cleanup crc setup crc start ``` -------------------------------- ### Install Sample Application (Go SDK) Source: https://github.com/minio/operator/blob/master/examples/kustomization/sts-example/README.md Installs the sample application that uses the Go SDK. This installs the application to the sts-client namespace and grants access to the sts-example-job. ```shell kubectl apply -k examples/kustomization/sts-example/ ``` -------------------------------- ### Start React page for JS debug Source: https://github.com/minio/operator/wiki/How-to-test-Operator-Login Installs dependencies, builds, and starts the React development server on port 5005 for JavaScript debugging. ```sh cd ~/console/portal-ui yarn install yarn build yarn run start ``` -------------------------------- ### Set Vault address and token for KES encryption example Source: https://github.com/minio/operator/blob/master/docs/examples.md Sets the Vault address and token environment variables required for the Vault KMS example. Replace `TOKEN` with the actual root token obtained from the pod logs. ```bash export VAULT_ADDR=http://localhost:8200 export VAULT_TOKEN=TOKEN ``` -------------------------------- ### Install cert-manager Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Installs cert-manager v1.6.2 using kubectl create from the official release manifest. ```sh kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.6.2/cert-manager.yaml ``` -------------------------------- ### Deploy MinIO Tenant with Vault KMS encryption using kustomize Source: https://github.com/minio/operator/blob/master/docs/examples.md Deploys a pre-configured MinIO Tenant with Server Side Encryption using KES and Hashicorp Vault. Requires prior setup of Vault, including role and policy configuration. ```bash kustomize build examples/kustomization/tenant-kes-encryption | kubectl apply -f - ``` -------------------------------- ### Deploy MinIO tenant with kustomize Source: https://github.com/minio/operator/blob/master/docs/lets-encrypt.md Build and apply the tenant-letsencrypt example from the operator project. ```bash kustomize build examples/kustomization/tenant-letsencrypt | kubectl apply -f - ``` -------------------------------- ### Install Sample Application (Go SDK) Source: https://github.com/minio/operator/blob/master/examples/kustomization/sts-example/README.md Installs the sample client using the MinIO Go SDK. ```shell kubectl apply -k examples/kustomization/sts-example/sample-clients/minio-sdk/go ``` -------------------------------- ### Install Sample Application (.NET SDK) Source: https://github.com/minio/operator/blob/master/examples/kustomization/sts-example/README.md Installs the sample client using the MinIO .NET SDK. ```shell kubectl apply -k examples/kustomization/sts-example/sample-clients/minio-sdk/dotnet ``` -------------------------------- ### Install MetalLB for bare metal clusters Source: https://github.com/minio/operator/blob/master/docs/lets-encrypt.md Install MetalLB to provide LoadBalancer support on bare metal Kubernetes clusters. ```bash kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml kubectl apply -f https://kind.sigs.k8s.io/examples/loadbalancer/metallb-configmap.yaml ``` -------------------------------- ### Install Nginx ingress controller Source: https://github.com/minio/operator/blob/master/docs/lets-encrypt.md Install the Nginx ingress controller with required settings for TLS termination and proxy protocol. ```bash helm repo add nginx-stable https://helm.nginx.com/stable helm repo update helm install nginx-ingress nginx-stable/nginx-ingress \ --set rbac.create=true \ --set controller.service.type=LoadBalancer \ --set controller.service.externalTrafficPolicy=Local \ --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-proxy-protocol"="*" \ --set controller.config.use-proxy-protocol="true" ``` -------------------------------- ### Full release example Source: https://github.com/minio/operator/wiki/How-to-make-Operator-Release Complete example of creating a release, including creating a PR branch, running the release command, committing, pushing, and opening a pull request. ```shell createPR operator operator-release RELEASE=5.0.15 make release git add . git commit -m 'Release 5.0.15' git push git push --set-upstream origin operator-release https://github.com/cniackz/operator/pull/new/operator-release ``` -------------------------------- ### Create MinIO Tenant with Kustomization Source: https://github.com/minio/operator/blob/master/README.md Applies a kustomization from the examples directory to create a 4-node MinIO Tenant with 4 volumes per node. The command uses kubectl apply with the -k flag. ```yaml kubectl apply -k github.com/minio/operator/examples/kustomization/base ``` -------------------------------- ### Install cert-manager Source: https://github.com/minio/operator/blob/master/docs/lets-encrypt.md Install cert-manager 1.7.2 or later, required for Let's Encrypt certificate management. ```bash kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.7.2/cert-manager.yaml ``` -------------------------------- ### Install Sample Application (Java SDK) Source: https://github.com/minio/operator/blob/master/examples/kustomization/sts-example/README.md Installs the sample client using the MinIO Java SDK. ```shell kubectl apply -k examples/kustomization/sts-example/sample-clients/minio-sdk/java ``` -------------------------------- ### Install Tenant with Cert Manager Source: https://github.com/minio/operator/blob/master/examples/kustomization/sts-example/README.md Installs the tenant example that uses cert-manager for certificate management. This is an alternative to the basic tenant installation. ```shell kubectl apply -k examples/kustomization/sts-example/tenant-certmanager ``` -------------------------------- ### Install tenant with Helm Source: https://github.com/minio/operator/wiki/Deploy-Prometheus-and-MinIO-using-Helm Installs the tenant Helm chart into the tenant-ns namespace, creating the namespace if needed, and uses the values file for configuration. ```sh cd ~/operator/helm helm install --debug --namespace tenant-ns \ --create-namespace tenant ./tenant \ -f ~/operator/helm/tenant/values.yaml ``` -------------------------------- ### nelm install output: progress and logs Source: https://github.com/minio/operator/blob/master/NELM.md Example output from the nelm install command, showing resource status, pod logs, and completed operations. The output ends with 'Succeeded release "operator" (namespace: "minio")'. ```bash Starting release "operator" (namespace: "minio") ┌ Progress status │ RESOURCE (→READY) STATE INFO │ Deployment/minio-operator WAITING Ready:0/2 │ • Pod/minio-operator-84867f7cd-67fhk UNKNOWN Status:ContainerCreating │ • Pod/minio-operator-84867f7cd-jqhxc UNKNOWN Status:ContainerCreating │ ClusterRole/minio-operator-role READY │ ClusterRoleBinding/minio-operator-binding READY │ Service/operator READY │ Service/sts READY │ ServiceAccount/minio-operator READY └ Progress status ┌ Logs for Pod/minio-operator-84867f7cd-67fhk, container/operator │ I0602 13:57:56.451191 1 controller.go:81] Starting MinIO Operator │ I0602 13:57:56.452231 1 main-controller.go:293] Setting up event handlers │ I0602 13:57:56.459735 1 main-controller.go:514] Using Kubernetes CSR Version: v1 │ I0602 13:57:56.459758 1 main-controller.go:534] Waiting for STS API to start │ I0602 13:57:56.459776 1 leaderelection.go:257] attempting to acquire leader lease minio/minio-operator-lock... │ I0602 13:57:56.459829 1 main-controller.go:399] Starting STS API server │ I0602 13:57:56.461263 1 tls.go:54] Waiting for the sts certificates secret to be issued │ I0602 13:57:56.468473 1 leaderelection.go:271] successfully acquired lease minio/minio-operator-lock │ I0602 13:57:56.468564 1 main-controller.go:577] minio-operator-84867f7cd-67fhk: I am the leader, applying leader labels on myself │ I0602 13:57:56.468645 1 main-controller.go:428] Waiting for Upgrade Server to start │ I0602 13:57:56.468674 1 main-controller.go:432] Starting Tenant controller │ I0602 13:57:56.468679 1 main-controller.go:435] Waiting for informer caches to sync │ I0602 13:57:56.468703 1 main-controller.go:388] Starting HTTP Upgrade Tenant Image server │ I0602 13:57:56.569827 1 main-controller.go:456] STS Autocert is enabled, starting API certificate setup. │ I0602 13:57:56.573769 1 tls.go:127] sts-tls TLS secret not found: secrets "sts-tls" not found │ I0602 13:57:56.591452 1 csr.go:183] Start polling for certificate of csr/sts-minio-csr, every 5s, timeout after 20m0s └ Logs for Pod/minio-operator-84867f7cd-67fhk, container/operator ┌ Logs for Pod/minio-operator-84867f7cd-jqhxc, container/operator │ I0602 13:57:57.107739 1 controller.go:81] Starting MinIO Operator │ I0602 13:57:57.108397 1 main-controller.go:293] Setting up event handlers │ I0602 13:57:57.115257 1 main-controller.go:514] Using Kubernetes CSR Version: v1 │ I0602 13:57:57.115285 1 main-controller.go:534] Waiting for STS API to start │ I0602 13:57:57.115306 1 leaderelection.go:257] attempting to acquire leader lease minio/minio-operator-lock... │ I0602 13:57:57.115368 1 main-controller.go:399] Starting STS API server │ I0602 13:57:57.116599 1 tls.go:54] Waiting for the sts certificates secret to be issued │ I0602 13:57:57.116653 1 main-controller.go:595] minio-operator-84867f7cd-67fhk: is the leader, removing any leader labels that I 'minio-operator-84867f7cd-jqhxc' might have └ Logs for Pod/minio-operator-84867f7cd-jqhxc, container/operator ┌ Progress status │ RESOURCE (→READY) STATE INFO │ Deployment/minio-operator READY Ready:2/2 │ • Pod/minio-operator-84867f7cd-67fhk READY Status:Running │ • Pod/minio-operator-84867f7cd-jqhxc READY Status:Running └ Progress status ┌ Completed operations │ Create resource: ClusterRole/minio-operator-role │ Create resource: ClusterRoleBinding/minio-operator-binding │ Create resource: CustomResourceDefinition/policybindings.sts.min.io │ Create resource: CustomResourceDefinition/tenants.minio.min.io │ Create resource: Deployment/minio-operator │ Create resource: Service/operator │ Create resource: Service/sts │ Create resource: ServiceAccount/minio-operator └ Completed operations Succeeded release "operator" (namespace: "minio") ``` -------------------------------- ### Apply Kafka Cluster CR Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Applies the Kafka cluster custom resource from the Strimzi example manifest. ```sh kubectl apply -f https://strimzi.io/examples/latest/kafka/kafka-persistent-single.yaml -n kafka ``` -------------------------------- ### Install Zookeeper Operator and Cluster Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Adds the Pravega Helm repo, installs the Zookeeper operator, and creates a ZookeeperCluster custom resource with one replica. ```sh helm repo add pravega https://charts.pravega.io helm repo update helm install zookeeper-operator --namespace=zookeeper --create-namespace pravega/zookeeper-operator kubectl create --namespace zookeeper -f - < tenant.yaml kubectl apply -f tenant.yaml ``` -------------------------------- ### Setup Vault and get credentials Source: https://github.com/minio/operator/wiki/Vault,-KES-and-MinIO-Deployed-in-k8s-Cluster Configures Vault with AppRole authentication and KV secrets engine, writes the KES policy, creates the kes-role, and retrieves the ROLE_ID and SECRET_ID for the KES configuration. ```sh VAULT_ROOT_TOKEN=$(kubectl logs -l app=vault | grep "Root Token: " | sed -e "s/Root Token: //g") kubectl exec $(kubectl get pods -l app=vault | grep -v NAME | awk '{print $1}') -- sh -c 'VAULT_TOKEN='$VAULT_ROOT_TOKEN' VAULT_ADDR="http://127.0.0.1:8200" vault auth enable approle' kubectl exec $(kubectl get pods -l app=vault | grep -v NAME | awk '{print $1}') -- sh -c 'VAULT_TOKEN='$VAULT_ROOT_TOKEN' VAULT_ADDR="http://127.0.0.1:8200" vault secrets enable kv' kubectl cp ~/operator/examples/vault/kes-policy.hcl $(kubectl get pods -l app=vault | grep -v NAME | awk '{print $1}'):/kes-policy.hcl kubectl exec $(kubectl get pods -l app=vault | grep -v NAME | awk '{print $1}') -- sh -c 'VAULT_TOKEN='$VAULT_ROOT_TOKEN' VAULT_ADDR="http://127.0.0.1:8200" vault policy write kes-policy /kes-policy.hcl' kubectl exec $(kubectl get pods -l app=vault | grep -v NAME | awk '{print $1}') -- sh -c 'VAULT_TOKEN='$VAULT_ROOT_TOKEN' VAULT_ADDR="http://127.0.0.1:8200" vault write auth/approle/role/kes-role token_num_uses=0 secret_id_num_uses=0 period=5m policies=kes-policy' ROLE_ID=$(kubectl exec $(kubectl get pods -l app=vault | grep -v NAME | awk '{print $1}') -- sh -c 'VAULT_TOKEN='$VAULT_ROOT_TOKEN' VAULT_ADDR="http://127.0.0.1:8200" vault read auth/approle/role/kes-role/role-id' | grep "role_id " | sed -e "s/role_id //g") SECRET_ID=$(kubectl exec $(kubectl get pods -l app=vault | grep -v NAME | awk '{print $1}') -- sh -c 'VAULT_TOKEN='$VAULT_ROOT_TOKEN' VAULT_ADDR="http://127.0.0.1:8200" vault write -f auth/approle/role/kes-role/secret-id' | grep "secret_id " | sed -e "s/secret_id //g") echo $ROLE_ID echo $SECRET_ID ``` -------------------------------- ### Install tenant lite Source: https://github.com/minio/operator/wiki/decommission Installs the tenant-lite tenant using the nodeport configuration. ```bash installtenant nodeport ``` -------------------------------- ### Install MinIO Operator via Kustomization Source: https://github.com/minio/operator/blob/master/README.md Uses kubectl's built-in kustomize support to install the MinIO Operator from the GitHub repository at version v7.1.1. The command pipes the kustomized output directly to kubectl apply. ```sh kubectl kustomize github.com/minio/operator\?ref=v7.1.1 | kubectl apply -f - ``` -------------------------------- ### Generate TLS certificates with mkcert Source: https://github.com/minio/operator/blob/master/docs/examples.md Generate certificate keypairs for the MinIO tenant and its headless service using mkcert. Assumes tenant name 'myminio' and namespace 'minio-tenant'. ```sh mkcert "minio.minio-tenant.svc.cluster.local" mkcert "*.myminio.minio-tenant.svc.cluster.local" mkcert "*.myminio-hl.minio-tenant.svc.cluster.local" ``` -------------------------------- ### Example tenant spec with zones and securityContext for v3.x.x to v4.x.x upgrade Source: https://github.com/minio/operator/blob/master/UPGRADE.md Example tenant spec showing required zone names and securityContext for the v3.x.x to v4.x.x upgrade. Apply via kubectl edit or kubectl apply. ```yaml image: "minio/minio:$(LATEST-VERSION)" ... zones: - servers: 4 name: "zone-0" volumesPerServer: 4 volumeClaimTemplate: metadata: name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Ti securityContext: runAsUser: 0 runAsGroup: 0 runAsNonRoot: false fsGroup: 0 - servers: 4 name: "zone-1" volumesPerServer: 4 volumeClaimTemplate: metadata: name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Ti securityContext: runAsUser: 0 runAsGroup: 0 runAsNonRoot: false fsGroup: 0 ``` -------------------------------- ### Build tenant-certmanager kustomization Source: https://github.com/minio/operator/wiki/cert‐manager Builds the tenant-certmanager kustomization from the MinIO operator examples and saves it to a local file. ```sh kustomize build github.com/minio/operator/examples/kustomization/tenant-lite\?ref\=v5.0.10 > /Users/cniackz/bash-config/config-files/kustomize/Tenant/kustomize-tenant-5-0-10.yaml ``` -------------------------------- ### Install MinIO Tenant with Helm Source: https://github.com/minio/operator/wiki/Deploy-Operator-with-Helm Installs a MinIO tenant using the tenant-5.0.14.tgz chart into the tenant-ns namespace, creating the namespace if it does not exist. ```shell helm install \ --namespace tenant-ns \ --create-namespace \ tenant-ns ./tenant-5.0.14.tgz ``` -------------------------------- ### Install MinIO Operator Source: https://github.com/minio/operator/wiki/cert‐manager Installs the MinIO Operator using the `installoperator` command with ingress support. ```sh installoperator ingress ``` -------------------------------- ### Create MinIO tenant with oc minio Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Creates a MinIO tenant with specified servers, volumes, capacity, namespace, and storage class. Output includes generated credentials and service details. ```sh $ oc minio tenant create minio-tenant-1 \ --servers 1 \ --volumes 4 \ --capacity 1Gi \ --namespace minio-tenant-1 \ --storage-class local-storage \ Tenant 'minio-tenant-1' created in 'minio-tenant-1' Namespace Username: admin Password: 7ca988c1-2963-4729-9084-4230723fb51b Note: Copy the credentials to a secure location. MinIO will not display these again. +-------------+------------------------+----------------+--------------+--------------+ | APPLICATION | SERVICE NAME | NAMESPACE | SERVICE TYPE | SERVICE PORT | +-------------+------------------------+----------------+--------------+--------------+ | MinIO | minio | minio-tenant-1 | ClusterIP | 443 | | Console | minio-tenant-1-console | minio-tenant-1 | ClusterIP | 9443 | +-------------+------------------------+----------------+--------------+--------------+ ``` -------------------------------- ### Enable Kafka bucket notification Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Creates a bucket, adds a Kafka event notification for .jpg files, and lists the events. ```sh mc mb myminio/images --insecure mc event add myminio/images arn:minio:sqs::1:kafka --suffix .jpg --insecure mc event list myminio/images --insecure ``` -------------------------------- ### Example tenant spec with securityContext for v4.2.2 to v4.2.3 upgrade Source: https://github.com/minio/operator/blob/master/UPGRADE.md Example tenant spec showing the required securityContext in each pool for the v4.2.2 to v4.2.3 upgrade. Apply via kubectl edit or kubectl apply. ```yaml image: "minio/minio:$(LATEST-VERSION)" ... pools: - servers: 4 name: "pool-0" volumesPerServer: 4 volumeClaimTemplate: metadata: name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Ti securityContext: runAsUser: 0 runAsGroup: 0 runAsNonRoot: false fsGroup: 0 ``` -------------------------------- ### Run Kafka consumer Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Runs a Kafka console consumer to read messages from the 'my-topic' topic from the beginning. ```sh kubectl -n kafka run kafka-consumer -ti --image=quay.io/strimzi/kafka:0.28.0-kafka-3.1.0 --rm=true --restart=Never -- bin/kafka-console-consumer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic --from-beginning ``` -------------------------------- ### Example output of pool statefulset names Source: https://github.com/minio/operator/blob/master/docs/DECOMISSION.md Example output showing the statefulset names for two pools. This is the expected output from the previous command when the tenant has two pools. ```bash ~ kubectl get tenants -n ns-1 all-nodes -o json | jq '.status.pools[].ssName' "worker-ss-0" "worker-ss-1" ``` -------------------------------- ### Follow operator logs Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Streams the logs of the Strimzi cluster operator deployment. ```sh kubectl logs deployment/strimzi-cluster-operator -n kafka -f ``` -------------------------------- ### Expected output for adding Kafka endpoint Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Shows the expected output when the Kafka endpoint is configured successfully, including a prompt to restart the server. ```sh mc admin config set myminio notify_kafka:1 tls_skip_verify="off" queue_dir="" queue_limit="0" sasl="off" sasl_password="" sasl_username="" tls_client_auth="0" tls="off" client_tls_cert="" client_tls_key="" brokers="172.17.0.11:29093" topic="my-topic" version="" --insecure Successfully applied new settings. Please restart your server 'mc admin service restart myminio'. ``` -------------------------------- ### Expected output from helm install Source: https://github.com/minio/operator/wiki/Deploy-MinIO-Operator-with-Helm Shows the expected output from the helm install command, including the JWT retrieval and console URL instructions. ```sh $ helm repo add minio https://operator.min.io/ "minio" has been added to your repositories $ helm install \ > --namespace minio-operator \ > --create-namespace \ > minio-operator minio/operator NAME: minio-operator LAST DEPLOYED: Wed Sep 14 12:14:05 2022 NAMESPACE: minio-operator STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: 1. Get the JWT for logging in to the console: kubectl apply -f - < tenant.yaml kustomize apply -f tenant.yaml ``` -------------------------------- ### Expected output for enabling Kafka bucket notification Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Shows the expected output when the bucket is created, the event is added, and the event list is displayed. ```sh $ mc mb myminio/images --insecure Bucket created successfully `myminio/images`. $ mc event add myminio/images arn:minio:sqs::1:kafka --suffix .jpg --insecure Successfully added arn:minio:sqs::1:kafka $ mc event list myminio/images --insecure arn:minio:sqs::1:kafka s3:ObjectCreated:*,s3:ObjectRemoved:*,s3:ObjectAccessed:* Filter: suffix=".jpg" ``` -------------------------------- ### Create a MinIO Tenant Source: https://github.com/minio/operator/blob/master/helm/operator/README.md Creates a MinIO Tenant (cluster) in the 'tenant-ns' namespace, creating the namespace if needed. This results in a 4-node MinIO Tenant by default; customize via the tenant chart's values.yaml. ```bash helm install --namespace tenant-ns \ --create-namespace tenant minio/tenant ``` -------------------------------- ### Run Kafka producer Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Runs a Kafka console producer in a pod to send messages to the 'my-topic' topic. ```sh kubectl -n kafka run kafka-producer -ti --image=quay.io/strimzi/kafka:0.28.0-kafka-3.1.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9092 --topic my-topic ``` -------------------------------- ### Deploy MinIO Tenant with NodePort using kustomize Source: https://github.com/minio/operator/blob/master/docs/examples.md Use this command to deploy a MinIO tenant with NodePort services. It builds the Kubernetes manifests from the examples/kustomization/tenant-nodeport directory and applies them to the cluster. NodePort services expose the tenant by opening a port on each Kubernetes cluster node, avoiding the need for load balancers or ingress configurations. ```bash kustomize build examples/kustomization/tenant-nodeport | kubectl apply -f - ``` -------------------------------- ### Verify data encryption with mc admin kms key status Source: https://github.com/minio/operator/blob/master/docs/examples.md Connects to MinIO via port-forward and checks the KMS key status. The output shows the key `my-minio-key` with encryption and decryption enabled. ```bash kubectl port-forward svc/minio 9000:443 -n tenant-kms-encrypted mc alias set alias https://127.0.0.1:9000 minio minio123 --insecure mc admin kms key status alias --insecure Key: my-minio-key - Encryption ✔ - Decryption ✔ ``` -------------------------------- ### Create Kubernetes TLS secrets from certificates Source: https://github.com/minio/operator/blob/master/docs/examples.md Create Kubernetes secrets for the MinIO tenant and operator certificates. The operator secret 'operator-ca-tls-minio-tls-cert' is created separately. ```bash kubectl create secret tls minio-tls-cert --key="minio.minio-tenant.svc.cluster.local-key.pem" --cert="minio.minio-tenant.svc.cluster.local.pem" -n minio-tenant kubectl create secret tls minio-buckets-cert --key="_wildcard.myminio.minio-tenant.svc.cluster.local-key.pem" --cert="_wildcard.myminio.minio-tenant.svc.cluster.local.pem" -n minio-tenant kubectl create secret tls minio-hl-cert --key="_wildcard.myminio-hl.minio-tenant.svc.cluster.local-key.pem" --cert="_wildcard.myminio-hl.minio-tenant.svc.cluster.local.pem" -n minio-tenant # create a new secret for the operator certs kubectl create secret tls operator-ca-tls-minio-tls-cert --key="minio.minio-tenant.svc.cluster.local-key.pem" --cert="minio.minio-tenant.svc.cluster.local.pem" -n minio-tenant ``` -------------------------------- ### Add MinIO alias Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Sets up an alias for the MinIO server using the mc client, with insecure mode enabled. ```sh mc alias set myminio https://localhost:9000 minio minio123 --insecure ``` -------------------------------- ### Apply static YAML with kubectl apply -f Source: https://github.com/minio/operator/blob/master/docs/tenant-creation.md Applies the generated `minio-tenant.yaml` file to create the tenant. ```shell kubectl apply -f minio-tenant.yaml ``` -------------------------------- ### Generate KES identity keypair Source: https://github.com/minio/operator/blob/master/docs/examples.md Generates a new KES identity keypair for mTLS authentication between MinIO and KES. The resulting app.key and app.cert are used to create a Kubernetes secret. ```sh kes tool identity new --key="./app.key" --cert="app.cert" app ``` -------------------------------- ### Customized securityContext values in values.yaml Source: https://github.com/minio/operator/wiki/Deploy-MinIO-Operator-with-Helm Example of customized securityContext values in the values.yaml file, changing the user and group IDs to 900. ```sh securityContext: runAsUser: 900 runAsGroup: 900 runAsNonRoot: true fsGroup: 900 ``` -------------------------------- ### Deploy pods for front end engineer login Source: https://github.com/minio/operator/wiki/How-to-test-Operator-Login Sets up a kind cluster and deploys the MinIO Operator and tenant, similar to the end-user setup but without port-forwarding. Use this before running the console server in operator mode. ```sh kind delete cluster kind create cluster --config ~/operator/testing/kind-config.yaml kubectl apply -k ~/operator/resources kubectl apply -k ~/operator/examples/kustomization/tenant-tiny ``` -------------------------------- ### Tenant YAML overlay with externalClientCertSecret Source: https://github.com/minio/operator/blob/master/docs/examples.md Adds the externalClientCertSecret field to the tenant YAML overlay to provide the mTLS certificate secret. The field may need to be added if it does not exist. ```yaml spec: ... externalClientCertSecret: name: minio-kes-mtls type: kubernetes.io/tls ``` -------------------------------- ### Run Kafka consumer with OpenShift Source: https://github.com/minio/operator/wiki/MinIO-and-Koperator Executes a Kafka console consumer in a pod to read messages from a topic. Output shows LEADER_NOT_AVAILABLE warnings followed by the consumed messages. ```sh $ kubectl -n kafka run kafka-consumer -ti --image=quay.io/strimzi/kafka:0.28.0-kafka-3.1.0 --rm=true --restart=Never -- bin/kafka-console-consumer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic --from-beginning If you don't see a command prompt, try pressing enter. [2022-04-07 23:39:13,028] WARN [Consumer clientId=console-consumer, groupId=console-consumer-6036] Error while fetching metadata with correlation id 2 : {my-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2022-04-07 23:39:13,185] WARN [Consumer clientId=console-consumer, groupId=console-consumer-6036] Error while fetching metadata with correlation id 4 : {my-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2022-04-07 23:39:13,371] WARN [Consumer clientId=console-consumer, groupId=console-consumer-6036] Error while fetching metadata with correlation id 6 : {my-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2022-04-07 23:39:13,679] WARN [Consumer clientId=console-consumer, groupId=console-consumer-6036] Error while fetching metadata with correlation id 8 : {my-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) message1 message2 message3 ``` -------------------------------- ### Start decommission Source: https://github.com/minio/operator/wiki/decommission Starts the decommission of the first pool. The expected output confirms the decommission started successfully. ```bash mc admin decommission start myminio/ https://myminio-pool-0-{0...3}.myminio-hl.tenant-lite.svc.cluster.local/export{0...1} --insecure ``` ```bash $ mc admin decommission start myminio/ https://myminio-pool-0-{0...3}.myminio-hl.tenant-lite.svc.cluster.local/export{0...1} --insecure Decommission started successfully for `https://myminio-pool-0-{0...3}.myminio-hl.tenant-lite.svc.cluster.local/export{0...1}`. ``` -------------------------------- ### Set Prometheus values in values.yaml Source: https://github.com/minio/operator/wiki/Deploy-Prometheus-and-MinIO-using-Helm Values file entries for the Prometheus setup. Defaults are provided for image, sidecarimage, initimage, diskCapacityGB, storageClassName, and securityContext; empty values fall back to the defaults noted in comments. ```yaml ## Prometheus setup for MinIO Tenant. prometheus: image: "" # defaults to quay.io/prometheus/prometheus:latest sidecarimage: "" # defaults to alpine initimage: "" # defaults to busybox:1.33.1 diskCapacityGB: 1 storageClassName: standard annotations: { } labels: { } nodeSelector: { } affinity: nodeAffinity: { } podAffinity: { } podAntiAffinity: { } resources: { } serviceAccountName: "" securityContext: runAsUser: 1000 runAsGroup: 1000 runAsNonRoot: true fsGroup: 1000 ``` -------------------------------- ### Apply customized tenant with kubectl apply -k . Source: https://github.com/minio/operator/blob/master/docs/tenant-creation.md Creates the tenant with the MinIO version specified in the overlay `tenant.yaml`. Run this command from the directory containing both `kustomization.yaml` and `tenant.yaml`. ```shell kubectl apply -k . ``` -------------------------------- ### Add a Pool to Tenant YAML Source: https://github.com/minio/operator/blob/master/docs/expansion.md Example YAML snippet showing how to add a pool with 4 drives to the tenant's .spec.pools section. The pool name should be sequential, volumesPerServer specifies the number of volumes per pod, and the volumeClaimTemplate defines the storage request. ```yaml spec: pools: ... - servers: 4 ## For naming of the pool, you can use any name, but keeping sequential numbers is recommended. name: pool-1 ## volumesPerServer specifies the number of volumes attached per MinIO Tenant Pod / Server. volumesPerServer: 4 ## This VolumeClaimTemplate is used across all the volumes provisioned for MinIO Tenant in this Pool. volumeClaimTemplate: metadata: name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi ``` -------------------------------- ### Install operator Source: https://github.com/minio/operator/wiki/decommission Installs the MinIO operator using the nodeport configuration. ```bash installoperator nodeport ``` -------------------------------- ### Start pool decommission with mc admin decommission start Source: https://github.com/minio/operator/blob/master/docs/DECOMISSION.md Starts decommissioning a pool by specifying the alias and the pool's endpoint with a range of statefulset pods and export paths. Replace the alias, hostname, and ranges with your environment values. ```bash mc admin decommission start myminio/ https://minio-ss-0-{0...3}.minio-hl..svc.cluster.local/export{0...3} ``` -------------------------------- ### Create a cluster in kind Source: https://github.com/minio/operator/wiki/How-to-test-MinIOJob Command to create a Kubernetes cluster using kind. ```shell createcluster ```