### Deploy Bytewax Platform

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/demo-keycloak.md

Deploys or upgrades the Bytewax Platform using Helm. It installs the platform from a local chart `./platform` into the `bytewax-system` namespace, using the specified `values.yaml` file for configuration.

```bash
helm upgrade --install bytewax-platform ./platform -nbytewax-system -f ./values.yaml
```

--------------------------------

### Install Prerequisites for Bytewax Platform

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

This snippet outlines the essential command-line tools required for installing the Bytewax Platform. Ensure these are installed and accessible in your environment before proceeding with the platform installation.

```bash
# Install AWS CLI
# Refer to: https://aws.amazon.com/cli/

# Install kubectl
# Refer to: https://kubernetes.io/docs/tasks/tools/

# Install Helm
# Refer to: https://helm.sh/docs/intro/install/
```

--------------------------------

### Deploy Bytewax Platform using Helm

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Installs or upgrades the Bytewax Platform using the Helm chart, applying the specified values file and targeting the 'bytewax-system' namespace.

```bash
helm upgrade --install bytewax-platform ./platform -n bytewax-system -f ./values.yaml
```

--------------------------------

### Prepare Helm Chart Directory

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Creates a new directory 'awsmp-chart' and changes the current directory into it, preparing for Helm chart operations.

```bash
mkdir awsmp-chart
cd awsmp-chart
```

--------------------------------

### Download Bytewax Platform Helm Chart

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Downloads the Bytewax Platform Helm chart (version 0.2.1) from the specified OCI repository.

```bash
helm pull oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/bytewax/platform --version 0.2.1
```

--------------------------------

### Create Kubernetes Namespace

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Creates the 'bytewax-system' namespace in the Kubernetes cluster. This namespace is used to isolate Bytewax Platform resources.

```bash
kubectl create namespace bytewax-system
```

--------------------------------

### WaxAPI Authentication Example

Source: https://github.com/bytewax/platform-docs-content/blob/main/reference/waxapi.md

Demonstrates how to authenticate with WaxAPI using a bearer token in the Authorization header. This is required for all API calls.

```bash
curl https://waxapi.yourcompany.com/dataflows/
-H "Authorization: Bearer <TOKEN>"
```

--------------------------------

### Configure Bytewax Platform Values

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/demo-keycloak.md

Creates a `values.yaml` file to configure the Bytewax Platform's OIDC settings and enable the demo Keycloak instance. This file specifies the authentication issuer, client ID, and client secret.

```bash
cat << EOF > ../values.yaml
oidc:
  authIssuer: "http://bytewax-platform-demokeycloak.bytewax-system.svc.cluster.local:8880/realms/bytewax"
  clientId: "bytewax-platform"
  clientSecret: "**********"

demokeycloak:
  enabled: true
EOF
```

--------------------------------

### Set Local Environment Variables

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Sets the AWS Marketplace token and role ARN as local environment variables. These are required for subsequent steps in the installation process.

```bash
AWSMP_TOKEN=<CREATED_TOKEN_ABOVE>
AWSMP_ROLE_ARN=<CREATED_ROLE_ABOVE>
```

--------------------------------

### Configure Bytewax Platform Helm Chart for EKS Anywhere

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Creates a 'values.yaml' file for the Bytewax Platform Helm chart, specifying the license configuration secret name for EKS Anywhere installations.

```yaml
billing:
  awsMarketplace:
    licenseConfigSecretName: awsmp-license-token-secret
```

--------------------------------

### Authenticate Helm Registry

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Authenticates the Helm client with the AWS ECR registry using credentials obtained from the AWS CLI.

```bash
aws ecr get-login-password \
    --region us-east-1 | helm registry login \
    --username AWS \
    --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
```

--------------------------------

### Deploy Bytewax Platform using Helm

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-eks.md

Deploys the Bytewax Platform to your EKS cluster using the Helm chart. It installs or upgrades the 'bytewax-platform' release in the 'bytewax-system' namespace, using the specified 'values.yaml' file for configuration.

```bash
helm upgrade --install bytewax-platform ./platform -nbytewax-system -f ./values.yaml
```

--------------------------------

### Create Docker Registry Secret for Image Pulling

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Creates a Kubernetes secret for Docker registry authentication, allowing the cluster to pull Bytewax Platform images from AWS ECR. It uses AWS CLI to get login credentials.

```bash
kubectl create secret docker-registry awsmp-image-pull-secret \
--docker-server=709825985650.dkr.ecr.us-east-1.amazonaws.com \
--docker-username=AWS \
--docker-password=$(aws ecr get-login-password --region us-east-1) \
--namespace bytewax-system
```

--------------------------------

### Update Hosts File

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/demo-keycloak.md

Adds an entry to the `/etc/hosts` file to map the Keycloak service's hostname to the local loopback address (127.0.0.1). This is necessary for accessing the Keycloak instance via port-forwarding.

```bash
echo "127.0.0.1 bytewax-platform-demokeycloak.bytewax-system.svc.cluster.local" >> /etc/hosts
```

--------------------------------

### Create Bytewax Namespace

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-eks.md

Creates a Kubernetes namespace named 'bytewax-system' for the Bytewax Platform. This namespace is crucial for organizing Bytewax resources within the cluster.

```bash
kubectl create namespace bytewax-system
```

--------------------------------

### Extract Helm Chart Archive

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Extracts the downloaded Helm chart archive and removes the compressed file. Assumes the archive is a tarball.

```bash
tar xf $(pwd)/* && find $(pwd) -maxdepth 1 -type f -delete
```

--------------------------------

### Download and Extract Bytewax Helm Chart

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-eks.md

Downloads the Bytewax Platform Helm chart (version 0.2.1) from the AWS ECR OCI registry and then extracts the chart archive. It also cleans up the original archive file.

```bash
helm pull oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/bytewax/platform --version 0.2.1 
tar xf $(pwd)/* && find $(pwd) -maxdepth 1 -type f -delete
```

--------------------------------

### Check Default Storage Class

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/installation.md

Verifies if a default Storage Class is configured in your Kubernetes cluster. The output should show at least one Storage Class with a '(default)' designation.

```bash
kubectl get sc
```

--------------------------------

### Export AWS Credentials

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Exports the obtained AWS access key ID, secret access key, and session token into environment variables. These are used by the AWS CLI for subsequent operations.

```bash
export AWS_ACCESS_KEY_ID=$(echo $AWSMP_ROLE_CREDENTIALS | awk '{print $1}' | xargs)
export AWS_SECRET_ACCESS_KEY=$(echo $AWSMP_ROLE_CREDENTIALS | awk '{print $3}' | xargs)
export AWS_SESSION_TOKEN=$(echo $AWSMP_ROLE_CREDENTIALS | awk '{print $4}' | xargs)
```

--------------------------------

### Port Forwarding for Bytewax Platform Access

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/demo-keycloak.md

Establishes port forwarding to access the Bytewax Platform services. This includes forwarding the `waxapi` service to local port 8080, the `demokeycloak` service to local port 8880, and the dashboard service to local port 3000.

```bash
kubectl port-forward svc/bytewax-platform-waxapi 8080
```

```bash
kubectl port-forward svc/bytewax-platform-demokeycloak 8880
```

```bash
kubectl port-forward svc/bytewax-platform-dashboard 3000
```

--------------------------------

### Create AWS Marketplace License Secret

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Creates a Kubernetes secret named 'awsmp-license-token-secret' in the 'bytewax-system' namespace. This secret stores the AWS Marketplace token and role ARN for license verification.

```bash
kubectl create secret generic awsmp-license-token-secret \
--from-literal=license_token=$AWSMP_TOKEN \
--from-literal=iam_role=$AWSMP_ROLE_ARN \
--namespace bytewax-system
```

--------------------------------

### Assume AWS Role with Web Identity

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Assumes an AWS role using web identity token, obtaining temporary AWS credentials. These credentials are required for interacting with AWS services like ECR.

```bash
AWSMP_ROLE_CREDENTIALS=$(aws sts assume-role-with-web-identity \
                --region 'us-east-1' \
                --role-arn $AWSMP_ROLE_ARN \
                --role-session-name 'AWSMP-guided-deployment-session' \
                --web-identity-token $AWSMP_ACCESS_TOKEN \
                --query 'Credentials' \
                --output text)
```

--------------------------------

### Create IAM Service Account for Bytewax

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-eks.md

Creates an IAM service account for the Bytewax Platform controller manager within the 'bytewax-system' namespace. It attaches the AWSLicenseManagerConsumptionPolicy and approves existing service accounts. Ensure you replace <YOUR_CLUSTER_NAME> and <YOUR_CLUSTER_REGION> with your specific cluster details.

```bash
eksctl create iamserviceaccount \
  --name bytewax-platform-controller-manager \
  --namespace bytewax-system \
  --cluster <YOUR_CLUSTER_NAME> \
  --attach-policy-arn arn:aws:iam::aws:policy/service-role/AWSLicenseManagerConsumptionPolicy \
  --approve \
  --override-existing-serviceaccounts \
  --region <YOUR_CLUSTER_REGION>
```

--------------------------------

### Retrieve AWS License Manager Access Token

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-anywhere.md

Fetches an access token from AWS License Manager using the provided AWSMP_TOKEN. This token is used to authenticate with AWS services.

```bash
AWSMP_ACCESS_TOKEN=$(aws license-manager get-access-token \
    --output text --query '*' --token $AWSMP_TOKEN --region us-east-1)
```

--------------------------------

### Authenticate to AWS ECR and Login Helm

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/aws-marketplace-eks.md

Authenticates to the AWS Elastic Container Registry (ECR) using your AWS CLI credentials and logs into the Helm registry to pull the Bytewax Platform chart. It uses the us-east-1 region for authentication.

```bash
aws ecr get-login-password \
  --region us-east-1 | helm registry login \
  --username AWS \
  --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
```

--------------------------------

### Waxctl Dataflow Deploy Flags for Bytewax Platform on Kubernetes

Source: https://github.com/bytewax/platform-docs-content/blob/main/reference/waxctl-for-platform.md

This section details the specific flags available when deploying a dataflow to a Kubernetes cluster with the Bytewax Platform installed using the `waxctl df deploy` command. These flags control aspects like concurrency policy, recovery mechanisms, and cloud storage integration for state backups.

```bash
      --concurrency-policy string                         specifies how to treat concurrent executions of a Scheduled Dataflow (the value must be Allow, Forbid or Replace) (default "Forbid")
      --platform                                          deploy the dataflow as a bytewax.io/dataflow Custom Resource (requires Bytewax Platform installed)
      --recovery                                          stores recovery files in Kubernetes persistent volumes to allow resuming after a restart (your dataflow must have recovery enabled: https://bytewax.io/docs/getting-started/recovery)
      --recovery-backup                                   back up worker state DBs to cloud storage (must have recovery flag present and provide S3 parameters)
      --recovery-backup-interval int                      System time duration in seconds to keep extra state snapshots around
      --recovery-backup-s3-aws-access-key-id string       AWS credentials access key id
      --recovery-backup-s3-aws-secret-access-key string   AWS credentials secret access key
      --recovery-backup-s3-k8s-secret string              name of the Kubernetes secret storing AWS credentials.
      --recovery-backup-s3-url string                     s3 url to store state backups. For example, s3://mybucket/mydataflow-state-backups
      --recovery-parts int                                number of recovery parts (default 1)
      --recovery-single-volume                            use only one persistent volume for all dataflow's pods in Kubernetes
      --recovery-size string                              size of the persistent volume claim to be assign to each dataflow pod in Kubernetes (default "10Gi")
      --recovery-snapshot-interval int                    system time duration in seconds to snapshot state for recovery
      --recovery-storageclass string                      storage class of the persistent volume claim to be assign to each dataflow pod in Kubernetes
      --schedule string                                   dataflow schedule in Cron format, see https://en.wikipedia.org/wiki/Cron
```

--------------------------------

### Waxctl for Platform Management

Source: https://github.com/bytewax/platform-docs-content/blob/main/concepts/overview.md

Command-line tool for managing Bytewax dataflows within the Platform. This section links to detailed documentation for `waxctl`.

```bash
# For more information about waxctl, see the documentation [here](/reference/waxctl-for-platform).
```

--------------------------------

### Bytewax Platform Architecture

Source: https://github.com/bytewax/platform-docs-content/blob/main/concepts/overview.md

Visual representation of the Bytewax Platform components and their interactions, including user interfaces, command-line tools, and Kubernetes integration.

```mermaid
graph LR;
  client1([user])-->waxctl
  client1 --> dashboard
  k8sapi[k8s API]
  subgraph Bytewax Platform;
  dashboard(Dashboard)
  waxctl(Waxctl)
  waxapi[WaxAPI]
  operator[Operator]
  end
  waxctl --> k8sapi;
  waxapi--> k8sapi
  dashboard --> waxapi
  operator --> k8sapi
  k8sapi --> dataflow(Dataflow<br>Stack)
  classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000;
  classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff;
  classDef cluster fill:#fff,stroke:#fab90f,stroke-width:2px,color:#000;
  classDef bw fill:#fab90f,stroke:#fff,stroke-width:2px,color:#fff;
  class ingress,k8sapi,dataflow k8s;
  class client1,client2 plain;
  class cluster cluster;
  class dashboard,waxapi,waxctl,operator bw;
```

--------------------------------

### Generate Helm Chart Documentation

Source: https://github.com/bytewax/platform-docs-content/blob/main/README.md

Generates a markdown table for the Helm chart documentation. This is done by running a Docker container from the `deployment/chart` directory of the waxctl repository. The output is sent to standard output and needs to be manually copied into `reference/helm-chart.md`.

```bash
docker run --rm --volume "$(pwd):/helm-docs" -u $(id -u) jnorwood/helm-docs:latest --dry-run
```

--------------------------------

### Configure Bytewax Helm Chart for SPA PKCE Authentication

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/identity-provider.md

This YAML configuration shows how to set up the Bytewax Platform Helm chart for authentication using the Auth Code Flow with PKCE for a SPA. It includes OIDC settings and references a Kubernetes secret containing the SPA's client credentials.

```yaml
oidc:
  authIssuer: "<IDP AUTH ISSUER URL>"
  clientId: "<WEB APP CLIENT ID>"
  clientSecret: "<WEB APP CLIENT SECRET>"

dashboard:
  oidc:
    pkce: true
    existingSecret: "dashboard-pkce-data"
    clientIdKey: "client-id"
    clientSecretKey: "client-secret"
```

--------------------------------

### ByteBake Platform Configuration Parameters

Source: https://github.com/bytewax/platform-docs-content/blob/main/reference/helm-chart.md

This section details the configuration parameters for the ByteBake platform. It covers settings related to pods, services, and resource management. These parameters are typically used in deployment or configuration files.

```APIDOC
waxapi.podAnnotations: object
  Description: Annotations to be added to the pods.
  Default: {}

waxapi.podLabels: object
  Description: Labels to be applied to the pods.
  Default: {}

waxapi.port: int
  Description: The port number for the application within the pod.
  Default: 8080

waxapi.replicas: int
  Description: The desired number of replicas for the application.
  Default: 1

waxapi.resources.limits.cpu: string
  Description: The CPU limit for the pod.
  Example: "500m"

waxapi.resources.limits.memory: string
  Description: The memory limit for the pod.
  Example: "512Mi"

waxapi.resources.requests.cpu: string
  Description: The CPU request for the pod.
  Example: "100m"

waxapi.resources.requests.memory: string
  Description: The memory request for the pod.
  Example: "128Mi"

waxapi.service.annotations: object
  Description: Annotations to be added to the Kubernetes service.
  Default: {}

waxapi.service.labels: object
  Description: Labels to be applied to the Kubernetes service.
  Default: {}

waxapi.service.loadBalancerSourceRanges: list
  Description: List of IP CIDR ranges that are allowed to connect to the load balancer.
  Default: []

waxapi.service.port: int
  Description: The port number for the Kubernetes service.
  Default: 8080

waxapi.service.type: string
  Description: The type of the Kubernetes service (e.g., ClusterIP, NodePort, LoadBalancer).
  Default: "ClusterIP"

waxapi.tolerations: list
  Description: Tolerations for the pods, allowing them to be scheduled on nodes with matching taints.
  Default: []

waxapi.topologySpreadConstraints: list
  Description: Topology spread constraints for the pods, ensuring even distribution across the cluster.
  Default: []
```

--------------------------------

### Bytewax Platform Helm Chart Configuration

Source: https://github.com/bytewax/platform-docs-content/blob/main/reference/helm-chart.md

This entry details the configuration options available for the Bytewax Platform Helm Chart. It includes settings for various components such as the controller manager, dashboard, and API, along with global settings like OIDC and Kubernetes cluster domain. Each key is described with its type, default value, and a brief explanation.

```APIDOC
Bytewax Platform Helm Chart Configuration:

- billing.awsMarketplace.enabled (bool, default: `true`): Enables AWS Marketplace billing.
- controllerManager.kubeRbacProxy.image.imagePullPolicy (string, default: `"IfNotPresent"`): Image pull policy for the kube-rbac-proxy.
- controllerManager.kubeRbacProxy.image.repository (string, default: `"709825985650.dkr.ecr.us-east-1.amazonaws.com/bytewax/kube-rbac-proxy"`): Container image repository for the kube-rbac-proxy.
- controllerManager.kubeRbacProxy.image.tag (string, default: `"v0.13.0"`): Container image tag for the kube-rbac-proxy.
- controllerManager.kubeRbacProxy.resources.limits.cpu (string, default: `"500m"`): CPU limit for the kube-rbac-proxy.
- controllerManager.kubeRbacProxy.resources.limits.memory (string, default: `"128Mi"`): Memory limit for the kube-rbac-proxy.
- controllerManager.kubeRbacProxy.resources.requests.cpu (string, default: `"5m"`): CPU request for the kube-rbac-proxy.
- controllerManager.kubeRbacProxy.resources.requests.memory (string, default: `"64Mi"`): Memory request for the kube-rbac-proxy.
- controllerManager.manager.image.imagePullPolicy (string, default: `"IfNotPresent"`): Image pull policy for the controller manager.
- controllerManager.manager.image.repository (string, default: `"709825985650.dkr.ecr.us-east-1.amazonaws.com/bytewax/operator"`): Container image repository for the controller manager.
- controllerManager.manager.image.tag (string, default: `"0.3.6"`): Container image tag for the controller manager.
- controllerManager.manager.logLevel (string, default: `"info"`): Log level for the controller manager.
- controllerManager.manager.resources.limits.cpu (string, default: `"500m"`): CPU limit for the controller manager.
- controllerManager.manager.resources.limits.memory (string, default: `"128Mi"`): Memory limit for the controller manager.
- controllerManager.manager.resources.requests.cpu (string, default: `"10m"`): CPU request for the controller manager.
- controllerManager.manager.resources.requests.memory (string, default: `"64Mi"`): Memory request for the controller manager.
- controllerManager.replicas (int, default: `1`): Number of replicas for the controller manager.
- dashboard.affinity (object, default: `{}`): Affinity settings for the dashboard pods.
- dashboard.annotations (object, default: `{}`): Annotations for the dashboard pods.
- dashboard.apiUrl (string, default: `"http://localhost:8080/"`): API URL for the dashboard.
- dashboard.baseUrl (string, default: `"http://localhost:3000/"`): Base URL for the dashboard.
- dashboard.faviconUrl (string, default: `"/favicon.ico"`): URL for the dashboard favicon.
- dashboard.image.imagePullPolicy (string, default: `"IfNotPresent"`): Image pull policy for the dashboard.
- dashboard.image.repository (string, default: `"709825985650.dkr.ecr.us-east-1.amazonaws.com/bytewax/dashboard"`): Container image repository for the dashboard.
- dashboard.image.tag (string, default: `"0.1.16"`): Container image tag for the dashboard.
- dashboard.ingress.annotations (object, default: `{}`): Annotations for the dashboard ingress.
- dashboard.ingress.enabled (bool, default: `false`): Enables ingress for the dashboard.
- dashboard.ingress.labels (object, default: `{}`): Labels for the dashboard ingress.
- dashboard.labels (object, default: `{}`): Labels for the dashboard pods.
- dashboard.nodeSelector (object, default: `{}`): Node selector for the dashboard pods.
- dashboard.oidc (object, default: `{"clientId":"","clientIdKey":"","clientSecret":"","clientSecretKey":"","existingSecret":"","pkce":false}`): OpenID Connect configuration for the dashboard. If not set, global OIDC settings are used.
- dashboard.oidc.pkce (bool, default: `false`): Enables Proof Key for Code Exchange (PKCE) for the dashboard. If true, clientSecret is not set.
- dashboard.podAnnotations (object, default: `{}`): Annotations for the dashboard pods.
- dashboard.podLabels (object, default: `{}`): Labels for the dashboard pods.
- dashboard.replicas (int, default: `1`): Number of replicas for the dashboard.
- dashboard.resources.limits.cpu (string, default: `"500m"`): CPU limit for the dashboard.
- dashboard.resources.limits.memory (string, default: `"512Mi"`): Memory limit for the dashboard.
- dashboard.resources.requests.cpu (string, default: `"100m"`): CPU request for the dashboard.
- dashboard.resources.requests.memory (string, default: `"128Mi"`): Memory request for the dashboard.
- dashboard.service.annotations (object, default: `{}`): Annotations for the dashboard service.
- dashboard.service.labels (object, default: `{}`): Labels for the dashboard service.
- dashboard.service.loadBalancerSourceRanges (list, default: `[]`): Load balancer source ranges for the dashboard service.
- dashboard.service.port (int, default: `3000`): Port for the dashboard service.
- dashboard.service.type (string, default: `"ClusterIP"`): Type of the dashboard service.
- dashboard.themeUrl (string, default: `"/"`): URL for the dashboard theme.
- dashboard.title (string, default: `"Bytewax"`): Title for the dashboard.
- dashboard.tolerations (list, default: `[]`): Tolerations for the dashboard pods.
- dashboard.topologySpreadConstraints (list, default: `[]`): Topology spread constraints for the dashboard pods.
- demokeycloak.enabled (bool, default: `false`): Enables Keycloak installation for testing the Bytewax Platform.
- demokeycloak.keycloak.keycloak.image.repository (string, default: `"709825985650.dkr.ecr.us-east-1.amazonaws.com/bytewax/keycloak"`): Container image repository for Keycloak.
- demokeycloak.keycloak.keycloak.image.tag (string, default: `"20.0.1"`): Container image tag for Keycloak.
- fullnameOverride (string, default: `"bytewax-platform"`): Overrides the default full name of the release.
- kubernetesClusterDomain (string, default: `"cluster.local"`): The Kubernetes cluster domain.
- metricsService.ports[0].name (string, default: `"https"`): Name of the metrics service port.
- metricsService.ports[0].port (int, default: `8443`): Port number for the metrics service.
- metricsService.ports[0].protocol (string, default: `"TCP"`): Protocol for the metrics service port.
- metricsService.ports[0].targetPort (string, default: `"https"`): Target port for the metrics service.
- metricsService.type (string, default: `"ClusterIP"`): Type of the metrics service.
- oidc (object, default: `{"authIssuer":"https://identiyprovider.com/realms/bytewax","authIssuerKey":"","clientId":"clientid","clientIdKey":"","clientSecret":"clientsecret","clientSecretKey":"","existingSecret":""}`): Global OpenID Connect configuration.
- waxapi.affinity (object, default: `{}`): Affinity settings for the waxapi pods.
- waxapi.allowedHosts (string, default: `"*"`): Allowed hosts for the waxapi.
- waxapi.annotations (object, default: `{}`): Annotations for the waxapi pods.
- waxapi.image.imagePullPolicy (string, default: `"IfNotPresent"`): Image pull policy for the waxapi.
- waxapi.image.repository (string, default: `"709825985650.dkr.ecr.us-east-1.amazonaws.com/bytewax/waxapi-ecr"`): Container image repository for the waxapi.
- waxapi.image.tag (string, default: `"0.2.0"`): Container image tag for the waxapi.
- waxapi.ingress.annotations (object, default: `{}`): Annotations for the waxapi ingress.
- waxapi.ingress.enabled (bool, default: `false`): Enables ingress for the waxapi.
- waxapi.ingress.labels (object, default: `{}`): Labels for the waxapi ingress.
- waxapi.labels (object, default: `{}`): Labels for the waxapi pods.
- waxapi.nodeSelector (object, default: `{}`): Node selector for the waxapi pods.
```

--------------------------------

### List Dataflows

Source: https://github.com/bytewax/platform-docs-content/blob/main/dataflow-guides/cloud-backup.md

Lists all deployed dataflows. The `-A` flag shows dataflows across all namespaces.

```sh
waxctl df ls -A
```

--------------------------------

### Bytewax Platform Ecosystem Diagram

Source: https://github.com/bytewax/platform-docs-content/blob/main/concepts/architecture.md

Illustrates the broader Bytewax Platform ecosystem, including user interaction via Waxctl and Dashboard, integration with Kubernetes API, and external services like Prometheus, OpenTelemetry, and S3 for metrics, traces, and recovery snapshots.

```mermaid
graph LR;
  client([user])
  waxctl[Waxctl]
  client-.terminal.->waxctl;
  waxctl --> k8sapi;
  client --> dashboard
  dashboard --> service
  idp(Identity<br>Provider) <--> |OpenID Connect<br>flow|service;
  idp <--> |OpenID Connect<br>flow|dashboard
  pod1 --> |metrics|prom[(Prometheus)];
  pod1 --> |traces|otel[(OpenTelemetry)];
  pod1 --> |recovery<br>snapshots|bucket[(S3)];
  subgraph Kubernetes cluster
  k8sapi[kubeAPI]
  pod1[Dataflow<br>stack]
  subgraph Bytewax Platform
  operator[Operator]
  service[WaxAPI]
  dashboard[Dashboard]
  end
  service --> k8sapi
  operator --> k8sapi
  k8sapi-->pod1;
  end
  classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000;
  classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff;
  classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5;
  classDef bw fill:#fab90f,stroke:#fff,stroke-width:2px,color:#fff;
  class ingress,pod1,k8sapi k8s;
  class client plain;
  class cluster cluster;
  class service,dashboard,operator,waxctl bw;
```

--------------------------------

### Bytewax Platform Architecture Diagram

Source: https://github.com/bytewax/platform-docs-content/blob/main/concepts/architecture.md

Visual representation of the Bytewax Platform components and their interactions within a Kubernetes cluster, showing the flow of data and control from user interactions to Dataflow pods.

```mermaid
graph LR;
  client([user])-. CRUD Dataflows<br>using waxctl, waxapi or dashboard .->ingress[Dataflow CR];
  ingress-->|events|service[Bytewax<br>Platform];
  subgraph Kubernetes cluster
  ingress;
  service-->pod1[Dataflow Pod 1];
  service-->pod2[Dataflow Pod 2];
  end
  classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000;
  classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff;
  classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5;
  classDef bw fill:#fab90f,stroke:#fff,stroke-width:2px,color:#fff;
  class ingress,pod1,pod2 k8s;
  class client plain;
  class cluster cluster;
  class service bw;
```

--------------------------------

### Dataflow Artifacts Download Configuration

Source: https://github.com/bytewax/platform-docs-content/blob/main/reference/dataflow-crd.md

Configures the download of tar files from a URL, supporting public URLs or private GitHub repositories. Includes options for authentication.

```APIDOC
Dataflow.spec.artifactsDownload:
  url: string (required) - Url of the tar file to download
  secretName: string (optional) - Name of the Kubernetes secret storing Personal Access Token (must contain the key TOKEN)
  token: string (optional) - Personal Access Token
```

--------------------------------

### Deploy Dataflow with Cloud Recovery

Source: https://github.com/bytewax/platform-docs-content/blob/main/dataflow-guides/cloud-backup.md

Deploys a dataflow with cloud recovery and backup enabled, specifying an S3 bucket for backups and Kubernetes secrets for credentials. This command initiates the dataflow and sets up its state persistence.

```sh
waxctl df deploy simple_slow.py \
  --platform \
  --name demo01 \
  --recovery=true \
  --recovery-backup=true \
  --recovery-backup-s3-url=s3://backup-tests/demo01 \
  --recovery-backup-s3-k8s-secret=aws-credentials
```

--------------------------------

### Ingress Controller Configuration

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/external-access.md

Configuration for enabling external access via an Ingress Controller. This includes specifying hostnames, TLS secrets, and annotations for controller-specific settings.

```yaml
ingress:
  enabled: false
  # For Kubernetes >= 1.18 you should specify the ingress-controller via the field ingressClassName
  # See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
  # ingressClassName: nginx
  annotations: {}
  labels: {}
  hosts:
    - waxapi.yourdomain.com
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  labels:
    name: value
  tls:
    # Secrets must be manually created in the namespace.
    - secretName: waxapi-tls
      hosts:
        - waxapi.yourdomain.com
```

--------------------------------

### Configure Bytewax Helm Chart with Plain OIDC Values

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/identity-provider.md

This YAML configuration demonstrates setting OIDC values directly within the Bytewax Platform Helm chart values file. This method is suitable for non-production environments due to security considerations.

```yaml
oidc:
  authIssuer: "<YOUR IDP AUTH ISSUER URL>"
  clientId: "<YOUR CLIENT ID>"
  clientSecret: "<YOUR CLIENT SECRET>"
```

--------------------------------

### LoadBalancer Service Configuration

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/external-access.md

Configuration for exposing the service externally using a Kubernetes LoadBalancer. This includes setting the service type, port, and optional annotations and labels.

```yaml
waxapi:
  service:
    annotations: {}
    labels: {}
    type: LoadBalancer
    # List of IP ranges that are allowed to access the load balancer (if supported)
    loadBalancerSourceRanges: []
    port: 8080
```

--------------------------------

### Create Kubernetes Secret for OIDC Configuration

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/identity-provider.md

This bash script demonstrates how to create a Kubernetes secret to store OIDC configuration details such as the authentication issuer URL, client ID, and client secret. These values are essential for the Bytewax Platform to authenticate users via an external Identity Provider.

```bash
kubectl create secret generic my-oidc-config \
--from-literal=auth-issuer=<YOUR IDP AUTH ISSUER URL> \
--from-literal=client-id=<YOUR CLIENT ID> \
--from-literal=client-secret=<YOUR CLIENT SECRET> \
--namespace bytewax-system
```

--------------------------------

### Configure Bytewax Helm Chart with Existing OIDC Secret

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/identity-provider.md

This YAML configuration shows how to reference an existing Kubernetes secret for OIDC settings within the Bytewax Platform Helm chart. It specifies the secret name and the keys within the secret that hold the authentication issuer URL, client ID, and client secret.

```yaml
oidc:
  existingSecret: "my-oidc-config"
  authIssuerKey: "auth-issuer"
  clientIdKey: "client-id"
  clientSecretKey: "client-secret"
```

--------------------------------

### WaxAPI OpenAPI Specifications

Source: https://github.com/bytewax/platform-docs-content/blob/main/reference/waxapi.md

Information on accessing the OpenAPI (Swagger) documentation for WaxAPI. This includes a UI and a JSON specification file for detailed endpoint information and import into tools like Postman.

```APIDOC
OpenAPI v2 Specification:
  - UI: /swagger/index.html
  - JSON Spec: /swagger/doc.json

Related Resources:
  - Bytewax Platform Documentation: /api/openapi/v0.7.2
```

--------------------------------

### Create Kubernetes Secret for Dashboard SPA PKCE Data

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/identity-provider.md

This bash script creates a Kubernetes secret to store client ID and client secret specifically for a Single Page Application (SPA) using the Auth Code Flow with PKCE. This is used when the Identity Provider requires this flow for web applications.

```bash
kubectl create secret generic dashboard-pkce-data \
  --from-literal=client-id=<YOUR SPA CLIENT ID> \
  --from-literal=client-secret=<YOUR SPA CLIENT SECRET> \
  --namespace bytewax-system
```

--------------------------------

### Update Dataflow CRD Reference

Source: https://github.com/bytewax/platform-docs-content/blob/main/README.md

Generates the `reference/dataflow-crd.md` file by running a Docker container. This process requires defining absolute paths for the waxctl and platform-docs-content repositories. The command uses `ghcr.io/fybrik/crdoc:latest` to process CRD resources and a markdown template.

```bash
export waxctl_repo_path=<ABSOLUTE PATH OF WAXCTL REPOSITORY DIRECTORY>
export content_path=<ABSOLUTE PATH OF PLATFORM CONTENT REPOSITORY DIRECTORY>

docker run -u $(id -u):$(id -g) --rm \
  -v $waxctl_repo_path:/workdir \
  -v $content_path/reference:/reference \
  ghcr.io/fybrik/crdoc:latest \
  --resources /workdir/operator/config/crd/bases \
  --output /reference/dataflow-crd.md \
  --template /reference/markdown.tmpl
```

--------------------------------

### NodePort Service Configuration

Source: https://github.com/bytewax/platform-docs-content/blob/main/setup/external-access.md

Configuration for exposing the service externally using a Kubernetes NodePort. This allows access via a specific port on each node, including specifying the nodePort and service port.

```yaml
service:
  annotations: {}
  labels: {}
  type: NodePort
  # Specify a specific node port when type is NodePort
  nodePort: 32500
  port: 8080
```

--------------------------------

### Dataflow Resource Schema

Source: https://github.com/bytewax/platform-docs-content/blob/main/reference/dataflow-crd.md

Defines the schema for the Dataflow resource, including apiVersion, kind, metadata, spec, and status. It references Kubernetes API documentation for metadata and provides links to DataflowSpec and DataflowStatus definitions.

```APIDOC
Dataflow:
  apiVersion: string (bytewax.io/v1alpha1)
  kind: string (Dataflow)
  metadata: object (Refer to the Kubernetes API documentation for the fields of the `metadata` field.)
  spec: object (DataflowSpec defines the desired state of Dataflow)
  status: object (DataflowStatus defines the observed state of Dataflow)
```

--------------------------------

### Switch Kubectl Context

Source: https://github.com/bytewax/platform-docs-content/blob/main/dataflow-guides/cloud-backup.md

Changes the current kubectl context to a different Kubernetes cluster, allowing management of resources in that cluster.

```sh
kubectl config use-context demo1
```