### Install Istio in OKE Clusters

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Install Istio using the 'istioctl install' command with a default profile and a cluster-specific configuration file.

```bash
for cluster in c1 c2; do
  ktx $cluster
  istioctl install --set profile=default -f $HOME/$cluster.yaml
done
```

--------------------------------

### Operator Configuration with Cloud-Init Example

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

This example demonstrates how to configure the operator instance with custom cloud-init scripts for initialization. It specifies creation settings, upgrade behavior, user, image drift allowance, cloud-init content, and the shape of the operator instance.

```hcl
create_operator            = true
operator_upgrade           = false
operator_user              = "opc"
operator_allow_image_drift = true

operator_cloud_init = [
  {
    content      = <<-EOT
    runcmd:
    - echo "Operator cloud_init using cloud-config"
    EOT
    content_type = "text/cloud-config"
  }
]

operator_shape = {
  shape                     = "VM.Standard.E4.Flex"
  ocpus                     = 1
  memory                    = 4
  boot_volume_size          = 50
  baseline_ocpu_utilization = 100
}
```

--------------------------------

### Verify Istio Installation

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Run 'istioctl verify-install' in each cluster to confirm that Istio has been installed correctly.

```bash
for cluster in c1 c2; do
  ktx $cluster
  istioctl verify-install
done
```

--------------------------------

### Test Terraform Installation

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/prerequisites.md

Verifies that Terraform has been installed correctly and is accessible from the command line. Displays the installed Terraform version.

```bash
terraform -v
Terraform v1.x.x
```

--------------------------------

### Basic OKE Cluster Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Example of essential parameters for creating a basic OKE cluster.

```hcl
cluster_name       = "oke-example"
kubernetes_version = "v1.34.2"
```

--------------------------------

### Install Terraform on macOS

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/prerequisites.md

Installs Terraform version 1.3.0+ on macOS using Homebrew. Ensure Homebrew is installed and updated.

```bash
brew install terraform
```

--------------------------------

### Enhanced OKE Cluster Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Example demonstrating parameters for an enhanced OKE cluster, including network and control plane settings.

```hcl
cluster_name                      = "oke"
cluster_type                      = "enhanced"
cni_type                          = "flannel"
kubernetes_version                = "v1.34.2"
assign_public_ip_to_control_plane = true
```

--------------------------------

### Verify Cluster Connectivity and Nodes

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Use the 'ktx' command to switch contexts and 'k get nodes' to check the status of nodes in each cluster.

```bash
for cluster in c1 c2; do
  ktx $cluster
  k get nodes
done
```

--------------------------------

### Configure Bastion Host

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Example configuration for creating and customizing a bastion host. This snippet shows how to enable bastion creation, define allowed SSH CIDR blocks, specify image type, disable upgrades, set the SSH user, and configure the instance shape.

```hcl
create_bastion              = true
bastion_allowed_cidrs       = ["0.0.0.0/0"]
bastion_image_type          = "platform"
bastion_upgrade             = false
bastion_user                = "opc"

bastion_shape = {
  shape                     = "VM.Standard.E4.Flex"
  ocpus                     = 1
  memory                    = 4
  boot_volume_size          = 50
  baseline_ocpu_utilization = 100
}
```

--------------------------------

### Install Terraform on Oracle Linux

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/prerequisites.md

Installs Terraform version 1.3.0+ on Oracle Linux using yum. Ensure your system has the developer release repository configured.

```bash
yum -y install oraclelinux-developer-release-el7 && yum -y install terraform
```

--------------------------------

### Basic Node Pool Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Defines a basic node pool with a specified size and shape. Use this for standard worker node setups.

```hcl
worker_pool_mode = "node-pool"
worker_pool_size = 1

worker_pools = {
  oke-vm-standard = {}
  oke-vm-standard-large = {
    size             = 1
    shape            = "VM.Standard.E4.Flex"
    ocpus            = 8
    memory           = 128
    boot_volume_size = 200
  }
}
```

--------------------------------

### Retrieve OKE Cluster Information

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

After applying the Terraform configuration, use 'terraform output' to get essential cluster details such as cluster ID, endpoints, and SSH access information.

```bash
terraform output cluster_id
terraform output cluster_endpoints
terraform output ssh_to_bastion
terraform output ssh_to_operator
```

--------------------------------

### Verify Kubernetes Node Connectivity

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

After connecting to the cluster (either via operator host or kubeconfig), use 'kubectl get nodes' to verify that the Kubernetes nodes are accessible and registered with the cluster.

```bash
kubectl get nodes
```

--------------------------------

### Cluster Network Worker Pool Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Example of configuring a self-managed worker pool in cluster-network mode for HPC/GPU workloads, including secondary VNIC settings.

```hcl
worker_pools = {
  oke-bm-gpu-rdma = {
    mode          = "cluster-network"
    size          = 1
    shape         = "BM.GPU.B4.8"
    placement_ads = [1]
    image_id      = "ocid1.image..."
    secondary_vnics = {
      "vnic-display-name" = {
        nic_index = 1
        subnet_id = "ocid1.subnet..."
      }
    }
  }
}
```

--------------------------------

### Managed Node Pool in Compute Cluster Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Example of configuring a managed node pool within a compute cluster for RDMA workloads. Set `use_compute_cluster = true` to create a dedicated cluster or provide `compute_cluster_id` to attach to an existing one.

```hcl
worker_pools = {
  oke-bm-gpu-rdma-np = {
    mode                = "node-pool"
    size                = 2
    shape               = "BM.GPU.B4.8"
    placement_ads       = [1]
    use_compute_cluster = true
  }
}
```

--------------------------------

### Node Pool with GVA Secondary VNICs

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Example of configuring a node pool with GVA secondary VNICs for advanced networking. This requires `cni_type = "npn"` and is not compatible with `mode = "virtual-node-pool"`. The `pods` map within `gva_secondary_vnics` allows customization of IP count, source/destination checks, and network security groups for pod subnets.

```hcl
cni_type = "npn"

worker_pools = {
  gva-node-pool = {
    mode                = "node-pool"
    size                = 2
    shape               = "VM.Standard..."
    network_launch_type = "..."

    gva_secondary_vnics = {
      pods = {
        # Defaults to subnet_key = "pods"; set subnet_id to use an explicit subnet OCID.
        ip_count               = 16
        skip_source_dest_check = true
        nsg_ids                = ["pods"]
      }
    }
  }
}
```

--------------------------------

### Deploy HelloWorld Service in Both Clusters

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Deploys the HelloWorld service and enables Istio injection in the 'sample' namespace for both clusters. Ensure 'c1' and 'c2' are valid context names.

```bash
for cluster in c1 c2; do
  kubectl create --context="${cluster}" namespace sample
  kubectl label --context="${cluster}" namespace sample istio-injection=enabled
  kubectl apply --context="${cluster}" -f samples/helloworld/helloworld.yaml -l service=helloworld -n sample
done
```

--------------------------------

### Deploy v1 to Cluster c1

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Applies version 1 of the HelloWorld service to the first cluster and verifies the pod is running. Replace CTX_CLUSTER1 with your cluster context.

```bash
kubectl apply --context="${CTX_CLUSTER1}" \
    -f samples/helloworld/helloworld.yaml \
    -l version=v1 -n sample

kubectl get pod --context="${CTX_CLUSTER1}" -n sample -l app=helloworld
```

--------------------------------

### Initialize, Plan, and Apply Terraform Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

Execute Terraform commands to initialize the project, review the planned changes, and apply them to provision the OCI OKE cluster.

```bash
terraform init
terraform plan
terraform apply
```

--------------------------------

### Deploy v2 to Cluster c2

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Applies version 2 of the HelloWorld service to the second cluster and verifies the pod is running. Replace CTX_CLUSTER2 with your cluster context.

```bash
kubectl apply --context="${CTX_CLUSTER2}" \
    -f samples/helloworld/helloworld.yaml \
    -l version=v2 -n sample

kubectl get pod --context="${CTX_CLUSTER2}" -n sample -l app=helloworld
```

--------------------------------

### Apply Terraform Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Execute Terraform to provision the OKE clusters and associated resources.

```bash
terraform apply --auto-approve
```

--------------------------------

### Create and Label Istio System Namespace

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Create the 'istio-system' namespace in each cluster and label it with the network topology information.

```bash
for cluster in c1 c2; do
  ktx $cluster
  k create ns istio-system
  k label namespace istio-system topology.istio.io/network=$cluster
done
```

--------------------------------

### Apply Gateway to Expose Services

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Apply the 'expose-services.yaml' configuration to create an Istio Gateway that exposes all services through the east-west ingress gateway in each cluster.

```bash
cd $ISTIO_HOME
for cluster in c1 c2; do
  ktx $cluster
  k apply -f samples/multicluster/expose-services.yaml
done
```

--------------------------------

### Additional NSG Rules for Workers

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Add custom Network Security Group rules to worker nodes. This example allows TCP traffic on port 8080 from a specific VCN CIDR block.

```hcl
allow_rules_workers = {
  "Allow TCP 8080 from VCN" = {
    protocol = 6, port = 8080, source = "10.0.0.0/16", source_type = "CIDR_BLOCK",
  },
}
```

--------------------------------

### Check Istio Service Load Balancers

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Inspect the services in the 'istio-system' namespace to verify that load balancers have been provisioned correctly.

```bash
for cluster in c1 c2; do
  ktx $cluster
  k -n istio-system get svc
done
```

--------------------------------

### NSG Rules Attached to Subnets

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Define Network Security Group rules directly within the NSG configuration for standard or custom NSGs. This example shows rules for workers and a custom 'app' NSG.

```hcl
nsgs = {
  workers = {
    rules = {
      "Allow TCP 8080 from VCN" = {
        protocol = 6, port = 8080, source = "10.0.0.0/16", source_type = "CIDR_BLOCK",
      }
    }
  }
  app = {
    create = "always"
    rules = {
      "Allow HTTPS from workers" = {
        protocol = 6, port = 443, source = "workers", source_type = "NETWORK_SECURITY_GROUP",
      }
    }
  }
}
```

--------------------------------

### Clone the Terraform OCI OKE Repository

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

Clone the repository to your local machine and navigate into the created directory. This sets up the necessary files for Terraform.

```bash
git clone https://github.com/oracle-terraform-modules/terraform-oci-oke.git tfoke

cd tfoke
```

--------------------------------

### Configure OKE Cluster Add-ons

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Enable or remove cluster add-ons like CertManager and NvidiaGpuPlugin. You can specify resource deletion behavior and custom configurations.

```hcl
cluster_addons = {
  "CertManager" = {
    remove_addon_resources_on_delete = true
    override_existing                = true
    configurations = [
      { key = "numOfReplicas", value = "1" }
    ]
  }
  "NvidiaGpuPlugin" = {
    remove_addon_resources_on_delete = true
  }
}

cluster_addons_to_remove = {
  Flannel = { remove_k8s_resources = true }
}
```

--------------------------------

### Check Istio Pod Status

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Verify that all Istio pods in the 'istio-system' namespace are running as expected.

```bash
for cluster in c1 c2; do
  ktx $cluster
  k -n istio-system get pods
done
```

--------------------------------

### SSH to OKE Operator Host via Bastion

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

Connect to the OKE operator host by first SSHing into the bastion host, then using the bastion's IP to proxy the connection to the operator. Ensure you use the correct SSH key.

```bash
# Use the output from terraform output ssh_to_operator
ssh -o ProxyCommand='ssh -W %h:%p -i ~/.ssh/oke_key opc@<bastion_ip>' -i ~/.ssh/oke_key opc@<operator_ip>
```

--------------------------------

### Configure OKE Provider Parameters

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Set up the OCI provider details for Terraform, including API credentials and region information.

```hcl
api_fingerprint = ""

api_private_key_path = "~/.oci/oci_rsa.pem"

home_region = "ashburn"

tenancy_id = "ocid1.tenancy.oc1.."

user_id = "ocid1.user.oc1.."

compartment_id = "ocid1.compartment.oc1.."
```

--------------------------------

### Configure Terraform Path on Linux/macOS

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/prerequisites.md

Moves the Terraform binary to a directory included in your system's PATH for easy command-line access. Adjust the source path to where you extracted the binary.

```bash
sudo mv /path/to/terraform /usr/local/bin
```

--------------------------------

### Configure SSH Key Pair for OKE

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Specify the paths to your SSH private and public key files for cluster access.

```hcl
# ssh
ssh_private_key_path = "~/.ssh/id_rsa"
ssh_public_key_path  = "~/.ssh/id_rsa.pub"
```

--------------------------------

### Deploy Sleep Client Pod in Both Clusters

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Deploys the Sleep client pod to both clusters. This client will be used to generate traffic for testing.

```bash
kubectl apply --context="${CTX_CLUSTER1}" \
    -f samples/sleep/sleep.yaml -n sample
kubectl apply --context="${CTX_CLUSTER2}" \
    -f samples/sleep/sleep.yaml -n sample
```

--------------------------------

### SSH to OKE Operator Host

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Connect to the operator host using the provided SSH command, which includes a proxy command for bastion access.

```bash
ssh_to_operator = "ssh -o ProxyCommand='ssh -W %h:%p -i ~/.ssh/id_rsa opc@<bastion_ip>' -i ~/.ssh/id_rsa opc@<operator_ip>"
```

--------------------------------

### Configure OKE Node Pools

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Define the shape, OCPUs, memory, size, and boot volume size for your OKE node pools.

```hcl
nodepools = {
  np1 = {
    shape            = "VM.Standard.E4.Flex",
    ocpus            = 2,
    memory           = 64,
    size             = 2,
    boot_volume_size = 150,
  }
}
```

--------------------------------

### Configure Additional OKE Parameters

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Specify the Kubernetes version, cluster type, and control plane configuration.

```hcl
kubernetes_version = "v1.32.1"

cluster_type = "basic"

oke_control_plane = "private"
```

--------------------------------

### Configure Kubeconfig for kubectl Access

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

Save the cluster's kubeconfig output to a file and set the KUBECONFIG environment variable to point to this file. This allows `kubectl` to communicate with your OKE cluster.

```bash
terraform output -raw cluster_kubeconfig > ~/.kube/config-oke
export KUBECONFIG=~/.kube/config-oke
```

--------------------------------

### Create Istio CA Certificates Secret

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Create a Kubernetes secret named 'cacerts' in the 'istio-system' namespace for each cluster, containing the generated CA certificates and keys.

```bash
for cluster in c1 c2; do
  ktx $cluster
  kubectl create secret generic cacerts -n istio-system \
      --from-file=$cluster/ca-cert.pem \
      --from-file=$cluster/ca-key.pem \
      --from-file=$cluster/root-cert.pem \
      --from-file=$cluster/cert-chain.pem
done
```

--------------------------------

### Generate Istio Self-Signed Certificates

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Generate CA certificates and keys for each cluster using Istio's provided Makefile. This is necessary for establishing trust in a multi-cluster environment.

```bash
export ISTIO_HOME=/home/opc/istio-1.20.2
cd $ISTIO_HOME/tools/certs 
make -f Makefile.selfsigned.mk c1-cacerts
make -f Makefile.selfsigned.mk c2-cacerts
```

--------------------------------

### Using Existing Subnets

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Reference existing subnets by their OCIDs. This is useful when managing network resources outside of this module.

```hcl
subnets = {
  operator = { id = "ocid1.subnet..." }
  cp       = { id = "ocid1.subnet..." }
  workers  = { id = "ocid1.subnet..." }
}
```

--------------------------------

### Configure OKE Clusters Regions and Networking

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Define the network CIDR blocks and regions for your OKE clusters.

```hcl
# clusters
clusters = {
  c1 = { region = "sydney", vcn = "10.1.0.0/16", pods = "10.201.0.0/16", services = "10.101.0.0/16", enabled = true }
  c2 = { region = "melbourne", vcn = "10.2.0.0/16", pods = "10.202.0.0/16", services = "10.102.0.0/16", enabled = true }
}
```

--------------------------------

### Enable Endpoint Discovery with Remote Secrets

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Create and apply remote secrets to enable endpoint discovery between clusters. This allows clusters to be aware of each other's services.

```bash
istioctl create-remote-secret \
  --context="${CTX_CLUSTER1}" \
  --name="${CTX_CLUSTER1}" | \
  kubectl apply -f - --context="${CTX_CLUSTER2}"

istioctl create-remote-secret \
  --context="${CTX_CLUSTER2}" \
  --name="${CTX_CLUSTER2}" | \
  kubectl apply -f - --context="${CTX_CLUSTER1}"
```

--------------------------------

### Autoscaled Node Pool Configuration

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Configures a node pool with autoscaling enabled. Set `min_size` and `max_size` to define the scaling boundaries. `ignore_initial_pool_size` is useful to let the autoscaler manage the initial count.

```hcl
worker_pools = {
  np-autoscaled = {
    size                     = 2
    min_size                 = 1
    max_size                 = 3
    autoscale                = true
    ignore_initial_pool_size = true
  }
}
```

--------------------------------

### Define Cluster and Worker Pool Parameters

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

Configure essential parameters for cluster creation and worker pool settings. This includes enabling cluster creation, naming the cluster, and specifying Kubernetes version and worker pool size.

```hcl
# Cluster
create_cluster     = true
cluster_name       = "oke-cluster"
kubernetes_version = "v1.34.2"

# Workers
worker_pool_mode = "node-pool"
worker_pool_size = 1

worker_pools = {
  np1 = {
    size  = 1
  }
}
```

--------------------------------

### Generate Traffic from c1

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Generates 100 requests from the Sleep client in cluster c1 to the HelloWorld service. The responses should alternate between v1 (from c1) and v2 (from c2), verifying cross-cluster communication.

```bash
for i in $(seq 1 100); do
kubectl exec --context="${CTX_CLUSTER1}" -n sample -c sleep \
    "$(kubectl get pod --context="${CTX_CLUSTER1}" -n sample -l \
    app=sleep -o jsonpath='{.items[0].metadata.name}')" \
    -- curl -sS helloworld.sample:5000/hello
done
```

--------------------------------

### Configure Service Accounts with RBAC

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Enable the creation of Kubernetes service accounts with Role-Based Access Control (RBAC). Define specific service accounts with their names, namespaces, cluster roles, and role bindings.

```hcl
create_service_account = true

service_accounts = {
  example_cluster_role_binding = {
    sa_name                 = "sa1"
    sa_namespace            = "kube-system"
    sa_cluster_role         = "cluster-admin"
    sa_cluster_role_binding = "sa1-crb"
  }
}
```

--------------------------------

### OIDC Authentication for GitHub Actions

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Configuration for enabling OIDC token authentication for enhanced OKE clusters, specifically for GitHub Actions.

```hcl
cluster_type                      = "enhanced"
oidc_token_auth_enabled           = true
oidc_token_authentication_config  = {
  client_id      = "oke-kubernetes-cluster"
  issuer_url     = "https://token.actions.githubusercontent.com"
  username_claim = "sub"
  required_claims = [
    { key = "repository", value = "GITHUB_ACCOUNT/GITHUB_REPOSITORY" },
    { key = "workflow",   value = "oke-oidc" },
    { key = "ref",        value = "refs/heads/main" },
  ]
}
```

--------------------------------

### Automatic Subnet Creation

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/terraformoptions.md

Configure subnets by specifying the number of bits for automatic creation. The module calculates CIDR blocks based on these values.

```hcl
subnets = {
  bastion  = { newbits = 13 }
  operator = { newbits = 13 }
  cp       = { newbits = 13 }
  int_lb   = { newbits = 11 }
  pub_lb   = { newbits = 11 }
  workers  = { newbits = 4 }
  pods     = { newbits = 2 }
}
```

--------------------------------

### Set Environment Variables for Multi-Cluster Verification

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Define environment variables to easily reference the contexts of the two clusters for subsequent commands.

```bash
export CTX_CLUSTER1=c1
export CTX_CLUSTER2=c2
```

--------------------------------

### Generate SSH Key Pair

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/prerequisites.md

Generates an RSA SSH key pair with a key length of 4096 bits, named `oke_key`, in the `~/.ssh/` directory. This key pair is used for accessing bastion and operator hosts.

```bash
ssh-keygen -t rsa -b 4096 -f ~/.ssh/oke_key
```

--------------------------------

### Enable Kubeconfig Output in Terraform

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

Set 'output_detail = true' in your Terraform configuration to enable the output of the cluster's kubeconfig, which is necessary for direct kubectl access.

```hcl
output_detail = true
```

--------------------------------

### Generate Traffic from c2

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/examples/istio-mc/README.md

Generates 100 requests from the Sleep client in cluster c2 to the HelloWorld service. The responses should alternate between v1 (from c1) and v2 (from c2), verifying cross-cluster communication.

```bash
for i in $(seq 1 100); do
kubectl exec --context="${CTX_CLUSTER2}" -n sample -c sleep \
    "$(kubectl get pod --context="${CTX_CLUSTER2}" -n sample -l \
    app=sleep -o jsonpath='{.items[0].metadata.name}')" \
    -- curl -sS helloworld.sample:5000/hello
done
```

--------------------------------

### Update OKE Cluster Infrastructure

Source: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/quickstart.md

Modify your `terraform.tfvars` file to reflect desired changes, such as Kubernetes version, worker pool size, or adding new worker pools. Then, run 'terraform plan' and 'terraform apply' to update the infrastructure.

```bash
# Modify terraform.tfvars as needed
terraform plan
terraform apply
```