### Download CCM Provider Configuration Example

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/install-oci-ccm.md

This command downloads an example configuration file for the OCI Cloud Controller Manager (CCM) that utilizes Instance Principals for authentication. This file needs to be updated with specific OCI resource OCIDs and settings before use.

```shell
curl -L https://raw.githubusercontent.com/oracle/oci-cloud-controller-manager/master/manifests/provider-config-instance-principals-example.yaml -o cloud-provider-example.yaml
```

--------------------------------

### Initialize Management Cluster with CAPOCI

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/install-cluster-api.md

Initializes the management cluster and installs CAPOCI. It shows commands for using the latest version or a specific released version of CAPOCI.

```bash
clusterctl init --infrastructure oci
```

```bash
clusterctl init --infrastructure oci:vX.X.X
```

--------------------------------

### Build and Start Kind Cluster with Tilt (Make)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/local-dev.md

This command uses Make to build a Kind cluster and start Tilt. This is the primary command for initiating the local development environment for the OCI Cluster API Provider.

```makefile
make tilt-up
```

--------------------------------

### Example Cluster Template (YAML)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/proposals/20230117-cluster-level-identity.md

This YAML template demonstrates the configuration of OCICluster and OCIClusterIdentity resources. It shows how to link an OCICluster to an OCIClusterIdentity using `identityRef` and defines the necessary fields for the identity, including secret references and allowed namespaces.

```yaml
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OCICluster
spec:
  identityRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: OCIClusterIdentity
    name: <test-identity>
    namespace: <test>
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OCIClusterIdentity
metadata:
  name: test-identity
  namespace: test
spec:
  type: UserPrincipal
  clientSecret: 
    name: user-credentials
    namespace: secret-namespace
  allowedNamespaces:
    list:
      - test
---
apiVersion: v1
kind: Secret
metadata:
  name: user-credentials
  namespace: secret-namespace
type: Opaque
data:
  tenancy: <>
  user: <>
  passphrase: <> -> optional
  key: <>
  fingerprint: <>
```

--------------------------------

### Export OCI Authentication Settings (Instance Principal)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/test/README.md

Exports an environment variable to indicate that instance principal should be used for OCI authentication. This is a simpler setup for resources running within OCI that have the necessary IAM policies attached.

```bash
export USE_INSTANCE_PRINCIPAL_B64="$(echo -n \"true\" | base64 | tr -d '\n')"
```

--------------------------------

### Install Calico CNI Provider

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-workload-cluster.md

Installs the Calico CNI (Container Network Interface) provider into the workload cluster using the provided kubeconfig file. This step is crucial for enabling network communication between pods and is a prerequisite for cluster nodes to reach the 'Ready' state.

```bash
kubectl --kubeconfig=<cluster-name>.kubeconfig \
  apply -f https://docs.projectcalico.org/v3.21/manifests/calico.yaml
```

--------------------------------

### Execute Test Script with Focus Parameter

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/test/README.md

Executes the test script (`ci-e2e.sh`) while controlling test execution using GINKGO environment variables. `GINKGO_SKIP` can be used to skip tests, and `GINKGO_FOCUS` allows running only tests matching a specific pattern, such as 'VCNPeering'.

```bash
GINKGO_SKIP="" GINKGO_FOCUS="VCNPeering" scripts/ci-e2e.sh
```

--------------------------------

### Configure OKE Addons in OCIManagedControlPlane

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/managed/features.md

This configuration specifies Addons to be installed in the OKE cluster. It is part of the `OCIManagedControlPlane` spec, and an example shows how to enable the 'CertManager' addon. More details on parameters are in the CAPOCI API Reference.

```yaml
kind: OCIManagedControlPlane
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
spec:
  clusterType: "ENHANCED_CLUSTER"
  addons:
  - name: CertManager
```

--------------------------------

### Install Nvidia GPU Operator with Helm

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-gpu-workload-cluster.md

Installs the Nvidia GPU Operator using Helm. Requires Helm to be installed and the Nvidia Helm repository configured. It creates a dedicated namespace 'gpu-operator' and waits for the installation to complete.

```bash
helm install --wait --generate-name \
     -n gpu-operator --create-namespace \
     nvidia/gpu-operator
```

--------------------------------

### Set KUBECONFIG Environment Variable

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/install-cluster-api.md

Sets the KUBECONFIG environment variable to point to the correct Kubeconfig file, necessary when not using kind for the management cluster.

```shell
export KUBECONFIG=/path/to/kubeconfig
```

--------------------------------

### Configure Preemptible Instances with OCIMachineTemplate

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/customize-worker-node.md

This configuration sets up preemptible instances, which are cost-effective but can be terminated by OCI. The example ensures that the boot volume is preserved upon preemption.

```yaml
kind: OCIMachineTemplate
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
spec:
  template:
    spec:
      preemptibleInstanceConfig:
        terminatePreemptionAction:
          preserveBootVolume: false
```

--------------------------------

### Execute OCI Test Script

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/test/README.md

Sets the container registry and a flag for local testing, then executes the main end-to-end test script (`ci-e2e.sh`). This command initiates the automated testing process for the OCI Cluster API Provider.

```bash
export REGISTRY="iad.ocir.io/<namespace>"
export LOCAL_ONLY=false
scripts/ci-e2e.sh
```

--------------------------------

### Download Calico Release

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-windows-workload-cluster.md

Downloads a specific version of the Calico release archive using curl. This archive contains the necessary manifests for Calico installation.

```bash
curl -L https://github.com/projectcalico/calico/releases/download/v3.24.5/release-v3.24.5.tgz -o calico-v3.24.5.tgz
```

--------------------------------

### Heterogeneous Cluster Setup - KubeadmConfigTemplate (YAML)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/advanced.md

This YAML defines a KubeadmConfigTemplate for the bootstrap process of new Linux nodes joining a Windows workload cluster. It configures the kubelet to use an external cloud provider and sets the provider-id format for OCI instances, ensuring proper integration into the Kubernetes cluster.

```yaml
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha4
kind: KubeadmConfigTemplate
metadata:
  name: "${CLUSTER_NAME}-md-0"
spec:
  template:
    spec:
      joinConfiguration:
        nodeRegistration:
          kubeletExtraArgs:
            cloud-provider: external
            provider-id: oci://{{ ds["id"] }}
```

--------------------------------

### Download CCM Deployment Manifests

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/install-oci-ccm.md

These commands download the necessary YAML manifests for deploying the OCI Cloud Controller Manager (CCM) and its associated RBAC rules. The CCM_RELEASE_VERSION environment variable should be set to the desired version before execution.

```shell
export CCM_RELEASE_VERSION=<update-version-here>

curl -L "https://github.com/oracle/oci-cloud-controller-manager/releases/download/${CCM_RELEASE_VERSION}/oci-cloud-controller-manager.yaml" -o oci-cloud-controller-manager.yaml

curl -L "https://github.com/oracle/oci-cloud-controller-manager/releases/download/${CCM_RELEASE_VERSION}/oci-cloud-controller-manager-rbac.yaml" -o oci-cloud-controller-manager-rbac.yaml
```

--------------------------------

### Verify CCM Pod Status and Logs

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/install-oci-ccm.md

These commands check the status of the OCI Cloud Controller Manager (CCM) pods in the 'kube-system' namespace and display their logs. This is used to verify that the CCM is running correctly and to troubleshoot any potential issues.

```shell
kubectl -n kube-system get po | grep oci

kubectl -n kube-system logs oci-cloud-controller-manager-ds-k2txq
```

--------------------------------

### Configure OCI Instance Principal Authentication

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/install-cluster-api.md

Enables Instance Principals authentication for CAPOCI when the management cluster is hosted within OCI. This method is recommended and overrides user principal settings. It involves exporting a boolean flag and its base64 encoded version.

```bash
export USE_INSTANCE_PRINCIPAL="true"
export USE_INSTANCE_PRINCIPAL_B64="$(echo -n "$USE_INSTANCE_PRINCIPAL" | base64 | tr -d '\n')"
```

--------------------------------

### Export OCI Authentication Settings (User Principal)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/test/README.md

Exports necessary Oracle Cloud Infrastructure (OCI) authentication environment variables when using a user principal. It includes base64 encoded versions of sensitive credentials for secure transmission or storage. Ensure you replace placeholders with actual values and secure your private key file.

```bash
export OCI_TENANCY_ID=<tenancy-id>
export OCI_USER_ID=<use-id>
export OCI_CREDENTIALS_FINGERPRINT=<fingerprint>
export OCI_REGION=<region>
# if Passphrase is present
export OCI_CREDENTIALS_PASSPHRASE=<passphrase>
export OCI_TENANCY_ID_B64="$(echo -n "$OCI_TENANCY_ID" | base64 | tr -d '\n')"
export OCI_CREDENTIALS_FINGERPRINT_B64="$(echo -n "$OCI_CREDENTIALS_FINGERPRINT" | base64 | tr -d '\n')"
export OCI_USER_ID_B64="$(echo -n "$OCI_USER_ID" | base64 | tr -d '\n')"
export OCI_REGION_B64="$(echo -n "$OCI_REGION" | base64 | tr -d '\n')"
export OCI_CREDENTIALS_KEY_B64=$( cat <path-to-api-private-key-file> | base64 | tr -d '\n' )
# if Passphrase is present
export OCI_CREDENTIALS_PASSPHRASE_B64="$(echo -n "$OCI_CREDENTIALS_PASSPHRASE" | base64 | tr -d '\n')"
```

--------------------------------

### Linux Node Selector Example (YAML)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/advanced.md

This YAML snippet shows how to configure a Kubernetes Deployment to specifically schedule pods on Linux nodes. It utilizes the `nodeSelector` field with the label `kubernetes.io/os: linux`, ensuring that Linux-specific workloads are directed to the appropriate worker nodes.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx-linux
spec:
  selector:
    matchLabels:
      run: my-nginx-linux
  replicas: 2
  template:
    metadata:
      labels:
        run: my-nginx-linux
    spec:
      nodeSelector:
        kubernetes.io/os: linux
      containers:
      - args:
        - /bin/sh
        - -c
        - sleep 3600
        name: nginx
        image: nginx:latest
```

--------------------------------

### Heterogeneous Cluster Setup - MachineDeployment (YAML)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/advanced.md

This YAML defines a MachineDeployment for managing the lifecycle of Linux worker nodes in a heterogeneous cluster. It links the infrastructure (OCIMachineTemplate) and bootstrap (KubeadmConfigTemplate) configurations, specifying the cluster name, desired replica count, and Kubernetes version.

```yaml
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: "${CLUSTER_NAME}-md-0"
spec:
  clusterName: "${CLUSTER_NAME}"
  replicas: ${NODE_MACHINE_COUNT}
  selector:
    matchLabels:
  template:
    spec:
      clusterName: "${CLUSTER_NAME}"
      version: "${KUBERNETES_VERSION}"
      bootstrap:
        configRef:
          name: "${CLUSTER_NAME}-md-0"
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
      infrastructureRef:
        name: "${CLUSTER_NAME}-md-0"
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: OCIMachineTemplate
```

--------------------------------

### Heterogeneous Cluster Setup - OCIMachineTemplate (YAML)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/advanced.md

This YAML defines an OCIMachineTemplate for provisioning new Linux machines to be added to an existing Windows workload cluster. It specifies image, compartment, machine shape, OCPUs, SSH key, and encrypted transit for the persistent volume. This is a crucial part of setting up a heterogeneous cluster.

```yaml
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: OCIMachineTemplate
metadata:
  name: "${CLUSTER_NAME}-md-0"
spec:
  template:
    spec:
      imageId: "${OCI_IMAGE_ID}"
      compartmentId: "${OCI_COMPARTMENT_ID}"
      shape: "${OCI_NODE_MACHINE_TYPE=VM.Standard.E4.Flex}"
      shapeConfig:
        ocpus: "${OCI_NODE_MACHINE_TYPE_OCPUS=1}"
      metadata:
        ssh_authorized_keys: "${OCI_SSH_KEY}"
      isPvEncryptionInTransitEnabled: ${OCI_NODE_PV_TRANSIT_ENCRYPTION=true}
```

--------------------------------

### Controller Manager Initialization and Execution (Go)

Source: https://context7.com/oracle/cluster-api-provider-oci/llms.txt

This Go program initializes and runs the CAPOCI controller manager. It sets up various controllers (OCICluster, OCIMachine), configures OCI client authentication, and integrates with the Kubernetes controller-runtime manager. Dependencies include the controller-runtime library and CAPOCI's internal packages. It takes OCI authentication details from the environment and cluster configuration.

```go
package main

import (
    "context"
    "flag"
    "os"
    "time"

    infrastructurev1beta2 "github.com/oracle/cluster-api-provider-oci/api/v1beta2"
    "github.com/oracle/cluster-api-provider-oci/cloud/config"
    "github.com/oracle/cluster-api-provider-oci/cloud/scope"
    "github.com/oracle/cluster-api-provider-oci/controllers"
    ctrl "sigs.k8s.io/controller-runtime"
)

func main() {
    var (
        metricsAddr               = ":8080"
        enableLeaderElection      = false
        ociClusterConcurrency     = 5
        ociMachineConcurrency     = 10
        ociMachinePoolConcurrency = 5
        initOciClientsOnStartup   = true
    )

    // Setup manager
    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
        Scheme:             scheme,
        MetricsBindAddress: metricsAddr,
        LeaderElection:     enableLeaderElection,
        LeaderElectionID:   "controller-leader-elect-capoci",
    })
    if err != nil {
        setupLog.Error(err, "unable to start manager")
        os.Exit(1)
    }

    ctx := ctrl.SetupSignalHandler()

    // Initialize OCI client provider
    authConfig, err := config.FromDir(os.Getenv("AUTH_CONFIG_DIR"))
    if err != nil {
        setupLog.Error(err, "invalid auth config")
        os.Exit(1)
    }

    ociAuthConfigProvider, err := config.NewConfigurationProvider(authConfig)
    if err != nil {
        setupLog.Error(err, "authentication provider could not be initialised")
        os.Exit(1)
    }

    region, err := ociAuthConfigProvider.Region()
    if err != nil {
        setupLog.Error(err, "unable to get region")
        os.Exit(1)
    }

    clientProvider, err := scope.NewClientProvider(scope.ClientProviderParams{
        OciAuthConfigProvider: ociAuthConfigProvider,
    })
    if err != nil {
        setupLog.Error(err, "unable to create OCI ClientProvider")
        os.Exit(1)
    }

    // Setup cluster controller
    if err = (&controllers.OCIClusterReconciler{
        Client:         mgr.GetClient(),
        Scheme:         mgr.GetScheme(),
        Region:         region,
        ClientProvider: clientProvider,
        Recorder:       mgr.GetEventRecorderFor("ocicluster-controller"),
    }).SetupWithManager(ctx, mgr, controller.Options{
        MaxConcurrentReconciles: ociClusterConcurrency,
    }); err != nil {
        setupLog.Error(err, "unable to create controller", "controller", "OCICluster")
        os.Exit(1)
    }

    // Setup machine controller
    if err = (&controllers.OCIMachineReconciler{
        Client:         mgr.GetClient(),
        Scheme:         mgr.GetScheme(),
        ClientProvider: clientProvider,
        Region:         region,
        Recorder:       mgr.GetEventRecorderFor("ocimachine-controller"),
    }).SetupWithManager(ctx, mgr, controller.Options{
        MaxConcurrentReconciles: ociMachineConcurrency,
    }); err != nil {
        setupLog.Error(err, "unable to create controller", "controller", "OCIMachine")
        os.Exit(1)
    }

    // Setup webhooks
    if err = (&infrastructurev1beta2.OCICluster{}).SetupWebhookWithManager(mgr); err != nil {
        setupLog.Error(err, "unable to create webhook", "webhook", "OCICluster")
        os.Exit(1)
    }

    setupLog.Info("starting manager")
    if err := mgr.Start(ctx); err != nil {
        setupLog.Error(err, "problem running manager")
        os.Exit(1)
    }
}

```

--------------------------------

### OCICluster Spec with Security Lists

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/networking/custom-networking.md

This example demonstrates how to configure the OCICluster resource to use Security Lists for network security, specifying ingress and egress rules for different subnets. It requires a Kubernetes environment with the Cluster API Provider OCI installed.

```yaml
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: OCICluster
metadata:
  name: "${CLUSTER_NAME}"
spec:
  compartmentId: "${OCI_COMPARTMENT_ID}"
  networkSpec:
    vcn:
      name: ${CLUSTER_NAME}
      subnets:
        - name: ep-subnet
          role: control-plane-endpoint
          type: public
          securityList:
            name: ep-seclist
            egressRules:
              - destination: "10.0.0.0/29"
                protocol: "6"
                destinationType: "CIDR_BLOCK"
                description: "All traffic to control plane nodes"
                tcpOptions:
                  destinationPortRange:
                    max: 6443
                    min: 6443
            ingressRules:
              - source: "0.0.0.0/0"
                protocol: "6"
                sourceType: "CIDR_BLOCK"
                description: "External access to Kubernetes API endpoint"
                tcpOptions:
                  destinationPortRange:
                    max: 6443
                    min: 6443
              - source: "10.0.0.0/29"
                protocol: "6"
                sourceType: "CIDR_BLOCK"
                description: "Control plane worker nodes to API Server endpoint"
              - source: "0.0.0.0/0"
                protocol: "6"
                sourceType: "CIDR_BLOCK"
                description: "SSH access"
                tcpOptions:
                  destinationPortRange:
                    max: 22
                    min: 22
        - name: cp-mc-subnet
          role: control-plane
          type: private
          securityList:
            name: cp-mc-seclist
            egressRules:
              - destination: "0.0.0.0/0"
                protocol: "6"
                destinationType: "CIDR_BLOCK"
                description: "control plane machine access to internet"
            ingressRules:
              - source: "10.0.0.0/16"
                protocol: "all"
                sourceType: "CIDR_BLOCK"
                description: "Allow inter vcn communication"
              - source: "0.0.0.0/0"
                protocol: "6"
                sourceType: "CIDR_BLOCK"
                description: "SSH access"
                tcpOptions:
                  destinationPortRange:
                    max: 22
                    min: 22
        - name: worker-subnet
          role: worker
          type: private
          securityList:
            name: node-seclist
            egressRules:
              - destination: "0.0.0.0/0"
                protocol: "6"
                destinationType: "CIDR_BLOCK"
                description: "Worker Nodes access to Internet"
            ingressRules:
              - source: "10.0.0.0/16"
                protocol: "all"
                sourceType: "CIDR_BLOCK"
                description: "Allow inter vcn communication"
        - name: svc-lb-subnet
          role: service-lb
          type: public
          securityList:
            name: service-lb-seclist
            ingressRules:
              - source: "10.0.0.0/16"
                protocol: "all"
                sourceType: "CIDR_BLOCK"
                description: "Allow ingress from vcn subnets"

```

--------------------------------

### Deploy OCI Clusters with clusterctl (Bash)

Source: https://context7.com/oracle/cluster-api-provider-oci/llms.txt

This bash script demonstrates how to initialize the Cluster API Provider OCI (CAPOCI) and deploy Kubernetes clusters on OCI using the `clusterctl` command-line tool. It involves setting OCI environment variables for authentication, initializing the provider, generating a cluster manifest with specified parameters, applying the manifest, and retrieving the cluster's kubeconfig.

```bash
# Initialize the management cluster with CAPOCI
export OCI_TENANCY_ID="ocid1.tenancy.oc1..aaaaaa..."
export OCI_USER_ID="ocid1.user.oc1..aaaaaa..."
export OCI_CREDENTIALS_FINGERPRINT="aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99"
export OCI_REGION="us-phoenix-1"
export OCI_CREDENTIALS_KEY_PATH="$HOME/.oci/oci_api_key.pem"

clusterctl init --infrastructure oci

# Generate cluster manifest
export CLUSTER_NAME="my-cluster"
export NAMESPACE="default"
export KUBERNETES_VERSION="v1.28.2"
export OCI_COMPARTMENT_ID="ocid1.compartment.oc1..aaaaaa..."
export OCI_IMAGE_ID="ocid1.image.oc1.phx.aaaaaa..."
export OCI_SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB..."
export CONTROL_PLANE_MACHINE_COUNT=3
export WORKER_MACHINE_COUNT=2

clusterctl generate cluster ${CLUSTER_NAME} \
  --infrastructure oci \
  --kubernetes-version ${KUBERNETES_VERSION} \
  --control-plane-machine-count ${CONTROL_PLANE_MACHINE_COUNT} \
  --worker-machine-count ${WORKER_MACHINE_COUNT} \
  > cluster.yaml

# Apply the cluster manifest
kubectl apply -f cluster.yaml

# Get cluster kubeconfig
clusterctl get kubeconfig ${CLUSTER_NAME} > ${CLUSTER_NAME}.kubeconfig

# Check cluster status
kubectl get ocicluster,ocimachine,cluster,machine

```

--------------------------------

### Disable OCI Client Initialization on Startup (Shell)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/advanced.md

This snippet shows how to disable the automatic initialization of OCI clients when CAPOCI starts. This is useful when OCI principals are set at the cluster level, allowing CAPOCI to be installed without direct OCI user credentials. Ensure Cluster Identity is configured if this option is used to avoid OCICluster errors.

```shell
export INIT_OCI_CLIENTS_ON_STARTUP=false
```

--------------------------------

### Launch OCI Compute Instance using Go SDK

Source: https://context7.com/oracle/cluster-api-provider-oci/llms.txt

This Go code snippet demonstrates how to launch a compute instance in OCI using the OCI Go SDK. It configures instance details including availability domain, compartment, shape, image, and network interface (VNIC) with public IP assignment and security group IDs. The function returns an error if the instance launch fails and updates the machine scope with the new instance ID.

```go
package scope

import (
    "context"
    "fmt"

    "github.com/oracle/oci-go-sdk/v65/common"
    "github.com/oracle/oci-go-sdk/v65/core"
    "github.com/oracle/oci-go-sdk/v65/loadbalancer"
    "github.com/oracle/oci-go-sdk/v65/networkloadbalancer"
)

type ClusterScope struct {
    VCNClient                     core.VirtualNetworkClient
    LoadBalancerClient            loadbalancer.LoadBalancerClient
    NetworkLoadBalancerClient     networkloadbalancer.NetworkLoadBalancerClient
    ComputeClient                 core.ComputeClient
    ComputeManagementClient       core.ComputeManagementClient
}

// LaunchInstance creates a compute instance
func (s *ClusterScope) LaunchInstance(ctx context.Context, machineScope *MachineScope) error {
    instanceDetails := core.LaunchInstanceDetails{
        AvailabilityDomain: common.String(machineScope.getAvailabilityDomain()),
        CompartmentId:      common.String(machineScope.OCIMachine.Spec.CompartmentId),
        DisplayName:        common.String(machineScope.Name()),
        Shape:              common.String(machineScope.OCIMachine.Spec.Shape),
        ImageId:            common.String(machineScope.OCIMachine.Spec.ImageId),
        CreateVnicDetails: &core.CreateVnicDetails{
            SubnetId:       machineScope.getSubnetId(),
            AssignPublicIp: common.Bool(machineScope.OCIMachine.Spec.NetworkDetails.AssignPublicIp),
            NsgIds:         machineScope.getNSGIds(),
            FreeformTags:   machineScope.getFreeformTags(),
        },
        Metadata: machineScope.OCIMachine.Spec.Metadata,
        ShapeConfig: &core.LaunchInstanceShapeConfigDetails{
            Ocpus:       common.Float32(parseOcpus(machineScope.OCIMachine.Spec.ShapeConfig.Ocpus)),
            MemoryInGBs: common.Float32(parseMemory(machineScope.OCIMachine.Spec.ShapeConfig.MemoryInGBs)),
        },
        FreeformTags: machineScope.getFreeformTags(),
        DefinedTags:  machineScope.getDefinedTags(),
    }

    request := core.LaunchInstanceRequest{
        LaunchInstanceDetails: instanceDetails,
    }

    response, err := s.ComputeClient.LaunchInstance(ctx, request)
    if err != nil {
        return fmt.Errorf("failed to launch instance: %w", err)
    }

    machineScope.OCIMachine.Spec.InstanceId = response.Id
    return nil
}
```

--------------------------------

### Create Cluster on Virtual Instances (Ubuntu Image)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-workload-cluster.md

This command creates a workload cluster on OCI virtual instances using an Ubuntu custom image. It requires specifying the OCI compartment ID, image ID, SSH key, and optionally Kubernetes version and node counts. The output is piped to kubectl for applying the cluster configuration.

```bash
OCI_COMPARTMENT_ID=<compartment-id> \
OCI_IMAGE_ID=<ubuntu-custom-image-id> \
OCI_SSH_KEY=<ssh-key>  \
CONTROL_PLANE_MACHINE_COUNT=1 \
KUBERNETES_VERSION=v1.20.10 \
NAMESPACE=default \
NODE_MACHINE_COUNT=1 \
clusterctl generate cluster <cluster-name>\
--from cluster-template.yaml | kubectl apply -f -
```

--------------------------------

### OCICluster: NSG as Destination in Security Rule

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/networking/custom-networking.md

This example illustrates how to specify a Network Security Group (NSG) as the destination in an egress security rule within an OCICluster configuration. It's crucial that all required NSGs are defined within the template, as CAPOCI will not automatically retrieve them from the VCN. The example defines an 'ep-nsg' for control-plane endpoints and a 'cp-mc-nsg' for control-plane machines, with specific egress rules.

```yaml
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: OCICluster
metadata:
  name: "${CLUSTER_NAME}"
spec:
  compartmentId: "${OCI_COMPARTMENT_ID}"
  networkSpec:
    vcn:
      name: ${CLUSTER_NAME}
      cidr: "172.16.0.0/16"
      networkSecurityGroup:
        list:
          - name: ep-nsg
            role: control-plane-endpoint
            egressRules:
              - egressRule:
                  isStateless: false
                  destination: "cp-mc-nsg"
                  protocol: "6"
                  destinationType: "NETWORK_SECURITY_GROUP"
                  description: "All traffic to control plane nodes"
                  tcpOptions:
                    destinationPortRange:
                      max: 6443
                      min: 6443
          - name: cp-mc-nsg
            role: control-plane
            egressRules:
              - egressRule:
                  isStateless: false
                  destination: "0.0.0.0/0"
                  protocol: "6"
                  destinationType: "CIDR_BLOCK"
                  description: "control plane machine access to internet"

```

--------------------------------

### Create Cluster on Bare Metal Instances (Ubuntu Image)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-workload-cluster.md

This command creates a workload cluster on OCI bare metal instances using an Ubuntu custom image. It allows specifying bare metal shapes for control plane and worker nodes, and disables data transit encryption. Environment variables for OCI configuration, machine types, and node counts are used.

```bash
OCI_COMPARTMENT_ID=<compartment-id> \
OCI_IMAGE_ID=<ubuntu-custom-image-id> \
OCI_SSH_KEY=<ssh-key>  \
OCI_CONTROL_PLANE_MACHINE_TYPE=BM.Standard2.52 \
OCI_CONTROL_PLANE_MACHINE_TYPE_OCPUS=52 \
OCI_CONTROL_PLANE_PV_TRANSIT_ENCRYPTION=false \
OCI_NODE_MACHINE_TYPE=BM.Standard2.52 \
OCI_NODE_MACHINE_TYPE_OCPUS=52 \
OCI_NODE_PV_TRANSIT_ENCRYPTION=false \
CONTROL_PLANE_MACHINE_COUNT=1 \
KUBERNETES_VERSION=v1.20.10 \
NAMESPACE=default \
WORKER_MACHINE_COUNT=1 \
clusterctl generate cluster <cluster-name>\
--from cluster-template.yaml| kubectl apply -f -
```

--------------------------------

### Apply Heterogeneous Cluster Template (Bash)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/advanced.md

This bash script demonstrates how to apply the heterogeneous cluster template using `clusterctl`. It sets various environment variables required for provisioning Linux nodes, such as image OCID, compartment ID, machine type, and SSH key, before piping the output to `kubectl apply`.

```bash
OCI_IMAGE_ID=<your new linux image OCID> \
OCI_NODE_IMAGE_ID=<your new linux image OCID> \
OCI_COMPARTMENT_ID=<your compartment> \
NODE_MACHINE_COUNT=2 \
OCI_NODE_MACHINE_TYPE=<shape> \
OCI_NODE_MACHINE_TYPE_OCPUS=4 \
OCI_SSH_KEY="<your public ssh key>" \
clusterctl generate cluster <cluster-name> --kubernetes-version <kubernetes-version> \
--target-namespace default \
--from cluster-template-windows-calico-heterogeneous.yaml | kubectl apply -f -
```

--------------------------------

### Get OKE Cluster OCID from Management Cluster

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/managed/managedcluster.md

Retrieves the Oracle Cloud Infrastructure (OCI) Managed Control Plane OCID for a workload cluster. This is the first step in accessing the cluster's kubeconfig.

```bash
kubectl get ocimanagedcontrolplane <workload-cluster-name> -n <workload-cluster-namespace> -o jsonpath='{.spec.id}'
```

--------------------------------

### Create Public OKE Cluster using Cluster API

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/managed/managedcluster.md

Creates a new public OKE cluster using clusterctl with default configurations. Requires OCI credentials, image ID, SSH key, Kubernetes version, and namespace. The node pool utilizes VCN native pod networking.

```bash
OCI_COMPARTMENT_ID=<compartment-id> \
OCI_MANAGED_NODE_IMAGE_ID=<ubuntu-custom-image-id> \
OCI_SSH_KEY=<ssh-key>  \
KUBERNETES_VERSION=v1.24.1 \
NAMESPACE=default \
clusterctl generate cluster <cluster-name> \
--from cluster-template-managed.yaml | kubectl apply -f -
```

--------------------------------

### Initialize CAPOCI Provider

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-workload-cluster.md

Initializes the Cluster API Provider OCI (CAPOCI) with the specified infrastructure. This command assumes the OCI_REGION environment variable is already configured for authentication.

```bash
export OCI_REGION=us-phoenix-1

clusterctl init --infrastructure oci
```

--------------------------------

### Create Cluster on Virtual Instances (Oracle Linux Image)

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-workload-cluster.md

This command creates a workload cluster on OCI virtual instances using an Oracle Linux custom image. It requires specifying the OCI compartment ID, image ID, SSH key, and optionally Kubernetes version and node counts. The output is piped to kubectl for applying the cluster configuration.

```bash
OCI_COMPARTMENT_ID=<compartment-id> \
OCI_IMAGE_ID=<oracle-linux-custom-image-id> \
OCI_SSH_KEY=<ssh-key>  \
CONTROL_PLANE_MACHINE_COUNT=1 \
KUBERNETES_VERSION=v1.20.10 \
NAMESPACE=default \
WORKER_MACHINE_COUNT=1 \
clusterctl generate cluster <cluster-name>\
--from cluster-template-oraclelinux.yaml | kubectl apply -f -
```

--------------------------------

### Check GPU Operator Pod Status

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-gpu-workload-cluster.md

Retrieves the status of pods within the 'gpu-operator' namespace. This command is used to monitor the deployment progress of the GPU Operator after installation.

```bash
kubectl --<cluster-name>.kubeconf get pods -n gpu-operator
```

--------------------------------

### OCI Cluster API Example YAML Configuration

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-gpu-workload-cluster.md

This YAML defines a Kubernetes cluster using Cluster API with the OCI provider. It includes configurations for the main cluster object, OCI specific infrastructure, the Kubeadm control plane, OCIMachineTemplates for control plane and worker nodes, and a KubeadmConfigTemplate for bootstrapping. It also demonstrates setting up a MachineDeployment targeting a specific availability domain (failure domain).

```yaml
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  labels:
    cluster.x-k8s.io/cluster-name: "${CLUSTER_NAME}"
  name: "${CLUSTER_NAME}"
  namespace: "${NAMESPACE}"
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
        - ${POD_CIDR:="192.168.0.0/16"}
    serviceDomain: ${SERVICE_DOMAIN:="cluster.local"}
    services:
      cidrBlocks:
        - ${SERVICE_CIDR:="10.128.0.0/12"}
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
    kind: OCICluster
    name: "${CLUSTER_NAME}"
    namespace: "${NAMESPACE}"
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: "${CLUSTER_NAME}-control-plane"
    namespace: "${NAMESPACE}"
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: OCICluster
metadata:
  labels:
    cluster.x-k8s.io/cluster-name: "${CLUSTER_NAME}"
  name: "${CLUSTER_NAME}"
spec:
  compartmentId: "${OCI_COMPARTMENT_ID}"
---
kind: KubeadmControlPlane
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
metadata:
  name: "${CLUSTER_NAME}-control-plane"
  namespace: "${NAMESPACE}"
spec:
  version: "${KUBERNETES_VERSION}"
  replicas: ${CONTROL_PLANE_MACHINE_COUNT}
  machineTemplate:
    infrastructureRef:
      kind: OCIMachineTemplate
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
      name: "${CLUSTER_NAME}-control-plane"
      namespace: "${NAMESPACE}"
  kubeadmConfigSpec:
    clusterConfiguration:
      kubernetesVersion: ${KUBERNETES_VERSION}
      apiServer:
        certSANs: [localhost, 127.0.0.1]
      dns: {}
      etcd: {}
      networking: {}
      scheduler: {}
    initConfiguration:
      nodeRegistration:
        criSocket: /var/run/containerd/containerd.sock
        kubeletExtraArgs:
          cloud-provider: external
          provider-id: oci://{{ ds["id"] }}
    joinConfiguration:
      discovery: {}
      nodeRegistration:
        criSocket: /var/run/containerd/containerd.sock
        kubeletExtraArgs:
          cloud-provider: external
          provider-id: oci://{{ ds["id"] }}
---
kind: OCIMachineTemplate
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
metadata:
  name: "${CLUSTER_NAME}-control-plane"
spec:
  template:
    spec:
      imageId: "${OCI_IMAGE_ID}"
      compartmentId: "${OCI_COMPARTMENT_ID}"
      shape: "${OCI_CONTROL_PLANE_MACHINE_TYPE=VM.Standard.E4.Flex}"
      shapeConfig:
        ocpus: "${OCI_CONTROL_PLANE_MACHINE_TYPE_OCPUS=1}"
      metadata:
        ssh_authorized_keys: "${OCI_SSH_KEY}"
      isPvEncryptionInTransitEnabled: ${OCI_CONTROL_PLANE_PV_TRANSIT_ENCRYPTION=true}
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: OCIMachineTemplate
metadata:
  name: "${CLUSTER_NAME}-md"
spec:
  template:
    spec:
      imageId: "${OCI_IMAGE_ID}"
      compartmentId: "${OCI_COMPARTMENT_ID}"
      shape: "${OCI_NODE_MACHINE_TYPE=VM.Standard.E4.Flex}"
      shapeConfig:
        ocpus: "${OCI_NODE_MACHINE_TYPE_OCPUS=1}"
      metadata:
        ssh_authorized_keys: "${OCI_SSH_KEY}"
      isPvEncryptionInTransitEnabled: ${OCI_NODE_PV_TRANSIT_ENCRYPTION=true}
---
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha4
kind: KubeadmConfigTemplate
metadata:
  name: "${CLUSTER_NAME}-md"
spec:
  template:
    spec:
      joinConfiguration:
        nodeRegistration:
          kubeletExtraArgs:
            cloud-provider: external
            provider-id: oci://{{ ds["id"] }}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: "${CLUSTER_NAME}-fd-2-md-0"
spec:
  clusterName: "${CLUSTER_NAME}"
  replicas: ${NODE_MACHINE_COUNT}
  selector:
    matchLabels:
  template:
    spec:
      clusterName: "${CLUSTER_NAME}"
      version: "${KUBERNETES_VERSION}"
      bootstrap:
        configRef:
          name: "${CLUSTER_NAME}-md"
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
      infrastructureRef:
        name: "${CLUSTER_NAME}-md"
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
        kind: OCIMachineTemplate
      # Cluster-API calls them Failure Domains while OCI calls them Availability Domains
      # In the example this would be targeting US-ASHBURN-AD-2
      failureDomain: "2"

```

--------------------------------

### Get Kube-DNS Service IP

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-windows-workload-cluster.md

Retrieves the cluster IP address for the kube-dns service in the kube-system namespace using kubectl. This IP is used for configuring DNS settings within Calico manifests.

```bash
kubectl  get svc kube-dns -n kube-system  -o jsonpath='{.spec.clusterIP}'
```

--------------------------------

### OCIClusterIdentity using Workload Identity

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/multi-tenancy.md

This YAML defines an OCIClusterIdentity resource configured for Workload Identity. It specifies the identity type and allows all namespaces. This setup requires specific OCI policies to be configured for the associated Kubernetes Service Account.

```yaml
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: OCIClusterIdentity
metadata:
  name: cluster-identity
  namespace: default
spec:
  type: Workload
  allowedNamespaces: {}
```

--------------------------------

### Extend Root Partition using Cloud Init Script

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/managed/boot-volume-expansion.md

This cloud-init script extends the root partition of an instance to utilize the increased boot volume size. It fetches and executes an OKE provisioning script and then uses `oci-growfs` to resize the filesystem. A timestamp is created to indicate completion.

```bash
#!/bin/bash

# DO NOT MODIFY
curl --fail -H "Authorization: Bearer Oracle" -L0 http://169.254.169.254/opc/v2/instance/metadata/oke_init_script | base64 --decode >/var/run/oke-init.sh

## run oke provisioning script
bash -x /var/run/oke-init.sh

### adjust block volume size
/usr/libexec/oci-growfs -y

touch /var/log/oke.done
```

--------------------------------

### Get Kubernetes Host IP with kubectl

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-windows-workload-cluster.md

Retrieves the Kubernetes service host IP address from the OCI cluster specification using kubectl. This IP is essential for configuring network components like Calico.

```bash
kubectl get OCICluster <cluster-name> -o jsonpath='{.spec.controlPlaneEndpoint.host}'
```

--------------------------------

### Create CCM Kubernetes Secret

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/install-oci-ccm.md

This command creates a Kubernetes secret named 'oci-cloud-controller-manager' in the 'kube-system' namespace. This secret stores the customized cloud provider configuration file, which the CCM uses for authentication and configuration.

```shell
kubectl  create secret generic oci-cloud-controller-manager \
  -n kube-system \
  --from-file=cloud-provider.yaml=cloud-provider-example.yaml
```

--------------------------------

### Apply Windows Kube-Proxy Manifest

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-windows-workload-cluster.md

Applies the kube-proxy configuration for Windows nodes. This manifest ensures that the kube-proxy component functions correctly with the specified Kubernetes and Windows versions.

```bash
kubectl apply -f windows-kube-proxy.yaml
```

--------------------------------

### OCI Policies for Workload Identity

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/multi-tenancy.md

These are example OCI IAM policies required for Workload Identity to manage OKE or self-managed clusters. They grant necessary permissions to the CAPOCI controller manager's Service Account within a specified compartment.

```hcl
Allow any-user to manage virtual-network-family in compartment <compartment> where all { request.principal.type = 'workload', request.principal.namespace = 'cluster-api-provider-oci-system', request.principal.service_account = 'capoci-controller-manager'}
Allow any-user to manage cluster-family in compartment <compartment> where all { request.principal.type = 'workload', request.principal.namespace = 'cluster-api-provider-oci-system', request.principal.service_account = 'capoci-controller-manager'}
Allow any-user to manage volume-family in compartment <compartment> where all { request.principal.type = 'workload', request.principal.namespace = 'cluster-api-provider-oci-system', request.principal.service_account = 'capoci-controller-manager'}
Allow any-user to manage instance-family in compartment <compartment> where all { request.principal.type = 'workload', request.principal.namespace = 'cluster-api-provider-oci-system', request.principal.service_account = 'capoci-controller-manager'}
Allow any-user to inspect compartments in compartment <compartment> where all { request.principal.type = 'workload', request.principal.namespace = 'cluster-api-provider-oci-system', request.principal.service_account = 'capoci-controller-manager'}
```

--------------------------------

### View CUDA Vector Add Test Pod Logs

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-gpu-workload-cluster.md

Fetches the logs from the 'cuda-vector-add' pod. This is used to verify the results of the CUDA vector addition test, confirming that the GPU is functioning correctly.

```bash
kubectl --kubeconfig=<cluster-name>.kubeconf logs cuda-vector-add -n default
```

--------------------------------

### Access Workload Cluster Kubeconfig using clusterctl

Source: https://github.com/oracle/cluster-api-provider-oci/blob/main/docs/src/gs/create-windows-workload-cluster.md

These commands retrieve the kubeconfig for a specific workload cluster. The first lists all clusters, and the second downloads the kubeconfig file for the named cluster. This allows you to interact with your created cluster using `kubectl`.

```bash
kubectl get clusters -A
```

```bash
clusterctl get kubeconfig <cluster-name> -n default > <cluster-name>.kubeconfig
```