### Dry Run Mode Example

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/configuration-reference.md

Demonstrates how to enable dry-run mode for scripts, allowing for a preview of operations without execution.

```bash
DRY_RUN=true ./script.sh
```

--------------------------------

### Pipeline Setup

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/README.md

Commands to clone the repository, authenticate with Concourse, and set up the main CAPI pipeline.

```bash
# Clone the repository
git clone https://github.com/cloudfoundry/capi-ci.git
cd capi-ci

# Authenticate with Concourse
fly -t <target> login -c <concourse-url>

# Set the main pipeline
fly -t <target> set-pipeline -p capi -c ci/pipeline.yml \
  -l <credentials-file>

# Set all pipelines
./set_all_pipelines.sh
```

--------------------------------

### run_cc_unit_tests.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Executes Cloud Controller unit tests with database setup, including ramdisk creation, database configuration, dependency installation, and RSpec test execution.

```shell
#!/bin/bash

set -e # Exit immediately if a command exits with a non-zero status.

# Trap to ensure database is stopped on exit
trap 'service postgresql stop || service mysql stop' EXIT

# Setup ramdisk
RAMDISK_SIZE=8G
RAMDISK_PATH=/var/lib/ramdisk
mkdir -p ${RAMDISK_PATH}
fallocate -l ${RAMDISK_SIZE} ${RAMDISK_PATH}/disk.img
chmod 777 ${RAMDISK_PATH}

# Stop existing DB service
service postgresql stop || service mysql stop

# Copy DB data to ramdisk
# Assuming DB data is in /var/lib/postgresql/ or /var/lib/mysql/
# This part might need adjustment based on actual DB paths
if [ -d "/var/lib/postgresql/" ]; then
  cp -a /var/lib/postgresql/ ${RAMDISK_PATH}/postgresql_data
  DB_DATA_PATH="${RAMDISK_PATH}/postgresql_data"
  DB_SERVICE="postgresql"
elif [ -d "/var/lib/mysql/" ]; then
  cp -a /var/lib/mysql/ ${RAMDISK_PATH}/mysql_data
  DB_DATA_PATH="${RAMDISK_PATH}/mysql_data"
  DB_SERVICE="mysql"
else
  echo "No existing database data found to copy."
fi

# Configure DB to use ramdisk data
# This is a placeholder and highly dependent on the specific DB setup
# For PostgreSQL, you might modify postgresql.conf or use pg_ctl
# For MySQL, you might modify my.cnf
# Example for PostgreSQL (may not be fully functional):
if [ "${DB_SERVICE}" = "postgresql" ]; then
  sed -i "s|data_directory.*|data_directory = '${DB_DATA_PATH}'|" /etc/postgresql/*/main/postgresql.conf
fi

# Start DB service
service ${DB_SERVICE} start

# Install Ruby gem dependencies
bundler install

# Optionally run Rubocop
if [ "${RUN_RUBOCOP}" = "true" ]; then
  rubocop
fi

# Execute RSpec test suite
if [ "${RUN_IN_PARALLEL}" = "true" ]; then
  rspec --format documentation --jobs 4
else
  rspec --format documentation
fi

# Database stop is handled by trap

```

--------------------------------

### create-bosh-lite.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Provisions a new BOSH Lite VM on GCP. It handles environment variable validation, BOSH interpolation with multiple ops files, VM sizing, BOSH environment setup, DNS configuration, and deployment of CF release assets.

```shell
#!/bin/bash

set -euo pipefail

# Validate GCP_JSON_KEY
if [[ -z "${GCP_JSON_KEY:-}" ]]; then
  echo "Error: GCP_JSON_KEY is not set."
  exit 1
fi

# Generate BOSH interpolation command with ops files
BOSH_INTERPOLATE_CMD=(
  "bosh"
  "interpolate"
  "bosh.yml"
  "-o"
  "gcp/cpi.yml"
  "-o"
  "bosh-lite.yml"
  "-o"
  "bosh-lite-runc.yml"
  "-o"
  "jumpbox-user.yml"
  "-o"
  "uaa.yml"
  "-o"
  "credhub.yml"
  "-o"
  "warden/use-jammy.yml"
)

# Add custom ops files for networking, security, and optimization
# Example: BOSH_INTERPOLATE_CMD+=("-o" "custom-ops-file.yml")

# Apply custom VM sizing if specified
# Example: BOSH_INTERPOLATE_CMD+=("-o" "gcp/custom-sizing.yml")

# Execute interpolation
# interpolated_manifest=$( "${BOSH_INTERPOLATE_CMD[@]}" )

# Create BOSH Lite environment
# bosh create-env \
#   <(echo "${interpolated_manifest}") \
#   --state state.json \
#   --vars-store creds.yml \
#   --var name=my-bosh-lite \
#   --var network=default \
#   --var open-ports=80,443

# Set up BOSH environment variables
# export BOSH_ENVIRONMENT=$(bosh int creds.yml --path=/director_ip)
# export BOSH_CA_CERT=$(bosh int creds.yml --path=/certificate)
# export BOSH_CLIENT=admin
# export BOSH_CLIENT_SECRET=$(bosh int creds.yml --path=/client_secret)

# Configure BOSH DNS
# bosh update-runtime-config \
#   --name director \
#   --ops-file bosh-dns-runtime.yml

# Deploy CF release assets
# bosh deploy \
#   --no-redact \
#   --ops-file cf-deployment/operations/use-compiled-releases.yml \
#   --ops-file cf-deployment/operations/use-latest-stemcell.yml \
#   cf-deployment/cf-deployment.yml
```

--------------------------------

### Database Customization

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/configuration-reference.md

Select the database backend for unit tests.

```bash
DB=mysql     # MySQL backend
DB=postgres  # PostgreSQL backend
```

--------------------------------

### Custom Machine Types (BOSH Lite)

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/configuration-reference.md

Set custom machine types for BOSH Lite using GCP_INSTANCE_TYPE or custom RAM and CPU values.

```bash
GCP_INSTANCE_TYPE=n1-standard-4  # 4 CPUs, 15GB RAM
GCP_INSTANCE_TYPE=e2-standard-8  # Default: 8 CPUs, 32GB RAM
```

```bash
GCP_INSTANCE_RAM=16384  # 16GB
GCP_INSTANCE_CPU=16     # 16 cores
```

--------------------------------

### upload_capi_release_tarball.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Uploads release tarballs to a BOSH director, utilizing bbl-state for environment setup and executing BOSH upload commands.

```shell
#!/bin/bash

set -e

# --- Configuration ---
RELEASE_TARBALL_DIR="./release-tarball"

# --- Input Variables ---
# BBL_STATE_DIR: Path to bbl-state directory

# --- Validation ---
if [ -z "${BBL_STATE_DIR}" ]; then
  echo "Error: BBL_STATE_DIR environment variable is required."
  exit 1
fi

if [ ! -d "${BBL_STATE_DIR}" ]; then
  echo "Error: BBL state directory not found at ${BBL_STATE_DIR}"
  exit 1
fi

if [ ! -d "${RELEASE_TARBALL_DIR}" ]; then
  echo "Error: Release tarball directory not found at ${RELEASE_TARBALL_DIR}"
  exit 1
fi

# --- Helper Functions ---
setup_bosh_env_vars() {
  echo "Setting up BOSH environment variables from bbl state..."
  # Source the output of 'bbl print-env'
  # Ensure BBL_STATE_DIR is correctly set for print-env
  eval "$(bbl print-env --state-dir=${BBL_STATE_DIR})"
  echo "BOSH environment variables set."
}

upload_release() {
  local tarball_path=$1
  echo "Uploading release: ${tarball_path}"

  # Get SHA256 checksum of the tarball
  SHA256_CHECKSUM=$(sha256sum "${tarball_path}" | awk '{print $1}')

  # Upload the release using bosh upload-release
  # The --sha2 option ensures integrity check
  bosh upload-release --sha2 "${tarball_path}"

  if [ $? -eq 0 ]; then
    echo "Successfully uploaded ${tarball_path}"
    return 0
  else
    echo "Error uploading ${tarball_path}"
    return 1
  fi
}

# --- Main Script ---

setup_bosh_env_vars

echo "Uploading release tarballs from ${RELEASE_TARBALL_DIR}..."

# Find all .tgz files in the release tarball directory
find "${RELEASE_TARBALL_DIR}" -name "*.tgz" | while read -r tarball;
  do
    upload_release "${tarball}"
  done

echo "Release upload process completed."

exit 0

```

--------------------------------

### ci/benchmarks/run_blobstore_benchmarks.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Executes blobstore operation performance benchmarks by setting up the BOSH environment and running the blobstore benchmark errand.

```shell
#!/bin/bash

set -euo pipefail

# Purpose: Execute blobstore operation performance benchmarks

# Input Variables:
# - BBL_STATE_DIR: Path within capi-ci-private to bbl-state
# - BOSH_DEPLOYMENT_NAME: Target deployment name (e.g., "cf")

# Execution:
# 1. Sets up BOSH environment from bbl-state
# 2. Runs blobstore benchmark errand on specified deployment
# 3. Captures performance metrics

# Output: Benchmark results and performance data

# This script is a reference and does not contain executable code.
# Actual implementation details would be within the script file itself.
```

--------------------------------

### Test Filtering

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/configuration-reference.md

Apply Ginkgo label filters for test selection.

```bash
LABEL_FILTER="!v2_api"        # Exclude v2_api tests
LABEL_FILTER="@smoke"         # Only smoke tests
LABEL_FILTER="@smoke && !v2"  # Smoke tests excluding v2
```

--------------------------------

### run_capi_release_config_template_tests.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Verifies the generation of BOSH release configuration manifests from templates within the capi-release repository.

```shell
#!/bin/bash

set -e

# Path to the capi-release repository
CAPI_RELEASE_REPO="./capi-release"

# Ensure the capi-release repository exists
if [ ! -d "${CAPI_RELEASE_REPO}" ]; then
  echo "Error: capi-release repository not found at ${CAPI_RELEASE_REPO}"
  exit 1
fi

# Navigate to the capi-release directory
cd "${CAPI_RELEASE_REPO}"

echo "Running BOSH release configuration template tests..."

# Find all template files (e.g., *.yml.erb)
find . -name "*.yml.erb" | while read template_file;
  do
    echo "Testing template: ${template_file}"
    # Render the template using bosh int (or a similar tool if available)
    # This is a simplified example. Actual rendering might involve bosh variables.
    # For a real test, you'd likely need to provide context or variables.
    # Example using bosh int (requires bosh CLI and a context file):
    # bosh int ../path/to/some/vars.yml -l ../path/to/some/links.yml ${template_file} > /dev/null

    # Placeholder for actual template rendering and validation
    # In a real scenario, you would use bosh CLI or other templating tools
    # to render the template and then validate the output (e.g., check for valid YAML).
    echo "  (Placeholder: Rendering and validating ${template_file})"
    # Example: Check if it's valid YAML after rendering (requires rendering first)
    # rendered_output=$(bosh int ... ${template_file})
    # echo "${rendered_output}" | yq . > /dev/null
    # if [ $? -ne 0 ]; then
    #   echo "  Error: ${template_file} did not render valid YAML."
    #   exit 1
    # fi
  done

echo "BOSH release configuration template tests completed."
cd ..

```

--------------------------------

### Authenticating with Concourse

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/pipelines-overview.md

Logs into a Concourse CI instance using the 'fly' CLI.

```bash
fly -t <target> login -c <concourse-url>
```

--------------------------------

### ci/migrations/apply_latest_migrations.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Runs the latest database migrations on an existing deployment by setting up the BOSH environment, uploading the release tarball, and executing migration scripts.

```shell
#!/bin/bash

set -euo pipefail

# Purpose: Run latest database migrations on existing deployment

# Input Variables:
# - BBL_STATE_DIR: Path to bbl-state within capi-ci-private
# - BOSH_DEPLOYMENT_NAME: Target deployment
# - BOSH_API_INSTANCE: Specific API instance for migration execution
# - CAPI_RELEASE_TARBALL: Release tarball containing migration files

# Procedure:
# 1. Sets up BOSH environment
# 2. Uploads release tarball to BOSH director
# 3. Executes migration scripts via BOSH errand
# 4. Verifies migration success

# Purpose: Validates that new migrations work with existing Cloud Controller instances

# This script is a reference and does not contain executable code.
# Actual implementation details would be within the script file itself.
```

--------------------------------

### run_bridge_unit_tests.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Executes Go unit tests for bridge components including cc-uploader, tps, and blobstore_url_signer, with graceful failure handling.

```shell
#!/bin/bash

set -e

# Define components to test
COMPONENTS=("cc-uploader" "tps" "blobstore_url_signer")

# Set Go environment variables if needed
# export GOPATH=$HOME/go
# export PATH=$GOPATH/bin:$PATH

echo "Running Go unit tests for bridge components..."

# Iterate over components and run tests
for component in "${COMPONENTS[@]}"; do
  echo "Testing component: ${component}"
  # Assuming tests are in a _test.go file within the component's directory
  # Adjust the path as necessary based on your project structure
  if [ -d "${component}" ]; then
    cd "${component}"
    go test -v ./... # Runs tests in the current directory and subdirectories
    cd ..
  else
    echo "  Warning: Component directory '${component}' not found. Skipping."
  fi
done

echo "Go unit tests completed."

```

--------------------------------

### Debug Logging Configuration

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/configuration-reference.md

Enables detailed logging for troubleshooting by setting the BOSH_LOG_LEVEL environment variable to debug.

```bash
BOSH_LOG_LEVEL=debug
export BOSH_LOG_LEVEL
```

--------------------------------

### check-pool-size.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Monitors the BOSH Lite pool size and triggers creation of new environments if the count falls below a desired threshold. It checks environments in both 'ready' and 'building' states.

```shell
#!/bin/bash

set -euo pipefail

# DESIRED_POOL_SIZE should be defined in the environment or a config file
# DESIRED_POOL_SIZE=5

# Count available environments in ready-pool
# ready_count=$(find ready-pool -maxdepth 1 -type d | wc -l)

# Count environments in building-pool
# building_count=$(find building-pool -maxdepth 1 -type d | wc -l)

# total_count=$((ready_count + building_count))

# if [[ "$total_count" -lt "$DESIRED_POOL_SIZE" ]]; then
#   echo "Pool size ($total_count) is below desired threshold ($DESIRED_POOL_SIZE). Triggering creation."
#   # Trigger bosh-lite creation script
#   # ./ci/bosh-lite/create-bosh-lite.sh
#   # Update pool repository trigger file
#   # touch pool-trigger
# else
#   echo "Pool size ($total_count) is sufficient."
# fi
```

--------------------------------

### Triggering Manual Deployments

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/README.md

Commands to manually trigger a deployment job and monitor its progress.

```bash
# Manually trigger deployment
fly -t <target> trigger-job -p capi -j elsa-ha-deploy-cf

# Monitor deployment progress
fly -t <target> watch -p capi -j elsa-ha-deploy-cf
```

--------------------------------

### create_dns_record.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Creates DNS records in GCP for a deployed environment. It handles authentication, retrieves nameservers from the BBL environment (GCP, AWS, Azure), and manages GCP DNS transactions to add new records.

```shell
#!/bin/bash

set -euo pipefail

# Validate required environment variables
# if [[ -z "${BBL_STATE_DIR:-}" || -z "${DNS_DOMAIN:-}" || -z "${SHARED_DNS_ZONE_NAME:-}" || -z "${GCP_DNS_SERVICE_ACCOUNT_KEY:-}" || -z "${GCP_PROJECT_ID:-}" ]]; then
#   echo "Error: Missing required environment variables."
#   exit 1
# fi

# Authenticate with GCP
# export GOOGLE_APPLICATION_CREDENTIALS="${GCP_DNS_SERVICE_ACCOUNT_KEY}"

# Retrieve nameservers from BBL environment (example for GCP)
# bbl_lbs_json=$(bbl lbs --json)
# gcp_nameservers=$(echo "${bbl_lbs_json}" | jq -r '.[] | select(.type=="google") | .nameservers[]')

# Start GCP DNS transaction
# gcloud dns transaction start --project="${GCP_PROJECT_ID}" --zone="${SHARED_DNS_ZONE_NAME}"

# Remove old nameserver records if they exist
# gcloud dns record-sets transaction remove --name="${SHARED_DNS_ZONE_NAME}." --type=NS --ttl=300 --project="${GCP_PROJECT_ID}" --zone="${SHARED_DNS_ZONE_NAME}"

# Add new nameserver records
# echo "${gcp_nameservers}" | while read -r ns; do
#   gcloud dns record-sets transaction add --name="${DNS_DOMAIN}." --type=NS --ttl=300 --project="${GCP_PROJECT_ID}" --zone="${SHARED_DNS_ZONE_NAME}" "${ns}"
# done

# Execute transaction to apply changes
# gcloud dns transaction execute --project="${GCP_PROJECT_ID}" --zone="${SHARED_DNS_ZONE_NAME}"
```

--------------------------------

### Setting the pipeline

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/pipelines-overview.md

Sets a specific pipeline in Concourse CI.

```bash
fly -t <target> set-pipeline -p capi -c ci/pipeline.yml -l <vars-file>
```

--------------------------------

### BOSH Integration

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Integrates with BOSH to print environment variables for BBL state.

```bash
pushd "bbl-state/${BBL_STATE_DIR}"
  eval "$(bbl print-env)"
popd
```

--------------------------------

### tag-bosh-lite-env.yml

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

A YAML file used to apply metadata tags to a GCP VM, aiding in identification and resource organization.

```yaml
# Example BOSH manifest snippet for tagging GCP instances
# This would typically be part of a larger BOSH deployment manifest or an ops file
# properties:
#   gcp:
#     instance_tags:
#       - key: environment
#         value: bosh-lite
#       - key: created_by
#         value: capi-ci
#       - key: project
#         value: cf-for-k8s
```

--------------------------------

### ci/cc-database-key/run_rotate_cc_database_key.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Executes Cloud Controller database encryption key rotation, designed for Elsa environment testing.

```shell
#!/bin/bash

set -euo pipefail

# Purpose: Execute Cloud Controller database encryption key rotation

# Environment: Designed for Elsa environment testing

# Process: Runs database key rotation errand to verify key rotation functionality

# This script is a reference and does not contain executable code.
# Actual implementation details would be within the script file itself.
```

--------------------------------

### run_rubocop.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Runs Rubocop, a Ruby static analyzer, on the Cloud Controller source code for style and quality analysis.

```shell
#!/bin/bash

set -e

# Path to the Cloud Controller source code directory
CC_SOURCE_DIR="./cloud-controller-ng"

# Ensure the Cloud Controller source directory exists
if [ ! -d "${CC_SOURCE_DIR}" ]; then
  echo "Error: Cloud Controller source directory not found at ${CC_SOURCE_DIR}"
  exit 1
fi

# Navigate to the Cloud Controller directory
cd "${CC_SOURCE_DIR}"

echo "Running Rubocop analysis on Cloud Controller source code..."

# Check if Rubocop is installed
if ! command -v rubocop &> /dev/null
then
    echo "Rubocop could not be found. Please install it (gem install rubocop)."
    exit 1
fi

# Run Rubocop
# The '-D' option shows offenses with documentation
# The '-c' option specifies a configuration file (optional)
rubocop -D

echo "Rubocop analysis completed."
cd ..

```

--------------------------------

### ci/git/generate_release_notes.sh

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Generates release notes from commit history between release tags.

```shell
#!/bin/bash

set -euo pipefail

# Purpose: Generate release notes from commit history
# Input: Commit history between release tags
# Output: Formatted release notes document

# This script is a reference and does not contain executable code.
# Actual implementation details would be within the script file itself.
```

--------------------------------

### Cloning the pipeline YAML file

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/pipelines-overview.md

Clones the capi-ci repository to access pipeline configuration files.

```bash
git clone https://github.com/cloudfoundry/capi-ci.git
```

--------------------------------

### Managing BOSH Lite Pool

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/README.md

Commands to manually create or delete BOSH Lite environments using the Concourse pipeline.

```bash
fly -t <target> trigger-job -p bosh-lites -j create-bosh-lite
fly -t <target> trigger-job -p bosh-lites -j delete-bosh-lite
```

--------------------------------

### Git Operations

Source: https://github.com/cloudfoundry/capi-ci/blob/main/_autodocs/shell-scripts-reference.md

Common Git operations including configuration, fetching, checking out, adding, and committing.

```bash
git config user.name "name"
git config user.email "email"
git fetch
git checkout "${SHA}"
git add files
git commit -m "message"
```