### Initialize Example Module with Kitchen Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Initialize the working directory for a specific example module using the kitchen_do command. This prepares the environment for convergence and verification. ```bash kitchen_do create ``` -------------------------------- ### Apply Example Module with Kitchen Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Apply the example module to the test environment using the kitchen_do converge command. This provisions the resources defined in the module. ```bash kitchen_do converge ``` -------------------------------- ### Set Region for Cloud SQL Example Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-web-proxy.md Set the REGION environment variable for the Cloud SQL example. ```bash export REGION="us-central1" ``` -------------------------------- ### Verify Example Module with Kitchen Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Test the applied example module using the kitchen_do verify command. This runs InSpec tests to ensure the module behaves as expected. ```bash kitchen_do verify ``` -------------------------------- ### Destroy Example Module with Kitchen Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Clean up the resources created for an example module using the kitchen_do destroy command. This reverts the test environment to its initial state. ```bash kitchen_do destroy ``` -------------------------------- ### Set Region for BigQuery and Internal Web Server Examples Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-web-proxy.md Set the REGION environment variable for BigQuery and Internal Web Server examples. ```bash export REGION="us-west1" ``` -------------------------------- ### Start Testing Docker Container Interactively Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Start the Docker container for testing in interactive mode. This allows for manual execution of Kitchen commands to manage the test lifecycle. ```bash make docker_run ``` -------------------------------- ### Initialize Terraform and Get Serverless Project ID Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Initializes Terraform for the production environment and retrieves the serverless project ID. ```bash terraform -chdir="gcp-projects/business_unit_1/production" init export SERVERLESS_PROJECT_ID=$(terraform -chdir="gcp-projects/business_unit_1/production" output -raw serverless_project_id) echo "SERVERLESS_PROJECT_ID = ${SERVERLESS_PROJECT_ID}" ``` -------------------------------- ### Initialize Terraform and Get App Infrastructure Service Account Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Initializes Terraform for the shared business unit configuration and extracts the email address of the application infrastructure service account. ```bash terraform -chdir="gcp-projects/business_unit_1/shared" init export app_infra_sa=$(terraform -chdir="gcp-projects/business_unit_1/shared" output -json terraform_service_accounts | jq '."bu1-scf-app"' --raw-output) echo "APP_INFRA_SA_EMAIL = ${app_infra_sa}" ``` -------------------------------- ### Deploy Secure Cloud Function (2nd Gen) Core Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/modules/secure-cloud-function-core/README.md Example of how to use the secure_cloud_function_core module to deploy a 2nd Gen Cloud Function with various security and networking configurations. ```hcl module "secure_cloud_function_core" { source = "GoogleCloudPlatform/cloud-functions/google//modules/secure-cloud-function-core" version = "~> 0.9" function_name = function_description = project_id = project_number = labels = location = runtime = entry_point = storage_source = build_environment_variables = event_trigger = encryption_key = service_config = { vpc_connector = service_account_email = ingress_settings = "ALLOW_INTERNAL_AND_GCLB" all_traffic_on_latest_revision = true vpc_connector_egress_settings = "ALL_TRAFFIC" runtime_env_variables = runtime_secret_env_variables = secret_volumes = } } ``` -------------------------------- ### Run Integration Tests Noninteractively Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Execute all integration tests for the example modules noninteractively using Docker. This command verifies the functionality of the modules against the prepared test project. ```bash make docker_test_integration ``` -------------------------------- ### Deploy Cloud Functions (Gen 2) with Source Code Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/README.md Basic usage example for deploying a Cloud Function (Gen 2) using the Terraform module. Ensure all required variables like function name, project ID, location, runtime, entrypoint, and storage source details are provided. ```hcl module "cloud_functions2" { source = "GoogleCloudPlatform/cloud-functions/google" version = "~> 0.9" # Required variables function_name = "" project_id = "" location = "" runtime = "" entrypoint = "" storage_source = { bucket = "" object = "" generation = "" } } ``` -------------------------------- ### Get Serverless Cloud Build Service Account Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Retrieves the service account email for Cloud Build operations within the serverless project. ```bash export SERVERLESS_PROJECT_CB_SA=$(terraform -chdir="gcp-projects/business_unit_1/production" output -raw serverless_project_cb_sa) echo "SERVERLESS_PROJECT_CB_SA = ${SERVERLESS_PROJECT_CB_SA}" ``` -------------------------------- ### Terraform Configuration for Production Environment Setup Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md This Terraform configuration sets up the necessary infrastructure for a production environment, including KMS for encryption, artifact registry service accounts, and a storage bucket for Cloud Functions. It ensures that security and serverless projects are correctly provisioned and configured. ```hcl resource "google_kms_key_ring" "serverless" { count = var.enable_scf ? 1 : 0 project = module.security_project[0].project_id name = local.serverless_keyring_name location = local.default_region } resource "google_kms_crypto_key" "serverless" { count = var.enable_scf ? 1 : 0 name = local.serverless_key_name key_ring = google_kms_key_ring.serverless[0].id rotation_period = var.key_rotation_period # Use HSM for higher security purpose = "ENCRYPT_DECRYPT" version_template { protection_level = "HSM" } } module "artifact_registry_kms" { source = "terraform-google-modules/kms/google" version = "~> 2.1" count = var.enable_scf ? 1 : 0 project_id = module.security_project[0].project_id location = local.default_region keyring = local.serverless_keyring_name keys = [local.serverless_key_name] set_decrypters_for = [local.serverless_key_name] set_encrypters_for = [local.serverless_key_name] decrypters = [local.decrypters] encrypters = [local.encrypters] prevent_destroy = false key_rotation_period = var.key_rotation_period key_protection_level = "HSM" depends_on = [ time_sleep.sa_propagation ] } resource "google_project_service_identity" "artifact_sa" { provider = google-beta count = var.enable_scf ? 1 : 0 project = module.security_project[0].project_id service = "artifactregistry.googleapis.com" } module "cloudfunction_source_bucket" { source = "terraform-google-modules/cloud-storage/google//modules/simple_bucket" version = "~>3.4" count = var.enable_scf ? 1 : 0 project_id = module.serverless_project[0].project_id name = "bkt-${local.default_region}-${module.serverless_project[0].project_number}-cfv2-zip-files" location = local.default_region storage_class = "REGIONAL" force_destroy = true encryption = { default_kms_key_name = module.artifact_registry_kms[0].keys[local.serverless_key_name] } depends_on = [ module.artifact_registry_kms ] } ``` -------------------------------- ### Initialize Terraform and Get Default Region and Project ID Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Initializes Terraform in the shared environment and retrieves the default region and Cloud Build project ID. This is a prerequisite for subsequent build trigger operations. ```bash terraform -chdir="./gcp-bootstrap/envs/shared" init export DEFAULT_REGION=$(terraform -chdir="./gcp-bootstrap/envs/shared" output -json common_config | jq '.default_region' --raw-output) export CICD_PROJECT_ID=$(terraform -chdir="./gcp-bootstrap/envs/shared" output -raw cloudbuild_project_id) echo "trigger page = https://console.cloud.google.com/cloud-build/triggers;region=${DEFAULT_REGION}?project=${CICD_PROJECT_ID}" ``` -------------------------------- ### Generate Self-Signed Certificate for SWP Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md A bash script to generate a self-signed certificate and private key using openssl, and then create a Certificate Manager certificate resource in Google Cloud. Ensure openssl is installed. ```bash #!/bin/bash # Copyright 2023 Google LLC # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. project_id=${1} location=${2} generate_self_signed_certificate() { if [[ ! -x "$(command -v openssl)" ]]; then echo "openssl not found" exit 1 fi openssl req -x509 -newkey rsa:2048 \ -keyout key.pem \ -out cert.pem -days 365 \ -subj '/CN=myswp.example.com' -nodes \ -addext "subjectAltName=DNS:myswp.example.com" gcloud certificate-manager certificates create swp-certificate \ --certificate-file=cert.pem \ --private-key-file=key.pem \ --location="${location}" \ --project="${project_id}" } generate_self_signed_certificate ``` -------------------------------- ### Define Security Project Module Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Configures a security project, similar to the serverless project setup, with specific roles and network configurations. This module is used for establishing a dedicated security environment. ```terraform module "security_project" { source = "../single_project" count = var.enable_scf ? 1 : 0 org_id = local.org_id billing_account = local.billing_account folder_id = local.env_folder_name environment = var.env vpc_type = "restricted" shared_vpc_host_project_id = local.restricted_host_project_id shared_vpc_subnets = local.restricted_subnets_self_links project_budget = var.project_budget project_prefix = local.project_prefix enable_cloudbuild_deploy = local.enable_cloudbuild_deploy app_infra_pipeline_service_accounts = local.app_infra_pipeline_service_accounts sa_roles = { ``` -------------------------------- ### Prepare Test Project with Docker Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Use the make command to prepare the test project environment using Docker. This command sets up the necessary infrastructure for integration testing. ```bash make docker_test_prepare ``` -------------------------------- ### Get Serverless GCS Service Account Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Retrieves the service account email for GCS operations within the serverless project. ```bash export SERVERLESS_PROJECT_GCS_SA=$(terraform -chdir="gcp-projects/business_unit_1/production" output -raw serverless_project_gcs_sa) echo "SERVERLESS_PROJECT_GCS_SA = ${SERVERLESS_PROJECT_GCS_SA}" ``` -------------------------------- ### Conditional resource creation based on enable_scf and enable_swp flags Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md These resources, including `null_resource` for certificate generation and `time_sleep` for waiting, are conditionally created only when both `enable_scf` and `enable_swp` are true. The `secure_web_proxy` module is also conditionally included. ```hcl resource "null_resource" "generate_certificate" { count = var.enable_scf && var.enable_swp ? 1 : 0 ... ``` ```hcl resource "time_sleep" "wait_upload_certificate" { count = var.enable_scf && var.enable_swp ? 1 : 0 ... ``` ```hcl module "secure_web_proxy" { source = "github.com/GoogleCloudPlatform/terraform-google-cloud-functions //modules/secure-web-proxy" count = var.enable_scf && var.enable_swp ? 1 : 0 ... } ``` -------------------------------- ### Enable Secure Cloud Function and Set Web Proxy IPs in main.tf Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Configures the `base_env` module in `main.tf` by setting `enable_scf` to `true` and assigning the `restricted_subnet_secure_web_proxy_ip` local value. ```hcl module "base_env" { source = "../../modules/base_env" env = local.env environment_code = local.environment_code access_context_manager_policy_id = var.access_context_manager_policy_id ... enable_scf = true restricted_subnet_secure_web_proxy_ip = local.restricted_subnet_secure_web_proxy_ip } ``` -------------------------------- ### Delete Secure Web Proxy Gateway Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-web-proxy.md Deletes the Secure Web Proxy gateway using the gcloud command. Ensure you are in the correct example directory. ```bash export PROJECT_ID=$(terraform output -raw network_project_id) gcloud network-services gateways delete secure-web-proxy --location=${REGION} --project=${PROJECT_ID} ``` -------------------------------- ### Enable SSH Ingress Firewall Rule Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/examples/secure_cloud_function_internal_server/README.md Create a firewall rule to allow SSH ingress from IAP to the web server machine. Replace `` and `` with your specific network and project IDs. ```bash gcloud compute firewall-rules create allow-ssh-ingress-from-iap \ --direction=INGRESS \ --action=allow \ --rules=tcp:22 \ --source-ranges=35.235.240.0/20 \ --network= \ --project= ``` -------------------------------- ### Get Serverless Project GCS Service Account Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Retrieves the default Google Cloud Storage service account for the serverless project. This account is often used for GCS-related operations. ```terraform data "google_storage_project_service_account" "serverless_project_gcs_account" { count = var.enable_scf ? 1 : 0 project = module.serverless_project[0].project_id } ``` -------------------------------- ### Generate Documentation Tables Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Run the make generate_docs command to automatically generate or refresh the Inputs and Outputs tables in the README files. This should be done whenever module interfaces change. ```bash make generate_docs ``` -------------------------------- ### Add APIs to serviceusage_allow_basic_apis.yaml Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Include these APIs in the `services` list within the `serviceusage_allow_basic_apis.yaml` file to enable necessary functionalities for Secure Cloud Functions. ```yaml - "cloudfunctions.googleapis.com" - "eventarc.googleapis.com" - "eventarcpublishing.googleapis.com" - "run.googleapis.com" - "vpcaccess.googleapis.com" - "containerscanning.googleapis.com" - "certificatemanager.googleapis.com" - "networkservices.googleapis.com" - "networksecurity.googleapis.com" ``` -------------------------------- ### Create Function Infrastructure Files Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Sets up the directory structure and creates essential Terraform configuration files for the cloud function. It also copies necessary build scripts and gitignore file from the foundation repository. ```bash cd bu1-scf-app git checkout -b plan mkdir -p business_unit_1/production/functions/bq-to-cf mkdir -p business_unit_1/production/templates touch business_unit_1/production/backend.tf touch business_unit_1/production/versions.tf touch business_unit_1/production/variables.tf touch business_unit_1/production/terraform.tfvars touch business_unit_1/production/main.tf cp ../terraform-example-foundation/build/cloudbuild-tf-* . cp ../terraform-example-foundation/build/tf-wrapper.sh . cp ../terraform-example-foundation/5-app-infra/.gitignore . chmod 755 ./tf-wrapper.sh ``` -------------------------------- ### Configure Security Project with IAM and APIs Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Sets up the security project, enabling necessary APIs and configuring VPC Service Controls. This project hosts resources like artifact registries. ```terraform module "security_project" { source = "terraform-google-modules/project-factory/google" version = "~> 13.0" name = "${var.business_code}-sec-app" billing_code = "1234" folder_id = var.folder_id labels = { environment = var.environment application = "${var.business_code}-sec-app" } apis = [ "cloudbilling.googleapis.com", "cloudkms.googleapis.com", "artifactregistry.googleapis.com", "compute.googleapis.com", "cloudfunctions.googleapis.com", "cloudbuild.googleapis.com", "iam.googleapis.com", "eventarc.googleapis.com", "run.googleapis.com", "iam.googleapis.com", "vpcservicecontrol.googleapis.com", "secretmanager.googleapis.com", "logging.googleapis.com", "storage.googleapis.com", "pubsub.googleapis.com", "cloudresourcemanager.googleapis.com", "serviceusage.googleapis.com", "containerregistry.googleapis.com", "cloudsecurityscanner.googleapis.com", "osconfig.googleapis.com", "tpu.googleapis.com", "dataflow.googleapis.com", "bigquery.googleapis.com", "cloudkms.googleapis.com", "cloudfunctions.googleapis.com", "cloudbuild.googleapis.com", "iam.googleapis.com", "eventarc.googleapis.com", "run.googleapis.com", "iam.googleapis.com", "vpcservicecontrol.googleapis.com", "secretmanager.googleapis.com", "logging.googleapis.com", "storage.googleapis.com", "pubsub.googleapis.com", "cloudresourcemanager.googleapis.com", "serviceusage.googleapis.com", "containerregistry.googleapis.com", "cloudsecurityscanner.googleapis.com", "osconfig.googleapis.com", "tpu.googleapis.com", "dataflow.googleapis.com", "bigquery.googleapis.com" ] service_account_roles = { "roles/storage.admin" = [ "roles/bigquery.admin", "roles/serviceusage.serviceUsageAdmin", "roles/cloudkms.admin" ] } activate_apis = [ "cloudbilling.googleapis.com", "cloudkms.googleapis.com", "artifactregistry.googleapis.com", "compute.googleapis.com" ] vpc_service_control_attach_enabled = "true" vpc_service_control_perimeter_name = "accessPolicies/${local.access_context_manager_policy_id}/servicePerimeters/${local.perimeter_name}" vpc_service_control_sleep_duration = "60s" # Metadata project_suffix = "sec" application_name = "${var.business_code}-sec" billing_code = "1234" primary_contact = "example@example.com" secondary_contact = "example2@example.com" business_code = var.business_code } ``` -------------------------------- ### Configure Secure Web Proxy in Terraform Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Defines the configuration for the Secure Web Proxy, including certificates, IP addresses, ports, and URL lists for restricted access. This is used within a Terraform setup. ```hcl certificates = ["projects/${local.restricted_project_id}/locations/${local.default_region}/certificates/swp-certificate"] addresses = [var.restricted_subnet_secure_web_proxy_ip[local.default_region]] ports = [443] proxy_ip_range = "10.129.0.0/23" # This list of URL was obtained through Cloud Function imports # It will change depending on what imports your CF are using. url_lists = [ "*google.com/go*", "*github.com/GoogleCloudPlatform*", "*github.com/cloudevents*", "*golang.org/x*", "*google.golang.org/*", "*github.com/golang/*", "*github.com/google/*", "*github.com/googleapis/*", "*github.com/json-iterator/go", "*github.com/modern-go/concurrent", "*github.com/modern-go/reflect2", "*go.opencensus.io", "*go.uber.org/atomic", "*go.uber.org/multierr", "*go.uber.org/zap" ] depends_on = [ null_resource.generate_certificate, time_sleep.wait_upload_certificate ] ``` -------------------------------- ### Insert Data into BigQuery Table Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md This SQL command demonstrates how to insert a sample row into a BigQuery table. Replace `` with your actual project ID and ensure the table path matches your schema. ```sql INSERT INTO `.dst_secure_cloud_function.tbl_test` VALUES ``` -------------------------------- ### Set Test Environment Variables Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Set the necessary environment variables for the test project, including organization ID, folder ID, and billing account ID. These are used by the Terraform setup for integration testing. ```bash export TF_VAR_org_id="your_org_id" export TF_VAR_folder_id="your_folder_id" export TF_VAR_billing_account="your_billing_account_id" ``` -------------------------------- ### Enable Secure Cloud Function infrastructure in production environment Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Set the `enable_scf` variable to `true` in the production environment's main configuration file. This activates the creation of necessary infrastructure for Secure Cloud Functions. ```hcl module "env" { source = "../../modules/env_baseline" env = "production" environment_code = "p" monitoring_workspace_users = var.monitoring_workspace_users remote_state_bucket = var.remote_state_bucket enable_scf = true ... } ``` -------------------------------- ### Configure Secure Web Proxy toggle in production main.tf Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Set the `enable_swp` toggle to `false` in the production environment's main.tf to disable the Secure Web Proxy. Ensure `restricted_subnet_secure_web_proxy_ip` is also set. ```hcl module "base_env" { source = "../../modules/base_env" env = local.env environment_code = local.environment_code access_context_manager_policy_id = var.access_context_manager_policy_id ... enable_scf = true enable_swp = false restricted_subnet_secure_web_proxy_ip = local.restricted_subnet_secure_web_proxy_ip } ``` -------------------------------- ### Basic Usage of Secure Cloud Function Module Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/modules/secure-cloud-function/README.md This snippet demonstrates the basic configuration for deploying a secure cloud function using the Terraform module. Ensure all placeholder values are replaced with actual resource names and IDs. ```hcl module "secure_cloud_function" { source = "GoogleCloudPlatform/cloud-functions/google//modules/secure-cloud-function" version = "~> 0.9" function_name = function_description = location = region = serverless_project_id = vpc_project_id = kms_project_id = key_name = keyring_name = service_account_email = connector_name = subnet_name = create_subnet = false shared_vpc_name = ip_cidr_range = "10.0.0.0/28" storage_source = { bucket = object = } runtime = entry_point = } ``` -------------------------------- ### Run Linting and Formatting Checks Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/CONTRIBUTING.md Execute the make docker_test_lint command to run linting and formatting checks across the repository files. This helps maintain code quality and consistency. ```bash make docker_test_lint ``` -------------------------------- ### Set Environment Variables for Deployment Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Exports default region and pipeline project ID, and provides a link to the Cloud Build builds page for monitoring. ```bash export DEFAULT_REGION=$(terraform -chdir="gcp-bootstrap/envs/shared" output -json common_config | jq '.default_region' --raw-output) echo ${DEFAULT_REGION} export INFRA_PIPELINE_PROJECT_ID=$(terraform -chdir="gcp-projects/business_unit_1/shared/" output -raw cloudbuild_project_id) echo ${INFRA_PIPELINE_PROJECT_ID} echo "builds page = https://console.cloud.google.com/cloud-build/builds;region=${DEFAULT_REGION}?project=${INFRA_PIPELINE_PROJECT_ID}" ``` -------------------------------- ### Activate APIs for App Infra Module Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Add BigQuery and Service Usage APIs to the activate_apis list within the app_infra_cloudbuild_project module. ```hcl activate_apis = [ "cloudbuild.googleapis.com", "sourcerepo.googleapis.com", "cloudkms.googleapis.com", "iam.googleapis.com", "artifactregistry.googleapis.com", "cloudresourcemanager.googleapis.com", "bigquery.googleapis.com", "serviceusage.googleapis.com", ] ``` -------------------------------- ### Make Certificate Generation Script Executable Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Sets execute permissions for the bash script responsible for generating Secure Web Proxy certificates. This command must be run in the terminal. ```bash chmod 755 gcp-networks/modules/base_env/generate_swp_certificate.sh ``` -------------------------------- ### Commit and Push Code Changes Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md These bash commands are used to stage all changes, commit them with a message, and push the code to the 'plan' branch of the origin repository. Ensure the 'plan' build completes before proceeding. ```bash git add . git commit -m "Secure Cloud Function initial commit" git push --set-upstream origin plan ``` -------------------------------- ### Tail Web Server Request Logs Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/examples/secure_cloud_function_internal_server/README.md Connect to the internal server via SSH and execute this command to view real-time request logs. This is useful for debugging interactions between the Cloud Function and the internal server. ```bash tail -f /tmp/request_logs.log ``` -------------------------------- ### Merge and Push to Production Branch Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md This sequence of bash commands creates a new 'production' branch, checks it out, and then pushes it to the origin repository. This is typically done after successful testing on the 'plan' branch. ```bash git checkout -b production git push --set-upstream origin production ``` -------------------------------- ### Run INSERT command for BigQuery Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/examples/secure_cloud_function_bigquery_trigger/README.md Execute this SQL command in the BigQuery console to trigger the Cloud Function. Replace `` with your actual project ID. ```sql INSERT INTO `.dst_secure_cloud_function.tbl_test` VALUES ("AX","American Express","American Express","30006041298416","Gerson Beahan","688","09/2008","04/2013","26",9287,"77443") ``` -------------------------------- ### Configure Serverless Project with IAM and APIs Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Defines the serverless project, activates necessary APIs, and configures VPC Service Controls. This is used for creating the primary project for Cloud Functions. ```terraform module "serverless_project" { source = "terraform-google-modules/project-factory/google" version = "~> 13.0" name = "${var.business_code}-scf-app" billing_code = "1234" folder_id = var.folder_id labels = { environment = var.environment application = "${var.business_code}-scf-app" } apis = [ "cloudbilling.googleapis.com", "cloudkms.googleapis.com", "artifactregistry.googleapis.com", "compute.googleapis.com", "cloudfunctions.googleapis.com", "cloudbuild.googleapis.com", "iam.googleapis.com", "eventarc.googleapis.com", "run.googleapis.com", "iam.googleapis.com", "vpcservicecontrol.googleapis.com", "secretmanager.googleapis.com", "logging.googleapis.com", "storage.googleapis.com", "pubsub.googleapis.com", "cloudresourcemanager.googleapis.com", "serviceusage.googleapis.com", "containerregistry.googleapis.com", "cloudsecurityscanner.googleapis.com", "osconfig.googleapis.com", "tpu.googleapis.com", "dataflow.googleapis.com", "bigquery.googleapis.com", "cloudkms.googleapis.com", "cloudfunctions.googleapis.com", "cloudbuild.googleapis.com", "iam.googleapis.com", "eventarc.googleapis.com", "run.googleapis.com", "iam.googleapis.com", "vpcservicecontrol.googleapis.com", "secretmanager.googleapis.com", "logging.googleapis.com", "storage.googleapis.com", "pubsub.googleapis.com", "cloudresourcemanager.googleapis.com", "serviceusage.googleapis.com", "containerregistry.googleapis.com", "cloudsecurityscanner.googleapis.com", "osconfig.googleapis.com", "tpu.googleapis.com", "dataflow.googleapis.com", "bigquery.googleapis.com" ] service_account_roles = { "roles/storage.admin" = [ "roles/bigquery.admin", "roles/serviceusage.serviceUsageAdmin", "roles/cloudkms.admin" ] } activate_apis = [ "cloudbilling.googleapis.com", "cloudkms.googleapis.com", "artifactregistry.googleapis.com", "compute.googleapis.com" ] vpc_service_control_attach_enabled = "true" vpc_service_control_perimeter_name = "accessPolicies/${local.access_context_manager_policy_id}/servicePerimeters/${local.perimeter_name}" vpc_service_control_sleep_duration = "60s" # Metadata project_suffix = "app" application_name = "${var.business_code}-app" billing_code = "1234" primary_contact = "example@example.com" secondary_contact = "example2@example.com" business_code = var.business_code } ``` -------------------------------- ### Create Artifact Registry Repository Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Sets up a Docker-formatted Artifact Registry repository in the security project, secured with a KMS key. This repository will store Cloud Function container images. ```terraform resource "google_artifact_registry_repository" "repo" { count = var.enable_scf ? 1 : 0 project = module.security_project[0].project_id location = local.default_region repository_id = "rep-secure-cloud-function" description = "Secure Cloud Run Artifact Registry Repository" format = "DOCKER" kms_key_name = module.artifact_registry_kms[0].keys[local.serverless_key_name] } ``` -------------------------------- ### Define Serverless Project Module Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Configures a serverless project with specified roles, enabled APIs, and VPC Service Controls. Use this to set up the foundational infrastructure for Cloud Functions. ```terraform module "serverless_project" { source = "../single_project" count = var.enable_scf ? 1 : 0 org_id = local.org_id billing_account = local.billing_account folder_id = local.env_folder_name environment = var.env vpc_type = "restricted" shared_vpc_host_project_id = local.restricted_host_project_id shared_vpc_subnets = local.restricted_subnets_self_links project_budget = var.project_budget project_prefix = local.project_prefix enable_cloudbuild_deploy = local.enable_cloudbuild_deploy app_infra_pipeline_service_accounts = local.app_infra_pipeline_service_accounts sa_roles = { "${var.business_code}-scf-app" = [ "roles/artifactregistry.admin", "roles/bigquery.admin", "roles/bigquery.jobUser", "roles/cloudbuild.builds.editor", "roles/cloudbuild.workerPoolOwner", "roles/cloudfunctions.admin", "roles/iam.serviceAccountAdmin", "roles/iam.serviceAccountUser", "roles/serviceusage.serviceUsageAdmin", "roles/storage.admin", "roles/resourcemanager.projectIamAdmin", "roles/pubsub.admin", ] } activate_apis = [ "accesscontextmanager.googleapis.com", "vpcaccess.googleapis.com", "container.googleapis.com", "run.googleapis.com", "eventarc.googleapis.com", "cloudbuild.googleapis.com", "cloudfunctions.googleapis.com", "cloudresourcemanager.googleapis.com", "storage-api.googleapis.com", "serviceusage.googleapis.com", "servicenetworking.googleapis.com", "iam.googleapis.com", "dns.googleapis.com", "pubsub.googleapis.com", "bigquery.googleapis.com", "cloudbilling.googleapis.com", "cloudkms.googleapis.com", "cloudbuild.googleapis.com", "artifactregistry.googleapis.com", "compute.googleapis.com", "networksecurity.googleapis.com", ] vpc_service_control_attach_enabled = "true" vpc_service_control_perimeter_name = "accessPolicies/${local.access_context_manager_policy_id}/servicePerimeters/${local.perimeter_name}" vpc_service_control_sleep_duration = "60s" # Metadata project_suffix = "c-func" application_name = "${var.business_code}-c-func" billing_code = "1234" primary_contact = "example@example.com" secondary_contact = "example2@example.com" business_code = var.business_code } ``` -------------------------------- ### Define Variables for Secure Cloud Function and Secure Web Proxy in variables.tf Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Introduces two new variables in `variables.tf`: `enable_scf` to control Secure Cloud Function infrastructure deployment and `restricted_subnet_secure_web_proxy_ip` for Secure Web Proxy IPs. ```hcl variable "enable_scf" { description = "Set to true to create the infrastructure needed by the Secure Cloud Function." type = bool default = false } variable "restricted_subnet_secure_web_proxy_ip" { description = "The IPs the will be used by the Secure Web Proxy instance." type = map(string) default = {} } ``` -------------------------------- ### Add Network Connector and Web Proxy IP Outputs in production/outputs.tf Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Adds `restricted_serverless_network_connector_id` and `restricted_subnet_secure_web_proxy_ip` to the outputs in the production environment's `outputs.tf` file. ```hcl output "restricted_serverless_network_connector_id" { description = "VPC serverless connector ID for the restricted network." value = module.base_env.restricted_serverless_network_connector_id } output "restricted_subnet_secure_web_proxy_ip" { description = "The IPs the will be used by the Secure Web Proxy instance." value = local.restricted_subnet_secure_web_proxy_ip } ``` -------------------------------- ### Clone Source Repository Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Clones the source repository for the business unit's secure cloud function application. ```bash gcloud source repos clone bu1-scf-app --project=${INFRA_PIPELINE_PROJECT_ID} ``` -------------------------------- ### Define New Locals for Secure Network Configuration Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Update the `example_secure_cloud_function_projects.tf` file to define new local variables for secure network configurations. These locals fetch necessary network details from the remote state. ```hcl restricted_network_name = data.terraform_remote_state.network_env.outputs.restricted_network_name restricted_serverless_network_connector_id = try(data.terraform_remote_state.network_env.outputs.restricted_serverless_network_connector_id, "") restricted_subnet_secure_web_proxy_ip = try(data.terraform_remote_state.network_env.outputs.restricted_subnet_secure_web_proxy_ip, {}) ``` -------------------------------- ### Copy Cloud Function Code and BigQuery Template Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Copies the necessary Cloud Function source code and BigQuery schema template from a specified path to the production directory. ```bash SCF_PATH="../terraform-google-cloud-functions/examples/secure_cloud_function_bigquery_trigger" cp "${SCF_PATH}/templates/bigquery_schema.template" ./business_unit_1/production/templates/bigquery_schema.template cp "${SCF_PATH}/functions/bq-to-cf/go.mod" ./business_unit_1/production/functions/bq-to-cf/go.mod cp "${SCF_PATH}/functions/bq-to-cf/main.go" ./business_unit_1/production/functions/bq-to-cf/main.go ``` -------------------------------- ### Add Restricted Outputs to Production Environment Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Configure the `outputs.tf` file for the production environment to expose the restricted network and host project outputs. This ensures production deployments have access to these critical configurations. ```hcl output "restricted_host_project_id" { description = "Restricted host project id." value = module.env.restricted_host_project_id } output "restricted_network_name" { description = "Restricted network name." value = module.env.restricted_network_name } output "restricted_serverless_network_connector_id" { description = "VPC serverless connector ID for the restricted network." value = module.env.restricted_serverless_network_connector_id } output "restricted_subnet_secure_web_proxy_ip" { description = "The IPs the will be used by the Secure Web Proxy instance." value = module.env.restricted_subnet_secure_web_proxy_ip } ``` -------------------------------- ### Initialize KMS Module for Artifact Registry Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Initializes the KMS module to manage encryption keys for the Artifact Registry. This ensures that container images stored in the registry are encrypted. ```terraform module "artifact_registry_kms" { source = "terraform-google-modules/kms/google" version = "~> 2.2" } ``` -------------------------------- ### Enable Cloud Functions in Production Environment Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Update the `enable_scf` variable to `true` in the `gcp-projects/business_unit_1/production/main.tf` file to enable Cloud Functions deployment in the production environment. This change should be committed and pushed after the `gcp-projects` build completes. ```hcl module "env" { source = "../../modules/base_env" env = "production" business_code = "bu1" business_unit = "business_unit_1" remote_state_bucket = var.remote_state_bucket location_kms = var.location_kms location_gcs = var.location_gcs peering_module_depends_on = var.peering_module_depends_on enable_scf = true } ``` -------------------------------- ### Enable Additional APIs for Restricted Shared VPC Host Project Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Add the specified Google Cloud APIs to the `activate_apis` list within the `restricted_shared_vpc_host_project` module. This is required for services like Serverless VPC Access connector and Secure Web Proxy. ```hcl activate_apis = [ "compute.googleapis.com", "dns.googleapis.com", "servicenetworking.googleapis.com", "container.googleapis.com", "logging.googleapis.com", "cloudresourcemanager.googleapis.com", "accesscontextmanager.googleapis.com", "billingbudgets.googleapis.com", "vpcaccess.googleapis.com", "certificatemanager.googleapis.com", "networkservices.googleapis.com", "networksecurity.googleapis.com", ] ``` -------------------------------- ### Base Environment Outputs for Secure Cloud Function Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md Outputs related to Secure Cloud Function infrastructure, including project IDs, bucket names, and service account emails. These are conditionally created based on the `enable_scf` variable. ```hcl output "serverless_project_id" { description = "The ID of the project in which Secure Cloud Functions serverless resources will be created." value = var.enable_scf ? module.serverless_project[0].project_id : "" } output "serverless_project_number" { description = "The project number in which Secure Cloud Functions serverless resources will be created." value = var.enable_scf ? module.serverless_project[0].project_number : "" } output "security_project_id" { description = "The ID of the project in which Secure Cloud Functions security resources will be created." value = var.enable_scf ? module.security_project[0].project_id : "" } output "cloudfunction_source_bucket_name" { description = "Cloud Function Source Bucket." value = var.enable_scf ? module.cloudfunction_source_bucket[0].bucket.name : "" } output "serverless_service_account_email" { description = "The service account created in the serverless project." value = var.enable_scf ? module.service_accounts[0].email : "" } output "default_region" { description = "Default region to create resources where applicable." value = local.default_region } output "serverless_project_cb_sa" { description = "The Cloud Build service account created for the serverless project." value = var.enable_scf ? google_project_service_identity.cloudbuild_sa[0].email : "" } output "serverless_project_gcs_sa" { description = "The Google Cloud Storage service account created for the serverless project." value = var.enable_scf ? data.google_storage_project_service_account.serverless_project_gcs_account[0].email_address : "" } ``` -------------------------------- ### Configure Secure Web Proxy Module Source: https://github.com/googlecloudplatform/terraform-google-cloud-functions/blob/main/docs/secure-cloud-function-on-foundation-3-0-0.md This Terraform configuration sets up the Secure Web Proxy module, including certificate generation and sleep duration for certificate upload. ```hcl /** * Copyright 2023 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ resource "null_resource" "generate_certificate" { count = var.enable_scf ? 1 : 0 triggers = { project_id = local.restricted_project_id region = local.default_region } provisioner "local-exec" { when = create command = <