### Example Node Setup for Deployments Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/bootstrap.md Illustrates Node ID and Cluster naming conventions for single-node and multi-node Envoy deployments. ```text Single-node deployment: opts.Node = "ESPv2" → Node.Id = "ESPv2" → Node.Cluster = "ESPv2_cluster" Multi-node deployment: opts.Node = "esp-node-1" → Node.Id = "esp-node-1" → Node.Cluster = "esp-node-1_cluster" ``` -------------------------------- ### Install Other Development Tools Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Install additional utilities like jq, Node.js, npm, and Python 3. ```bash sudo apt-get install jq nodejs npm python-is-python3 ``` -------------------------------- ### Install ESPv2 Dependencies Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Run this command to install all necessary project dependencies. ```bash make depend.install ``` -------------------------------- ### Install jq Source: https://github.com/googlecloudplatform/esp-v2/blob/master/prow/README.md Installs the jq utility, which is required for the HTTP Bookstore on Google Cloud Run test. Use this command if jq is not already installed. ```shell sudo apt-get install jq ``` -------------------------------- ### NewConfigManager Example Usage Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configmanager.md Example demonstrating how to initialize a ConfigManager for a GCP deployment with a managed rollout strategy. Ensure necessary URLs and options are configured. ```go // GCP deployment with managed rollout opts := options.DefaultConfigGeneratorOptions() opts.ServiceManagementURL = "https://servicemanagement.googleapis.com" opts.ServiceControlURL = "https://servicecontrol.googleapis.com" mf := metadata.NewMetadataFetcher(opts.CommonOptions) cm, err := configmanager.NewConfigManager(mf, opts) if err != nil { log.Fatal(err) } // cm.Cache() returns the snapshot cache for use with xDS server ``` -------------------------------- ### MetadataFetcher Initialization Examples Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Demonstrates how to initialize MetadataFetcher for both standard GCP deployments and non-GCP environments. The example shows setting common options like metadata URL and HTTP request timeout. ```go // GCP deployment with standard metadata server opts := options.DefaultCommonOptions() mf := metadata.NewMetadataFetcher(opts) // Non-GCP deployment (will fail on metadata access) opts.NonGCP = true mf := metadata.NewMetadataFetcher(opts) ``` -------------------------------- ### Example Usage of LoadConfigIdFromRollouts Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/serviceconfig.md Demonstrates how to use LoadConfigIdFromRollouts to get a configuration ID for managed rollout strategy and handle potential errors. ```go // Used for managed rollout strategy configId, err := fetcher.LoadConfigIdFromRollouts() if err != nil { log.Fatalf("Failed to load config from rollouts: %v", err) } // configId = "2024-01-15" (highest traffic version) ``` -------------------------------- ### Install Build Tools Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Install essential build tools including CMake, Ninja, Protobuf compiler, and libraries for Brotli and ICU. ```bash sudo apt install -y cmake ninja-build protobuf-compiler brotli \ libicu-dev libbrotli-dev ``` -------------------------------- ### Get Default Common Options Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/options.md Returns CommonOptions with sensible defaults. Use this as a starting point for common configurations. ```go opts := options.DefaultCommonOptions() opts.HttpRequestTimeout = 60 * time.Second opts.Node = "my-esp-node" ``` -------------------------------- ### FetchAccessToken Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Shows how to call FetchAccessToken and handle potential errors. It logs the token's validity duration if the fetch is successful. ```go token, expiry, err := mf.FetchAccessToken() if err != nil { log.Printf("Failed to fetch token: %v", err) return } // token is valid for at least 60 seconds log.Printf("Token valid for: %v", expiry) ``` -------------------------------- ### Install Clang-14 and Related Tools Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Install Clang-14 and its associated development libraries and tools. If these commands fail, consult the official Clang installation instructions for your operating system. ```bash sudo apt-get update sudo apt-get install -y llvm-14-dev libclang-14-dev clang-14 \ clang-tools-14 clang-format-14 xz-utils lld-14 ``` -------------------------------- ### Install Envoy Dependencies Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Follow the instructions from the Envoy project's README to set up the necessary dependencies for building Envoy locally. ```bash # Follow instructions from https://github.com/envoyproxy/envoy/blob/master/bazel/README.md#quick-start-bazel-build-for-developers ``` -------------------------------- ### OpenCensus Configuration Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tracing.md Example of OpenCensus configuration for tracing, including Stackdriver exporter, project ID, address, and context propagation settings. ```yaml OpenCensusConfig: - TraceConfig: Span limits (attributes, annotations, etc.) - StackdriverExporterEnabled: true - StackdriverProjectId: - StackdriverAddress: - IncomingTraceContext: [TRACE_CONTEXT, CLOUD_TRACE_CONTEXT] - OutgoingTraceContext: [TRACE_CONTEXT, CLOUD_TRACE_CONTEXT] ``` -------------------------------- ### FetchServiceName Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Demonstrates how to retrieve the service name and log an error if it's not found in the metadata. ```go serviceName, err := mf.FetchServiceName() if err != nil { log.Printf("No service name in metadata: %v", err) return } // serviceName = "my-api.endpoints.example.com" ``` -------------------------------- ### Envoy Bootstrap Configuration Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/src/envoy/README.md Example of configuring Envoy filters within the bootstrap configuration file. ```yaml filter_chains: - filters: - name: envoy.filters.network.http_connection_manager config: ... other config options ... http_filters: ... list of filters... ``` -------------------------------- ### Non-GCP Sidecar Deployment Startup Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tokengenerator.md Configures and starts the token agent, then initializes the config manager for a non-GCP sidecar deployment. ```go // Startup (main.go) opts := commonflags.DefaultCommonOptionsFromFlags() // Start token agent handler := tokengenerator.MakeTokenAgentHandler(opts.ServiceAccountKey) go http.ListenAndServe(fmt.Sprintf(":%d", opts.TokenAgentPort), handler) // Create config manager (uses token from token agent) cm, err := configmanager.NewConfigManager(nil, opts) if err != nil { log.Fatal(err) } ``` -------------------------------- ### Non-GCP Sidecar Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/README.md Example command for configuring ESPv2 as a non-GCP sidecar. This setup requires a service account key and specifies the backend address. ```bash espv2 \ --non_gcp=true \ --service_json_path=/etc/esp/service.json \ --service_account_key=/etc/esp/key.json \ --backend_address=http://localhost:8081 \ --listener_port=8080 ``` -------------------------------- ### Install LLVM and LibFuzzer Dependencies Source: https://github.com/googlecloudplatform/esp-v2/blob/master/tests/fuzz/README.md Installs necessary LLVM and LibFuzzer components using apt. Ensure you have clang-10 and related libraries. ```shell sudo apt install llvm-10-dev libclang-10-dev clang-10 xz-utils lld ``` -------------------------------- ### Service Configuration JSON Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/configuration.md This is an example of a service configuration file in JSON format, matching the Google service config protobuf format. It defines API details, authentication providers, and backend routing rules. ```json { "type": "google.api.Service", "config_version": 3, "name": "my-api.endpoints.example.com", "title": "My API", "id": "2024-01-15", "apis": [ { "name": "myapi.v1.MyService", "version": "v1", "methods": [ { "name": "GetResource", "request_type_url": "type.googleapis.com/myapi.v1.GetResourceRequest", "response_type_url": "type.googleapis.com/myapi.v1.GetResourceResponse" } ] } ], "authentication": { "providers": [ { "id": "google_service_account", "issuer": "accounts.google.com", "jwks_uri": "https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com" } ] }, "http": { "rules": [ { "selector": "myapi.v1.MyService.GetResource", "get": "/v1/resources/{resource_id}" } ] }, "backend": { "rules": [ { "selector": "myapi.v1.MyService.GetResource", "address": "grpc://backend:50051", "operation_deadline": 15.0 } ] } } ``` -------------------------------- ### Source File and Line Reference Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/README.md This format indicates the exact location of code within the source repository, allowing developers to easily navigate to the implementation details. ```text **File:** `src/go/configmanager/config_manager.go:77-203` ``` -------------------------------- ### Download ESPv2 Source Code with Go Get Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Use `go get` to download the ESPv2 source code. This command will place the code under your `$HOME/go/src/github.com/GoogleCloudPlatform/` directory. ```go go get github.com/GoogleCloudPlatform/esp-v2 ``` -------------------------------- ### GCP with Managed Rollouts Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/README.md Example command for configuring ESPv2 in a GCP environment with managed rollouts. Ensure --check_metadata is set to true for GCP deployments. ```bash espv2 \ --service=my-api.endpoints.example.com \ --rollout_strategy=managed \ --check_metadata=true \ --backend_address=grpc://backend:50051 \ --listener_port=8080 ``` -------------------------------- ### Install Dependent Libraries Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Installs libraries required for building Config Manager. ```bash make depend.install make depend.install.endpoints ``` -------------------------------- ### Run Docker Instance for ESPv2 Development Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md This command starts a Docker container for ESPv2 development. Ensure you replace `${latest_image_link}` with the actual image URL. The `--privileged` flag and volume mounts are necessary for proper operation. ```bash docker run -ti --privileged -v /var/run/docker.sock:/var/run/docker.sock -v ~/esp-v2:/esp-v2 ${latest_image_link} /bin/bash ``` -------------------------------- ### Verbose Annotations Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tracing.md Example JSON objects representing verbose annotations with timing events, such as 'request_started' and 'response_sent'. ```json { "event": "request_started", "timestamp": "2024-01-15T10:00:00Z" } ``` ```json { "event": "backend_request_sent", "timestamp": "2024-01-15T10:00:00.001Z" } ``` ```json { "event": "backend_response_received", "timestamp": "2024-01-15T10:00:00.050Z" } ``` ```json { "event": "response_sent", "timestamp": "2024-01-15T10:00:00.051Z" } ``` -------------------------------- ### Get Default ConfigGenerator Options Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/options.md Returns ConfigGeneratorOptions with sensible defaults. Use this for generating ESPv2 configurations. ```go opts := options.DefaultConfigGeneratorOptions() opts.BackendAddress = "grpc://backend.example.com:50051" opts.ListenerPort = 8080 opts.ServiceManagementURL = "https://servicemanagement.googleapis.com" ``` -------------------------------- ### API Request Flow Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/OVERVIEW.md Illustrates the step-by-step processing of an incoming HTTP request through ESPv2, including various filters and interactions with backend services. ```text 1. Request arrives at Envoy listener (port 8080) 2. HTTP Connection Manager processes: - Normalizes path - Routes to virtual host 3. Custom filters in sequence: - JWT Authentication Filter ├─ Extract JWT from Authorization header ├─ Fetch JWKS from issuer ├─ Validate signature └─ Extract claims - Service Control Filter (Check Call) ├─ Call Service Control Check API ├─ Verify API key / quota └─ Proceed if allowed - gRPC-JSON Transcoding Filter (if needed) ├─ Convert JSON request to gRPC └─ Convert gRPC response to JSON 4. Router Filter ├─ Match route: GET /v1/resources/{id} ├─ Extract path parameters └─ Select upstream cluster 5. Load Balancer ├─ Select backend connection pool └─ Rotate through healthy endpoints 6. Upstream Connection ├─ Forward request to backend ├─ Wait for response └─ Forward response to client 7. Service Control Filter (Report Call) ├─ Report metrics to Service Control API ├─ Include status, size, latency └─ Non-blocking (fail-open) 8. Response sent to client ``` -------------------------------- ### Parse Command-line Flags with CommonFlags Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/commonflags.md This example demonstrates how to parse command-line flags using the commonflags package and obtain a CommonOptions structure. ```go import "github.com/GoogleCloudPlatform/esp-v2/src/go/commonflags" // Parse command-line flags (os.Args) opts := commonflags.DefaultCommonOptionsFromFlags() // opts now contains all flag values as CommonOptions ``` -------------------------------- ### Test HTTP API with URL Binding Parameter Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-k8s.md Test the HTTP API using URL binding parameters for specific resources. This example targets a specific shelf and book. ```bash curl --header "x-api-key: $API_KEY " http://$HOST/v1/shelves/1/books/1?key=$API_KEY"&echo ``` -------------------------------- ### Configure OpenAPI Host and Backend Address Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-cloudrun.md Example snippet showing how to configure the OpenAPI document for ESPv2. Update 'host' with your PROXY_SERVICE_URL and 'x-google-backend.address' with your BACKEND_SERVICE_URL. ```json ... "host": "PROXY_SERVICE_URL", "x-google-backend": { "address": "https://BACKEND_SERVICE_URL" }, ... ``` -------------------------------- ### Sampling Rate Behavior Example Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tracing.md Illustrates how the SamplingRate field in TracingOptions determines the percentage of requests traced. Note that rates are rounded to 4 decimal places, and the internal storage might represent percentages. ```go opts.TracingOptions.SamplingRate = 0.01 // Internally stored as 1.0 (for percent: 0.01 * 100 = 1.0%) ``` -------------------------------- ### MethodInfo IsGRPCPath Check Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configinfo.md Checks if a given path starts with the method's gRPC path. Returns true if the path matches. ```go func (m *MethodInfo) IsGRPCPath(path string) bool ``` -------------------------------- ### Initialize ConfigManager with Common Options Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/commonflags.md Demonstrates a typical integration pattern in a Go application's `main.go` file, showing how to parse flags, obtain common options, and initialize a `configmanager.ConfigManager`. ```go package main import ( "flag" "github.com/GoogleCloudPlatform/esp-v2/src/go/commonflags" "github.com/GoogleCloudPlatform/esp-v2/src/go/configmanager" ) func main() { // Define application-specific flags // (ConfigManager flags, ConfigGenerator flags, etc.) // Parse all flags flag.Parse() // Get CommonOptions from flags opts := commonflags.DefaultCommonOptionsFromFlags() // Use opts to create other structures cm, err := configmanager.NewConfigManager(mf, opts) if err != nil { log.Fatal(err) } } ``` -------------------------------- ### ConfigManager Initialization and Usage Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/serviceconfig.md Illustrates the typical integration pattern with ConfigManager, including creating a ServiceConfigFetcher and handling both fixed and managed rollout strategies. ```go // In configmanager.NewConfigManager // Create fetcher with OAuth2 token source fetcher := serviceconfig.NewServiceConfigFetcher( client, opts.ServiceManagementURL, serviceName, accessToken, // Callable to get fresh tokens ) // Fixed strategy: fetch specified config if rolloutStrategy == "fixed" { config, err := fetcher.FetchConfig(configId) // ... } // Managed strategy: fetch latest if rolloutStrategy == "managed" { latestConfigId, err := fetcher.LoadConfigIdFromRollouts() config, err := fetcher.FetchConfig(latestConfigId) // Start polling for updates detector := NewRolloutIdChangeDetector(...) detector.SetDetectRolloutIdChangeTimer(checkInterval, func() { newConfigId, err := fetcher.LoadConfigIdFromRollouts() if newConfigId != currentConfigId { fetcher.FetchConfig(newConfigId) // Reload Envoy configuration } }) } ``` -------------------------------- ### Deploy Bookstore Backend Service on Cloud Run Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-cloudrun.md Deploys the sample Bookstore backend service to Google Cloud Run. Replace CLOUD_RUN_SERVICE_NAME and YOUR_PROJECT_ID with your specific values. ```bash gcloud run deploy CLOUD_RUN_SERVICE_NAME \ --image="gcr.io/endpoints-release/bookstore:1" \ --allow-unauthenticated \ --platform managed \ --project=YOUR_PROJECT_ID ``` -------------------------------- ### Configure gRPC Backend with Transcoding Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/options.md Sets up options for a gRPC backend with transcoding enabled. Ignores unknown query parameters and preserves proto field names. ```go opts := options.DefaultConfigGeneratorOptions() opts.BackendAddress = "grpc://backend:50051" opts.TranscodingIgnoreUnknownQueryParameters = true opts.TranscodingPreserveProtoFieldNames = true ``` -------------------------------- ### Configure ESPv2 for Multi-Backend Load Balancing Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/configuration.md Use this command to launch ESPv2 with multiple backends configured for load balancing. Specify the service configuration, backend addresses, and retry policies. ```bash espv2 \ --service_json_path=/etc/esp/service_config.json \ --backend_address=grpc://backend1:50051 \ --backend_cluster_max_requests=100 \ --backend_retry_num=3 \ --backend_retry_ons=reset,connect-failure,refused-stream \ --listener_port=8080 ``` -------------------------------- ### Set Go Environment Variables Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Configure GOPATH, GOBIN, and PATH environment variables for Go development. Remember to source your .profile file after adding these. ```bash export GOPATH=$HOME/go export GOBIN=$GOPATH/bin export PATH=$GOBIN:$PATH ``` -------------------------------- ### MethodInfo.Operation Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configinfo.md Gets the fully-qualified operation name for a given MethodInfo. ```APIDOC ## MethodInfo.Operation ### Description Returns fully-qualified operation name. ### Method Signature ```go func (m *MethodInfo) Operation() string ``` ### Parameters None ### Returns Operation name in format "ApiName.MethodName" ### Example "myapi.v1.MyService.GetResource" ``` -------------------------------- ### Invalid Sampling Rate Validation Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tracing.md Example demonstrating the validation of the sampling rate. Setting it above 1.0 results in an error. ```go opts.TracingOptions.SamplingRate = 1.5 // Invalid tracingConfig, err := tracing.CreateTracing(opts.TracingOptions) // err: invalid trace sampling rate: 1.5. It must be >= 0.0 and <= 1.0 ``` -------------------------------- ### Get Default Common Options Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/commonflags.md Access the default common options for ESPv2 using the `options.DefaultCommonOptions()` function in Go. ```go var defaults = options.DefaultCommonOptions() ``` -------------------------------- ### ESPv2 Options and Configuration Flow Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/OVERVIEW.md Shows how command-line flags are processed to generate common options used throughout ESPv2. ```text Command-line Flags ↓ commonflags.DefaultCommonOptionsFromFlags() ↓ CommonOptions ├─ Used by: bootstrap, metadata, tracing └─ Extended by: ConfigGeneratorOptions └─ Used by: configmanager, configgenerator ``` -------------------------------- ### Build ESPv2 Binaries Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md These commands build the main ESPv2 binaries and the Envoy proxy. ```bash make build make build-envoy ``` -------------------------------- ### IsDiscoveryAPI Check Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configinfo.md Determines if an operation name belongs to a Google Discovery API. This is true if the operation name starts with the 'google.discovery' prefix. ```go func IsDiscoveryAPI(operationName string) bool ``` -------------------------------- ### Deploy Endpoints Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-k8s.md Deploy the proto descriptor and configuration files to Google Service Management. ```bash gcloud endpoints services deploy api_descriptor.pb api_config_auth.yaml ``` -------------------------------- ### Deploy OpenAPI Specification Source: https://github.com/googlecloudplatform/esp-v2/blob/master/examples/README.md Deploys an OpenAPI specification to Google Service Management to generate a service configuration. Ensure you have the gcloud CLI installed and authenticated. ```shell gcloud endpoints services deploy ./examples/$DIRECTORY/openapi_swagger.json ``` -------------------------------- ### Deploy ESPv2 Service on Kubernetes Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-k8s.md Use this command to deploy the bookstore service and ESPv2 to your Kubernetes cluster. Ensure your bookstore.yaml is correctly configured. ```bash kubectl create -f tests/e2e/testdata/bookstore_grpc/gke/bookstore.yaml ``` -------------------------------- ### Deploy Endpoints Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-cloudrun.md Deploys the OpenAPI configuration to Google Service Management. Ensure bookstore_swagger_template.json is correctly configured before running. ```bash gcloud endpoints services deploy bookstore_swagger_template.json ``` -------------------------------- ### SetDetectRolloutIdChangeTimer Function Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/serviceconfig.md Starts periodic checks for rollout ID changes. The callback function is invoked when a new rollout is detected. This is used internally by ConfigManager. ```Go // Start periodic rollout checks func (r *RolloutIdChangeDetector) SetDetectRolloutIdChangeTimer( interval time.Duration, callback func()) // Called when new rollout detected ``` -------------------------------- ### Test HTTP API with Query Parameter and Request Body Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-k8s.md This command demonstrates testing the HTTP API with a POST request, including a query parameter and an API key. Ensure API_KEY and JWT_TOKEN are set. ```bash curl -X POST --header "x-api-key: $API_KEY " http://$HOST/v1/shelves?access_token=$JWT_TOKEN&id=3 curl --header "x-api-key: $API_KEY " http://$HOST/v1/shelves?access_token=$JWT_TOKEN&echo ``` -------------------------------- ### Project ID Requirement Validation Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tracing.md Example showing the requirement for a Project ID. An empty Project ID results in tracing being silently disabled with a warning log. ```go opts.TracingOptions.ProjectId = "" tracingConfig, _ := tracing.CreateTracing(opts.TracingOptions) // Returns: nil, nil (silently disabled with warning log) ``` -------------------------------- ### Configure Local Sidecar Deployment Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/options.md Sets up options for a local sidecar deployment. Ensures the backend address is local and metadata is available. ```go opts := options.DefaultConfigGeneratorOptions() opts.BackendAddress = "http://localhost:8081" opts.ListenerPort = 8080 opts.NonGCP = false // metadata available ``` -------------------------------- ### Get GKE Cluster Credentials Source: https://github.com/googlecloudplatform/esp-v2/blob/master/prow/README.md Retrieves credentials for a GKE cluster, allowing your shell to interact with it. Replace placeholders with your cluster's name, zone, and project. ```shell gcloud container clusters get-credentials ${CLUSTER_NAME} --zone ${ZONE} --project cloudesf-testing ``` -------------------------------- ### Assemble Complete Envoy Bootstrap Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/bootstrap.md Combines Node, Admin, and LayeredRuntime configurations with static or dynamic resources to create a complete Envoy bootstrap configuration. ```go // In ads/main/main.go or similar opts := commonflags.DefaultCommonOptionsFromFlags() bootstrap := &bootstrappb.Bootstrap{ Node: bootstrap.CreateNode(opts), Admin: bootstrap.CreateAdmin(opts), LayeredRuntime: bootstrap.CreateLayeredRuntime(), StaticResources: &bootstrappb.Bootstrap_StaticResources{ Listeners: [...], }, DynamicResources: &bootstrappb.Bootstrap_DynamicResources{ AdsConfig: [...], }, } // Marshal to JSON or YAML for envoy -c ``` -------------------------------- ### Build ESPv2 Docker Image for Cloud Run Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-cloudrun.md Automates the process of building a new ESPv2 Docker image with the service configuration baked in. Requires gcloud SDK and the provided bash script. ```bash chmod +x gcloud_build_image && ./gcloud_build_image -s ENDPOINTS_SERVICE_NAME -c ENDPOINTS_SERVICE_CONFIG_ID -p YOUR_PROJECT_ID ``` -------------------------------- ### FetchRolloutStrategy Method Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Fetches the rollout strategy for the service from the metadata server. The strategy can be either 'fixed' or 'managed'. ```go func (mf *MetadataFetcher) FetchRolloutStrategy() (string, error) ``` -------------------------------- ### Run HTTP Bookstore on Cloud Run Test Source: https://github.com/googlecloudplatform/esp-v2/blob/master/prow/README.md Executes the end-to-end test for the HTTP Bookstore deployed on Google Cloud Run. Ensure you have the Cloud Run Admin role and jq installed. ```shell ./prow/e2e-cloud-run-http-bookstore.sh ``` -------------------------------- ### Get External IP Address of the Service Source: https://github.com/googlecloudplatform/esp-v2/blob/master/doc/esp-v2-on-k8s.md Retrieve the external IP address of your deployed service. This IP is needed to send requests to your API. It may take a few minutes for the IP to become available. ```bash HOST=$(kubectl get service app | awk '{print $4}' | grep -v EXTERNAL-IP) ``` -------------------------------- ### Static File Configuration Source Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configmanager.md Use the --service_json_path flag to specify a local service configuration file. This method has the highest priority. ```bash --service_json_path=/etc/esp/service_config.json ``` -------------------------------- ### MakeTokenAgentHandler Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tokengenerator.md Creates an HTTP request handler for Envoy's token agent. This handler listens for GET requests on the `/local/access_token` endpoint and returns a JSON object containing the access token and its expiration time. It handles token generation and caching internally. ```APIDOC ## MakeTokenAgentHandler ### Description Creates an HTTP request handler for Envoy's token agent. This handler listens for GET requests on the `/local/access_token` endpoint and returns a JSON object containing the access token and its expiration time. It handles token generation and caching internally. ### Method Signature ```go func MakeTokenAgentHandler(serviceAccountKey string) http.Handler ``` ### Parameters #### Path Parameters - **serviceAccountKey** (string) - Required - Path to Google service account JSON key file ### Returns - **http.Handler:** HTTP handler implementing token agent API. ### Token Agent Endpoint ``` GET /local/access_token ``` ### Response (Success) ```json { "access_token": "", "expires_in": } ``` ### Response (Error) HTTP 500 with error message if token generation fails. ### Example ```go import ( "net/http" "github.com/GoogleCloudPlatform/esp-v2/src/go/tokengenerator" ) // Create token agent handler handler := tokengenerator.MakeTokenAgentHandler("/etc/esp/sa-key.json") // Start token agent server go http.ListenAndServe(":8791", handler) // Envoy can now fetch tokens from http://localhost:8791/local/access_token ``` ``` -------------------------------- ### Handle Non-GCP Deployments Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Shows how to pass a nil MetadataFetcher to ConfigManager for non-GCP deployments where the metadata server is unavailable. ```Go // Non-GCP deployment: no metadata server var mf *metadata.MetadataFetcher = nil cm, err := configmanager.NewConfigManager(mf, opts) if err != nil { // Required: --service flag and --service_config_id or service JSON file } ``` -------------------------------- ### Create ServiceConfigFetcher Instance Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/serviceconfig.md Initializes a new ServiceConfigFetcher. Requires an HTTP client, Service Management API endpoint URL, the service name, and a function to obtain OAuth2 access tokens. ```go import ( "net/http" "github.com/GoogleCloudPlatform/esp-v2/src/go/serviceconfig" "github.com/GoogleCloudPlatform/esp-v2/src/go/metadata" ) // Create HTTP client with TLS opts := options.DefaultConfigGeneratorOptions() client := &http.Client{ Timeout: opts.HttpRequestTimeout, Transport: &http.Transport{ // TLS configuration }, } // Create fetcher accessToken := func() (string, time.Duration, error) { return mf.FetchAccessToken() // From metadata or token generator } fetcher := serviceconfig.NewServiceConfigFetcher( client, "https://servicemanagement.googleapis.com", "my-api.endpoints.example.com", accessToken, ) ``` -------------------------------- ### Build Docker Images Source: https://github.com/googlecloudplatform/esp-v2/blob/master/prow/README.md Builds local binaries, Docker images using Google Cloud Build, and pushes them to Google Container Registry. Run from the repository root. ```shell ./prow/gcpproxy-build.sh ``` -------------------------------- ### Error Handling: Token Generation Failure Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tokengenerator.md Illustrates error handling for failures during token generation when exchanging service account credentials. ```go token, _, err := tokengenerator.GenerateAccessTokenFromFile(keyFile) // err: error exchanging service account credentials for token ``` -------------------------------- ### CreateAdmin Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/bootstrap.md Creates an Envoy Admin structure for bootstrap configuration. It takes common options and returns a configured Admin protobuf or an empty Admin if the port is 0. ```APIDOC ## CreateAdmin ### Description Creates an Envoy Admin structure for bootstrap configuration. ### Method ```go func CreateAdmin(opts options.CommonOptions) *bootstrappb.Admin ``` ### Parameters #### Path Parameters - **opts** (options.CommonOptions) - Required - Common options with AdminAddress and AdminPort ### Returns Configured Admin protobuf, or empty Admin if port is 0. **Admin Fields (if port > 0):** - **Address:** SocketAddress with AdminAddress and AdminPort - **Protocol:** TCP socket **Returns Empty Admin If:** `AdminPort == 0` (disables admin interface) ### Request Example ```go // Enable admin interface opts := options.DefaultCommonOptions() opts.AdminPort = 8001 opts.AdminAddress = "127.0.0.1" admin := bootstrap.CreateAdmin(opts) // admin.Address.SocketAddress.Address = "127.0.0.1" // admin.Address.SocketAddress.PortValue = 8001 // Disable admin interface opts.AdminPort = 0 admin := bootstrap.CreateAdmin(opts) // Returns empty Admin struct ``` **Security Note:** Admin interface is unauthenticated and grants full control. Do not expose publicly. Recommended for development only. ``` -------------------------------- ### Deploy ESPv2 with Local Backend Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/configuration.md This command deploys ESPv2 as a sidecar with a local backend. It specifies the service configuration path, backend address, and listener details. ```bash espv2 \ --service_json_path=/etc/esp/service_config.json \ --backend_address=http://localhost:8081 \ --listener_address=127.0.0.1 \ --listener_port=8080 \ --admin_port=0 ``` -------------------------------- ### Development Tracing Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tracing.md Go code snippet for configuring tracing with all traces enabled for development. Sets project ID, sampling rate to 1.0, and enables verbose annotations. ```go opts := options.DefaultCommonOptions() opts.TracingOptions.ProjectId = "my-dev-project" opts.TracingOptions.SamplingRate = 1.0 // Trace all opts.TracingOptions.DisableTracing = false opts.TracingOptions.EnableVerboseAnnotations = true tracingConfig, _ := tracing.CreateTracing(opts.TracingOptions) ``` -------------------------------- ### Generate Go Protobuf Files Script Source: https://github.com/googlecloudplatform/esp-v2/blob/master/api/README.md Generates Go protobuf files, typically used after updating the configuration version to ensure compatibility. ```bash api/scripts/go_proto_gen.sh ``` -------------------------------- ### Fetch Full Service Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/examples/README.md Fetches the complete service configuration, including proto descriptors, directly from the Service Management API. This is crucial when updating the service configuration. ```shell curl --fail -o "service.json" -H "Authorization: Bearer $(gcloud auth print-access-token)" \ "https://servicemanagement.googleapis.com/v1/services/${SERVICE}/configs/${CONFIG_ID}?view=FULL" ``` -------------------------------- ### Fixed Rollout Strategy Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/OVERVIEW.md This strategy uses a single, unchanging configuration ID. The configuration is loaded once during initialization and remains static throughout the ESPv2 lifetime. ```text 1. Initialization: - Load config with specified ID - Generate Envoy configuration - Cache result 2. Runtime: - No polling or updates - Configuration static for lifetime ``` -------------------------------- ### Build ESPv2 Components Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Builds the Config Manager and Envoy components of ESPv2. ```bash # Config Manager make build # Envoy make build-envoy ``` -------------------------------- ### Token Agent Configuration Options Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tokengenerator.md Sets the token agent port and service account key path for ESPv2 configuration. ```go opts := options.DefaultConfigGeneratorOptions() opts.TokenAgentPort = 8791 // ESPv2 listens here opts.ServiceAccountKey = "/etc/esp/sa-key.json" // Envoy is configured to fetch from http://localhost:8791/local/access_token ``` -------------------------------- ### Error Handling: Invalid Service Account Key Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tokengenerator.md Demonstrates error handling when an invalid service account key file path is provided to GenerateAccessTokenFromFile. ```go token, _, err := tokengenerator.GenerateAccessTokenFromFile("/invalid/path") // err: open /invalid/path: no such file or directory ``` -------------------------------- ### FetchConfigId Method Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Fetches the service configuration ID from the metadata server. This ID represents the version of the service configuration being used. ```go func (mf *MetadataFetcher) FetchConfigId() (string, error) ``` -------------------------------- ### NewServiceInfoFromServiceConfig Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configinfo.md Parses a Google service configuration protobuf into a ServiceInfo object, performing various processing steps to initialize the configuration. ```APIDOC ## NewServiceInfoFromServiceConfig ### Description Parses a Google service configuration into a ServiceInfo object. ### Function Signature ```go func NewServiceInfoFromServiceConfig(serviceConfig *confpb.Service, opts options.ConfigGeneratorOptions) (*ServiceInfo, error) ``` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Parameter | Type | Required | Description | |---|---|---|---| | serviceConfig | *confpb.Service | Yes | Protobuf service configuration | | opts | options.ConfigGeneratorOptions | Yes | Runtime configuration options | ### Returns Initialized ServiceInfo or error. ### Processing Steps 1. Validates service configuration is non-empty and has at least one API 2. Parses backend address and creates local cluster 3. Processes endpoints (determines CORS support) 4. Processes all APIs and methods 5. Processes quota configuration 6. Processes backend rules (routing, authentication, retry policies) 7. Processes HTTP rules (gRPC-JSON transcoding) 8. Processes usage rules (quota metrics) 9. Processes authentication configuration 10. Processes gRPC HTTP rules (auto-generated transcoding) 11. Validates all types referenced in method signatures ### Errors - Service config is nil or empty - Service has no APIs - Invalid backend address format - Invalid HTTP rule URI templates - Invalid backend rule configuration - Circular or missing type dependencies - Conflicting route definitions ### Example ```go // Parse static service config opts := options.DefaultConfigGeneratorOptions() opts.BackendAddress = "grpc://backend.example.com:50051" opts.BackendDnsLookupFamily = "v4preferred" serviceConfig := &confpb.Service{ Name: "my-api.endpoints.example.com", Id: "2024-01-15", Apis: []*confpb.Api{ { Name: "myapi.v1.MyService", Methods: []*confpb.Method{ { Name: "GetResource", RequestTypeUrl: "type.googleapis.com/myapi.v1.GetResourceRequest", }, }, }, }, } serviceInfo, err := configinfo.NewServiceInfoFromServiceConfig(serviceConfig, opts) if err != nil { log.Fatal(err) } // Access parsed configuration for _, operation := range serviceInfo.Operations { methodInfo := serviceInfo.Methods[operation] // Generate filter configuration based on methodInfo } ``` ``` -------------------------------- ### Fetch Platform Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Internal function to detect the compute platform (GAE Flex, GKE, or GCE). ```Go func (mf *MetadataFetcher) fetchPlatform() string ``` -------------------------------- ### Apply Service Configuration Method Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configmanager.md The applyServiceConfig method parses the service configuration, generates a new Envoy snapshot, and caches it. It validates the configuration, fetches GCP attributes, and creates necessary Envoy resources. ```go func (m *ConfigManager) applyServiceConfig(serviceConfig *confpb.Service) error ``` -------------------------------- ### Production Tracing Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/tracing.md Go code snippet for configuring tracing with sampled traces for production. Sets project ID and a sampling rate of 0.001. ```go opts := options.DefaultCommonOptions() opts.TracingOptions.ProjectId = "my-prod-project" opts.TracingOptions.SamplingRate = 0.001 // 1 in 1000 opts.TracingOptions.EnableVerboseAnnotations = false tracingConfig, _ := tracing.CreateTracing(opts.TracingOptions) ``` -------------------------------- ### FetchRolloutStrategy Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/metadata.md Fetches the rollout strategy from the metadata server. ```APIDOC ## FetchRolloutStrategy ### Description Fetches the rollout strategy from the metadata server. ### Method *MetadataFetcher.FetchRolloutStrategy ### Endpoint `/computeMetadata/v1/instance/attributes/endpoints-rollout-strategy` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None explicitly provided in source. ### Response #### Success Response (200) - **string** - Rollout strategy ("fixed" or "managed") #### Response Example None explicitly provided in source. ``` -------------------------------- ### NewConfigManager Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configmanager.md Creates a new ConfigManager instance and initializes it with a service configuration. It handles loading configurations from files or GCP metadata, and setting up periodic polling for updates in managed rollout strategies. ```APIDOC ## NewConfigManager ### Description Creates a new ConfigManager instance and initializes it with a service configuration. It handles loading configurations from files or GCP metadata, and setting up periodic polling for updates in managed rollout strategies. ### Function Signature ```go func NewConfigManager(mf *metadata.MetadataFetcher, opts options.ConfigGeneratorOptions) (*ConfigManager, error) ``` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **mf** (*metadata.MetadataFetcher) - Optional - Metadata fetcher for GCP deployments; nil for non-GCP. - **opts** (options.ConfigGeneratorOptions) - Required - Configuration options controlling Envoy generation. ### Returns - **(*ConfigManager, error)** - Initialized ConfigManager or error. ### Errors - Fails if service name cannot be determined (required) - Fails if service config ID cannot be determined (required) - Fails if initial service configuration fetch fails - Fails if service configuration is invalid ### Initialization Steps: 1. If `--service_json_path` flag is set, loads static service configuration from file and disables all cloud-based fetching 2. If `--check_metadata` is enabled, fetches service name and rollout strategy from GCP metadata 3. For fixed rollout strategy: fetches single config ID and keeps it constant 4. For managed rollout strategy: fetches latest config ID from rollouts and starts periodic polling for updates 5. Initializes Envoy cache with first snapshot ### Example: ```go // GCP deployment with managed rollout opts := options.DefaultConfigGeneratorOptions() opts.ServiceManagementURL = "https://servicemanagement.googleapis.com" opts.ServiceControlURL = "https://servicecontrol.googleapis.com" mf := metadata.NewMetadataFetcher(opts.CommonOptions) cm, err := configmanager.NewConfigManager(mf, opts) if err != nil { log.Fatal(err) } // cm.Cache() returns the snapshot cache for use with xDS server ``` ``` -------------------------------- ### Parse Service Configuration Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configinfo.md Parses a Google service configuration into a ServiceInfo object. Use this to initialize ServiceInfo from a protobuf service configuration and runtime options. Ensure backend address and DNS lookup family are configured. ```go func NewServiceInfoFromServiceConfig(serviceConfig *confpb.Service, opts options.ConfigGeneratorOptions) (*ServiceInfo, error) ``` ```go // Parse static service config opts := options.DefaultConfigGeneratorOptions() opts.BackendAddress = "grpc://backend.example.com:50051" opts.BackendDnsLookupFamily = "v4preferred" serviceConfig := &confpb.Service{ Name: "my-api.endpoints.example.com", Id: "2024-01-15", Apis: []*confpb.Api{ { Name: "myapi.v1.MyService", Methods: []*confpb.Method{ { Name: "GetResource", RequestTypeUrl: "type.googleapis.com/myapi.v1.GetResourceRequest", }, }, }, }, } serviceInfo, err := configinfo.NewServiceInfoFromServiceConfig(serviceConfig, opts) if err != nil { log.Fatal(err) } // Access parsed configuration for _, operation := range serviceInfo.Operations { methodInfo := serviceInfo.Methods[operation] // Generate filter configuration based on methodInfo } ``` -------------------------------- ### ESPv2 Token Generation Flow (Non-GCP) Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/OVERVIEW.md Explains the process of generating OAuth2 access tokens from a service account key file for non-GCP environments. ```text Service Account Key File ↓ tokengenerator.GenerateAccessTokenFromFile() ├─ Google OAuth2 token exchange ├─ Token caching (60s refresh margin) └─ Returns: (token, expiry, error) ↓ Available to: configmanager, serviceconfig fetcher, metadata-based token source ``` -------------------------------- ### GetServiceConfig Method Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configinfo.md Retrieves the underlying Google service configuration protobuf from a ServiceInfo object. This method returns the original configuration used to initialize the ServiceInfo. ```go func (s *ServiceInfo) ServiceConfig() *confpb.Service ``` -------------------------------- ### Clone ESPv2 Repository Source: https://github.com/googlecloudplatform/esp-v2/blob/master/DEVELOPER.md Use this command to clone the ESPv2 repository to your local machine. ```bash git clone https://github.com/GoogleCloudPlatform/esp-v2.git ``` -------------------------------- ### Configure mTLS with Backend Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/options.md Enables mutual TLS (mTLS) for backend communication by specifying client certificate, root CA certificate, and cipher suites. ```go opts := options.DefaultConfigGeneratorOptions() opts.SslBackendClientCertPath = "/etc/certs/client.crt" opts.SslBackendClientRootCertsPath = "/etc/certs/ca.crt" opts.SslBackendClientCipherSuites = "TLS_AES_256_GCM_SHA384" ``` -------------------------------- ### Fetch and Apply Service Configuration Method Source: https://github.com/googlecloudplatform/esp-v2/blob/master/_autodocs/api-reference/configmanager.md The fetchAndApplyServiceConfig method fetches a service configuration by ID and applies it if it has changed. It is called by the rollout change detector when using the managed strategy and returns early if the config ID hasn't changed. ```go func (m *ConfigManager) fetchAndApplyServiceConfig(latestConfigId string) error ```