### Install vals with Nix / NixOS Source: https://github.com/helmfile/vals/blob/main/README.md Install vals on Nix or NixOS using the nix profile command. ```bash nix profile install nixpkgs#vals ``` -------------------------------- ### Install vals with Scoop (Windows) Source: https://github.com/helmfile/vals/blob/main/README.md Install vals on Windows using the Scoop package manager. ```bash scoop install vals ``` -------------------------------- ### Exec Provider Command Examples Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/providers.md Examples of using the Exec provider to run arbitrary commands and use their output. Supports arguments, timeouts, and environment variables. ```plaintext ref+exec://echo/hello ref+exec://my-script.sh?args=--key,value ref+exec://my-tool?timeout=10&env_API_TOKEN=xyz ``` -------------------------------- ### Install vals with Homebrew Source: https://github.com/helmfile/vals/blob/main/README.md Use this command to install vals on macOS and Linux systems using Homebrew. ```bash brew install vals ``` -------------------------------- ### OpenBao Example: Kubernetes Authentication Source: https://github.com/helmfile/vals/blob/main/README.md Demonstrates using the Kubernetes authentication method with a specified role ID. ```plaintext ref+openbao://mykv/foo?role_id=my-kube-role#/bar ``` -------------------------------- ### Vals Documentation Structure Example Source: https://github.com/helmfile/vals/blob/main/_autodocs/INDEX.md Illustrates the directory structure of the generated Vals documentation, showing the location of INDEX.md and other reference files. ```text output/ ├── INDEX.md ← You are here ├── README.md Entry point, overview, concepts ├── types.md Type definitions ├── configuration.md Options and environment variables ├── errors.md Error reference └── api-reference/ ├── runtime.md Core Runtime API ├── expansion.md Template expansion ├── config.md Configuration access ├── io.md File I/O └── providers.md Provider backends ``` -------------------------------- ### Install vals with Alpine Linux Edge Source: https://github.com/helmfile/vals/blob/main/README.md Install vals on Alpine Linux Edge using the apk package manager. ```bash apk add vals ``` -------------------------------- ### Vault and OpenBao URI Examples Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/providers.md Examples of URIs for accessing secrets from Vault and OpenBao, demonstrating different authentication methods like token, approle, kubernetes, and userpass. ```plaintext ref+vault://secret/db?address=https://vault.example.com:8200#/password ref+vault://secret/db?auth_method=kubernetes&role_id=my-role#/password ref+vault://secret/db?auth_method=approle&role_id=role&secret_id=secret#/password ``` -------------------------------- ### OpenBao Example: Userpass Authentication with File Source: https://github.com/helmfile/vals/blob/main/README.md Uses the userpass authentication method, retrieving the password from a file. ```plaintext ref+openbao://mykv/foo?auth_method=userpass&username=some-user&password_file=PATH/TO/FILE#/bar ``` -------------------------------- ### EnvSubst Provider Example Source: https://github.com/helmfile/vals/blob/main/README.md Perform environment variable substitution. Loads values from specified environment variables. ```bash ref+envsubst://$VAR1 ``` -------------------------------- ### Vault Backend Configuration Source: https://github.com/helmfile/vals/blob/main/README.md Examples of how to configure and authenticate with the Vault backend. ```APIDOC ## Vault Backend Configuration This section details the various ways to configure and authenticate with the Vault backend for vals. ### Vault Token Authentication ``` ref+vault://PATH/TO/KVBACKEND[?address=VAULT_ADDR:PORT&token_file=PATH/TO/FILE&token_env=VAULT_TOKEN&namespace=VAULT_NAMESPACE]#/fieldkey ``` ### Vault AppRole Authentication ``` ref+vault://PATH/TO/KVBACKEND[?address=VAULT_ADDR:PORT&auth_method=approle&role_id=ce5e571a-f7d4-4c73-93dd-fd6922119839&secret_id=5c9194b9-585e-4539-a865-f45604bd6f56]#/fieldkey ``` ### Vault Kubernetes Authentication ``` ref+vault://PATH/TO/KVBACKEND[?address=VAULT_ADDR:PORT&auth_method=kubernetes&role_id=K8S-ROLE ``` ### Vault Userpass Authentication ``` ref+vault://PATH/TO/KVBACKEND[?address=VAULT_ADDR:PORT&auth_method=userpass&username=some-user&password_file=PATH/TO/FILE&password_env=VAULT_PASSWORD]#/fieldkey ``` ### Notes on Parameters * `address`: Defaults to the value of the `VAULT_ADDR` environment variable. * `namespace`: Defaults to the value of the `VAULT_NAMESPACE` environment variable. * `auth_method`: Defaults to `token` and can also be set to the value of the `VAULT_AUTH_METHOD` environment variable. * `role_id`: Defaults to the value of the `VAULT_ROLE_ID` environment variable. * `secret_id`: Defaults to the value of the `VAULT_SECRET_ID` environment variable. * `version`: Specifies the exact version of the secret to retrieve. Useful for accessing previous versions of a secret. ``` -------------------------------- ### Install vals with Arch Linux Package Manager Source: https://github.com/helmfile/vals/blob/main/README.md Install vals on Arch Linux using the pacman package manager. ```bash sudo pacman -S vals ``` -------------------------------- ### Core API Functions Source: https://github.com/helmfile/vals/blob/main/_autodocs/SUMMARY.txt Documentation for the core API functions including Runtime, Eval, Get, and Exec. Each function is fully documented with its signature, parameters, return type, error conditions, and code examples. ```APIDOC ## Core API Reference This section details the core API functions available in the helmfile/vals project. ### Runtime **Description**: Provides runtime capabilities for vals operations. **Signature**: `Runtime()` **Parameters**: None **Return Type**: `*Runtime` **Error Conditions**: None explicitly documented. **Code Example**: ```go // Example usage of Runtime rt := vals.Runtime() ``` ### Eval **Description**: Evaluates expressions or values within the vals context. **Signature**: `Eval(ctx context.Context, v interface{}) (interface{}, error)` **Parameters**: - `ctx` (context.Context) - Required - The context for the operation. - `v` (interface{}) - Required - The value or expression to evaluate. **Return Type**: `interface{}, error` **Error Conditions**: Errors during evaluation. **Code Example**: ```go // Example usage of Eval result, err := vals.Eval(ctx, "${VAR:-default}") ``` ### Get **Description**: Retrieves a value from the vals configuration. **Signature**: `Get(key string) (interface{}, error)` **Parameters**: - `key` (string) - Required - The key of the value to retrieve. **Return Type**: `interface{}, error` **Error Conditions**: Key not found or other retrieval errors. **Code Example**: ```go // Example usage of Get value, err := vals.Get("database.host") ``` ### Exec **Description**: Executes a command or operation within the vals environment. **Signature**: `Exec(ctx context.Context, cmd string, args ...string) error` **Parameters**: - `ctx` (context.Context) - Required - The context for the operation. - `cmd` (string) - Required - The command to execute. - `args` (...string) - Optional - Arguments for the command. **Return Type**: `error` **Error Conditions**: Errors during command execution. **Code Example**: ```go // Example usage of Exec err := vals.Exec(ctx, "ls", "-l") ``` ``` -------------------------------- ### Install vals with MacPorts Source: https://github.com/helmfile/vals/blob/main/README.md Install vals on macOS using the MacPorts package manager. ```bash sudo port install vals ``` -------------------------------- ### AWS KMS Example URIs Source: https://github.com/helmfile/vals/blob/main/README.md Illustrative examples of AWS KMS reference URIs, demonstrating different parameter combinations. ```plaintext ref+awskms://AQICAHhy_i8hQoGLOE46PVJyinH...WwHKT0i3H0znHRHwfyC7AGZ8ek= ``` ```plaintext ref+awskms://AQICAHhy...nHRHwfyC7AGZ8ek=#/foo/bar ``` ```plaintext ref+awskms://AQICAHhy...WwHKT0i3AGZ8ek=?context=%7B%22foo%22%3A%22bar%22%2C%22hello%22%2C%22world%22%7D ``` ```plaintext ref+awskms://AQICAVJyinH...WwHKT0iC7AGZ8ek=?alg=RSAES_OAEP_SHA1&key=alias%2FExampleAlias ``` ```plaintext ref+awskms://AQICA...fyC7AGZ8ek=?alg=RSAES_OAEP_SHA256&key=arn%3Aaws%3Akms%3Aus-east-2%3A111122223333%3Akey%2F1234abcd-12ab-34cd-56ef-1234567890ab&context=%7B%22foo%22%3A%22bar%22%2C%22hello%22%2C%22world%22%7D ``` -------------------------------- ### Terraform Output Lookup Example Source: https://github.com/helmfile/vals/blob/main/README.md Example of retrieving a Terraform output value using tfstate-lookup and its equivalent with vals. ```shell $ tfstate-lookup -s ./terraform.tfstate output.mystack_apply ``` ```shell $ echo 'foo: ref+tfstate://terraform.tfstate/output.mystack_apply' | vals eval -f - ``` -------------------------------- ### AWS S3 Examples Source: https://github.com/helmfile/vals/blob/main/README.md Demonstrates various ways to reference S3 objects, including basic references, path extraction for JSON/YAML content, and specifying region or profile. ```plaintext ref+s3://mybucket/mykey ``` ```plaintext ref+s3://mybucket/myjsonobj#/foo/bar ``` ```plaintext ref+s3://mybucket/myyamlobj#/foo/bar ``` ```plaintext ref+s3://mybucket/mykey?region=us-west-2 ``` ```plaintext ref+s3://mybucket/mykey?profile=prod ``` -------------------------------- ### OpenBao KV Backend Reference Source: https://github.com/helmfile/vals/blob/main/README.md Examples of OpenBao references for Key-Value (KV) backends, demonstrating different authentication methods and parameters. ```plaintext ref+openbao://PATH/TO/KVBACKEND[?address=BAO_ADDR:PORT&token_file=PATH/TO/FILE&token_env=BAO_TOKEN&namespace=BAO_NAMESPACE]#/fieldkey ``` ```plaintext ref+openbao://PATH/TO/KVBACKEND[?address=BAO_ADDR:PORT&auth_method=approle&role_id=ce5e571a-f7d4-4c73-93dd-fd6922119839&secret_id=5c9194b9-585e-4539-a865-f45604bd6f56]#/fieldkey ``` ```plaintext ref+openbao://PATH/TO/KVBACKEND[?address=BAO_ADDR:PORT&auth_method=kubernetes&role_id=K8S-ROLE]#/fieldkey ``` ```plaintext ref+openbao://PATH/TO/KVBACKEND[?address=BAO_ADDR:PORT&auth_method=userpass&username=some-user&password_file=PATH/TO/FILE&password_env=BAO_PASSWORD]#/fieldkey ``` -------------------------------- ### Terraform State Lookup Example Source: https://github.com/helmfile/vals/blob/main/README.md Example of using tfstate-lookup to retrieve a resource attribute and then using vals to evaluate the same reference. ```shell $ tfstate-lookup -s ./terraform.tfstate module.vpc.aws_vpc.this[0].arn arn:aws:ec2:us-east-2:ACCOUNT_ID:vpc/vpc-0cb48a12e4df7ad4c ``` ```shell $ echo 'foo: ref+tfstate://terraform.tfstate/module.vpc.aws_vpc.this[0].arn' | vals eval -f - ``` -------------------------------- ### OpenBao Example: Token File Authentication Source: https://github.com/helmfile/vals/blob/main/README.md Reads a secret using a token stored in a specified file for authentication. ```plaintext ref+openbao://mykv/foo?token_file=~/.bao_token_bao1&address=https://bao.example.com:8200#/bar ``` -------------------------------- ### Google Cloud Storage Example URIs Source: https://github.com/helmfile/vals/blob/main/README.md Examples of Google Cloud Storage reference URIs for accessing objects. ```plaintext ref+gcs://mybucket/mykey ``` ```plaintext ref+gcs://mybucket/myjsonobj#/foo/bar ``` ```plaintext ref+gcs://mybucket/myyamlobj#/foo/bar ``` ```plaintext ref+gcs://mybucket/mykey?generation=1639567476974625 ``` -------------------------------- ### OpenBao Example: Userpass Authentication with Environment Variable Source: https://github.com/helmfile/vals/blob/main/README.md Uses the userpass authentication method, retrieving the password from an environment variable. ```plaintext ref+openbao://mykv/foo?auth_method=userpass&username=some-user&password_env=BAO_PASSWORD#/bar ``` -------------------------------- ### Terraform S3 State Lookup Example Source: https://github.com/helmfile/vals/blob/main/README.md Example of using tfstate-lookup to retrieve a resource attribute from S3 and its equivalent with vals. ```shell $ tfstate-lookup -s s3://bucket-with-terraform-state/terraform.tfstate module.vpc.aws_vpc.this[0].arn arn:aws:ec2:us-east-2:ACCOUNT_ID:vpc/vpc-0cb48a12e4df7ad4c ``` ```shell $ echo 'foo: ref+tfstates3://bucket-with-terraform-state/terraform.tfstate/module.vpc.aws_vpc.this[0].arn' | vals eval -f - ``` -------------------------------- ### Get (Package Level) Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/runtime.md Convenience function that creates a new Runtime and evaluates a string in one call. ```APIDOC ## Get (Package Level) ### Description Convenience function that creates a new Runtime and evaluates a string in one call. ### Method ```go func Get(code string, opts Options) (string, error) ``` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | code | string | Yes | The string to evaluate | | opts | Options | No | Evaluation options | ### Returns - `string` — evaluated string - `error` — evaluation error ### Example ```go result, err := vals.Get("API_KEY=ref+awssecrets://myapp/key", vals.Options{}) if err != nil { panic(err) } ``` ``` -------------------------------- ### AWS SSM Parameter Store Examples Source: https://github.com/helmfile/vals/blob/main/README.md Illustrates various ways to reference SSM parameters, including direct values, nested JSON/YAML, and recursive retrieval. Demonstrates successful and failing reference scenarios. ```plaintext ref+awsssm://foo/bar ``` ```plaintext ref+awsssm://foo#/bar ``` ```plaintext ref+awsssm://foo/bar/a ``` ```plaintext ref+awsssm://foo/bar?#/a ``` ```plaintext ref+awsssm://foo?recursive=true#/bar/a ``` ```plaintext ref+awsssm://foo/bar?mode=singleparam#/BAR ``` ```plaintext ref+awsssm://foo/bar#/BAR ``` ```plaintext ref+awsssm://foo?recursive=true#/bar ``` -------------------------------- ### AWS Secrets Manager Examples Source: https://github.com/helmfile/vals/blob/main/README.md Provides examples of referencing secrets, including basic references, path extraction, and region specification. Cross-account referencing is also demonstrated. ```plaintext ref+awssecrets://myteam/mykey ``` ```plaintext ref+awssecrets://myteam/mydoc#/foo/bar ``` ```plaintext ref+awssecrets://myteam/mykey?region=us-west-2 ``` ```plaintext ref+awssecrets://arn:aws:secretsmanager:::secret:/myteam/mydoc/?region=ap-southeast-2#/secret/key ``` -------------------------------- ### Exec Provider Examples Source: https://github.com/helmfile/vals/blob/main/README.md Execute arbitrary CLI commands and use their stdout as secret values. Supports custom arguments, timeouts, whitespace trimming, and environment variables. ```bash ref+exec://echo/hello ``` ```bash ref+exec://bw/get/password/item-id ``` ```bash ref+exec://my-script.sh?args=--key,my-secret ``` ```bash ref+exec://vault-helper.sh?args=read,secret/db#/password ``` ```bash ref+exec:///usr/local/bin/my-tool?args=fetch,key1 ``` ```bash ref+exec://my-tool?env_API_TOKEN=xyz&timeout=10 ``` -------------------------------- ### Access configuration values Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Demonstrates how to access configuration values after they have been processed by Vals. Provides methods to get string values and check for existence. ```go conf := config.Map(data) value := conf.String("key", "nested") if conf.Exists("key") { /* ... */ } ``` -------------------------------- ### Expand InString Method Example Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/expansion.md Demonstrates how to use the `InString` method to expand all matched references within a string. References are replaced with their resolved string values. Supports nested references and multiple references in a single string. ```go expand := expansion.ExpandRegexMatch{ Target: expansion.DefaultRefRegexp, Lookup: func(key string) (interface{}, error) { if key == "vault://secret/db#/password" { return "p@ssw0rd", nil } return nil, fmt.Errorf("unknown ref") }, } result, err := expand.InString("Database password: ref+vault://secret/db#/password") // result: "Database password: p@ssw0rd" ``` -------------------------------- ### GCP Secrets Manager Example URIs Source: https://github.com/helmfile/vals/blob/main/README.md Examples of GCP Secrets Manager reference URIs, including with and without project specification. ```plaintext ref+gcpsecrets://myproject/mysecret ``` ```plaintext ref+gcpsecrets://myproject/mysecret?version=3 ``` ```plaintext ref+gcpsecrets://myproject/mysecret?version=3#/yaml_or_json_key/in/secret ``` ```plaintext ref+gcpsecrets://mysecret ``` ```plaintext ref+gcpsecrets://mysecret?version=3 ``` -------------------------------- ### OpenBao Example: Default Token Authentication Source: https://github.com/helmfile/vals/blob/main/README.md Reads a secret using default token authentication, where the token is sourced from the BAO_TOKEN environment variable or ~/.bao-token. ```plaintext ref+openbao://mykv/foo?address=https://bao.example.com:8200#/bar ``` -------------------------------- ### Google Cloud Providers URI Examples Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/providers.md Examples of URIs for Google Cloud providers, including GCP Secrets, GCS, and GCP KMS. Assumes credentials are set via environment variables. ```plaintext ref+gcpsecrets://PROJECT/SECRET ref+gcs://BUCKET/OBJECT ref+gkms://CIPHERTEXT?project=...&location=...&keyring=...&crypto_key=... ``` -------------------------------- ### OpenBao Example: Custom Token Environment Variable Source: https://github.com/helmfile/vals/blob/main/README.md Reads a secret from a specific namespace using a custom token environment variable for authentication. ```plaintext ref+openbao://mykv/foo?token_env=BAO_TOKEN_BAO1&namespace=ns1&address=https://bao.example.com:8200#/bar ``` -------------------------------- ### GitLab Secrets Provider Examples Source: https://github.com/helmfile/vals/blob/main/README.md Fetch CI/CD variables from GitLab projects or groups. Requires GITLAB_TOKEN environment variable. Supports custom server URLs and SSL verification options. ```bash ref+gitlab://gitlab.com/11111/password ``` ```bash ref+gitlab://gitlab.com/projects/11111/password ``` ```bash ref+gitlab://gitlab.com/groups/2222/password ``` ```bash ref+gitlab://my-gitlab.org/11111/password?ssl_verify=true&scheme=https ``` -------------------------------- ### Encrypting with GCP KMS CLI Source: https://github.com/helmfile/vals/blob/main/README.md Example of how to encrypt plaintext using the GCP CLI and KMS, outputting URL-safe base64 ciphertext. ```bash echo test | gcloud kms encrypt \ --project myproject \ --location global \ --keyring mykeyring \ --key mykey \ --plaintext-file - \ --ciphertext-file - \ | base64 -w0 \ | tr '/+' '_-' ``` -------------------------------- ### Example of Secret Value Interpolation Source: https://github.com/helmfile/vals/blob/main/README.md Demonstrates how to use multiple secret references for string interpolation. Each `ref+` expression is evaluated sequentially. ```console foo ref+SECRET1+ ref+SECRET2+ bar ``` -------------------------------- ### Encrypting with AWS KMS CLI Source: https://github.com/helmfile/vals/blob/main/README.md Example of how to encrypt plaintext using the AWS CLI and KMS, outputting URL-safe base64 ciphertext. ```bash aws kms encrypt \ --key-id alias/example \ --plaintext $(echo -n "hello, world" | base64 -w0) \ --query CiphertextBlob \ --output text | \ tr '/+' '_-' ``` -------------------------------- ### Configure AWS SDK Logging Level Source: https://github.com/helmfile/vals/blob/main/_autodocs/configuration.md Control AWS SDK request and response logging. This example enables logging for both requests and responses. The AWS_SDK_GO_LOG_LEVEL environment variable can override this setting. ```go runtime, err := vals.New(vals.Options{ AWSLogLevel: "request,response", }) ``` ```go os.Setenv("AWS_SDK_GO_LOG_LEVEL", "request,response") ``` -------------------------------- ### Configure Log Output Source: https://github.com/helmfile/vals/blob/main/_autodocs/configuration.md Specify where debug logs are written. Defaults to stderr if nil. This example directs logs to a file named 'vals.log'. ```go logFile, _ := os.Create("vals.log") runtime, err := vals.New(vals.Options{ LogOutput: logFile, }) ``` -------------------------------- ### Handle SOPS Provider Errors Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md Demonstrates error handling for SOPS-encrypted files accessed through Vals. The example covers common issues like decryption failures due to incorrect keys and file not found errors, offering specific recovery advice. ```go result, err := runtime.Get("ref+sops://secrets.enc.yaml#/database/password") if err != nil { if strings.Contains(err.Error(), "decrypt") { log.Printf("SOPS decryption failed - check KMS key access") } else if strings.Contains(err.Error(), "file") { log.Printf("SOPS file not found - check path") } } ``` -------------------------------- ### Azure Key Vault Examples Source: https://github.com/helmfile/vals/blob/main/README.md Retrieve secrets from Azure Key Vault. Specify vault name, secret name, and optionally a secret version. Authentication is handled by the azidentity Go module. ```bash ref+azurekeyvault://my-vault/secret-a ``` ```bash ref+azurekeyvault://my-vault/secret-a/ba4f196b15f644cd9e949896a21bab0d ``` ```bash ref+azurekeyvault://gov-cloud-test.vault.usgovcloudapi.net/secret-b ``` -------------------------------- ### Handle Vault Provider Errors Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md Illustrates how to recover from errors when accessing HashiCorp Vault via Vals, including authentication failures, permission denied, unsupported paths, and invalid requests. The example checks for specific error substrings to provide targeted feedback. ```go result, err := runtime.Get("ref+vault://secret/db#/password") if err != nil { if strings.Contains(err.Error(), "auth") { log.Printf("Vault authentication failed - check VAULT_TOKEN") } else if strings.Contains(err.Error(), "permission") { log.Printf("Vault policy denies access - check policy") } else if strings.Contains(err.Error(), "unsupported") { log.Printf("Check if using KV v1 or v2") } } ``` -------------------------------- ### Initialize Runtime Source: https://github.com/helmfile/vals/blob/main/_autodocs/INDEX.md Initializes a Runtime instance with provided options. This is a method on the Runtime type. ```go (*Runtime).New(opts Options) error ``` -------------------------------- ### Vals CLI Help Source: https://github.com/helmfile/vals/blob/main/README.md Display the available commands and usage information for the vals CLI. ```bash vals [command] Available Commands: eval Evaluate a JSON/YAML document and replace any template expressions in it and prints the result exec Populates the environment variables and executes the command env Renders environment variables to be consumed by eval or a tool like direnv get Evaluate a string value passed as the first argument and replace any expressiosn in it and prints the result ksdecode Decode YAML document(s) by converting Secret resources' "data" to "stringData" for use with "vals eval" version Print vals version Use "vals [command] --help" for more information about a comman ``` -------------------------------- ### Create New Runtime Instance Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/runtime.md Creates a new Runtime instance with specified options, initializing internal caches. Use this for managing runtime configurations and evaluations. ```go import "github.com/helmfile/vals" runtime, err := vals.New(vals.Options{ CacheSize: 256, }) if err != nil { panic(err) } defer runtime.Close() // if applicable result, err := runtime.Eval(map[string]interface{}{ "db_password": "ref+vault://secret/db#/password", }) if err != nil { panic(err) } ``` -------------------------------- ### Create Runtime with Options Source: https://github.com/helmfile/vals/blob/main/_autodocs/types.md Instantiates a new vals Runtime with custom options. Configure cache size and AWS logging level. ```go opts := vals.Options{ CacheSize: 256, AWSLogLevel: "request,response", ExcludeSecret: false, } runtime, err := vals.New(opts) ``` -------------------------------- ### Terraform GCS State Lookup Example Source: https://github.com/helmfile/vals/blob/main/README.md Example of using tfstate-lookup to retrieve a resource attribute from GCS and its equivalent with vals. ```shell $ tfstate-lookup -s gs://bucket-with-terraform-state/terraform.tfstate google_compute_disk.instance.source_image_id 5449927740744213880 ``` ```shell $ echo 'foo: ref+tfstategs://bucket-with-terraform-state/terraform.tfstate/google_compute_disk.instance.source_image_id' | vals eval -f - ``` -------------------------------- ### Basic Usage Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Demonstrates how to create a vals runtime and evaluate a map containing secret references. ```APIDOC ## Basic Usage ```go import "github.com/helmfile/vals" // Create a runtime runtime, err := vals.New(vals.Options{ CacheSize: 256, }) if err != nil { panic(err) } // Evaluate a template result, err := runtime.Eval(map[string]interface{}{ "database_password": "ref+vault://secret/db#/password", "api_key": "ref+awssecrets://myapp/key", }) if err != nil { panic(err) } // result["database_password"] contains the actual password // result["api_key"] contains the actual API key ``` ``` -------------------------------- ### Runtime.New Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/runtime.md Creates a new Runtime instance with the specified options. Initializes internal LRU caches for both documents and strings. ```APIDOC ## New ### Description Creates a new Runtime instance with the specified options. Initializes internal LRU caches for both documents and strings. ### Method ```go func New(opts Options) (*Runtime, error) ``` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | opts | Options | No | Empty Options | Configuration for the runtime | ### Returns - `*Runtime` — new runtime instance - `error` — initialization error (e.g., cache creation failure) ### Example ```go import "github.com/helmfile/vals" runtime, err := vals.New(vals.Options{ CacheSize: 256, }) if err != nil { panic(err) } defer runtime.Close() // if applicable result, err := runtime.Eval(map[string]interface{}{ "db_password": "ref+vault://secret/db#/password", }) if err != nil { panic(err) } ``` ``` -------------------------------- ### Define Options Structure Source: https://github.com/helmfile/vals/blob/main/_autodocs/types.md Defines the configuration options for a Runtime instance. Use this to customize logging, caching, and secret expansion behavior. ```go type Options struct { LogOutput io.Writer AWSLogLevel string CacheSize int ExcludeSecret bool FailOnMissingKeyInMap bool } ``` -------------------------------- ### AWS Providers URI Examples Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/providers.md Examples of URIs for AWS providers, showing region specification, profile selection, cross-account access, and accessing secrets from different AWS services. ```plaintext ref+awssecrets://myapp/dbpass?region=us-west-2 ref+s3://my-bucket/key.json?region=eu-west-1&profile=prod#/database/url ref+awsssm://config/app#/api_key ``` -------------------------------- ### Example of Maximum Nesting Depth Exceeded Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md Illustrates a scenario where nested references exceed the maximum allowed depth of 10 levels, leading to an error. It contrasts this with valid nesting examples. ```text // Too deep: ref+echo://ref+echo://ref+echo://... (10+ levels) // Error: maximum nesting depth (10) exceeded // Valid nesting: ref+echo://ref+envsubst://$VAR+/path (2 levels - OK) ref+s3://ref+secretsmanager://ref+vault://secret (3 levels - OK) ``` -------------------------------- ### Create Runtime and Evaluate Secrets Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Demonstrates how to create a vals runtime and evaluate a map containing secret references. The runtime manages provider initialization, credential management, and caching. Ensure proper error handling for runtime creation and evaluation. ```go import "github.com/helmfile/vals" // Create a runtime runtime, err := vals.New(vals.Options{ CacheSize: 256, }) if err != nil { panic(err) } // Evaluate a template result, err := runtime.Eval(map[string]interface{}{ "database_password": "ref+vault://secret/db#/password", "api_key": "ref+awssecrets://myapp/key", }) if err != nil { panic(err) } // result["database_password"] contains the actual password // result["api_key"] contains the actual API key ``` -------------------------------- ### Single Secret Retrieval Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Demonstrates how to retrieve a single secret using the `Get` method. ```APIDOC ## Single Secret Retrieval ```go runtime, _ := vals.New(vals.Options{}) password, err := runtime.Get("ref+vault://secret/db#/password") ``` ``` -------------------------------- ### Execute Command with ExecConfig Source: https://github.com/helmfile/vals/blob/main/_autodocs/types.md Executes a command using vals, resolving environment variables and streaming YAML content. Inherit environment variables and direct output to specified writers. ```go err := vals.Exec( map[string]interface{}{ "DB_PASSWORD": "ref+vault://secret/db#/password", }, []string{"kubectl", "apply", "-f", "-"}, vals.ExecConfig{ InheritEnv: true, StreamYAML: "manifests.yaml", Stdout: os.Stdout, Stderr: os.Stderr, }, ) ``` -------------------------------- ### End-to-End Vals Usage Pattern Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/io.md Demonstrates the typical workflow for processing YAML files using Vals, from reading inputs to evaluating and outputting results. Ensure necessary packages are imported. ```go import ( "fmt" "os" "github.com/helmfile/vals" ) func main() { // 1. Read YAML documents nodes, err := vals.Inputs("manifests.yaml") if err != nil { fmt.Fprintf(os.Stderr, "Read failed: %v\n", err) os.Exit(1) } // 2. Configure evaluation opts := vals.Options{ CacheSize: 256, AWSLogLevel: "off", } // 3. Evaluate all refs in the documents result, err := vals.EvalNodes(nodes, opts) if err != nil { fmt.Fprintf(os.Stderr, "Evaluation failed: %v\n", err) os.Exit(1) } // 4. Write result err = vals.Output(os.Stdout, "yaml", result) if err != nil { fmt.Fprintf(os.Stderr, "Output failed: %v\n", err) os.Exit(1) } } ``` -------------------------------- ### Get Value by Code Source: https://github.com/helmfile/vals/blob/main/_autodocs/INDEX.md Retrieves a string value based on a provided code string. Requires runtime options. ```go Get(code string, opts Options) (string, error) ``` -------------------------------- ### Process Files with vals Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Demonstrates processing YAML files using vals. It involves reading input files, evaluating expressions within them, and outputting the results. Ensure the input file 'manifests.yaml' exists and is correctly formatted. ```go nodes, _ := vals.Inputs("manifests.yaml") result, _ := vals.EvalNodes(nodes, vals.Options{}) vals.Output(os.Stdout, "yaml", result) ``` -------------------------------- ### Execute with Arguments Source: https://github.com/helmfile/vals/blob/main/_autodocs/INDEX.md Executes a template with specific arguments and optional execution configuration. Returns an error if execution fails. ```go Exec(template map[string]interface{}, args []string, config ...ExecConfig) error ``` -------------------------------- ### Handle Fragment Path Not Found Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md Shows how to manage errors when a specified fragment key does not exist within a retrieved map. It demonstrates setting `FailOnMissingKeyInMap` to `false` to allow missing keys and return nil instead of an error. ```go // Secret contains: {"database": {"host": "localhost"}} _, err := runtime.Get("ref+vault://secret/db#/username") // Key doesn't exist if err != nil { log.Printf("Key not found: %v", err) } // To allow missing keys (return nil): runtime, _ := vals.New(vals.Options{ FailOnMissingKeyInMap: false, }) // Returns nil instead of error ``` -------------------------------- ### Create New Runtime Source: https://github.com/helmfile/vals/blob/main/_autodocs/INDEX.md Creates a new vals Runtime instance with specified options. This is the entry point for most vals operations. ```go New(opts Options) (*Runtime, error) ``` -------------------------------- ### Retrieve Nested Config from MapConfig Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/config.md Use to get a nested configuration as a new StaticConfig. This is useful for accessing sub-sections of the configuration independently. ```go conf := config.MapConfig{ M: map[string]interface{}{ "provider": map[string]interface{}{ "name": "vault", "address": "https://vault.example.com:8200", }, }, } providerConf := conf.Config("provider") name := providerConf.String("name") // "vault" ``` -------------------------------- ### Handle Cache Creation Errors Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md Demonstrates how to handle potential errors during vals runtime initialization, specifically when the LRU cache fails to create, typically due to insufficient memory. It shows retrying with a smaller cache size. ```go runtime, err := vals.New(vals.Options{CacheSize: 1000000}) if err != nil { log.Printf("Failed to create runtime: %v", err) // Retry with smaller cache runtime, _ = vals.New(vals.Options{CacheSize: 256}) } ``` -------------------------------- ### Path Error Handling for Empty Paths Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/io.md Demonstrates error handling for empty path arguments passed to `vals.Inputs` and `vals.TextInput`. These functions require a valid file path. ```go nodes, err := vals.Inputs("") // Empty path // err: "Nothing to eval: No file specified" nodes, err := vals.TextInput("") // Empty path // err: "Nothing to read: No file specified" ``` -------------------------------- ### Get Value by Code on Runtime Source: https://github.com/helmfile/vals/blob/main/_autodocs/INDEX.md Retrieves a string value using a code string with an existing Runtime instance. Returns the value or an error. ```go (*Runtime).Get(code string) (string, error) ``` -------------------------------- ### Command Execution with Secrets Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Illustrates how to execute a command, injecting secrets as environment variables. ```APIDOC ## Command Execution with Secrets ```go vals.Exec( map[string]interface{}{ "DB_PASSWORD": "ref+vault://secret/db#/password", }, []string{"kubectl", "apply", "-f", "-"}, vals.ExecConfig{InheritEnv: true}, ) ``` ``` -------------------------------- ### Safe Navigation with Exists Check Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/config.md Safely access configuration values by first checking for their existence using the `Exists` method. This prevents panics when a key might be missing. ```go conf := config.Map(configData) if conf.Exists("provider", "credentials") { creds := conf.String("provider", "credentials") // Process credentials } ``` -------------------------------- ### MapConfig Access Methods Source: https://github.com/helmfile/vals/blob/main/_autodocs/INDEX.md Provides methods to access configuration values from a MapConfig. Use these to retrieve strings, maps, slices, check existence, or get nested configurations. ```go // Access (MapConfig).String(path ...string) string (MapConfig).Map(path ...string) map[string]interface{} (MapConfig).StringSlice(path ...string) []string (MapConfig).Exists(path ...string) bool (MapConfig).Config(path ...string) StaticConfig ``` -------------------------------- ### Correct Type Conversion in Vals Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md Demonstrates how to correctly access configuration values using the appropriate accessor methods (e.g., StringSlice for arrays) to avoid type conversion errors. Incorrect usage may result in empty strings or unexpected behavior. ```go conf := config.Map(map[string]interface{}{ "count": 123, "items": []string{"a", "b"}, }) // Wrong: trying to get int as string count := conf.String("count") // "123" - OK (converts) // Wrong: trying to get array as string val := conf.String("items") // "" - returns empty string // Correct: use StringSlice items := conf.StringSlice("items") // []string{"a", "b"} ``` -------------------------------- ### Retrieve a Single Secret Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Shows how to retrieve a single secret value using the vals runtime's Get method. This is useful for fetching individual secret values directly. ```go runtime, _ := vals.New(vals.Options{}) password, err := runtime.Get("ref+vault://secret/db#/password") ``` -------------------------------- ### Create Vals Runtime Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Initializes a new Vals runtime with a specified cache size. This is the entry point for using the Vals Go API. ```go runtime, err := vals.New(vals.Options{CacheSize: 256}) ``` -------------------------------- ### Retrieve String Value from MapConfig Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/config.md Use to get a string value from a nested map. It supports path-based access and uses a fallback function for single-component paths if the value is not found. ```go conf := config.MapConfig{ M: map[string]interface{}{ "database": map[string]interface{}{ "host": "localhost", "port": 5432, }, }, } host := conf.String("database", "host") // "localhost" port := conf.String("database", "port") // "5432" missing := conf.String("database", "user") // "" ``` -------------------------------- ### Integrate Helm with Vals for Secret Resolution Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Use vals to template and resolve secrets within Helm manifests before applying them to Kubernetes. This example demonstrates piping Helm output through vals. ```bash # Template and resolve secrets helm template myrelease mychart \ --set dbPassword='ref+vault://secret/db#/password' | \ vals eval -f - | \ kubectl apply -f - ``` -------------------------------- ### Execute Command with Environment Variables Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/runtime.md Use `vals.Exec` to evaluate a template, populate environment variables, and execute a command. Supports streaming YAML to command stdin. Set `InheritEnv` to true to inherit existing environment variables and specify `Stdout` to direct command output. ```Go err := vals.Exec( map[string]interface{}{ "DB_PASSWORD": "ref+vault://secret/db#/password", "API_KEY": "ref+awssecrets://app/key", }, []string{"kubectl", "apply", "-f", "manifests.yaml"}, vals.ExecConfig{ InheritEnv: true, Stdout: os.Stdout, }, ) if err != nil { panic(err) } ``` -------------------------------- ### vals Reference Expression Format Examples Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Illustrates the pattern for vals secret references, including provider, path, parameters, and fragment. The '+' indicates an optional explicit terminator for complex expressions. ```text ref+vault://secret/db#/password ref+awssecrets://myapp/credentials?region=us-west-2#/database/host ref+k8s://v1/Secret/default/myapp/DB_PASSWORD ref+echo://ref+envsubst://$HOME+/path (nested reference) ``` -------------------------------- ### CLI Pipe Integration: File to File Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/io.md Shows how to use Vals with command-line arguments to read from an input file and write to an output file. The `-f` flag specifies the input file, and `-o` specifies the output file. ```bash # Read from file, write to file vals eval -f input.yaml -o output.yaml ``` -------------------------------- ### File Processing Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Shows how to process configuration files, evaluate expressions within them, and output the result. ```APIDOC ## File Processing ```go nodes, _ := vals.Inputs("manifests.yaml") result, _ := vals.EvalNodes(nodes, vals.Options{}) vals.Output(os.Stdout, "yaml", result) ``` ``` -------------------------------- ### Evaluate String (Package Level) Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/runtime.md A convenience function to create a new Runtime and evaluate a string in one call. Use this for quick, single-string evaluations with optional configurations. ```go result, err := vals.Get("API_KEY=ref+awssecrets://myapp/key", vals.Options{}) if err != nil { panic(err) } ``` -------------------------------- ### Retrieve Map Data by Path Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/config.md Use this method to get a map at a specific path within the configuration. An empty path retrieves the entire configuration. Returns nil if the path does not exist or is not a map. ```go conf := config.MapConfig{ M: map[string]interface{}{ "database": map[string]interface{}{ "host": "localhost", "port": 5432, }, }, } dbConf := conf.Map("database") // Result: map[string]interface{}{"host": "localhost", "port": 5432} rootConf := conf.Map() // Result: entire configuration map missing := conf.Map("nonexistent") // Result: nil ``` -------------------------------- ### Batch vs. Streaming Processing Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/io.md Compares batch processing (loading all data into memory) with streaming (processing one document at a time) for large files. Streaming is recommended for memory efficiency. ```go // Batch (all in memory) nodes, _ := vals.Inputs("large.yaml") result, _ := vals.EvalNodes(nodes, opts) // Streaming (process one at a time) // Use Runtime.Eval() on each document individually nodes, _ := vals.Inputs("large.yaml") for _, node := range nodes { var v interface{} node.Decode(&v) result, _ := runtime.Eval(v.(map[string]interface{})) // Process result immediately } ``` -------------------------------- ### Execute Command with Secrets Source: https://github.com/helmfile/vals/blob/main/_autodocs/README.md Illustrates executing a command with secrets injected as environment variables. The `InheritEnv: true` option in `ExecConfig` ensures that existing environment variables are preserved. ```go vals.Exec( map[string]interface{}{ "DB_PASSWORD": "ref+vault://secret/db#/password", }, []string{"kubectl", "apply", "-f", "-"}, vals.ExecConfig{InheritEnv: true}, ) ``` -------------------------------- ### Fail on Missing Key in Map Source: https://github.com/helmfile/vals/blob/main/_autodocs/configuration.md Configure Vals to return an error if a fragment path is not found in a retrieved map. When false, missing paths return nil. This example demonstrates enabling the 'FailOnMissingKeyInMap' option and shows an error scenario. ```go // Error on missing key runtime, err := vals.New(vals.Options{ FailOnMissingKeyInMap: true, }) // If the key doesn't exist, GetString returns an error _, err := runtime.Get("value: ref+vault://secret#/nonexistent") // Error: no value found for key nonexistent ``` -------------------------------- ### Reusing Vals Runtime for Caching Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/io.md Explains how to improve performance by creating a single Vals Runtime instance and reusing it for multiple evaluations. This allows the cache to be shared across evaluations. ```go runtime, _ := vals.New(opts) // Evaluate multiple node sets with same runtime nodes1, _ := vals.Inputs("file1.yaml") nodes2, _ := vals.Inputs("file2.yaml") result1, _ := runtime.Eval(decodeToMap(nodes1[0])) result2, _ := runtime.Eval(decodeToMap(nodes2[0])) // Cache is shared across evaluations ``` -------------------------------- ### Example of Nested Reference Type Mismatch Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md Demonstrates an invalid use case where a nested reference resolves to a map or array type, which is not permitted when used within a URI. It contrasts this with a valid case where the nested reference resolves to a scalar value. ```text // Invalid (nested ref returns object): ref+s3://bucket/ref+vault://secret/db#/config // Valid (nested ref returns string): ref+s3://bucket/ref+vault://secret/db#/path ``` -------------------------------- ### Expand InValue Method Examples Source: https://github.com/helmfile/vals/blob/main/_autodocs/api-reference/expansion.md Illustrates the `InValue` method for expanding references in a string value while preserving the type for complete matches. If the entire string is a single reference, the resolved value's type is maintained; otherwise, it's expanded as a string. ```go expand := expansion.ExpandRegexMatch{ Target: expansion.DefaultRefRegexp, Lookup: func(key string) (interface{}, error) { if key == "echo://123" { return 123, nil } return nil, fmt.Errorf("unknown ref") }, } // Full match: type preserved result, _ := expand.InValue("ref+echo://123") // result is int 123, not string "123" // Partial match: converted to string result, _ := expand.InValue("Value: ref+echo://123") // result is string "Value: 123" ``` -------------------------------- ### Fallback to Alternative Source on Failure Source: https://github.com/helmfile/vals/blob/main/_autodocs/errors.md This Go code demonstrates a fallback pattern where if the primary data source fails, it attempts to retrieve data from a secondary file-based source. It returns an error if both sources fail. ```go result, err := runtime.Get("ref+vault://secret/primary#/key") if err != nil { log.Printf("Primary source failed: %v", err) // Try fallback result, err = runtime.Get("ref+file://fallback.txt") if err != nil { return fmt.Errorf("both primary and fallback failed: %v", err) } } ```