### Install a Steampipe Plugin Source: https://github.com/turbot/steampipe/blob/develop/tests/acceptance/test_data/templates/expected_plugin_help_output.txt Installs a specified plugin from the Steampipe plugin registry. Ensure the plugin name is correct for successful installation. ```bash steampipe plugin install aws ``` -------------------------------- ### Install Steampipe CLI (Linux/Windows WSL2) Source: https://github.com/turbot/steampipe/blob/develop/README.md Installs the Steampipe CLI on Linux or Windows (WSL2) by downloading and executing an installation script. This script fetches and runs the official Steampipe installer. ```shell # Linux or Windows (WSL2) sudo /bin/sh -c "$(curl -fsSL https://steampipe.io/install/steampipe.sh)" ``` -------------------------------- ### Set Installation Prefix and Linker Flags Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Configures environment variables for the installation directory and linker flags. LDFLAGS are set to ensure shared libraries are found at runtime, crucial for relocatable installations. ```bash export PREFIX=/postgres-binaries-14.19/linux-$(uname -m) mkdir -p "$PREFIX" export LDFLAGS='-Wl,-rpath,$ORIGIN/../lib/postgresql -Wl,--enable-new-dtags' ``` -------------------------------- ### Run First Steampipe Query Source: https://github.com/turbot/steampipe/blob/develop/README.md Executes an initial query using Steampipe. This example demonstrates inspecting the 'steampipe' plugin and selecting data from the 'steampipe_registry_plugin' table. ```sh steampipe query > .inspect steampipe +-----------------------------------+ | TABLE | +-----------------------------------+ | steampipe_registry_plugin | | steampipe_registry_plugin_version | +-----------------------------------+ > select * from steampipe_registry_plugin; ``` -------------------------------- ### Build and Install PostgreSQL Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Compiles the PostgreSQL source code and installs the binaries and libraries to the specified prefix. The '-j2' flag enables parallel compilation using 2 jobs. ```bash make -j2 make install ``` -------------------------------- ### Build and Install Contrib Extensions Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Compiles and installs additional contributed modules for PostgreSQL. This step ensures that optional functionalities are available. ```bash make -C contrib -j2 make -C contrib install ``` -------------------------------- ### Install Steampipe Plugin Source: https://github.com/turbot/steampipe/blob/develop/README.md Installs a Steampipe plugin. This command is used to add new data sources or functionalities to Steampipe. ```sh steampipe plugin install steampipe ``` -------------------------------- ### Install Steampipe CLI (macOS) Source: https://github.com/turbot/steampipe/blob/develop/README.md Installs the Steampipe CLI on macOS using Homebrew. This command adds the Turbot tap and installs the steampipe package. ```shell # MacOS brew install turbot/tap/steampipe ``` -------------------------------- ### Build Steampipe Binary Source: https://github.com/turbot/steampipe/blob/develop/README.md Builds the Steampipe binary from the cloned repository. After cloning, navigate to the directory and run 'make'. The binary is typically installed in '/usr/local/bin/steampipe' unless an alternative OUTPUT_DIR is specified. ```sh cd steampipe make ``` -------------------------------- ### Install Steampipe on Linux Source: https://context7.com/turbot/steampipe/llms.txt Installs Steampipe on Linux systems using a shell script downloaded from the official Steampipe website. It also includes a command to verify the installation by checking the version. ```bash # Install via shell script sudo /bin/sh -c "$(curl -fsSL https://steampipe.io/install/steampipe.sh)" # Verify installation steampipe --version # Output: steampipe version 2.3.5 ``` -------------------------------- ### Wrap error, replacing root message in Go Source: https://github.com/turbot/steampipe/blob/develop/design/sperr.md Demonstrates how to replace the root error message using `sperr.WrapWithRootMessage` or `sperr.WithRootMessage`. This is useful for providing a user-friendly summary while retaining the original error details internally. The example concerns installation failure. ```go if _, err := installFDW(ctx, false); err != nil { log.Printf("[TRACE] installFDW failed: %v", err) return sperr.WrapWithRootMessage(err, "Update steampipe-postgres-fdw... FAILED!") } ``` ```go if _, err := installFDW(ctx, false); err != nil { log.Printf("[TRACE] installFDW failed: %v", err) return sperr.Wrap(err, sperr.WithRootMessage("Update steampipe-postgres-fdw... FAILED!")) } ``` -------------------------------- ### Pack PostgreSQL Binaries into .txz Archive Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Creates a compressed tar archive (.txz) of the PostgreSQL installation directory. This is useful for distributing or backing up the compiled binaries. ```bash cd $(dirname "$PREFIX") tar -cJf postgres-14.19-$(uname -m).txz $(basename "$PREFIX") ``` -------------------------------- ### Install Steampipe Plugin Source: https://github.com/turbot/steampipe/blob/develop/README.md Installs a Steampipe plugin for a specific service, such as Hacker News. This command allows Steampipe to query data from the specified service. ```shell steampipe plugin install hackernews ``` -------------------------------- ### Example of Wrapping Errors with Options (Go) Source: https://github.com/turbot/steampipe/blob/develop/design/sperr.md Demonstrates how to wrap an error using multiple options like WithMessage and WithDetail to add context and specific information to the error. ```Go sperr.Wrap( err, sperr.WithMessage("operation '%s' failed", operation), sperr.WithDetail("argument: %d", input), ) ``` -------------------------------- ### Configure PostgreSQL Build on macOS Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Configures the PostgreSQL build process on macOS, specifying installation paths, enabling OpenSSL support, and setting include/library directories for OpenSSL. Ensure `libdir` and `datadir` point to the correct subdirectories within the installation path for Steampipe compatibility. ```bash ./configure --prefix=location/where/you/want/the/files \ --libdir=/location/where/you/want/the/files/lib/postgresql \ --datadir=/location/where/you/want/the/files/share/postgresql \ --with-openssl \ --with-includes=$(brew --prefix openssl)/include \ --with-libraries=$(brew --prefix openssl)/lib ``` -------------------------------- ### Configure PostgreSQL Build Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Runs the configure script to prepare the build environment. It specifies installation paths, enables OpenSSL support, and points to system include and library directories. ```bash ./configure \ --prefix="$PREFIX" \ --libdir="$PREFIX/lib/postgresql" \ --datadir="$PREFIX/share/postgresql" \ --with-openssl \ --with-includes=/usr/include \ --with-libraries=/usr/lib/$(uname -m)-linux-gnu ``` -------------------------------- ### Minimal Example Program for Error Handling (Go) Source: https://github.com/turbot/steampipe/blob/develop/design/sperr.md A minimal Go program illustrating error creation using sperr.WrapWithRootMessage and subsequent wrapping with additional messages and details. ```Go func readFile() error { path := "/imaginary/path" _, err := os.Open(path) if err != nil { return sperr.WrapWithRootMessage(err, "could not open file at %s", path) } return nil } func wrapWithMessageAndDetail() error { err := readFile() return sperr.Wrap( err, sperr.WithMessage("message from wrapWithMessageAndDetail"), sperr.WithDetail("detail from wrapWithMessageAndDetail"), ) } showCaseErr := sperr.Wrap( err, sperr.WithMessage("message from main"), sperr.WithDetail("detail from main"), ) ``` -------------------------------- ### Install Steampipe on macOS Source: https://context7.com/turbot/steampipe/llms.txt Installs Steampipe on macOS using the Homebrew package manager. This is a straightforward installation method for macOS users. ```bash # Install via Homebrew brew install turbot/tap/steampipe ``` -------------------------------- ### Install PostgreSQL Dependencies on Linux Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Installs the necessary build tools and development libraries required to compile PostgreSQL from source on Ubuntu 24 (amd64 or arm64). This includes essential packages like `build-essential`, `libssl-dev`, and `patchelf`. ```bash apt update apt install -y build-essential wget ca-certificates \ libreadline-dev zlib1g-dev flex bison \ libssl-dev patchelf file ``` -------------------------------- ### Check Steampipe Version Source: https://github.com/turbot/steampipe/blob/develop/README.md Verifies the installed Steampipe version. This command is useful after building or installing Steampipe to confirm the correct version is active. ```sh steampipe --version ``` -------------------------------- ### List Installed Steampipe Plugins Source: https://github.com/turbot/steampipe/blob/develop/tests/acceptance/test_data/templates/expected_plugin_help_output.txt Displays a list of all plugins currently installed in your Steampipe environment. This helps in managing your installed plugins and checking their status. ```bash steampipe plugin list ``` -------------------------------- ### Connecting to Steampipe Service (Bash) Source: https://context7.com/turbot/steampipe/llms.txt Instructions for connecting to the Steampipe PostgreSQL service using external clients like psql. Examples show connecting via host, port, user, and database, as well as using a connection string. ```bash # Connect using psql psql -h localhost -p 9193 -U steampipe -d steampipe # Connect using connection string psql "postgres://steampipe:password@localhost:9193/steampipe" # Example query from psql # steampipe=> SELECT name, instance_type FROM aws_ec2_instance WHERE instance_state = 'running'; ``` -------------------------------- ### Start OpenTelemetry Collector with Docker Compose Source: https://github.com/turbot/steampipe/blob/develop/pkg/otel/README.md This command starts the OpenTelemetry Collector in detached mode using docker-compose. Ensure you are in the 'otel' directory before running. It spins up containers for Jaeger and Prometheus, making them accessible locally. ```shell docker-compose up -d ``` -------------------------------- ### Create error with message and detail in Go Source: https://github.com/turbot/steampipe/blob/develop/design/sperr.md Shows how to create an error with both a primary message and additional details using `sperr.New` and `sperr.WithDetail`. This provides more context for debugging. The example validates an integer argument. ```go func validateData(data int) error { if data > 10 { return sperr.Wrap( sperr.New("invalid argument: %d", data), sperr.WithDetail("error occurred with %d argument", data), ) } return nil } ``` -------------------------------- ### Manage Steampipe Plugins Source: https://context7.com/turbot/steampipe/llms.txt Provides commands for managing Steampipe plugins, which extend its functionality to connect to various services. This includes installing, listing, updating, and uninstalling plugins. ```bash # Install a plugin with default configuration steampipe plugin install aws # Install a specific version steampipe plugin install turbot/azure@0.1.0 # Install plugin without creating default config file steampipe plugin install --skip-config aws # Install all plugins referenced in configuration files steampipe plugin install # Hide progress bars during installation steampipe plugin install --progress=false aws # List all installed plugins in table format steampipe plugin list # List plugins in JSON format steampipe plugin list --output json # Check for available updates steampipe plugin list --outdated # Update a specific plugin steampipe plugin update aws # Update all installed plugins steampipe plugin update --all # Update without progress bars steampipe plugin update --progress=false aws # Remove a plugin steampipe plugin uninstall aws ``` -------------------------------- ### Remove Include Directory Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Deletes the 'include' directory from the installation prefix. This is often done to reduce the size of the final package if the header files are not needed. ```bash rm -rf "$PREFIX/include" ``` -------------------------------- ### Patch RPATHs for Relocatability Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Modifies the RPATH (Run-time Search Path) for executable files within the installation prefix. This ensures that libraries can be found when the installation is moved to a different location. ```bash cd "$PREFIX" for f in bin/*; do if [ -x "$f" ] && file "$f" | grep -q ELF; then patchelf --set-rpath '$ORIGIN/../lib/postgresql' "$f" fi done ``` -------------------------------- ### Configure Steampipe Environment Variables (Bash) Source: https://context7.com/turbot/steampipe/llms.txt Sets key environment variables for Steampipe, including installation directory, database configuration, cache settings, Turbot Pipes integration, query timeouts, telemetry, and logging levels. ```bash # Installation directory export STEAMPIPE_INSTALL_DIR=~/.steampipe # Database configuration export STEAMPIPE_DATABASE_PORT=9193 export STEAMPIPE_DATABASE_PASSWORD=mypassword # Cache configuration export STEAMPIPE_CACHE=true export STEAMPIPE_CACHE_TTL=300 # Turbot Pipes integration export PIPES_HOST=pipes.turbot.com export PIPES_TOKEN=tpt_... # Query timeout (in seconds) export STEAMPIPE_DATABASE_QUERY_TIMEOUT=300 # Telemetry export STEAMPIPE_TELEMETRY=none # Logging export STEAMPIPE_LOG_LEVEL=WARN ``` -------------------------------- ### Wrap an error with a message in Go Source: https://github.com/turbot/steampipe/blob/develop/design/sperr.md Demonstrates wrapping an error and adding a custom message using `sperr.WrapWithMessage`. This is useful for providing higher-level context to an underlying error. The example includes file path information. ```go if err := json.Unmarshal(byteContent, &data); err != nil { return nil, sperr.WrapWithMessage(err, "error unmarshalling file content in %s", filePath) } ``` ```go if err := json.Unmarshal(byteContent, &data); err != nil { return nil, sperr.Wrap(err, sperr.WithMessage("error unmarshalling file content in %s", filePath)) } ``` -------------------------------- ### Steampipe Service Management (Bash) Source: https://context7.com/turbot/steampipe/llms.txt Commands for managing the Steampipe service, including starting, checking status, stopping, and restarting. Options allow customization of ports, network access, passwords, and running in foreground mode. ```bash # Start service with default settings (localhost:9193) steampipe service start # Start on a custom port steampipe service start --database-port 5432 # Allow network connections steampipe service start --database-listen-addresses network # Start with a custom password steampipe service start --service-password mypassword # Run in foreground (useful for containers) steampipe service start --foreground # Show password in output steampipe service start --show-password # Check status of current installation steampipe service status # Show database password steampipe service status --show-password # Check all running Steampipe instances steampipe service status --all # Stop the service steampipe service stop # Force stop (disconnects all clients) steampipe service stop --force # Restart the service steampipe service restart # Force restart steampipe service restart --force ``` -------------------------------- ### Wrap an error with detail in Go Source: https://github.com/turbot/steampipe/blob/develop/design/sperr.md Shows how to wrap an error and add specific details using `sperr.Wrap` combined with `sperr.WithDetail`. This adds contextual information to the error chain, aiding in debugging. The example relates to data validation. ```go err := validateData(userInput.numAttacks) if err!= nil { return sperr.Wrap(err, sperr.WithDetail("error occurred with %d argument", userInput.numAttacks)) } ``` -------------------------------- ### Create Steampipe Database Client (Go) Source: https://context7.com/turbot/steampipe/llms.txt Demonstrates how to create a new database client using the Steampipe Go SDK. It requires a connection string and handles potential errors during client creation. The client can then be used to access server settings. ```go package main import ( "context" "fmt" "log" "github.com/turbot/steampipe/v2/pkg/db/db_client" ) func main() { ctx := context.Background() // Create a new database client connectionString := "postgres://steampipe:password@localhost:9193/steampipe" client, err := db_client.NewDbClient(ctx, connectionString) if err != nil { log.Fatal(err) } defer client.Close(ctx) // Get server settings settings := client.ServerSettings() if settings != nil { fmt.Printf("Server Version: %s\n", settings.ServerVersion) } } ``` -------------------------------- ### Work with Steampipe Connection Configuration (Go) Source: https://context7.com/turbot/steampipe/llms.txt Shows how to load Steampipe configuration and access connection details and database options using the Go SDK. It iterates through configured connections and prints their names and plugin types. ```go package main import ( "context" "fmt" "log" "github.com/turbot/steampipe/v2/pkg/steampipeconfig" ) func main() { ctx := context.Background() // Load Steampipe configuration config, errorsAndWarnings := steampipeconfig.LoadSteampipeConfig(ctx, ".", "query") if errorsAndWarnings.GetError() != nil { log.Fatal(errorsAndWarnings.GetError()) } // List all connections for name, conn := range config.Connections { fmt.Printf("Connection: %s, Plugin: %s\n", name, conn.Plugin) } // Get database options if config.DatabaseOptions != nil { fmt.Printf("Database Port: %d\n", config.DatabaseOptions.Port) } } ``` -------------------------------- ### Verify Library Linkage Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Uses 'ldd' to verify that the 'libpq' library is correctly linked to the 'initdb' executable. This ensures that the PostgreSQL client library can be found at runtime. ```bash ldd bin/initdb | grep libpq ``` -------------------------------- ### Uninstall a Steampipe Plugin Source: https://github.com/turbot/steampipe/blob/develop/tests/acceptance/test_data/templates/expected_plugin_help_output.txt Removes a specified plugin from your Steampipe installation. Use this command when a plugin is no longer needed to free up resources. ```bash steampipe plugin uninstall aws ``` -------------------------------- ### Steampipe Query Search Path Configuration (Bash) Source: https://context7.com/turbot/steampipe/llms.txt Demonstrates how to control the order and selection of schemas searched by Steampipe queries using the `--search-path` and `--search-path-prefix` command-line arguments. ```bash # Set search path for query session steampipe query --search-path aws,github,public # Add prefix to existing search path steampipe query --search-path-prefix aws_prod # Example with multiple connections: steampipe query --search-path aws_prod,aws_dev "select * from aws_s3_bucket" ``` -------------------------------- ### Execute Steampipe Queries Source: https://context7.com/turbot/steampipe/llms.txt Demonstrates how to execute SQL queries using the Steampipe CLI in both interactive and batch modes. Supports various output formats like table, JSON, and CSV, along with options for timing and custom separators. ```bash # Start interactive query session steampipe query # Example interactive session: # > select * from aws_s3_bucket limit 5; # +---------------------------+---------------------+--------+ # | name | creation_date | region | # +---------------------------+---------------------+--------+ # | my-bucket | 2023-01-15 10:30:00 | us-east-1 | # | logs-bucket | 2023-02-20 14:45:00 | us-west-2 | # +---------------------------+---------------------+--------+ # Use .inspect to view available tables # > .inspect aws # +-----------------------------+----------------------------------------+ # | TABLE | DESCRIPTION | # +-----------------------------+----------------------------------------+ # | aws_s3_bucket | AWS S3 Bucket | # | aws_ec2_instance | AWS EC2 Instance | # +-----------------------------+----------------------------------------+ # Exit interactive mode # > .exit # Run a single query steampipe query "select name, region from aws_s3_bucket" # Run query from SQL file steampipe query query.sql # Run multiple queries steampipe query "select count(*) from aws_s3_bucket" "select count(*) from aws_ec2_instance" # Query with JSON output steampipe query "select * from aws_s3_bucket limit 2" --output json # Output: # { # "columns": [ # {"name": "name", "data_type": "text"}, # {"name": "region", "data_type": "text"} # ], # "rows": [ # {"name": "my-bucket", "region": "us-east-1"}, # {"name": "logs-bucket", "region": "us-west-2"} # ] # } # Query with CSV output steampipe query "select name, region from aws_s3_bucket" --output csv # Query with timing information steampipe query "select * from aws_ec2_instance" --timing # Query with custom separator for CSV steampipe query "select * from aws_s3_bucket" --output csv --separator "|" # Pipe query from stdin echo "select * from aws_s3_bucket limit 5" | steampipe query # Table output (default) steampipe query "select name from aws_s3_bucket" --output table ``` -------------------------------- ### Update a Steampipe Plugin Source: https://github.com/turbot/steampipe/blob/develop/tests/acceptance/test_data/templates/expected_plugin_help_output.txt Updates an already installed plugin to its latest version. This command ensures you have the most recent features and bug fixes for the specified plugin. ```bash steampipe plugin update aws ``` -------------------------------- ### Verify RPATH Setting Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Checks the RPATH setting for the 'initdb' executable using 'readelf'. This confirms that the RPATH has been correctly updated for relocatability. ```bash readelf -d bin/initdb | grep -i rpath ``` -------------------------------- ### Clone Steampipe Repository Source: https://github.com/turbot/steampipe/blob/develop/README.md Clones the Steampipe GitHub repository to your local machine. This is the first step to begin developing the core Steampipe binary. ```sh git clone git@github.com:turbot/steampipe ``` -------------------------------- ### Steampipe Query Output Formats Source: https://context7.com/turbot/steampipe/llms.txt Demonstrates how to execute Steampipe queries and specify different output formats for the results. Supported formats include JSON, CSV, line (key-value pairs), and SPS (snapshot format). ```bash steampipe query "select name from aws_s3_bucket" --output json ``` ```bash steampipe query "select name from aws_s3_bucket" --output csv ``` ```bash steampipe query "select name from aws_s3_bucket" --output line ``` ```bash steampipe query "select name from aws_s3_bucket" --output sps ``` -------------------------------- ### Define Go Struct for Options with Interface Methods Source: https://github.com/turbot/steampipe/blob/develop/design/adding_to_workspace_profile.md Defines a Go struct for options, implementing interface methods for configuration mapping, merging, and string serialization. This is a common pattern for handling extensible configuration in Steampipe. ```go type Query struct {} // ConfigMap :: this is merged with viper // Only add keys which are not nil func (t *Query) ConfigMap() map[string]interface{} {} // Merge :: merge other options over the top of this options object // i.e. if a property is set in otherOptions, it takes precedence func (t *Query) Merge(otherOptions Options) { // make sure this is the type we want if _, ok := otherOptions.(*Query); !ok { return } } // String serialize for printing func (t *Query) String() string {} ``` -------------------------------- ### Enable Steampipe Shell Completion (Bash, Zsh, Fish) Source: https://context7.com/turbot/steampipe/llms.txt Provides commands to generate and load shell completion scripts for Bash, Zsh, and Fish. This enhances the command-line experience by offering autocompletion for Steampipe commands. ```bash # Generate Bash completion script steampipe completion bash > /etc/bash_completion.d/steampipe # Generate Zsh completion script steampipe completion zsh > "${fpath[1]}/_steampipe" # Generate Fish completion script steampipe completion fish > ~/.config/fish/completions/steampipe.fish # Load completion in current session (Bash) source <(steampipe completion bash) # Load completion in current session (Zsh) source <(steampipe completion zsh) ``` -------------------------------- ### Steampipe AWS Connection Configuration (HCL) Source: https://context7.com/turbot/steampipe/llms.txt Defines how to configure AWS connections for Steampipe using HCL files in `~/.steampipe/config/`. Examples cover single connections using default or specific profiles, and explicit credentials. ```hcl # ~/.steampipe/config/aws.spc # Single AWS connection using default credentials connection "aws" { plugin = "aws" # Uses AWS_PROFILE, AWS_ACCESS_KEY_ID, etc. from environment } # AWS connection with specific profile connection "aws_prod" { plugin = "aws" profile = "production" regions = ["us-east-1", "us-west-2"] } # AWS connection with explicit credentials connection "aws_dev" { plugin = "aws" access_key = "AKIA..." secret_key = "..." regions = ["us-east-1"] } ``` -------------------------------- ### Steampipe Snapshot Sharing to Turbot Pipes (Bash) Source: https://context7.com/turbot/steampipe/llms.txt Instructions for sharing Steampipe snapshots to Turbot Pipes. This includes logging into Turbot Pipes, sharing snapshots with link visibility, creating private snapshots, and adding tags. ```bash # Login to Turbot Pipes steampipe login # Opens browser for authentication, enter verification code when prompted # Share snapshot with link visibility steampipe query "select * from aws_s3_bucket" --share # Create private snapshot in workspace steampipe query "select * from aws_s3_bucket" --snapshot # Add tags to snapshot steampipe query "select * from aws_s3_bucket" --share --snapshot-tag "team=security" --snapshot-tag "env=prod" ``` -------------------------------- ### Steampipe Database Options Configuration (HCL) Source: https://context7.com/turbot/steampipe/llms.txt Shows how to configure global database options for Steampipe, such as port, listen address, search path, and caching behavior. These settings are typically defined in a `default.spc` file. ```hcl # ~/.steampipe/config/default.spc options "database" { port = 9193 listen = "local" search_path = "aws,github,public" cache = true cache_ttl = 300 } ``` -------------------------------- ### Query Data with Steampipe Source: https://github.com/turbot/steampipe/blob/develop/README.md Opens the Steampipe query interface and executes a SQL query to retrieve the top 10 new items from Hacker News. This demonstrates basic data retrieval using Steampipe. ```shell steampipe query > select * from hackernews_new limit 10 ``` -------------------------------- ### Navigate to PostgreSQL Source Directory Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md Changes the current working directory to the PostgreSQL source code location. This is the initial step before proceeding with the build process. ```bash cd /postgres/source/dir ``` -------------------------------- ### Get CloudTrail Event Selectors (AWS CLI) Source: https://github.com/turbot/steampipe/blob/develop/tests/acceptance/test_data/mods/sample_workspace/cis_v130/docs/cis_v130_3_10.md Retrieves the event selectors for a specified CloudTrail trail to determine if Data events logging is enabled for S3 buckets. An empty array indicates that S3 object-level API operations are not being logged. ```bash aws cloudtrail get-event-selectors --region --trail-name --query EventSelectors[*].DataResources[] ``` -------------------------------- ### Get S3 Bucket Policy using AWS CLI (Bash) Source: https://github.com/turbot/steampipe/blob/develop/tests/acceptance/test_data/mods/sample_workspace/cis_v130/docs/cis_v130_2_1_2.md This command retrieves the existing bucket policy for a specified S3 bucket and saves it to a JSON file. It's the first step in modifying the policy via the command line. ```bash aws s3api get-bucket-policy --bucket --query Policy --output text > policy.json ``` -------------------------------- ### Steampipe Snapshot Export and Creation (Bash) Source: https://context7.com/turbot/steampipe/llms.txt Commands for exporting query results into snapshot files (`.sps`) and creating snapshots with custom titles. Snapshots are useful for saving and sharing query results. ```bash # Export query results as snapshot steampipe query "select * from aws_s3_bucket" --export snapshot.sps # Create snapshot with custom title steampipe query "select * from aws_s3_bucket" --snapshot-title "S3 Bucket Inventory" ``` -------------------------------- ### Enable S3 Default Bucket Encryption with AWS CLI Source: https://github.com/turbot/steampipe/blob/develop/tests/acceptance/test_data/mods/sample_workspace/cis_v130/docs/cis_v130_2_1_1.md This snippet demonstrates how to enable default server-side encryption for an S3 bucket using the AWS CLI. It supports both AES-256 and AWS-KMS encryption algorithms. Ensure you have the AWS CLI installed and configured with appropriate permissions. ```bash aws s3api put-bucket-encryption --bucket --server-side-encryption-configuration '{"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}]}' ``` ```bash aws s3api put-bucket-encryption --bucket --server-side-encryption-configuration '{"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "aws:kms","KMSMasterKeyID": "aws/s3"}}]}' ``` -------------------------------- ### Fix RPATHs for PostgreSQL Binaries on macOS Source: https://github.com/turbot/steampipe/blob/develop/design/embedded_postgres_build_instructions.md A bash script to fix the RPATHs of PostgreSQL binaries on macOS, ensuring they can find shared libraries like libpq. It modifies install names and adds necessary RPATH entries for relocatable binaries. This script is crucial for the packaged PostgreSQL binaries to function correctly within Steampipe. ```bash #!/bin/bash set -euo pipefail # --- CONFIGURE --- # Adjust if your libpq lives in lib/ not lib/postgresql LIB_SUBDIR="lib/postgresql" BUNDLE_ROOT="$(pwd)" LIBPQ_PATH="$BUNDLE_ROOT/$LIB_SUBDIR/libpq.5.dylib" echo "🔧 Fixing libpq install name..." install_name_tool -id "@rpath/libpq.5.dylib" "$LIBPQ_PATH" echo "🔍 Processing binaries in bin/..." for binfile in "$BUNDLE_ROOT"/bin/*; do [[ -x "$binfile" && ! -d "$binfile" ]] || continue echo "➡️ Patching $(basename "$binfile")" # Ensure an rpath to ../lib/postgresql exists install_name_tool -add_rpath "@loader_path/../$LIB_SUBDIR" "$binfile" 2>/dev/null || true # Rewrite any absolute reference to libpq install_name_tool -change "$BUNDLE_ROOT/$LIB_SUBDIR/libpq.5.dylib" "@rpath/libpq.5.dylib" "$binfile" 2>/dev/null || true done echo "✅ Verification:" for binfile in "$BUNDLE_ROOT"/bin/*; do [[ -x "$binfile" && ! -d "$binfile" ]] || continue echo "--- $(basename "$binfile") ---" otool -L "$binfile" | grep libpq || echo "⚠️ No libpq linkage" otool -l "$binfile" | grep -A2 LC_RPATH | grep path || echo "⚠️ No RPATH" done ``` -------------------------------- ### Define Steampipe ScanMetadata Structure (Go) Source: https://github.com/turbot/steampipe/blob/develop/design/timing_output.md Defines the ScanMetadata struct used to store information about a Steampipe FDW scan. It includes details like table name, cache status, rows fetched, hydration calls, columns, qualifiers, limit, start time, and duration. This struct is central to tracking scan performance and details. ```go type ScanMetadata struct { Id int Table string CacheHit bool RowsFetched int64 HydrateCalls int64 Columns []string Quals map[string]*proto.Quals Limit int64 StartTime time.Time Duration time.Duration } ``` -------------------------------- ### Execute Steampipe Queries Programmatically (Go) Source: https://context7.com/turbot/steampipe/llms.txt Provides a function to execute SQL queries programmatically using the Steampipe Go SDK. It returns a streamer to process results row by row and handles potential errors during query execution. ```go package main import ( "context" "fmt" "github.com/turbot/steampipe/v2/pkg/db/db_common" "github.com/turbot/pipe-fittings/v2/modconfig" ) func executeQuery(ctx context.Context, client db_common.Client, sql string) error { // Execute query and get result streamer streamer, err := db_common.ExecuteQuery(ctx, client, sql) if err != nil { return err } // Process results for result := range streamer.Results { for row := range result.RowChan { fmt.Printf("Row: %v\n", row) } streamer.AllResultsRead() } return nil } ``` -------------------------------- ### Steampipe Search Path in Interactive Mode (SQL) Source: https://context7.com/turbot/steampipe/llms.txt Shows how to manage the Steampipe query search path directly within the interactive SQL session using `.search_path` and `.search_path_prefix` commands. ```sql -- View current search path .search_path -- Set search path in session .search_path aws_prod,aws_dev,public -- Add prefix to search path .search_path_prefix aws_prod ``` -------------------------------- ### Display Steampipe Query Execution Time (Bash) Source: https://context7.com/turbot/steampipe/llms.txt Shows query execution time, with an option for verbose timing that includes hydrate function calls and cache status. Can be toggled interactively. ```bash # Show timing with query steampipe query "select * from aws_s3_bucket" --timing # Verbose timing (includes hydrate function calls, cache status) steampipe query "select * from aws_s3_bucket" --timing verbose # Interactive timing toggle # > .timing on # > select * from aws_s3_bucket; # Time: 1.234s ``` -------------------------------- ### HCL Startup with Invalid Plugin Instance Source: https://github.com/turbot/steampipe/blob/develop/design/internal_introspection_tables_tests.md Tests a Steampipe configuration where a connection refers to a plugin instance by name, but the plugin instance itself is configured with an invalid source. The expected behavior is the same as a missing plugin. ```hcl connection "aws" { plugin = "aws_bar" } plugin "aws_bar"{ source="aws" } ```