Steampipe

https://github.com/devops-ia/steampipe
Admin
Steampipe is a tool that uses SQL to instantly query cloud services including AWS, Azure, GCP and...
Tokens:13,183
Snippets:134
Trust Score:8.4
License:MIT
Update:6 months ago
Show doc for...
Tokens:
Raw
### Run and Install GCP Plugin

Source: https://context7.com/devops-ia/steampipe/llms.txt

Commands to start Steampipe with GCP credentials and install the plugin.

```bash
# Mount GCP config and service account credentials
docker run -d --name steampipe \
  -p 9193:9193 \
  -v "$PWD/gcp.spc:/home/steampipe/.steampipe/config/gcp.spc:ro" \
  -v "$PWD/service-account.json:/home/steampipe/.config/gcloud/application_default_credentials.json:ro" \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

# Install GCP plugin
docker exec steampipe steampipe plugin install gcp
```

--------------------------------

### Install Steampipe Plugin with Docker

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Create a named volume for persistent plugin storage, start the Steampipe service, and then install a plugin like AWS using `docker exec`.

```bash
# Create a named volume so plugins survive container restarts
docker volume create steampipe-data

docker run -d --name steampipe \
  -p 9193:9193 \
  -v steampipe-data:/home/steampipe/.steampipe \
  -e STEAMPIPE_DATABASE_PASSWORD=mypassword \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

# Install the AWS plugin
docker exec steampipe steampipe plugin install aws

# List installed plugins
docker exec steampipe steampipe plugin list
```

--------------------------------

### Install Steampipe with Helm

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Add the repository and install the Steampipe chart with specific database and image configurations.

```bash
helm repo add devops-ia https://devops-ia.github.io/helm-charts
helm repo update

helm install steampipe devops-ia/steampipe \
  --set image.repository=ghcr.io/devops-ia/steampipe \
  --set image.tag=2.4.1 \
  --set bbdd.enabled=true \
  --set bbdd.listen=network \
  --namespace steampipe \
  --create-namespace
```

--------------------------------

### Install AWS Plugin and Query S3 Buckets

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Installs the AWS plugin and then lists all S3 buckets, ordered by creation date. Ensure Docker is running and the steampipe container is available.

```bash
docker exec steampipe steampipe plugin install aws

# List all S3 buckets
docker exec steampipe steampipe query \
  "select name, region, creation_date from aws_s3_bucket order by creation_date desc"
```

--------------------------------

### Run and Connect to Steampipe Container

Source: https://github.com/devops-ia/steampipe/blob/main/README.md

Commands to start the Steampipe service in a container, install a plugin, and connect using a PostgreSQL client.

```bash
# Run Steampipe as a service (PostgreSQL endpoint on port 9193)
docker run -d --name steampipe \
  -p 9193:9193 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

# Install a plugin
docker exec steampipe steampipe plugin install aws

# Connect with any PostgreSQL client
psql -h localhost -p 9193 -U steampipe -d steampipe
```

```bash
# Run Steampipe as a service (PostgreSQL endpoint on port 9193)
docker run -d --name steampipe \
  -p 9193:9193 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  service start --foreground --database-listen network

# Install a plugin
docker exec steampipe steampipe plugin install aws

# Connect with any PostgreSQL client
psql -h localhost -p 9193 -U steampipe -d steampipe
```

--------------------------------

### Run Steampipe with Docker Compose

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Start Steampipe using Docker Compose, then install a plugin.

```bash
cd examples
docker compose up -d
docker compose exec steampipe steampipe plugin install aws
```

--------------------------------

### Install Plugins via Init Container

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Specify plugins to be installed during the initialization phase of the pod.

```yaml
initContainer:
  plugins:
    - aws
    - azure
    - gcp
```

--------------------------------

### Connect to Steampipe from Go

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Provides a Go example for connecting to Steampipe using the standard database/sql package and the pq driver to query S3 buckets.

```go
package main

import (
    "database/sql"
    "fmt"
    _ "github.com/lib/pq"
)

func main() {
    db, _ := sql.Open("postgres",
        "host=localhost port=9193 dbname=steampipe user=steampipe password=your-password sslmode=disable")
    defer db.Close()

    rows, _ := db.Query("SELECT name, region FROM aws_s3_bucket")
    defer rows.Close()
    for rows.Next() {
        var name, region string
        rows.Scan(&name, &region)
        fmt.Printf("%s (%s)\n", name, region)
    }
}
```

--------------------------------

### Query Multi-Account AWS Setup

Source: https://context7.com/devops-ia/steampipe/llms.txt

Commands to mount multi-account configuration and execute queries across the aggregator.

```bash
# Mount the config and credentials
docker run -d --name steampipe \
  -p 9193:9193 \
  -v "$PWD/multi-account-aws.spc:/home/steampipe/.steampipe/config/aws.spc:ro" \
  -v "$HOME/.aws:/home/steampipe/.aws:ro" \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

# Query across all accounts using the aggregator
docker exec steampipe steampipe query \
  "select _ctx->>'connection_name' as account, name, region from aws_all.aws_s3_bucket"
```

--------------------------------

### Execute Multi-Cloud Queries

Source: https://context7.com/devops-ia/steampipe/llms.txt

Installs multiple plugins and performs cross-provider SQL queries.

```bash
# Install multiple cloud plugins
docker exec steampipe steampipe plugin install aws azure gcp

# Compare running VMs across AWS and Azure
docker exec steampipe steampipe query "
  select 'aws' as cloud, instance_id as id, instance_type as size, region
  from aws_ec2_instance where instance_state = 'running'
  union all
  select 'azure', id, size, location
  from azure_compute_virtual_machine where power_state = 'running'
  order by cloud, region
"
```

--------------------------------

### Run Steampipe and Powerpipe Together

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Demonstrates setting up Steampipe as a query backend and Powerpipe as a dashboard frontend using Docker Compose. Includes installing plugins and mods.

```bash
# See examples/docker-compose-with-powerpipe.yml
docker compose -f examples/docker-compose-with-powerpipe.yml up -d

# Install AWS plugin in Steampipe
docker compose exec steampipe steampipe plugin install aws

# Install a compliance mod in Powerpipe
docker compose exec powerpipe powerpipe mod install github.com/turbot/steampipe-mod-aws-compliance

# Open the dashboard
open http://localhost:9033
```

--------------------------------

### Connect to Steampipe with psql

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Provides examples for connecting to Steampipe using the psql client, including interactive connections, running queries from files, and executing one-liners with CSV output.

```bash
# Connect interactively
psql -h localhost -p 9193 -U steampipe -d steampipe

# Run a file
psql -h localhost -p 9193 -U steampipe -d steampipe -f my-query.sql

# One-liner with output formatting
psql -h localhost -p 9193 -U steampipe -d steampipe \
  -c "select name, region from aws_s3_bucket" \
  --csv > buckets.csv
```

--------------------------------

### Conventional Commit Examples

Source: https://github.com/devops-ia/steampipe/blob/main/CONTRIBUTING.md

Examples of commit messages following the Conventional Commits specification.

```text
feat: add support for multi-arch builds
fix: correct plugin directory permissions
chore: bump steampipe to 2.5.0
docs: add GCP plugin configuration example
```

--------------------------------

### Manage Steampipe and Powerpipe services via CLI

Source: https://context7.com/devops-ia/steampipe/llms.txt

Commands to initialize the services and install plugins or mods within the containers.

```bash
# Start both services
docker compose -f docker-compose-with-powerpipe.yml up -d

# Install AWS plugin in Steampipe
docker compose exec steampipe steampipe plugin install aws

# Install AWS compliance mod in Powerpipe
docker compose exec powerpipe powerpipe mod install github.com/turbot/steampipe-mod-aws-compliance

# Open the dashboard at http://localhost:9033
```

--------------------------------

### Install Steampipe Plugins

Source: https://context7.com/devops-ia/steampipe/llms.txt

Installs Steampipe plugins for querying specific cloud providers. Plugins are installed into the running Steampipe service container and persisted via Docker volumes.

```bash
docker exec steampipe steampipe plugin install aws
```

```bash
docker exec steampipe steampipe plugin install aws azure gcp
```

```bash
docker exec steampipe steampipe plugin list
```

--------------------------------

### Run Unit Tests

Source: https://github.com/devops-ia/steampipe/blob/main/CONTRIBUTING.md

Install test dependencies and execute the pytest suite.

```bash
pip install -r tests/requirements.txt
python3 -m pytest tests/ --cov=compare_snapshots --cov-report=term-missing
```

--------------------------------

### Connect to Steampipe in Kubernetes

Source: https://context7.com/devops-ia/steampipe/llms.txt

Examples of connecting to the Steampipe service running in Kubernetes, both from within the same namespace and from any namespace using the full DNS name.

```bash
# Connect from another pod in the same namespace
psql -h steampipe -p 9193 -U steampipe -d steampipe

# Using full service DNS from any namespace
psql -h steampipe.steampipe.svc.cluster.local -p 9193 -U steampipe -d steampipe
```

--------------------------------

### Run Steampipe with AWS Credentials

Source: https://context7.com/devops-ia/steampipe/llms.txt

Starts the Steampipe container using either environment variables or mounted credential files.

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -v "$PWD/aws.spc:/home/steampipe/.steampipe/config/aws.spc:ro" \
  -e AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
  -e AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
  -e AWS_DEFAULT_REGION=us-east-1 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -v "$HOME/.aws:/home/steampipe/.aws:ro" \
  -v "$PWD/aws.spc:/home/steampipe/.steampipe/config/aws.spc:ro" \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Wait for Steampipe Service Readiness

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

When connecting to a newly started Steampipe service, it may take up to 30 seconds for PostgreSQL to bootstrap. Use this loop to wait until the service is ready before attempting to connect.

```bash
# Wait for the service to be ready
until docker exec steampipe pg_isready -h localhost -p 9193 -U steampipe; do
  echo "Waiting for Steampipe..."; sleep 5
done
```

--------------------------------

### Query Multiple Clouds (AWS vs Azure)

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Compares running virtual machines across AWS and Azure by querying both cloud providers and uniting the results. Requires AWS and Azure plugins to be installed.

```bash
docker exec steampipe steampipe plugin install aws azure gcp

# AWS vs Azure: compare running VMs
docker exec steampipe steampipe query "
  select 'aws' as cloud, instance_id as id, instance_type as size, region
  from aws_ec2_instance where instance_state = 'running'
  union all
  select 'azure', id, size, location
  from azure_compute_virtual_machine where power_state = 'running'
  order by cloud, region
"
```

--------------------------------

### Deploy Steampipe to Kubernetes with Helm

Source: https://context7.com/devops-ia/steampipe/llms.txt

Install or upgrade Steampipe on Kubernetes using the official Helm chart. This includes adding the repository, installing with basic configuration, and upgrading to a new version.

```bash
# Add the Helm repository
helm repo add devops-ia https://devops-ia.github.io/helm-charts
helm repo update

# Install with basic configuration
helm install steampipe devops-ia/steampipe \
  --set image.repository=ghcr.io/devops-ia/steampipe \
  --set image.tag=2.4.1 \
  --set bbdd.enabled=true \
  --set bbdd.listen=network \
  --namespace steampipe \
  --create-namespace

# Upgrade to a new version
helm upgrade steampipe devops-ia/steampipe \
  --set image.tag=2.5.0 \
  --reuse-values
```

--------------------------------

### Run Steampipe as a PostgreSQL Service

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Start Steampipe as a persistent background service accessible on port 9193. Use `--foreground` to keep the container running and `--database-listen network` to allow external connections.

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Run Steampipe as a PostgreSQL Service

Source: https://context7.com/devops-ia/steampipe/llms.txt

Starts Steampipe as a background service, exposing a PostgreSQL-compatible endpoint on port 9193. A named volume is used for plugin persistence. Ensure the STEAMPIPE_DATABASE_PASSWORD environment variable is set.

```bash
docker volume create steampipe-data
docker run -d --name steampipe \
  -p 9193:9193 \
  -v steampipe-data:/home/steampipe/.steampipe \
  -e STEAMPIPE_DATABASE_PASSWORD=mypassword \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

```bash
docker logs steampipe
```

```bash
docker exec steampipe pg_isready -h localhost -p 9193 -U steampipe
```

--------------------------------

### Query AWS S3 Buckets

Source: https://context7.com/devops-ia/steampipe/llms.txt

Executes SQL queries against AWS S3 buckets using the Steampipe Docker service. Requires the AWS plugin to be installed.

```bash
docker exec steampipe steampipe query 
  "select name, region, creation_date from aws_s3_bucket order by creation_date desc"
```

```bash
docker exec steampipe steampipe query 
  "select name, region from aws_s3_bucket where bucket_policy_is_public = true"
```

--------------------------------

### Enable External Connections to Steampipe Database

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

To allow external connections to the Steampipe PostgreSQL database within a container, the `--database-listen network` flag must be included during service startup. This example also shows how to publish the port.

```bash
# Must include --database-listen network to accept external connections
docker run -d -p 9193:9193 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Configure Steampipe Helm Deployment

Source: https://context7.com/devops-ia/steampipe/llms.txt

Define custom resources, environment variables, and plugins for Steampipe's Helm deployment in a values.yaml file. This configuration is applied during Helm installation.

```yaml
# values.yaml
image:
  repository: ghcr.io/devops-ia/steampipe
  tag: "2.4.1"

bbdd:
  enabled: true
  listen: network
  port: 9193

resources:
  requests:
    cpu: "250m"
    memory: "512Mi"
  limits:
    cpu: "2000m"
    memory: "2Gi"

env:
  - name: STEAMPIPE_MEMORY_MAX_MB
    value: "1536"
  - name: STEAMPIPE_PLUGIN_MEMORY_MAX_MB
    value: "1024"
  - name: STEAMPIPE_DATABASE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: steampipe-credentials
        key: password

initContainer:
  plugins:
    - aws
    - azure
    - gcp
```

```bash
helm install steampipe devops-ia/steampipe -f values.yaml \
  --namespace steampipe --create-namespace
```

--------------------------------

### Fix Volume Permissions in Kubernetes

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

In Kubernetes, use an `initContainer` to set the correct ownership for the Steampipe data volume before the main container starts. This ensures the container can write to the volume.

```yaml
initContainers:
  - name: fix-permissions
    image: busybox
    command: ["chown", "-R", "9193:0", "/home/steampipe/.steampipe"]
    volumeMounts:
      - name: steampipe-data
        mountPath: /home/steampipe/.steampipe
```

--------------------------------

### Persist Steampipe Data with Named Volumes

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

To prevent plugins from being lost when a container is removed, use a named volume for the Steampipe home directory (`/home/steampipe/.steampipe`). This ensures that installed plugins survive container recreation.

```bash
# Use a named volume for the Steampipe home directory
docker volume create steampipe-data

docker run -d --name steampipe \
  -p 9193:9193 \
  -v steampipe-data:/home/steampipe/.steampipe \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

# Plugins installed now survive container recreation
docker exec steampipe steampipe plugin install aws
```

--------------------------------

### Fix Volume Ownership for Steampipe Container

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

When mounting a volume for Steampipe's home directory, ensure the ownership is correct. The container runs as UID 9193, so the host directory should be owned by this UID. This example shows how to fix ownership on the host.

```bash
# Fix ownership on the host before mounting
sudo chown -R 9193:0 ./steampipe-data

# Or use Docker's --user flag is NOT recommended — always run as UID 9193
```

--------------------------------

### Connect to Steampipe from Node.js

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Shows how to connect to Steampipe using the 'pg' library in Node.js, execute a query for S3 buckets, and log the results.

```javascript
const { Client } = require("pg");

const client = new Client({
  host: "localhost",
  port: 9193,
  database: "steampipe",
  user: "steampipe",
  password: "your-password",
  ssl: false,
});

await client.connect();
const res = await client.query("SELECT name, region FROM aws_s3_bucket");
console.log(res.rows);
await client.end();
```

--------------------------------

### Run Container Structure Tests

Source: https://github.com/devops-ia/steampipe/blob/main/CONTRIBUTING.md

Verify the structure of the built image against a configuration file.

```bash
docker run --rm \
  -v "$PWD/structure-tests.yaml:/structure-tests.yaml:ro" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gcr.io/gcp-runtimes/container-structure-test:latest \
  test --image steampipe:dev --config /structure-tests.yaml
```

--------------------------------

### Connect to Steampipe from Python

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Demonstrates how to connect to Steampipe using Python's psycopg2 library and execute a query to fetch S3 bucket names and regions.

```python
import psycopg2

conn = psycopg2.connect(
    host="localhost",
    port=9193,
    dbname="steampipe",
    user="steampipe",
    password="your-password",
    sslmode="disable",
)
cur = conn.cursor()
cur.execute("SELECT name, region FROM aws_s3_bucket")
for row in cur.fetchall():
    print(row)
conn.close()
```

--------------------------------

### Connect to Steampipe with psql

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Connect to the running Steampipe PostgreSQL service using the `psql` command-line client.

```bash
# psql
psql -h localhost -p 9193 -U steampipe -d steampipe
```

--------------------------------

### GCP plugin configuration and execution

Source: https://github.com/devops-ia/steampipe/blob/main/docs/configuration.md

Configures a GCP connection and runs the container with mounted service account credentials.

```hcl
connection "gcp" {
  plugin  = "gcp"
  project = "my-project-id"
}
```

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -v "$PWD/gcp.spc:/home/steampipe/.steampipe/config/gcp.spc:ro" \
  -v "$PWD/service-account.json:/home/steampipe/.config/gcloud/application_default_credentials.json:ro" \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Build Steampipe Docker Image

Source: https://github.com/devops-ia/steampipe/blob/main/CONTRIBUTING.md

Build the image locally using the default version or a specific Steampipe version.

```bash
# Build with the default Steampipe version from the Dockerfile
docker build -t steampipe:dev .

# Build with a specific version
docker build --build-arg STEAMPIPE_VERSION=2.4.1 -t steampipe:dev .
```

--------------------------------

### Run Security Scan

Source: https://github.com/devops-ia/steampipe/blob/main/CONTRIBUTING.md

Scan the image for critical vulnerabilities using Trivy.

```bash
docker run --rm \
  -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy image --severity CRITICAL --ignore-unfixed steampipe:dev
```

--------------------------------

### Configure Steampipe with Custom Values

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Define custom resource limits, environment variables, and database settings in a values file.

```yaml
# values.yaml
image:
  repository: ghcr.io/devops-ia/steampipe
  tag: "2.4.1"

bbdd:
  enabled: true
  listen: network
  port: 9193

resources:
  requests:
    cpu: "250m"
    memory: "512Mi"
  limits:
    cpu: "2000m"
    memory: "2Gi"

env:
  - name: STEAMPIPE_MEMORY_MAX_MB
    value: "1536"
  - name: STEAMPIPE_PLUGIN_MEMORY_MAX_MB
    value: "1024"
  - name: STEAMPIPE_DATABASE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: steampipe-credentials
        key: password
```

```bash
helm install steampipe devops-ia/steampipe -f values.yaml \
  --namespace steampipe --create-namespace
```

--------------------------------

### Configure Health Checks

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Define liveness and readiness probes using the pg_isready utility to monitor the PostgreSQL endpoint.

```yaml
livenessProbe:
  exec:
    command:
      - pg_isready
      - -h
      - localhost
      - -p
      - "9193"
      - -U
      - steampipe
  initialDelaySeconds: 30
  periodSeconds: 10

readinessProbe:
  exec:
    command:
      - pg_isready
      - -h
      - localhost
      - -p
      - "9193"
      - -U
      - steampipe
  initialDelaySeconds: 15
  periodSeconds: 5
```

--------------------------------

### Run Steampipe as a Query Shell

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Execute a one-off interactive SQL session with Steampipe using Docker.

```bash
docker run -it --rm \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe query
```

--------------------------------

### Verify Steampipe Query Engine

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Execute a simple SQL query through Docker to verify the Steampipe query engine is functioning correctly.

```bash
docker exec steampipe steampipe query "select 1 as test"
# Expected output:
# +------+
# | test |
# +------+
# | 1    |
# +------+
```

--------------------------------

### Run Steampipe with AWS environment variables

Source: https://github.com/devops-ia/steampipe/blob/main/docs/configuration.md

Mounts an AWS configuration file and provides credentials via environment variables.

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -v "$PWD/aws.spc:/home/steampipe/.steampipe/config/aws.spc:ro" \
  -e AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
  -e AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
  -e AWS_DEFAULT_REGION=us-east-1 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Configure Steampipe with environment variables

Source: https://context7.com/devops-ia/steampipe/llms.txt

Adjust runtime behavior such as caching, logging, and timeouts using environment variables.

```bash
# Run with custom environment configuration
docker run -d --name steampipe \
  -p 9193:9193 \
  -e STEAMPIPE_DATABASE_PASSWORD=supersecret \
  -e STEAMPIPE_UPDATE_CHECK=false \
  -e STEAMPIPE_TELEMETRY=none \
  -e STEAMPIPE_LOG_LEVEL=warn \
  -e STEAMPIPE_MEMORY_MAX_MB=2048 \
  -e STEAMPIPE_PLUGIN_MEMORY_MAX_MB=1024 \
  -e STEAMPIPE_CACHE=true \
  -e STEAMPIPE_CACHE_TTL=300 \
  -e STEAMPIPE_QUERY_TIMEOUT=240 \
  -e STEAMPIPE_MAX_PARALLEL=10 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

# Environment variable reference:
# STEAMPIPE_DATABASE_PASSWORD  - PostgreSQL password (default: random)
# STEAMPIPE_UPDATE_CHECK       - Disable update checking (default: false)
# STEAMPIPE_TELEMETRY          - Disable telemetry (default: none)
# STEAMPIPE_LOG_LEVEL          - Log verbosity: trace/debug/info/warn/error (default: warn)
# STEAMPIPE_MEMORY_MAX_MB      - Process memory soft limit in MB (default: 1024)
# STEAMPIPE_PLUGIN_MEMORY_MAX_MB - Per-plugin memory limit in MB (default: 1024)
# STEAMPIPE_CACHE              - Enable query result cache (default: true)
# STEAMPIPE_CACHE_TTL          - Cache TTL in seconds (default: 300)
# STEAMPIPE_QUERY_TIMEOUT      - Query timeout in seconds (default: 240)
# STEAMPIPE_MAX_PARALLEL       - Maximum parallel executions (default: 10)
```

--------------------------------

### Run Steampipe Interactive Query Shell

Source: https://context7.com/devops-ia/steampipe/llms.txt

Launches an interactive SQL session within a temporary Docker container. The container is automatically removed upon exit.

```bash
docker run -it --rm \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe query
```

```bash
docker run -it --rm \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe query "select 1 as test"
```

--------------------------------

### Create AWS Configuration File

Source: https://context7.com/devops-ia/steampipe/llms.txt

Defines the AWS plugin connection and target regions in a .spc file.

```hcl
cat > aws.spc << 'EOF'
connection "aws" {
  plugin  = "aws"
  regions = ["us-east-1", "eu-west-1", "ap-southeast-1"]
}
EOF
```

--------------------------------

### Set a fixed database password

Source: https://github.com/devops-ia/steampipe/blob/main/docs/configuration.md

Configures a custom database password and enables network listening for the Steampipe service.

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -e STEAMPIPE_DATABASE_PASSWORD=supersecret \
  -e STEAMPIPE_DATABASE_LISTEN=network \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Connect to Steampipe with psql

Source: https://context7.com/devops-ia/steampipe/llms.txt

Connects to the Steampipe PostgreSQL-compatible endpoint using the psql client. Supports running query files and exporting results to CSV.

```bash
psql -h localhost -p 9193 -U steampipe -d steampipe
```

```bash
psql -h localhost -p 9193 -U steampipe -d steampipe -f my-query.sql
```

```bash
psql -h localhost -p 9193 -U steampipe -d steampipe \
  -c "select name, region from aws_s3_bucket" \
  --csv > buckets.csv
```

--------------------------------

### Connect to Steampipe from Another Pod

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Use psql to connect to the Steampipe service from within the cluster.

```bash
# From any pod in the same namespace
psql -h steampipe -p 9193 -U steampipe -d steampipe

# Using the full service DNS
psql -h steampipe.steampipe.svc.cluster.local -p 9193 -U steampipe -d steampipe
```

--------------------------------

### Wait for Steampipe Service Readiness

Source: https://context7.com/devops-ia/steampipe/llms.txt

A script to wait for the Steampipe service to be fully ready before executing queries. It checks for PostgreSQL connection readiness using `docker exec` and `pg_isready`.

```bash
# Wait for PostgreSQL to accept connections
until docker exec steampipe pg_isready -h localhost -p 9193 -U steampipe; do
  echo "Waiting for Steampipe..."; sleep 5
done
echo "Steampipe is ready!"

# Now safe to run queries
docker exec steampipe steampipe plugin install aws
```

--------------------------------

### Steampipe Connection Settings for SQL Clients

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Connection details for PostgreSQL-compatible SQL clients like DBeaver, TablePlus, and DataGrip.

```text
# DBeaver, TablePlus, DataGrip — use these connection settings:
# Host:     localhost
# Port:     9193
# Database: steampipe
# User:     steampipe
# Password: (see STEAMPIPE_DATABASE_PASSWORD or docker logs)
```

--------------------------------

### Verify Steampipe Service

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Check the Docker logs for the Steampipe container to confirm the database is running.

```bash
docker logs steampipe
# Look for: "Database is now running"
```

--------------------------------

### AWS plugin configuration

Source: https://github.com/devops-ia/steampipe/blob/main/docs/configuration.md

Defines an AWS connection in an HCL file.

```hcl
connection "aws" {
  plugin  = "aws"
  regions = ["us-east-1", "eu-west-1"]
}
```

--------------------------------

### Export Query Results to Markdown Table

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Demonstrates exporting query results into a Markdown table format, useful for documentation or reports.

```bash
# Markdown table
docker exec steampipe steampipe query \
  "select name, region from aws_s3_bucket limit 10" --output table
```

--------------------------------

### Export Query Results to JSON and CSV

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Shows how to export the results of a Steampipe query to JSON and CSV files using the --output flag.

```bash
# JSON
docker exec steampipe steampipe query \
  "select * from aws_s3_bucket" --output json > buckets.json

# CSV
docker exec steampipe steampipe query \
  "select * from aws_s3_bucket" --output csv > buckets.csv
```

--------------------------------

### Mount Plugin Configuration via ConfigMap

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Use a ConfigMap to store plugin configuration files and mount them into the Steampipe container.

```yaml
# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: steampipe-plugin-config
  namespace: steampipe
data:
  aws.spc: |
    connection "aws" {
      plugin  = "aws"
      regions = ["us-east-1", "eu-west-1"]
    }
```

```yaml
extraVolumes:
  - name: plugin-config
    configMap:
      name: steampipe-plugin-config

extraVolumeMounts:
  - name: plugin-config
    mountPath: /home/steampipe/.steampipe/config/aws.spc
    subPath: aws.spc
    readOnly: true
```

--------------------------------

### Run Steampipe with mounted AWS profile

Source: https://github.com/devops-ia/steampipe/blob/main/docs/configuration.md

Mounts the local AWS credentials directory into the container.

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -v "$HOME/.aws:/home/steampipe/.aws:ro" \
  -v "$PWD/aws.spc:/home/steampipe/.steampipe/config/aws.spc:ro" \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Query AWS EC2 Instances

Source: https://context7.com/devops-ia/steampipe/llms.txt

Retrieves a list of EC2 instances, ordered by their state, using a SQL query executed via the Steampipe Docker service.

```bash
docker exec steampipe steampipe query 
  "select instance_id, instance_type, instance_state, region from aws_ec2_instance order by instance_state"
```

--------------------------------

### Configure GCP Plugin

Source: https://context7.com/devops-ia/steampipe/llms.txt

Defines the GCP plugin connection with a project ID.

```hcl
# gcp.spc
connection "gcp" {
  plugin  = "gcp"
  project = "my-project-id"
}
```

--------------------------------

### Tune memory for large datasets

Source: https://context7.com/devops-ia/steampipe/llms.txt

Increase container memory limits and Steampipe internal settings for heavy workloads.

```bash
# Increase memory limits for heavy workloads
docker run -d --name steampipe \
  --memory=6g --memory-swap=6g \
  -p 9193:9193 \
  -e STEAMPIPE_MEMORY_MAX_MB=4096 \
  -e STEAMPIPE_PLUGIN_MEMORY_MAX_MB=2048 \
  -e STEAMPIPE_MAX_PARALLEL=20 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Lint Dockerfile

Source: https://github.com/devops-ia/steampipe/blob/main/CONTRIBUTING.md

Validate the Dockerfile using hadolint.

```bash
docker run --rm -i hadolint/hadolint < Dockerfile
```

--------------------------------

### Deploy with Docker Compose

Source: https://context7.com/devops-ia/steampipe/llms.txt

Defines a Docker Compose service for Steampipe with persistent storage and health checks.

```yaml
# docker-compose.yml
services:
  steampipe:
    image: ghcr.io/devops-ia/steampipe:2.4.1
    container_name: steampipe
    command: steampipe service start --foreground --database-listen network
    ports:
      - "9193:9193"
    volumes:
      - steampipe-data:/home/steampipe/.steampipe
    environment:
      STEAMPIPE_DATABASE_PASSWORD: steampipe
      STEAMPIPE_UPDATE_CHECK: "false"
      STEAMPIPE_TELEMETRY: none
      STEAMPIPE_LOG_LEVEL: warn
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h localhost -p 9193 -U steampipe"]
      interval: 10s
      timeout: 5s
      retries: 10
      start_period: 30s
    restart: unless-stopped

volumes:
  steampipe-data:
```

```bash
# Start the service
docker compose up -d

# Install plugins
docker compose exec steampipe steampipe plugin install aws

# View logs
docker compose logs -f steampipe
```

--------------------------------

### Enable Steampipe Debug Logging

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

To diagnose unexpected behavior, enable debug logging by setting the `STEAMPIPE_LOG_LEVEL` environment variable to `debug`. View the detailed logs using `docker logs -f steampipe`.

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -e STEAMPIPE_LOG_LEVEL=debug \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

docker logs -f steampipe
```

--------------------------------

### List EC2 Instances by State

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Retrieves a list of EC2 instances, including their ID, type, state, and region, ordered by their current state.

```bash
docker exec steampipe steampipe query \
  "select instance_id, instance_type, instance_state, region from aws_ec2_instance order by instance_state"
```

--------------------------------

### Export query results

Source: https://context7.com/devops-ia/steampipe/llms.txt

Output query results to JSON, CSV, or table formats using the CLI.

```bash
# Export to JSON
docker exec steampipe steampipe query \
  "select * from aws_s3_bucket" --output json > buckets.json

# Export to CSV
docker exec steampipe steampipe query \
  "select * from aws_s3_bucket" --output csv > buckets.csv

# Output as formatted table
docker exec steampipe steampipe query \
  "select name, region from aws_s3_bucket limit 10" --output table
```

--------------------------------

### Upgrade Steampipe Image Version

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Update the existing Steampipe deployment to a new image tag while preserving previous values.

```bash
helm upgrade steampipe devops-ia/steampipe \
  --set image.tag=2.5.0 \
  --reuse-values
```

--------------------------------

### Define Steampipe and Powerpipe services in Docker Compose

Source: https://context7.com/devops-ia/steampipe/llms.txt

Use this configuration to run Steampipe and Powerpipe as linked services in a Docker environment.

```yaml
services:
  steampipe:
    image: ghcr.io/devops-ia/steampipe:2.4.1
    container_name: steampipe
    command: steampipe service start --foreground --database-listen network
    ports:
      - "9193:9193"
    volumes:
      - steampipe-data:/home/steampipe/.steampipe
    environment:
      STEAMPIPE_DATABASE_PASSWORD: steampipe
      STEAMPIPE_UPDATE_CHECK: "false"
      STEAMPIPE_TELEMETRY: none
      STEAMPIPE_LOG_LEVEL: warn
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h localhost -p 9193 -U steampipe"]
      interval: 10s
      timeout: 5s
      retries: 10
      start_period: 30s
    restart: unless-stopped

  powerpipe:
    image: ghcr.io/devops-ia/powerpipe:1.5.1
    container_name: powerpipe
    ports:
      - "9033:9033"
    volumes:
      - workspace:/workspace
    environment:
      POWERPIPE_DATABASE: "postgres://steampipe:steampipe@steampipe:9193/steampipe"
      POWERPIPE_UPDATE_CHECK: "false"
      POWERPIPE_TELEMETRY: none
      POWERPIPE_LISTEN: network
    depends_on:
      steampipe:
        condition: service_healthy
    restart: unless-stopped

volumes:
  steampipe-data:
  workspace:
```

--------------------------------

### OpenShift Security Context Configuration

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

The image is compatible with OpenShift restricted SCC by default, requiring no securityContext overrides.

```yaml
# No securityContext overrides needed for OpenShift restricted SCC
securityContext: {}
```

--------------------------------

### Enable Debug Logging for Steampipe

Source: https://context7.com/devops-ia/steampipe/llms.txt

Run Steampipe in a Docker container with debug logging enabled by setting the `STEAMPIPE_LOG_LEVEL` environment variable. This is useful for troubleshooting.

```bash
# Run with debug logging
docker run -d --name steampipe \
  -p 9193:9193 \
  -e STEAMPIPE_LOG_LEVEL=debug \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network

# Follow the logs
docker logs -f steampipe
```

--------------------------------

### Run Steampipe Container in Foreground

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

When running Steampipe as a service in a container, use the `--foreground` flag. This flag is essential for container orchestration systems to keep the container running.

```bash
# Wrong — exits after starting the background daemon
docker run ghcr.io/devops-ia/steampipe:2.4.1 steampipe service start

# Correct — blocks in foreground (required for containers)
docker run ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Pull Steampipe Docker Images

Source: https://github.com/devops-ia/steampipe/blob/main/README.md

Commands to pull the Steampipe image from GitHub Container Registry or Docker Hub.

```bash
# GitHub Container Registry
docker pull ghcr.io/devops-ia/steampipe:2.4.1

# Docker Hub
docker pull devopsiaci/steampipe:2.4.1
```

--------------------------------

### Find Public S3 Buckets

Source: https://github.com/devops-ia/steampipe/blob/main/docs/examples.md

Queries for S3 buckets that have a public access policy. This helps identify potential security risks.

```bash
docker exec steampipe steampipe query \
  "select name, region from aws_s3_bucket where bucket_policy_is_public = true"
```

--------------------------------

### Configure Multi-Account AWS Aggregator

Source: https://context7.com/devops-ia/steampipe/llms.txt

Defines multiple AWS connections and an aggregator to query them simultaneously.

```hcl
# multi-account-aws.spc
connection "aws_dev" {
  plugin  = "aws"
  profile = "dev"
  regions = ["us-east-1"]
}

connection "aws_prod" {
  plugin  = "aws"
  profile = "prod"
  regions = ["us-east-1", "eu-west-1"]
}

# Aggregator combines all accounts into a single connection
connection "aws_all" {
  plugin      = "aws"
  type        = "aggregator"
  connections = ["aws_dev", "aws_prod"]
}
```

--------------------------------

### Memory tuning for Steampipe

Source: https://github.com/devops-ia/steampipe/blob/main/docs/configuration.md

Adjusts memory limits and parallel execution settings for high-load environments.

```bash
docker run -d --name steampipe \
  -p 9193:9193 \
  -e STEAMPIPE_MEMORY_MAX_MB=4096 \
  -e STEAMPIPE_PLUGIN_MEMORY_MAX_MB=2048 \
  -e STEAMPIPE_MAX_PARALLEL=20 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

```bash
docker run -d --name steampipe \
  --memory=6g --memory-swap=6g \
  -p 9193:9193 \
  -e STEAMPIPE_MEMORY_MAX_MB=4096 \
  ghcr.io/devops-ia/steampipe:2.4.1 \
  steampipe service start --foreground --database-listen network
```

--------------------------------

### Configure multiple connections with an aggregator

Source: https://github.com/devops-ia/steampipe/blob/main/docs/configuration.md

Groups multiple AWS connections into a single aggregator connection.

```hcl
connection "aws_dev" {
  plugin  = "aws"
  regions = ["us-east-1"]
  # profile = "dev"
}

connection "aws_prod" {
  plugin  = "aws"
  regions = ["us-east-1", "eu-west-1"]
  # profile = "prod"
}

connection "aws_all" {
  plugin      = "aws"
  type        = "aggregator"
  connections = ["aws_dev", "aws_prod"]
}
```

--------------------------------

### Query AWS IAM Users for MFA Status

Source: https://context7.com/devops-ia/steampipe/llms.txt

Finds IAM users who have console access enabled but do not have MFA configured, sorted by creation date. This query helps identify potential security risks.

```sql
SELECT
  user_name,
  create_date,
  password_last_used
FROM aws_iam_user
WHERE
  password_enabled = true
  AND mfa_enabled = false
ORDER BY create_date
```

--------------------------------

### Pull Steampipe Docker Image

Source: https://github.com/devops-ia/steampipe/blob/main/docs/getting-started.md

Pull the latest Steampipe Docker image from either GitHub Container Registry or Docker Hub.

```bash
# GitHub Container Registry (recommended)
docker pull ghcr.io/devops-ia/steampipe:2.4.1
```

```bash
# Docker Hub
docker pull devopsiaci/steampipe:2.4.1
```

--------------------------------

### Inject Credentials via Kubernetes Secrets

Source: https://github.com/devops-ia/steampipe/blob/main/docs/kubernetes.md

Define a Kubernetes Secret and reference it in the Helm values to securely inject credentials.

```yaml
# secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: aws-credentials
  namespace: steampipe
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: "AKIAIOSFODNN7EXAMPLE"
  AWS_SECRET_ACCESS_KEY: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
  AWS_DEFAULT_REGION: "us-east-1"
```

```yaml
envFrom:
  - secretRef:
      name: aws-credentials
```

--------------------------------

### Configure Kubernetes Memory Resources for Steampipe

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

In Kubernetes, set memory requests and limits for the Steampipe pod. Additionally, configure Steampipe's internal memory usage with environment variables `STEAMPIPE_MEMORY_MAX_MB` and `STEAMPIPE_PLUGIN_MEMORY_MAX_MB`.

```yaml
resources:
  requests:
    memory: "1Gi"
  limits:
    memory: "4Gi"
env:
  - name: STEAMPIPE_MEMORY_MAX_MB
    value: "3072"
  - name: STEAMPIPE_PLUGIN_MEMORY_MAX_MB
    value: "2048"
```

--------------------------------

### Kubernetes ConfigMap for Steampipe Plugin Configuration

Source: https://context7.com/devops-ia/steampipe/llms.txt

Mount plugin configuration files from a Kubernetes ConfigMap into the Steampipe container. This allows for externalizing and managing plugin settings.

```yaml
# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: steampipe-plugin-config
  namespace: steampipe
data:
  aws.spc: |
    connection "aws" {
      plugin  = "aws"
      regions = ["us-east-1", "eu-west-1"]
    }
```

```yaml
# Reference in Helm values
extraVolumes:
  - name: plugin-config
    configMap:
      name: steampipe-plugin-config

extraVolumeMounts:
  - name: plugin-config
    mountPath: /home/steampipe/.steampipe/config/aws.spc
    subPath: aws.spc
    readOnly: true
```

--------------------------------

### Publish Steampipe Port

Source: https://github.com/devops-ia/steampipe/blob/main/docs/troubleshooting.md

Ensure the Steampipe PostgreSQL port (default 9193) is published from the container to the host using the `-p` flag in `docker run`.

```bash
# Publish the port explicitly
docker run -d -p 9193:9193 ...
```
Tokens:
Raw