# S3Scanner

S3Scanner is a security reconnaissance tool designed to find open and misconfigured S3 buckets across AWS and other S3-compatible cloud storage providers. The tool supports multi-threaded scanning, permission enumeration, and can discover security misconfigurations in bucket access control lists (ACLs) for both anonymous and authenticated users.

The scanner works with multiple cloud providers including AWS, DigitalOcean, DreamHost, Google Cloud Platform (GCP), Linode, Scaleway, and custom S3-compatible endpoints. It can output results in human-readable or JSON format, save results to a PostgreSQL database, and integrate with RabbitMQ for large-scale automated scanning operations.

## Basic Bucket Scanning

Scan a single bucket to check if it exists and identify permission misconfigurations. The tool checks READ, WRITE, READ_ACP, WRITE_ACP, and FULL_CONTROL permissions for both public users and authenticated AWS users.

```bash
# Scan a single bucket in AWS (default provider)
s3scanner -bucket my-test-bucket

# Example output:
# exists    | my-test-bucket | us-east-1 | AuthUsers: [] | AllUsers: [READ]
```

## Batch Scanning from File

Scan multiple buckets from a text file containing one bucket name per line. Duplicate bucket names are automatically deduplicated, and invalid bucket names are skipped with a warning.

```bash
# Create a file with bucket names
cat > buckets.txt << EOF
company-assets
backup-files
public-images
EOF

# Scan all buckets in the file
s3scanner -bucket-file buckets.txt

# Example output:
# not_exist | company-assets
# exists    | backup-files | eu-west-1 | AuthUsers: [] | AllUsers: []
# exists    | public-images | us-west-2 | AuthUsers: [] | AllUsers: [READ, READ_ACP]
```

## Multi-threaded Scanning

Increase scanning speed by using multiple threads. Each thread processes buckets in parallel for faster reconnaissance of large bucket lists.

```bash
# Scan with 8 concurrent threads
s3scanner -bucket-file large-bucket-list.txt -threads 8

# Scan a single bucket with verbose output for debugging
s3scanner -bucket sensitive-data -threads 4 -verbose
```

## Object Enumeration

Enumerate all objects stored in publicly readable buckets. This reveals file names, sizes, and calculates total bucket storage used.

```bash
# Enumerate objects in a bucket
s3scanner -bucket public-assets -enumerate

# Example output:
# exists    | public-assets | us-east-1 | AuthUsers: [] | AllUsers: [READ] | 1523 objects (2.4 GB)

# Enumerate with file input
s3scanner -bucket-file targets.txt -enumerate -threads 4
```

## JSON Output Format

Output scan results as JSON for integration with other tools, pipelines, or analysis scripts. Each result is printed as a single JSON object per line.

```bash
# Scan with JSON output
s3scanner -bucket company-backup -json

# Example output:
# {"bucket":{"name":"company-backup","region":"us-east-1","exists":1,"perm_all_users_read":1,...}}

# Filter results with jq - show only existing buckets with their regions
s3scanner -bucket-file names.txt -json | jq -r '. | select(.bucket.exists==1) | [.bucket.name, .bucket.region] | join(" - ")'

# Output:
# company-backup - us-east-1
# public-assets - ap-southeast-1
# staging-files - us-west-2
```

## Scanning Non-AWS Providers

Scan S3-compatible storage services from other cloud providers using the `-provider` flag.

```bash
# Scan DigitalOcean Spaces
s3scanner -bucket my-space -provider digitalocean

# Scan Google Cloud Storage
s3scanner -bucket gcs-bucket -provider gcp

# Scan Linode Object Storage
s3scanner -bucket linode-bucket -provider linode

# Scan Scaleway Object Storage
s3scanner -bucket scaleway-bucket -provider scaleway

# Scan DreamHost Objects
s3scanner -bucket dreamhost-bucket -provider dreamhost

# Batch scan GCP buckets with enumeration
s3scanner -bucket-file gcp-buckets.txt -provider gcp -enumerate
```

## Custom Provider Configuration

Configure custom S3-compatible endpoints for unsupported providers or self-hosted S3 services. Requires a config.yml file.

```yaml
# config.yml - Custom provider configuration for Vultr Object Storage
providers:
  custom:
    address_style: "path"    # "path" or "vhost"
    endpoint_format: "https://$REGION.vultrobjects.com"
    insecure: false
    regions:
      - "ewr1"
      - "sjc1"
      - "ams1"
```

```bash
# Scan using custom provider (config.yml must exist in ., /etc/s3scanner/, or ~/.s3scanner/)
s3scanner -bucket custom-bucket -provider custom

# Scan custom provider with enumeration
s3scanner -bucket-file vultr-buckets.txt -provider custom -enumerate
```

## Database Storage

Save scan results to a PostgreSQL database for persistent storage and later analysis. The schema is auto-migrated on first connection.

```yaml
# config.yml - Database configuration
db:
  uri: "postgresql://user:password@localhost:5432/s3scanner"
```

```bash
# Scan and save results to database
s3scanner -bucket company-data -db

# Batch scan with database storage
s3scanner -bucket-file enterprise-buckets.txt -db -threads 8

# Combine with JSON output for both console and database
s3scanner -bucket-file targets.txt -db -json -enumerate
```

## RabbitMQ Integration

Consume bucket names from a RabbitMQ queue for large-scale distributed scanning. Messages should be JSON-encoded Bucket objects.

```yaml
# config.yml - RabbitMQ configuration
mq:
  queue_name: "s3-scan-queue"
  uri: "amqp://user:pass@rabbitmq-host:5672"
```

```bash
# Start scanner as RabbitMQ consumer
s3scanner -mq -threads 8

# Combine with database storage for distributed scanning
s3scanner -mq -db -enumerate -threads 16
```

## MQ Ingest Utility

Publish bucket names to RabbitMQ queue using the mqingest utility for distributed scanning workflows.

```bash
# Build the mqingest utility
go build -o mqingest ./cmd/mqingest/

# Publish bucket names to queue
./mqingest -file buckets.txt -queue s3-scan-queue -url "amqp://guest:guest@localhost:5672/"

# Output:
# 1500 bucket names published to queue s3-scan-queue
```

## Go Library Usage

Use S3Scanner as a Go library for custom integrations and programmatic bucket scanning.

```go
package main

import (
    "fmt"
    "github.com/sa7mon/s3scanner/bucket"
    "github.com/sa7mon/s3scanner/provider"
)

func main() {
    // Create a new AWS provider
    p, err := provider.NewProvider("aws")
    if err != nil {
        panic(err)
    }

    // Create a bucket to scan
    b := bucket.NewBucket("target-bucket")

    // Check if bucket exists
    result, err := p.BucketExists(&b)
    if err != nil {
        panic(err)
    }

    if result.Exists == bucket.BucketExists {
        fmt.Printf("Bucket exists in region: %s\n", result.Region)

        // Scan permissions
        if err := p.Scan(result, false); err != nil {
            panic(err)
        }

        // Check permissions
        if result.PermAllUsersRead == bucket.PermissionAllowed {
            fmt.Println("WARNING: Bucket is publicly readable!")
        }

        // Enumerate objects if readable
        if result.PermAllUsersRead == bucket.PermissionAllowed {
            if err := p.Enumerate(result); err != nil {
                panic(err)
            }
            fmt.Printf("Found %d objects, total size: %d bytes\n",
                result.NumObjects, result.BucketSize)
        }
    }
}
```

## Bucket Validation

Validate bucket names according to S3 naming rules before scanning to avoid unnecessary API calls.

```go
package main

import (
    "fmt"
    "github.com/sa7mon/s3scanner/bucket"
)

func main() {
    names := []string{
        "valid-bucket-name",
        "ab",                    // Too short (min 3 chars)
        "bucket_with_underscore", // Invalid character
        "xn--invalid-prefix",    // Reserved prefix
        "bucket-s3alias",        // Reserved suffix
        ".starts-with-dot",      // Must start with letter/number
    }

    for _, name := range names {
        if bucket.IsValidS3BucketName(name) {
            fmt.Printf("✓ Valid: %s\n", name)
        } else {
            fmt.Printf("✗ Invalid: %s\n", name)
        }
    }
}
```

## Docker Usage

Run S3Scanner in a Docker container for isolated execution without local installation.

```bash
# Pull and run the latest image
docker run ghcr.io/sa7mon/s3scanner -bucket test-bucket

# Scan with a bucket file (mount local directory)
docker run -v $(pwd):/data ghcr.io/sa7mon/s3scanner -bucket-file /data/buckets.txt

# Use custom config file
docker run -v $(pwd)/config.yml:/etc/s3scanner/config.yml \
    ghcr.io/sa7mon/s3scanner -bucket custom-bucket -provider custom

# Run with database storage
docker run -v $(pwd)/config.yml:/etc/s3scanner/config.yml \
    ghcr.io/sa7mon/s3scanner -bucket-file /data/targets.txt -db -enumerate
```

## Summary

S3Scanner is primarily used for security auditing, penetration testing, and cloud security assessments to identify misconfigured S3 buckets that may expose sensitive data. Security teams can integrate it into their continuous monitoring workflows using the RabbitMQ consumer mode and PostgreSQL storage for tracking bucket permissions over time.

The tool's flexible provider system allows scanning across multiple cloud platforms using a unified interface. For enterprise deployments, the combination of multi-threaded scanning, database persistence, and message queue integration enables large-scale reconnaissance operations. Custom provider support extends functionality to any S3-compatible storage system, making it suitable for both public cloud assessments and internal infrastructure security testing.