### Full polling implementation example Source: https://sw33tlie.github.io/bbscope/library/polling.html A complete example demonstrating how to initialize a database, configure a poller, and execute the polling process with a custom logger. ```go package main import ( "context" "fmt" "log" h1 "github.com/sw33tLie/bbscope/v2/pkg/platforms/hackerone" "github.com/sw33tLie/bbscope/v2/pkg/polling" "github.com/sw33tLie/bbscope/v2/pkg/storage" ) type myLogger struct{} func (myLogger) Infof(f string, a ...interface{}) { log.Printf("[INFO] "+f, a...) } func (myLogger) Warnf(f string, a ...interface{}) { log.Printf("[WARN] "+f, a...) } func (myLogger) Errorf(f string, a ...interface{}) { log.Printf("[ERROR] "+f, a...) } func (myLogger) Debugf(f string, a ...interface{}) {} func main() { db, _ := storage.Open("postgres://user:pass@localhost/bbscope?sslmode=disable") defer db.Close() poller := h1.NewPoller("user", "token") result, err := polling.PollPlatform(context.Background(), polling.PlatformConfig{ Poller: poller, DB: db, Concurrency: 10, Log: myLogger{}, OnProgramDone: func(url string, changes []storage.Change, isFirstRun bool) { if !isFirstRun { for _, c := range changes { fmt.Printf("[%s] %s %s\n", c.ChangeType, c.ProgramURL, c.TargetRaw) } } }, }) if err != nil { log.Fatal(err) } fmt.Printf("Polled %d programs, %d changes\n", len(result.PolledProgramURLs), len(result.ProgramChanges)) } ``` -------------------------------- ### Find programs examples Source: https://sw33tlie.github.io/bbscope/web/api.html Demonstrates searching for programs using specific hostnames. ```bash # Find programs with example.com in scope curl "https://your-instance.com/api/v1/find?q=example.com" # Find via subdomain — expands to root domain matching curl "https://your-instance.com/api/v1/find?q=sub.example.com" ``` -------------------------------- ### Full workflow example Source: https://sw33tlie.github.io/bbscope/library/targets.html Demonstrates opening a database, listing entries, and printing sorted wildcards. ```go db, _ := storage.Open(dbURL) defer db.Close() entries, _ := db.ListEntries(ctx, storage.ListOptions{InScope: true}) wildcards := targets.CollectWildcardsSorted(entries, false) for _, w := range wildcards { fmt.Println(w) } ``` -------------------------------- ### Install bbscope via Go Source: https://sw33tlie.github.io/bbscope/getting-started/installation.html Requires Go 1.24 or later to install the latest version directly from the repository. ```bash go install github.com/sw33tLie/bbscope/v2@latest ``` -------------------------------- ### Deploy bbscope with Docker Compose Source: https://sw33tlie.github.io/bbscope/web/docker.html Standard commands to initialize the environment and start the services. ```bash cd website/ cp .env.example .env # Edit .env with your credentials docker compose up -d ``` -------------------------------- ### Install bbscope dependency Source: https://sw33tlie.github.io/bbscope/library/overview.html Use this command to add the bbscope library to your Go project. ```bash go get github.com/sw33tLie/bbscope/v2@latest ``` -------------------------------- ### Start the bbscope web server Source: https://sw33tlie.github.io/bbscope/web/self-hosting.html Launches the web interface with development mode enabled and a custom listen address. ```bash bbscope serve --dev --listen :8080 ``` -------------------------------- ### Scope updates example Source: https://sw33tlie.github.io/bbscope/web/api.html Demonstrates retrieving scope changes from the last 7 days. ```bash # Changes in the last 7 days curl "https://your-instance.com/api/v1/updates?since=7d" ``` -------------------------------- ### Initializing platform pollers Source: https://sw33tlie.github.io/bbscope/library/platforms.html Examples of how to initialize different platform pollers, noting that some require explicit authentication while others do not. ```go // HackerOne — credentials passed at construction poller := h1.NewPoller("username", "api_token") // Bugcrowd — authenticate after construction poller := &bc.Poller{} err := poller.Authenticate(ctx, platforms.AuthConfig{ Email: "...", Password: "...", OtpSecret: "...", }) // Bugcrowd public-only (no auth) poller := bc.NewPollerPublicOnly() // Intigriti — authenticate with token poller := it.NewPoller() err := poller.Authenticate(ctx, platforms.AuthConfig{Token: "..."}) // YesWeHack — authenticate after construction poller := &ywh.Poller{} err := poller.Authenticate(ctx, platforms.AuthConfig{ Email: "...", Password: "...", OtpSecret: "...", }) // Immunefi — no auth needed poller := imm.NewPoller() ``` -------------------------------- ### Run PostgreSQL via Docker Source: https://sw33tlie.github.io/bbscope/database/setup.html Quick setup for a local development database instance using Docker. ```bash docker run -d \ --name bbscope-db \ -e POSTGRES_DB=bbscope \ -e POSTGRES_USER=bbscope \ -e POSTGRES_PASSWORD=secret \ -p 5432:5432 \ postgres:16-alpine ``` ```yaml db: url: "postgres://bbscope:secret@localhost:5432/bbscope?sslmode=disable" ``` -------------------------------- ### Fetch HackerOne program scope Source: https://sw33tlie.github.io/bbscope/library/overview.html Example demonstrating how to initialize a HackerOne poller and iterate through program scope entries. ```go package main import ( "context" "fmt" "log" "github.com/sw33tLie/bbscope/v2/pkg/platforms" h1 "github.com/sw33tLie/bbscope/v2/pkg/platforms/hackerone" ) func main() { ctx := context.Background() poller := h1.NewPoller("your_user", "your_token") handles, err := poller.ListProgramHandles(ctx, platforms.PollOptions{}) if err != nil { log.Fatal(err) } for _, h := range handles { pd, err := poller.FetchProgramScope(ctx, h, platforms.PollOptions{}) if err != nil { log.Printf("error: %s: %v", h, err) continue } for _, s := range pd.InScope { fmt.Printf("%s %s %s\n", pd.Url, s.Category, s.Target) } } } ``` -------------------------------- ### Target extraction examples Source: https://sw33tlie.github.io/bbscope/web/api.html Demonstrates common usage patterns for extracting target data using curl. ```bash # All in-scope wildcards as plain text curl https://your-instance.com/api/v1/targets/wildcards # HackerOne wildcards as JSON curl "https://your-instance.com/api/v1/targets/wildcards?platform=h1&format=json" # Out-of-scope URLs curl "https://your-instance.com/api/v1/targets/urls?scope=out" ``` -------------------------------- ### GET /api/v1/programs Source: https://sw33tlie.github.io/bbscope/web/api.html Returns all programs with their scope data. The response is cached for 5 minutes. ```APIDOC ## GET /api/v1/programs ### Description Returns all programs with their scope data. Response is cached for 5 minutes. ### Method GET ### Endpoint /api/v1/programs ### Parameters #### Query Parameters - **raw** (boolean) - Optional - Set to true to return raw (non-AI-normalized) scope data ``` -------------------------------- ### Extract URLs Source: https://sw33tlie.github.io/bbscope/cli/targets.html Retrieves targets starting with http:// or https://. ```bash bbscope db get urls ``` -------------------------------- ### GET /api/v1/programs/{platform}/{handle} Source: https://sw33tlie.github.io/bbscope/web/api.html Returns full scope data for a specific program identified by platform and handle. ```APIDOC ## GET /api/v1/programs/{platform}/{handle} ### Description Returns full scope data for a specific program. ### Method GET ### Endpoint /api/v1/programs/{platform}/{handle} ### Parameters #### Path Parameters - **platform** (string) - Required - The platform name - **handle** (string) - Required - The program handle ``` -------------------------------- ### Advanced SQL Queries Source: https://sw33tlie.github.io/bbscope/database/querying.html Example SQL queries for analyzing program statistics, tracking scope changes, and identifying recently added programs. ```sql -- Programs with most in-scope targets SELECT p.url, COUNT(*) as target_count FROM programs p JOIN targets_raw t ON t.program_id = p.id WHERE t.in_scope = 1 GROUP BY p.url ORDER BY target_count DESC LIMIT 20; -- Recent wildcard additions SELECT sc.occurred_at, sc.program_url, sc.target_raw FROM scope_changes sc WHERE sc.change_type = 'added' AND sc.category = 'wildcard' ORDER BY sc.occurred_at DESC LIMIT 50; -- Programs added in the last 7 days SELECT url, platform, first_seen_at FROM programs WHERE first_seen_at > NOW() - INTERVAL '7 days' ORDER BY first_seen_at DESC; ``` -------------------------------- ### Dry-run output example Source: https://sw33tlie.github.io/bbscope/cli/reports.html The --dry-run flag displays a table of reports that match the criteria without initiating a download. ```text ID PROGRAM STATE SEVERITY CREATED TITLE 123456 google resolved high 2024-01-15T10:30:00.000Z XSS in login page 123457 google triaged critical 2024-02-20T14:00:00.000Z IDOR in user profile ``` -------------------------------- ### GET /api/v1/find Source: https://sw33tlie.github.io/bbscope/web/api.html Find programs whose scope matches a given hostname or domain, with automatic root domain expansion. ```APIDOC ## GET /api/v1/find ### Description Find programs whose scope matches a given hostname or domain. Automatically expands to root domain matching. ### Method GET ### Endpoint /api/v1/find ### Parameters #### Query Parameters - **q** (string) - Required - Search query (hostname, domain, etc.) ``` -------------------------------- ### GET /api/v1/updates Source: https://sw33tlie.github.io/bbscope/print.html Returns paginated scope changes. ```APIDOC ## GET /api/v1/updates ### Description Returns paginated scope changes. ### Method GET ### Endpoint /api/v1/updates ### Parameters #### Query Parameters - **since** (string) - Optional - Time filter: today, yesterday, 7d, 30d, 90d, 1y, or an ISO date (YYYY-MM-DD) - **page** (integer) - Optional - Page number (default: 1) ``` -------------------------------- ### GET /api/v1/updates Source: https://sw33tlie.github.io/bbscope/web/api.html Returns paginated scope changes filtered by time. ```APIDOC ## GET /api/v1/updates ### Description Returns paginated scope changes. ### Method GET ### Endpoint /api/v1/updates ### Parameters #### Query Parameters - **since** (string) - Optional - Time filter: 'today', 'yesterday', '7d', '30d', '90d', '1y', or an ISO date (YYYY-MM-DD) - **page** (integer) - Optional - Page number (default: 1) ``` -------------------------------- ### Create Environment Configuration Source: https://sw33tlie.github.io/bbscope/web/docker.html Initialize the .env file from the provided template. ```bash cp .env.example .env ``` -------------------------------- ### Build bbscope from source Source: https://sw33tlie.github.io/bbscope/getting-started/installation.html Clone the repository and compile the binary locally using the Go toolchain. ```bash git clone https://github.com/sw33tLie/bbscope.git cd bbscope go build -o bbscope . ``` -------------------------------- ### Create Database via CLI Source: https://sw33tlie.github.io/bbscope/database/setup.html Commands to initialize the bbscope database using shell utilities or SQL. ```bash createdb bbscope ``` ```sql CREATE DATABASE bbscope; ``` -------------------------------- ### Initialize Storage Connection Source: https://sw33tlie.github.io/bbscope/library/storage.html Import the storage package and establish a database connection. The Open function automatically handles schema creation and index initialization. ```go import "github.com/sw33tLie/bbscope/v2/pkg/storage" ``` ```go db, err := storage.Open("postgres://user:pass@localhost/bbscope?sslmode=disable") if err != nil { log.Fatal(err) } defer db.Close() ``` -------------------------------- ### Authentication methods Source: https://sw33tlie.github.io/bbscope/cli/reports.html Credentials can be provided via command-line flags or a configuration file. ```bash # CLI flags bbscope reports h1 --user your_username --token your_api_token --output-dir ./reports ``` ```yaml # ~/.bbscope.yaml hackerone: username: "your_username" token: "your_api_token" ``` -------------------------------- ### Open database connection Source: https://sw33tlie.github.io/bbscope/print.html Establishes a connection to the PostgreSQL database. This automatically creates necessary tables and indexes if they are missing. ```go db, err := storage.Open("postgres://user:pass@localhost/bbscope?sslmode=disable") if err != nil { log.Fatal(err) } defer db.Close() ``` -------------------------------- ### Authenticate Intigriti via CLI and Environment Source: https://sw33tlie.github.io/bbscope/print.html CLI command and environment variable for Intigriti authentication. ```bash bbscope poll it --token your_token ``` ```bash IT_TOKEN=your_token ``` -------------------------------- ### GET /api/v1/targets/{type} Source: https://sw33tlie.github.io/bbscope/web/api.html Extract specific target types across all programs, including wildcards, domains, URLs, IPs, and CIDRs. ```APIDOC ## GET /api/v1/targets/{type} ### Description Extract specific target types across all programs. ### Method GET ### Endpoint /api/v1/targets/{type} ### Parameters #### Path Parameters - **type** (string) - Required - Target category (wildcards, domains, urls, ips, cidrs) #### Query Parameters - **scope** (string) - Optional - 'in' (default), 'out', or 'all' - **platform** (string) - Optional - Filter by platform name - **type** (string) - Optional - Filter by target category - **raw** (boolean) - Optional - 'true' for raw (non-AI) data - **format** (string) - Optional - 'text' (default) or 'json' ``` -------------------------------- ### Configure bbscope via YAML Source: https://sw33tlie.github.io/bbscope/getting-started/configuration.html Define platform credentials, database URLs, and AI provider settings in the configuration file. ```yaml __ hackerone: username: "your_h1_username" token: "your_h1_api_token" bugcrowd: email: "you@example.com" password: "your_password" otpsecret: "YOUR_TOTP_SECRET" intigriti: token: "your_intigriti_token" yeswehack: email: "you@example.com" password: "your_password" otpsecret: "YOUR_TOTP_SECRET" db: url: "postgres://user:pass@localhost:5432/bbscope?sslmode=disable" ai: api_key: "sk-..." model: "gpt-4.1-mini" # provider: "openai" # default # endpoint: "" # custom OpenAI-compatible endpoint # max_batch: 0 # items per API call (0 = auto) # max_concurrency: 0 # parallel API calls (0 = auto) # proxy: "" # HTTP proxy for AI calls ``` -------------------------------- ### Importing the platforms package Source: https://sw33tlie.github.io/bbscope/library/platforms.html Import the base platforms package to access shared types and interfaces. ```go import "github.com/sw33tLie/bbscope/v2/pkg/platforms" ``` -------------------------------- ### Import storage package Source: https://sw33tlie.github.io/bbscope/print.html Import the storage package to access the PostgreSQL persistence layer. ```go import "github.com/sw33tLie/bbscope/v2/pkg/storage" ``` -------------------------------- ### Iterating through multiple platforms Source: https://sw33tlie.github.io/bbscope/library/platforms.html Demonstrates using the PlatformPoller interface to iterate through multiple platforms and fetch scope data. ```go ctx := context.Background() pollers := []platforms.PlatformPoller{ h1.NewPoller("user", "token"), imm.NewPoller(), } opts := platforms.PollOptions{BountyOnly: true} for _, p := range pollers { handles, _ := p.ListProgramHandles(ctx, opts) for _, h := range handles { pd, _ := p.FetchProgramScope(ctx, h, opts) fmt.Printf("%s: %d in-scope targets\n", pd.Url, len(pd.InScope)) } } ``` -------------------------------- ### Configure platform credentials Source: https://sw33tlie.github.io/bbscope/getting-started/quickstart.html Define platform credentials in the ~/.bbscope.yaml configuration file. ```yaml __ hackerone: username: "your_username" token: "your_api_token" ``` -------------------------------- ### Update and Rebuild Services Source: https://sw33tlie.github.io/bbscope/web/docker.html Commands to update images or rebuild the application from source. ```bash docker compose pull docker compose up -d ``` ```bash docker compose up -d --build ``` -------------------------------- ### Track scope changes with database Source: https://sw33tlie.github.io/bbscope/cli/polling.html Use the --db flag to enable state tracking, which silently populates the database on the first run and reports changes in subsequent runs. ```bash # First run — silent population bbscope poll --db # Second run — prints changes bbscope poll --db # 🆕 h1 https://hackerone.com/example *.new-target.com # ❌ bc https://bugcrowd.com/example removed-target.com ``` -------------------------------- ### Set YesWeHack environment variables Source: https://sw33tlie.github.io/bbscope/platforms/yeswehack.html Configure authentication using environment variables for shell sessions. ```bash YWH_EMAIL=you@example.com YWH_PASSWORD=your_password YWH_OTP=YOUR_TOTP_SECRET ``` -------------------------------- ### Set Environment Variables Source: https://sw33tlie.github.io/bbscope/platforms/bugcrowd.html Environment variables for configuring the web server or authentication. ```bash BC_PUBLIC_ONLY=1 ``` ```bash BC_EMAIL=you@example.com BC_PASSWORD=your_password BC_OTP=YOUR_TOTP_SECRET ``` ```bash BC_PUBLIC_ONLY=1 ``` -------------------------------- ### Authenticate with CLI Flags Source: https://sw33tlie.github.io/bbscope/platforms/bugcrowd.html CLI commands for different authentication modes. ```bash bbscope poll bc --email you@example.com --password pass --otp-secret SECRET ``` ```bash bbscope poll bc --token "your_bugcrowd_session_cookie" ``` ```bash bbscope poll bc --public-only ``` -------------------------------- ### Manage Program Entries and Changes Source: https://sw33tlie.github.io/bbscope/library/storage.html Operations for building entries, performing upserts, syncing platform programs, and logging changes. ```go // Build entries from target items entries, err := storage.BuildEntries(programURL, platform, handle, items) // Upsert and get changes changes, err := db.UpsertProgramEntries(ctx, programURL, platform, handle, entries) ``` ```go changes, err := db.SyncPlatformPrograms(ctx, "h1", polledProgramURLs) ``` ```go err := db.LogChanges(ctx, changes) ``` -------------------------------- ### Deploy with Cloudflare DNS Challenge Source: https://sw33tlie.github.io/bbscope/web/docker.html Use this command for wildcard certificates or when port 80 is unavailable. ```bash docker compose -f docker-compose.cloudflare.yml up -d ``` -------------------------------- ### Query Targets and Statistics Source: https://sw33tlie.github.io/bbscope/library/storage.html Methods for retrieving entries, searching targets, listing recent changes, and fetching system statistics. ```go // List entries with filters entries, err := db.ListEntries(ctx, storage.ListOptions{ Platform: "h1", InScope: true, }) // Full-text search results, err := db.SearchTargets(ctx, "example.com") // Recent changes (limit, since, until — use zero time for no filter) changes, err := db.ListRecentChanges(ctx, 50, time.Time{}, time.Time{}) // Statistics stats, err := db.GetStats(ctx) ``` ```go // Get program count count, err := db.GetActiveProgramCount(ctx, "h1") // Get ignored programs ignored, err := db.GetIgnoredPrograms(ctx, "h1") // AI enhancements cache enhancements, err := db.ListAIEnhancements(ctx, programURL) ``` -------------------------------- ### Configure Database Connection Source: https://sw33tlie.github.io/bbscope/database/setup.html Methods to provide the database connection string via configuration file, command-line flags, or environment variables. ```yaml db: url: "postgres://user:password@localhost:5432/bbscope?sslmode=disable" ``` ```bash bbscope db stats --db-url "postgres://user:password@localhost:5432/bbscope?sslmode=disable" ``` ```bash export DB_URL="postgres://user:password@localhost:5432/bbscope?sslmode=disable" ``` -------------------------------- ### Display AI normalization mapping in CLI Source: https://sw33tlie.github.io/bbscope/ai/overview.html Shows the output format when running the CLI with the --db flag to visualize the transformation from raw scope to normalized scope. ```text __ 🆕 h1 https://hackerone.com/program *.example.com (main site) -> *.example.com ``` -------------------------------- ### bbscope Configuration File Source: https://sw33tlie.github.io/bbscope/print.html The default configuration file is located at ~/.bbscope.yaml and supports platform credentials, database URLs, and AI settings. ```yaml hackerone: username: "your_h1_username" token: "your_h1_api_token" bugcrowd: email: "you@example.com" password: "your_password" otpsecret: "YOUR_TOTP_SECRET" intigriti: token: "your_intigriti_token" yeswehack: email: "you@example.com" password: "your_password" otpsecret: "YOUR_TOTP_SECRET" db: url: "postgres://user:pass@localhost:5432/bbscope?sslmode=disable" ai: api_key: "sk-..." model: "gpt-4.1-mini" # provider: "openai" # default # endpoint: "" # custom OpenAI-compatible endpoint # max_batch: 0 # items per API call (0 = auto) # max_concurrency: 0 # parallel API calls (0 = auto) # proxy: "" # HTTP proxy for AI calls ``` -------------------------------- ### Importing specific platform packages Source: https://sw33tlie.github.io/bbscope/library/platforms.html Import individual platform packages to access their specific poller implementations. ```go import ( h1 "github.com/sw33tLie/bbscope/v2/pkg/platforms/hackerone" bc "github.com/sw33tLie/bbscope/v2/pkg/platforms/bugcrowd" it "github.com/sw33tLie/bbscope/v2/pkg/platforms/intigriti" ywh "github.com/sw33tLie/bbscope/v2/pkg/platforms/yeswehack" imm "github.com/sw33tLie/bbscope/v2/pkg/platforms/immunefi" ) ``` -------------------------------- ### Retrieve program metadata and cache Source: https://sw33tlie.github.io/bbscope/print.html Fetches program counts, ignored programs, and AI enhancement cache data. ```go // Get program count count, err := db.GetActiveProgramCount(ctx, "h1") // Get ignored programs ignored, err := db.GetIgnoredPrograms(ctx, "h1") // AI enhancements cache enhancements, err := db.ListAIEnhancements(ctx, programURL) ``` -------------------------------- ### Importing the polling package Source: https://sw33tlie.github.io/bbscope/library/polling.html Import the package to access the high-level orchestrator for platform polling. ```go import "github.com/sw33tLie/bbscope/v2/pkg/polling" ``` -------------------------------- ### Pipe wildcards to recon tools Source: https://sw33tlie.github.io/bbscope/cli/targets.html Demonstrates piping wildcard output into tools like subfinder and httpx. ```bash # Subdomain enumeration bbscope db get wildcards | subfinder -silent | httpx -silent # Aggressive mode for broader coverage bbscope db get wildcards --aggressive | subfinder -silent ``` -------------------------------- ### Retrieve all programs Source: https://sw33tlie.github.io/bbscope/web/api.html Fetches all programs with their scope data. The response is cached for 5 minutes. ```http GET /api/v1/programs ``` -------------------------------- ### Run bbscope with Docker Source: https://sw33tlie.github.io/bbscope/getting-started/installation.html Pull the latest image and execute the poll command with user and token credentials. ```bash docker pull ghcr.io/sw33tlie/bbscope:latest docker run --rm ghcr.io/sw33tlie/bbscope:latest poll h1 --user x --token y ``` -------------------------------- ### Configure database connection Source: https://sw33tlie.github.io/bbscope/getting-started/quickstart.html Set the PostgreSQL connection string in the configuration file. ```yaml db: url: "postgres://user:pass@localhost:5432/bbscope?sslmode=disable" ``` -------------------------------- ### Configure and use AI scope cleaning Source: https://sw33tlie.github.io/bbscope/getting-started/quickstart.html Add an OpenAI API key to the configuration and enable AI-assisted scope cleaning during polling. ```yaml ai: api_key: "sk-..." ``` ```bash bbscope poll --db --ai ``` -------------------------------- ### Query and list operations Source: https://sw33tlie.github.io/bbscope/print.html Provides methods for listing entries with filters, full-text search, retrieving recent changes, and fetching statistics. ```go // List entries with filters entries, err := db.ListEntries(ctx, storage.ListOptions{ Platform: "h1", InScope: true, }) // Full-text search results, err := db.SearchTargets(ctx, "example.com") // Recent changes (limit, since, until — use zero time for no filter) changes, err := db.ListRecentChanges(ctx, 50, time.Time{}, time.Time{}) // Statistics stats, err := db.GetStats(ctx) ``` -------------------------------- ### View platform statistics Source: https://sw33tlie.github.io/bbscope/cli/database.html Displays a table containing program counts, in-scope targets, and out-of-scope targets per platform. ```bash bbscope db stats ``` -------------------------------- ### Access Database Shell Source: https://sw33tlie.github.io/bbscope/database/querying.html Open a psql session with the database schema pre-loaded for advanced manual queries. ```bash bbscope db shell ``` -------------------------------- ### Importing the targets package Source: https://sw33tlie.github.io/bbscope/library/targets.html Import the package to access target extraction and filtering functions. ```go import "github.com/sw33tLie/bbscope/v2/pkg/targets" ``` -------------------------------- ### Sync platform programs Source: https://sw33tlie.github.io/bbscope/print.html Synchronizes programs for a platform, marking missing programs as disabled and returning removal changes. ```go changes, err := db.SyncPlatformPrograms(ctx, "h1", polledProgramURLs) ``` -------------------------------- ### Set Intigriti Token Environment Variable Source: https://sw33tlie.github.io/bbscope/platforms/intigriti.html Set the authentication token as an environment variable. ```bash __ IT_TOKEN=your_token ``` -------------------------------- ### Configure YesWeHack credentials Source: https://sw33tlie.github.io/bbscope/platforms/yeswehack.html Define YesWeHack credentials in a configuration file for automated access. ```yaml yeswehack: email: "you@example.com" password: "your_password" otpsecret: "YOUR_TOTP_SECRET" ``` -------------------------------- ### Configure Intigriti Token in YAML Source: https://sw33tlie.github.io/bbscope/platforms/intigriti.html Set the Intigriti token in the configuration file. ```yaml __ intigriti: token: "your_intigriti_token" ``` -------------------------------- ### Utility Functions for Target Processing Source: https://sw33tlie.github.io/bbscope/library/storage.html Helper functions for normalizing target strings, building cache keys, and performing domain transformations. ```go // Normalize a target string (lowercase, strip ports, etc.) normalized := storage.NormalizeTarget(raw, category) // Build cache key for AI enhancement lookups key := storage.BuildTargetCategoryKey(target, category) // Aggressive transform: extract root domain via publicsuffix root := storage.AggressiveTransform(target) ``` -------------------------------- ### Search and Filter Targets via CLI Source: https://sw33tlie.github.io/bbscope/database/querying.html Use these commands to perform full-text searches, filter by platform or program, and export data in JSON format. ```bash bbscope db find "example.com" bbscope db print bbscope db print --platform h1 bbscope db print --program "https://hackerone.com/example" bbscope db print --since 2025-01-01T00:00:00Z bbscope db print --format json ``` -------------------------------- ### Configure AI settings in bbscope Source: https://sw33tlie.github.io/bbscope/ai/configuration.html Define the API key, model, and provider settings within the configuration file. ```yaml ai: api_key: "sk-..." model: "gpt-4.1-mini" # default # provider: "openai" # default # endpoint: "" # custom OpenAI-compatible endpoint # max_batch: 0 # items per API call (0 = auto) # max_concurrency: 0 # parallel API calls (0 = auto) # proxy: "" # HTTP proxy for AI API calls ``` -------------------------------- ### Retrieve Programs Source: https://sw33tlie.github.io/bbscope/print.html Endpoints for fetching program lists and specific program details. ```http GET /api/v1/programs ``` ```http GET /api/v1/programs/{platform}/{handle} ``` -------------------------------- ### Define Target and Change Structures Source: https://sw33tlie.github.io/bbscope/library/storage.html Data models for representing target items and scope changes within the system. ```go type TargetItem struct { URI string Category string Description string InScope bool IsBBP bool Variants []TargetVariant // AI-normalized variants } ``` ```go type Change struct { OccurredAt time.Time ProgramURL string Platform string Handle string TargetNormalized string TargetRaw string TargetAINormalized string Category string InScope bool IsBBP bool ChangeType string // "added", "removed", "updated" } ``` -------------------------------- ### View Service Logs Source: https://sw33tlie.github.io/bbscope/web/docker.html Commands to monitor logs for all services or specific components. ```bash # All services docker compose logs -f # Just the web server docker compose logs -f bbscope-web # Just the poller output docker compose logs -f bbscope-web | grep "Poller:" ``` -------------------------------- ### Defining AuthConfig and PollOptions structures Source: https://sw33tlie.github.io/bbscope/library/platforms.html Configuration structures used for platform authentication and filtering program results. ```go type AuthConfig struct { Username string Email string Password string Token string OtpSecret string Proxy string } type PollOptions struct { BountyOnly bool PrivateOnly bool Categories string // comma-separated or "all" } ``` -------------------------------- ### Set HackerOne credentials as environment variables Source: https://sw33tlie.github.io/bbscope/platforms/hackerone.html Export credentials to the shell environment to avoid passing them as command-line arguments. ```bash H1_USERNAME=your_username H1_TOKEN=your_api_token ``` -------------------------------- ### Project Directory Structure Source: https://sw33tlie.github.io/bbscope/advanced/architecture.html The layout of the bbscope repository, separating CLI commands, library packages, and web server components. ```text bbscope/ ├── cmd/ # CLI commands (Cobra) │ ├── root.go # Root command, global flags │ ├── poll.go # bbscope poll │ ├── poll_*.go # Platform-specific poll subcommands │ ├── db.go # bbscope db │ ├── db_*.go # DB subcommands (stats, changes, find, etc.) │ ├── serve.go # bbscope serve │ └── dev.go # bbscope dev ├── pkg/ # Library packages (importable) │ ├── platforms/ # Platform interface + implementations │ │ ├── platform.go # PlatformPoller interface │ │ ├── hackerone/ │ │ ├── bugcrowd/ │ │ ├── intigriti/ │ │ ├── yeswehack/ │ │ ├── immunefi/ │ │ └── dev/ │ ├── polling/ # Shared polling orchestrator │ │ └── polling.go │ ├── storage/ # PostgreSQL persistence │ │ ├── storage.go # DB operations │ │ ├── types.go # Entry, Change, UpsertEntry, etc. │ │ ├── normalize.go # Target normalization │ │ ├── transform.go # Aggressive transforms │ │ └── extra.go # Additional queries │ ├── scope/ # Core types and category normalization │ ├── targets/ # Target extraction (wildcards, domains, etc.) │ ├── ai/ # AI normalization │ ├── whttp/ # HTTP client wrapper (retryablehttp) │ └── otp/ # TOTP generation ├── website/ # Web server │ ├── pkg/core/ # Server core (routes, handlers, poller) │ ├── static/ # CSS, JS, images │ ├── docker-compose.yml │ └── Dockerfile ├── internal/ │ └── utils/ # Logging setup └── docs/ # This documentation (mdBook) ``` -------------------------------- ### PlatformConfig structure Source: https://sw33tlie.github.io/bbscope/library/polling.html Configuration struct defining the poller, database connection, concurrency settings, and optional callbacks. ```go type PlatformConfig struct { Poller platforms.PlatformPoller // required Options platforms.PollOptions // category/bounty/private filters DB *storage.DB // required Concurrency int // defaults to 5 if <= 0 Normalizer ai.Normalizer // optional; nil = skip AI Log Logger // optional; nil = no logging // Called per-program after upsert (from worker goroutines). // Nil = no callback. OnProgramDone func(programURL string, changes []storage.Change, isFirstRun bool) } ``` -------------------------------- ### Configure HackerOne credentials in YAML Source: https://sw33tlie.github.io/bbscope/platforms/hackerone.html Define the username and API token in the configuration file for persistent access. ```yaml hackerone: username: "your_username" token: "your_api_token" ``` -------------------------------- ### Upsert program entries Source: https://sw33tlie.github.io/bbscope/print.html Builds entries from target items and performs an atomic upsert, returning the detected changes. ```go // Build entries from target items entries, err := storage.BuildEntries(programURL, platform, handle, items) // Upsert and get changes changes, err := db.UpsertProgramEntries(ctx, programURL, platform, handle, entries) ``` -------------------------------- ### Execute bbscope CLI commands Source: https://sw33tlie.github.io/bbscope Common command-line operations for fetching, storing, and serving bug bounty scope data. ```bash # Print all HackerOne in-scope targets bbscope poll h1 --user your_user --token your_token # Poll all platforms, store in DB, print changes bbscope poll --db # Extract wildcards for subdomain enumeration bbscope db get wildcards # Start the web interface bbscope serve ``` -------------------------------- ### Download and filter reports via CLI Source: https://sw33tlie.github.io/bbscope/cli/reports.html Commands for downloading reports from HackerOne with various filtering and operational flags. ```bash # Download all your reports bbscope reports h1 --output-dir ./reports # Preview what would be downloaded (dry-run) bbscope reports h1 --output-dir ./reports --dry-run # Filter by program bbscope reports h1 --output-dir ./reports --program google --program microsoft # Filter by state (e.g., resolved, triaged, new, duplicate, informative, not-applicable, spam) bbscope reports h1 --output-dir ./reports --state resolved --state triaged # Filter by severity bbscope reports h1 --output-dir ./reports --severity critical --severity high # Combine filters bbscope reports h1 --output-dir ./reports --program google --state resolved --severity critical # Overwrite existing files bbscope reports h1 --output-dir ./reports --overwrite ``` -------------------------------- ### Set environment variables for AI Source: https://sw33tlie.github.io/bbscope/ai/configuration.html Use environment variables to configure the AI API key and model for the web server. ```bash OPENAI_API_KEY=sk-... OPENAI_MODEL=gpt-4.1-mini ``` -------------------------------- ### Poll targets with database integration Source: https://sw33tlie.github.io/bbscope/getting-started/quickstart.html Use the --db flag to store and track changes in the database. ```bash # First run populates the database silently bbscope poll --db # Subsequent runs print only changes (new/removed/updated targets) bbscope poll --db ``` -------------------------------- ### Enable AI normalization Source: https://sw33tlie.github.io/bbscope/print.html Configure an OpenAI API key to clean up scope entries and cache results in the database. ```yaml ai: api_key: "sk-..." ``` -------------------------------- ### Search and Print Targets Source: https://sw33tlie.github.io/bbscope/print.html Commands for searching the database and printing targets with various filters and output formats. ```bash bbscope db find "example.com" ``` ```bash bbscope db print ``` ```bash bbscope db print --platform h1 ``` ```bash bbscope db print --program "https://hackerone.com/example" ``` ```bash bbscope db print --since 2025-01-01T00:00:00Z ``` ```bash bbscope db print --format json ``` -------------------------------- ### Configure Bugcrowd TOTP Authentication Source: https://sw33tlie.github.io/bbscope/platforms/bugcrowd.html Use these credentials in the configuration file to perform a full login flow with TOTP. ```yaml bugcrowd: email: "you@example.com" password: "your_password" otpsecret: "YOUR_BASE32_TOTP_SECRET" ``` -------------------------------- ### Normalizing scope strings Source: https://sw33tlie.github.io/bbscope/library/targets.html Strips prefixes and artifacts from scope strings to prepare them for subdomain enumeration tools. ```go // Clean a scope string for use with subdomain enumeration tools cleaned := targets.NormalizeForSubdomainTools(scopeEntry) ``` -------------------------------- ### Poll program scopes from platforms Source: https://sw33tlie.github.io/bbscope/cli/polling.html Commands to fetch program scopes from all configured platforms or a specific platform. ```bash # Poll all configured platforms bbscope poll # Poll a specific platform bbscope poll h1 bbscope poll bc bbscope poll it bbscope poll ywh bbscope poll immunefi ``` -------------------------------- ### Poll Intigriti via CLI Source: https://sw33tlie.github.io/bbscope/platforms/intigriti.html Use the command-line interface to poll Intigriti using a token. ```bash __ bbscope poll it --token your_token ``` -------------------------------- ### Find programs by hostname Source: https://sw33tlie.github.io/bbscope/web/api.html Searches for programs whose scope matches a given hostname or domain, with automatic root domain expansion. ```http GET /api/v1/find ``` -------------------------------- ### Dump scope database with filters Source: https://sw33tlie.github.io/bbscope/cli/database.html Exports scope data using various filters, formats, and time-based constraints. ```bash # All in-scope targets bbscope db print # Include out-of-scope bbscope db print --oos # Filter by platform bbscope db print --platform h1 # Filter by program bbscope db print --program "https://hackerone.com/example" # JSON output bbscope db print --format json # CSV output bbscope db print --format csv # Custom text output bbscope db print -o tdu -d $'\t' # Only changes since a date bbscope db print --since 2025-01-01T00:00:00Z ``` -------------------------------- ### Extract domains, URLs, IPs, and CIDRs Source: https://sw33tlie.github.io/bbscope/print.html Commands to retrieve specific target types from the database. ```bash bbscope db get domains ``` ```bash bbscope db get urls ``` ```bash bbscope db get ips ``` ```bash bbscope db get cidrs ``` -------------------------------- ### Poll with inline credentials Source: https://sw33tlie.github.io/bbscope/cli/polling.html Use platform-specific flags to provide credentials for one-off runs without a configuration file. ```bash # HackerOne bbscope poll h1 --user your_user --token your_token # Bugcrowd bbscope poll bc --email you@example.com --password pass --otp-secret SECRET # Bugcrowd public-only (no auth) bbscope poll bc --public-only # Intigriti bbscope poll it --token your_token # YesWeHack bbscope poll ywh --email you@example.com --password pass --otp-secret SECRET ``` -------------------------------- ### Enable AI normalization in background poller Source: https://sw33tlie.github.io/bbscope/ai/configuration.html Run the background poller with the --ai flag to enable AI normalization, which requires a database connection. ```bash bbscope poll --db --ai ``` -------------------------------- ### Add a target to the database Source: https://sw33tlie.github.io/bbscope/cli/database.html Adds a new target to the scope database with specified category and program URL. ```bash bbscope db add --target "*.custom.com" --category wildcard --program-url "https://example.com/program" ``` -------------------------------- ### Define TargetItem structure Source: https://sw33tlie.github.io/bbscope/print.html Represents the input data structure for building scope entries. ```go type TargetItem struct { URI string Category string Description string InScope bool IsBBP bool Variants []TargetVariant // AI-normalized variants } ``` -------------------------------- ### Poll YesWeHack programs via CLI Source: https://sw33tlie.github.io/bbscope/platforms/yeswehack.html Execute program polling using email/password or a bearer token. ```bash bbscope poll ywh --email you@example.com --password pass --otp-secret SECRET ``` ```bash bbscope poll ywh --token "your_bearer_token" ``` -------------------------------- ### Poll HackerOne programs via CLI Source: https://sw33tlie.github.io/bbscope/platforms/hackerone.html Use command-line flags to provide authentication credentials directly during execution. ```bash bbscope poll h1 --user your_username --token your_api_token ``` -------------------------------- ### Format database output Source: https://sw33tlie.github.io/bbscope/cli/output.html Specify the output format for the db print command using the --format flag. ```bash # Plain text (default) bbscope db print # JSON bbscope db print --format json # CSV bbscope db print --format csv ``` -------------------------------- ### Configure custom OpenAI-compatible endpoint Source: https://sw33tlie.github.io/bbscope/ai/configuration.html Specify a custom endpoint and model for self-hosted or alternative AI providers. ```yaml ai: api_key: "your-key" endpoint: "http://localhost:11434/v1" model: "llama3" ``` -------------------------------- ### Configure output fields with -o Source: https://sw33tlie.github.io/bbscope/cli/output.html Use the -o flag to specify which fields are included in the command output. Flags can be combined to display multiple fields. ```bash # Target only (default for db get) bbscope poll -o t # Target + program URL (default for poll) bbscope poll -o tu # All fields bbscope poll -o tdcu ``` -------------------------------- ### Logger interface Source: https://sw33tlie.github.io/bbscope/library/polling.html Interface for custom logging implementations. Compatible with logrus and standard library loggers. ```go type Logger interface { Infof(format string, args ...interface{}) Warnf(format string, args ...interface{}) Errorf(format string, args ...interface{}) Debugf(format string, args ...interface{}) } ```