### Install gau from Source Source: https://github.com/lc/gau/blob/master/README.md Install the gau tool by fetching the latest version from GitHub using go install. ```bash $ go install github.com/lc/gau/v2/cmd/gau@latest ``` -------------------------------- ### Install gau from Binary Source: https://github.com/lc/gau/blob/master/README.md Install gau by downloading a pre-built binary archive, extracting it, and moving the executable to a directory in your system's PATH. ```bash $ tar xvf gau_2.0.6_linux_amd64.tar.gz $ mv gau /usr/bin/gau ``` -------------------------------- ### Basic Usage Examples for gau Source: https://github.com/lc/gau/blob/master/README.md Demonstrates common ways to use the gau command-line tool to fetch URLs for a domain. Input can be piped or provided as arguments. ```bash $ printf example.com | gau ``` ```bash $ cat domains.txt | gau --threads 5 ``` ```bash $ gau example.com google.com ``` ```bash $ gau --o example-urls.txt example.com ``` ```bash $ gau --blacklist png,jpg,gif example.com ``` -------------------------------- ### Install gau from GitHub Source: https://github.com/lc/gau/blob/master/README.md Build the gau executable from source after cloning the GitHub repository and then move it to the system's PATH. ```bash git clone https://github.com/lc/gau.git; \ cd gau/cmd; \ go build; \ sudo mv gau /usr/local/bin/; \ gau --version; ``` -------------------------------- ### Integrate Gau as a Go Library Source: https://context7.com/lc/gau/llms.txt Use Gau programmatically within Go applications by importing its packages. This example demonstrates initializing the runner, configuring providers, and processing results. ```go package main import ( "context" "fmt" "sync" "github.com/lc/gau/v2/pkg/providers" "github.com/lc/gau/v2/pkg/providers/wayback" "github.com/lc/gau/v2/runner" "github.com/valyala/fasthttp" ) func main() { // Create provider configuration config := &providers.Config{ Threads: 5, Timeout: 45, MaxRetries: 5, IncludeSubdomains: true, Client: &fasthttp.Client{}, } // Initialize runner with specific providers gau := &runner.Runner{} filters := providers.Filters{} err := gau.Init(config, []string{"wayback", "otx"}, filters) if err != nil { panic(err) } // Create channels for work and results results := make(chan string) workChan := make(chan runner.Work) ctx, cancel := context.WithCancel(context.Background()) defer cancel() // Start workers gau.Start(ctx, workChan, results) // Collect results in background var wg sync.WaitGroup wg.Add(1) go func() { defer wg.Done() for url := range results { fmt.Println(url) } }() // Send work for domain domain := "example.com" for _, provider := range gau.Providers { workChan <- runner.NewWork(domain, provider) } close(workChan) // Wait for completion gau.Wait() close(results) wg.Wait() } ``` -------------------------------- ### Run Gau with Default Configuration Source: https://context7.com/lc/gau/llms.txt Execute Gau using the default configuration file located at `~/.gau.toml`. This simplifies command-line usage by applying predefined settings. ```bash gau example.com ``` -------------------------------- ### Selecting URL Providers Source: https://context7.com/lc/gau/llms.txt Specify which archives to query to optimize performance or focus on specific data sources. ```bash # Use only the Wayback Machine gau --providers wayback example.com # Use multiple specific providers gau --providers wayback,commoncrawl example.com # Use only OTX (AlienVault Open Threat Exchange) gau --providers otx example.com # Use only URLScan gau --providers urlscan example.com # Available providers: wayback, commoncrawl, otx, urlscan ``` -------------------------------- ### Run Gau with Custom Configuration File Source: https://context7.com/lc/gau/llms.txt Specify a custom path to a TOML configuration file for Gau. This allows for different sets of default settings without modifying the standard configuration location. ```bash gau --config /path/to/custom-config.toml example.com ``` -------------------------------- ### Basic Domain Scanning with gau Source: https://context7.com/lc/gau/llms.txt Fetch URLs for single or multiple domains using direct arguments, pipes, or input files. ```bash # Scan a single domain gau example.com # Scan multiple domains in one command gau example.com google.com github.com # Pipe domains from stdin echo "example.com" | gau # Scan multiple domains from a file cat domains.txt | gau # Output from domains.txt with multiple workers cat domains.txt | gau --threads 5 ``` -------------------------------- ### Build Docker Image for gau Source: https://github.com/lc/gau/blob/master/README.md Build a custom Docker image for the gau tool using the provided Dockerfile. ```bash docker build -t gau . ``` -------------------------------- ### Display Help for gau Source: https://github.com/lc/gau/blob/master/README.md Use the -h flag to display the help message and available options for the gau tool. ```bash $ gau -h ``` -------------------------------- ### Run gau from Docker Image Source: https://github.com/lc/gau/blob/master/README.md Run a gau container using a locally built Docker image to scan a target domain. ```bash docker run gau example.com ``` -------------------------------- ### Run Gau Docker Container with Options Source: https://context7.com/lc/gau/llms.txt Execute Gau from a Docker container and pass command-line arguments such as `--subs` for subdomains and `-o /dev/stdout` for output redirection. This demonstrates how to use Gau's features within a containerized environment. ```bash docker run --rm sxcurity/gau:latest --subs --o /dev/stdout example.com ``` -------------------------------- ### Run gau with Docker Source: https://github.com/lc/gau/blob/master/README.md Execute the gau tool within a Docker container. This command shows how to display help within the container. ```bash docker run --rm sxcurity/gau:latest --help ``` -------------------------------- ### Run Custom Built Gau Docker Image Source: https://context7.com/lc/gau/llms.txt Execute a locally built Gau Docker image. This is useful after customizing the image or for testing purposes. ```bash docker run --rm gau example.com ``` -------------------------------- ### Provider Interface Definition Source: https://context7.com/lc/gau/llms.txt Defines the Provider interface for fetching URLs from new sources. Implement this interface to create custom URL providers. ```go package providers import "context" // Provider interface that all URL sources must implement type Provider interface { // Fetch retrieves URLs for a domain and sends them to the results channel Fetch(ctx context.Context, domain string, results chan string) error // Name returns the provider's identifier Name() string } ``` ```go // Config holds shared configuration for all providers type Config struct { Threads uint Timeout uint MaxRetries uint IncludeSubdomains bool RemoveParameters bool Client *fasthttp.Client Providers []string Blacklist mapset.Set[string] Output string JSON bool URLScan URLScan OTX string } ``` ```go // Filters for date ranges and response filtering type Filters struct { From string // Start date (YYYYMM format) To string // End date (YYYYMM format) MatchStatusCodes []string // Only include these status codes MatchMimeTypes []string // Only include these MIME types FilterStatusCodes []string // Exclude these status codes FilterMimeTypes []string // Exclude these MIME types } ``` -------------------------------- ### Gau TOML Configuration File Source: https://context7.com/lc/gau/llms.txt Define default settings for Gau using a TOML file located at `~/.gau.toml`. This file allows persistent configuration of threads, verbosity, retries, subdomains, parameters, providers, blacklisted extensions, JSON output, and provider-specific API keys. ```toml # ~/.gau.toml - Example configuration file # Number of concurrent workers threads = 2 # Show verbose output verbose = false # Number of HTTP retries retries = 15 # Include subdomains subdomains = false # Remove duplicate parameters parameters = false # Providers to use providers = ["wayback", "commoncrawl", "otx", "urlscan"] # File extensions to skip blacklist = ["ttf", "woff", "svg", "png", "jpg"] # Output as JSON json = false # URLScan API configuration [urlscan] apikey = "your-urlscan-api-key" # Filter configuration [filters] from = "" to = "" matchstatuscodes = [] matchmimetypes = [] filterstatuscodes = [] filtermimetypes = ["image/png", "image/jpg", "image/svg+xml"] ``` -------------------------------- ### Configuring gau Output Source: https://context7.com/lc/gau/llms.txt Save results to files or format output as JSON for integration into automated pipelines. ```bash # Write results to a file gau --o example-urls.txt example.com # Output as JSON (one JSON object per line) gau --json example.com # JSON output example: # {"url":"https://example.com/page1"} # {"url":"https://example.com/page2"} # Combine JSON output with file writing gau --json --o results.json example.com ``` -------------------------------- ### Configure HTTP Proxy for Gau Source: https://context7.com/lc/gau/llms.txt Utilize an HTTP proxy for network requests. Specify the proxy address and port to route traffic through an intermediary. ```bash gau --proxy http://proxy.example.com:8080 example.com ``` -------------------------------- ### Filtering by File Extension Source: https://context7.com/lc/gau/llms.txt Exclude static assets like images or fonts from results using the blacklist flag. ```bash # Skip common image and font files gau --blacklist ttf,woff,svg,png,jpg,gif example.com # Skip all static assets gau --blacklist png,jpg,gif,jpeg,svg,woff,woff2,ttf,eot,css example.com # Skip specific file types during bug bounty recon gau --blacklist pdf,doc,docx,xls,xlsx example.com ``` -------------------------------- ### Combine Gau Performance Options Source: https://context7.com/lc/gau/llms.txt Apply multiple performance tuning flags simultaneously to customize threading, timeouts, and retries for a specific scan. This allows for fine-grained control over resource usage and reliability. ```bash gau --threads 5 --timeout 60 --retries 10 example.com ``` -------------------------------- ### Mass Scanning with Parallel Source: https://context7.com/lc/gau/llms.txt Performs mass scanning by reading targets from a file and processing them in parallel using 'gau' with individual output files. ```bash # Mass scanning with parallel cat targets.txt | parallel -j 5 "gau --o {}.txt {}" ``` -------------------------------- ### Override Configuration with Command-Line Flags Source: https://context7.com/lc/gau/llms.txt Use command-line flags to override settings defined in the Gau configuration file. This provides flexibility to adjust specific parameters for individual runs. ```bash gau --threads 10 example.com # Overrides threads from config ``` -------------------------------- ### Find Parameters for Fuzzing Source: https://context7.com/lc/gau/llms.txt Extracts URLs containing an equals sign, likely indicating parameters, and sorts the unique results. ```bash # Find parameters for fuzzing gau example.com | grep "=" | sort -u > params.txt ``` -------------------------------- ### Configure Gau Timeout Settings Source: https://context7.com/lc/gau/llms.txt Set a longer timeout duration in seconds for Gau to handle slow or unresponsive network connections. This prevents premature termination of scans on unreliable networks. ```bash gau --timeout 120 example.com ``` -------------------------------- ### Enable Verbose Logging in Gau Source: https://context7.com/lc/gau/llms.txt Activate verbose output to monitor the scanning progress and diagnose issues. This mode provides detailed information about each step of the process. ```bash gau --verbose example.com ``` -------------------------------- ### Comprehensive Recon with Filters Source: https://context7.com/lc/gau/llms.txt Performs comprehensive reconnaissance including subdomains and filtering out common image and font file types, then sorts the unique results. ```bash # Comprehensive recon: subdomains + clean output gau --subs --fp --blacklist png,jpg,gif,svg,woff,ttf,css example.com | sort -u ``` -------------------------------- ### Including Subdomains Source: https://context7.com/lc/gau/llms.txt Expand the scope of the scan to include all subdomains of the target. ```bash # Include all subdomains gau --subs example.com # This will return URLs from: # - example.com # - www.example.com # - api.example.com # - mail.example.com # - etc. # Combine with output file gau --subs --o all-subdomains-urls.txt example.com ``` -------------------------------- ### Run Gau Docker Container with JSON Output Source: https://context7.com/lc/gau/llms.txt Execute Gau within a Docker container and enable JSON output format. This is useful for programmatic parsing of results in automated workflows. ```bash docker run --rm sxcurity/gau:latest --json example.com ``` -------------------------------- ### Run Gau from Docker Hub Source: https://context7.com/lc/gau/llms.txt Execute Gau within a Docker container by pulling the latest image from Docker Hub. This is ideal for isolated environments or CI/CD pipelines. ```bash docker run --rm sxcurity/gau:latest example.com ``` -------------------------------- ### Sort Unique Endpoints Source: https://context7.com/lc/gau/llms.txt Finds all unique endpoints for a domain and sorts them, saving the output to a file. ```bash # Find all unique endpoints and sort them gau --fp example.com | sort -u > endpoints.txt ``` -------------------------------- ### Feed URLs to Other Tools Source: https://context7.com/lc/gau/llms.txt Pipes discovered URLs to httpx for silent status code checking. ```bash # Feed URLs to other security tools gau example.com | httpx -silent -status-code ``` -------------------------------- ### Find JavaScript Files Source: https://context7.com/lc/gau/llms.txt Finds JavaScript files by filtering with the 'application/javascript' MIME type and sorts the unique results. ```bash # Find JavaScript files for analysis gau --mt application/javascript example.com | sort -u > js-files.txt ``` -------------------------------- ### Configure SOCKS5 Proxy for Gau Source: https://context7.com/lc/gau/llms.txt Route traffic through a SOCKS5 proxy, commonly used for anonymization services like Tor. Provide the SOCKS5 URI including the address and port. ```bash gau --proxy socks5://127.0.0.1:9050 example.com ``` -------------------------------- ### Filtering by MIME Type Source: https://context7.com/lc/gau/llms.txt Target specific content types such as HTML, JSON, or JavaScript files. ```bash # Only get HTML pages gau --mt text/html example.com # Get HTML and JSON responses (useful for API discovery) gau --mt text/html,application/json example.com # Filter out image content types gau --ft image/png,image/jpeg,image/gif,image/svg+xml example.com # Find JavaScript files gau --mt application/javascript,text/javascript example.com ``` -------------------------------- ### Export to JSON Source: https://context7.com/lc/gau/llms.txt Exports discovered URLs in JSON format, extracts the 'url' field, and sorts the unique results. ```bash # Export to JSON for processing gau --json example.com | jq -r '.url' | sort -u ``` -------------------------------- ### Find Potential API Endpoints Source: https://context7.com/lc/gau/llms.txt Filters URLs to find potential API endpoints using grep and sorts the unique results. ```bash # Find potential API endpoints gau example.com | grep -E "api|v[0-9]|graphql" | sort -u ``` -------------------------------- ### Filtering by Status Code Source: https://context7.com/lc/gau/llms.txt Include or exclude URLs based on their HTTP response status codes. ```bash # Only get URLs that returned 200 OK gau --mc 200 example.com # Match multiple successful status codes gau --mc 200,201,301,302 example.com # Filter out error and redirect responses gau --fc 404,403,500,301,302 example.com # Find only server errors gau --mc 500,502,503 example.com ``` -------------------------------- ### Increase Gau Retries Source: https://context7.com/lc/gau/llms.txt Boost the number of retries for unreliable network connections to improve scan success rates. This is beneficial in environments with intermittent connectivity. ```bash gau --retries 15 example.com ``` -------------------------------- ### Filtering by Date Range Source: https://context7.com/lc/gau/llms.txt Restrict results to a specific timeframe using YYYYMM format. ```bash # Get URLs from January 2021 onwards gau --from 202101 example.com # Get URLs up to December 2022 gau --to 202212 example.com # Get URLs from a specific date range gau --from 202001 --to 202112 example.com # Get only recent URLs (last year) gau --from 202401 example.com ``` -------------------------------- ### Deduplicating Endpoints Source: https://context7.com/lc/gau/llms.txt Remove duplicate endpoints that differ only by query parameters to reduce noise. ```bash # Deduplicate endpoints with different parameters gau --fp example.com # Without --fp: # https://example.com/search?q=test # https://example.com/search?q=hello # https://example.com/search?q=world # With --fp: # https://example.com/search?q=test # (only first occurrence of each endpoint is shown) # Combine with blacklist for cleaner results gau --fp --blacklist png,jpg,gif,css,js example.com ``` -------------------------------- ### Adjust Gau Worker Threads Source: https://context7.com/lc/gau/llms.txt Increase the number of worker threads to speed up scanning processes. This is useful for handling a large number of targets or when faster results are needed. ```bash gau --threads 10 example.com ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.