### Generate HTTP Probe with Detailed Debug Output Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md This example demonstrates how to start the Blackbox Exporter with `--log.prober=debug` to get more detailed logs for each probe. It shows the full request and response lifecycle. ```shell ~ $ ./blackbox_exporter --config.file ./blackbox.yml --log.prober=debug time=2025-09-09T00:58:21.483-04:00 level=INFO source=main.go:95 msg="Starting blackbox_exporter" version="(version=0.27.0, branch=feat/make-scrape-logger-independent, revision=47e27d09847edf2ade2732b50663c37ed8177485)" time=2025-09-09T00:58:21.483-04:00 level=INFO source=main.go:96 msg="(go=go1.24.4, platform=linux/amd64, user=tjhop@contraband, date=20250909-04:57:55, tags=unknown)" time=2025-09-09T00:58:21.483-04:00 level=INFO source=main.go:108 msg="Loaded config file" time=2025-09-09T00:58:21.484-04:00 level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9115 time=2025-09-09T00:58:21.484-04:00 level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9115 time=2025-09-09T00:58:26.604-04:00 level=DEBUG source=handler.go:116 msg="Beginning probe" module=http_2xx target=prometheus.io probe=http timeout_seconds=119.5 time=2025-09-09T00:58:26.604-04:00 level=DEBUG source=utils.go:61 msg="Resolving target address" module=http_2xx target=prometheus.io target=prometheus.io ip_protocol=ip4 time=2025-09-09T00:58:26.605-04:00 level=DEBUG source=utils.go:96 msg="Resolved target address" module=http_2xx target=prometheus.io target=prometheus.io ip=104.21.60.220 time=2025-09-09T00:58:26.605-04:00 level=DEBUG source=http.go:209 msg="Making HTTP request" module=http_2xx target=prometheus.io url=http://104.21.60.220 host=prometheus.io time=2025-09-09T00:58:26.645-04:00 level=WARN source=http.go:490 msg="Received redirect" module=http_2xx target=prometheus.io location=https://prometheus.io/ time=2025-09-09T00:58:26.645-04:00 level=DEBUG source=http.go:209 msg="Making HTTP request" module=http_2xx target=prometheus.io url=https://prometheus.io/ host="" time=2025-09-09T00:58:26.645-04:00 level=DEBUG source=http.go:224 msg="Address does not match first address, not sending TLS ServerName" module=http_2xx target=prometheus.io first=104.21.60.220 address=prometheus.io time=2025-09-09T00:58:26.765-04:00 level=DEBUG source=http.go:590 msg="Received HTTP response" module=http_2xx target=prometheus.io status_code=200 time=2025-09-09T00:58:26.800-04:00 level=DEBUG source=http.go:721 msg="Response timings for roundtrip" module=http_2xx target=prometheus.io roundtrip=0 start=2025-09-09T00:58:26.605-04:00 dnsDone=2025-09-09T00:58:26.605-04:00 connectDone=2025-09-09T00:58:26.619-04:00 gotConn=2025-09-09T00:58:26.619-04:00 responseStart=2025-09-09T00:58:26.645-04:00 tlsStart=0001-01-01T00:00:00.000Z tlsDone=0001-01-01T00:00:00.000Z end=0001-01-01T00:00:00.000Z time=2025-09-09T00:58:26.800-04:00 level=DEBUG source=http.go:721 msg="Response timings for roundtrip" module=http_2xx target=prometheus.io roundtrip=1 start=2025-09-09T00:58:26.645-04:00 dnsDone=2025-09-09T00:58:26.646-04:00 connectDone=2025-09-09T00:58:26.656-04:00 gotConn=2025-09-09T00:58:26.717-04:00 responseStart=2025-09-09T00:58:26.765-04:00 tlsStart=2025-09-09T00:58:26.657-04:00 tlsDone=2025-09-09T00:58:26.717-04:00 end=2025-09-09T00:58:26.800-04:00 time=2025-09-09T00:58:26.801-04:00 level=DEBUG source=handler.go:127 msg="Probe succeeded" module=http_2xx target=prometheus.io duration_seconds=0.196876958 ``` -------------------------------- ### GetByTargetAndModule Example Usage Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/history.md Examples demonstrating how to retrieve the latest result for a target, either for any module or a specific one. The method prioritizes preserved failures. ```go // Get latest result for target, any module result := rh.GetByTargetAndModule("https://example.com", "") // Get latest result for specific target + module combination result := rh.GetByTargetAndModule("https://example.com", "http_2xx") if result != nil { fmt.Println(result.DebugOutput) } ``` -------------------------------- ### YAML Configuration Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/config.md Provides examples of how to configure different probe modules (HTTP, TCP, ICMP, DNS) in YAML format. This includes settings like status codes, methods, headers, preferred IP protocols, and query details. ```yaml modules: http_2xx: prober: http timeout: 30s http: valid_status_codes: [200, 201] method: GET headers: User-Agent: Prometheus tcp_connect: prober: tcp timeout: 10s tcp: preferred_ip_protocol: "ip4" icmp_echo: prober: icmp icmp: preferred_ip_protocol: "ip4" dns_query: prober: dns dns: query_name: "example.com" query_type: "A" ``` -------------------------------- ### TCP Connection Check Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md This example demonstrates how to check TCP connectivity to a specific host and port. The 'tcp_connect' module should be defined. ```bash curl "http://localhost:9115/probe?target=example.com:443&module=tcp_connect" ``` -------------------------------- ### HTTP Probe Configuration Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/configuration.md An example configuration for an HTTP probe, demonstrating how to set status codes, HTTP versions, headers, and body matching rules. Use this to define custom HTTP checks. ```yaml modules: http_2xx: prober: http timeout: 30s http: valid_status_codes: [200, 201] valid_http_versions: ["HTTP/1.1", "HTTP/2.0"] preferred_ip_protocol: "ip4" ip_protocol_fallback: true method: GET headers: User-Agent: Mozilla/5.0 Accept: text/html fail_if_ssl: false fail_if_not_ssl: false fail_if_body_matches_regexp: - "error" - "failure" fail_if_body_not_matches_regexp: - "success" fail_if_header_matches: - header: "Content-Type" regexp: "text/plain" allow_missing: false body: | {"key": "value"} # OR body_file: /path/to/body.json compression: gzip body_size_limit: 10MB follow_redirects: true enable_http2: true enable_http3: false tls_config: insecure_skip_verify: false basic_auth: username: user password: pass http_proxy_url: http://proxy.example.com:8080 ``` -------------------------------- ### Ping Google Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of performing a ping to Google's IP address using the ICMP probe. ```bash # Ping Google curl "http://localhost:9115/probe?target=8.8.8.8&module=icmp_echo" ``` -------------------------------- ### Module Configuration Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Example YAML structure for defining module configurations. Specifies the prober type and protocol-specific settings. ```yaml modules: module_name: prober: http|tcp|icmp|dns|grpc|unix|websocket timeout: 30s # Optional, default 120s http: { ... } # Protocol-specific config tcp: { ... } icmp: { ... } dns: { ... } grpc: { ... } unix: { ... } websocket: { ... } ``` -------------------------------- ### Blackbox Exporter /metrics Output Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/metrics.md An example of the metrics output from the Blackbox exporter's /metrics endpoint, showcasing various exporter and Go runtime metrics. ```text # HELP blackbox_exporter_config_last_reload_successful Blackbox exporter config loaded successfully. # TYPE blackbox_exporter_config_last_reload_successful gauge blackbox_exporter_config_last_reload_successful 1 # HELP blackbox_exporter_config_last_reload_success_timestamp_seconds Timestamp of the last successful configuration reload. # TYPE blackbox_exporter_config_last_reload_success_timestamp_seconds gauge blackbox_exporter_config_last_reload_success_timestamp_seconds 1.6234567e+09 # HELP blackbox_module_unknown_total Count of unknown modules requested by probes # TYPE blackbox_module_unknown_total counter blackbox_module_unknown_total 2 # HELP process_resident_memory_bytes Resident memory size in bytes. # TYPE process_resident_memory_bytes gauge process_resident_memory_bytes 5.2428e+07 # HELP process_virtual_memory_bytes Virtual memory size in bytes. # TYPE process_virtual_memory_bytes gauge process_virtual_memory_bytes 7.89312e+08 # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge go_goroutines 12 ``` -------------------------------- ### SMTP Banner Check Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of checking an SMTP banner using the TCP probe. ```bash # SMTP banner check curl "http://localhost:9115/probe?target=mail.example.com:25&module=tcp_smtp" ``` -------------------------------- ### TLS Certificate Check Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of checking an HTTPS certificate using the TCP probe with TLS enabled. ```bash # HTTPS certificate check curl "http://localhost:9115/probe?target=example.com:443&module=tcp_tls" ``` -------------------------------- ### Creating a New Regexp Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/config.md Example of how to create a new Regexp object using the config.NewRegexp function. ```go re, err := config.NewRegexp("^error.*") ``` -------------------------------- ### Example DNS Probe Usage with cURL Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Demonstrates how to use cURL to trigger DNS probes against a Blackbox Exporter instance. Examples include a general query to Google DNS, an A record lookup, and an MX record lookup. ```bash # Query Google public DNS curl "http://localhost:9115/probe?target=8.8.8.8&module=dns_google" # A record lookup curl "http://localhost:9115/probe?target=1.1.1.1&module=dns_a_query" # MX record lookup curl "http://localhost:9115/probe?target=8.8.8.8&module=dns_mx_query" ``` -------------------------------- ### IPv6 Ping Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of performing an IPv6 ping using the ICMP probe. ```bash # IPv6 ping curl "http://localhost:9115/probe?target=2001:4860:4860::8888&module=icmp_echo" ``` -------------------------------- ### Run Blackbox Exporter with Docker Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md Start the exporter container, mapping the local configuration directory to the container. ```bash docker run --rm \ -p 9115/tcp \ --name blackbox_exporter \ -v $(pwd):/config \ quay.io/prometheus/blackbox-exporter:latest --config.file=/config/blackbox.yml ``` -------------------------------- ### HTTP Probe YAML Configuration Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/config.md Example of how to configure an HTTP probe in YAML, specifying status codes, method, headers, body, and validation rules. ```yaml http_2xx: prober: http http: valid_status_codes: [200, 201, 202] method: POST headers: Content-Type: application/json body: '{"test": true}' fail_if_body_matches_regexp: - "error" fail_if_header_not_matches: - header: "Content-Type" regexp: "application/json" fail_if_not_ssl: true compression: gzip ``` -------------------------------- ### Example Blackbox Exporter Configuration Output Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/endpoints.md This is an example of the YAML output you might receive when querying the /config endpoint, showing the structure of loaded modules and their settings. ```yaml modules: http_2xx: prober: http timeout: 0s http: valid_status_codes: [200, 201] valid_http_versions: [] preferred_ip_protocol: ip4 ip_protocol_fallback: true skip_resolve_phase_with_proxy: false fail_if_ssl: false fail_if_not_ssl: false method: GET headers: null fail_if_body_matches_regexp: [] fail_if_body_not_matches_regexp: [] ``` -------------------------------- ### Basic HTTP Probe Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Use this command to perform a basic HTTP probe against a target URL. Ensure the 'http_2xx' module is configured. ```bash curl "http://localhost:9115/probe?target=https://example.com&module=http_2xx" ``` -------------------------------- ### GetById Example Usage Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/history.md Example of how to use the GetById method to retrieve and print debug output of a result. Ensure the ID exists in the history. ```go result := rh.GetById(42) if result != nil { fmt.Println(result.DebugOutput) } ``` -------------------------------- ### Complete Blackbox Exporter Configuration Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/configuration.md A comprehensive configuration file demonstrating various probe modules including HTTP, HTTPS, TCP, DNS, ICMP, and gRPC. ```yaml modules: # Simple HTTP 2xx check http_2xx: prober: http timeout: 5s http: valid_status_codes: [200, 201] follow_redirects: true # HTTPS with body validation https_check: prober: http timeout: 10s http: fail_if_not_ssl: true fail_if_body_matches_regexp: - "error" fail_if_body_not_matches_regexp: - "OK" # TCP SMTP probe smtp_check: prober: tcp timeout: 10s tcp: query_response: - expect: "^220 .* " - send: "EHLO prober " expect: "^250-.* " - send: "QUIT " expect: "^221 .* " # DNS resolution check dns_query: prober: dns timeout: 5s dns: query_name: "example.com" query_type: "A" preferred_ip_protocol: "ip4" # ICMP echo check icmp_check: prober: icmp timeout: 5s icmp: preferred_ip_protocol: "ip4" # gRPC health check grpc_check: prober: grpc timeout: 10s grpc: service: "example.v1.ExampleService" ``` -------------------------------- ### Reload Configuration and Handle Errors Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/config.md Example demonstrating how to call the ReloadConfig method and log any errors that occur during the configuration reload process. ```go err := sc.ReloadConfig("blackbox.yml", logger) if err != nil { logger.Error("Config reload failed", "err", err) } ``` -------------------------------- ### TLS Configuration Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/configuration.md Configure TLS settings for secure probes. Ensure certificate files are correctly specified and server name matches the target. ```yaml tls_config: insecure_skip_verify: false ca_file: /path/to/ca.crt cert_file: /path/to/cert.crt key_file: /path/to/key.key server_name: example.com min_version: "TLSv1.2" max_version: "TLSv1.3" cipher_suites: - TLS_RSA_WITH_AES_128_CBC_SHA ``` -------------------------------- ### DNS Probe Configuration Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example YAML configuration for a DNS probe. It specifies preferred IP protocol, query name and type, recursion desired, valid response codes, and answer record validation. ```yaml dns: preferred_ip_protocol: "ip4" ip_protocol_fallback: true query_name: "example.com" query_type: "A" query_class: "IN" recursion_desired: true valid_rcodes: ["NOERROR", "NXDOMAIN"] dns_over_tls: false transport_protocol: "udp" validate_answer_rrs: fail_if_matches_regexp: - "192\.0\.2\.1" validate_authority_rrs: {} validate_additional_rrs: {} ``` -------------------------------- ### Instantiate SafeConfig with Prometheus Registry Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/config.md Example of how to create a new SafeConfig instance, passing the default Prometheus registry for metric registration. ```go registry := prometheus.DefaultRegisterer sc := config.NewSafeConfig(registry) ``` -------------------------------- ### Basic Probe Request Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/handler.md Example of how to probe google.com using the http_2xx module. This is a standard probe request. ```bash curl "http://localhost:9115/probe?target=google.com&module=http_2xx" ``` -------------------------------- ### Example Blackbox Exporter Probe Response Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/metrics.md This is an example of the raw metrics output from a Blackbox Exporter probe. It includes details like probe duration, HTTP status code, content length, SSL expiry, and success status. ```text # HELP probe_duration_seconds Returns how long the probe took to complete in seconds # TYPE probe_duration_seconds gauge probe_duration_seconds 0.123456 # HELP probe_http_content_length Length of http response # TYPE probe_http_content_length gauge probe_http_content_length 1234 # HELP probe_http_status_code Response HTTP status code # TYPE probe_http_status_code gauge probe_http_status_code 200 # HELP probe_http_version Returns the HTTP protocol version used by the probe # TYPE probe_http_version gauge probe_http_version 2 # HELP probe_ssl_earliest_cert_expiry Returns last SSL chain expiry in unixtime # TYPE probe_ssl_earliest_cert_expiry gauge probe_ssl_earliest_cert_expiry 1.7234567e+09 # HELP probe_success Displays whether or not the probe was a success # TYPE probe_success gauge probe_success 1 # HELP probe_timeout_seconds Returns how long the probe timeout is in seconds # TYPE probe_timeout_seconds gauge probe_timeout_seconds 119.5 ``` -------------------------------- ### Configure DNS Probe Module Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/configuration.md Example configuration for the DNS probe module, specifying query parameters and validation rules. ```yaml modules: dns_query: prober: dns timeout: 10s dns: preferred_ip_protocol: "ip4" ip_protocol_fallback: true source_ip_address: "0.0.0.0" query_name: "example.com" query_type: "A" query_class: "IN" recursion_desired: true dns_over_tls: false transport_protocol: "udp" valid_rcodes: - NOERROR - NXDOMAIN validate_answer_rrs: fail_if_matches_regexp: - "^example\.com\.\s+IN\s+A\s+127\.0\.0\.1$" validate_authority_rrs: fail_if_none_matches_regexp: - "example\.com" validate_additional_rrs: fail_if_all_match_regexp: - "127\.0\.0\.1" ``` -------------------------------- ### Create CELProgram Instance Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/config.md Example of creating a new CELProgram instance with a given expression string. This is used for evaluating configuration conditions. ```go prog, err := config.NewCELProgram("body.status == 'ok'") ``` -------------------------------- ### DNS Query Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Perform a DNS query for a given target using the 'dns_query' module. This is useful for verifying DNS resolution. ```bash curl "http://localhost:9115/probe?target=8.8.8.8&module=dns_query" ``` -------------------------------- ### Get Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Retrieves the current configuration of the Blackbox Exporter in YAML format. ```APIDIDOC ## GET /config ### Description Retrieves the current configuration of the Blackbox Exporter in YAML format. ### Method GET ### Endpoint /config ### Response #### Success Response (200) - **config** (string) - The exporter configuration in YAML format. ``` -------------------------------- ### HTTP Probe with Debug Output Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Illustrates how to enable debug output for an HTTP probe using curl. This is helpful for troubleshooting probe failures. ```bash # Debug output curl "http://localhost:9115/probe?target=https://example.com&module=http_2xx&debug=true" ``` -------------------------------- ### Probe Request with Debug Output Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/handler.md Example of probing google.com with the http_2xx module and enabling debug output. This provides detailed logs and metrics. ```bash curl "http://localhost:9115/probe?target=google.com&module=http_2xx&debug=true" ``` -------------------------------- ### Test WebSocket Server Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of testing a WebSocket server endpoint using curl. This demonstrates targeting a standard ws:// URL. ```bash # Test WebSocket server curl "http://localhost:9115/probe?target=ws://api.example.com/socket&module=websocket" ``` -------------------------------- ### gRPC Probe Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example configuration for the gRPC probe, specifying service name, TLS settings, preferred IP protocol, and custom metadata headers. ```yaml grpc: service: "myservice.v1.MyService" tls: true preferred_ip_protocol: "ip4" ip_protocol_fallback: true metadata: authorization: - "Bearer token123" x-custom-header: - "value" tls_config: insecure_skip_verify: false ca_file: /path/to/ca.crt ``` -------------------------------- ### HTTP Probe Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example YAML configuration for the HTTP probe, specifying valid status codes, HTTP versions, headers, SSL settings, body matching, redirects, and compression. ```yaml http: valid_status_codes: [200, 201] valid_http_versions: ["HTTP/1.1", "HTTP/2.0"] method: GET headers: User-Agent: MyProbe/1.0 fail_if_ssl: false fail_if_not_ssl: false fail_if_body_matches_regexp: - "error" fail_if_body_not_matches_regexp: - "success" follow_redirects: true compression: gzip body_size_limit: 10MB enable_http2: true enable_http3: false ``` -------------------------------- ### Test Docker Unix Socket Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of testing a Docker Unix domain socket using curl. This demonstrates how to target a specific socket file. ```bash # Test Docker socket curl "http://localhost:9115/probe?target=/var/run/docker.sock&module=unix_docker" ``` -------------------------------- ### Test Custom Protocol Unix Socket Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of testing a custom protocol over a Unix domain socket. This shows how to specify a module for a non-standard protocol. ```bash # Custom protocol socket curl "http://localhost:9115/probe?target=/tmp/myservice.sock&module=unix_custom" ``` -------------------------------- ### Validate Blackbox Exporter Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/configuration.md Use the `--config.check` flag to validate the configuration file syntax and options without starting the exporter. ```bash ./blackbox_exporter --config.file=blackbox.yml --config.check ``` -------------------------------- ### Test Secure WebSocket Server Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example of testing a secure WebSocket (wss://) server endpoint using curl. This demonstrates targeting a secure WebSocket URL. ```bash # Secure WebSocket curl "http://localhost:9115/probe?target=wss://api.example.com/socket&module=websocket" ``` -------------------------------- ### Generate HTTP Probe with Debug Output Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md Use curl to send a probe request to the Blackbox Exporter and view the debug output. This example shows the output when the prober log level is set to its default (info). ```shell curl -sL "http://localhost:9115/probe?target=prometheus.io&module=http_2xx&debug=true" ``` ```shell ~ $ ./blackbox_exporter --config.file ./blackbox.yml time=2025-09-09T00:58:02.557-04:00 level=INFO source=main.go:95 msg="Starting blackbox_exporter" version="(version=0.27.0, branch=feat/make-scrape-logger-independent, revision=47e27d09847edf2ade2732b50663c37ed8177485)" time=2025-09-09T00:58:02.557-04:00 level=INFO source=main.go:96 msg="(go=go1.24.4, platform=linux/amd64, user=tjhop@contraband, date=20250909-04:57:55, tags=unknown)" time=2025-09-09T00:58:02.557-04:00 level=INFO source=main.go:108 msg="Loaded config file" time=2025-09-09T00:58:02.558-04:00 level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9115 time=2025-09-09T00:58:02.558-04:00 level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9115 time=2025-09-09T00:58:06.756-04:00 level=WARN source=http.go:490 msg="Received redirect" module=http_2xx target=prometheus.io location=https://prometheus.io/ ^Ctime=2025-09-09T00:58:12.257-04:00 level=INFO source=main.go:290 msg="Received SIGTERM, exiting gracefully..." ``` ```shell ~ $ curl -sL "http://localhost:9115/probe?target=prometheus.io&module=http_2xx&debug=true" | sed '/Metrics that would have been returned/q' Logs for the probe: time=2025-09-09T00:58:06.756-04:00 level=WARN source=http.go:490 msg="Received redirect" module=http_2xx target=prometheus.io location=https://prometheus.io/ Metrics that would have been returned: ``` -------------------------------- ### Get Blackbox Exporter Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/endpoints.md Retrieve the currently loaded configuration of the blackbox exporter in YAML format. This is useful for verifying the active settings. ```bash curl "http://localhost:9115/config" ``` -------------------------------- ### Prometheus Blackbox Exporter Basic Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md Configure Prometheus to use the Blackbox Exporter for probing targets. This example uses static targets and relabeling to set the probe target and exporter address. ```yaml scrape_configs: - job_name: 'blackbox' metrics_path: /probe params: module: [http_2xx] # Look for a HTTP 200 response. static_configs: - targets: - http://prometheus.io # Target to probe with http. - https://prometheus.io # Target to probe with https. - http://example.com:8080 # Target to probe with http on port 8080. relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ replacement: 127.0.0.1:9115 # The blackbox exporter's real hostname:port. - job_name: 'blackbox_exporter' # collect blackbox exporter's operational metrics. static_configs: - targets: ['127.0.0.1:9115'] ``` -------------------------------- ### HTTP Probe with Body Validation Example Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Shows how to use curl to perform an HTTP probe that includes body validation against a target API endpoint. This is useful for verifying API health. ```bash # With body validation curl "http://localhost:9115/probe?target=https://api.example.com/health&module=http_api" ``` -------------------------------- ### Probe Request with Custom Hostname Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/handler.md Example of probing an IP address with a custom hostname for testing virtual hosts. This sets the Host header and TLS ServerName. ```bash curl "http://localhost:9115/probe?target=192.0.2.1&module=http_2xx&hostname=example.com" ``` -------------------------------- ### Blackbox Exporter with DNS Service Discovery and Hostname Parameter Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md Configure Prometheus to use Blackbox Exporter with DNS service discovery. This example probes targets found via DNS and sets the 'Host' header and TLS SNI using the discovered domain name. ```yaml scrape_configs: - job_name: blackbox_all metrics_path: /probe params: module: [ http_2xx ] # Look for a HTTP 200 response. dns_sd_configs: - names: - example.com - prometheus.io type: A port: 443 relabel_configs: - source_labels: [__address__] target_label: __param_target replacement: https://$1/ # Make probe URL be like https://1.2.3.4:443/ - source_labels: [__param_target] target_label: instance - target_label: __address__ replacement: 127.0.0.1:9115 # The blackbox exporter's real hostname:port. - source_labels: [__meta_dns_name] target_label: __param_hostname # Make domain name become 'Host' header for probe requests - source_labels: [__meta_dns_name] target_label: vhost # and store it in 'vhost' label ``` -------------------------------- ### DNS Validation Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Sets up a DNS module to query for 'example.com' of type 'A', validating the return code and ensuring the answer does not match a specific IP range. ```yaml modules: dns_example: prober: dns dns: query_name: "example.com" query_type: "A" valid_rcodes: [NOERROR] validate_answer_rrs: fail_if_matches_regexp: - "127\.0\.0\.1" ``` -------------------------------- ### GET /metrics Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/endpoints.md Retrieves Prometheus metrics for the Blackbox Exporter itself. This endpoint does not return probe-specific metrics. ```APIDOC ## GET /metrics ### Description Returns Prometheus metrics for the blackbox exporter itself (not probe metrics). ### Method GET ### Endpoint /metrics ### Response Format Prometheus text format metrics including: - **blackbox_exporter_config_last_reload_successful** - 0 or 1, success status of last config reload - **blackbox_exporter_config_last_reload_success_timestamp_seconds** - Timestamp of last successful reload - **blackbox_module_unknown_total** - Counter of probes requesting unknown modules - **process_*** - Go runtime metrics - **go_*** - Go runtime details ### Request Example ```bash curl "http://localhost:9115/metrics" ``` ``` -------------------------------- ### Get Debug Logs Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Retrieves debug output for previously executed probes, useful for troubleshooting. ```APIDOC ## GET /logs ### Description Retrieves debug output for previous probe executions. ### Method GET ### Endpoint /logs ### Parameters #### Query Parameters - **id** (int64) - Optional - The ID of the probe result to retrieve debug output for. - **target** (string) - Optional - Filter by target. - **module** (string) - Optional - Filter by module name. ``` -------------------------------- ### Run Blackbox Exporter binary Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md Execute the binary directly using command-line flags. ```bash ./blackbox_exporter ``` -------------------------------- ### Check Specific gRPC Service Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example using curl to check the health of a specific gRPC service by name. ```bash # Check specific service curl "http://localhost:9115/probe?target=api.example.com:50051&module=grpc_myservice" ``` -------------------------------- ### GET /logs Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/history.md Retrieves debug output for a specific probe result or the latest result for a given target and module. ```APIDOC ## GET /logs ### Description Retrieve debug output for a specific probe result or the latest result for a given target and module. ### Method GET ### Endpoint /logs ### Parameters #### Query Parameters - **id** (int64, optional) - Result ID from history - **target** (string, optional) - Target to search for - **module** (string, optional) - Module filter (only works with target parameter) ### Response #### Success Response (200) - **Id** (int64) - Unique identifier of the probe result. - **ModuleName** (string) - Name of the probe module executed. - **Target** (string) - The target that was probed. - **DebugOutput** (string) - Full plaintext output including logs, metrics, and configuration. - **Success** (bool) - Whether the probe succeeded (true) or failed (false). #### Response Example { "Id": 42, "ModuleName": "http_2xx", "Target": "https://example.com", "DebugOutput": "...debug output...", "Success": true } ### Error Handling - Returns 400 if neither `id` nor `target` are specified, or if both are specified. - Returns 404 if the result is not found. ``` -------------------------------- ### Check gRPC Service Health Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/probes.md Example using curl to check the health of a gRPC service at a specific target and module. ```bash # Check gRPC service health curl "http://localhost:9115/probe?target=api.example.com:50051&module=grpc_health" ``` -------------------------------- ### Build Blackbox Exporter Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md Commands to build the software locally or via Docker. ```shell make ``` ```shell docker build -t blackbox_exporter . ``` -------------------------------- ### Get Blackbox Exporter Metrics Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/endpoints.md Use this endpoint to retrieve Prometheus metrics about the blackbox exporter itself. This does not include probe-specific metrics. ```bash curl "http://localhost:9115/metrics" ``` -------------------------------- ### Execute Probe Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Executes a probe against a target and returns the collected metrics. This endpoint can be accessed via GET or POST requests. ```APIDOC ## GET /probe ### Description Executes a probe against a target and returns the collected metrics. ### Method GET ### Endpoint /probe ### Parameters #### Query Parameters - **module** (string) - Required - The name of the module to use for the probe. - **target** (string) - Required - The target to probe. - **debug** (string) - Optional - Enable debug output. ### Response #### Success Response (200) - **metrics** (object) - The collected metrics from the probe. - **debug_output** (string) - Debug information if requested. ``` ```APIDOC ## POST /probe ### Description Executes a probe against a target and returns the collected metrics. This endpoint is useful for providing probe details in the request body. ### Method POST ### Endpoint /probe ### Parameters #### Request Body - **module** (string) - Required - The name of the module to use for the probe. - **target** (string) - Required - The target to probe. ### Response #### Success Response (200) - **metrics** (object) - The collected metrics from the probe. - **debug_output** (string) - Debug information if requested. ``` -------------------------------- ### Regexp Constructor Functions Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/types.md Provides functions to create new Regexp instances. NewRegexp returns a Regexp and an error, while MustNewRegexp panics if the input string is invalid. ```go func NewRegexp(s string) (Regexp, error) func MustNewRegexp(s string) Regexp ``` -------------------------------- ### Probe Endpoint Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/MANIFEST.md Initiates a probe to a specified target using various protocols. Supports both GET and POST methods for flexibility. ```APIDOC ## GET/POST /probe ### Description Initiates a probe to a specified target. The probe type and parameters are defined in the request body or query parameters. ### Method GET, POST ### Endpoint /probe ### Parameters #### Query Parameters - **target** (string) - Required - The target address to probe. - **module** (string) - Optional - The probe module to use (e.g., http, tcp, dns). #### Request Body - **target** (string) - Required - The target address to probe. - **module** (string) - Optional - The probe module to use. - **params** (object) - Optional - Module-specific parameters. ``` -------------------------------- ### GET /logs Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/endpoints.md Retrieves debug output and logs for a previous probe result. You can query by a specific result ID or by target and module. ```APIDOC ## GET /logs ### Description Retrieves debug output and logs for a previous probe result. ### Method GET ### Endpoint /logs ### Parameters #### Query Parameters - **id** (int64) - Required - Mutually exclusive with target. Result ID from history. - **target** (string) - Required - Mutually exclusive with id. Target to search for. - **module** (string) - Optional - Only with target. Module name filter. ### Response Codes - **200** - Success - **404** - Result not found ### Response Format Returns the complete debug output from ResultHistory, including: 1. Logs section with probe execution logs 2. Metrics section with Prometheus metrics that would have been returned 3. Configuration section showing the module used ### Request Example ```bash # Retrieve result by ID curl "http://localhost:9115/logs?id=42" # Retrieve latest result for a target curl "http://localhost:9115/logs?target=https://example.com" # Retrieve latest result for target + module curl "http://localhost:9115/logs?target=https://example.com&module=http_2xx" ``` ``` -------------------------------- ### Define basic probe module Source: https://github.com/prometheus/blackbox_exporter/blob/master/README.md Minimal configuration for an HTTP probe module. ```yaml # blackbox.yml modules: http_2xx: prober: http ``` -------------------------------- ### Current Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/INDEX.md Fetches the currently loaded configuration of the Blackbox Exporter. ```APIDOC ## GET /config ### Description Returns the current configuration of the Blackbox Exporter. ### Method GET ### Endpoint /config ### Response #### Success Response (200) - **Configuration** (object) - The exporter's configuration in YAML format. ``` -------------------------------- ### Module Configuration Structure Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/README.md Defines the structure for configuring individual probes within the Blackbox Exporter. Specifies the prober type and protocol-specific settings. ```go type Module struct { Prober string // "http", "tcp", "icmp", "dns", "grpc", "unix", "websocket" Timeout time.Duration HTTP HTTPProbe TCP TCPProbe ICMP ICMPProbe DNS DNSProbe GRPC GRPCProbe Unix UnixProbe Websocket WebsocketProbe } ``` -------------------------------- ### HTTP Client Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/CONFIGURATION.md Standard HTTP client settings including authentication, proxy configuration, and TLS options. ```yaml # The HTTP basic authentication credentials. basic_auth: [ username: ] [ password: ] [ username_file: ] [ password_file: ] # Sets the `Authorization` header on every request with # the configured credentials. authorization: # Sets the authentication type of the request. [ type: | default: Bearer ] # Sets the credentials of the request. It is mutually exclusive with # `credentials_file`. [ credentials: ] # Sets the credentials of the request with the credentials read from the # configured file. It is mutually exclusive with `credentials`. [ credentials_file: ] # OAuth 2.0 configuration to use to connect to the targets. oauth2: [ ] # Whether to enable HTTP2. [ enable_http2: | default: true ] # Configuration for TLS protocol of HTTP probe. tls_config: [ ] # HTTP proxy server to use to connect to the targets. [ proxy_url: ] # Comma-separated string that can contain IPs, CIDR notation, domain names # that should be excluded from proxying. IP and domain names can # contain port numbers. [ no_proxy: ] # Use proxy URL indicated by environment variables (HTTP_PROXY, https_proxy, HTTPs_PROXY, https_proxy, and no_proxy) [ proxy_from_environment: | default: false ] # Specifies headers to send to proxies during CONNECT requests. [ proxy_connect_header: [ : [, ...] ] ] # Whether or not the probe will follow any redirects. [ follow_redirects: | default = true ] ``` -------------------------------- ### Retrieve Latest Probe Result for a Target Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/history.md Use this curl command to get the most recent probe result for a given target URL or hostname. ```bash # Retrieve latest result for a target curl "http://localhost:9115/logs?target=https://example.com" ``` -------------------------------- ### UnixProbe Configuration Structure Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/types.md Defines the configuration for a Unix domain socket probe. Supports sending and expecting patterns, and TLS settings. ```go type UnixProbe struct { QueryResponse []QueryResponse `yaml:"query_response,omitempty"` TLS bool `yaml:"tls,omitempty"` TLSConfig config.TLSConfig `yaml:"tls_config,omitempty"` } ``` -------------------------------- ### Initialize ResultHistory Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/api-reference/history.md Initialize ResultHistory with the desired history capacity. Set the MaxResults field to control the size of the rolling window. ```go rh := &ResultHistory{MaxResults: 100} ``` -------------------------------- ### Custom Module Probe Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/endpoints.md Uses a custom probe module for the blackbox exporter. Ensure the custom module is correctly configured within the exporter. ```bash curl "http://localhost:9115/probe?target=10.0.0.1&module=my_custom_module" ``` -------------------------------- ### Configure Unix Domain Socket Probe Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/configuration.md This configuration sets up a probe for Unix domain sockets. It allows defining send and expect patterns for communication. ```yaml modules: unix_check: prober: unix timeout: 10s unix: query_response: - send: "PING\n" expect: "^PONG\n" tls: false ``` -------------------------------- ### Configure WebSocket Probe Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/configuration.md Configure a WebSocket probe for checking WebSocket services. This snippet includes settings for IP protocol, fallback, custom headers, and query/response patterns. ```yaml modules: websocket_check: prober: websocket timeout: 10s websocket: preferred_ip_protocol: "ip4" ip_protocol_fallback: true headers: User-Agent: MyProber/1.0 query_response: - send: '{"action": "ping"}' expect: '{"action": "pong"}' ``` -------------------------------- ### CELProgram Constructor Functions Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/types.md Provides functions to create new CELProgram instances. NewCELProgram returns a CELProgram and an error, while MustNewCELProgram panics if the input string is invalid. ```go func NewCELProgram(s string) (CELProgram, error) func MustNewCELProgram(s string) CELProgram ``` -------------------------------- ### Define WebsocketProbe Configuration Source: https://github.com/prometheus/blackbox_exporter/blob/master/_autodocs/types.md Defines the structure for configuring a WebSocket probe, including HTTP client settings, custom headers, query/response patterns, and IP protocol preferences. ```go type WebsocketProbe struct { HTTPClientConfig config.HTTPClientConfig `yaml:"http_config,omitempty"` Headers config.Headers `yaml:"headers,omitempty"` QueryResponse []QueryResponse `yaml:"query_response,omitempty"` IPProtocol string `yaml:"preferred_ip_protocol,omitempty"` IPProtocolFallback bool `yaml:"ip_protocol_fallback,omitempty"` } ```