### Development Setup Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command for a development Pushgateway setup with debug logging. ```bash pushgateway \ --web.listen-address=127.0.0.1:9091 \ --persistence.file=/tmp/pushgateway.db \ --log.level=debug \ --push.enable-utf8-names ``` -------------------------------- ### Basic Setup Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command for a basic in-memory Pushgateway setup without persistence. ```bash pushgateway \ --web.listen-address=:9091 \ --web.telemetry-path=/metrics \ --log.level=info ``` -------------------------------- ### Start with Defaults Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command to start Pushgateway with default configuration. ```bash ./pushgateway ``` -------------------------------- ### Production Setup Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command for a production Pushgateway setup with persistence and security. ```bash pushgateway \ --web.listen-address=:9091 \ --web.external-url=http://pushgateway.example.com/ \ --web.config.file=/etc/pushgateway/web-config.yml \ --persistence.file=/var/lib/pushgateway/metrics.db \ --persistence.interval=5m \ --log.level=info \ --log.format=json ``` -------------------------------- ### Development Setup with Admin API Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command for a development Pushgateway setup with admin API enabled. ```bash pushgateway \ --web.listen-address=127.0.0.1:9091 \ --persistence.file=/tmp/pushgateway.db \ --web.enable-lifecycle \ --web.enable-admin-api \ --log.level=debug ``` -------------------------------- ### computeRoutePrefix examples Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Examples demonstrating the behavior of the computeRoutePrefix function with different inputs. ```go // Example 1: Explicit prefix computeRoutePrefix("/pushgateway", &url.URL{Path: "/"}) // Returns: "/pushgateway" // Example 2: From external URL computeRoutePrefix("", &url.URL{Scheme: "http", Host: "example.com", Path: "/pg"}) // Returns: "/pg" // Example 3: Root path computeRoutePrefix("/", &url.URL{Path: "/"}) // Returns: "" (empty for root) // Example 4: Path normalization computeRoutePrefix("/api/", &url.URL{}) // Returns: "/api" (trailing slash removed) ``` -------------------------------- ### Enable UTF-8 Names Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command to allow UTF-8 characters in metric and label names, and an example of a metric that would be accepted. ```bash pushgateway --push.enable-utf8-names # Now accepts metrics like: # métrique_température{région="Paris"} 20.5 ``` -------------------------------- ### Log Format Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command to set the log output format to JSON. ```bash pushgateway --log.format=json ``` -------------------------------- ### decodeRequest example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Example demonstrating how to wrap a handler with decodeRequest for automatic request decompression. ```go // Raw handler metricsHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // r.Body is already decompressed parseMetrics(r.Body) }) // Wrapped with decompression wrapped := decodeRequest(metricsHandler) // Can now handle: // - Content-Encoding: gzip // - Content-Encoding: snappy // - No encoding ``` -------------------------------- ### Route Prefix Application Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Illustrates how routes are prefixed with --web.route-prefix, using /pg as an example. ```shell Example with --web.route-prefix=/pg: GET /pg/-/healthy GET /pg/-/ready GET /pg/metrics GET /pg/status POST /pg/metrics/job/my_job PUT /api/v1/ → /pg/api/v1/ (via mux) ``` -------------------------------- ### Enable Lifecycle Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example of enabling the --web.enable-lifecycle flag and triggering a graceful shutdown. ```bash pushgateway --web.enable-lifecycle curl -X POST http://localhost:9091/-/quit ``` -------------------------------- ### Log Level Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command to set the log level to debug. ```bash pushgateway --log.level=debug ``` -------------------------------- ### API Status Endpoint Example Source: https://github.com/prometheus/pushgateway/blob/master/README.md Example of how to retrieve build information, command line flags, and start time from the /api/v1/status endpoint. ```shell curl -X GET http://pushgateway.example.org:9091/api/v1/status | jq ``` ```json { "status": "success", "data": { "build_information": { "branch": "master", "buildDate": "20200310-20:14:39", "buildUser": "flipbyte@localhost.localdomain", "goVersion": "go1.13.6", "revision": "eba0ec4100873d23666bcf4b8b1d44617d6430c4", "version": "1.1.0" }, "flags": { "log.format": "logfmt", "log.level": "info", "persistence.file": "", "persistence.interval": "5m0s", "push.disable-consistency-check": "false", "web.enable-admin-api": "false", "web.enable-lifecycle": "false", "web.external-url": "", "web.listen-address": ":9091", "web.route-prefix": "", "web.telemetry-path": "/metrics" }, "start_time": "2020-03-11T01:44:49.9189758+05:30" } } ``` -------------------------------- ### Status Endpoint Response Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-api-v1.md Example JSON structure for the response of the GET /api/v1/status endpoint, including build information, flags, and start time. ```json { "status": "success", "data": { "build_information": { "version": "0.15.0", "revision": "a1b2c3d", "branch": "main", "buildUser": "builder", "buildDate": "2024-01-01T00:00:00Z", "goVersion": "go1.25.0" }, "flags": { "web.listen-address": ":9091", "web.telemetry-path": "/metrics", "persistence.file": "" }, "start_time": "2024-01-01T12:00:00Z" } } ``` -------------------------------- ### NewDiskMetricStore Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-storage.md Example usage of the NewDiskMetricStore function. ```go persistenceFile := "/var/lib/pushgateway/metrics.db" persistenceInterval := 5 * time.Minute logger := promslog.New(&promslog.Config{}) ms := storage.NewDiskMetricStore(persistenceFile, persistenceInterval, nil, logger) defers ms.Shutdown() ``` -------------------------------- ### Web Listen Address Examples Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Examples demonstrating how to set the --web.listen-address flag to specify the address and port for the Pushgateway to listen on. ```bash pushgateway --web.listen-address=:9091 pushgateway --web.listen-address=127.0.0.1:9091 pushgateway --web.listen-address=0.0.0.0:9091 ``` -------------------------------- ### GetMetricFamilies Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-storage.md Example of retrieving and iterating over MetricFamilies. ```go families := ms.GetMetricFamilies() for _, family := range families { fmt.Printf("Metric: %s (Type: %s)\n", family.GetName(), family.GetType()) for _, metric := range family.GetMetric() { fmt.Printf(" Labels: %v\n", metric.GetLabel()) } } ``` -------------------------------- ### Example: shutdownServerOnQuit Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Go code demonstrating how to use the shutdownServerOnQuit function. ```go quitCh := make(chan struct{}) server := &http.Server{Handler: mux} // Run in background go shutdownServerOnQuit(server, quitCh, logger) // Later: trigger shutdown via channel close(quitCh) // Or via signal: kill -TERM // Or via HTTP: curl -X POST http://localhost:9091/-/quit ``` -------------------------------- ### Enable Admin API Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example of enabling the --web.enable-admin-api flag and using the wipe endpoint. ```bash pushgateway --web.enable-admin-api curl -X PUT http://localhost:9091/api/v1/admin/wipe ``` -------------------------------- ### SortedLabels method example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/types.md Example usage of the SortedLabels method. ```go labels := metricGroup.SortedLabels() // Returns: ["job", "custom_label", "instance"] ``` -------------------------------- ### Ready Handler Example Usage Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-handler.md Example of how to use the Ready handler with HTTP routing. ```go handler := handler.Ready(ms) r.Get("/-/ready", handler.ServeHTTP) ``` -------------------------------- ### Server Startup Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Initializes and starts the HTTP server, including graceful shutdown. ```go server := &http.Server{Handler: mux} go shutdownServerOnQuit(server, quitCh, logger) err := web.ListenAndServe(server, webConfig, logger) ``` -------------------------------- ### Persistence File Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example of setting the --persistence.file flag to specify a path for persisting metrics. ```bash pushgateway --persistence.file=/var/lib/pushgateway/metrics.db ``` -------------------------------- ### Persistence Interval Examples Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Examples of setting the --persistence.interval flag to control the frequency of persistence. ```bash pushgateway --persistence.interval=1m pushgateway --persistence.interval=30s ``` -------------------------------- ### SubmitWriteRequest Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-storage.md Example of submitting a write request to DiskMetricStore. ```go ms.SubmitWriteRequest(storage.WriteRequest{ Labels: map[string]string{ "job": "batch_job_1", "instance": "server1", }, Timestamp: time.Now(), MetricFamilies: metricFamilies, Replace: true, }) ``` -------------------------------- ### 405 Method Not Allowed Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md Example of attempting to use a GET request on the /-/quit endpoint. ```bash curl -X GET http://localhost:9091/-/quit # Response: 405 Method Not Allowed # Body: "Only POST or PUT requests allowed." ``` -------------------------------- ### Telemetry Path Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example of setting the --web.telemetry-path flag and accessing the metrics endpoint. ```bash pushgateway --web.telemetry-path=/prometheus_metrics curl http://localhost:9091/prometheus_metrics ``` -------------------------------- ### LastPushSuccess method example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/types.md Example usage of the LastPushSuccess method. ```go if !metricGroup.LastPushSuccess() { fmt.Println("Last push for this group failed") } ``` -------------------------------- ### GET /api/v1/metrics Response Body Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-api-v1.md Example JSON response body for the GET /api/v1/metrics endpoint, showing organized pushed metrics. ```json { "status": "success", "data": [ { "labels": { "job": "batch_job", "instance": "server1" }, "last_push_successful": true, "process_cpu_seconds_total": { "type": "COUNTER", "help": "Total user and system CPU time spent in seconds.", "time_stamp": "2024-01-01T12:30:00Z", "metrics": [ { "labels": {}, "value": "123.45" } ] }, "request_duration_seconds": { "type": "HISTOGRAM", "help": "Request duration in seconds", "time_stamp": "2024-01-01T12:30:00Z", "metrics": [ { "labels": {"handler": "/push"}, "count": "100", "sum": "45.67", "buckets": { "0.1": "10", "0.5": "60", "1.0": "95", "+Inf": "100" } } ] } } ] } ``` -------------------------------- ### Example Usage: pprof Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Bash commands to interact with the pprof endpoints. ```bash # Get profile index curl http://localhost:9091/debug/pprof/ # Get 30-second CPU profile curl http://localhost:9091/debug/pprof/profile?seconds=30 > cpu.prof # Analyze with pprof go tool pprof cpu.prof ``` -------------------------------- ### Route Prefix Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example of setting the --web.route-prefix flag to configure the prefix for web endpoints. ```bash pushgateway --web.route-prefix=/pushgateway # Routes become: # GET /pushgateway/metrics # GET /pushgateway/status # POST /pushgateway/metrics/job/... ``` -------------------------------- ### GetMetricFamily method example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/types.md Example usage of the GetMetricFamily method. ```go family := tmf.GetMetricFamily() fmt.Printf("Metric: %s (type: %s)\n", family.GetName(), family.GetType()) ``` -------------------------------- ### APIBucket Example (Float Histogram) Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-histogram.md Example of creating an APIBucket for a float histogram. ```Go bucket := APIBucket[uint64]{ Boundaries: 1, // [) bucket Lower: 0.1, Upper: 0.5, Count: 25, } // Represents bucket [0.1, 0.5) with 25 samples ``` -------------------------------- ### Example curl command for /api/v1/metrics Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-api-v1.md Example command to fetch metrics from the Pushgateway API. ```bash curl http://localhost:9091/api/v1/metrics | jq . ``` -------------------------------- ### Web Config File Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command to specify the path to a YAML configuration file for TLS, basic auth, and OAuth2 configuration. ```bash pushgateway --web.config.file=/etc/pushgateway/web-config.yml ``` -------------------------------- ### GetMetricFamiliesMap Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-storage.md Example of retrieving and iterating over metrics organized by grouping key. ```go groupMap := ms.GetMetricFamiliesMap() for groupingKey, metricGroup := range groupMap { fmt.Printf("Group: %s\n", groupingKey) fmt.Printf(" Job: %s\n", metricGroup.Labels["job"]) for metricName, tmf := range metricGroup.Metrics { fmt.Printf(" Metric: %s (timestamp: %s)\n", metricName, tmf.Timestamp) } } ``` -------------------------------- ### GetAPIFloatBuckets Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-histogram.md Example usage of GetAPIFloatBuckets with a sample FloatHistogram. ```Go floatHistogram := &model.FloatHistogram{ Count: 100.5, Sum: 45.67, PositiveBuckets: []float64{10.2, 30.1, 5.3}, // ... other fields } buckets := histogram.GetAPIFloatBuckets(floatHistogram) for _, bucket := range buckets { fmt.Printf("Bucket [%f, %f): %f\n", bucket.Lower, bucket.Upper, bucket.Count) } ``` -------------------------------- ### Pushing Metrics Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md Example of pushing metrics to the Pushgateway. ```bash cat > metrics.txt << EOF # HELP process_cpu_seconds Total CPU time # TYPE process_cpu_seconds counter process_cpu_seconds_total 123.45 EOF curl -X POST -H "Content-Type: text/plain" --data-binary @metrics.txt \ http://localhost:9091/metrics/job/batch_job/instance/server1 ``` -------------------------------- ### Example: Registering API Routes Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-api-v1.md Shows how to create a router and register the API handlers with it. ```go r := route.New() api.Register(r) // Routes are now: /status, /metrics, /:path (OPTIONS) ``` -------------------------------- ### External URL Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example of setting the --web.external-url flag to define the Pushgateway's external reachability. ```bash pushgateway --web.external-url=http://pushgateway.example.com/pushgateway ``` -------------------------------- ### Push Handler Example Usage Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-handler.md Example of how to use the Push handler with HTTP routing. ```go handler := handler.Push(ms, true, true, false, logger) r.Put("/metrics/job/:job/*labels", handler) r.Post("/metrics/job/:job/*labels", handler) ``` -------------------------------- ### 403 Forbidden Example Request/Response Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md Example of attempting to use the Lifecycle API without it being enabled. ```bash # Without --web.enable-lifecycle curl -X POST http://localhost:9091/-/quit # Response: 403 Forbidden # Body: "Lifecycle API is not enabled." ``` -------------------------------- ### Gzip and Snappy Compression Examples Source: https://github.com/prometheus/pushgateway/blob/master/README.md Examples demonstrating how to send gzip-compressed and snappy-compressed metric data to the Pushgateway using curl. ```bash echo "some_metric 3.14" | gzip | curl -H 'Content-Encoding: gzip' --data-binary @- http://pushgateway.example.org:9091/metrics/job/some_job echo "some_metric 3.14" | snzip | curl -H 'Content-Encoding: snappy' --data-binary @- http://pushgateway.example.org:9091/metrics/job/some_job ``` -------------------------------- ### APIBucket Usage Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/types.md Shows how to retrieve and iterate through APIBuckets for a histogram. ```go apiHistogram := histogram.GetAPIBuckets(h) for _, bucket := range apiHistogram { fmt.Printf("Bucket [%f, %f): %d\n", bucket.Lower, bucket.Upper, bucket.Count) } ``` -------------------------------- ### Startup Options - With Persistence Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md Starting the Pushgateway with disk persistence enabled. ```bash ./pushgateway \ --persistence.file=/data/metrics.db \ --persistence.interval=5m ``` -------------------------------- ### API Metrics Endpoint Example Source: https://github.com/prometheus/pushgateway/blob/master/README.md Example of how to retrieve pushed metric families from the /api/v1/metrics endpoint. ```shell curl -X GET http://pushgateway.example.org:9091/api/v1/metrics | jq ``` ```json { "status": "success", "data": [ { "labels": { "job": "batch" }, "last_push_successful": true, "my_job_duration_seconds": { "time_stamp": "2020-03-11T02:02:27.716605811+05:30", "type": "GAUGE", "help": "Duration of my batch job in seconds", "metrics": [ { "labels": { "instance": "", "job": "batch" }, "value": "0.2721322309989773" } ] }, "push_failure_time_seconds": { "time_stamp": "2020-03-11T02:02:27.716605811+05:30", "type": "GAUGE", "help": "Last Unix time when changing this group in the Pushgateway failed.", "metrics": [ { "labels": { "instance": "", "job": "batch" }, "value": "0" } ] }, "push_time_seconds": { "time_stamp": "2020-03-11T02:02:27.716605811+05:30", "type": "GAUGE", "help": "Last Unix time when changing this group in the Pushgateway succeeded.", "metrics": [ { "labels": { "instance": "", "job": "batch" }, "value": "1.5838723477166057e+09" } ] } } ] } ``` -------------------------------- ### Delete Handler Example Usage Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-handler.md Example of how to use the Delete handler with HTTP routing. ```go handler := handler.Delete(ms, false, logger) r.Del("/metrics/job/:job/*labels", handler) r.Del("/metrics/job/:job", handler) ``` -------------------------------- ### Healthy Handler Example Usage Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-handler.md Example of how to use the Healthy handler with HTTP routing. ```go handler := handler.Healthy(ms) r.Get("/-/healthy", handler.ServeHTTP) ``` -------------------------------- ### Startup Options - With Admin API Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md Starting the Pushgateway with the admin API enabled. ```bash ./pushgateway \ --web.enable-admin-api \ --web.enable-lifecycle ``` -------------------------------- ### Pushgateway Build Info Metric Source: https://github.com/prometheus/pushgateway/blob/master/README.md Example of the pushgateway_build_info metric, providing build details. ```text # HELP pushgateway_build_info A metric with a constant '1' value labeled by version, revision, branch, and goversion from which pushgateway was built. # TYPE pushgateway_build_info gauge pushgateway_build_info{branch="master",goversion="go1.10.2",revision="8f88ccb0343fc3382f6b93a9d258797dcb15f770",version="0.5.2"} 1 ``` -------------------------------- ### Complete Histogram Processing Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-histogram.md Example demonstrating the complete processing of histogram data, from API v1 metrics response building to model representation and conversion for API responses. ```Go import "github.com/prometheus/pushgateway/histogram" // From API v1 metrics response building dtoHistogram := metric.GetHistogram() // *dto.Histogram // Convert to model representation h, fh := histogram.NewModelHistogram(dtoHistogram) if h != nil { // Classic histogram apiBuckets := histogram.GetAPIBuckets(h) jsonBuckets := histogram.BucketsAsJson(apiBuckets) metricResponse["buckets"] = jsonBuckets metricResponse["count"] = fmt.Sprint(h.Count) } else if fh != nil { // Float histogram apiBuckets := histogram.GetAPIFloatBuckets(fh) jsonBuckets := histogram.BucketsAsJson(apiBuckets) metricResponse["buckets"] = jsonBuckets metricResponse["count"] = fmt.Sprint(fh.Count) } ``` -------------------------------- ### Typical Usage: Push/Update metrics Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/types.md Example of how to construct and submit a WriteRequest to push or update metrics. ```go req := storage.WriteRequest{ Labels: map[string]string{ "job": "my_job", "instance": "server1", }, Timestamp: time.Now(), MetricFamilies: metricFamilies, Replace: true, Done: make(chan error, 1), } ms.SubmitWriteRequest(req) if err := <-req.Done; err != nil { log.Printf("Push failed: %v", err) } ``` -------------------------------- ### GroupingKeyToMetricGroup Usage Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/types.md Demonstrates how to iterate through metric groups using the GroupingKeyToMetricGroup map. ```go groups := ms.GetMetricFamiliesMap() for groupingKey, metricGroup := range groups { fmt.Printf("Grouping Key: %s\n", groupingKey) fmt.Printf("Labels: %v\n", metricGroup.Labels) for metricName := range metricGroup.Metrics { fmt.Printf(" Metric: %s\n", metricName) } } ``` -------------------------------- ### NameToTimestampedMetricFamilyMap Usage Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/types.md Illustrates iterating through metrics in a MetricGroup and accessing their details. ```go for metricName, tmf := range metricGroup.Metrics { fmt.Printf("Metric: %s, Pushed at: %s\n", metricName, tmf.Timestamp) family := tmf.GetMetricFamily() for _, metric := range family.GetMetric() { fmt.Printf(" Sample labels: %v\n", metric.GetLabel()) } } ``` -------------------------------- ### WriteRequest Error Handling Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md Shows how to submit a WriteRequest and handle potential errors through the Done channel. ```go req := storage.WriteRequest{ // ... request details ... Done: make(chan error, 1), } ms.SubmitWriteRequest(req) // Block until processing is complete for err := range req.Done { if err != nil { log.Printf("Write failed: %v", err) } } ``` -------------------------------- ### Content Encoding - Snappy Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md Example of sending compressed metrics using snappy. ```bash # Snappy compression curl -X POST \ -H "Content-Encoding: snappy" \ --data-binary @metrics.snappy \ http://localhost:9091/metrics/job/my_job ``` -------------------------------- ### Go Profiling Endpoints Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/endpoints.md Examples of accessing Go profiling endpoints via pprof. ```bash curl http://localhost:9091/debug/pprof/ ``` ```bash curl http://localhost:9091/debug/pprof/profile?seconds=30 > cpu.prof ``` -------------------------------- ### Query API v1 Request Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md Example of a GET request to the v1 metrics API. ```bash GET /api/v1/metrics HTTP/1.1 ``` -------------------------------- ### Push metrics using PowerShell on Windows Source: https://github.com/prometheus/pushgateway/blob/master/README.md Example of pushing a single metric using PowerShell on MS Windows. ```powershell Invoke-WebRequest -Uri http://localhost:9091/metrics/job/MyJob -Method POST -Body "Some_Metric 3.14`n" ``` -------------------------------- ### Example: Creating a New API Instance Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-api-v1.md Demonstrates how to initialize the buildInfo map and create a new API instance using the New function. ```go buildInfo := map[string]string{ "version": "0.15.0", "revision": "a1b2c3d", "branch": "main", "buildUser": "builder", "buildDate": "2024-01-01T00:00:00Z", "goVersion": "go1.25.0", } api := api_v1.New(logger, ms, flags, buildInfo) ``` -------------------------------- ### Push Request (Protobuf Format) Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md Example of a POST request to push metrics in Protobuf format. ```bash POST /metrics/job/my_job HTTP/1.1 Content-Type: application/vnd.google.protobuf; encoding=delimited; proto=io.prometheus.client.MetricFamily Content-Length: 512 [binary protobuf data] ``` -------------------------------- ### init() function signature Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Initializes Prometheus metrics for version tracking on startup. Registers a Prometheus version collector and collects build information. ```go func init() ``` -------------------------------- ### main() function signature Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Application entry point. Parses command-line flags, initializes the metric store and HTTP server, and handles graceful shutdown. ```go func main() ``` -------------------------------- ### DiskMetricStore Constructor Parameters Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Go code snippet showing the instantiation of DiskMetricStore with its parameters. ```go ms := storage.NewDiskMetricStore( *persistenceFile, // --persistence.file value *persistenceInterval, // --persistence.interval value prometheus.DefaultGatherer, logger, ) ``` -------------------------------- ### Build Information Collection Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Gathers build information from the Prometheus version package. ```go buildInfo := map[string]string{ "version": version.Version, "revision": version.Revision, "branch": version.Branch, "buildUser": version.BuildUser, "buildDate": version.BuildDate, "goVersion": version.GoVersion, } ``` -------------------------------- ### Startup Options - With UTF-8 Support Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md Starting the Pushgateway with UTF-8 support for metric and label names. ```bash ./pushgateway --push.enable-utf8-names ``` -------------------------------- ### Disable Consistency Check Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/configuration.md Example command to disable consistency validation of pushed metrics. ```bash pushgateway --push.disable-consistency-check ``` -------------------------------- ### Ready Method Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-storage.md Reports whether the MetricStore is ready for use, indicating that all files are opened and checkpoints restored. Currently delegates to Healthy(). ```go func (dms *DiskMetricStore) Ready() error ``` ```go if err := ms.Ready(); err != nil { // Persistence is not yet loaded http.Error(w, "MetricStore not ready", http.StatusServiceUnavailable) return } ``` -------------------------------- ### BucketsAsJson Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-histogram.md Example usage of BucketsAsJson to convert APIBucket structures to a JSON-serializable format. ```go h, _ := histogram.NewModelHistogram(dtoHistogram) apiBuckets := histogram.GetAPIBuckets(h) jsonBuckets := histogram.BucketsAsJson(apiBuckets) // Result (after JSON marshal): // [ // [1, "0.1", "0.5", "25"], // [1, "0.5", "1.0", "50"], // [1, "1.0", "+Inf", "100"] // ] ``` -------------------------------- ### errorInternal Example (API v1) Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md Example JSON response for an errorInternal type in API v1. ```json { "status": "error", "errorType": "internal", "error": "internal server error" } ``` -------------------------------- ### errorBadData Example (API v1) Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md Example JSON response for an errorBadData type in API v1. ```json { "status": "error", "errorType": "bad_data", "error": "error marshaling JSON: {details}" } ``` -------------------------------- ### computeRoutePrefix function signature Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Computes the effective route prefix from command-line flags. ```go func computeRoutePrefix(prefix string, externalURL *url.URL) string ``` -------------------------------- ### New API Instance Constructor Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-api-v1.md Function to create a new API instance, requiring a logger, metric store, flags, and build information. ```go func New( l *slog.Logger, ms storage.MetricStore, flags map[string]string, buildInfo map[string]string, ) *API ``` -------------------------------- ### Example Error Response (API v1) Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md An example of a bad_data error response from API v1. ```json { "status": "error", "errorType": "bad_data", "error": "invalid base64 encoding in job name \"invalid\": illegal base64 data at input byte 0" } ``` -------------------------------- ### Static Files Route Registration Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Go code snippet for registering the static file serving endpoint. ```go r.Get(*routePrefix+"/static/*filepath", handler.Static(asset.Assets, *routePrefix).ServeHTTP) ``` -------------------------------- ### Complete Push Workflow Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/README.md A comprehensive bash script demonstrating the workflow of creating a metrics file, pushing it to the Pushgateway, querying metrics, checking push status, and optionally deleting metrics. ```bash # 1. Create metrics file cat > batch_metrics.txt << 'EOF' # HELP batch_duration_seconds Batch job duration # TYPE batch_duration_seconds histogram batch_duration_seconds_bucket{le="0.1"} 0 batch_duration_seconds_bucket{le="0.5"} 5 batch_duration_seconds_bucket{le="1.0"} 10 batch_duration_seconds_bucket{le="+Inf"} 15 batch_duration_seconds_sum 8.234 batch_duration_seconds_count 15 # HELP batch_last_success Last successful batch run # TYPE batch_last_success gauge batch_last_success 1704110400 EOF # 2. Push metrics to pushgateway JOB="batch_job" INSTANCE="scheduler_1" PG="http://localhost:9091" curl -X POST \ -H "Content-Type: text/plain" \ --data-binary @batch_metrics.txt \ "$PG/metrics/job/$JOB/instance/$INSTANCE" # Response: HTTP 202 Accepted (or 400 Bad Request if invalid) # 3. Query metrics curl "$PG/metrics" | grep batch_ # This command is illustrative and might not be fully accurate depending on grep's behavior with binary data. # 4. Check last push status curl "$PG/api/v1/metrics" | jq '.[0].last_push_successful' # 5. Delete metrics when done (optional) curl -X DELETE "$PG/metrics/job/$JOB/instance/$INSTANCE" ``` -------------------------------- ### Template Logic for Runtime Information Source: https://github.com/prometheus/pushgateway/blob/master/resources/template.html This snippet displays runtime information including the start time, build information (key-value pairs), and startup flags (key-value pairs). ```go-html-template Runtime Information ------------------- Started {{.Birth}} Build Information ----------------- {{- range $key, $value := .BuildInfo}} {{- end}} {{$key}} {{$value}} Startup Flags ------------- {{- range $key, $value := .Flags}} {{- end}} {{$key}} {{$value}} ``` -------------------------------- ### NewModelHistogram Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-histogram.md Example usage of NewModelHistogram to convert a DTO histogram to Prometheus model histograms. ```go metricFamily := getMetricFromPush() dtoHistogram := metricFamily.GetMetric()[0].GetHistogram() h, fh := histogram.NewModelHistogram(dtoHistogram) if h != nil { fmt.Printf("Classic histogram with %d buckets\n", len(h.PositiveBuckets)) } else if fh != nil { fmt.Printf("Float histogram with count: %v\n", fh.Count) } ``` -------------------------------- ### GetAPIBuckets Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-histogram.md Example usage of GetAPIBuckets to extract histogram buckets from a classic Prometheus model histogram. ```go classicHistogram := &model.Histogram{ Count: 100, Sum: 45.67, PositiveBuckets: []int64{10, 30, 5}, // Delta-encoded // ... other fields } buckets := histogram.GetAPIBuckets(classicHistogram) for _, bucket := range buckets { fmt.Printf("Bucket [%f, %f): %d\n", bucket.Lower, bucket.Upper, bucket.Count) } ``` -------------------------------- ### Handler Logging Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md Demonstrates how to use structured logging with slog.Logger for error, debug, and warning messages. ```go logger.Error(message, key, value, ...) logger.Debug(message, key, value, ...) logger.Warn(message, key, value, ...) ``` -------------------------------- ### Lifecycle Endpoints (Conditional) Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Configures the /-/quit endpoint for graceful shutdown, conditional on the --web.enable-lifecycle flag. ```go if *enableLifeCycle { r.Put(*routePrefix+"/-/quit", quitHandler) r.Post(*routePrefix+"/-/quit", quitHandler) } else { r.Put(*routePrefix+"/-/quit", forbiddenAPINotEnabled) r.Post(*routePrefix+"/-/quit", forbiddenAPINotEnabled) } ``` -------------------------------- ### API v1 Routes Registration Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Registers API v1 routes including /status, /metrics, and conditionally /admin/wipe. ```go apiv1 := api_v1.New(logger, ms, flags, buildInfo) av1 := route.New() apiv1.Register(av1) // Registers /status, /metrics, OPTIONS if *enableAdminAPI { av1.Put("/admin/wipe", handler.WipeMetricStore(ms, logger).ServeHTTP) } mux.Handle(apiPath+"/v1/", http.StripPrefix(apiPath+"/v1", av1)) ``` -------------------------------- ### Status Page Display Example Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-histogram.md Example of how histogram utilities are used in the status page template to format histogram data for human-readable display. ```Go // In handler/status.go template functions: "formatHistogram": func(m *dto.Histogram) string { h, fh := histogram.NewModelHistogram(m) if h == nil { return fh.String() // Float histogram string representation } return h.String() // Classic histogram string representation }, ``` -------------------------------- ### 500 Internal Server Error Health/Ready Endpoint Examples Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/errors.md Examples of curl requests to the /-/healthy and /-/ready endpoints when they return a 500 Internal Server Error. ```bash # Unhealthy due to full write queue curl -I http://localhost:9091/-/healthy # HTTP/1.1 500 Internal Server Error # Body: "write queue is full" # Not ready (during startup) curl -I http://localhost:9091/-/ready # HTTP/1.1 500 Internal Server Error # Body: (error message from Ready() check) ``` -------------------------------- ### HTTP Server Error Handling Source: https://github.com/prometheus/pushgateway/blob/master/_autodocs/api-reference-main.md Logs server shutdown errors, distinguishing between normal closure and unexpected errors. ```go if err == http.ErrServerClosed { logger.Info("HTTP server stopped") } else { logger.Error("HTTP server stopped", "err", err) } ``` -------------------------------- ### Pushgateway Metrics Source: https://github.com/prometheus/pushgateway/blob/master/README.md Example metrics exposed by the Pushgateway. ```prometheus # HELP pushgateway_http_push_duration_seconds HTTP request duration for pushes to the Pushgateway. # TYPE pushgateway_http_push_duration_seconds summary pushgateway_http_push_duration_seconds{method="post",quantile="0.1"} 0.000116755 pushgateway_http_push_duration_seconds{method="post",quantile="0.5"} 0.000192608 pushgateway_http_push_duration_seconds{method="post",quantile="0.9"} 0.000327593 pushgateway_http_push_duration_seconds_sum{method="post"} 0.001622878 pushgateway_http_push_duration_seconds_count{method="post"} 8 # HELP pushgateway_http_push_size_bytes HTTP request size for pushes to the Pushgateway. # TYPE pushgateway_http_push_size_bytes summary pushgateway_http_push_size_bytes{method="post",quantile="0.1"} 166 pushgateway_http_push_size_bytes{method="post",quantile="0.5"} 182 pushgateway_http_push_size_bytes{method="post",quantile="0.9"} 196 pushgateway_http_push_size_bytes_sum{method="post"} 1450 pushgateway_http_push_size_bytes_count{method="post"} 8 # HELP pushgateway_http_requests_total Total HTTP requests processed by the Pushgateway, excluding scrapes. # TYPE pushgateway_http_requests_total counter pushgateway_http_requests_total{code="200",handler="static",method="get"} 5 pushgateway_http_requests_total{code="200",handler="status",method="get"} 8 pushgateway_http_requests_total{code="202",handler="delete",method="delete"} 1 pushgateway_http_requests_total{code="202",handler="push",method="post"} 6 pushgateway_http_requests_total{code="400",handler="push",method="post"} 2 ```