### Example GitHub Actions Workflow for Production Prober

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This YAML defines a GitHub Actions workflow that runs the Sigstore prober on a schedule or manually. It checks out the code, sets up Go, runs the prober with production logging, and includes a step to alert PagerDuty on failure. The workflow uses secrets for sensitive information like the PagerDuty integration key.

```yaml
name: Production Proberon:
  schedule:
    - cron: '*/5 * * * *'  # Every 5 minutes
  workflow_dispatch:

jobs:
  prober:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-go@v5
        with:
          go-version: '1.21'

      - name: Run prober
        run: |
          go run ./prober/prober \
            --one-time \
            --logStyle prod

      - name: Alert on failure
        if: failure()
        env:
          PAGERDUTY_INTEGRATION_KEY: ${{ secrets.PAGERDUTY_KEY }}
        run: |
          echo '{"summary":"Prober failed","source":"github-actions","severity":"error","component":"prober","group":"monitoring"}' | \
            go run ./pager-duty
```

--------------------------------

### Example Git Commit Message

Source: https://github.com/sigstore/sigstore-probers/blob/main/CONTRIBUTORS.md

Demonstrates a well-formed Git commit message, including a concise subject line, detailed body explaining the 'why' behind the change, and issue tracking references. This format aids reviewers and automated release note generation.

```git
Summarize changes in around 50 characters or less

More detailed explanatory text, if necessary. Wrap it to about 72
characters or so. In some contexts, the first line is treated as the
subject of the commit and the rest of the text as the body. The
blank line separating the summary from the body is critical (unless
you omit the body entirely); various tools like `log`, `shortlog`
and `rebase` can get confused if you run the two together.

Explain the problem that this commit is solving. Focus on why you
are making this change as opposed to how (the code explains that).
Are there side effects or other unintuitive consequences of this
change? Here's the place to explain them.

Further paragraphs come after blank lines.

 - Bullet points are okay, too

 - Typically a hyphen or asterisk is used for the bullet, preceded
   by a single space, with blank lines in between, but conventions
   vary here

If you use an issue tracker, put references to them at the bottom,
like this:

Resolves: #123
See also: #456, #789

```

--------------------------------

### Fetch Prometheus Metrics from Running Prober

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This command fetches the Prometheus metrics exposed by a running Sigstore prober instance. It uses `curl` to make an HTTP GET request to the `/metrics` endpoint. The output contains time-series data for monitoring various aspects of the prober and the services it monitors.

```bash
curl http://localhost:8080/metrics
```

--------------------------------

### Test Rekor v1 API Endpoints (Bash)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This section outlines common Rekor v1 API endpoints tested by the prober, including retrieving log information, public keys, entries by index or UUID, searching entries by artifact hash, and getting consistency proofs. These tests ensure the integrity and availability of the Rekor transparency log.

```bash
# Endpoints automatically tested by the prober:

# GET /api/v1/log - Get log info (tree size, inactive shards)
curl -H "Accept: application/json" https://rekor.sigstore.dev/api/v1/log
# Response: {"treeSize":12345678,"inactiveShards":[{"treeID":"...","treeSize":1000000}]}

# GET /api/v1/log/publicKey - Get Rekor's public key
curl -H "Accept: application/x-pem-file" https://rekor.sigstore.dev/api/v1/log/publicKey
# Response: -----BEGIN PUBLIC KEY-----...-----END PUBLIC KEY-----

# GET /api/v1/log/entries?logIndex=<index> - Retrieve entry by log index
curl -H "Accept: application/json" "https://rekor.sigstore.dev/api/v1/log/entries?logIndex=100"

# POST /api/v1/log/entries/retrieve - Retrieve entries by UUID
curl -X POST -H "Content-Type: application/json" -H "Accept: application/json" \
  -d '{"entryUUIDs":["362f8ecba72f432663d86f73f83a1a1032637c79679113b224e91d757c835cd08379739f9097cf8a"]}' \
  https://rekor.sigstore.dev/api/v1/log/entries/retrieve

# POST /api/v1/index/retrieve - Search entries by artifact hash
curl -X POST -H "Content-Type: application/json" -H "Accept: application/json" \
  -d '{"hash":"sha256:2bd37672a9e472c79c64f42b95e362db16870e28a90f3b17fee8faf952e79b4b"}' \
  https://rekor.sigstore.dev/api/v1/index/retrieve

# GET /api/v1/log/proof - Get consistency proof
curl -H "Accept: application/json" "https://rekor.sigstore.dev/api/v1/log/proof?firstSize=1&lastSize=12345678"
```

--------------------------------

### Request Signing Certificate (POST /api/v2/signingCert)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This prober automatically generates an ECDSA P-256 key, obtains an OIDC token, and requests a signing certificate from the Fulcio API. It requires a public key and a proof of possession in the request body.

```go
# The prober automatically generates an ECDSA P-256 key, obtains OIDC token, and requests certificate
# Request body structure:
# {
#   "publicKeyRequest": {
#     "publicKey": {"content": "-----BEGIN PUBLIC KEY-----..."},
#     "proofOfPossession": "<base64-encoded-signature>"
#   }
# }
```

--------------------------------

### Run Sigstore Prober for Service Health Monitoring (Bash)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This command runs the main Sigstore prober to test Rekor, Fulcio, and TSA endpoints. It collects latency metrics and verifies service functionality, supporting both read-only and write probes. Options include specifying staging/production environments, write probes, custom signing configurations, and one-time checks.

```bash
# Run the prober against production Sigstore (read-only mode)
go run ./prober/prober \
  --addr ":8080" \
  --frequency 10 \
  --logStyle prod

# Run against staging environment
go run ./prober/prober \
  --staging \
  --addr ":8080" \
  --frequency 30

# Run with write probers enabled (requires OIDC authentication)
# This will create actual entries in Rekor, Fulcio certificates, and TSA timestamps
go run ./prober/prober \
  --staging \
  --write-prober \
  --frequency 60

# Run against a custom deployment with signing config and trusted root
go run ./prober/prober \
  --signing-config /path/to/signing-config.json \
  --trusted-root /path/to/trusted-root.json \
  --write-prober

# Run one-time check and exit (useful for CI/CD)
go run ./prober/prober \
  --staging \
  --one-time

# Run with custom Rekor v2 URL override
go run ./prober/prober \
  --rekor-v2-url "https://log2025-1.rekor.sigstore.dev" \
  --write-prober

# Add additional custom endpoint checks via JSON flags
go run ./prober/prober \
  --rekor-requests '[{"endpoint":"/api/v1/log/entries","method":"GET","queries":{"logIndex":"100"}}]' \
  --fulcio-requests '[{"endpoint":"/api/v2/configuration","method":"GET"}]'

# Expected Prometheus metrics exposed at :8080/metrics
# api_endpoint_latency{endpoint="/api/v1/log",host="https://rekor.sigstore.dev",status_code="200",method="GET"}
# api_endpoint_latency_histogram_bucket{...}
# verification{verified="true"}
# sigstore_prober_build_info{version="...",revision="...",build_date="...",goversion="..."}
```

--------------------------------

### Customize Request Limit for Rate Limiting Prober

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This command demonstrates how to run the rate-limiting prober with a custom request limit. It utilizes the Go toolchain to execute the prober script. The `--limit` flag controls the maximum number of requests allowed within a certain timeframe.

```bash
go run ./prober/rate-limiting.go --limit 500
```

--------------------------------

### Fulcio Signing Certificate API

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Requests a signing certificate from Fulcio by providing a public key and proof of possession. The prober automatically generates keys, obtains an OIDC token, and requests the certificate.

```APIDOC
## POST /api/v2/signingCert

### Description
Requests a signing certificate from Fulcio. This endpoint requires an OIDC token and is used by the prober to automatically generate keys, obtain an OIDC token, and request a certificate.

### Method
POST

### Endpoint
/api/v2/signingCert

### Parameters
#### Request Body
- **publicKeyRequest** (object) - Required - Contains the public key and proof of possession.
  - **publicKey** (object) - Required - The public key details.
    - **content** (string) - Required - The public key in PEM format (e.g., "-----BEGIN PUBLIC KEY-----...").
  - **proofOfPossession** (string) - Required - A base64-encoded signature of the public key.

### Request Example
```json
{
  "publicKeyRequest": {
    "publicKey": {
      "content": "-----BEGIN PUBLIC KEY-----..."
    },
    "proofOfPossession": "<base64-encoded-signature>"
  }
}
```

### Response
#### Success Response (200)
- **certificate** (string) - The DER-encoded signing certificate.
```

--------------------------------

### Test Fulcio gRPC API (GetTrustBundle)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This prober tests Fulcio's gRPC API, specifically the GetTrustBundle endpoint, which is typically available on port 443 or a custom --grpc-port.

```go
# The prober also tests Fulcio's gRPC API at port 443 (or custom --grpc-port)
# gRPC endpoint testing: GetTrustBundle
```

--------------------------------

### Rekor v2 Tiles Consistency Check (tiles-fsck)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

The tiles-fsck tool performs Merkle tree consistency checks on Rekor v2 tile-based logs using the Tessera library. It downloads TUF materials, identifies log URLs, retrieves public keys, and runs parallel consistency checks, verifying checkpoint signatures.

```bash
# Run consistency check against production Rekor v2 logs
go run ./prober/tiles-fsck

# Run against staging environment
go run ./prober/tiles-fsck --staging

# The tool:
# 1. Downloads TUF materials (trusted_root.json, signing_config.json)
# 2. Identifies all Rekor v2 log URLs from signing config
# 3. Retrieves public keys for each log from trusted root
# 4. Runs Merkle tree consistency checks in parallel
# 5. Verifies checkpoint signatures using note format

# Expected output on success:
# INFO Running consistency check for log=log2025-1.rekor.sigstore.dev
# INFO Verified consistency for log=log2025-1.rekor.sigstore.dev
# INFO Finished!
```

--------------------------------

### Rekor v2 API - Create Hashed Rekord Entry

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Creates a hashed rekord entry in the Rekor v2 transparency log. This involves providing a signature, verifier details, and the digest of the data.

```APIDOC
## POST /api/v2/log/entries

### Description
Creates a hashed rekord entry in the Rekor v2 transparency log. This endpoint is used to add records to the log.

### Method
POST

### Endpoint
/api/v2/log/entries

### Parameters
#### Request Body
- **hashedRekordRequestV002** (object) - Required - The request object for creating a hashed rekord entry.
  - **signature** (object) - Required - The signature of the data.
    - **content** (string) - Required - The base64-encoded signature.
    - **verifier** (object) - Required - Details about the verifier.
      - **keyDetails** (string) - Required - The type of key used (e.g., "PKIX_ECDSA_P256_SHA_256").
      - **x509Certificate** (object) - Required - The X.509 certificate.
        - **rawBytes** (string) - Required - The DER-encoded certificate.
  - **digest** (string) - Required - The SHA256 hash of the data.

### Request Example
```json
{
  "hashedRekordRequestV002": {
    "signature": {
      "content": "<base64-signature>",
      "verifier": {
        "keyDetails": "PKIX_ECDSA_P256_SHA_256",
        "x509Certificate": {"rawBytes": "<der-encoded-cert>"}
      }
    },
    "digest": "<sha256-hash>"
  }
}
```
```

--------------------------------

### Test Fulcio Certificate Authority API Endpoints (Bash)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This section details the Fulcio API endpoints tested by the prober, including v1 and v2 certificate issuance APIs and gRPC endpoints. It covers retrieving the root certificate chain, configuration details, and the trust bundle, ensuring the Fulcio CA is functioning correctly.

```bash
# Endpoints automatically tested by the prober:

# GET /api/v1/rootCert - Get Fulcio root certificate chain
curl -H "Accept: application/pem-certificate-chain" https://fulcio.sigstore.dev/api/v1/rootCert
# Response: -----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----

# GET /api/v2/configuration - Get Fulcio configuration
curl -H "Accept: application/json" https://fulcio.sigstore.dev/api/v2/configuration

# GET /api/v2/trustBundle - Get trust bundle
curl -H "Accept: application/json" https://fulcio.sigstore.dev/api/v2/trustBundle
```

--------------------------------

### Create HashedRekord Entry (Rekor v2 API)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This prober tests the Rekor v2 API by creating a hashedRekord entry. It requires a base64-encoded signature, the public key details (type and certificate), and the SHA256 hash of the data to be logged.

```bash
# Write prober: POST /api/v2/log/entries - Create hashedrekord entry
# Request body (protobuf JSON):
# {
#   "hashedRekordRequestV002": {
#     "signature": {
#       "content": "<base64-signature>",
#       "verifier": {
#         "keyDetails": "PKIX_ECDSA_P256_SHA_256",
#         "x509Certificate": {"rawBytes": "<der-encoded-cert>"}
#       }
#     },
#     "digest": "<sha256-hash>"
#   }
# }
```

--------------------------------

### Rekor v2 Tiles Consistency Checker

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Performs Merkle tree consistency checks on Rekor v2 tile-based logs using the Tessera library. It verifies the integrity of the transparency log.

```APIDOC
## Rekor v2 Tiles Consistency Checker Tool

### Description
Performs Merkle tree consistency checks on Rekor v2 tile-based logs using the Tessera library. This tool ensures the integrity and consistency of the transparency log data.

### Usage
```bash
go run ./prober/tiles-fsck [--staging]
```

### Parameters
- **--staging** (flag) - Optional - Use the staging environment instead of production.

### Process
1. Downloads TUF materials (trusted_root.json, signing_config.json).
2. Identifies all Rekor v2 log URLs from the signing configuration.
3. Retrieves public keys for each log from the trusted root.
4. Runs Merkle tree consistency checks in parallel.
5. Verifies checkpoint signatures using the note format.

### Example Usage
```bash
# Run consistency check against production Rekor v2 logs
go run ./prober/tiles-fsck

# Run against staging environment
go run ./prober/tiles-fsck --staging
```

### Expected Output on Success
```
INFO Running consistency check for log=log2025-1.rekor.sigstore.dev
INFO Verified consistency for log=log2025-1.rekor.sigstore.dev
INFO Finished!
```
```

--------------------------------

### PagerDuty Alert Integration

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Sends alerts to PagerDuty when probers detect failures. Accepts JSON input via stdin to create PagerDuty events.

```APIDOC
## PagerDuty Alert Tool

### Description
Sends alerts to PagerDuty based on JSON input provided via standard input. This tool is used to notify teams of detected failures.

### Usage
```bash
# Set the PagerDuty integration key
export PAGERDUTY_INTEGRATION_KEY="your-integration-key"

# Send an alert via stdin
echo '{...}' | go run ./pager-duty
```

### Parameters (JSON Input)
- **summary** (string) - Required - A short summary of the alert.
- **source** (string) - Required - The source of the event (e.g., "sigstore-prober-prod").
- **severity** (string) - Required - The severity level of the alert (critical, error, warning, info).
- **component** (string) - Required - The component generating the event (e.g., "rekor").
- **group** (string) - Required - The group the event belongs to (e.g., "transparency-log").
- **dedup_key** (string) - Optional - A key to deduplicate events.
- **details** (object) - Optional - Additional details about the event.
- **links** (array) - Optional - Links to relevant dashboards or status pages.
  - **href** (string) - The URL of the link.
  - **text** (string) - The display text for the link.

### Example JSON Input
```json
{
  "summary": "Rekor endpoint latency exceeded threshold",
  "source": "sigstore-prober-prod",
  "severity": "error",
  "component": "rekor",
  "group": "transparency-log",
  "dedup_key": "rekor-latency-alert-2024",
  "details": {
    "endpoint": "/api/v1/log",
    "latency_ms": 5000,
    "threshold_ms": 2000
  },
  "links": [
    {"href": "https://grafana.sigstore.dev/dashboard/rekor", "text": "Grafana Dashboard"},
    {"href": "https://status.sigstore.dev", "text": "Status Page"}
  ]
}
```

### Command-line Options
- **--retry-count** (integer) - Custom retry count for sending the alert.
```

--------------------------------

### Fulcio Certificate Authority API Endpoints

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

The prober tests Fulcio's certificate issuance endpoints for both v1 (legacy) and v2 APIs, including gRPC endpoints.

```APIDOC
## Tested Endpoints - Fulcio Certificate Authority API

### Description
The prober tests Fulcio's certificate issuance endpoints for both v1 (legacy) and v2 APIs, including gRPC endpoints.

### Method
GET

### Endpoints
- `/api/v1/rootCert`
- `/api/v2/configuration`
- `/api/v2/trustBundle`

### Parameters
None for these specific endpoints.

### Request Example
```bash
# GET /api/v1/rootCert - Get Fulcio root certificate chain
curl -H "Accept: application/pem-certificate-chain" https://fulcio.sigstore.dev/api/v1/rootCert

# GET /api/v2/configuration - Get Fulcio configuration
curl -H "Accept: application/json" https://fulcio.sigstore.dev/api/v2/configuration

# GET /api/v2/trustBundle - Get trust bundle
curl -H "Accept: application/json" https://fulcio.sigstore.dev/api/v2/trustBundle
```

### Response
#### Success Response (200)
Responses vary based on the endpoint:
- `/api/v1/rootCert`: PEM-encoded certificate chain.
- `/api/v2/configuration`: JSON object with Fulcio configuration details.
- `/api/v2/trustBundle`: JSON object containing trust bundle information.

#### Response Example
```json
{
  "some_config_key": "some_value"
}
```
```

--------------------------------

### Rekor v1 API Endpoints

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

The prober tests Rekor's transparency log API endpoints including log info, public key retrieval, entry queries, and proof verification.

```APIDOC
## Tested Endpoints - Rekor v1 API

### Description
The prober tests Rekor's transparency log API endpoints including log info, public key retrieval, entry queries, and proof verification.

### Method
GET, POST

### Endpoints
- `/api/v1/log`
- `/api/v1/log/publicKey`
- `/api/v1/log/entries`
- `/api/v1/log/entries/retrieve`
- `/api/v1/index/retrieve`
- `/api/v1/log/proof`

### Parameters
#### Query Parameters
- `logIndex` (integer) - Optional - Used with `/api/v1/log/entries` to retrieve an entry by its index.
- `firstSize` (integer) - Optional - Used with `/api/v1/log/proof` for consistency proof.
- `lastSize` (integer) - Optional - Used with `/api/v1/log/proof` for consistency proof.

#### Request Body
- `entryUUIDs` (array of strings) - Required for `/api/v1/log/entries/retrieve` - List of entry UUIDs to retrieve.
- `hash` (string) - Required for `/api/v1/index/retrieve` - Artifact hash to search for (e.g., "sha256:...").

### Request Example
```bash
# GET /api/v1/log - Get log info (tree size, inactive shards)
curl -H "Accept: application/json" https://rekor.sigstore.dev/api/v1/log

# GET /api/v1/log/publicKey - Get Rekor's public key
curl -H "Accept: application/x-pem-file" https://rekor.sigstore.dev/api/v1/log/publicKey

# GET /api/v1/log/entries?logIndex=<index> - Retrieve entry by log index
curl -H "Accept: application/json" "https://rekor.sigstore.dev/api/v1/log/entries?logIndex=100"

# POST /api/v1/log/entries/retrieve - Retrieve entries by UUID
curl -X POST -H "Content-Type: application/json" -H "Accept: application/json" \
  -d '{"entryUUIDs":["362f8ecba72f432663d86f73f83a1a1032637c79679113b224e91d757c835cd08379739f9097cf8a"]}' \
  https://rekor.sigstore.dev/api/v1/log/entries/retrieve

# POST /api/v1/index/retrieve - Search entries by artifact hash
curl -X POST -H "Content-Type: application/json" -H "Accept: application/json" \
  -d '{"hash":"sha256:2bd37672a9e472c79c64f42b95e362db16870e28a90f3b17fee8faf952e79b4b"}' \
  https://rekor.sigstore.dev/api/v1/index/retrieve

# GET /api/v1/log/proof - Get consistency proof
curl -H "Accept: application/json" "https://rekor.sigstore.dev/api/v1/log/proof?firstSize=1&lastSize=12345678"
```

### Response
#### Success Response (200)
Responses vary based on the endpoint. Examples:
- `/api/v1/log`: `{"treeSize":12345678,"inactiveShards":[...]} `
- `/api/v1/log/publicKey`: PEM-encoded public key.

#### Response Example
```json
{
  "treeSize": 12345678,
  "inactiveShards": [
    {
      "treeID": "...",
      "treeSize": 1000000
    }
  ]
}
```
```

--------------------------------

### Test Rate Limiting

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

The rate-limiting tool tests Sigstore services' rate limiting implementation by sending concurrent requests until HTTP 429 responses are received. It can target production or staging environments and use a custom signing configuration.

```bash
# Test rate limiting against production services
go run ./prober/rate-limiting.go

# Test against staging
go run ./prober/rate-limiting.go --staging

# Use custom signing config
go run ./prober/rate-limiting.go --signing-config /path/to/config.json
```

--------------------------------

### Verify CT Log STH (ctlog)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

The ctlog-sth tool verifies Certificate Transparency log Signed Tree Heads (STH) for Sigstore's CTFE instances. It supports different environments (production, staging) and shards, with an option to specify a custom retry count.

```bash
# Verify production CT log STH for the 2022 shard
go run ./prober/ctlog \
  --env production \
  --shard 2022

# Verify staging CT log STH for test shard
go run ./prober/ctlog \
  --env staging \
  --shard test

# Available environments and shards:
# production:
#   - test (https://ctfe.sigstore.dev/test)
#   - 2022 (https://ctfe.sigstore.dev/2022)
# staging:
#   - test (https://ctfe.sigstage.dev/test)
#   - 2022 (https://ctfe.sigstage.dev/2022)
#   - 2022-2 (https://ctfe.sigstage.dev/2022-2)

# Custom retry count
go run ./prober/ctlog \
  --env production \
  --shard 2022 \
  --retry-count 10

# Expected output on success:
# received STH: {TreeSize:... Timestamp:... SHA256RootHash:... TreeHeadSignature:...}
# STH verified
```

--------------------------------

### Main Prober - Sigstore Service Health Monitoring

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

The main prober continuously tests Rekor, Fulcio, and TSA endpoints, collecting latency metrics and verifying service functionality. It supports both read-only probes and write probes that create actual entries in the transparency logs.

```APIDOC
## Main Prober - Sigstore Service Health Monitoring

### Description
The main prober continuously tests Rekor, Fulcio, and TSA endpoints, collecting latency metrics and verifying service functionality. It supports both read-only probes and write probes that create actual entries in the transparency logs.

### Method
`go run`

### Endpoint
`./prober/prober`

### Parameters
#### Command-line Flags
- `--addr` (string) - Optional - The address to expose metrics on (default: ":8080").
- `--frequency` (int) - Optional - The interval in seconds between probes (default: 10).
- `--logStyle` (string) - Optional - The logging style to use (e.g., "prod").
- `--staging` - Optional - Use staging environment endpoints.
- `--write-prober` - Optional - Enable write probes (requires OIDC authentication).
- `--signing-config` (string) - Optional - Path to signing configuration JSON file.
- `--trusted-root` (string) - Optional - Path to trusted root JSON file.
- `--one-time` - Optional - Run probes once and exit.
- `--rekor-v2-url` (string) - Optional - Override for Rekor v2 URL.
- `--rekor-requests` (string) - Optional - JSON string for custom Rekor endpoint checks.
- `--fulcio-requests` (string) - Optional - JSON string for custom Fulcio endpoint checks.

### Request Example
```bash
# Run the prober against production Sigstore (read-only mode)
go run ./prober/prober \
  --addr ":8080" \
  --frequency 10 \
  --logStyle prod

# Run against staging environment
go run ./prober/prober \
  --staging \
  --addr ":8080" \
  --frequency 30

# Run with write probers enabled (requires OIDC authentication)
# This will create actual entries in Rekor, Fulcio certificates, and TSA timestamps
go run ./prober/prober \
  --staging \
  --write-prober \
  --frequency 60
```

### Response
Prometheus metrics are exposed at the specified `--addr` (default ":8080") on the `/metrics` endpoint.

#### Expected Prometheus Metrics
- `api_endpoint_latency{endpoint,host,status_code,method}`
- `api_endpoint_latency_histogram_bucket{...}`
- `verification{verified}`
- `sigstore_prober_build_info{version,revision,build_date,goversion}`
```

--------------------------------

### Send PagerDuty Alert

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

The pager-duty tool sends alerts to PagerDuty when probers detect failures. It accepts JSON input via stdin, requiring fields like summary, source, severity, component, and group. Optional fields include dedup_key, details, and links.

```bash
# Set the PagerDuty integration key
export PAGERDUTY_INTEGRATION_KEY="your-integration-key"

# Send an alert via stdin
echo '{
  "summary": "Rekor endpoint latency exceeded threshold",
  "source": "sigstore-prober-prod",
  "severity": "error",
  "component": "rekor",
  "group": "transparency-log",
  "dedup_key": "rekor-latency-alert-2024",
  "details": {
    "endpoint": "/api/v1/log",
    "latency_ms": 5000,
    "threshold_ms": 2000
  },
  "links": [{"href": "https://grafana.sigstore.dev/dashboard/rekor", "text": "Grafana Dashboard"},
    {"href": "https://status.sigstore.sigstore.dev", "text": "Status Page"}
  ]
}' | go run ./pager-duty

# Run with custom retry count
echo '{"summary":"Test alert","source":"test","severity":"info","component":"test","group":"test"}' | \
  go run ./pager-duty --retry-count 3

# Severity levels: critical, error, warning, info
# All fields except dedup_key, details, and links are required
```

--------------------------------

### Squashing Commits with Git Rebase

Source: https://github.com/sigstore/sigstore-probers/blob/main/CONTRIBUTORS.md

Illustrates the process of squashing multiple Git commits into a single commit using interactive rebase. This is useful for cleaning up commit history before merging a pull request. It involves marking commits to be squashed and then amending the commit message.

```git
git rebase -i HEAD~3

```

```git
pick eb36612 initial commit
squash 9ac8968 second commit
squash a760569 final commit

```

```git
git commit --amend

```

```git
git push origin your-branch --force

```

--------------------------------

### Rate Limiting Tester

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Tests Sigstore services for proper rate limiting implementation by sending concurrent requests until HTTP 429 responses are received.

```APIDOC
## Rate Limiting Tester Tool

### Description
Tests Sigstore services to ensure they correctly implement rate limiting. The tool sends concurrent requests to the services until it receives HTTP 429 responses, indicating that rate limits are in effect.

### Usage
```bash
go run ./prober/rate-limiting.go [--staging] [--signing-config <path>]
```

### Parameters
- **--staging** (flag) - Optional - Test against the staging environment instead of production.
- **--signing-config** (string) - Optional - Path to a custom signing configuration file.

### Example Usage
```bash
# Test rate limiting against production services
go run ./prober/rate-limiting.go

# Test against staging
go run ./prober/rate-limiting.go --staging

# Use custom signing config
go run ./prober/rate-limiting.go --signing-config /path/to/config.json
```
```

--------------------------------

### CT Log STH Verifier

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Verifies Certificate Transparency log Signed Tree Heads (STH) for Sigstore's CTFE instances. It checks STHs for specified environments and shards.

```APIDOC
## CT Log STH Verifier Tool

### Description
Verifies the Signed Tree Heads (STH) of Certificate Transparency (CT) logs for Sigstore's CTFE instances. This tool ensures the integrity of the CT logs.

### Usage
```bash
go run ./prober/ctlog \
  --env <environment> \
  --shard <shard>
```

### Parameters
- **--env** (string) - Required - The environment to check (e.g., "production", "staging").
- **--shard** (string) - Required - The shard of the CT log to check (e.g., "test", "2022", "2022-2").
- **--retry-count** (integer) - Optional - Custom retry count for the verification process.

### Available Environments and Shards
- **production**:
  - test (https://ctfe.sigstore.dev/test)
  - 2022 (https://ctfe.sigstore.dev/2022)
- **staging**:
  - test (https://ctfe.sigstage.dev/test)
  - 2022 (https://ctfe.sigstage.dev/2022)
  - 2022-2 (https://ctfe.sigstage.dev/2022-2)

### Example Usage
```bash
# Verify production CT log STH for the 2022 shard
go run ./prober/ctlog \
  --env production \
  --shard 2022

# Verify staging CT log STH for test shard with custom retry count
go run ./prober/ctlog \
  --env staging \
  --shard test \
  --retry-count 10
```

### Expected Output on Success
```
received STH: {TreeSize:... Timestamp:... SHA256RootHash:... TreeHeadSignature:...}
STH verified
```
```

--------------------------------

### Request RFC 3161 Timestamp (TSA)

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

This prober tests the Timestamp Authority (TSA) endpoints by sending RFC 3161 timestamp requests and verifying the received timestamp responses. The TSA URL is dynamically obtained from the signing configuration.

```bash
# POST <tsa-url> - Request timestamp
# Request: RFC 3161 timestamp request (application/timestamp-query)
# Response: RFC 3161 timestamp response (application/timestamp-reply)

# Example with openssl (for reference):
# Create timestamp request
echo "test data" | openssl ts -query -sha256 -out request.tsq

# The prober sends timestamp requests and verifies responses against trusted root
```

--------------------------------

### Rekor v2 API - Health Check

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Tests the Rekor v2 health check endpoint to ensure the service is operational.

```APIDOC
## GET /healthz

### Description
Checks the health status of the Rekor v2 service.

### Method
GET

### Endpoint
/healthz

### Request Example
```bash
curl https://log2025-1.rekor.sigstore.dev/healthz
```
```

--------------------------------

### TSA Timestamp Authority API

Source: https://context7.com/sigstore/sigstore-probers/llms.txt

Tests the Timestamp Authority (TSA) endpoints for requesting RFC 3161 timestamps. The prober sends timestamp requests and verifies responses against trusted roots.

```APIDOC
## POST <tsa-url>

### Description
Requests an RFC 3161 timestamp from the Timestamp Authority. The prober sends timestamp requests and verifies the responses against trusted roots.

### Method
POST

### Endpoint
<tsa-url> (URL comes from signing config)

### Parameters
#### Request Body
- **timestamp request** (application/timestamp-query) - An RFC 3161 timestamp request.

### Response
#### Success Response (200)
- **timestamp response** (application/timestamp-reply) - An RFC 3161 timestamp response.

### Request Example (using openssl for reference)
```bash
# Create timestamp request
echo "test data" | openssl ts -query -sha256 -out request.tsq
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.