Prometheus Operator

https://prometheus-operator.dev/docs/
Admin
Prometheus Operator provides Kubernetes native deployment and management of Prometheus and related...
Tokens:285,408
Snippets:2,151
Trust Score:9.4
Update:1 month ago
Show doc for...
Tokens:
Raw
### Deploy Example Application and ServiceMonitor

Source: https://prometheus-operator.dev/docs/platform/sharding

Deploys an example application, its service, and a ServiceMonitor to enable monitoring.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: example-app
  template:
    metadata:
      labels:
        app: example-app
    spec:
      containers:
      - name: example-app
        image: quay.io/brancz/prometheus-example-app:v0.5.0
        ports:
        - name: web
          containerPort: 8080
```

```yaml
kind: Service
apiVersion: v1
metadata:
  name: example-app
  namespace: default
  labels:
    app: example-app
spec:
  selector:
    app: example-app
  ports:
  - name: web
    port: 8080
```

```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: example-app
  namespace: default
  labels:
    team: frontend
spec:
  selector:
    matchLabels:
      app: example-app
  endpoints:
  - port: web
```

--------------------------------

### Deploy example application Deployment

Source: https://prometheus-operator.dev/docs/developer/getting-started

Deploys a simple example application with 3 replicas that listens and exposes metrics on port 8080. The image is quay.io/brancz/prometheus-example-app:v0.5.0.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: example-app
  template:
    metadata:
      labels:
        app: example-app
    spec:
      containers:
      - name: example-app
        image: quay.io/brancz/prometheus-example-app:v0.5.0
        ports:
        - name: web
          containerPort: 8080

```

--------------------------------

### Deploy Prometheus Instance

Source: https://prometheus-operator.dev/docs/platform/platform-guide

Example manifest for a basic Prometheus instance.

```yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
spec:
  serviceAccountName: prometheus
```

--------------------------------

### staticConfig resource example

Source: https://prometheus-operator.dev/docs/proposals/implemented/scrape-config

Defines a static configuration with a target and labels.

```yaml
targets:
  - target:9100
labels:
  labelA: placeholder

```

--------------------------------

### Shard distribution example with num_shards=2 and zones=[A,B,C]

Source: https://prometheus-operator.dev/docs/proposals/implemented/zone-aware-sharding

Example distribution showing shard indices, zones, and assignment indices when num_shards is 2 and len(zones) is 3. This case leads to targets in zone C not being scraped.

```plaintext
0 1   | shard index
A B C | zone
0 0 0 | assignment index

```

--------------------------------

### Deploy Alertmanager Instance

Source: https://prometheus-operator.dev/docs/platform/platform-guide

Example manifest to create 3 replicas of Alertmanager.

```yaml
apiVersion: monitoring.coreos.com/v1
kind: Alertmanager
metadata:
  name: example
spec:
  replicas: 3
```

--------------------------------

### httpSDConfig resource example

Source: https://prometheus-operator.dev/docs/proposals/implemented/scrape-config

Defines an HTTP-based service discovery configuration with a URL and refresh interval.

```yaml
url: http://localhost:1234
refreshInterval: 60s

```

--------------------------------

### Shard distribution example with num_shards=10 and zones=[A,B,C]

Source: https://prometheus-operator.dev/docs/proposals/implemented/zone-aware-sharding

Example distribution showing shard indices, zones, and assignment indices when num_shards is 10 and len(zones) is 3. This case triggers the second assert warning about double scraping in zone A.

```plaintext
0 1 2 | 3 4 5 | 6 7 8 | 9 | shard index
A B C | A B C | A B C | A | zone
0 0 0 | 1 1 1 | 2 2 2 | 0 | assignment index

```

--------------------------------

### Example Service definition

Source: https://prometheus-operator.dev/docs/platform/troubleshooting

A standard Kubernetes Service definition with a named port 'web' set to 8080.

```yaml
kind: Service
apiVersion: v1
metadata:
  name: example-app
  labels:
    app: example-app
spec:
  selector:
    app: example-app
  ports:
  - name: web
    port: 8080
```

--------------------------------

### Global scrape TLS configuration example

Source: https://prometheus-operator.dev/docs/proposals/implemented/scrapeclasses

A hypothetical configuration using spec.scrapeTlsConfig to apply a default TLS configuration to all monitors.

```yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
spec:
  scrapeTlsConfig:
    caFile: "/etc/istio-certs/root-cert.pem"
    certFile: "/etc/istio-certs/cert-chain.pem"
    keyFile: "/etc/istio-certs/key.pem"
    insecureSkipVerify: true
```

--------------------------------

### Clone kube-prometheus repository

Source: https://prometheus-operator.dev/docs/getting-started/installation

Clones the kube-prometheus repository from GitHub, which is the easiest way to get started with the Prometheus Operator.

```bash
git clone https://github.com/prometheus-operator/kube-prometheus.git

```

--------------------------------

### Initialize podman machine for Kind on macOS

Source: https://prometheus-operator.dev/docs/community/testing

Recommended setup for running Kind on macOS with Podman. Use 4 CPUs and 8 GiB memory to avoid resource-related test failures.

```bash
podman machine init --cpus=4 --memory=8192 --rootful --now

```

--------------------------------

### Install in custom namespace using Kustomize

Source: https://prometheus-operator.dev/docs/getting-started/installation

Sets a custom NAMESPACE, uses Kustomize to generate a custom bundle, and applies it. Requires Kustomize and works for any namespace other than default.

```bash
NAMESPACE=my_namespace
TMPDIR=$(mktemp -d)
LATEST=$(curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name)
curl -s "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/refs/tags/$LATEST/kustomization.yaml" > "$TMPDIR/kustomization.yaml"
curl -s "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/refs/tags/$LATEST/bundle.yaml" > "$TMPDIR/bundle.yaml"
(cd $TMPDIR && kustomize edit set namespace $NAMESPACE) && kubectl create -k "$TMPDIR"

```

--------------------------------

### Create Service for example-app

Source: https://prometheus-operator.dev/docs/developer/getting-started

Exposes the example application with a Service object that selects all Pods with the label app: example-app and specifies the port where metrics are exposed (port 8080).

```yaml
kind: Service
apiVersion: v1
metadata:
  name: example-app
  labels:
    app: example-app
spec:
  selector:
    app: example-app
  ports:
  - name: web
    port: 8080

```

--------------------------------

### Prometheus remoteWriteLimits example

Source: https://prometheus-operator.dev/docs/proposals/accepted/remote-write

Prometheus CRD example setting administrative limits on remote write options, such as disabling metadata sending and limiting queue capacity and shards.

```yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: example
  namespace: default
spec:
  remoteWriteLimits:
    metadataConfig:
      send: false
    queueConfig:
      capacity: 1024
      maxShards: 16

```

--------------------------------

### Install Operator and CRDs in default namespace

Source: https://prometheus-operator.dev/docs/getting-started/installation

Downloads the latest release, retrieves the bundle.yaml, and applies it using kubectl create -f -. This installs the CRDs and the operator into the default namespace.

```bash
LATEST=$(curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name)
curl -sL https://github.com/prometheus-operator/prometheus-operator/releases/download/${LATEST}/bundle.yaml | kubectl create -f -

```

--------------------------------

### Apply kube-prometheus manifests in one command

Source: https://prometheus-operator.dev/docs/getting-started/installation

Alternative to applying setup and manifests separately; creates both in one kubectl create command. May need to be run multiple times for all components to be created successfully.

```bash
kubectl create -f manifests/setup -f manifests

```

--------------------------------

### fileSDConfig resource example

Source: https://prometheus-operator.dev/docs/proposals/implemented/scrape-config

Defines a file-based service discovery configuration, referencing files in the Prometheus pod and an optional refresh interval.

```yaml
files:
  # Files here are string referencing a file existing in the Prometheus Pod. prometheus-operator is not responsible for
  # these SD files. The operator should use Prometheus.ConfigMaps to mount these files in the pods and have them usable
  # by ScrapeConfig. No validation on the content of the SD files is expected from prometheus-operator.
  - /etc/prometheus/configmaps/inventory/file.json
refreshInterval: 5m

```

--------------------------------

### ServiceMonitor with status.bindings

Source: https://prometheus-operator.dev/docs/proposals/accepted/configuration-object-status-subresource

Example ServiceMonitor manifest showing the status subresource with bindings to Prometheus and PrometheusAgent resources. The status includes conditions with type Accepted, status True or False, observedGeneration, reason, and message. The example shows a successful binding and two rejected bindings with error messages.

```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: example-servicemonitor
  namespace: monitoring
  generation: 3
  labels:
    team: backend
spec:
  selector:
    matchLabels:
      app: my-service
  namespaceSelector:
    matchNames:
      - default
  endpoints:
    - port: http
      path: /metrics
      interval: 30s
      scheme: http
      basicAuth:
        username:
          name: my-secret
          key: basic-auth-username
        password:
          name: my-secret
          key: basic-auth-password
  status:
    bindings:
      - group: monitoring.coreos.com
        resource: prometheuses
        name: main
        namespace: monitoring
        conditions:
          - type: Accepted
            status: "True"
            observedGeneration: 3
            lastTransitionTime: "2025-05-20T12:34:56Z"
            reason: ""
            message: ""
      - group: monitoring.coreos.com
        resource: prometheuses
        name: example
        namespace: default
        conditions:
          - type: Accepted
            status: "False"
            observedGeneration: 2
            lastTransitionTime: "2024-02-08T23:52:22Z"
            reason: InvalidConfiguration
            message: "'KeepEqual' relabel action is only supported with Prometheus >= 2.41.0"
      - group: monitoring.coreos.com
        resource: prometheusagents
        name: agent
        namespace: monitoring
        conditions:
          - type: Accepted
            status: "False"
            observedGeneration: 3
            lastTransitionTime: "2024-02-08T23:52:22Z"
            reason: InvalidConfiguration
            message: "Referenced Secret 'my-secret' in namespace 'monitoring' is missing or does not contain the required key 'basic-auth-password'"

```

--------------------------------

### ByteSize

Source: https://prometheus-operator.dev/docs/api-reference/api

A valid memory size type based on powers-of-2, with supported units and example.

```APIDOC
## ByteSize

### Description
A valid memory size type based on powers-of-2, so 1KB is 1024B. Supported units: B, KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB. Example: 512MB.
```

--------------------------------

### Build and load images into Kind with Podman

Source: https://prometheus-operator.dev/docs/community/testing

Run this command before automated e2e tests when using Podman as the container CLI.

```bash
CONTAINER_CLI=podman KIND_CONTEXT=e2e make test-e2e-images

```
Tokens:
Raw