Metricbeat

https://www.elastic.co/guide/en/beats/metricbeat/current/
Admin
Metricbeat is a lightweight shipper that collects metrics from the operating system and services...
Tokens:272,368
Snippets:2,080
Trust Score:10
Update:6 months ago
Show doc for...
Tokens:
Raw
### Manually Load Index Template (Windows PowerShell)

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-template.html

Execute the `metricbeat setup` command from a PowerShell prompt as Administrator. This command is for Windows installations.

```powershell
PS > .\metricbeat.exe setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'
```

--------------------------------

### Manually Load Index Template (deb/rpm)

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-template.html

Use the `metricbeat setup` command with index management enabled and Elasticsearch output configured. This command is for deb and rpm package installations.

```bash
metricbeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'
```

--------------------------------

### Example Cloudfoundry module configuration

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-cloudfoundry.html

A complete example configuration for the Cloudfoundry module including all available metricsets.

```yaml
metricbeat.modules:
- module: cloudfoundry
  metricsets:
    - container
    - counter
    - value
  enabled: true
  api_address: '${CLOUDFOUNDRY_API_ADDRESS:""}'
  doppler_address: '${CLOUDFOUNDRY_DOPPLER_ADDRESS:""}'
  uaa_address: '${CLOUDFOUNDRY_UAA_ADDRESS:""}'
  rlp_address: '${CLOUDFOUNDRY_RLP_ADDRESS:""}'
  client_id: '${CLOUDFOUNDRY_CLIENT_ID:""}'
  client_secret: '${CLOUDFOUNDRY_CLIENT_SECRET:""}'
  shard_id: metricbeat
  version: v1
```

--------------------------------

### Run Metricbeat setup command

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/could-not-locate-index-pattern.html

Executes the setup command to attempt automatic creation of the index pattern.

```bash
./metricbeat setup
```

--------------------------------

### Example Benchmark Module Configuration

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-benchmark.html

Example configuration for the Benchmark module within metricbeat.modules. This example has the module disabled.

```yaml
metricbeat.modules:
- module: benchmark
  metricsets:
    - info
  enabled: false
  period: 10s
```

--------------------------------

### Kubernetes state_deployment metricset Example

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kubernetes-state_deployment.html

An example of a document generated by the state_deployment metricset, showing the structure and fields collected.

```APIDOC
## Kubernetes state_deployment metricset Example

### Description
This example demonstrates the structure of a document collected by the `state_deployment` metricset of the Kubernetes module in Metricbeat.

### Request Example
```json
{
    "@timestamp": "2019-03-01T08:05:34.853Z",
    "event": {
        "dataset": "kubernetes.deployment",
        "duration": 115000,
        "module": "kubernetes"
    },
    "kubernetes": {
        "deployment": {
            "name": "local-path-provisioner",
            "paused": false,
            "replicas": {
                "available": 1,
                "desired": 1,
                "unavailable": 0,
                "updated": 1
            },
            "status": {
                "available": "true",
                "progressing": "true"
            }
        },
        "namespace": "local-path-storage"
    },
    "metricset": {
        "name": "state_deployment",
        "period": 10000
    },
    "service": {
        "address": "127.0.0.1:55555",
        "type": "kubernetes"
    }
}
```

### Response
#### Success Response (200)
This metricset does not have a direct API endpoint for requests. The data is collected by Metricbeat agents.

#### Response Example
See Request Example above, as this represents the data collected by the metricset.
```

--------------------------------

### Load Metricbeat dashboards via setup command

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/load-kibana-dashboards.html

Use the setup command to deploy sample dashboards and index templates to Kibana.

```bash
metricbeat setup --dashboards
```

```bash
metricbeat setup --dashboards
```

```bash
./metricbeat setup --dashboards
```

```bash
metricbeat setup --dashboards
```

```bash
docker run --rm --net="host" docker.elastic.co/beats/metricbeat:9.3.3 setup --dashboards
```

```powershell
PS > .\metricbeat.exe setup --dashboards
```

--------------------------------

### Example System uptime metricset document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-uptime.html

This is an example document generated by the system.uptime metricset, showing the structure and fields collected.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "agent": {
        "hostname": "host.example.com",
        "name": "host.example.com"
    },
    "event": {
        "dataset": "system.uptime",
        "duration": 115000,
        "module": "system"
    },
    "metricset": {
        "name": "uptime"
    },
    "service": {
        "type": "system"
    },
    "system": {
        "uptime": {
            "duration": {
                "ms": 1295582000
            }
        }
    }
}
		
```

--------------------------------

### Kubernetes state_statefulset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kubernetes-state_statefulset.html

This is an example document generated by the state_statefulset metricset. It shows the structure of the data collected for each StatefulSet.

```json
{
    "@timestamp": "2019-03-01T08:05:34.853Z",
    "event": {
        "dataset": "kubernetes.statefulset",
        "duration": 115000,
        "module": "kubernetes"
    },
    "kubernetes": {
        "namespace": "default",
        "statefulset": {
            "created": 1744115807,
            "generation": {
                "desired": 1,
                "observed": 1
            },
            "name": "basic-sts",
            "replicas": {
                "desired": 1,
                "observed": 1,
                "ready": 1
            }
        }
    },
    "metricset": {
        "name": "state_statefulset",
        "period": 10000
    },
    "service": {
        "address": "127.0.0.1:55555",
        "type": "kubernetes"
    }
}
		
```

--------------------------------

### Example System Service Metricset Document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-service.html

This is an example document generated by the system service metricset, showing details about a systemd service.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "system.service",
        "duration": 115000,
        "module": "system"
    },
    "metricset": {
        "name": "service",
        "period": 10000
    },
    "process": {
        "exit_code": 0,
        "pid": 259
    },
    "service": {
        "type": "system"
    },
    "system": {
        "service": {
            "exec_code": "exited",
            "load_state": "loaded",
            "name": "dracut-pre-udev.service",
            "state": "inactive",
            "state_since": "2020-08-26T18:05:23.525244-07:00",
            "sub_state": "dead",
            "unit_file": {
                "state": "static",
                "vendor_preset": "disabled"
            }
        }
    },
    "systemd": {
        "fragment_path": "/usr/lib/dracut/modules.d/98dracut-systemd/dracut-pre-udev.service",
        "unit": "dracut-pre-udev.service"
    }
}
		
```

--------------------------------

### Regexp Condition Example

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/defining-processors.html

The `regexp` condition checks a field against a regular expression. This example checks if the process name starts with 'foo'.

```yaml
regexp:
  system.process.name: "^foo.*"
```

--------------------------------

### Configure Metricbeat Nginx Module

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-nginx.html

Example configuration for the Nginx module in Metricbeat. Ensure Metricbeat is installed and configured to use this module. The `server_status_path` defaults to `nginx_status` if not specified.

```yaml
metricbeat.modules:
- module: nginx
  metricsets:
  - "stubstatus"
  enabled: true
  period: 10s

  # Nginx hosts
  hosts: ["http://127.0.0.1"]

  # Path to server status. Default nginx_status
  server_status_path: "nginx_status"
```

--------------------------------

### Configure Windows perfmon metricset

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-windows-perfmon.html

Example configuration for collecting processor time and disk writes every 10 seconds. This setup includes options for handling non-existent counters and grouping measurements by instance.

```yaml
- module: windows
  metricsets: [perfmon]
  period: 10s
  perfmon.ignore_non_existent_counters: true
  perfmon.group_measurements_by_instance: true
  perfmon.extract_object_from_counter: true
  perfmon.queries:
  - object: "Process"
    instance: ["svchost*", "conhost*"]
    counters:
    - name: "% Processor Time"
      field: time.processor.pct
      format: "float"
    - name: "Thread Count"
      field: thread_count
    - name: "IO Read Operations/sec"
  - object: "PhysicalDisk"
    field : "disk"
    instance: "*"
    counters:
    - name: "Disk Writes/sec"
    - name: "% Disk Write Time"
      field: "write_time"
      format: "float"
```

--------------------------------

### Configure general Beat options

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-general-options.html

Example configuration for common Beat settings including name and tags.

```yaml
name: "my-shipper"
tags: ["service-X", "web-tier"]
		
```

--------------------------------

### Manually Load Index Template (mac/linux)

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-template.html

Execute the `metricbeat setup` command with index management enabled and Elasticsearch output specified. This command is for macOS and Linux systems.

```bash
./metricbeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'
```

--------------------------------

### Start Metricbeat on macOS/Linux

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-starting.html

Start Metricbeat on macOS or Linux. Ensure correct file ownership for configuration files or use the --strict.perms=false flag. This command starts Metricbeat in the foreground.

```bash
sudo chown root metricbeat.yml
sudo chown root modules.d/{modulename}.yml
sudo ./metricbeat -e		

```

--------------------------------

### PHP_FPM Process Metricset Example

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-php_fpm-process.html

Example document structure for the PHP_FPM process metricset.

```APIDOC
## PHP_FPM process metricset

### Description
This metricset collects process-level metrics from the PHP_FPM module.

### Response Example
{
    "@timestamp": "2019-03-01T08:05:34.853Z",
    "event": {
        "dataset": "php_fpm.process",
        "duration": 115000,
        "module": "php_fpm"
    },
    "http": {
        "request": {
            "method": "get"
        },
        "response": {
            "body": {
                "bytes": 0
            }
        }
    },
    "metricset": {
        "name": "process",
        "period": 10000
    },
    "php_fpm": {
        "pool": {
            "name": "www"
        },
        "process": {
            "last_request_cpu": 0,
            "last_request_memory": 0,
            "request_duration": 135,
            "requests": 9,
            "script": "-",
            "start_since": 3471,
            "start_time": 1551792028,
            "state": "Running"
        }
    },
    "process": {
        "pid": 24
    },
    "service": {
        "address": "127.0.0.1:55555",
        "type": "php_fpm"
    },
    "url": {
        "original": "/status?full&json"
    },
    "user": {
        "name": "-"
    }
}
```

--------------------------------

### Configure Kibana protocol and path

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/setup-kibana-endpoint.html

Example showing how to specify the protocol and path for the Kibana connection.

```yaml
setup.kibana.host: "192.0.2.255:5601"
setup.kibana.protocol: "http"
setup.kibana.path: /kibana
```

--------------------------------

### Example state_deployment document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kubernetes-state_deployment.html

A sample JSON document generated by the state_deployment metricset showing deployment metadata and status fields.

```json
{
    "@timestamp": "2019-03-01T08:05:34.853Z",
    "event": {
        "dataset": "kubernetes.deployment",
        "duration": 115000,
        "module": "kubernetes"
    },
    "kubernetes": {
        "deployment": {
            "name": "local-path-provisioner",
            "paused": false,
            "replicas": {
                "available": 1,
                "desired": 1,
                "unavailable": 0,
                "updated": 1
            },
            "status": {
                "available": "true",
                "progressing": "true"
            }
        },
        "namespace": "local-path-storage"
    },
    "metricset": {
        "name": "state_deployment",
        "period": 10000
    },
    "service": {
        "address": "127.0.0.1:55555",
        "type": "kubernetes"
    }
}
```

--------------------------------

### System diskio metricset Example Document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-diskio.html

An example of a JSON document generated by the System diskio metricset.

```APIDOC
## System diskio metricset Example Document

### Description

This section provides an example of the JSON document structure generated by the `diskio` metricset when it collects disk I/O metrics from the operating system.

### Method

Data Collection

### Endpoint

N/A (Data is collected by Metricbeat agent)

### Parameters

N/A

### Request Example

N/A

### Response

#### Success Response (200)

- **`@timestamp`** (string) - The timestamp of the event.
- **`event.dataset`** (string) - The dataset name, typically `system.diskio`.
- **`event.duration`** (integer) - The duration of the I/O operation in microseconds.
- **`event.module`** (string) - The module name, typically `system`.
- **`metricset.name`** (string) - The name of the metricset, which is `diskio`.
- **`metricset.period`** (integer) - The collection period in milliseconds.
- **`service.type`** (string) - The type of service, typically `system`.
- **`system.diskio.io.ops`** (integer) - The number of I/O operations.
- **`system.diskio.io.time`** (integer) - The total time spent on I/O operations in microseconds.
- **`system.diskio.name`** (string) - The name of the disk device.
- **`system.diskio.read.bytes`** (integer) - The number of bytes read from the disk.
- **`system.diskio.read.count`** (integer) - The number of read operations.
- **`system.diskio.read.time`** (integer) - The total time spent on read operations in microseconds.
- **`system.diskio.write.bytes`** (integer) - The number of bytes written to the disk.
- **`system.diskio.write.count`** (integer) - The number of write operations.
- **`system.diskio.write.time`** (integer) - The total time spent on write operations in microseconds.

#### Response Example

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "system.diskio",
        "duration": 115000,
        "module": "system"
    },
    "metricset": {
        "name": "diskio",
        "period": 10000
    },
    "service": {
        "type": "system"
    },
    "system": {
        "diskio": {
            "io": {
                "ops": 0,
                "time": 545105
            },
            "name": "sda",
            "read": {
                "bytes": 4242313728,
                "count": 181371,
                "time": 2744086
            },
            "write": {
                "bytes": 9611375104,
                "count": 352596,
                "time": 10641320
            }
        }
    }
}
```

### Error Handling

N/A
```

--------------------------------

### RabbitMQ queue metricset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-rabbitmq-queue.html

Example JSON document generated by the RabbitMQ queue metricset.

```json
{
    "@timestamp": "2019-03-01T08:05:34.853Z",
    "event": {
        "dataset": "rabbitmq.queue",
        "duration": 115000,
        "module": "rabbitmq"
    },
    "metricset": {
        "name": "queue"
    },
    "rabbitmq": {
        "node": {
            "name": "rabbit@localhost"
        },
        "queue": {
            "arguments": {
                "max_priority": 9
            },
            "auto_delete": false,
            "consumers": {
                "count": 3,
                "utilisation": {
                    "pct": 0.7
                }
            },
            "disk": {
                "reads": {
                    "count": 212
                },
                "writes": {
                    "count": 121
                }
            },
            "durable": true,
            "exclusive": false,
            "memory": {
                "bytes": 232720
            },
            "messages": {
                "persistent": {
                    "count": 73
                },
                "ready": {
                    "count": 71,
                    "details": {
                        "rate": 0
                    }
                },
                "total": {
                    "count": 74,
                    "details": {
                        "rate": 2.2
                    }
                },
                "unacknowledged": {
                    "count": 3,
                    "details": {
                        "rate": 0.5
                    }
                }
            },
            "name": "queuenamehere",
            "state": "running"
        },
        "vhost": "/"
    },
    "service": {
        "address": "127.0.0.1:55555",
        "type": "rabbitmq"
    }
}
```

--------------------------------

### Example document generated by the system filesystem metricset

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-filesystem.html

This is an example of a JSON document produced by the system filesystem metricset, showing details like available space, device name, file counts, and mount point.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "system.filesystem",
        "duration": 115000,
        "module": "system"
    },
    "metricset": {
        "name": "filesystem",
        "period": 10000
    },
    "service": {
        "type": "system"
    },
    "system": {
        "filesystem": {
            "available": 148708327424,
            "device_name": "/dev/mapper/fedora-root",
            "files": 105089024,
            "free": 148708327424,
            "free_files": 103974920,
            "mount_point": "/",
            "total": 215211835392,
            "type": "xfs",
            "used": {
                "bytes": 66503507968,
                "pct": 0.309
            }
        }
    }
}
		
```

--------------------------------

### RabbitMQ shovel metricset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-rabbitmq-shovel.html

Example JSON document generated by the RabbitMQ shovel metricset.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "rabbitmq.shovel",
        "duration": 115000,
        "module": "rabbitmq"
    },
    "metricset": {
        "name": "shovel",
        "period": 10000
    },
    "rabbitmq": {
        "node": {
            "name": "rabbit@localhost"
        },
        "shovel": {
            "name": "testshovel",
            "state": "running",
            "type": "dynamic"
        },
        "vhost": "/"
    },
    "service": {
        "address": "127.0.0.1:60847",
        "type": "rabbitmq"
    }
}
```

--------------------------------

### Configure APT Repository

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/setup-repositories.html

Commands to add the Elastic APT repository and install Metricbeat.

```bash
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
```

```bash
sudo apt-get install apt-transport-https
```

```bash
echo "deb https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-9.x.list
```

```bash
echo "deb https://artifacts.elastic.co/packages/oss-9.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-9.x.list
```

```bash
sudo apt-get update && sudo apt-get install metricbeat
```

```bash
sudo systemctl enable metricbeat
```

--------------------------------

### PostgreSQL activity metricset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-postgresql-activity.html

Example JSON document generated by the PostgreSQL activity metricset.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "postgresql.activity",
        "duration": 115000,
        "module": "postgresql"
    },
    "metricset": {
        "name": "activity",
        "period": 10000
    },
    "postgresql": {
        "activity": {
            "application_name": "",
            "backend_start": "2021-03-05T19:01:40.467Z",
            "backend_type": "client backend",
            "client": {
                "address": "192.168.128.1",
                "hostname": "",
                "port": 33414
            },
            "database": {
                "name": "postgres",
                "oid": 13395
            },
            "pid": 2236,
            "query": "SELECT * FROM pg_stat_activity",
            "query_start": "2021-03-05T19:01:40.469Z",
            "state": "idle",
            "state_change": "2021-03-05T19:01:40.471Z",
            "user": {
                "id": 10,
                "name": "postgres"
            }
        }
    },
    "service": {
        "address": "192.168.128.2:5432",
        "type": "postgresql"
    }
}
```

--------------------------------

### Configure SQL Module in Metricbeat

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-sql.html

A complete example configuration for the SQL module, including driver selection, query definition, and commented-out SSL parameter examples.

```yaml
metricbeat.modules:
- module: sql
  metricsets:
    - query
  period: 10s
  hosts: ["postgres://postgres:mysecretpassword@localhost:5432"]
  # Example of using SSL parameters manually in the Postgres connection string (with ssl.* parameters unset). The Postgres SSL parameters "sslmode", "sslcert", "sslkey", and "sslrootcert" are passed in the connection string with slashes "/" being url-encoded to "%2F"
  # hosts: ["postgres://postgres:mysecretpassword@localhost:5432?sslmode=verify-full&sslcert=%2Fpath%2Fto%2Fcert.pem&sslkey=%2Fpath%2Fto%2Fkey.pem&sslrootcert=%2Fpath%2Fto%2Fca.pem"]
  # Example for SQL server
  # hosts: ["sqlserver://myuser:mypassword@localhost:1433?TrustServerCertificate=false&certificate=%2Fpath%2Fto%2Fca.pem&database=mydb&encrypt=true"]


  driver: "postgres"
  sql_query: "select now()"
  sql_response_format: table

  # List of root certificates for SSL/TLS server verification
  # ssl.certificate_authorities: ["/path/to/ca.pem"]

  # Certificate for SSL/TLS client authentication
  # ssl.certificate: "/path/to/client-cert.pem"

  # Client certificate key file
  # ssl.key: "/path/to/client-key.pem"

  # Controls the verification of server certificate
  # ssl.verification_mode: full
```

--------------------------------

### Full MySQL Module Configuration

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-mysql.html

A comprehensive configuration example including optional settings like SSL/TLS and additional metricsets.

```yaml
metricbeat.modules:
- module: mysql
  metricsets:
    - status
  #  - galera_status
  #  - performance
  #  - query
  period: 10s

  # Host DSN should be defined as "user:pass@tcp(127.0.0.1:3306)/"
  # or "unix(/var/lib/mysql/mysql.sock)/",
  # or another DSN format supported by <https://github.com/Go-SQL-Driver/MySQL/>.
  # The username and password can either be set in the DSN or using the username
  # and password config options. Those specified in the DSN take precedence.
  hosts: ["root:secret@tcp(127.0.0.1:3306)/"]

  # Username of hosts. Empty by default.
  #username: root

  # Password of hosts. Empty by default.
  #password: secret

  # By setting raw to true, all raw fields from the status metricset will be added to the event.
  #raw: false

  # Optional SSL/TLS. By default is false.
  #ssl.enabled: true

  # List of root certificates for SSL/TLS server verification
  #ssl.certificate_authorities: ["/etc/pki/root/ca.crt"]

  # Certificate for SSL/TLS client authentication
  #ssl.certificate: "/etc/pki/client/cert.crt"

  # Client certificate key file
  #ssl.key: "/etc/pki/client/cert.key"
```

--------------------------------

### Kafka partition metricset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kafka-partition.html

Example JSON document generated by the Kafka partition metricset.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "kafka.partition",
        "duration": 115000,
        "module": "kafka"
    },
    "kafka": {
        "broker": {
            "address": "172.21.0.2:9092",
            "id": 0
        },
        "partition": {
            "id": 0,
            "offset": {
                "newest": 1,
                "oldest": 0
            },
            "partition": {
                "insync_replica": true,
                "is_leader": true,
                "leader": 0,
                "replica": 0
            },
            "topic_broker_id": "0-metricbeat-generate-data-0",
            "topic_id": "0-metricbeat-generate-data"
        },
        "topic": {
            "name": "metricbeat-generate-data"
        }
    },
    "metricset": {
        "name": "partition",
        "period": 10000
    },
    "service": {
        "address": "172.21.0.2:9092",
        "type": "kafka"
    }
}
```

--------------------------------

### Example Jolokia Metricset Document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-jolokia-jmx.html

This is an example of a document generated by the Jolokia metricset, showcasing the structure and fields.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "jolokia.testnamespace",
        "duration": 115000,
        "module": "jolokia"
    },
    "jolokia": {
        "testnamespace": {
            "memory": {
                "heap_usage": {
                    "committed": 514850816,
                    "init": 536870912,
                    "max": 7635730432,
                    "used": 42335648
                },
                "non_heap_usage": {
                    "committed": 32243712,
                    "init": 2555904,
                    "max": -1,
                    "used": 29999896
                }
            },
            "uptime": 70795470
        }
    },
    "metricset": {
        "name": "jmx"
    },
    "service": {
        "address": "127.0.0.1:8778",
        "type": "jolokia"
    }
}
		
```

--------------------------------

### Example System Users Metricset Document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-users.html

This is an example document generated by the system/users metricset. It shows the structure and fields captured, including timestamp, event details, metricset information, process ID, service type, source IP, system user details, and user information.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "system.users",
        "duration": 115000,
        "module": "system"
    },
    "metricset": {
        "name": "users",
        "period": 10000
    },
    "process": {
        "pid": 10786
    },
    "service": {
        "type": "system"
    },
    "source": {
        "ip": "192.168.1.86"
    },
    "system": {
        "users": {
            "id": 6,
            "leader": 10786,
            "path": "/org/freedesktop/login1/session/_36",
            "remote": true,
            "remote_host": "192.168.1.86",
            "scope": "session-6.scope",
            "seat": "",
            "service": "sshd",
            "state": "active",
            "type": "tty"
        }
    },
    "user": {
        "id": "1000",
        "name": "alexk"
    }
}
		
```

--------------------------------

### Kafka consumergroup metricset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kafka-consumergroup.html

Example JSON document generated by the Kafka consumergroup metricset.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "kafka.consumergroup",
        "duration": 115000,
        "module": "kafka"
    },
    "kafka": {
        "broker": {
            "address": "172.21.0.2:9092",
            "id": 0
        },
        "consumergroup": {
            "client": {
                "host": "127.0.0.1",
                "id": "consumer-1",
                "member_id": "consumer-1-8653cb3a-afed-4b1b-87d0-2a208319b41e"
            },
            "consumer_lag": 77,
            "error": {
                "code": 0
            },
            "id": "console-consumer-40539",
            "meta": "",
            "offset": -1
        },
        "partition": {
            "id": 0,
            "topic_id": "0-test"
        },
        "topic": {
            "name": "test"
        }
    },
    "metricset": {
        "name": "consumergroup",
        "period": 10000
    },
    "service": {
        "address": "172.21.0.2:9092",
        "type": "kafka"
    }
}
```

--------------------------------

### Configure system core metricset

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-core.html

Example configuration for the system module using the core metricset with specific metric types enabled.

```yaml
metricbeat.modules:
- module: system
  metricsets: [core]
  core.metrics: [percentages, ticks]
  #use_performance_counters: false
```

--------------------------------

### Input Syslog Message Example

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/syslog.html

Example of a raw RFC 5424 syslog message stored in a JSON object.

```json
{
  "message": "<165>1 2022-01-11T22:14:15.003Z mymachine.example.com eventslog 1024 ID47 [exampleSDID@32473 iut=\"3\" eventSource=\"Application\" eventID=\"1011\"][examplePriority@32473 class=\"high\"] this is the message"
}
```

--------------------------------

### Example Linux Memory Document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-linux-memory.html

This is an example of a document generated by the linux.memory metricset, showing various memory statistics.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "linux.memory",
        "duration": 115000,
        "module": "linux"
    },
    "linux": {
        "memory": {
            "hugepages": {
                "default_size": 2097152,
                "free": 0,
                "reserved": 0,
                "surplus": 0,
                "swap": {
                    "out": {
                        "fallback": 0,
                        "pages": 0
                    }
                },
                "total": 0,
                "used": {
                    "bytes": 0,
                    "pct": 0
                }
            },
            "page_stats": {
                "direct_efficiency": {
                    "pct": 0.3419
                },
                "kswapd_efficiency": {
                    "pct": 0.8922
                },
                "pgfree": {
                    "pages": 16328309361
                },
                "pgscan_direct": {
                    "pages": 21037589
                },
                "pgscan_kswapd": {
                    "pages": 49831
                },
                "pgsteal_direct": {
                    "pages": 7193725
                },
                "pgsteal_kswapd": {
                    "pages": 44459
                }
            },
            "swap": {
                "free": 8586252288,
                "in": {
                    "pages": 72435
                },
                "out": {
                    "pages": 2032475
                },
                "readahead": {
                    "cached": 5,
                    "pages": 29
                },
                "total": 8589930496,
                "used": {
                    "bytes": 3678208,
                    "pct": 0.0004
                }
            },
            "vmstat": {
                "allocstall_dma": 0,
                "allocstall_dma32": 0,
                "allocstall_movable": 60572,
                "allocstall_normal": 1125,
                "balloon_deflate": 0,
                "balloon_inflate": 0,
                "balloon_migrate": 0,
                "cma_alloc_fail": 0,
                "cma_alloc_success": 0,
                "compact_daemon_free_scanned": 0,
                "compact_daemon_migrate_scanned": 0,
                "compact_daemon_wake": 2,
                "compact_fail": 0,
                "compact_free_scanned": 406096,
                "compact_isolated": 15241,
                "compact_migrate_scanned": 51056,
                "compact_stall": 0,
                "compact_success": 0,
                "direct_map_level2_splits": 864,
                "direct_map_level3_splits": 52,
                "drop_pagecache": 0,
                "drop_slab": 0,
                "htlb_buddy_alloc_fail": 0,
                "htlb_buddy_alloc_success": 0,
                "kswapd_high_wmark_hit_quickly": 0,
                "kswapd_inodesteal": 2,
                "kswapd_low_wmark_hit_quickly": 1,
                "nr_active_anon": 5553,
                "nr_active_file": 1478109,
                "nr_anon_pages": 1024369,
                "nr_anon_transparent_hugepages": 140,
                "nr_bounce": 0,
                "nr_dirtied": 87806596,
                "nr_dirty": 60,
                "nr_dirty_background_threshold": 1339484,
                "nr_dirty_threshold": 2682244,
                "nr_file_hugepages": 0,
                "nr_file_pages": 5302193,
                "nr_file_pmdmapped": 0,
                "nr_foll_pin_acquired": 2,
                "nr_foll_pin_released": 2,
                "nr_free_cma": 0,
                "nr_free_pages": 8219610,
                "nr_inactive_anon": 1046105,
                "nr_inactive_file": 3825289,
                "nr_isolated_anon": 0,
                "nr_isolated_file": 0,
                "nr_kernel_misc_reclaimable": 0,
                "nr_kernel_stack": 14144,
                "nr_mapped": 191441,
                "nr_mlock": 0,
                "nr_page_table_pages": 6756,
                "nr_shmem": 9446,
                "nr_shmem_hugepages": 0,
                "nr_shmem_pmdmapped": 0,
                "nr_slab_reclaimable": 362837,
                "nr_slab_unreclaimable": 167630,
                "nr_swapcached": 2,
                "nr_throttled_written": 0,
                "nr_unevictable": 768,
                "nr_unstable": 0,
                "nr_vmscan_immediate_reclaim": 10,
                "nr_vmscan_write": 2032594,
                "nr_writeback": 0,
                "nr_writeback_temp": 0,
                "nr_written": 77862049,
                "nr_zone_active_anon": 5553,
                "nr_zone_active_file": 1478109,
                "nr_zone_inactive_anon": 1046105,
                "nr_zone_inactive_file": 3825289
            }
        }
    }
}
```

--------------------------------

### Example system load document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-load.html

A sample JSON document generated by the system load metricset.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "agent": {
        "hostname": "host.example.com",
        "name": "host.example.com"
    },
    "event": {
        "dataset": "system.load",
        "duration": 115000,
        "module": "system"
    },
    "metricset": {
        "name": "load"
    },
    "service": {
        "type": "system"
    },
    "system": {
        "load": {
            "1": 2.33,
            "15": 2.03,
            "5": 1.93,
            "cores": 4,
            "norm": {
                "1": 0.5825,
                "15": 0.5075,
                "5": 0.4825
            }
        }
    }
}
```

--------------------------------

### System diskio metricset Configuration

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-diskio.html

Configuration example for the System diskio metricset, showing how to include specific disk devices.

```APIDOC
## System diskio metricset Configuration

### Description

This section details the configuration options for the `diskio` metricset within the `system` module. Specifically, it explains how to use `diskio.include_devices` to filter which disk devices are reported.

### Method

Configuration

### Endpoint

N/A (Configuration file setting)

### Parameters

#### Query Parameters

- **`diskio.include_devices`** (array of strings) - Optional - Defines a list of device names to pre-filter the devices that are reported. Filters only exact matches. If not set or given an empty array `[]`, all disk devices are returned.

### Request Example

```yaml
metricbeat.modules:
- module: system
  metricsets:
    - "diskio"
  diskio.include_devices:
    - "sda"
    - "sda1"
```

### Response

N/A (This is a configuration example, not an API response)

### Error Handling

N/A
```

--------------------------------

### NATS connection metricset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-nats-connection.html

Example JSON document structure generated by the NATS connection metricset.

```json
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "nats.connection",
        "duration": 115000,
        "module": "nats"
    },
    "metricset": {
        "name": "connection",
        "period": 10000
    },
    "nats": {
        "connection": {
            "id": 18,
            "idle_time": 416,
            "in": {
                "bytes": 0,
                "messages": 0
            },
            "ip": "127.0.0.1",
            "lang": "go",
            "last_activity": "2025-03-16T13:38:16.992424387Z",
            "name": "NATS CLI Version development",
            "out": {
                "bytes": 0,
                "messages": 0
            },
            "pending_bytes": 0,
            "port": 44042,
            "start": "2025-03-16T13:38:16.99188926Z",
            "subscriptions": 1,
            "uptime": 416,
            "version": "1.38.0"
        },
        "server": {
            "id": "NCLTOCCWWFT55ANXSNIMXTP2Q4CEL42V2K2BNDXGM5YNMTBMULQO5V7L"
        }
    },
    "service": {
        "address": "localhost:60207",
        "type": "nats"
    }
}
```

--------------------------------

### Full DNS Processor Configuration

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/processor-dns.html

A comprehensive configuration example demonstrating all available options, including cache settings, transport protocols, and custom nameservers.

```yaml
processors:
- dns:
    type: reverse
    action: append
    transport: tls
    fields:
      server.ip: server.domain
      client.ip: client.domain
    success_cache:
      capacity.initial: 1000
      capacity.max: 10000
      min_ttl: 1m
    failure_cache:
      capacity.initial: 1000
      capacity.max: 10000
      ttl: 1m
    nameservers: ['192.0.2.1', '203.0.113.1']
    timeout: 500ms
    tag_on_failure: [_dns_reverse_lookup_failed]
```

--------------------------------

### Example WQL Query and YAML Configuration

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-windows-wmi.html

Translate WQL queries into Metricbeat YAML configuration. Test queries using PowerShell's Get-CimInstance or WMI Explorer.

```yaml
- class: Win32_Process
  properties:
  - Name
  - ProcessId
  - WorkingSetSize
  where: "Name = 'lsass.exe' AND WorkingSetSize > 104857600"
```

--------------------------------

### Kubernetes event metricset example document

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-kubernetes-event.html

Example JSON document structure produced by the Kubernetes event metricset.

```json
{
    "@timestamp": "2017-05-15T08:07:12.945Z",
    "beat": {
        "hostname": "hostname",
        "name": "beatname",
        "version": "6.0.0-alpha2"
    },
    "kubernetes": {
        "event": {
            "count": 1,
            "involved_object": {
                "api_version": "extensions",
                "kind": "ReplicaSet",
                "name": "prometheus-2552087900",
                "resource_version": "1047038",
                "uid": "b2f92f14-2ad5-11e7-8cb8-e687a39f6e48"
            },
            "message": "Created pod: prometheus-2552087900-9fxh6",
            "metadata": {
                "generate_name": "",
                "name": "prometheus-2552087900.14bf266355fd16e0",
                "namespace": "default",
                "resource_version": "1047243",
                "self_link": "/api/v1/namespaces/default/events/prometheus-2552087900.14bf266355fd16e0",
                "timestamp": {
                    "created": "2017-05-16T10:30:09-07:00",
                    "deleted": ""
                },
                "uid": "4f3fe524-3a5d-11e7-b8f2-e687a39f6e48"
            },
            "reason": "SuccessfulCreate",
            "timestamp": {
                "first_occurrence": "2017-05-16T17:30:09Z",
                "last_occurrence": "2017-05-16T17:30:09Z"
            },
            "type": "Normal"
        }
    },
    "metricset": {
        "module": "kubernetes",
        "name": "event"
    }
}
```

--------------------------------

### Start Metricbeat on Debian/RPM

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-starting.html

Use this command to start Metricbeat on systems using init.d scripts. Command line flags cannot be specified when using this method.

```bash
sudo service metricbeat start		

```

--------------------------------

### Configure Jolokia JMX Collection with GET Method

Source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-jolokia-jmx.html

Configure Metricbeat to collect JMX metrics using the HTTP GET method. This is useful when POST is unavailable, but note that proxy requests are not allowed and multiple mappings result in multiple GET requests.

```yaml
- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
  hosts: ["localhost:8080"]
  namespace: "jolokia_metrics"
  path: "/jolokia"
  http_method: 'GET'
  jmx.mappings:
    - mbean: 'java.lang:type=Memory'
      attributes:
       - attr: HeapMemoryUsage
         field: memory.heap_usage
       - attr: NonHeapMemoryUsage
         field: memory.non_heap_usage
    - mbean: 'Catalina:name=*,type=ThreadPool'
      attributes:
       - attr: port
         field: catalina.port
       - attr: maxConnections
         field: catalina.maxConnections
    - mbean: 'java.lang:type=Runtime'
      attributes:
       - attr: Uptime
         field: uptime		

```
Tokens:
Raw