### Install Tilt

Source: https://github.com/elastic/beats/blob/main/dev-tools/kubernetes/README.md

Installs the Tilt development toolkit. Run this command to get started with Tilt.

```shell
curl -fsSL https://raw.githubusercontent.com/tilt-dev/tilt/master/scripts/install.sh | bash
```

--------------------------------

### Download Example Configuration

Source: https://github.com/elastic/beats/blob/main/docs/reference/auditbeat/running-on-docker.md

Download the example Auditbeat configuration file for Docker to use as a starting point.

```sh
curl -L -O https://raw.githubusercontent.com/elastic/beats/{{ version.stack | M.M }}/deploy/docker/auditbeat.docker.yml
```

--------------------------------

### Install Govcsim

Source: https://github.com/elastic/beats/blob/main/metricbeat/module/vsphere/_meta/README.md

Install the govcsim simulator using the go get command.

```go
go get -u github.com/vmware/vic/cmd/vcsim
```

--------------------------------

### Salesforce Setup Audit Trail Configuration Example

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/_set_up_the_oauth_app_in_the_salesforce_2.md

This example shows how to configure the Salesforce setup audit trail fileset with user-password authentication. It includes settings for initial data collection, API version, and real-time collection.

```yaml
- module: salesforce
  setupaudittrail:
    enabled: true
    var.initial_interval: 1d
    var.api_version: 56

    var.authentication:
      jwt_bearer_flow:
        enabled: false
        client.id: "my-client-id"
        client.username: "my.email@here.com"
        client.key_path: client_key.pem
        url: https://login.salesforce.com
        token_url: "<custom-token-url>"
      user_password_flow:
        enabled: true
        client.id: "my-client-id"
        client.secret: "my-client-secret"
        token_url: "https://login.salesforce.com"
        username: "my.email@here.com"
        password: "password"

    var.url: "https://instance-url.salesforce.com"

    var.real_time: true
    var.real_time_interval: 5m

```

--------------------------------

### Download Example Heartbeat Docker Configuration

Source: https://github.com/elastic/beats/blob/main/docs/reference/heartbeat/running-on-docker.md

Download the example Heartbeat configuration file for Docker to use as a starting point.

```shell
curl -L -O https://raw.githubusercontent.com/elastic/beats/{{ version.stack | M.M }}/deploy/docker/heartbeat.docker.yml
```

--------------------------------

### Download Example Configuration

Source: https://github.com/elastic/beats/blob/main/docs/reference/packetbeat/running-on-docker.md

Download the default Packetbeat Docker configuration file to use as a starting point.

```sh
curl -L -O https://raw.githubusercontent.com/elastic/beats/{{ version.stack | M.M }}/deploy/docker/packetbeat.docker.yml
```

--------------------------------

### Download Example Configuration

Source: https://github.com/elastic/beats/blob/main/docs/reference/metricbeat/running-on-docker.md

Download the default Metricbeat Docker configuration file to use as a starting point.

```shell
curl -L -O https://raw.githubusercontent.com/elastic/beats/{{ version.stack | M.M }}/deploy/docker/metricbeat.docker.yml
```

--------------------------------

### Download Example Filebeat Configuration

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/running-on-docker.md

Download the example Filebeat configuration file to use as a starting point for your Docker deployment.

```shell
curl -L -O https://raw.githubusercontent.com/elastic/beats/{{ version.stack | M.M }}/deploy/docker/filebeat.docker.yml
```

--------------------------------

### GCP Module Example Configurations

Source: https://github.com/elastic/beats/blob/main/docs/reference/metricbeat/metricbeat-module-gcp.md

These examples show how to configure the GCP module for different services like compute, pubsub, storage, and more. Adjust project IDs, credentials, and periods based on your GCP setup.

```yaml
metricbeat.modules:
- module: gcp
  metricsets:
    - compute
  region: "us-"
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m
  metadata_cache: true
  metadata_cache_refresh_period: 1h

- module: gcp
  metricsets:
    - pubsub
    - loadbalancing
    - firestore
  zone: "us-central1-a"
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m

- module: gcp
  metricsets:
    - storage
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 5m

- module: gcp
  metricsets:
    - metrics
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m
  location_label: "resource.labels.zone"
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types:
        - "instance/cpu/reserved_cores"
        - "instance/cpu/usage_time"
        - "instance/cpu/utilization"
        - "instance/uptime"

- module: gcp
  metricsets:
    - gke
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m

- module: gcp
  metricsets:
    - billing
  period: 24h
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  dataset_id: "dataset id"
  table_pattern: "table pattern"
  cost_type: "regular"

- module: gcp
  metricsets:
    - carbon
  period: 24h
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  endpoint: http://your-endpoint
  dataset_id: "dataset id"
  table_pattern: "table pattern"

- module: gcp
  metricsets:
    - dataproc
  zone: "us-central1-a"
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m
  collect_dataproc_user_labels: true

- module: gcp
  metricsets:
    - vertexai_logs
  period: 300s  # 5 minutes
  project_id: "your-project-id"
  table_id: "your-project-id.dataset.id.table_name"
  credentials_file_path: "/path/to/service-account.json"
  # credentials_json: '{"type": "service_account", ...}'
  time_lookback_hours: 1  # How many hours back to look for initial data fetch
```

--------------------------------

### View help for the setup command

Source: https://github.com/elastic/beats/blob/main/docs/extend/import-dashboards.md

Access command-line help for the `setup` command to understand its available flags and options. This is useful for learning about specific setup phases like dashboards, index management, or pipelines.

```shell
./metricbeat help setup
```

--------------------------------

### Run the Beat setup command

Source: https://github.com/elastic/beats/blob/main/docs/extend/import-dashboards.md

Use the `setup` command to import all dashboards and the index pattern for a Beat. This command also loads other dependencies like index mapping templates, ingest pipelines, and ILM policies.

```shell
./metricbeat setup
```

--------------------------------

### Translate LDAP GUIDs with Caching

Source: https://github.com/elastic/beats/blob/main/docs/reference/auditbeat/processor-translate-guid.md

This example demonstrates how to use the cache processor to store and retrieve LDAP lookups for GUIDs, significantly speeding up processing. It first attempts to get the common name from the cache and, if not found, uses the translate_ldap_attribute processor to query LDAP and then caches the result.

```yaml
processors:
  - cache:
      backend:
        memory:
          id: ldapguids
      get:
        key_field: winlog.event_data.ObjectGuid
        target_field: winlog.common_name
      ignore_missing: true
  - if:
      not:
        - has_fields: winlog.common_name
    then:
      - translate_ldap_attribute:
          field: winlog.event_data.ObjectGuid
          target_field: winlog.common_name
          ldap_address: "ldap://"
          ldap_base_dn: "dc=example,dc=com"
      - cache:
          backend:
            memory:
              id: ldapguids
            capacity: 10000
          put:
            key_field: winlog.event_data.ObjectGuid
            value_field: winlog.common_name
```

--------------------------------

### Setup Winlogbeat Environment

Source: https://github.com/elastic/beats/blob/main/docs/reference/winlogbeat/command-line-options.md

The `setup` command configures the initial environment for Winlogbeat, including index templates, ILM policies, and Kibana dashboards. Use flags to specify which components to set up.

```sh
winlogbeat setup --dashboards
```

```sh
winlogbeat setup --index-management
```

--------------------------------

### AWS CloudWatch Input Configuration

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/filebeat-input-aws-cloudwatch.md

Example configuration for the Filebeat aws-cloudwatch input. This setup specifies the log group ARN, scan frequency, credential profile, and starting position for log collection.

```yaml
filebeat.inputs:
- type: aws-cloudwatch
  log_group_arn: arn:aws:logs:us-east-1:428152502467:log-group:test:*
  scan_frequency: 1m
  credential_profile_name: elastic-beats
  start_position: beginning
```

--------------------------------

### Setup and Run System Integration Tests (Python)

Source: https://github.com/elastic/beats/blob/main/docs/extend/testing.md

This snippet shows the complete workflow for setting up the Python testing environment, building necessary binaries, and running all system and integration tests. It includes commands for activating the virtual environment, building Docker Compose containers, and executing tests with specific environment variables.

```bash
# Create and activate the system test virtual environment (assumes a Unix system).
source $(mage pythonVirtualEnv)/bin/activate

# Pull and build the containers. Only needs to be done once unless you change the containers.
mage docker:composeBuild

# Bring up all containers, wait until they are healthy, and put them in the background.
mage docker:composeUp

# Build the test binary
mage buildSystemTestBinary

# Run all system and integration tests.
INTEGRATION_TESTS=1 \
BEAT_STRICT_PERMS=false \
ES_USER="admin" \
ES_PASS="testing" \
pytest ./tests/system

# Stop all started containers.
mage docker:composeDown
```

--------------------------------

### Example Log Structure for Multiline Events

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/multiline-examples.md

This is an example of application log events that span multiple lines, starting with 'Start new event' and ending with 'End event'. This structure is suitable for multiline processing.

```shell
[2015-08-24 11:49:14,389] Start new event
[2015-08-24 11:49:14,395] Content of processing something
[2015-08-24 11:49:14,399] End event
```

--------------------------------

### Load Dashboards using `setup --dashboards`

Source: https://github.com/elastic/beats/blob/main/docs/reference/metricbeat/load-kibana-dashboards.md

Run this command to load the recommended index template and sample dashboards for visualizing Metricbeat data in Kibana. Ensure Kibana is running and accessible.

```sh
metricbeat setup --dashboards
```

```sh
metricbeat setup --dashboards
```

```sh
./metricbeat setup --dashboards
```

```sh
./metricbeat setup --dashboards
```

```sh
docker run --rm --net="host" docker.elastic.co/beats/metricbeat:{{version.stack}} setup --dashboards
```

```powershell
PS > .\metricbeat.exe setup --dashboards
```

--------------------------------

### Setup Metricbeat Environment

Source: https://github.com/elastic/beats/blob/main/docs/reference/metricbeat/command-line-options.md

Use the `setup` command to initialize the Metricbeat environment, including index templates, ILM policies, and Kibana dashboards. This command runs without ingesting data.

```sh
metricbeat setup --dashboards
```

```sh
metricbeat setup --index-management
```

--------------------------------

### Build and Setup Filebeat from Source

Source: https://github.com/elastic/beats/blob/main/x-pack/filebeat/module/suricata/README.md

Commands to build Filebeat from source and set up the Suricata module. Ensure you have the correct directory and configuration file paths.

```bash
cd x-pack/filebeat
make mage
mage build update
./filebeat setup --modules=suricata -e -d "*" -c filebeat.yml -E 'setup.dashboards.directory=build/kibana'
```

--------------------------------

### Filebeat Setup Command

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/command-line-options.md

Use the `setup` command to initialize Filebeat's environment. This includes setting up index templates, ILM policies, write aliases, Kibana dashboards, and machine learning jobs. Specify optional flags to set up only a subset of these assets.

```sh
filebeat setup --dashboards
```

```sh
filebeat setup --pipelines
```

```sh
filebeat setup --pipelines --modules system,nginx,mysql
```

```sh
filebeat setup --index-management
```

--------------------------------

### Install and Run Node.js for Statsd Client

Source: https://github.com/elastic/beats/blob/main/x-pack/metricbeat/module/statsd/README.md

Install the statsd-client npm package and start a Node.js interactive session to emit metrics.

```bash
$ npm install statsd-client
$ node
```

--------------------------------

### Run Winlogbeat Setup Command

Source: https://github.com/elastic/beats/blob/main/docs/reference/winlogbeat/could-not-locate-index-pattern.md

If the index pattern is not created correctly, try running the setup command again. This command attempts to set up the index template and pattern automatically.

```bash
./winlogbeat setup
```

--------------------------------

### Start Filebeat (DEB/RPM)

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/filebeat-starting.md

Use this command to start Filebeat when installed via DEB or RPM packages. If using an init.d script, command line flags cannot be specified.

```sh
sudo service filebeat start
```

--------------------------------

### Setup Packetbeat Environment

Source: https://github.com/elastic/beats/blob/main/docs/reference/packetbeat/command-line-options.md

Use the `setup` command to initialize the Packetbeat environment, including index templates, ILM policies, and Kibana dashboards. This command does not ingest data.

```sh
packetbeat setup --dashboards
```

```sh
packetbeat setup --index-management
```

--------------------------------

### Start Auditbeat on DEB/RPM systems

Source: https://github.com/elastic/beats/blob/main/docs/reference/auditbeat/auditbeat-starting.md

Use this command to start the Auditbeat service on systems using DEB or RPM package management. Ensure Auditbeat is installed and configured.

```sh
sudo service auditbeat start
```

--------------------------------

### Setup Okta Module and Dashboards

Source: https://github.com/elastic/beats/blob/main/x-pack/filebeat/module/okta/README.md

Run this command to set up the Okta module and load its dashboards.

```bash
./filebeat setup --modules=okta -e --dashboards
```

--------------------------------

### Example External Module Configuration File

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/filebeat-configuration-reloading.md

An example of an external configuration file for Filebeat modules. Each module definition must start with `- module` and should not include the `filebeat.config.modules` line.

```yaml
- module: apache
  access:
    enabled: true
    var.paths: [/var/log/apache2/access.log*]
  error:
    enabled: true
    var.paths: [/var/log/apache2/error.log*]
```

--------------------------------

### Metricbeat Configuration Example

Source: https://github.com/elastic/beats/blob/main/docs/reference/metricbeat/metricbeat-reference-yml.md

A full configuration example documenting all non-deprecated options in comments. Useful for understanding the complete range of settings.

```yaml
## Metricbeat Configuration ###########################

# This file is a full configuration example documenting all non-deprecated
# options in comments. For a shorter configuration example, that contains only
# the most common options, please see metricbeat.yml in the same directory.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/metricbeat/index.html

#============================  Config Reloading ===============================

# Config reloading allows to dynamically load modules. Each file which is
# monitored must contain one or multiple modules as a list.
metricbeat.config.modules:

  # Glob pattern for configuration reloading
  path: ${path.config}/modules.d/*.yml

  # Period on which files under path should be checked for changes
  reload.period: 10s

  # Set to true to enable config reloading
  reload.enabled: false

# Maximum amount of time to randomly delay the start of a metricset. Use 0 to
# disable startup delay.
metricbeat.max_start_delay: 10s

#============================== Autodiscover ===================================

# Autodiscover allows you to detect changes in the system and spawn new modules
# as they happen.

#metricbeat.autodiscover:
  # List of enabled autodiscover providers
#  providers:
#    - type: docker
#      templates:
#        - condition:
#            equals.docker.container.image: etcd
#          config:
#            - module: etcd
#              metricsets: ["leader", "self", "store"]
#              period: 10s
#              hosts: ["${host}:2379"]

#=========================== Timeseries instance ===============================

# Enabling this will add a `timeseries.instance` keyword field to all metric
# events. For a given metricset, this field will be unique for every single item
# being monitored.
# This setting is experimental.

#timeseries.enabled: false


#==========================  Modules configuration =============================
metricbeat.modules:

#-------------------------------- System Module --------------------------------
- module: system
  metricsets:
    - cpu             # CPU usage
    - load            # CPU load averages
    - memory          # Memory usage
    - network         # Network IO
    - process         # Per process metrics
    - process_summary # Process summary
    - uptime          # System Uptime
    - socket_summary  # Socket summary
    #- core           # Per CPU core usage
    #- diskio         # Disk IO
    #- filesystem     # File system usage for each mountpoint
    #- fsstat         # File system summary metrics
    #- raid           # Raid
    #- socket         # Sockets and connection info (linux only)
    #- service        # systemd service information
  enabled: true
  period: 10s
  processes: ['.*']

  # Configure the mount point of the host’s filesystem for use in monitoring a host from within a container
  #hostfs: "/hostfs"

  # Configure the metric types that are included by these metricsets.
  cpu.metrics:  ["percentages","normalized_percentages"]  # The other available option is ticks.
  core.metrics: ["percentages"]  # The other available option is ticks.

  # A list of filesystem types to ignore. The filesystem metricset will not
  # collect data from filesystems matching any of the specified types, and
  # fsstats will not include data from these filesystems in its summary stats.
  # If not set, types associated to virtual filesystems are automatically
  # added when this information is available in the system (e.g. the list of
  # `nodev` types in `/proc/filesystem`).
  #filesystem.ignore_types: []

  # These options allow you to filter out all processes that are not
  # in the top N by CPU or memory, in order to reduce the number of documents created.
  # If both the `by_cpu` and `by_memory` options are used, the union of the two sets
  # is included.
  #process.include_top_n:

    # Set to false to disable this feature and include all processes
    #enabled: true

    # How many processes to include from the top by CPU. The processes are sorted
    # by the `system.process.cpu.total.pct` field.
    #by_cpu: 0

```

--------------------------------

### Example External Input Configuration File

Source: https://github.com/elastic/beats/blob/main/docs/reference/filebeat/filebeat-configuration-reloading.md

An example of an external configuration file for Filebeat inputs. Each input definition must start with `- type` and should not include the `filebeat.config.inputs` line.

```yaml
- type: filestream
  id: first
  paths:
    - /var/log/mysql.log
  prospector.scanner.check_interval: 10s

- type: filestream
  id: second
  paths:
    - /var/log/apache.log
  prospector.scanner.check_interval: 5s
```

--------------------------------

### Install Winlogbeat Index Template Manually

Source: https://github.com/elastic/beats/blob/main/docs/reference/winlogbeat/winlogbeat-template.md

Install the exported index template into Elasticsearch using Invoke-RestMethod. Ensure the specified URI and template name match your Elasticsearch setup.

```powershell
PS > Invoke-RestMethod -Method Put -ContentType "application/json" -InFile winlogbeat.template.json -Uri http://localhost:9200/_index_template/winlogbeat-{{version.stack}}
```

--------------------------------

### Build and Install Elastic Logging Plugin from Source

Source: https://github.com/elastic/beats/blob/main/docs/reference/loggingplugin/log-driver-installation.md

Builds and installs the Elastic Logging Plugin from its source code. Requires a development environment setup as per the Beats GitHub repository.

```sh
cd x-pack/dockerlogbeat
mage BuildAndInstall
```

--------------------------------

### System module configuration example

Source: https://github.com/elastic/beats/blob/main/docs/reference/metricbeat/metricbeat-module-system.md

Example of how to configure the System module in `modules.d/system.yml`. Default metricsets are listed, and commented-out metricsets can be disabled.

```yaml
metricbeat.modules:
- module: system
  period: 10s
  processes:
    enabled: true
    net: true
    process.include_top_n:
      by_cpu: 10
      by_memory: 10
  #cpu:
  #  metrics:
  #    - cpu"
  #load:
  #  memory:
  #network:
  #process:
  #process_summary:
  #socket_summary:
  #filesystem:
  #fsstat:
  #uptime:
  #core:
  #diskio:
  #socket:
  #entropy:

```

--------------------------------

### Winlogbeat Configuration Example

Source: https://github.com/elastic/beats/blob/main/docs/reference/winlogbeat/winlogbeat-reference-yml.md

This example highlights common Winlogbeat configuration options, including event log monitoring and general settings. It serves as a starting point for customizing Winlogbeat's behavior.

```yaml
## Winlogbeat Configuration Example ########################

# This file is an example configuration file highlighting only the most common
# options. The winlogbeat.reference.yml file from the same directory contains
# all the supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/winlogbeat/index.html

# ======================== Winlogbeat specific options =========================

# The registry file is where Winlogbeat persists its state so that the beat can
# resume after shutdown or an outage. The default is .winlogbeat.yml in the
# directory in which it was started.
#winlogbeat.registry_file: .winlogbeat.yml

# The timeout value that controls when registry entries are written to disk
# (flushed). When an unwritten update exceeds this value, it triggers a write
# to disk. When flush is set to 0s, the registry is written to disk after each
# batch of events has been published successfully. The default value is 5s.
#winlogbeat.registry_flush: 5s

# By default Ingest pipelines are not updated if a pipeline with the same ID
# already exists. If this option is enabled Winlogbeat overwrites pipelines
# every time a new Elasticsearch connection is established.
#winlogbeat.overwrite_pipelines: false

# event_logs specifies a list of event logs to monitor as well as any
# accompanying options. The YAML data type of event_logs is a list of
# dictionaries.
#
# The supported keys are name, id, xml_query, tags, fields, fields_under_root,
# forwarded, ignore_older, level, event_id, provider, include_xml, and 
# ignore_missing_channel. 
# The xml_query key requires an id and must not be used with the name,
# ignore_older, level, event_id, or provider keys. Please visit the
# documentation for the complete details of each option.
# https://go.es.io/WinlogbeatConfig

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h

  - name: System

  - name: Security

  - name: ForwardedEvents
    tags: [forwarded]

  - name: Windows PowerShell
    event_id: 400, 403, 600, 800

  - name: Microsoft-Windows-PowerShell/Operational
    event_id: 4103, 4104, 4105, 4106


# ================================== General ===================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
# If this option is not defined, the hostname is used.
#name:

# The tags of the shipper are included in their field with each
# transaction published. Tags make it easy to group servers by different
# logical properties.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output. Fields can be scalar values, arrays, dictionaries, or any nested
# combination of these.
#fields:
#  env: staging

# If this option is set to true, the custom fields are stored as top-level
# fields in the output document instead of being grouped under a field
# sub-dictionary. Default is false.
#fields_under_root: false

# Configure the precision of all timestamps in Winlogbeat.
# Available options: millisecond, microsecond, nanosecond
#timestamp.precision: millisecond

# Internal queue configuration for buffering events to be published.
# Queue settings may be overridden by performance presets in the
# Elasticsearch output. To configure them manually use "preset: custom".
#queue:
  # Queue type by name (default 'mem')
  # The memory queue will present all available events (up to the outputs
  # bulk_max_size) to the output, the moment the output is ready to serve
  # another batch of events.
  #mem:
    # Max number of events the queue can buffer.
    #events: 3200

    # Hints the minimum number of events stored in the queue,
    # before providing a batch of events to the outputs.
    # The default value is set to 2048.
    # A value of 0 ensures events are immediately available
    # to be sent to the outputs.
    #flush.min_events: 1600

    # Maximum duration after which events are available to the outputs,
    # if the number of events stored in the queue is < `flush.min_events`.
    #flush.timeout: 10s

```