### Logstash Initial Setup and Build Commands

Source: https://github.com/elastic/logstash/blob/main/AGENTS.md

Execute these commands from the repository root for initial setup, dependency installation, and building distribution artifacts. Ensure a JDK 17+ is available; JDK 21 is preferred.

```bash
# Initial setup (required once)
./gradlew bootstrap
```

```bash
# Install development dependencies
./gradlew installDevelopmentGems
```

```bash
# Install default plugins (80+ plugins)
./gradlew installDefaultGems
```

```bash
# Build distribution artifact (tar.gz)
./gradlew assembleTarDistribution
```

--------------------------------

### Start Logstash

Source: https://github.com/elastic/logstash/blob/main/README.md

Use this command to start Logstash after setting up the environment and installing plugins. Note that no plugins are installed by default before running `rake plugin:install-default`.

```sh
bin/logstash
```

--------------------------------

### Start Logstash and Send First Event

Source: https://github.com/elastic/logstash/blob/main/README.md

Use this command to start Logstash with stdin input and send a test event to verify the installation. The output will show the processed event.

```sh
bin/logstash -e 'input { stdin { } } output { stdout {} }'
```

```text
hello world
2016-11-11T01:22:14.405+0000 0.0.0.0 hello world
```

--------------------------------

### Example Java Version Output

Source: https://github.com/elastic/logstash/blob/main/docs/reference/getting-started-with-logstash.md

This is an example of the output you might see when checking your Java version.

```text
openjdk 21.0.10 2026-01-20 LTS
OpenJDK Runtime Environment Temurin-21.0.10+7 (build 21.0.10+7-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.10+7 (build 21.0.10+7-LTS, mixed mode, sharing)
```

--------------------------------

### Install a Logstash Plugin

Source: https://github.com/elastic/logstash/blob/main/docs/reference/working-with-plugins.md

Retrieve and install plugins from the RubyGems.org public repository using the `install` subcommand.

```sh
bin/logstash-plugin install logstash-input-github
```

--------------------------------

### Example Test Structure in Logstash Framework

Source: https://github.com/elastic/logstash/blob/main/qa/README.md

Illustrates how to structure tests using ServiceTester and shared examples within the Logstash acceptance test framework. Use this as a starting point for new tests.

```ruby
logstash = ServiceTester::Artifact.new()

## your test code goes here.

# example:
it_behaves_like "installable_with_jdk", logstash
it_behaves_like "updated", logstash, from_release_branch="7.17"
```

--------------------------------

### Example Input Plugin Output

Source: https://github.com/elastic/logstash/blob/main/docs/extend/input-new-plugin.md

This is a sample output from the example input plugin, showing the default message and timestamp.

```json
{
       "message" => "Hello World!",
      "@version" => "1",
    "@timestamp" => "2015-01-27T19:17:18.932Z",
          "host" => "cadenza"
}
```

--------------------------------

### Start Logstash with Configuration

Source: https://github.com/elastic/logstash/blob/main/docs/extend/java-output-plugin.md

Command to start Logstash with the specified configuration file. Ensure the path to the configuration file is correct.

```text
bin/logstash -f /path/to/java_output.conf
```

--------------------------------

### Test Plugin Installation

Source: https://github.com/elastic/logstash/blob/main/docs/extend/filter-new-plugin.md

Test your plugin by installing it into a clean Logstash instance. Download Logstash, extract it, and use the plugin tool to install your built gem.

```sh
curl -O https://download.elastic.co/logstash/logstash/logstash-9.0.0.tar.gz
tar xzvf logstash-9.0.0.tar.gz
cd logstash-9.0.0
```

```sh
bin/logstash-plugin install /my/logstash/plugins/logstash-filter-example/logstash-filter-example-0.1.0.gem
```

--------------------------------

### Changelog Entry Examples

Source: https://github.com/elastic/logstash/blob/main/docs/extend/community-maintainer.md

Examples of how to format changelog entries, distinguishing between user-facing changes and internal refactors.

```markdown
 - internal: Refactored specs for better testing
 - config: Default timeout configuration changed from 10s to 5s
```

--------------------------------

### Install Logstash using APT

Source: https://github.com/elastic/logstash/blob/main/docs/reference/installing-logstash.md

Updates the package list and installs Logstash after configuring the Elastic APT repository.

```sh
sudo apt-get update && sudo apt-get install logstash
```

--------------------------------

### Clone Logstash Input Plugin Example

Source: https://github.com/elastic/logstash/blob/main/docs/extend/input-new-plugin.md

Clone the example input plugin from GitHub and copy its contents to your new plugin's directory. Remember to delete the example's .git directory before copying.

```bash
git clone https://github.com/logstash-plugins/logstash-input-example.git
cd logstash-input-example
rm -rf .git
cp -R * /path/to/logstash-input-mypluginname/
```

--------------------------------

### Clone Logstash Output Plugin Example

Source: https://github.com/elastic/logstash/blob/main/docs/extend/output-new-plugin.md

Clone the example output plugin from GitHub and copy its contents to your new plugin's repository. Remember to delete the example's .git directory before copying.

```bash
git clone https://github.com/logstash-plugins/logstash-output-example.git
cd logstash-output-example
rm -rf .git
cp -R * /path/to/logstash-output-mypluginname/
```

--------------------------------

### Install Plugin Dependencies with Bundler

Source: https://github.com/elastic/logstash/blob/main/docs/extend/codec-new-plugin.md

Install your plugin's dependencies using bundler. This command should be run after cloning the plugin repository.

```bash
bundle install
```

--------------------------------

### Start Logstash Pipeline

Source: https://github.com/elastic/logstash/blob/main/docs/reference/use-filebeat-modules-kafka.md

Execute this command to start Logstash with your defined pipeline configuration file.

```shell
bin/logstash -f mypipeline.conf
```

--------------------------------

### Run Filebeat setup command

Source: https://github.com/elastic/logstash/blob/main/docs/reference/use-filebeat-modules-kafka.md

Use this command to set up the Filebeat index template and sample Kibana dashboards. The -e flag sends output to standard error.

```shell
filebeat -e setup
```

--------------------------------

### Start Logstash Service with Systemd

Source: https://github.com/elastic/logstash/blob/main/docs/reference/running-logstash.md

Use this command to start the Logstash service on systems that use systemd. This is typically done after installing the Logstash package.

```sh
sudo systemctl start logstash.service
```

--------------------------------

### Hash Configuration Example

Source: https://github.com/elastic/logstash/blob/main/docs/reference/configuration-file-structure.md

Demonstrates the 'hash' value type for key-value pairs. Entries are separated by spaces, not commas. This example shows a 'match' setting with multiple key-value pairs, both in a multi-line and single-line format.

```js
match => {
  "field1" => "value1"
  "field2" => "value2"
  ...
}
# or as a single line. No commas between entries:
match => { "field1" => "value1" "field2" => "value2" }
```

--------------------------------

### Logstash Input and Output Example

Source: https://github.com/elastic/logstash/blob/main/docs/reference/first-event.md

This is an example of how Logstash processes input. After starting Logstash with the basic pipeline, enter text at the prompt. Logstash will add a timestamp and IP address to your message.

```shell
hello world
2013-11-21T01:22:14.405+0000 0.0.0.0 hello world
```

--------------------------------

### Example Benchmark CLI Execution

Source: https://github.com/elastic/logstash/blob/main/tools/benchmark-cli/README.md

An example of running the benchmark tool with specific configurations for working directory, test case, and Logstash distribution version. The output includes ASCII art, benchmark summary, and performance metrics.

```bash
java -cp 'benchmark-cli.jar:*' org.logstash.benchmark.cli.Main --workdir=/tmp/benchmark2 --testcase=baseline --distribution-version=5.5.0
```

--------------------------------

### Logstash Comment Example

Source: https://github.com/elastic/logstash/blob/main/docs/reference/configuration-file-structure.md

Logstash supports comments starting with '#', which can appear at the beginning of a line or after code.

```logstash
# this is a comment

input { # comments can appear at the end of a line, too
  # ...
}
```

--------------------------------

### Manual Integration Test Setup

Source: https://github.com/elastic/logstash/blob/main/qa/AGENTS.md

Manually set up integration test dependencies, including gems and external services like Elasticsearch and Filebeat.

```bash
./gradlew installIntegrationTestGems
./gradlew copyEs
./gradlew copyFilebeat
```

--------------------------------

### Boolean Setting Example

Source: https://github.com/elastic/logstash/blob/main/docs/reference/configuration-file-structure.md

Provides an example of a boolean setting, 'ssl_enable', which must be set to either 'true' or 'false'. These keywords should not be enclosed in quotes.

```js
ssl_enable => true
```

--------------------------------

### Clone Logstash Example Filter Plugin

Source: https://github.com/elastic/logstash/blob/main/docs/extend/java-filter-plugin.md

Clone the example filter plugin repository to start developing your custom Java filter. Ensure you use the correct branch for the Java plugin API.

```shell
git clone --branch <branch_name> --single-branch https://github.com/logstash-plugins/logstash-filter-java_filter_example.git <target_folder>
```

--------------------------------

### Example Travis CI Configuration for Logstash Plugins

Source: https://github.com/elastic/logstash/blob/main/docs/extend/index.md

This example shows how to import the standard Logstash Travis CI configuration file. Ensure your plugin has specs in the spec folder.

```yaml
language: ruby

before_install:
  - gem install bundler

install:
  - bundle install --without development test

before_script:
  - bundle exec rspec spec/ --format documentation

script:
  - bundle exec rspec spec/ --format documentation

notifications:
  email:
    on_success: change
    on_failure: always
```

--------------------------------

### Update Logstash Plugins

Source: https://github.com/elastic/logstash/blob/main/docs/reference/working-with-plugins.md

Use the `update` subcommand to get the latest versions of installed plugins. You can update all plugins or specify a single plugin.

```sh
bin/logstash-plugin update <1>
```

```sh
bin/logstash-plugin update logstash-input-github <2>
```

--------------------------------

### Logstash Configuration with Java Output Plugin

Source: https://github.com/elastic/logstash/blob/main/docs/extend/java-output-plugin.md

A minimal Logstash configuration to test the installed Java output plugin. This example uses the generator input plugin to send a message to the custom java_output_example.

```logstash
input {
  generator { message => "Hello world!" count => 1 }
}
output {
  java_output_example {}
}
```

--------------------------------

### Changelog Entry Examples with Labels and Keywords

Source: https://github.com/elastic/logstash/blob/main/CONTRIBUTING.md

Demonstrates the standard format for changelog entries, including optional labels like [BREAKING], [DOC], [SECURITY], and keywords such as Added, Changed, Fixed. Each entry should link to the relevant pull request.

```markdown
## 4.0.0
- Changed default value of `number_of_threads` from 2 to 1 [#101](http://example.org)
- Changed default value of `execution_bugs` from 30 to 0 [#104](http://example.org)
- [BREAKING] Removed obsolete option `enable_telnet` option [#100](http://example.org)

## 3.3.3
- [DOC] Fixed incorrect formatting of code sample [#85](http://example.org)

## 3.3.2
- Fixed incorrect serialization of input data when encoding was `Emacs-Mule` [#84](http://example.org)

## 3.3.1
- Fixed memory leak by removing calls to `leak_lots_of_memory` [#86](http://example.org)
```

--------------------------------

### IRB Session for Grok Pattern Testing

Source: https://github.com/elastic/logstash/wiki/Testing-your-Grok-patterns-(--logstash-1.1.0-and-above-)

Use this IRB session to test Grok patterns. Ensure you have the 'jls-grok' gem installed. This example shows adding a custom pattern, compiling it, and matching a string.

```ruby
$ irb
require 'rubygems'
# => true
require 'grok-pure'
# => true
g = Grok.new
# => #<Grok:0x99176f @patterns={}, @logger=#<Cabin::Channel:0x1627c16 @data={}, @outputs=[#
 <Cabin::Outputs::StdlibLogger:0x666d83 @logger=#<Logger:0x9fd062 @logdev=#<Logger::LogDevice:0x29f93b 
  @shift_age=nil, @filename=nil, @dev=#<IO:0x1a998c7>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x16d4b50 
   @mon_count=0, @mon_mutex=#<Mutex:0x6b93c5>, @mon_owner=nil>, @shift_size=nil>, @formatter=nil, @progname=nil, 
   @default_formatter=#<Logger::Formatter:0xb28980 @datetime_format=nil>, @level=0>>], @level=:warn>>
g.add_pattern("foo", ".*") # pattern name, and the regex
# => nil
g.compile("%{foo}") # compile the named pattern
# => nil
str = "some string"
# => "some string"
g.match(str).captures # match str against pattern, and output the captures 
# => {"foo"=>["some string"]}
```

```ruby
g.add_patterns_from_file("/home/alcy/downloads/logstash/patterns/grok-patterns")
# => nil
g.add_patterns_from_file("/home/alcy/downloads/logstash/patterns/linux-syslog")
# => nil
g.compile("%{SYSLOGLINE}")
# => nil
sample_syslog = "May 16 12:17:47 ub1104 ntpdate[704]: step time server 91.189.94.4 offset 0.003341 sec"
# => "May 16 12:17:47 ub1104 ntpdate[704]: step time server 91.189.94.4 offset 0.003341 sec"
g.match(sample_syslog).captures
# => {"SYSLOGLINE"=>["May 16 12:17:47 ub1104 ntpdate[704]: step time server 91.189.94.4 offset 0.003341 sec"], 
 "SYSLOGBASE2"=>["May 16 12:17:47 ub1104 ntpdate[704]:"], 
 "SYSLOGTIMESTAMP:timestamp"=>["May 16 12:17:47"], "MONTH"=>["May"], "MONTHDAY"=>["16", nil], 
 "TIME"=>["12:17:47"], "HOUR"=>["12", nil, nil], "MINUTE"=>["17", nil, nil], "SECOND"=>["47", nil], 
 "TIMESTAMP_ISO8601:timestamp8601"=>[nil], "YEAR"=>[nil], "MONTHNUM"=>[nil], "ISO8601_TIMEZONE"=>[nil], 
 "SYSLOGFACILITY"=>[nil], "POSINT:facility"=>[nil], "POSINT:priority"=>[nil], "SYSLOGHOST:logsource"=>["ub1104"], 
 "IPORHOST"=>["ub1104"], "HOSTNAME"=>["ub1104"], "IP"=>[nil], "SYSLOGPROG"=>["ntpdate[704]"], "PROG:program"=>["ntpdate"], 
 "POSINT:pid"=>["704"], "GREEDYDATA:message"=>["step time server 91.189.94.4 offset 0.003341 sec"]}
```

--------------------------------

### Configure keystore for client certificate

Source: https://github.com/elastic/logstash/blob/main/docs/reference/monitoring-internal-collection-legacy.md

Optionally, set up client certificates using a keystore (a Java Keystore file) or a certificate and key file pair.

```yaml
xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
xpack.monitoring.elasticsearch.ssl.keystore.password: password
```

```yaml
xpack.monitoring.elasticsearch.ssl.certificate: /path/to/certificate
xpack.monitoring.elasticsearch.ssl.key: /path/to/key
```

--------------------------------

### Simulate a stalled shutdown with unsafe shutdown

Source: https://github.com/elastic/logstash/blob/main/docs/reference/shutdown.md

This example demonstrates a stalled shutdown scenario where a slow Ruby filter prevents clean termination. Logstash is started with `--pipeline.unsafe_shutdown`, leading to data loss.

```shell
bin/logstash -e 'input { generator { } } filter { ruby { code => "sleep 10000" } }
  output { stdout { codec => dots } }' -w 1 --pipeline.unsafe_shutdown
Pipeline main started
^CSIGINT received. Shutting down the agent. {:level=>:warn} 
stopping pipeline {:id=>"main", :level=>:warn}
Received shutdown signal, but pipeline is still waiting for in-flight events
to be processed. Sending another ^C will force quit Logstash, but this may cause
data loss. {:level=>:warn}
{"inflight_count"=>125, "stalling_thread_info"=>{["LogStash::Filters::Ruby",
{"code"=>"sleep 10000"}]=>[{"thread_id"=>19, "name"=>"[main]>worker0",
"current_call"=>"(ruby filter code):1:in `sleep'"}]}} {:level=>:warn}
The shutdown process appears to be stalled due to busy or blocked plugins.
Check the logs for more information. {:level=>:error}
{"inflight_count"=>125, "stalling_thread_info"=>{["LogStash::Filters::Ruby",
{"code"=>"sleep 10000"}]=>[{"thread_id"=>19, "name"=>"[main]>worker0",
"current_call"=>"(ruby filter code):1:in `sleep'"}]}} {:level=>:warn}
{"inflight_count"=>125, "stalling_thread_info"=>{["LogStash::Filters::Ruby",
{"code"=>"sleep 10000"}]=>[{"thread_id"=>19, "name"=>"[main]>worker0",
"current_call"=>"(ruby filter code):1:in `sleep'"}]}} {:level=>:warn}
Forcefully quitting logstash.. {:level=>:fatal}
```

--------------------------------

### Run Integration Test Directly

Source: https://github.com/elastic/logstash/blob/main/qa/AGENTS.md

After setup, navigate to the integration test directory and run a specific spec file using Bundler and JRuby.

```bash
cd qa/integration
BUNDLE_PATH=../../build/qa/integration/vendor \
  ../../vendor/jruby/bin/jruby -S bundle exec rspec specs/dlq_spec.rb
```

--------------------------------

### Enrich Events with Data from Elasticsearch

Source: https://github.com/elastic/logstash/blob/main/docs/reference/lookup-enrichment.md

The Elasticsearch filter retrieves data from previous log events in Elasticsearch to enrich current events. This example finds a 'start' event based on an operation ID and copies its timestamp to the 'end' event for duration calculation.

```json
if [type] == "end" {
   elasticsearch {
      hosts => ["es-server"]
      query => "type:start AND operation:%{[opid]}"
      fields => { "@timestamp" => "started" }
   }
   date {
      match => ["[started]", "ISO8601"]
      target => "[started]"
   }
   ruby {
      code => 'event.set("duration_hrs", (event.get("@timestamp") - event.get("started")) / 3600) rescue nil'
  }
}
```

--------------------------------

### Run Benchmark CLI

Source: https://github.com/elastic/logstash/blob/main/tools/benchmark-cli/README.md

Execute the benchmark tool using the `java` command. The classpath should include the benchmark jar and any necessary libraries. Use the `--help` flag to see all available options.

```bash
java -cp 'benchmark-cli.jar:*' org.logstash.benchmark.cli.Main --help
```

--------------------------------

### Install Plugin Gem

Source: https://github.com/elastic/logstash/blob/main/lib/pluginmanager/templates/codec-plugin/README.md

Install the built plugin gem into an installed Logstash instance.

```sh
bin/logstash-plugin install /your/local/plugin/logstash-codec-awesome.gem
```

--------------------------------

### Package All Filter Plugins for Offline Use

Source: https://github.com/elastic/logstash/blob/main/docs/reference/offline-plugins.md

Example using a wildcard to package all filter plugins and their dependencies for offline use.

```shell
bin/logstash-plugin prepare-offline-pack logstash-filter-* <2>
```

--------------------------------

### Install Published Plugin

Source: https://github.com/elastic/logstash/blob/main/docs/extend/input-new-plugin.md

After publishing, Logstash users can install your plugin from RubyGems.org using the plugin install command.

```sh
bin/logstash-plugin install logstash-input-mypluginname
```

--------------------------------

### Logstash Configuration Example

Source: https://github.com/elastic/logstash/wiki/Cool-links-to-help-with-Logstash-and-The-Elk-Stack

Sample configuration for Logstash. Refer to the linked article for a detailed dissection.

```logstash
input {
  file {
    path => "/var/log/myapp.log"
  }
}

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
}
```

--------------------------------

### Install Plugin Gem into Logstash

Source: https://github.com/elastic/logstash/blob/main/lib/pluginmanager/templates/filter-plugin/README.md

Install a built plugin gem into an existing Logstash installation. This method is used when not running Logstash from a local clone.

```sh
bin/logstash-plugin install /your/local/plugin/logstash-filter-awesome.gem
```

--------------------------------

### Complete first-pipeline.conf Example

Source: https://github.com/elastic/logstash/blob/main/docs/reference/advanced-pipeline.md

This is the complete configuration for the first Logstash pipeline, including Beats input and stdout output.

```logstash
input {
    beats {
        port => "5044"
    }
}
# The filter part of this file is commented out to indicate that it is
# optional.
# filter {
#
# }
output {
    stdout {
        codec => rubydebug
    }
}
```

--------------------------------

### List Installed Logstash Plugins

Source: https://github.com/elastic/logstash/blob/main/docs/extend/codec-new-plugin.md

Use the Logstash plugin tool to list all currently installed plugins. This can help in verifying your plugin's installation.

```bash
bin/logstash-plugin list
```

--------------------------------

### Install Logstash Dependencies with Bundler

Source: https://github.com/elastic/logstash/blob/main/README.md

Use this command to install development dependencies via Bundler. Ensure you have Bundler installed. This is an alternative to using Gradle.

```sh
bundle config set --local path vendor/bundle
bundle install
```

--------------------------------

### Install Plugin Development Dependencies

Source: https://github.com/elastic/logstash/blob/main/README.md

After installing a plugin using the plugin manager, run this command to install its development dependencies. This is crucial for running plugin tests.

```sh
bin/logstash-plugin install --development
```

--------------------------------

### Start Logstash

Source: https://github.com/elastic/logstash/blob/main/docs/reference/multiple-input-output-plugins.md

Run Logstash with your configuration file to begin processing and outputting data.

```shell
bin/logstash -f second-pipeline.conf
```

--------------------------------

### Package Multiple Plugins for Offline Use

Source: https://github.com/elastic/logstash/blob/main/docs/reference/offline-plugins.md

Example of packaging a mix of plugins, including all filter plugins and the Beats input plugin, along with their dependencies for offline use.

```shell
bin/logstash-plugin prepare-offline-pack logstash-filter-* logstash-input-beats <3>
```

--------------------------------

### Test Plugin Installation in Logstash

Source: https://github.com/elastic/logstash/blob/main/docs/extend/codec-new-plugin.md

Install your built plugin gem into a clean Logstash installation using the Logstash plugin tool. Ensure paths and version numbers are correct.

```bash
bin/logstash-plugin install /my/logstash/plugins/logstash-codec-example/logstash-codec-example-0.1.0.gem
```

--------------------------------

### Install Logstash using YUM

Source: https://github.com/elastic/logstash/blob/main/docs/reference/installing-logstash.md

Installs Logstash after the Elastic YUM repository has been configured.

```sh
sudo yum install logstash
```

--------------------------------

### Run All Integration Tests

Source: https://github.com/elastic/logstash/blob/main/qa/AGENTS.md

Execute all integration tests using Gradle. This command handles all necessary setup automatically.

```bash
./gradlew runIntegrationTests
```

--------------------------------

### Install Logstash Development Gems

Source: https://github.com/elastic/logstash/blob/main/README.md

Use Gradle to install development dependencies for Logstash.

```sh
./gradlew installDevelopmentGems
```