### Start Dsiem, ELK, and Suricata with Docker Compose

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Pulls the latest Docker images for the Dsiem components and then starts the ELK stack, Suricata, and Dsiem services in standalone mode. This command brings up the entire testing environment.

```shell
docker-compose pull && \
    docker-compose up
```

--------------------------------

### Install and Import Afero

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Instructions for installing the Afero library using `go get` and how to import it into your Go project for use.

```bash
go get github.com/spf13/afero
```

```go
import "github.com/spf13/afero"
```

--------------------------------

### Getting Started: Creating an Elasticsearch Client

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/olivere/elastic/v7/README.md

Provides a basic example of how to initialize an Elasticsearch client using the olivere/elastic library. The client connects to Elasticsearch on `http://127.0.0.1:9200` by default and is typically instantiated once per application.

```go
import (
	"fmt"
	"log"

	"github.com/olivere/elastic/v7"
)

func main() {
	// Create a new client that connects to the default Elasticsearch instance
	client, err := elastic.NewClient()
	if err != nil {
		log.Fatalf("Failed to create client: %s", err)
	}

	// Ping the Elasticsearch cluster to get information about the cluster.
	info, code, err := client.Ping().Index("myindex").Do(context.Background())
	if err != nil {
		panic(err)
	}

	if code == 200 {
		fmt.Println("Connected to Elasticsearch")
		fmt.Printf("Cluster name: %s\n", info.ClusterName)
	} else {
		fmt.Printf("Connection failed with status code: %d\n", code)
	}
}
```

--------------------------------

### Install Dsiem Server Binary (Linux)

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Downloads and installs the Dsiem server binary for Linux AMD64 architecture into a specified directory. This script requires root privileges to execute.

```shell
# [ "$EUID" -ne 0 ] && echo must be run as root! || (\
export DSIEM_DIR=/var/dsiem && \
mkdir -p $DSIEM_DIR && \
wget https://github.com/defenxor/dsiem/releases/latest/download/dsiem-server_linux_amd64.zip -O /tmp/dsiem.zip && \
unzip /tmp/dsiem.zip -d $DSIEM_DIR && rm -rf /tmp/dsiem.zip  && \
cd $DSIEM_DIR

)
```

--------------------------------

### Basic fasthttprouter Usage

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/buaazp/fasthttprouter/README.md

Demonstrates the basic setup of fasthttprouter, including defining routes for the root path and a path with a named parameter, and starting the HTTP server.

```go
package main

import (
	"fmt"
	"log"

	"github.com/buaazp/fasthttprouter"
	"github.com/valyala/fasthttp"
)

func Index(ctx *fasthttp.RequestCtx) {
	fmt.Fprint(ctx, "Welcome!\n")
}

func Hello(ctx *fasthttp.RequestCtx) {
	fmt.Fprintf(ctx, "hello, %s!\n", ctx.UserValue("name"))
}

func main() {
	router := fasthttprouter.New()
	router.GET("/", Index)
	router.GET("/hello/:name", Hello)

	log.Fatal(fasthttp.ListenAndServe(":8080", router.Handler))
}
```

--------------------------------

### Install mapstructure Go Library

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/mitchellh/mapstructure/README.md

Standard Go installation command using 'go get'. This command fetches and installs the mapstructure library and its dependencies into your Go workspace.

```bash
go get github.com/mitchellh/mapstructure
```

--------------------------------

### Install NATS Go Client

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/nats-io/nats.go/README.md

Installs the NATS Go client library using the go get command. This is the standard method for obtaining Go packages.

```bash
go get github.com/nats-io/nats.go/
```

--------------------------------

### Unzip and Navigate Dsiem Repository

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Downloads and extracts the Dsiem project archive, then navigates into the extracted directory. This is the initial step for setting up the Dsiem environment.

```shell
unzip dsiem-master.zip && cd dsiem-master
```

--------------------------------

### Install and Upgrade Go Package

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/magiconair/properties/README.md

Provides the command to install or update the properties Go library using the go get command. This ensures you have the latest version or can install it for the first time.

```shell
go get -u github.com/magiconair/properties
```

--------------------------------

### Install NATS Go Client with Modules

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/nats-io/nats.go/README.md

Installs the NATS Go client library using Go modules, allowing for specific version pinning or fetching the latest version. Also shows how to get the NATS server.

```bash
# Go client latest or explicit version
go get github.com/nats-io/nats.go/@latest
go get github.com/nats-io/nats.go/@v1.13.0

# For latest NATS Server, add /v2 at the end
go get github.com/nats-io/nats-server/v2

# NATS Server v1 is installed otherwise
# go get github.com/nats-io/nats-server
```

--------------------------------

### Run Dsiem Demo Setup Script

Source: https://github.com/defenxor/dsiem/blob/master/demo/README.md

This script automates the setup and launch of the Dsiem demo environment. It prompts the user for network interface and hostname/IP details, then executes docker-compose to bring up all services. It also performs necessary initializations like setting file ownership and configuring indices.

```shell
#!/bin/bash

# ... (script content from the example) ...

# Example usage:
# cd demo/
# ./run.sh
```

--------------------------------

### Configure Dsiem Systemd Service

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Sets up Dsiem to auto-start on a systemd-based Linux system. This involves creating a service unit file and enabling/starting the service.

```shell
# [ "$EUID" -ne 0 ] && echo must be run as root! || ( \
cat <<EOF > /etc/systemd/system/dsiem.service 
[Unit]
Description=Dsiem
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/dsiem
ExecStart=/var/dsiem/dsiem serve
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload && \
systemctl enable dsiem.service && \
systemctl start dsiem.service && \
systemctl status dsiem.service)

```

--------------------------------

### Install plist Go Package

Source: https://github.com/defenxor/dsiem/blob/master/vendor/howett.net/plist/README.md

Installs the plist Go package using the go get command. This command fetches and installs the specified package and its dependencies into your Go workspace.

```shell
$ go get howett.net/plist
```

--------------------------------

### Build and Setup Filebeat Module

Source: https://github.com/defenxor/dsiem/blob/master/demo/docker/conf/filebeat-es/module/suricata/README.md

Instructions to build Filebeat from source and set up the Suricata module. This involves navigating to the Filebeat directory, running build commands, and executing the Filebeat setup command with specific flags.

```bash
cd x-pack/filebeat
make mage
mage build update
./filebeat setup --modules=suricata -e -d "*" -c filebeat.yml -E 'setup.dashboards.directory=build/kibana'
```

--------------------------------

### Start Suricata Filebeat Module

Source: https://github.com/defenxor/dsiem/blob/master/demo/docker/conf/filebeat-es/module/suricata/README.md

Command to start the Filebeat service with the Suricata module enabled. This command includes verbose logging and specifies the configuration file.

```bash
./filebeat --modules=suricata -e -d "*" -c filebeat.yml
```

--------------------------------

### Install and Use easyjson: Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/mailru/easyjson/README.md

This snippet shows how to install the easyjson tool using go get and then run it to generate Go code for JSON marshaling and unmarshaling. It requires a Go build environment and GOPATH.

```sh
# install
go get -u github.com/mailru/easyjson/...

# run
easyjson -all <file>.go
```

--------------------------------

### Start Suricata Service

Source: https://github.com/defenxor/dsiem/blob/master/demo/docker/conf/filebeat-es/module/suricata/README.md

Command to start the Suricata service, specifying network interfaces to monitor. Multiple interfaces can be monitored by adding more -i flags.

```bash
sudo suricata -i en0 # optionally more -i en1 -i en2...
```

--------------------------------

### Install Cobra CLI

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/cobra/README.md

Installs the latest version of the Cobra CLI generator and library using the go get command. This includes the library and its dependencies.

```bash
go get -u github.com/spf13/cobra/cobra
```

--------------------------------

### View pflag Package Documentation with Local godoc Server

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/pflag/README.md

This snippet demonstrates how to use Go's built-in `godoc` tool to serve package documentation locally. It allows you to browse the `pflag` package documentation, including its API, usage, and examples, by starting an HTTP server on port 6060.

```bash
godoc -http=:6060
```

--------------------------------

### Install s2 Command-line Tools

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/klauspost/compress/s2/README.md

Installs the s2c (compress) and s2d (decompress) command-line tools using Go. Ensure Go is installed and configured.

```shell
go install github.com/klauspost/compress/s2/cmd/s2c@latest
go install github.com/klauspost/compress/s2/cmd/s2d@latest
```

--------------------------------

### Install HighwayHash Go Package

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/minio/highwayhash/README.md

Installs the HighwayHash Go package using the go get command. This is the standard method for obtaining Go dependencies.

```go
go get -u github.com/minio/highwayhash
```

--------------------------------

### Install Elastic Go Client

Source: https://github.com/defenxor/dsiem/blob/master/vendor/gopkg.in/olivere/elastic.v5/README.md

Installs the Elastic Go client library using the `go get` command. This command fetches the specified version of the library and makes it available for use in your Go projects.

```sh
go get gopkg.in/olivere/elastic.v5
```

--------------------------------

### Install GoCSV Package

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/gocarina/gocsv/v2/README.md

Instructions for installing the GoCSV package using the go get command. This ensures you have the latest version available for use in your Go projects.

```go
go get -u github.com/gocarina/gocsv
```

--------------------------------

### Install and Use tomll CLI Tool

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/pelletier/go-toml/README.md

Provides instructions for installing the `tomll` command-line tool, which is used to read and lint TOML files. It also shows how to access its help message.

```shell
go install github.com/pelletier/go-toml/cmd/tomll
tomll --help
```

--------------------------------

### Install and Use jsontoml CLI Tool

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/pelletier/go-toml/README.md

Explains how to install the `jsontoml` command-line tool, which converts JSON files into TOML format. It also includes a command to display the tool's help.

```shell
go install github.com/pelletier/go-toml/cmd/jsontoml
jsontoml --help
```

--------------------------------

### Install pkg/errors

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/pkg/errors/README.md

Command to install the pkg/errors library using Go modules.

```bash
go get github.com/pkg/errors
```

--------------------------------

### Go TSV Parser Quickstart Example

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/dogenzaka/tsv/README.md

Demonstrates how to use the TSV parser to read data from a file and populate a Go struct based on field order. It shows opening a file, creating a parser, and iterating through rows.

```go
import (
    "fmt"
    "os"
    "testing"
    )

type TestRow struct {
  Name   string // 0
  Age    int    // 1
  Gender string // 2
  Active bool   // 3
}

func main() {

  file, _ := os.Open("example.tsv")
  defer file.Close()

  data := TestRow{}
  parser, _ := NewParser(file, &data)

  for {
    eof, err := parser.Next()
    if eof {
      return
    }
    if err != nil {
      panic(err)
    }
    fmt.Println(data)
  }

}
```

--------------------------------

### Install and Use tomljson CLI Tool

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/pelletier/go-toml/README.md

Details the installation of the `tomljson` command-line tool, used for converting TOML files to their JSON representation. It also shows how to view its help.

```shell
go install github.com/pelletier/go-toml/cmd/tomljson
tomljson --help
```

--------------------------------

### Start and Access Dsiem VM

Source: https://github.com/defenxor/dsiem/blob/master/demo/vagrant/README.md

Start the Vagrant virtual machine and connect to it via SSH to log in as the demo user. This command assumes you have already built the VM and are in the correct directory (alpine or ubuntu).

```shell
cd demo/vagrant/alpine    # or ubuntu, depending on which VM was built
vagrant up && vagrant ssh -c 'su - demo'
```

--------------------------------

### Install fsnotify with Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/fsnotify/fsnotify/CONTRIBUTING.md

Installs the fsnotify library using the Go toolchain. This command fetches the latest version of the fsnotify package from GitHub, making it available for use in your Go projects. Ensure you have Go installed and configured correctly.

```go
go get -u github.com/fsnotify/fsnotify
```

--------------------------------

### Install fasthttp

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/valyala/fasthttp/README.md

Provides the command to install the fasthttp library using Go modules. This is the standard method for adding external dependencies to a Go project.

```Go
go get -u github.com/valyala/fasthttp
```

--------------------------------

### Check Go and NPM Versions

Source: https://github.com/defenxor/dsiem/blob/master/docs/building.md

Verifies that Go and NPM are installed and accessible in the system's PATH. These are essential dependencies for building the project.

```shell
$ go version
$ npm -v
```

--------------------------------

### Install Zap Dependencies

Source: https://github.com/defenxor/dsiem/blob/master/vendor/go.uber.org/zap/CONTRIBUTING.md

Command to install all necessary dependencies for the zap project using the provided Makefile.

```bash
make dependencies
```

--------------------------------

### CopyOnWriteFs Example

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Demonstrates setting up a CopyOnWriteFs, which uses a read-only base filesystem and a writable overlay. Modifications to files only in the base layer copy them to the overlay first.

```go
base := afero.NewOsFs()
roBase := afero.NewReadOnlyFs(base)
ufs := afero.NewCopyOnWriteFs(roBase, afero.NewMemMapFs())

fh, _ := ufs.Create("/home/test/file2.txt")
fh.WriteString("This is a test")
fh.Close()
```

--------------------------------

### Go SizedWaitGroup Example

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/remeh/sizedwaitgroup/README.md

Demonstrates the typical use case of SizedWaitGroup to execute 50 queries concurrently while limiting the number of active goroutines to 8, preventing database overload. Includes setup, goroutine launching, and waiting for completion.

```go
package main

import (
        "fmt"
        "math/rand"
        "time"

        "github.com/remeh/sizedwaitgroup"
)

func main() {
        rand.Seed(time.Now().UnixNano())

        // Typical use-case:
        // 50 queries must be executed as quick as possible
        // but without overloading the database, so only
        // 8 routines should be started concurrently.
        swg := sizedwaitgroup.New(8)
        for i := 0; i < 50; i++ {
                swg.Add()
                go func(i int) {
                        defer swg.Done()
                        query(i)
                }(i)
        }

        swg.Wait()
}

func query(i int) {
        fmt.Println(i)
        ms := i + 500 + rand.Intn(500)
        time.Sleep(time.Duration(ms) * time.Millisecond)
}
```

--------------------------------

### fasthttp Connection Hijacking Example

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/valyala/fasthttp/README.md

Provides a comparison point for connection hijacking between net/http and fasthttp, referencing the specific methods available in each library.

```go
Compare [net/http connection hijacking](https://golang.org/pkg/net/http/#Hijacker)
    to [fasthttp connection hijacking](https://godoc.org/github.com/valyala/fasthttp#RequestCtx.Hijack).
```

--------------------------------

### Use go-toml Docker Image

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/pelletier/go-toml/README.md

Demonstrates how to use the go-toml tools via a Docker image. This example shows mounting a local directory and running `tomljson` on a file within that directory.

```shell
docker run -v $PWD:/workdir pelletier/go-toml tomljson /workdir/example.toml
```

--------------------------------

### Update DSIEM Demo Environment

Source: https://github.com/defenxor/dsiem/blob/master/demo/README.md

Steps to update the DSIEM demo environment to the latest version. This involves navigating to the demo directory, pulling the latest code changes, and updating Docker images.

```shell
cd dsiem/demo
git pull
./run.sh pull
```

--------------------------------

### Install YAML Go Package

Source: https://github.com/defenxor/dsiem/blob/master/vendor/gopkg.in/yaml.v2/README.md

Command to install the `gopkg.in/yaml.v2` package using Go's module system.

```shell
go get gopkg.in/yaml.v2
```

--------------------------------

### Quick Start: Logger in Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/go.uber.org/zap/README.md

Demonstrates using the Logger for high-performance, type-safe structured logging. It allocates less and is ideal for hot paths where performance is critical.

```go
logger, _ := zap.NewProduction()
def logger.Sync()
logger.Info("failed to fetch URL",
  // Structured context as strongly typed Field values.
  zap.String("url", url),
  zap.Int("attempt", 3),
  zap.Duration("backoff", time.Second),
)
```

--------------------------------

### RegexpFs Example

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Demonstrates creating a filtered filesystem view using RegexpFs. Files not matching the provided regular expression are treated as non-existing and cannot be created.

```go
fs := afero.NewRegexpFs(afero.NewMemMapFs(), regexp.MustCompile(`\.txt$`))
_, err := fs.Create("/file.html")
// err = syscall.ENOENT
```

--------------------------------

### HttpFs Setup

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Shows how to use Afero's http compatible backend to wrap existing filesystems for use with an HTTP server. It requires a specific Open implementation returning http.File.

```go
httpFs := afero.NewHttpFs(<ExistingFS>)
fileserver := http.FileServer(httpFs.Dir(<PATH>)))
http.Handle("/", fileserver)
```

--------------------------------

### Install fastjson Go Package

Source: https://github.com/defenxor/dsiem/blob/master/vendor/go.elastic.co/fastjson/README.md

Installs the fastjson Go package and its dependencies using the go get command.

```bash
go get -u go.elastic.co/fastjson/...
```

--------------------------------

### Fasthttp Websocket Chat Handler Setup

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/fasthttp-contrib/websocket/README.md

Demonstrates setting up a websocket server using the fasthttp-contrib/websocket package. It shows how to define a message handler and integrate it with a fasthttp request handler for real-time communication.

```go
import (
	"github.com/fasthttp-contrib/websocket"
	"github.com/vayala/fasthttp"
)

func chat(c *websocket.Conn) {
	// defer c.Close()
	// mt, message, err := c.ReadMessage()
	// c.WriteMessage(mt, message)
}

var upgrader = websocket.New(chat) // use default options
//var upgrader = websocket.Custom(chat, 1024, 1024) // customized options, read and write buffer sizes (int). Default: 4096
// var upgrader = websocket.New(chat).DontCheckOrigin() // it's useful when you have the websocket server on a different machine

func myChatHandler(ctx *fasthttp.RequestCtx) {
	err := upgrader.Upgrade(ctx)// returns only error, executes the handler you defined on the websocket.New before (the 'chat' function)
}

func main() {
	fasthttp.ListenAndServe(":8080", myChatHandler)
}
```

--------------------------------

### Install Suricata on macOS

Source: https://github.com/defenxor/dsiem/blob/master/demo/docker/conf/filebeat-es/module/suricata/README.md

Command to install Suricata on macOS using Homebrew, ensuring the jansson library is included for JSON output.

```bash
brew install suricata --with-jansson
```

--------------------------------

### Enable Basic CPU Profiling in Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/pkg/profile/README.md

Demonstrates the simplest way to enable CPU profiling in a Go application using the `profile` package. It requires the `github.com/pkg/profile` library and automatically writes profiling data upon program exit.

```Go
import "github.com/pkg/profile"

func main() {
    defer profile.Start().Stop()
    // Your application logic here
}
```

--------------------------------

### Read Configuration from io.Reader in Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/viper/README.md

Demonstrates how to read configuration from an io.Reader using Viper. It shows setting the configuration type and loading data from a byte buffer, with an example of retrieving a value.

```go
viper.SetConfigType("yaml") // or viper.SetConfigType("YAML")

// any approach to require this configuration into your program.
var yamlExample = []byte(`
Hacker: true
name: steve
hobbies:
- skateboarding
- snowboarding
- go
clothing:
  jacket: leather
  trousers: denim
age: 35
eyes : brown
beard: true
`)

vper.ReadConfig(bytes.NewBuffer(yamlExample))

vper.Get("name") // this would be "steve"
```

--------------------------------

### Dsiem Serve Command Flags and Usage

Source: https://github.com/defenxor/dsiem/blob/master/docs/commands.md

Details the 'dsiem serve' command, its purpose in standalone or clustered deployments, and lists all available startup flags with their descriptions, types, and default values. Includes examples of basic and customized command execution.

```APIDOC
dsiem serve
  Start dsiem server in a standalone or clustered deployment mode (either as frontend or backend).

  Frontends listen for normalized events from logstash and distribute them to backends through NATS message queue. Frontends also serve incoming requests for configuration management from web UI.
  Backends receive events on the message queue channel, perform correlation based on configured directive rules, and then send results/alarms to elasticsearch through local filebeat.
  Standalone mode performs both frontend and backend functions in a single dsiem instance directly, without the need for external message queue.

  Usage:
    dsiem serve [flags]

  Flags:
    -a, --address string         IP address for the HTTP server to listen on (default "0.0.0.0")
        --apm                    Enable elastic APM instrumentation
    -c, --cacheDuration int      Cache expiration time in minutes for intel and vuln query results (default 10)
        --frontend string        Frontend URL to pull configuration from, e.g. http://frontend:8080 (used only by backends).
    -h, --help                   help for serve
    -n, --holdDuration int       Duration in seconds before resetting overload condition state (default 10)
    -d, --maxDelay int           Max. processing delay in seconds before throttling incoming events (default 180)
    -e, --maxEPS int             Max. number of incoming events/second (default 1000)
        --medRiskMax int         Maximum alarm risk value to be classified as Medium risk. Higher value than this will be classified as High risk (default 6)
        --medRiskMin int         Minimum alarm risk value to be classified as Medium risk. Lower value than this will be classified as Low risk (default 3)
    -l, --minAlarmLifetime int   Min. alarm lifetime in minutes. Backlog won't expire sooner than this regardless rule timeouts. This is to support processing of delayed events
    -i, --minEPS int             Min. events/second rate allowed when throttling incoming events (default 100)
    -m, --mode string            Deployment mode, can be set to standalone, cluster-frontend, or cluster-backend (default "standalone")
        --msq string             Nats address to use for frontend - backend communication. (default "nats://dsiem-nats:4222")
        --node string            Unique node name to use when deployed in cluster mode.
    -p, --port int               TCP port for the HTTP server to listen on (default 8080)
        --pprof                  Enable go pprof on the web interface
    -s, --status strings         Alarm status to use, the first one will be assigned to new alarms (default [Open,In-Progress,Closed])
    -t, --tags strings           Alarm tags to use, the first one will be assigned to new alarms (default [Identified Threat,False Positive,Valid Threat,Security Incident])
        --trace                  Generate 10 seconds trace file for debugging.
        --websocket              Enable websocket endpoint that streams events/second measurement data
        --writeableConfig        Whether to allow configuration file update through HTTP

Global Flags:
        --debug   Enable debug messages for tracing and troubleshooting
        --dev     Enable development environment specific setting
```

```shell
$ ./dsiem serve
$ ./dsiem serve -e 5000
```

--------------------------------

### Initialize Procfs and Get Stats (Go)

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/prometheus/procfs/README.md

Demonstrates how to initialize the proc filesystem and retrieve general system statistics. It requires the path to the /proc mount point and returns system statistics or an error.

```Go
fs, err := procfs.NewFS("/proc")
stats, err := fs.Stat()
```

--------------------------------

### Install Elastic APM Agent for Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/go.elastic.co/apm/README.md

Installs the Elastic APM agent for Go using the go get command. This command fetches and installs the latest version of the package, making it available for use in your Go projects.

```bash
go get -u go.elastic.co/apm
```

--------------------------------

### Calling Afero Utilities Directly

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Demonstrates how to use Afero utility functions by passing the filesystem instance as the first argument to each function.

```go
fs := new(afero.MemMapFs)
f, err := afero.TempFile(fs,"", "ioutil-test")
```

--------------------------------

### Uninstall Dsiem with Docker Compose

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Stops and removes Dsiem containers and associated volumes when installed using Docker Compose. Supports default or cluster configurations.

```shell
$ cd dsiem/deployments/docker && \
docker-compose down -v
```

```shell
$ cd dsiem/deployments/docker && \
docker-compose -f docker-compose-cluster.yml down -v
```

--------------------------------

### Basic HttpRouter Usage in Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/julienschmidt/httprouter/README.md

Demonstrates how to set up a basic HTTP server using HttpRouter. It includes defining handlers for different routes, including one with a path parameter, and starting the HTTP server.

```Go
package main

import (
    "fmt"
    "net/http"
    "log"

    "github.com/julienschmidt/httprouter"
)

func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    fmt.Fprint(w, "Welcome!\n")
}

func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
    fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}

func main() {
    router := httprouter.New()
    router.GET("/", Index)
    router.GET("/hello/:name", Hello)

    log.Fatal(http.ListenAndServe(":8080", router))
}
```

--------------------------------

### Create Cobra CLI App with Subcommands

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/cobra/README.md

Demonstrates creating a basic CLI application using the Cobra library in Go. It includes defining root and subcommands, setting arguments, and adding flags for user input.

```go
package main

import (
  "fmt"
  "strings"

  "github.com/spf13/cobra"
)

func main() {
  var echoTimes int

  var cmdPrint = &cobra.Command{
    Use:   "print [string to print]",
    Short: "Print anything to the screen",
    Long: `print is for printing anything back to the screen.
For many years people have printed back to the screen.`,
    Args: cobra.MinimumNArgs(1),
    Run: func(cmd *cobra.Command, args []string) {
      fmt.Println("Print: " + strings.Join(args, " "))
    },
  }

  var cmdEcho = &cobra.Command{
    Use:   "echo [string to echo]",
    Short: "Echo anything to the screen",
    Long: `echo is for echoing anything back.
Echo works a lot like print, except it has a child command.`,
    Args: cobra.MinimumNArgs(1),
    Run: func(cmd *cobra.Command, args []string) {
      fmt.Println("Print: " + strings.Join(args, " "))
    },
  }

  var cmdTimes = &cobra.Command{
    Use:   "times [string to echo]",
    Short: "Echo anything to the screen more times",
    Long: `echo things multiple times back to the user by providing
a count and a string.`,
    Args: cobra.MinimumNArgs(1),
    Run: func(cmd *cobra.Command, args []string) {
      for i := 0; i < echoTimes; i++ {
        fmt.Println("Echo: " + strings.Join(args, " "))
      }
    },
  }

  cmdTimes.Flags().IntVarP(&echoTimes, "times", "t", 1, "times to echo the input")

  var rootCmd = &cobra.Command{Use: "app"}
  rootCmd.AddCommand(cmdPrint, cmdEcho)
  cmdEcho.AddCommand(cmdTimes)
  rootCmd.Execute()
}

```

--------------------------------

### Install TSV Parser for Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/dogenzaka/tsv/README.md

Installs the TSV parser library for Go using the go get command. This is the standard way to add external Go packages to your project.

```bash
go get github.com/dogenzaka/tsv
```

--------------------------------

### Initialize BigCache with Default Configuration (Go)

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/allegro/bigcache/README.md

Demonstrates the simplest way to initialize BigCache using default settings. It creates a cache with a 10-minute life window for entries and shows basic usage of setting and retrieving data.

```go
import "github.com/allegro/bigcache"

cache, _ := bigcache.NewBigCache(bigcache.DefaultConfig(10 * time.Minute))

cache.Set("my-unique-key", []byte("value"))

entry, _ := cache.Get("my-unique-key")
fmt.Println(string(entry))
```

--------------------------------

### Import Kibana Dashboard

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Imports a Kibana dashboard JSON file, which also installs associated Kibana index-patterns. Supports passing Elasticsearch credentials via environment variables for authentication.

```shell
$ ./scripts/kbndashboard-import.sh ${your-kibana-IP-or-hostname} ./deployments/kibana/dashboard-siem.json
```

```shell
$ export ES_USERNAME=elastic
$ export ES_PASSWORD=weak
$ ./scripts/kbndashboard-import.sh ${your-kibana-IP-or-hostname} ./deployments/kibana/dashboard-siem.json
```

--------------------------------

### Install zap Logging Library

Source: https://github.com/defenxor/dsiem/blob/master/vendor/go.uber.org/zap/README.md

Installs the latest version of the zap logging library for Go using the go get command. Note that zap only supports the two most recent minor versions of Go.

```go
go get -u go.uber.org/zap
```

--------------------------------

### Loading Properties Files in Go

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/magiconair/properties/README.md

Demonstrates various methods to load properties from files, URLs, maps, strings, and command-line flags using the properties library. It covers initialization from single or multiple files and URLs, including environment variable expansion in paths.

```go
import (
	"flag"
	"log"
	"time"
	"github.com/magiconair/properties"
)

func main() {
	// init from a file
	p := properties.MustLoadFile("${HOME}/config.properties", properties.UTF8)

	// or multiple files
	p = properties.MustLoadFiles([]string{
			"${HOME}/config.properties",
			"${HOME}/config-${USER}.properties",
		}, properties.UTF8, true)

	// or from a map
	p = properties.LoadMap(map[string]string{"key": "value", "abc": "def"})

	// or from a string
	p = properties.MustLoadString("key=value\nabc=def")

	// or from a URL
	p = properties.MustLoadURL("http://host/path")

	// or from multiple URLs
	p = properties.MustLoadURL([]string{
			"http://host/config",
			"http://host/config-${USER}",
		}, true)

	// or from flags
	p.MustFlag(flag.CommandLine)

	// get values through getters
	host := p.MustGetString("host")
	port := p.GetInt("port", 8080)

	// or through Decode
	type Config struct {
		Host    string        `properties:"host"`
		Port    int           `properties:"port,default=9000"`
		Accept  []string      `properties:"accept,default=image/png;image;gif"`
		Timeout time.Duration `properties:"timeout,default=5s"`
	}
	var cfg Config
	if err := p.Decode(&cfg); err != nil {
		log.Fatal(err)
	}
}
```

--------------------------------

### Afero Testing with MemMapFs

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Shows a common testing pattern using Afero's MemMapFs for a clean, fast, and reproducible file system state. This example tests file existence.

```go
func TestExist(t *testing.T) {
	appFS := afero.NewMemMapFs()
	// create test files and directories
	appFS.MkdirAll("src/a", 0755)
	afero.WriteFile(appFS, "src/a/b", []byte("file b"), 0644)
	afero.WriteFile(appFS, "src/c", []byte("file c"), 0644)
	name := "src/c"
	_, err := appFS.Stat(name)
	if os.IsNotExist(err) {
		t.Errorf("file \"%s\" does not exist.\n", name)
	}
}
```

--------------------------------

### Crypt CLI - Get Plaintext Configuration

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/viper/README.md

Example of using the 'crypt' command-line tool to retrieve a configuration value from a key/value store in plaintext.

```bash
$ crypt get -plaintext /config/hugo.json
```

--------------------------------

### Afero OsFs Backend

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Demonstrates the usage of Afero's OsFs backend, which wraps native OS file system calls, making it easy to use the actual file system.

```go
appfs := afero.NewOsFs()
appfs.MkdirAll("src/a", 0755))
```

--------------------------------

### Import Kibana Dashboard

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Imports the Dsiem Kibana dashboard and its dependencies into Kibana. This command requires Kibana to be running and accessible at the specified host and port.

```shell
./scripts/kbndashboard-import.sh localhost ./deployments/kibana/dashboard-siem.json
```

--------------------------------

### Configure Elasticsearch and Kibana URLs

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Specifies the Elasticsearch and Kibana endpoint URLs in the `esconfig.json` file. Supports basic authentication for Elasticsearch by including username and password in the URL.

```json
{
  "elasticsearch": "http://elasticsearch:9200",
  "kibana": "http://kibana:5601"
}
```

```json
{
  "elasticsearch": "http://username:password@elasticsearch:9200",
  "kibana": "http://kibana:5601"
}
```

--------------------------------

### Set Filebeat Configuration Ownership

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Changes the ownership of Filebeat configuration files to 'root' for security reasons, as recommended by the Beats documentation. This ensures proper permissions for the configuration files.

```shell
cd deployments/docker && \
    sudo chown root $(find conf/filebeat/ conf/filebeat-es/ -name "*.yml")
```

--------------------------------

### Afero MemMapFs Backend

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Illustrates the MemMapFs backend, an atomic, concurrent, memory-backed filesystem ideal for mocking and speeding up I/O operations when persistence is not required.

```go
mm := afero.NewMemMapFs()
mm.MkdirAll("src/a", 0755))
```

--------------------------------

### Set Promiscuous Network Interface

Source: https://github.com/defenxor/dsiem/blob/master/docs/installation.md

Exports an environment variable to specify the network interface that Suricata should monitor for traffic. Replace 'eth0' with your system's active network interface name.

```shell
export PROMISC_INTERFACE=eth0
```

--------------------------------

### Use Afero with Application Code

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/afero/README.md

Shows how to replace standard `os` package calls with Afero's `AppFs` methods for filesystem operations, enabling the use of the chosen backend throughout the application.

```go
// Before:
// os.Open("/tmp/foo")

// After:
// AppFs.Open("/tmp/foo")
```

--------------------------------

### Check and Set Required Elasticsearch Plugins

Source: https://github.com/defenxor/dsiem/blob/master/vendor/gopkg.in/olivere/elastic.v5/CHANGELOG-3.0.md

Provides helpers to check for installed Elasticsearch plugins and to ensure required plugins are present during client initialization. This is crucial for features like Delete-by-Query which are now plugins.

```go
err, found := client.HasPlugin("delete-by-query")
if err == nil && found {
	// ... Delete By Query API is available
}
```

```go
// Will raise an error if the "delete-by-query" plugin is NOT installed
client, err := elastic.NewClient(elastic.SetRequiredPlugins("delete-by-query"))
if err != nil {
  log.Fatal(err)
}
```

--------------------------------

### Check and Set Required Elasticsearch Plugins

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/olivere/elastic/v7/CHANGELOG-3.0.md

Provides helpers to check for installed Elasticsearch plugins and to ensure required plugins are present during client initialization. This is crucial for features like Delete-by-Query which are now plugins.

```go
err, found := client.HasPlugin("delete-by-query")
if err == nil && found {
	// ... Delete By Query API is available
}
```

```go
// Will raise an error if the "delete-by-query" plugin is NOT installed
client, err := elastic.NewClient(elastic.SetRequiredPlugins("delete-by-query"))
if err != nil {
  log.Fatal(err)
}
```

--------------------------------

### Check and Set Required Elasticsearch Plugins

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/olivere/elastic/CHANGELOG-3.0.md

Provides helpers to check for installed Elasticsearch plugins and to ensure required plugins are present during client initialization. This is crucial for features like Delete-by-Query which are now plugins.

```go
err, found := client.HasPlugin("delete-by-query")
if err == nil && found {
	// ... Delete By Query API is available
}
```

```go
// Will raise an error if the "delete-by-query" plugin is NOT installed
client, err := elastic.NewClient(elastic.SetRequiredPlugins("delete-by-query"))
if err != nil {
  log.Fatal(err)
}
```

--------------------------------

### NATS Go Client Basic Usage

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/nats-io/nats.go/README.md

Demonstrates fundamental operations of the NATS Go client, including connecting to a server, publishing messages, subscribing asynchronously and synchronously, handling requests and replies, and managing subscriptions and connections.

```go
import "github.com/nats-io/nats.go"
import "fmt"
import "time"

// Connect to a server
nc, _ := nats.Connect(nats.DefaultURL)

// Simple Publisher
nc.Publish("foo", []byte("Hello World"))

// Simple Async Subscriber
nc.Subscribe("foo", func(m *nats.Msg) {
    fmt.Printf("Received a message: %s\n", string(m.Data))
})

// Responding to a request message
nc.Subscribe("request", func(m *nats.Msg) {
    m.Respond([]byte("answer is 42"))
})

// Simple Sync Subscriber
sub, err := nc.SubscribeSync("foo")
m, err := sub.NextMsg(timeout)

// Channel Subscriber
ch := make(chan *nats.Msg, 64)
sub, err := nc.ChanSubscribe("foo", ch)
msg := <- ch

// Unsubscribe
sub.Unsubscribe()

// Drain
sub.Drain()

// Requests
msg, err := nc.Request("help", []byte("help me"), 10*time.Millisecond)

// Replies
nc.Subscribe("help", func(m *nats.Msg) {
    nc.Publish(m.Reply, []byte("I can help!"))
})

// Drain connection (Preferred for responders)
// Close() not needed if this is called.
nc.Drain()

// Close connection
nc.Close()
```

--------------------------------

### fasthttp Further Questions Link

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/valyala/fasthttp/README.md

Directs users to GitHub issues labeled 'question' for answers not found in the FAQ.

```go
Try exploring [these questions](https://github.com/valyala/fasthttp/issues?q=label%3Aquestion).
```

--------------------------------

### Build sys/unix with New System (Go)

Source: https://github.com/defenxor/dsiem/blob/master/vendor/golang.org/x/sys/unix/README.md

Executes `mkall.sh` to generate Go files using a Docker container for the new build system. This method is reproducible and requires bash, Go, and Docker on an amd64/Linux system.

```bash
mkall.sh
# To see commands without running:
mkall.sh -n
```

--------------------------------

### Install golang.org/x/sys

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/fsnotify/fsnotify/README.md

Ensures the latest version of the required system package is installed for fsnotify compatibility. This is a prerequisite for using the library.

```console
go get -u golang.org/x/sys/...
```

--------------------------------

### Basic Cobra Application Entrypoint

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/cobra/README.md

Shows a minimal main.go file for a Cobra application. Its primary purpose is to initialize and execute the Cobra command tree.

```go
package main

import (
  "{pathToYourApp}/cmd"
)

func main() {
  cmd.Execute()
}
```

--------------------------------

### Basic Logging with jWalterWeatherman

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/jwalterweatherman/README.md

Demonstrates how to use the jWalterWeatherman library for logging messages at different severity levels. It shows direct usage of logger instances like ERROR and WARN without explicit initialization.

```go
import (
    jww "github.com/spf13/jwalterweatherman"
)

...

if err != nil {
    // Log a serious error that will be printed to the terminal and logged.
    jww.ERROR.Println(err)
}

if err2 != nil {
    // Log a warning that might not be critical but should be logged.
    // By default, it's logged but not printed to the terminal.
    jww.WARN.Println(err2)
}

// Log informational messages that might be discarded by default thresholds.
jww.INFO.Printf("information %q", response)
```

--------------------------------

### Install goconcurrentqueue

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/enriquebris/goconcurrentqueue/readme.md

This snippet shows the command to install the goconcurrentqueue package using Go's module system. It is compatible with Go versions 1.7.x through 1.12.x.

```bash
go get github.com/enriquebris/goconcurrentqueue
```

--------------------------------

### Custom Bash Completion Function Example

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/spf13/cobra/bash_completions.md

An example of a custom bash completion function that retrieves namespace names using `kubectl` and provides them as completion suggestions.

```bash
__kubectl_get_namespaces()
{
    local template
    template="{{ range .items  }}{{ .metadata.name }} {{ end }}"
    local kubectl_out
    if kubectl_out=$(kubectl get -o template --template="${template}" namespace 2>/dev/null);
    then
        COMPREPLY=( $( compgen -W "${kubectl_out}[*]" -- "$cur" ) )
    fi
}
```

--------------------------------

### HCL Object and Nested Structure Example

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/hashicorp/hcl/README.md

Provides an example of defining nested objects in HCL, showing how to structure configuration data with keys and values.

```hcl
variable "ami" {
    description = "the AMI to use"
}
```

--------------------------------

### Fasthttp File Serving Mappings

Source: https://github.com/defenxor/dsiem/blob/master/vendor/github.com/valyala/fasthttp/README.md

Illustrates how to serve static files and handle URL path rewriting in fasthttp, comparing them to standard net/http functionalities. This includes serving directories and stripping URL prefixes.

```APIDOC
File Serving and Path Rewriting:

- `http.FileServer()` -> `fasthttp.FSHandler()`, `fasthttp.FS`
- `http.ServeFile()` -> `fasthttp.ServeFile()`
- `http.StripPrefix()` -> `fasthttp.PathRewriteFunc`
```

--------------------------------

### Example Threat Intel Configuration (JSON)

Source: https://github.com/defenxor/dsiem/blob/master/docs/ti_vuln_plugins.md

An example JSON configuration file for a threat intelligence plugin, specifying sources, their types, and configuration details like URLs.

```json
{
  "intel_sources": [
    {
      "name": "Wise",
      "plugin": "Wise",
      "type": "IP",
      "enabled": true,
      "config": "{ \"url\" : \"http://wise:8081/ip/${ip}\" }"
    }
  ]
}
```

--------------------------------

### Build dsiem Binary

Source: https://github.com/defenxor/dsiem/blob/master/docs/ti_vuln_plugins.md

Commands to clone the dsiem repository and build the main dsiem binary using Go.

```bash
$ git clone https://github.com/defenxor/dsiem
$ cd dsiem
$ go build ./cmd/dsiem
```