Tenzir

https://docs.tenzir.com/
Admin
Tenzir is the data pipeline engine for security teams that collects, transforms, enriches, and...
Tokens:557,551
Snippets:6,957
Trust Score:8.7
Update:6 months ago
Show doc for...
Tokens:
Raw
### Example package information (compact format)

Source: https://docs.tenzir.com/reference/operators/package/list

An example of the compact output format for a single installed package, showing its ID, name, author, description, and configuration.

```json
{
  "id": "suricata-ocsf",
  "name": "Suricata OCSF Mappings",
  "author": "Tenzir",
  "description": "[Suricata](https://suricata.io) is an open-source network monitor and\nthreat detection tool.\n\nThis package converts all Suricata events published on the topic `suricata` to\nOCSF and publishes the converted events on the topic `ocsf`.\n",
  "config": {
    "inputs": {},
    "overrides": {}
  }
}
```

--------------------------------

### Tenzir Platform Startup Output

Source: https://docs.tenzir.com/guides/platform-setup/run-the-platform

Example output shown when the Tenzir Platform starts. It indicates the status of various containers and logs from the platform services.

```text
[+] Running 5/5
 ✔ Container compose-app-1                Running
 ✔ Container compose-websocket-gateway-1  Running
 ✔ Container compose-seaweed-1            Running
 ✔ Container compose-platform-1           Running
Attaching to app-1, platform-1, postgres-1, seaweed-1, websocket-gateway-1
platform-1           | {"event": "connecting to postgres", "level": "debug", "ts": "2024-04-10T10:13:20.205616Z"}
platform-1           | {"event": "connected to postgres", "level": "debug", "ts": "2024-04-10T10:13:20.210667Z"}
platform-1           | {"event": "created table", "level": "info", "ts": "2024-04-10T10:13:20.210883Z"}
platform-1           | {"event": "connecting to postgres", "level": "debug", "ts": "2024-04-10T10:13:20.217700Z"}
platform-1           | {"event": "connected to postgres", "level": "debug", "ts": "2024-04-10T10:13:20.221194Z"}
platform-1           | {"event": "creating a table", "level": "info", "ts": "2024-04-10T10:13:20.221248Z"}
platform-1           | {"event": "connecting to postgres", "level": "debug", "ts": "2024-04-10T10:13:20.221464Z"}
platform-1           | {"event": "connected to postgres", "level": "debug", "ts": "2024-04-10T10:13:20.224226Z"}
app-1                | Listening on 0.0.0.0:3000
websocket-gateway-1  | {"event": "connecting to postgres", "level": "debug", "ts": "2024-04-10T10:15:37.033632Z"}
websocket-gateway-1  | {"event": "connected to postgres", "gtlevel": "debug", "ts": "2024-04-10T10:15:37.038510Z"}
websocket-gateway-1  | {"event": "created table", "level": "info", "ts": "2024-04-10T10:15:37.042555Z"}
websocket-gateway-1  | {"host": "0.0.0.0", "port": 5000, "common_env": {"base_path": "", "tenzir_proxy_timeout": 60.0}, "local_env": {"store": {"postgres_uri": "postgresql://postgres:postgres@postgres:5432/platform"}, "tenant_manager_app_api_key": "d3d185cc4d9a1bde0e07e24c2eb0bfe9d2726acb3a386f8882113727ac6e90cf"}, "tenant_manager_tenant_token_encryption_key": "CBOXE4x37RKRLHyUNKeAsfg8Tbejm2N251aKnBXakpU="}, "event": "HTTP server running", "level": "info", "ts": "2024-04-10T10:15:37.045244Z"}
...

```

--------------------------------

### Install Sysmon with configuration

Source: https://docs.tenzir.com/integrations/microsoft/windows-event-logs

Installs the Sysmon service using the downloaded configuration file.

```ps
.\Sysmon64.exe -accepteula -i sysmonconfig-export.xml
```

--------------------------------

### Examples

Source: https://docs.tenzir.com/reference/operators/from_s3

Examples demonstrating how to use the Tenzir file input operator with various configurations.

```APIDOC
## Examples

### Read every JSON file from a bucket
```
from_s3 "s3://my-bucket/data/**.json"
```

### Read CSV files using explicit credentials
```
from_s3 "s3://my-bucket/data.csv", aws_iam={
  access_key_id: "AKIAIOSFODNN7EXAMPLE",
  secret_access_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
```

### Read from S3-compatible service with custom endpoint
```
from_s3 "s3://my-bucket/data/**.json?endpoint_override=minio.example.com:9000&scheme=http"
```

### Read files continuously and assume IAM role
```
from_s3 "s3://logs/application/**.json", watch=true, aws_iam={
  assume_role: "arn:aws:iam::123456789012:role/LogReaderRole"
}
```

### Process files and move them to an archive bucket
```
from_s3 "s3://input-bucket/**.json",
  rename=(path => "archive/" + path)
```

### Add source path to events
```
from_s3 "s3://data-bucket/**.json", path_field=source_file
```

### Read Zeek logs with anonymous access
```
from_s3 "s3://public-bucket/zeek/**.log", anonymous=true {
  read_zeek_tsv
}
```
```

--------------------------------

### Pipeline action example

Source: https://docs.tenzir.com/reference/node/api/operations/pipelineupdate

Example of a valid pipeline state transition action.

```text
start
```

--------------------------------

### Examples

Source: https://docs.tenzir.com/reference/operators/to_hive

Illustrative examples of using the to_hive operator for different scenarios.

```APIDOC
## Examples

### Partition by a single field into local JSON files

This example demonstrates partitioning data by the field `a` and writing it as JSON files to a local directory.

```
from {a: 0, b: 0}, {a: 0, b: 1}, {a: 1, b: 2}
to_hive "/tmp/out/", partition_by=[a], format="json"
```

### Write a Parquet file into Azure Blob Store

This example shows how to write data in Parquet format to Azure Blob Storage, partitioned by `year`, `month`, and `day`.

```
to_hive "abfs://domain/bucket", partition_by=[year, month, day], format="parquet"
```

### Write partitioned JSON into an S3 bucket

This example writes JSON data to an S3 bucket, partitioned by `year` and `month`, with a `max_size` of 1GB to control file size.

```
year = ts.year()
month = ts.month()
to_hive "s3://my-bucket/some/subdirectory",
  partition_by=[year, month],
  format="json",
  max_size=1G
```
```

--------------------------------

### Install a package with TQL

Source: https://docs.tenzir.com/guides/packages/install-a-package

Use the package::add operator to install a package from a local path or with specific input configurations.

```tql
package::add "/path/to/pkg"
```

```tql
package::add "package.yaml", inputs={
  endpoint: "localhost:42000",
  policy: "block",
}
```

--------------------------------

### Example FTP Save Operation

Source: https://docs.tenzir.com/reference/operators/save_ftp

This example demonstrates how to use the save_ftp command with a simple FTP URL.

```text
save_ftp "ftp.example.org"
```

--------------------------------

### Install Tenzir System-Wide

Source: https://docs.tenzir.com/guides/development/build-from-source

Install Tenzir to the system path after building. Options include specifying a custom prefix, stripping debug symbols, or installing specific components.

```sh
cmake --install build
```

--------------------------------

### List installed packages (compact format)

Source: https://docs.tenzir.com/reference/operators/package/list

Shows all installed packages in a human-readable, compact format. This is the default output.

```text
package::list
```

--------------------------------

### Install Specific Skill Globally

Source: https://docs.tenzir.com/guides/ai-workbench/use-agent-skills

Installs a specific skill globally.

```bash
npx skills add -g tenzir/skills@ocsf
```

--------------------------------

### Get a live feed of error diagnostics

Source: https://docs.tenzir.com/reference/operators/diagnostics

Example of how to get a real-time feed of diagnostic events with severity 'error'.

```APIDOC
## GET /diagnostics

### Description
Get a live feed of error diagnostics.

### Method
GET

### Endpoint
/diagnostics

### Query Parameters
- **live** (bool) - Required - Set to `true` to emit diagnostic events as they are generated in real-time.
- **filter** (string) - Optional - Filter for diagnostics where severity is 'error'.

### Response
#### Success Response (200)
- **pipeline_id** (string) - The ID of the pipeline that created the diagnostic.
- **hidden** (bool) - Indicates whether the corresponding pipeline is hidden from the list of managed pipelines.
- **run** (uint64) - The number of the run, starting at 1 for the first run.
- **timestamp** (time) - The exact timestamp of the diagnostic creation.
- **message** (string) - The diagnostic message.
- **severity** (string) - The diagnostic severity.
- **notes** (list<record>) - The diagnostic notes. Can be empty.
- **annotations** (list<record>) - The diagnostic annotations. Can be empty.
- **rendered** (string) - The rendered diagnostic, as printed on the command-line.
```

--------------------------------

### Start development server

Source: https://docs.tenzir.com/guides/contribution/documentation

Launch the local development server to preview changes with hot module reloading.

```bash
bun run dev
```

--------------------------------

### List installed packages

Source: https://docs.tenzir.com/guides/packages/install-a-package

Verify installed packages and their status using the package::list operator.

```tql
package::list
```

```tql
{
  id: "your-package",
  install_status: "installed",
  // …
}
```

--------------------------------

### List installed packages (extended format)

Source: https://docs.tenzir.com/reference/operators/package/list

Lists all installed packages with detailed, machine-readable information. Use this format for programmatic consumption.

```text
package::list [format=string]
```

--------------------------------

### Remove an installed package

Source: https://docs.tenzir.com/reference/operators/package/remove

Example showing how to remove a package with the ID 'suricata-ocsf'.

```text
package::remove "suricata-ocsf"
```

--------------------------------

### Add release intro summary

Source: https://docs.tenzir.com/guides/packages/maintain-a-changelog

Include an introductory summary for the release notes via command line or file.

```sh
uvx tenzir-ship release create \
  --intro "Major stability improvements." \
  --yes
```

```sh
uvx tenzir-ship release create \
  --intro-file intro.md \
  --yes
```

--------------------------------

### Get value counts example

Source: https://docs.tenzir.com/reference/functions/value_counts

Demonstrates calculating the frequency of values within a dataset.

```text
from {x: 1}, {x: 2}, {x: 2}, {x: 3}
summarize counts=value_counts(x)
```

```json
{counts: [{value: 1, count: 1}, {value: 2, count: 2}, {value: 3, count: 1}]}
```

--------------------------------

### Install Package

Source: https://docs.tenzir.com/reference/operators

Installs a specified package. Use this to extend Tenzir's functionality with external modules or data.

```tql
package::add "suricata-ocsf"
```

--------------------------------

### Perform an API call

Source: https://docs.tenzir.com/reference/operators/load_http

Example of using load_http to perform a GET request with custom headers.

```text
load_http "example.org/api", headers={"X-API-Token": "0000-0000-0000"}
```

--------------------------------

### Install Package with Inputs using package::add

Source: https://docs.tenzir.com/guides/packages/configure-inputs

Install a package using the `package::add` operator and provide input values via the `inputs` record. Omit inputs with default values to use their defaults.

```tql
package::add "/path/to/package", inputs={
  refresh_interval: "24h",
  api_endpoint: "https://api.example.com/data",
}

```

```tql
// Uses default refresh_interval, overrides api_endpoint
package::add "/path/to/package", inputs={
  api_endpoint: "https://custom.example.com/feed",
}

```

--------------------------------

### Install All Tenzir Skills

Source: https://docs.tenzir.com/guides/ai-workbench/use-agent-skills

Installs all Tenzir skills into the current project. The CLI will prompt for agent selection.

```bash
npx skills add tenzir/skills
```

--------------------------------

### Start a TCP server to accept incoming connections

Source: https://docs.tenzir.com/guides/collecting/get-data-from-the-network

Listen on all interfaces on a specified port and parse incoming bytes as JSON. Ensure the port is not already in use.

```tql
from "tcp://0.0.0.0:9000" {
  read_json
}
```

--------------------------------

### Get the network address of a subnet

Source: https://docs.tenzir.com/reference/functions/network

Example usage of the network function within a select statement.

```sql
from {subnet: 192.168.0.0/16}
select ip = subnet.network()
```

```json
{ip: 192.168.0.0}
```

--------------------------------

### Create package examples in TQL

Source: https://docs.tenzir.com/guides/packages/create-a-package

Provides runnable TQL snippets for the Tenzir Library.

```tql
// Demonstrate the primary use case
acme::fetch
acme::transform
head 10
```

```tql
// Show a more complex scenario
acme::fetch
where severity == "high"
acme::enrich
publish "alerts"
```

--------------------------------

### Get the byte length of a Unicode string

Source: https://docs.tenzir.com/reference/functions/length_bytes

Example showing byte length for a Unicode string.

```text
from {x: "👻".length_bytes()}
```

```text
{x: 4}
```

--------------------------------

### Build and preview production site

Source: https://docs.tenzir.com/guides/contribution/documentation

Commands to generate a production-ready build and preview it locally.

```bash
# Build the site
bun run build


# Preview the build
bun run preview
```

--------------------------------

### Get the byte length of an ASCII string

Source: https://docs.tenzir.com/reference/functions/length_bytes

Example showing byte length for an ASCII string.

```text
from {x: "hello".length_bytes()}
```

```text
{x: 5}
```

--------------------------------

### Get the last non-null value

Source: https://docs.tenzir.com/reference/functions/last

Example usage of the last function within a summarize pipeline.

```Tenzir
from {x: 1}, {x: 2}, {x: null}
summarize last_value=last(x)
```

```JSON
{last_value: 2}
```

--------------------------------

### Clone the repository

Source: https://docs.tenzir.com/guides/contribution/documentation

Initial step to download the documentation source code.

```bash
git clone https://github.com/tenzir/docs.git
cd docs
```

--------------------------------

### Get distinct values in a list

Source: https://docs.tenzir.com/reference/functions/distinct

Example usage of the distinct function within a summarize operation.

```Tenzir
from {x: 1}, {x: 2}, {x: 2}, {x: 3}
summarize unique=distinct(x)
```

```JSON
{unique: [1, 2, 3]}
```

--------------------------------

### Show Dataset Summary Example

Source: https://docs.tenzir.com/tutorials/write-a-package

Inspects the context data and renders a pie chart of malware types.

```TQL
---
description: |
  Inspect all data in the context and count the different malware types,
  rendering the result as a pie chart.
---


context::inspect "sslbl"
select malware = value.malware[0].name
top malware
chart_pie x=malware, y=count
```

--------------------------------

### Initialize a new changelog project

Source: https://docs.tenzir.com/changelog/tenzir-ship/v1-4-0

Run this command to scaffold a new project interactively or non-interactively.

```bash
tenzir-ship init
```

--------------------------------

### Return every second event starting from the tenth

Source: https://docs.tenzir.com/reference/operators/slice

Use the stride parameter to select events at intervals. This example starts from the tenth event (index 9) and selects every second event thereafter.

```text
slice begin=9, stride=2
```

--------------------------------

### Match Regex: Full String Match

Source: https://docs.tenzir.com/reference/functions/match_regex

To ensure the entire string matches the regular expression, anchor the pattern with '^' for the start and '$' for the end. This example checks for 'example' optionally followed by '!' at the end of the string.

```string
from {input: "example"},
  {input: "Example!"},
  {input: "example?" }
output = input.match_regex("^[E|e]xample[!]?$")
```

```string
{input: "example", output: true}
{input: "example!", output: true}
{input: "example?", output: false}
```

--------------------------------

### Trim whitespace from string start

Source: https://docs.tenzir.com/reference/functions/trim_start

Removes leading whitespace characters from a string. No additional setup is required.

```Tenzir
from {x: " hello".trim_start()}
```

```Tenzir
{x: "hello"}
```

--------------------------------

### Combine multiple pipeline configurations

Source: https://docs.tenzir.com/guides/packages/add-pipelines

A comprehensive example demonstrating the use of metadata, disabled status, and error recovery settings.

```tql
---
name: Publish as OCSF
description: >
  Fetches threat intel hourly and publishes OCSF events to the `ocsf` topic.
disabled: true
restart-on-error: 5m
---


every 1h {
  acme::fetch
}
acme::ocsf::normalize
publish "ocsf"
```

--------------------------------

### Get an event whenever the node disconnected from the Tenzir Platform

Source: https://docs.tenzir.com/reference/operators/deduplicate

Example of using `deduplicate` with `distance` and `where` to detect disconnections.

```APIDOC
## Get an event whenever the node disconnected from the Tenzir Platform

### Description
Get an event whenever the node disconnected from the Tenzir Platform.

### Method
N/A (Operator)

### Endpoint
N/A (Operator)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```
metrics "platform", live=true
deduplicate connected, distance=1
where not connected
```

### Response
#### Success Response (200)
N/A (Operator)

#### Response Example
N/A
```

--------------------------------

### Get up to 10 warnings per hour for each run of a pipeline

Source: https://docs.tenzir.com/reference/operators/deduplicate

Example of using limit and create_timeout to throttle diagnostic output.

```Tenzir
diagnostics live=true
deduplicate pipeline_id, run, limit=10, create_timeout=1h
```

--------------------------------

### Get Tenzir version information

Source: https://docs.tenzir.com/guides/basic-usage/run-pipelines

Use this pipeline to retrieve version details of the Tenzir installation. The output is a JSON object.

```tql
{
  version: "5.0.1+g847fcc6334",
  tag: "g847fcc6334",
  major: 5,
  minor: 0,
  patch: 1,
  features: [
    "chart_limit",
    "modules",
    "tql2_from",
    "exact_schema",
    "tql2_only",
  ],
  build: {
    type: "Release",
    tree_hash: "ef28a81eb124cc46a646250d1fb17390",
    assertions: false,
    sanitizers: {
      address: false,
      undefined_behavior: false,
    },
  },
}
```

--------------------------------

### Initialize HTTP server

Source: https://docs.tenzir.com/reference/operators/from_http

Starts an HTTP server listening on port 8080.

```Tenzir
from_http "0.0.0.0:8080", server=true, metadata_field=metadata
```

--------------------------------

### CLI Command: init

Source: https://docs.tenzir.com/reference/ship-framework

Initializes a new changelog scaffold in either standalone or package mode.

```APIDOC
## CLI Command: init

### Description
Creates the initial changelog scaffold. Existing projects are never overwritten; the command fails if configuration or directory structures already exist.

### Method
CLI Command

### Endpoint
tenzir-ship init [options]

### Parameters
#### Options
- **--yes** (flag) - Optional - Initialize without interactive prompts
- **--package** (flag) - Optional - Force package mode using package.yaml metadata
- **--standalone** (flag) - Optional - Force standalone mode with config.yaml
- **--id** (string) - Optional - Project identifier for standalone mode
- **--name** (string) - Optional - Project display name for standalone mode
- **--description** (string) - Optional - Project description for standalone mode
- **--repository** (string) - Optional - GitHub repository slug (owner/repo) for standalone mode
```

--------------------------------

### Reset events to begin at Jan 1, 1984

Source: https://docs.tenzir.com/reference/operators/timeshift

Example of anchoring a Zeek log timestamp field to a specific start date.

```text
load_http "https://storage.googleapis.com/tenzir-datasets/M57/zeek-all.log.zst"
decompress "zstd"
read_zeek_tsv
timeshift ts, start=1984-01-01
```

--------------------------------

### Full Docker Compose Configuration

Source: https://docs.tenzir.com/reference/test-framework

A comprehensive configuration example demonstrating all available options for the docker-compose fixture.

```yaml
fixtures:
  - docker-compose:
      file: compose.yaml
      project_name: my-project
      profiles: [integration]
      services: [redis, postgres]
      env_file: compose.env
      env:
        COMPOSE_PARALLEL_LIMIT: 8
      pull: always
      build: true
      wait:
        timeout_seconds: 90
        poll_interval_seconds: 2.0
      down:
        volumes: true
        remove_orphans: true
        timeout_seconds: 30
      log_on_failure: true
```

--------------------------------

### Execute predicate pushdown with constant expressions

Source: https://docs.tenzir.com/changelog/tenzir/v4-25-0

Example of a pipeline starting with export and a constant predicate that no longer triggers internal errors.

```TQL
export | where 1 == 1
```

--------------------------------

### Get node version using remote operator

Source: https://docs.tenzir.com/reference/operators/remote

Example of using the `remote` operator to retrieve the version information from a Tenzir Node.

```tenzir
remote {
  version
}
```

--------------------------------

### Start Tenzir Node with Web Server via Command Line

Source: https://docs.tenzir.com/guides/node-setup/start-the-api

This command-line invocation is equivalent to the YAML configuration for starting the web server component.

```bash
tenzir-node --commands="web server [...]"
```

--------------------------------

### Get up to 10 warnings per hour for each run of a pipeline

Source: https://docs.tenzir.com/reference/operators/deduplicate

Example of using `deduplicate` with `limit` and `create_timeout` to control event frequency.

```APIDOC
## Get up to 10 warnings per hour for each run of a pipeline

### Description
Get up to 10 warnings per hour for each run of a pipeline.

### Method
N/A (Operator)

### Endpoint
N/A (Operator)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```
diagnostics live=true
deduplicate pipeline_id, run, limit=10, create_timeout=1h
```

### Response
#### Success Response (200)
N/A (Operator)

#### Response Example
N/A
```

--------------------------------

### Extract file name from path

Source: https://docs.tenzir.com/reference/functions/file_name

Use this function to get the file name component of a file path. No special setup is required.

```text
from {x: file_name("/path/to/log.json")}
```

```text
{x: "log.json"}
```

--------------------------------

### Install from a custom repository

Source: https://docs.tenzir.com/guides/packages/install-a-package

Clone a repository and register the directory in the Tenzir configuration to make packages available.

```bash
git clone https://github.com/your-org/my-packages.git
tenzir --package-dirs=/path/to/my-packages
```

```yaml
tenzir:
  package-dirs:
    - /path/to/my-packages
```

```tql
package::add "/path/to/my-packages/package-one"
```

--------------------------------

### Clean firewall logs pipeline

Source: https://docs.tenzir.com/guides/normalization/clean-up-values

A complete pipeline example demonstrating null replacement, type conversion, and field cleanup.

```tql
from_kafka "firewall-raw"
this = message.parse_json()


// Replace string nulls
replace what="N/A", with=null
replace what="-", with=null
replace what="", with=null


// Convert types
src_ip = src_ip?.ip()
dst_ip = dst_ip?.ip()
src_port = src_port?.int()
dst_port = dst_port?.int()
timestamp = timestamp?.time()
bytes = bytes?.int()


// Normalize booleans
blocked = action? == "block"


// Provide defaults
action = action? else "unknown"
protocol = protocol? else "unknown"


// Drop nulls from user context (e.g., domain, group may be missing)
drop_null_fields user
```

--------------------------------

### Get the last 5 events

Source: https://docs.tenzir.com/reference/operators/slice

A negative value for the begin parameter counts from the end of the stream. This example keeps the last 5 events.

```text
slice begin=-5
```

--------------------------------

### Calculate Absolute Value of a Number

Source: https://docs.tenzir.com/reference/functions/abs

Use the abs() function to get the absolute value of a number. This example shows how to apply it to a negative number.

```tenzir
from {x: -13.3}
x = x.abs()
```

```tenzir
{x: 13.3}
```

--------------------------------

### Start OpenWEC Server

Source: https://docs.tenzir.com/integrations/microsoft/windows-event-logs

This command starts the OpenWEC server with the provided configuration. It is recommended to manage this process using a service manager like systemd for reliability.

```bash
openwecd -c openwec.conf.toml
```

--------------------------------

### Universal Function Call Syntax Example

Source: https://docs.tenzir.com/changelog/tenzir/v4-23-0

Demonstrates the new universal function call syntax where methods are callable as functions and functions with positional arguments are callable as methods.

```tql
call_function(arg1, arg2)
method(arg1, arg2)
```

--------------------------------

### Get high-level statistics about a query using cache

Source: https://docs.tenzir.com/reference/operators/cache

This example shows how to retrieve statistics from a cache, with options for read timeout and reading from an existing cache.

```APIDOC
## Example: Query Statistics with Cache

### Description
Get high-level statistics about a query, utilizing cache for efficiency.

### Code
```tenzir
# Calculate cache again only if query does not exist, delete if unused for more than a minute
export
where @name == "suricata.flow"
summarize src_ip, total=sum(bytes_toserver), dest_ip
cache "some-unique-identifier", read_timeout=1min
summarize src_ip, total=sum(total), destinations=count(dest_ip)

# Get the same statistics, assuming the cache still exists
cache "some-unique-identifier", mode="read"
summarize src_ip, total=sum(total), destinations=count(dest_ip)
```
```

--------------------------------

### Pipeline Dataflow Example

Source: https://docs.tenzir.com/reference/statements

A pipeline demonstrating input, transformation, and output operators with implicit format inference.

```TQL
from "/path/to/file.json"
where src_ip in 10.0.0.0/8
to "s3://bucket/dir/file.parquet"
```

--------------------------------

### Get the first non-null value

Source: https://docs.tenzir.com/reference/functions/first

Example usage of the first function within a summarize operation to extract the first non-null value from a stream of records.

```Tenzir
from {x: null}, {x: 2}, {x: 3}
summarize first_value=first(x)
```

```json
{first_value: 2}
```

--------------------------------

### Get first N events with head

Source: https://docs.tenzir.com/guides/optimization/slice-and-sample-data

Use head to retrieve a specific number of events from the start of a stream. Omitting the argument returns all events.

```tql
from {id: 1, value: "a"},
     {id: 2, value: "b"},
     {id: 3, value: "c"},
     {id: 4, value: "d"},
     {id: 5, value: "e"}
head 3
```

```tql
from {id: 1, value: "first"},
     {id: 2, value: "second"},
     {id: 3, value: "third"}
head
```

--------------------------------

### Listen with TLS enabled

Source: https://docs.tenzir.com/reference/operators/from_opensearch

Example of configuring the operator to use TLS with specific certificate and key files.

```tql
from_opensearch tls=true, certfile="server.crt", keyfile="private.key"
```

--------------------------------

### Slice List with Negative Begin Offset

Source: https://docs.tenzir.com/reference/functions/slice

Selects elements from a list starting from a negative offset, which counts from the end of the list. This example selects the last three elements.

```list
from {xs: [1, 2, 3, 4, 5]}
xs = xs.slice(begin=-3)
```

```list
{xs: [3, 4, 5]}
```

--------------------------------

### Create Release with Custom Intro

Source: https://docs.tenzir.com/guides/packages/maintain-a-changelog

Create the release with a custom intro file. The tool writes release artifacts under `releases/v0.1.0/`.

```sh
uvx tenzir-ship release create v0.1.0 \
  --intro-file intro.md \
  --yes
```

--------------------------------

### Define a lookup-table context in package.yaml

Source: https://docs.tenzir.com/guides/packages/add-contexts

Define a context in the `contexts` section of your `package.yaml`. The node creates these contexts when you install the package. This example defines a lookup-table context for threat indicators.

```yaml
contexts:
  threat-indicators:
    type: lookup-table
    description: |
      A lookup table keyed by indicator values (hashes, IPs, domains)
      for threat intelligence enrichment.
    args: {}
    disabled: false
```

--------------------------------

### Create Release with Intro Text

Source: https://docs.tenzir.com/changelog/tenzir-ship/v1-2-0

Use the `--intro` or `--intro-file` flags with `release create` to publish a new release version without adding changelog entries. This is useful for re-publishing or retrying failed workflows.

```bash
release create --intro "My introductory text."
```

```bash
release create --intro-file ./intro.md
```

--------------------------------

### Get the second 100 events

Source: https://docs.tenzir.com/reference/operators/slice

Use the begin and end parameters to specify a range of events. This example retrieves events from index 100 (inclusive) up to index 200 (exclusive).

```text
slice begin=100, end=200
```

--------------------------------

### Create Intro Snippet for Release Notes

Source: https://docs.tenzir.com/guides/packages/maintain-a-changelog

Author an intro snippet that can include Markdown links, call-outs, or image references. Save it as `intro.md`.

```markdown
Welcome to the first release of the Tenzir changelog!


![Release Overview](images/release-overview.png)


We cover breaking changes, highlights, upgrades, and fixes below.
```

--------------------------------

### TQL Test with Stdin File Support

Source: https://docs.tenzir.com/changelog/tenzir-test/v1-1-0

Use a `.stdin` file next to your test to pipe its contents directly to the subprocess stdin. This example shows a TQL pipeline starting with a CSV parser.

```TQL
read_csv
where count > 10
```

--------------------------------

### Start REST API in Mutual TLS (mTLS) Mode

Source: https://docs.tenzir.com/guides/node-setup/start-the-api

Enable mTLS mode by passing `--mode=mtls`. Tenzir accepts HTTPS requests, requires TLS client certificates, and valid authentication tokens. Suitable for setups with a load balancer like nginx.

```bash
tenzir-node --commands="web server --certfile=/path/to/server.certificate --keyfile=/path/to/private.key --mode=mtls"
```

--------------------------------

### Configure Tenzir Plugins

Source: https://docs.tenzir.com/explanations/configuration

Specify plugins to load at startup in `tenzir.yaml`. Use `all` to load all available plugins. If `tenzir.plugins` is not specified, `all` is the default.

```yaml
tenzir:
  plugin-dirs:
    - .
    - /opt/foo/lib
  plugins:
    - example
    - /opt/bar/lib/libtenzir-plugin-example.so
```

--------------------------------

### Start REST API in Upstream TLS Mode

Source: https://docs.tenzir.com/guides/node-setup/start-the-api

Enable upstream TLS mode by passing `--mode=upstream`. Tenzir listens on localhost, accepts plain HTTP, but still checks authentication tokens. Suitable for setups with a separate TLS terminator like nginx.

```bash
tenzir-node --commands="web server --certfile=/path/to/server.certificate --keyfile=/path/to/private.key --mode=upstream"
```

--------------------------------

### Add a local package with inputs

Source: https://docs.tenzir.com/reference/operators/package/add

Installs a local package while providing a record of configuration inputs.

```tenzir

package::add "/mnt/config/tenzir/library/zeek",
  inputs={format: "tsv", "log-directory": "/opt/tenzir/logs"}

```

--------------------------------

### Install Tenzir via curl

Source: https://docs.tenzir.com/guides/installation

Downloads and installs binaries using the automated installer script.

```sh
curl https://get.tenzir.app | sh
```

--------------------------------

### Install tenzir-ship and verify CLI

Source: https://docs.tenzir.com/reference/ship-framework

Install the tenzir-ship Python package using uv (or pip) and verify the installation by checking the CLI help.

```bash
uv add tenzir-ship
uvx tenzir-ship --help
```

--------------------------------

### Release Manifest YAML Format

Source: https://docs.tenzir.com/guides/packages/maintain-a-changelog

Example of a `manifest.yaml` file recording the release date and intro content.

```yaml
created: 2025-10-18
intro: |-
  Welcome to the first release of the Tenzir changelog!


  ![Release Overview](images/release-overview.png)


  We cover breaking changes, highlights, upgrades, and fixes below.
```

--------------------------------

### Install Tenzir via Homebrew on macOS

Source: https://docs.tenzir.com/changelog/tenzir/v5-29-2

Install Tenzir on Apple Silicon macOS using Homebrew. This method keeps the installation in sync with releases.

```bash
brew tap tenzir/tenzir
brew install --cask tenzir
```

```bash
brew install --cask tenzir/tenzir/tenzir
```

--------------------------------

### Start Tenzir Platform (Foreground)

Source: https://docs.tenzir.com/guides/platform-setup/run-the-platform

Start the Tenzir Platform services in the foreground using Docker Compose. This command is useful for monitoring the startup process and logs in real-time.

```bash
docker compose up
```

--------------------------------

### Start Velociraptor frontend

Source: https://docs.tenzir.com/integrations/velociraptor

Launch the Velociraptor frontend using the specified server configuration.

```bash
velociraptor-binary -c server.yaml frontend
```

--------------------------------

### Annotation text example

Source: https://docs.tenzir.com/reference/node/api/operations/pipelineupdate

Example of an annotation label text.

```text
this option does not exist
```

--------------------------------

### Show Network Interfaces

Source: https://docs.tenzir.com/changelog/tenzir/v4-2-0

Use `show nics` to list network interfaces. Pipe to `select name` to get interface names for the `from nic` command.

```shell
show nics
show nics | select name
```

--------------------------------

### Pipeline name example

Source: https://docs.tenzir.com/reference/node/api/operations/pipelineupdate

Example of a human-readable pipeline name.

```text
zeek-monitoring-pipeline
```

--------------------------------

### Example: Route data to multiple TCP ports

Source: https://docs.tenzir.com/reference/operators/load_balance

Demonstrates how to use load_balance to send data to different TCP ports.

```APIDOC
## Example: Route data to multiple TCP ports

### Description
This example routes incoming events to different TCP ports based on a predefined list of addresses.

### Code
```tenzir
let $cfg = ["192.168.0.30:8080", "192.168.0.30:8081"]

subscribe "input"
load_balance $cfg {
  write_json
  save_tcp $cfg
}
```
```

--------------------------------

### Update Installed Skills

Source: https://docs.tenzir.com/guides/ai-workbench/use-agent-skills

Updates all installed skills to their latest versions.

```bash
npx skills update
```

--------------------------------

### Initialize Changelog Project

Source: https://docs.tenzir.com/guides/packages/maintain-a-changelog

Scaffolds a new changelog project structure in the current directory.

```sh
cd my-project
uvx tenzir-ship add \
  --title "Add pipeline builder" \
  --type feature \
  --description "Introduces the new pipeline builder UI." \
  --author alice \
  --pr 101
```

--------------------------------

### Tenzir Project Configuration Example

Source: https://docs.tenzir.com/reference/ship-framework

Example of a config.yaml file for a Tenzir project, specifying project details, components, and release settings.

```yaml
id: tenzir-core
name: Tenzir Core
description: Core pipeline engine
repository: tenzir/tenzir
export_style: compact
explicit_links: true
components:
  cli: Command-line interface and user commands
  engine: Core pipeline engine internals
  operators: Built-in pipeline operators
release:
  commit_message: "Release {version}"
  version_bump_mode: auto
  version_files:
    - ../python/pyproject.toml

```

--------------------------------

### Simulate Real-time Monitoring with Historical Logs

Source: https://docs.tenzir.com/guides/transformation/work-with-time

Replay historical logs as if they are happening now by loading them from a file, shifting their timestamps to the current time, and then replaying them at an accelerated speed.

```tql
// Load historical logs
from_file "/path/to/historical-logs.json"
// Shift timestamps to current time
timeshift timestamp, start=now()
// Replay at 5x speed to quickly review a day's worth of logs
delay timestamp, speed=5.0
// Continue with normal processing
// ...
```

--------------------------------

### Diagnostic note message example

Source: https://docs.tenzir.com/reference/node/api/operations/pipelineupdate

Example of a diagnostic note message.

```text
file <path> [-f|--follow] [-m|--mmap] [-t|--timeout <duration>]
```

--------------------------------

### Example Tenzir Node Configuration

Source: https://docs.tenzir.com/reference/node/configuration

This is a comprehensive example configuration file for Tenzir, showcasing all available options. Options enclosed in angle brackets indicate default values determined at runtime.

```yaml
# This is an example configuration file for Tenzir that shows all available
# options. Options in angle brackets have their default value determined at
# runtime.


```

--------------------------------

### Diagnostic note kind example

Source: https://docs.tenzir.com/reference/node/api/operations/pipelineupdate

Example of a diagnostic note kind.

```text
usage
```

--------------------------------

### Initialize OpenWEC Database

Source: https://docs.tenzir.com/integrations/microsoft/windows-event-logs

Run this command once to initialize the SQLite database for OpenWEC, using the specified configuration file. Ensure the configuration file path is correct.

```bash
openwec -c openwec.conf.toml db init
```
Tokens:
Raw