### Find substring at start Source: https://vector.dev/docs/reference/vrl/examples Finds the starting index of a substring that appears at the beginning of the string. ```VRL find("foobar", "foo") ``` -------------------------------- ### Check for string start with mismatch Source: https://vector.dev/docs/reference/vrl/examples Demonstrates a case-sensitive check where the string does not start with the specified substring. ```VRL starts_with("foobar", "F") ``` -------------------------------- ### Basic Remap Transform Example Source: https://vector.dev/docs/reference/vrl This example demonstrates a basic `remap` transform in a Vector configuration. It deletes the `user_info` field and adds a current timestamp to each event. ```yaml transforms: modify: type: remap inputs: - logs source: | del(.user_info) .timestamp = now() ``` -------------------------------- ### VRL Comment Example Source: https://vector.dev/docs/reference/vrl/expressions VRL comments start with a '#' character. Each line must be preceded by '#'. Block comments are not supported. ```VRL # comment ``` ```VRL # multi-line # comment ``` -------------------------------- ### Basic Filter Transform Example Source: https://vector.dev/docs/reference/vrl This example shows a `filter` transform in Vector configuration that filters out events where the `severity` field is equal to "info". ```yaml transforms: filter_out_info: type: filter inputs: - logs condition: '.severity != "info"' ``` -------------------------------- ### Example Log Event Source: https://vector.dev/docs/reference/vrl A sample Vector log event in JSON format. ```json { "message": "key1=value1 key2=value2" } ``` -------------------------------- ### SCREAMING_SNAKE_CASE conversion example Source: https://vector.dev/docs/reference/vrl/functions Converts a string to SCREAMING_SNAKE_CASE. This example shows a basic usage. ```vector screamingsnakecase("FooBarBaz", "kebab-case") ``` -------------------------------- ### Make a basic HTTP GET request Source: https://vector.dev/docs/reference/vrl/examples Makes an HTTP GET request to a specified URL and parses the JSON response. Useful for fetching data from external APIs. ```VRL . = parse_json!(http_request!("https://httpbin.org/get")) # Redact the origin ip .origin = redact!(.origin, filters: [r'.*'], redactor: {"type": "text", "replacement": "***"}) # Delete any ids in headers .headers = filter(object!(.headers)) -> |key, _value| { !contains(key, "id", false) } . ``` -------------------------------- ### Slice a string from a start index Source: https://vector.dev/docs/reference/vrl/examples Returns a portion of a string from the specified start index to the end of the string. ```VRL slice!("foobar", 3) ``` -------------------------------- ### Simple Block Example Source: https://vector.dev/docs/reference/vrl/expressions A block that defines a message and then parses it. The result of the last expression is returned. ```VRL { message = "{\"Hello\": \"World!\"}" parse_json!(message) } ``` -------------------------------- ### Assert Equal (Successful) Source: https://vector.dev/docs/reference/vrl/examples Asserts that two expressions have the same value. This example shows a successful assertion. ```VRL assert_eq!(1, 1) ``` -------------------------------- ### Check if a string starts with a substring (case insensitive) Source: https://vector.dev/docs/reference/vrl/examples Determines if a string begins with a specified substring, ignoring case. ```VRL starts_with("The Needle In The Haystack", "the needle", case_sensitive: false) ``` -------------------------------- ### VRL Array Push Example Source: https://vector.dev/docs/reference/vrl/examples Adds an item to the end of an array using the `push` function. ```VRL push([1, 2], 3) ``` ```text [1,2,3] ``` ```VRL push([], "bar") ``` ```text ["bar"] ``` -------------------------------- ### Upcase Object Keys Source: https://vector.dev/docs/reference/vrl/examples Maps object keys to their uppercase equivalents. This example demonstrates basic key transformation. ```Vector . = { "foo": "foo", "bar": "bar", "baz": {"nested key": "val"} } map_keys(.) -> |key| { upcase(key) } ``` -------------------------------- ### Assert Equal (Unsuccessful) Source: https://vector.dev/docs/reference/vrl/examples Asserts that two expressions have the same value. This example shows an unsuccessful assertion. ```VRL assert_eq!(127, [1, 2, 3]) ``` -------------------------------- ### Perform a basic DNS lookup Source: https://vector.dev/docs/reference/vrl/examples Executes a DNS lookup for a given domain name. The example shows how to process the results, setting a static TTL and filtering for specific records. ```Vector Reference Language res = dns_lookup!("dns.google") # reset non-static ttl so result is static res.answers = map_values(res.answers) -> |value| { value.ttl = 600 value } # remove extra responses for example res.answers = filter(res.answers) -> |_, value| { value.rData == "8.8.8.8" } # remove class since this is also dynamic res.additional = map_values(res.additional) -> |value| { del(value.class) value } res ``` -------------------------------- ### Compile-time Error Handling Example Source: https://vector.dev/docs/reference/vrl This example demonstrates how VRL's compile-time checks enforce error handling. Passing an incorrect type to `parse_json!` results in a compilation error with a detailed explanation and suggestions for correction. ```VRL parse_json!(1) ``` ```text error[E110]: invalid argument type ┌─ :2:13 │ 2 │ parse_json!(1) │ ^ │ │ │ this expression resolves to the exact type integer │ but the parameter "value" expects the exact type string │ = try: ensuring an appropriate type at runtime = = = 1 = string!(1) = = parse_json!(1) = = = try: coercing to an appropriate type and specifying a default value as a fallback in case coercion fails = = = 1 = to_string(1) ?? "default" = = parse_json!(1) = = = see documentation about error handling at https://errors.vrl.dev/#handling = = learn more about error code 110 at https://errors.vrl.dev/110 = = see language documentation at https://vrl.dev = = try your code in the VRL REPL, learn more at https://vrl.dev/examples = ``` -------------------------------- ### Find using regex Source: https://vector.dev/docs/reference/vrl/examples Locates the starting index of the first match for a regular expression within a string. ```VRL find("foobar", r'b.r') ``` -------------------------------- ### find Source: https://vector.dev/docs/reference/vrl/functions Finds the starting position of the first occurrence of a pattern within a string, with an optional starting offset. ```APIDOC ## find ### Description Determines from left to right the start position of the first found element in `value` that matches `pattern`. Returns `-1` if not found. ### Function Signature find(value: , pattern: , [from: ]) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Argument | Type | Description | Default | Required? | |---|---|---|---|---| | value | string | The string to find the pattern in. | | yes | | pattern | string, regex | The regular expression or string pattern to match against. | | yes | | from | integer | Offset to start searching. | `0` | no | ### Request Example ``` find("foobar", "bar") ``` ### Response #### Success Response Returns the starting index of the first match, or `-1` if not found. #### Response Example ``` 3 ``` ``` -------------------------------- ### Perform DNS lookup with custom class and qtype Source: https://vector.dev/docs/reference/vrl/functions Performs a DNS lookup with specified query class and record type. The example demonstrates how to query for specific DNS record types and classes, with post-processing for static output. ```vector res = dns_lookup!("dns.google", class: "IN", qtype: "A") # reset non-static ttl so result is static res.answers = map_values(res.answers) -> |value| { value.ttl = 600 value } # remove extra responses for example res.answers = filter(res.answers) -> |_, value| { value.rData == "8.8.8.8" } # remove class since this is also dynamic res.additional = map_values(res.additional) -> |value| { del(value.class) value } res ``` -------------------------------- ### Slice an array from a start index Source: https://vector.dev/docs/reference/vrl/examples Returns a portion of an array from the specified start index to the end of the array. ```VRL slice!([0, 1, 2], 1) ``` -------------------------------- ### Error Coalescing Example Source: https://vector.dev/docs/reference/vrl/expressions Demonstrates error coalescing by trying multiple parsing functions until one succeeds or a default is returned. ```VRL parse_syslog("not syslog") ?? parse_common_log("not common") ?? "malformed" ``` -------------------------------- ### Check if a string starts with a substring (case sensitive) Source: https://vector.dev/docs/reference/vrl/examples Determines if a string begins with a specified substring, performing a case-sensitive comparison. ```VRL starts_with("The Needle In The Haystack", "The Needle") ``` -------------------------------- ### Parse Gigabytes to Bytes (Binary) Source: https://vector.dev/docs/reference/vrl/functions Converts gigabytes to bytes using the binary base system. This example shows the difference in conversion when 'base: "2"' is used. ```Vector Script parse_bytes!("1GB", unit: "B", base: "2") ``` -------------------------------- ### Get system hostname Source: https://vector.dev/docs/reference/vrl/examples Retrieves the hostname of the local system. Useful for identifying the source of logs or metrics. ```VRL get_hostname!() ``` -------------------------------- ### get Source: https://vector.dev/docs/reference/vrl/functions Dynamically retrieves the value of a given path. Use this function when the path names are not known in advance. ```APIDOC ## get ### Description Dynamically get the value of a given path. If you know the path you want to look up, use static paths such as `.foo.bar[1]` to get the value of that path. However, if you do not know the path names, use the dynamic `get` function to get the requested value. ### Method `get(value: , path: )` ### Parameters #### Value - **value** (object | array) - Required - The object or array to query. #### Path - **path** (array) - Required - An array of path segments to look for the value. ### Errors The `get` function is **fallible**, which means that error handling is required for these errors: The `path` segment must be a string or an integer. ### Examples ##### Single-segment top-level field ``` get!(value: {"foo": "bar"}, path: ["foo"]) ``` ##### Returns null for unknown field ``` get!(value: {"foo": "bar"}, path: ["baz"]) ``` ##### Multi-segment nested field ``` get!(value: {"foo": { "bar": true }}, path: ["foo", "bar"]) ``` ##### Array indexing ``` get!(value: [92, 42], path: [0]) ``` ##### Array indexing (negative) ``` get!(value: ["foo", "bar", "baz"], path: [-2]) ``` ##### Nested indexing ``` get!(value: {"foo": { "bar": [92, 42] }}, path: ["foo", "bar", 1]) ``` ##### External target ``` { "foo": true } get!(value: ., path: ["foo"]) ``` ##### Variable ``` var = { "foo": true } get!(value: var, path: ["foo"]) ``` ##### Missing index ``` get!(value: {"foo": { "bar": [92, 42] }}, path: ["foo", "bar", 1, -1]) ``` ##### Invalid indexing ``` get!(value: [42], path: ["foo"]) ``` ##### Invalid segment type ``` get!(value: {"foo": { "bar": [92, 42] }}, path: ["foo", true]) ``` ``` -------------------------------- ### Upcase Object Values Source: https://vector.dev/docs/reference/vrl/examples Maps object values to their uppercase equivalents. This example shows a simple value transformation. ```Vector . = { "foo": "foo", "bar": "bar" } map_values(.) -> |value| { upcase(value) } ``` -------------------------------- ### Perform a basic DNS lookup Source: https://vector.dev/docs/reference/vrl/functions Performs a DNS lookup for the A record of a given domain. This function is synchronous and may introduce network latency. The example includes post-processing to make the output static for testing. ```vector res = dns_lookup!("dns.google") # reset non-static ttl so result is static res.answers = map_values(res.answers) -> |value| { value.ttl = 600 value } # remove extra responses for example res.answers = filter(res.answers) -> |_, value| { value.rData == "8.8.8.8" } # remove class since this is also dynamic res.additional = map_values(res.additional) -> |value| { del(value.class) value } res ``` -------------------------------- ### Assignment Block Example Source: https://vector.dev/docs/reference/vrl/expressions A block used within an assignment to parse a JSON string and assign the result to a field. ```VRL .structured = { message = "{\"Hello\": \"World!\"}" parse_json!(message) } ``` -------------------------------- ### HTTP POST Request with Body Source: https://vector.dev/docs/reference/vrl/functions Sends an HTTP POST request with a JSON body to a specified URL. The example demonstrates sending data and includes post-processing for origin IP redaction and header filtering. ```Vector Script . = parse_json!(http_request!("https://httpbin.org/post", method: "post", body: "{\"data\":{\"hello\":\"world\"}}")) # Redact the origin ip .origin = redact!(.origin, filters: [r'.*'], redactor: {"type": "text", "replacement": "***"}) # Delete any ids in headers .headers = filter(object!(.headers)) -> |key, _value| { !contains(key, "id", false) } . ``` ```JSON { "args": {}, "data": "{\"data\":{\"hello\":\"world\"}}", "files": {}, "form": {}, "headers": { "Accept": "*/*", "Content-Length": "26", "Host": "httpbin.org" }, "json": { "data": { "hello": "world" } }, "origin": "***", "url": "https://httpbin.org/post" } ``` -------------------------------- ### Perform DNS lookup with custom options Source: https://vector.dev/docs/reference/vrl/functions Performs a DNS lookup with custom resolver options, such as timeout and retry attempts. This allows tuning the DNS resolution behavior for specific network conditions. The example includes post-processing for static output. ```vector res = dns_lookup!("dns.google", options: {"timeout": 30, "attempts": 5}) res.answers = map_values(res.answers) -> |value| { value.ttl = 600 value } # remove extra responses for example res.answers = filter(res.answers) -> |_, value| { value.rData == "8.8.8.8" } ``` -------------------------------- ### Multiple Parsing Strategies Source: https://vector.dev/docs/reference/vrl/examples Demonstrates how to apply multiple parsing strategies sequentially, falling back to the next if the previous one fails. This is useful for handling logs that might conform to different formats. ```json { "message": "\u003c102\u003e1 2020-12-22T15:22:31.111Z vector-user.biz su 2666 ID389 - Something went wrong" } ``` ```VRL structured = parse_syslog(.message) ?? parse_common_log(.message) ?? parse_regex!(.message, r'^(?P\d+/\d+/\d+ \d+:\d+:\d+) \[(?P\w+) \[(?P\d+)#(?P\d+):(?: \*(?P\d+))? (?P.*)$') . = merge(., structured) ``` ```json { "log": { "appname": "su", "facility": "ntp", "hostname": "vector-user.biz", "message": "Something went wrong", "msgid": "ID389", "procid": 2666, "severity": "info", "timestamp": "2020-12-22T15:22:31.111Z", "version": 1 } } ``` -------------------------------- ### Get an element by negative array index Source: https://vector.dev/docs/reference/vrl/examples Use negative indices in the path array to access elements from the end of an array. For example, -1 refers to the last element. ```Vector get!(value: ["foo", "bar", "baz"], path: [-2]) ``` -------------------------------- ### VRL Array Chunks Example Source: https://vector.dev/docs/reference/vrl/examples Splits a string into fixed-size chunks. Note that this function does not respect Unicode code point boundaries. ```VRL chunks("abcdefgh", 4) ``` ```text ["abcd","efgh"] ``` ```VRL chunks("ab你好", 4) ``` ```text ["ab�","�好"] ``` -------------------------------- ### starts_with Source: https://vector.dev/docs/reference/vrl/functions Determines whether a string begins with a specified substring, with an option for case-sensitive comparison. ```APIDOC ## starts_with ### Description Determines whether `value` begins with `substring`. The comparison can be case-sensitive or case-insensitive. ### Method ``` starts_with(value: , substring: , [case_sensitive: ]) ``` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **value** (string) - Required - The string to search. - **substring** (string) - Required - The substring that the `value` must start with. - **case_sensitive** (boolean) - Optional - Whether the match should be case sensitive. Defaults to `true`. ### Request Example ``` starts_with("The Needle In The Haystack", "The Needle") starts_with("The Needle In The Haystack", "the needle", case_sensitive: false) starts_with("foobar", "F") ``` ### Response #### Success Response (200) - **return value** (boolean) - `true` if the string starts with the substring, `false` otherwise. #### Response Example ```json true true false ``` ``` -------------------------------- ### Get from a variable Source: https://vector.dev/docs/reference/vrl/examples Retrieve values from a variable by passing the variable name as the `value` argument to `get!`. ```Vector var = { "foo": true } get!(value: var, path: ["foo"]) ``` -------------------------------- ### String does not match regex Source: https://vector.dev/docs/reference/vrl/examples Demonstrates a case where a string does not match the provided regular expression. ```VRL match("I'm a little teapot", r'.*balloon') ``` -------------------------------- ### Parse Bytes to Terabytes (SI) Source: https://vector.dev/docs/reference/vrl/functions Converts a byte string to terabytes using the SI (decimal) base system. This example highlights the use of 'base: "10"' for decimal conversions. ```Vector Script parse_bytes!("4TB", unit: "MB", base: "10") ``` -------------------------------- ### Assert Equal (Unsuccessful) with Custom Message Source: https://vector.dev/docs/reference/vrl/examples Asserts that two expressions have the same value and aborts with a custom message if they do not. This example shows an unsuccessful assertion with a message. ```VRL assert_eq!(1, 0, message: "Unequal integers") ``` -------------------------------- ### Get from an external target Source: https://vector.dev/docs/reference/vrl/examples The `get!` function can operate on the current event context (`.`) by specifying it as the `value` argument. ```Vector get!(value: ., path: ["foo"]) ``` -------------------------------- ### Invalid Segment Type Example Source: https://vector.dev/docs/reference/vrl/examples This example demonstrates an invalid usage of `set!` where a boolean is used as a path segment, which is not permitted. ```vector set!({"foo": { "bar": [92, 42] }}, ["foo", true], "baz") ``` -------------------------------- ### Get returns null for an unknown field Source: https://vector.dev/docs/reference/vrl/examples If the specified path does not exist in the value, the `get!` function returns `null` without error. ```Vector get!(value: {"foo": "bar"}, path: ["baz"]) ``` -------------------------------- ### Parse AWS VPC Flow Log (v5 Fields with Format) Source: https://vector.dev/docs/reference/vrl/functions Parses an AWS VPC Flow log string, including v5 fields, using a specified format string. This example demonstrates parsing a more detailed log entry with explicit field mapping. ```vector parse_aws_vpc_flow_log( "5 52.95.128.179 10.0.0.71 80 34210 6 1616729292 1616729349 IPv4 14 15044 123456789012 vpc-abcdefab012345678 subnet-aaaaaaaa012345678 i-0c50d5961bcb2d47b eni-1235b8ca123456789 ap-southeast-2 apse2-az3 - - ACCEPT 19 52.95.128.179 10.0.0.71 S3 - - ingress OK", format: "version srcaddr dstaddr srcport dstport protocol start end type packets bytes account_id vpc_id subnet_id instance_id interface_id region az_id sublocation_type sublocation_id action tcp_flags pkt_srcaddr pkt_dstaddr pkt_src_aws_service pkt_dst_aws_service traffic_path flow_direction log_status" ) ``` -------------------------------- ### Encode to Snappy and then Base64 Source: https://vector.dev/docs/reference/vrl/examples Encodes a string to Snappy format and then encodes the result to Base64. ```VRL encode_base64(encode_snappy!("The quick brown fox jumps over 13 lazy dogs.")) ``` -------------------------------- ### Encode to Zlib and then Base64 Source: https://vector.dev/docs/reference/vrl/examples Encodes a string to Zlib format and then encodes the result to Base64. ```VRL encode_base64(encode_zlib("please encode me")) ``` -------------------------------- ### Get a single-segment top-level field Source: https://vector.dev/docs/reference/vrl/examples Use the `get!` function to retrieve a value from a top-level field using a path array. This is useful when the field name is determined dynamically. ```Vector get!(value: {"foo": "bar"}, path: ["foo"]) ``` -------------------------------- ### Basic HTTP GET Request Source: https://vector.dev/docs/reference/vrl/functions Performs a basic HTTP GET request to a specified URL and parses the JSON response. It also demonstrates redacting sensitive information like origin IP and filtering headers. ```Vector Script . = parse_json!(http_request!("https://httpbin.org/get")) # Redact the origin ip .origin = redact!(.origin, filters: [r'.*'], redactor: {"type": "text", "replacement": "***"}) # Delete any ids in headers .headers = filter(object!(.headers)) -> |key, _value| { !contains(key, "id", false) } . ``` ```JSON { "args": {}, "headers": { "Accept": "*/*", "Host": "httpbin.org" }, "origin": "***", "url": "https://httpbin.org/get" } ``` -------------------------------- ### Get length of an array Source: https://vector.dev/docs/reference/vrl/examples Returns the number of elements in an array. ```VRL length(["Trail Blazers", "Supersonics", "Grizzlies"]) ``` -------------------------------- ### Get length of an object Source: https://vector.dev/docs/reference/vrl/examples Returns the number of top-level keys in an object. ```VRL length({ "portland": "Trail Blazers", "seattle": "Supersonics" }) ``` -------------------------------- ### Get keys from an object Source: https://vector.dev/docs/reference/vrl/examples Retrieves an array of all top-level keys from an object. ```VRL keys({ "key1": "val1", "key2": "val2" }) ``` -------------------------------- ### Find with an offset Source: https://vector.dev/docs/reference/vrl/examples Searches for a substring starting from a specified index within the string. ```VRL find("foobarfoobarfoo", "bar", 4) ``` -------------------------------- ### Get length of a string Source: https://vector.dev/docs/reference/vrl/functions Use the `length` function to determine the number of bytes in a string. ```Vector length("The Planet of the Apes Musical") ``` -------------------------------- ### Split an empty path Source: https://vector.dev/docs/reference/vrl/examples Splitting an empty path results in an empty array. ```VRL split_path("") ``` -------------------------------- ### Find substring position Source: https://vector.dev/docs/reference/vrl/examples Finds the starting index of the first occurrence of a substring within a string. ```VRL find("foobar", "bar") ``` -------------------------------- ### Basic Regular Expression Source: https://vector.dev/docs/reference/vrl/expressions A basic regular expression literal, anchored to the start and end of the string. ```VRL r'^Hello, World!$' ``` -------------------------------- ### Encrypt using AES-128-CBC-PKCS7 Source: https://vector.dev/docs/reference/vrl/examples Encrypts a string using the AES-128-CBC-PKCS7 algorithm. Requires a 16-byte key and a 16-byte IV. The output is base64 encoded. ```VRL iv = "1234567890123456" # typically you would call random_bytes(16) key = "16_byte_keyxxxxx" encrypted_message = encrypt!("super secret message", "AES-128-CBC-PKCS7", key: key, iv: iv) encode_base64(encrypted_message) ``` -------------------------------- ### PascalCase specifying the wrong original case Source: https://vector.dev/docs/reference/vrl/examples Demonstrates PascalCasing when the original case is incorrectly specified, resulting in only the first letter being capitalized. ```vector pascalcase("foo_bar_baz", "kebab-case") ``` -------------------------------- ### Fix Missing Required Function Argument Source: https://vector.dev/docs/reference/vrl/errors Supply all required arguments to a function call to match its documented signature. ```VRL -parse_timestamp(.timestamp) +parse_timestamp(.timestamp, format: "%D") ``` -------------------------------- ### Get length of an object Source: https://vector.dev/docs/reference/vrl/functions Use the `length` function to determine the number of top-level keys in an object. ```Vector length({ "portland": "Trail Blazers", "seattle": "Supersonics" }) ``` -------------------------------- ### Get keys from an object Source: https://vector.dev/docs/reference/vrl/functions Use the `keys` function to extract all top-level keys from an object into an array. ```Vector keys({ "key1": "val1", "key2": "val2" }) ``` -------------------------------- ### Root directory dirname is itself Source: https://vector.dev/docs/reference/vrl/examples Demonstrates that the dirname of the root directory is the root directory itself. ```VRL dirname!("/") ``` -------------------------------- ### Extract dirname from directory path Source: https://vector.dev/docs/reference/vrl/examples Gets the parent directory name when the input is a directory path. ```VRL dirname!("/home/user/") ``` -------------------------------- ### Root Event Path Source: https://vector.dev/docs/reference/vrl/expressions Selects the entire event as the result. Use when you need to process the whole event object. ```VRL . ``` -------------------------------- ### Get length of a nested object Source: https://vector.dev/docs/reference/vrl/examples Returns the number of top-level keys in a nested object structure. ```VRL length({ "home": { "city": "Portland", "state": "Oregon" }, "name": "Trail Blazers", "mascot": { "name": "Blaze the Trail Cat" } }) ``` -------------------------------- ### Encode String to Base64 (Default) Source: https://vector.dev/docs/reference/vrl/examples Encodes a string to its Base64 representation using default settings, including padding. ```VRL encode_base64("please encode me") ``` -------------------------------- ### VRL Array Pop Example Source: https://vector.dev/docs/reference/vrl/examples Removes and returns the last item from an array using the `pop` function. ```VRL pop([1, 2, 3]) ``` ```text [1,2] ``` -------------------------------- ### VRL Array Append Example Source: https://vector.dev/docs/reference/vrl/examples Appends items from one array to another using the `append` function. ```VRL append([1, 2], [3, 4]) ``` ```text [1,2,3,4] ``` -------------------------------- ### Encode to Zstd and then Base64 Source: https://vector.dev/docs/reference/vrl/examples Encodes a string to Zstandard format and then encodes the result to Base64. ```VRL encode_base64(encode_zstd("please encode me")) ``` -------------------------------- ### Parse simple query string Source: https://vector.dev/docs/reference/vrl/examples Parses a basic query string into a key-value map. Each key maps to a single value. ```vrl parse_query_string("foo=1&bar=2") ``` -------------------------------- ### Log Message (Info) Source: https://vector.dev/docs/reference/vrl/examples Logs a message to stdout at the 'info' level with a specified rate limit. ```VRL log("Hello, World!", level: "info", rate_limit_secs: 60) ``` -------------------------------- ### Extract IPv4 subnet with /1 prefix Source: https://vector.dev/docs/reference/vrl/examples Demonstrates extracting an IPv4 subnet using `ip_subnet!` with a prefix length of /1. ```vector ip_subnet!("192.168.0.1", "/1") ``` -------------------------------- ### Get with a missing index Source: https://vector.dev/docs/reference/vrl/examples Accessing a path that includes an index beyond the bounds of an array results in `null`. ```Vector get!(value: {"foo": { "bar": [92, 42] }}, path: ["foo", "bar", 1, -1]) ``` -------------------------------- ### Make an HTTP PUT request Source: https://vector.dev/docs/reference/vrl/examples Makes an HTTP PUT request to a specified URL. Use this when you need to update or replace a resource on a server. ```VRL . = parse_json!(http_request!("https://httpbin.org/put", method: "put")) # Redact the origin ip .origin = redact!(.origin, filters: [r'.*'], redactor: {"type": "text", "replacement": "***"}) # Delete any ids in headers .headers = filter(object!(.headers)) -> |key, _value| { !contains(key, "id", false) } . ``` -------------------------------- ### Slice a string with positive indices Source: https://vector.dev/docs/reference/vrl/examples Returns a portion of a string between specified start and end positions. Indices are zero-based. ```VRL slice!("Supercalifragilisticexpialidocious", start: 5, end: 13) ``` -------------------------------- ### Check for Non-Empty String Source: https://vector.dev/docs/reference/vrl/examples Demonstrates checking a non-empty string with `is_empty`, which will return false. ```Vector is_empty("a string") ``` -------------------------------- ### Parse Nginx main log format Source: https://vector.dev/docs/reference/vrl/examples Parses a string according to the Nginx 'main' log format. This format includes client IP, user, timestamp, request details, status code, size, and referer. ```vrl parse_nginx_log!( s'172.24.0.3 - alice [31/Dec/2024:17:32:06 +0000] "GET / HTTP/1.1" 200 615 "https://domain.tld/path" "curl/8.11.1" "1.2.3.4, 10.10.1.1"', "main" ) ``` -------------------------------- ### Round-trip IP encryption and decryption Source: https://vector.dev/docs/reference/vrl/examples Demonstrates encrypting an IP address and then decrypting it using the same key and mode. ```VRL original_ip = "192.168.1.100" key = "sixteen byte key" mode = "aes128" encrypted = encrypt_ip!(original_ip, key, mode) decrypt_ip!(encrypted, key, mode) ``` -------------------------------- ### Integer Literal with Underscores Source: https://vector.dev/docs/reference/vrl/expressions Integer literals can use underscores for readability. This example shows a large integer value. ```VRL 1_000_000 ``` -------------------------------- ### Float Literal with Underscores Source: https://vector.dev/docs/reference/vrl/expressions Float literals can use underscores for readability. This example shows a large float value. ```VRL 1_000_000.01 ``` -------------------------------- ### Equal Integer and Float Comparison Source: https://vector.dev/docs/reference/vrl/expressions Compares an integer and a float for equality. ```VRL 1 == 1.0 ``` -------------------------------- ### Function Call with Abort Source: https://vector.dev/docs/reference/vrl/expressions Use the '!' suffix with fallible functions to abort the program on failure. Otherwise, errors must be handled explicitly. ```VRL result = f!() ``` ```VRL result, err = f() ``` -------------------------------- ### Nested Array Literal Source: https://vector.dev/docs/reference/vrl/expressions Arrays can be nested to create multi-dimensional structures. This example shows three levels of nesting. ```VRL ["first-level", ["second-level", ["third-level"]] ``` -------------------------------- ### Newline Delimited Expressions Source: https://vector.dev/docs/reference/vrl/expressions Expressions in VRL can be separated by newlines. This example shows multiple expressions on separate lines. ```VRL # newline delimited expressions del(.user_info) .timestamp = now() .message = "hello world" ``` -------------------------------- ### Get Current Timestamp Source: https://vector.dev/docs/reference/vrl/functions Retrieves the current timestamp in UTC with nanosecond precision. This function is infallible and pure. ```Vector Script now() ``` ```timestamp 2012-03-04T12:34:56.789012345Z ``` -------------------------------- ### Get with invalid indexing Source: https://vector.dev/docs/reference/vrl/examples Attempting to access an array element using a non-integer path segment results in `null`. ```Vector get!(value: [42], path: ["foo"]) ``` -------------------------------- ### Parse gigabytes to bytes (decimal units) Source: https://vector.dev/docs/reference/vrl/examples Converts a string representing gigabytes into bytes using decimal units. ```VRL parse_bytes!("1GB", unit: "B", base: "10") ``` -------------------------------- ### Attempt to get a non-existent secret Source: https://vector.dev/docs/reference/vrl/examples Calling `get_secret` with a name that does not correspond to any available secret will result in a null value. ```VRL get_secret("i_dont_exist") ``` -------------------------------- ### Encrypt using AES-256-CFB Source: https://vector.dev/docs/reference/vrl/examples Encrypts a string using the AES-256-CFB algorithm. Requires a 32-byte key and a 16-byte IV. The output is base64 encoded. ```VRL iv = "0123456789012345" # typically you would call random_bytes(16) key = "01234567890123456789012345678912" encrypted_message = encrypt!("data", "AES-256-CFB", key: key, iv: iv) encode_base64(encrypted_message) ``` -------------------------------- ### Find with no matches Source: https://vector.dev/docs/reference/vrl/examples Returns -1 when the specified substring or pattern is not found in the string. ```VRL find("foobar", "baz") ``` -------------------------------- ### Get String Length Source: https://vector.dev/docs/reference/vrl/examples Calculates the number of bytes in a string. Use `strlen` for Unicode scalar value counts. ```Vector length("The Planet of the Apes Musical") ``` -------------------------------- ### Assert Condition (False) with Message Source: https://vector.dev/docs/reference/vrl/examples Asserts a boolean condition and aborts with a message if false. This example shows an unsuccessful assertion. ```VRL assert!("foo" == "bar", message: "\"foo\" must be \"foo\"!") ``` -------------------------------- ### Parse gigabytes to megabytes (ambiguous decimal units) Source: https://vector.dev/docs/reference/vrl/examples Converts a string representing gigabytes into megabytes using decimal units. ```VRL parse_bytes!("1GB", unit: "MB", base: "2") ``` -------------------------------- ### Convert IPv6 address string to bytes and encode to Base64 Source: https://vector.dev/docs/reference/vrl/examples Converts an IPv6 address string to its binary byte representation using `ip_pton` and then encodes the result to Base64. ```vector encode_base64(ip_pton!("2001:db8:85a3::8a2e:370:7334")) ``` -------------------------------- ### Assert Condition (True) with Message Source: https://vector.dev/docs/reference/vrl/examples Asserts a boolean condition and aborts with a message if false. This example shows a successful assertion. ```VRL assert!("foo" == "foo", message: "\"foo\" must be \"foo\"!") ``` -------------------------------- ### Root Metadata Path Source: https://vector.dev/docs/reference/vrl/expressions Selects the event metadata as the result. Use when accessing metadata associated with the event. ```VRL % ``` -------------------------------- ### Replace first instance with processed string using replace_with Source: https://vector.dev/docs/reference/vrl/functions Demonstrates replacing only the first match of a pattern using `replace_with` and a closure that processes the matched string. ```vector replace_with("Apples and Apples", r'(?i)apples|cones', count: 1) -> |match| { "Pine" + downcase(match.string) } ``` -------------------------------- ### Semicolon Delimited Expressions Source: https://vector.dev/docs/reference/vrl/expressions Expressions in VRL can also be separated by semicolons. This example demonstrates multiple expressions on a single line. ```VRL # semicolon delimited expressions del(.user_info); .timestamp = now() .message = "hello world" ``` -------------------------------- ### Get length of a nested object Source: https://vector.dev/docs/reference/vrl/functions Use the `length` function to count the number of top-level keys in a nested object structure. ```Vector length({ "home": { "city": "Portland", "state": "Oregon" }, "name": "Trail Blazers", "mascot": { "name": "Blaze the Trail Cat" } }) ``` -------------------------------- ### No match for any pattern Source: https://vector.dev/docs/reference/vrl/examples Checks if a string matches any pattern in a list, returning false if none match. ```VRL match_any("My name is John Doe", patterns: [r'\d+', r'Jane']) ``` -------------------------------- ### Get an element by array index Source: https://vector.dev/docs/reference/vrl/examples Access elements within an array using their index in the path array. Indices are zero-based. ```Vector get!(value: [92, 42], path: [0]) ``` -------------------------------- ### Variable Assignment and Comparison Source: https://vector.dev/docs/reference/vrl/expressions Assigns a value to a variable and then compares it. Variables must start with a letter and can contain alphanumeric characters and underscores. ```VRL my_variable = 1 my_variable == 1 ``` -------------------------------- ### Parse using aliases from file Source: https://vector.dev/docs/reference/vrl/functions Parses a string using Grok patterns loaded from a file, allowing for reusable and complex pattern definitions. Ensure the alias file path is correct. ```vector parse_groks( "username=foo", patterns: [ "%{PATTERN_A}" ], alias_sources: [ "tests/data/grok/aliases.json" ] ) # aliases.json contents: # { # "PATTERN_A": "%{PATTERN_B}", # "PATTERN_B": "username=%{USERNAME:username}" ``` -------------------------------- ### Declare Integer Type from Field Source: https://vector.dev/docs/reference/vrl/examples Ensures a value from a data structure is an integer. This example first assigns an integer to a field and then validates it. ```Vector . = { "value": 42 } int(.value) ``` -------------------------------- ### Parse Nginx Main Log Format Source: https://vector.dev/docs/reference/vrl/functions Parses a string using the Nginx 'main' log format. This is a common format including client IP, user, timestamp, request, status, size, referer, user agent, and forwarded-for. ```vector parse_nginx_log!( s'172.24.0.3 - alice [31/Dec/2024:17:32:06 +0000] "GET / HTTP/1.1" 200 615 "https://domain.tld/path" "curl/8.11.1" "1.2.3.4, 10.10.1.1"', "main" ) ``` -------------------------------- ### Handle Runtime Errors with Assignment in VRL Source: https://vector.dev/docs/reference/vrl/errors Demonstrates assigning potential errors from fallible expressions to variables for handling. ```VRL structured, err = parse_json("not json") if err != null { log("Unable to parse JSON: " + err, level: "error") } else { . = merge(., structured) } ``` ```VRL # ".foo` can be `100` or `"not an int"` foo, err = to_int(.foo) ``` -------------------------------- ### Declare Float Type from Field Source: https://vector.dev/docs/reference/vrl/examples Ensures a value from a data structure is a float. This example first assigns a float to a field and then validates it. ```Vector . = { "value": 42.0 } float(.value) ``` -------------------------------- ### ip_pton Source: https://vector.dev/docs/reference/vrl/functions Converts IPv4 and IPv6 addresses from text form to their binary representation. ```APIDOC ## ip_pton ### Description Converts IPv4 and IPv6 addresses from text form to their binary representation. The binary form of IPv4 addresses is 4 bytes (32 bits) long, and for IPv6 addresses, it is 16 bytes (128 bits) long. This behavior mimics the `inet_pton` function. ### Function Signature ip_pton(value: ) :: , ### Parameters #### value - Type: string - Description: The IP address (v4 or v6) to convert to binary form. - Required: yes ### Notices The binary data from this function is not easily printable. However, functions such as `encode_base64` or `encode_percent` can still process it correctly. ### Errors - `value` is not a valid IP (v4 or v6) address in text form. ### Examples ##### Convert IPv4 address to bytes and encode to Base64 ``` encode_base64(ip_pton!("192.168.0.1")) ``` ##### Convert IPv6 address to bytes and encode to Base64 ``` encode_base64(ip_pton!("2001:db8:85a3::8a2e:370:7334")) ``` ``` -------------------------------- ### Infallible Function Invocation Source: https://vector.dev/docs/reference/vrl/expressions Demonstrates calling an infallible function 'split' with standard arguments, which is guaranteed not to fail at runtime. ```VRL split("apples and pears and bananas", " and ") ``` -------------------------------- ### Object Literal with JSON Structure Source: https://vector.dev/docs/reference/vrl/expressions Object literals are key/value structures similar to JSON. This example shows nested objects and arrays. ```VRL { "field1": "value1", "field2": [ "value2", "value3", "value4" ], "field3": { "field4": "value5" } } ``` -------------------------------- ### Filter Array by Value Source: https://vector.dev/docs/reference/vrl/functions Filters elements from an array based on a condition in a closure that checks the value. This example keeps elements less than 2. ```vector filter([1, 2]) -> |_index, value| { value < 2 } ``` -------------------------------- ### Calculate Negative Seahash Source: https://vector.dev/docs/reference/vrl/examples Demonstrates a Seahash calculation that results in a negative value due to the conversion from unsigned to signed 64-bit integers. ```VRL seahash("bar") ``` -------------------------------- ### Filter Object by Key Source: https://vector.dev/docs/reference/vrl/functions Filters key-value pairs from an object based on a condition in a closure that checks the key. This example keeps only the pair where the key is 'a'. ```vector filter({ "a": 1, "b": 2 }) -> |key, _value| { key == "a" } ``` -------------------------------- ### Parse kilobytes to bytes (binary units) Source: https://vector.dev/docs/reference/vrl/examples Converts a string representing kilobytes into bytes using binary units. ```VRL parse_bytes!("1KiB", unit: "B") ```