### Install rustfmt for Formatting Checks (Bash) Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Instructions to install the `rustfmt` component for Rust, which is necessary for code formatting checks before submitting pull requests. ```bash rustup component add rustfmt ``` -------------------------------- ### Install Angle Grinder via Package Managers Source: https://context7.com/rcoh/angle-grinder/llms.txt This section details how to install Angle Grinder using various package managers like Homebrew, MacPorts, pkg, and Cargo. It also provides instructions for manual installation on Linux systems using curl and tar. ```bash # macOS with Homebrew brew install angle-grinder # macOS with MacPorts sudo port selfupdate sudo port install angle-grinder # FreeBSD pkg install angle-grinder # Cargo (Rust package manager - works on most platforms) car go install ag # Linux (MUSL compatible) curl -L https://github.com/rcoh/angle-grinder/releases/download/v0.18.0/agrind-x86_64-unknown-linux-musl.tar.gz \ | tar Ozxf - \ | sudo tee /usr/local/bin/agrind > /dev/null && sudo chmod +x /usr/local/bin/agrind agrind --self-update ``` -------------------------------- ### Build and Install Angle Grinder (Bash) Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Commands to build, test, install, and run the Angle Grinder CLI. Includes formatting checks required for pull requests. ```bash cargo build cargo test cargo install --path . agrind --help # Required for PRs cargo fmt ``` -------------------------------- ### Count Downloads by Release using jq and agrind Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example demonstrates how to count downloads for angle-grinder releases by piping GitHub API data through `jq` for initial processing and then using `agrind` for aggregation and sorting. ```bash curl https://api.github.com/repos/rcoh/angle-grinder/releases | \ jq '.[] | .assets | .[]' -c | \ agrind '* | json \ | parse "download/*/" from browser_download_url as version \ | sum(download_count) by version | sort by version desc' ``` -------------------------------- ### Aggregate Operator: Count by source host Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example demonstrates using the 'count' aggregate operator with a 'by' clause to count the number of rows for each unique 'source_host'. ```agrind * | count by source_host ``` -------------------------------- ### Install Angle Grinder on Linux (MUSL) Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Downloads and installs the angle-grinder binary for MUSL-compatible Linux systems. It fetches the latest release, extracts it, and places the executable in the system's PATH. ```bash curl -L https://github.com/rcoh/angle-grinder/releases/download/v0.18.0/agrind-x86_64-unknown-linux-musl.tar.gz \ | tar Ozxf - \ | sudo tee /usr/local/bin/agrind > /dev/null && sudo chmod +x /usr/local/bin/agrind agrind --self-update ``` -------------------------------- ### Install Angle Grinder on FreeBSD Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Installs the angle-grinder command-line tool on FreeBSD systems using the pkg package manager. This command ensures the tool is available for log analysis on FreeBSD. ```bash pkg install angle-grinder ``` -------------------------------- ### Basic Angle Grinder Query Structure Source: https://context7.com/rcoh/angle-grinder/llms.txt Demonstrates the fundamental structure of Angle Grinder queries, which involve filters followed by operators connected by pipes. It shows examples of matching all lines, counting, parsing JSON, and processing real-time log streams. ```bash # Basic query structure agrind ' | operator1 | operator2 | ...' # Match all lines and count agrind '* | count' # Parse JSON logs and count by log level agrind '* | json | count by log_level' # Process a file cat access.log | agrind '* | json | count by status_code' # Stream logs in real-time tail -F /var/log/app.log | agrind '* | json | p50(response_time) by endpoint' ``` -------------------------------- ### Install Angle Grinder via Homebrew (macOS) Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Installs the angle-grinder command-line tool using the Homebrew package manager on macOS. This is a convenient way to get the tool set up for users of Homebrew. ```bash brew install angle-grinder ``` -------------------------------- ### Angle Grinder Alias Configuration Example Source: https://github.com/rcoh/angle-grinder/blob/main/README.md An example TOML file defining an alias for parsing Apache web server logs. This alias simplifies common parsing tasks by providing a shorthand for a complex parse operator. ```toml keyword = "apache" template = """ parse "* - * [*] \"* * *\" * *" as ip, name, timestamp, method, url, protocol, status, contentlength """ ``` -------------------------------- ### Aggregate Operator: Minimum response time Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example demonstrates the 'min' aggregate operator to find the minimum value in the 'response_time' column. Non-numeric values are ignored. The result is a single column with the minimum value. ```agrind * | json | min(response_time) ``` -------------------------------- ### Aggregate Operators: Renamed count and percentiles by status code Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example shows how to use the 'as' keyword to rename the output of the 'count' aggregate operator to 'num_requests', while also calculating 'p50' and 'p90' of 'response_ms', all grouped by 'status_code'. ```agrind * | json | count as num_requests, p50(response_ms), p90(response_ms) by status_code ``` -------------------------------- ### Install Angle Grinder via Cargo (Rust) Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Compiles and installs the angle-grinder command-line tool from source using Cargo, Rust's package manager. This method works on most platforms with a compatible Rust toolchain. ```bash cargo install ag ``` -------------------------------- ### Aggregate Operator: Maximum response time Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example demonstrates the 'max' aggregate operator to find the maximum value in the 'response_time' column. Non-numeric values are ignored. The result is a single column with the maximum value. ```agrind * | json | max(response_time) ``` -------------------------------- ### Angle Grinder Query with Combined Filters Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Demonstrates combining multiple filters in angle-grinder using logical operators AND, OR, and NOT. This query counts logs that are 'ERROR' or start with 'WARN' but are not from the 'staging' environment. ```agrind ("ERROR" OR WARN*) AND NOT staging | count ``` -------------------------------- ### Install Angle Grinder via Macports (macOS) Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Installs the angle-grinder command-line tool using the Macports package manager on macOS. This method is an alternative for users who prefer Macports over Homebrew. ```bash sudo port selfupdate sudo port install angle-grinder ``` -------------------------------- ### Angle Grinder Query Using an Alias Source: https://github.com/rcoh/angle-grinder/blob/main/README.md An example of using a pre-defined alias in angle-grinder to process Apache logs. The `apache` alias expands to a specific parsing pipeline, simplifying the command. ```bash * | apache | count by status ``` -------------------------------- ### Angle Grinder Logfmt Parsing Source: https://context7.com/rcoh/angle-grinder/llms.txt Explains the `logfmt` operator for parsing logfmt-formatted data (key=value pairs), commonly used with Heroku and Splunk logs. Examples include basic parsing, parsing from a specific field, and combining with other operators like `sum`. ```bash # Basic logfmt parsing echo 'level=info msg="request completed" duration=45ms' | agrind '* | logfmt | count by level' # Parse logfmt from a specific field echo '{"data": "status=200 method=GET"}' | agrind '* | json | logfmt from data | count by status' # Combine with other operators echo 'host=web-01 status=200 bytes=1024' | agrind '* | logfmt | sum(bytes) by host' ``` -------------------------------- ### Conditional Field: Byte counts for successful requests Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example shows the usage of the 'if' operator to conditionally select a value. It checks if the 'status' is 200 and assigns 'sc_bytes' if true, otherwise assigns 0, storing the result in 'ok_bytes'. ```agrind * | json | if(status == 200, sc_bytes, 0) as ok_bytes ``` -------------------------------- ### Aggregate Operators: Count info and error logs Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example demonstrates using conditional counting within the 'count' aggregate operator to specifically count logs where 'level' is 'info' and 'error', aliasing them as 'info_logs' and 'error_logs' respectively. ```agrind * | json | count(level == "info") as info_logs, count(level == "error") as error_logs ``` -------------------------------- ### Aggregate Operator: Count by status code Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example shows how to use the 'count' aggregate operator with a 'by' clause to group rows by 'status_code' and count the occurrences within each group. ```agrind * | json | count by status_code ``` -------------------------------- ### Field Expression: Multiply value by 100 Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This example demonstrates how to use a field expression to multiply a 'value' field by 100 and store the result in a new field named 'percentage'. It utilizes the '*' operator for multiplication. ```agrind * | json | value * 100 as percentage ``` -------------------------------- ### Limit Output Rows with 'limit' Operator Source: https://context7.com/rcoh/angle-grinder/llms.txt The 'limit' operator restricts the number of rows returned in the output. A positive number returns the first N rows, while a negative number returns the last N rows. It is often used in conjunction with 'sort' to get the top or bottom N results. ```bash agrind '* | json | limit 10' agrind '* | json | limit -10' agrind '* | json | count by endpoint | sort by _count desc | limit 5' ``` -------------------------------- ### Angle Grinder Query with Escaped Field Names Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Example of an angle-grinder query that counts logs by fields containing special characters or spaces. It uses bracket notation to properly escape field names like 'date received' and 'grpc.method'. ```bash agrind '* | json | count by ["date received"], ["grpc.method"]' ``` -------------------------------- ### Create and Use Aliases for Parsing Source: https://context7.com/rcoh/angle-grinder/llms.txt Aliases provide pre-defined parsing pipelines stored in `.agrind-aliases` directories. They simplify complex parsing tasks by allowing users to refer to them by a keyword. ```toml # File: .agrind-aliases/apache.toml keyword = "apache" template = """ parse "* - * [*] \"* * *\" * *" as ip, name, timestamp, method, url, protocol, status, contentlength """ ``` ```bash # Use the alias in queries agrind '* | apache | count by status' # Aliases can be chained with other operators agrind '* | apache | where status >= 400 | count by url' # Built-in aliases include: apache, nginx, k8s-ingress-nginx agrind '* | nginx | p50(request_time) by upstream_addr' ``` -------------------------------- ### Count Downloads by Release Version Source: https://context7.com/rcoh/angle-grinder/llms.txt This query counts download occurrences for each release version of a project from the GitHub API. It uses `curl` to fetch data, `jq` for JSON processing, and `agrind` for parsing and aggregation. ```bash curl https://api.github.com/repos/rcoh/angle-grinder/releases | \ jq '.[] | .assets | .[]' -c | \ agrind '* | json | parse "download/*/" from browser_download_url as version | sum(download_count) by version | sort by version desc' ``` -------------------------------- ### Calculate Response Time Percentiles by Endpoint Source: https://context7.com/rcoh/angle-grinder/llms.txt Calculates P50, P90, and P99 response times for each endpoint and status code from JSON logs. This helps in identifying performance bottlenecks and error patterns. ```bash tail -F /var/log/app.log | agrind '* | json | p50(response_time), p90(response_time), p99(response_time) by endpoint_url, status_code' ``` -------------------------------- ### Control Output Formats Source: https://context7.com/rcoh/angle-grinder/llms.txt Control the output format of the angle-grinder tool using the `--output` flag. Supported formats include default table, JSON, and logfmt. A custom format string can also be specified. ```bash # Default table output (live-updating for aggregates) tail -f app.log | agrind '* | json | count by status' # JSON output agrind --output json '* | json | count by status' < app.log # Logfmt output agrind --output logfmt '* | json | count by status' < app.log # Custom format string tail -f pcap.log | agrind --format '{src} => {dest} | length={length}' '* | parse "* > *:" as src, dest | parse "length *" as length' ``` -------------------------------- ### Split String Data with 'split' Command Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'split' command splits input data based on a specified separator (defaulting to ','). The output is an array. If no input or output field is specified, the split data is placed in the '_split' key. It can override the input field if no new field is specified. ```agrind * | split on " " ``` ```agrind * | parse "* *" as level, csv | split(csv) ``` ```agrind * | logfmt | split(raw) on "blah" as tokens | sum(tokens[1]) ``` -------------------------------- ### Find Slowest Endpoints Source: https://context7.com/rcoh/angle-grinder/llms.txt Identifies the top 10 slowest endpoints based on the 99th percentile of their duration. Results are sorted in descending order of duration. ```bash agrind '* | json | p99(duration) by endpoint | sort by _p99 desc | limit 10' ``` -------------------------------- ### Count Status Codes by URL Source: https://context7.com/rcoh/angle-grinder/llms.txt Counts the occurrences of each status code for every URL from Nginx access logs. Results are sorted by count in descending order to show the most frequent combinations. ```bash tail -F /var/log/nginx/access.log | agrind '* | json | count by status_code, url | sort by _count desc' ``` -------------------------------- ### Angle Grinder Parse Operator (Pattern Matching) Source: https://context7.com/rcoh/angle-grinder/llms.txt Demonstrates the `parse` operator for extracting data using pattern matching with the `*` wildcard. It covers parsing Apache-style logs, parsing from a specific field, keeping non-matching rows with `nodrop`, preventing type conversion with `noconvert`, and parsing multiple fields. ```bash # Extract status code from log line echo '[status_code=200] Request completed' | agrind '* | parse "[status_code=*]" as status_code' # Parse Apache-style logs agrind '* | parse "* - * [*] \"* * *\" * *" as ip, user, timestamp, method, url, protocol, status, size' # Parse from specific field agrind '* | json | parse "user_*" from username as user_id' # Keep non-matching rows with nodrop agrind '* | parse "[level=*]" as level nodrop | count by level' # Prevent automatic type conversion with noconvert agrind '* | parse "id=*" as id noconvert | count by id' # Parse multiple fields echo '10.0.0.1 GET /api/users 200 45ms' | agrind '* | parse "* * * * *" as ip, method, path, status, duration' ``` -------------------------------- ### Aggregate Data with 'count', 'sum', 'min', 'max', 'average' Operators Source: https://context7.com/rcoh/angle-grinder/llms.txt These aggregation operators allow for summarizing data. 'count' tallies rows, optionally with conditions and grouping. 'sum', 'min', 'max', and 'average' perform calculations on numeric fields, also supporting grouping and multiple aggregations in a single query. 'count_distinct' counts unique values. ```bash agrind '* | count' agrind '* | json | count by status_code' agrind '* | json | count(level == "error") as errors, count(level == "info") as info' agrind '* | json | count by method, status_code' agrind '* | json | count as total_requests by endpoint' agrind '* | json | count_distinct(user_id)' agrind '* | json | count_distinct(ip_address) by country' agrind '* | json | sum(bytes) by endpoint' agrind '* | json | sum(num_records) as total_records by action' agrind '* | json | min(response_time)' agrind '* | json | min(price) by category' agrind '* | json | max(response_time)' agrind '* | json | max(score) by player' agrind '* | json | average(response_time)' agrind '* | json | average(duration) by service' agrind '* | json | count, sum(bytes), min(response_time), max(response_time), average(response_time) by endpoint' ``` -------------------------------- ### Aggregate Operator: Count of source hosts Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This snippet shows a two-step aggregation: first, it counts rows by 'source_host', and then it applies another 'count' to count the number of unique 'source_host' groups. ```agrind * | count by source_host | count ``` -------------------------------- ### Group Data into Time Buckets Source: https://context7.com/rcoh/angle-grinder/llms.txt The timeslice operator groups data into time intervals for time-series analysis. It supports various duration units and allows custom naming for the time bucket field. ```bash # Group by 5-minute intervals agrind '* | json | timeslice(parseDate(timestamp)) 5m | count by _timeslice' # Group by hour agrind '* | json | timeslice(parseDate(ts)) 1h | count by _timeslice, status' # Custom field name agrind '* | json | timeslice(parseDate(timestamp)) 30s as time_bucket | count by time_bucket' # Supported duration units: ns, us, ms, s, m, h, d, w agrind '* | json | timeslice(parseDate(ts)) 1d | sum(bytes) by _timeslice' ``` -------------------------------- ### Split Input String into Array Source: https://context7.com/rcoh/angle-grinder/llms.txt The 'split' operator breaks an input string into an array of substrings. It defaults to using a comma as a separator but can be configured with a custom delimiter using 'on'. The 'as' keyword can be used to assign a custom name to the resulting array field. ```bash echo 'INFO web-001 influxd 127.0.0.1' | agrind '* | split on " "' echo 'INFO darren,hello,50' | agrind '* | parse "* *" as level, csv | split(csv)' agrind '* | logfmt | split(raw) on ":" as tokens | sum(tokens[1])' echo 'a,b,c,d' | agrind '* | split as items | count by items[0]' ``` -------------------------------- ### Analyze Error Rates Over Time Source: https://context7.com/rcoh/angle-grinder/llms.txt Analyzes error rates by counting requests with status codes >= 500 within 1-minute time intervals. It calculates both the number of errors and the total requests per interval. ```bash agrind '* | json | timeslice(parseDate(timestamp)) 1m | count(status >= 500) as errors, count as total by _timeslice' ``` -------------------------------- ### Calculate Error Rate Percentage Source: https://context7.com/rcoh/angle-grinder/llms.txt Calculates the percentage of errors (status codes >= 400) relative to the total number of requests. This provides a clear metric for overall system reliability. ```bash agrind '* | json | count(status >= 400) as errors, count as total | errors * 100 / total as error_rate' ``` -------------------------------- ### Angle Grinder Filter Operators Source: https://context7.com/rcoh/angle-grinder/llms.txt Explains how to use filter operators in Angle Grinder to select specific log lines. It covers wildcard matching, case-sensitive exact matching, and combining filters using boolean logic (AND, OR, NOT). ```bash # Match all lines agrind '* | count' # Case-insensitive wildcard match (no quotes) agrind 'error* | count' agrind 'warn* | count' # Case-sensitive exact match (with quotes) agrind '"ERROR" | count' agrind '"Connection refused" | count' # Boolean combinations with AND, OR, NOT agrind '("ERROR" OR WARN*) AND NOT staging | count' # Complex filter example agrind '("POST" OR "PUT") AND NOT healthcheck* | json | count by endpoint' ``` -------------------------------- ### Calculate Running Totals Source: https://context7.com/rcoh/angle-grinder/llms.txt The total operator computes running totals for streaming data. It can be used to track cumulative values like request counts or transferred bytes over time. ```bash # Running total of requests agrind '* | json | total(num_requests) as cumulative_requests' # Running total of bytes agrind '* | json | total(bytes) as total_bytes_transferred' ``` -------------------------------- ### Limit Output Rows with 'limit' Command Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'limit' command restricts the number of output rows. A positive number returns the first N rows, while a negative number returns the last N rows. ```agrind * | limit 10 ``` ```agrind * | limit -10 ``` -------------------------------- ### Sort Aggregate Results Source: https://context7.com/rcoh/angle-grinder/llms.txt Sort aggregated data based on column values or expressions. Supports ascending (default) and descending order, sorting by multiple columns, and sorting by function results. ```bash # Sort ascending (default) agrind '* | json | count by endpoint | sort by _count' # Sort descending agrind '* | json | count by endpoint | sort by _count desc' # Sort by multiple columns agrind '* | json | count by method, status | sort by method, _count desc' # Sort by expression agrind '* | json | count, sum(bytes) by endpoint | sort by _sum / _count' # Sort by function result agrind '* | json | count by url | sort by length(url)' ``` -------------------------------- ### Parse with Named Regex Captures using 'parse regex' Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'parse regex' command matches input text against a regular expression with named captures and populates the record with these captures. Lines not matching are dropped unless 'nodrop' is specified. Only named captures are supported, and it uses Rust's regular expression syntax. ```agrind * | parse regex "Hello, (?P\w+)" ``` -------------------------------- ### Calculate Percentiles of Data Fields Source: https://context7.com/rcoh/angle-grinder/llms.txt Calculate single or multiple percentiles for specified fields within JSON data. This is useful for understanding data distribution and identifying outliers. It supports grouping results by various dimensions. ```bash agrind '* | json | p50(response_time)' agrind '* | json | p99(latency) by service' agrind '* | json | p50(response_time), p90(response_time), p99(response_time) by endpoint' agrind '* | json | count, p50(duration), p90(duration), p99(duration) by status_code' agrind '* | json | p50(response_time), p90(response_time) by endpoint, method' ``` -------------------------------- ### Control Output Fields with 'fields' Operator Source: https://context7.com/rcoh/angle-grinder/llms.txt The 'fields' operator allows you to explicitly include or exclude specific fields from the output. Use '+' or 'only' to include fields, and '-' or 'except' to exclude them. Special field names can be escaped using square brackets. ```bash agrind '* | json | fields + event, timestamp' agrind '* | json | fields only event, timestamp' agrind '* | json | fields - password, secret' agrind '* | json | fields except password, secret' agrind '* | json | fields + ["date received"], ["grpc.method"] ``` -------------------------------- ### Manage Fields with 'fields' Command Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'fields' command allows selective inclusion or exclusion of fields. Modes include 'only' (or '+') to keep specified fields, and 'except' (or '-') to drop specified fields. ```agrind * | json | fields + event, timestamp ``` ```agrind * | fields except event ``` -------------------------------- ### Parse Network Capture Data Source: https://context7.com/rcoh/angle-grinder/llms.txt Parses network capture data to extract source and destination IP addresses, along with packet length. This is useful for basic network traffic analysis. ```bash tail -f live_pcap | agrind '* | parse "* > *:" as src, dest | parse "length *" as length' ``` -------------------------------- ### Aggregate Operators: Count, percentiles with complex grouping Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This snippet demonstrates using multiple aggregate operators ('count', 'p50', 'p90') with a complex 'by' clause that includes a condition ('status_code >= 400') and a field ('url'). ```agrind * | json | count, p50(response_ms), p90(response_ms), count by status_code >= 400, url ``` -------------------------------- ### Calculate 50th Percentile of Response Time by URL Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Calculates the 50th percentile (median) of response times from JSON logs, grouped by URL. This is useful for analyzing typical response performance. ```bash tail -F my_json_logs | agrind '* | json | pct50(response_time) by url' ``` -------------------------------- ### Sort Aggregate Data by Columns or Expressions Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Sorts aggregate data by one or more columns, with support for ascending (default) or descending order. It can also sort by arbitrary expressions. ```agrind * | json | count by endpoint_url, status_code | sort by endpoint_url desc ``` ```agrind * | json | sort by num_requests / num_responses ``` ```agrind * | json | sort by length(endpoint_url) ``` -------------------------------- ### Aggregate Operator: Average response time Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This snippet shows the 'average' aggregate operator used to calculate the mean of the 'response_time' column. Non-numeric values are ignored. The result is a single column with the average value. ```agrind * | json | average(response_time) ``` -------------------------------- ### Aggregate Operator: Count all rows Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This snippet demonstrates the basic 'count' aggregate operator, which counts the total number of input rows. The output column defaults to '_count'. ```agrind * | count ``` -------------------------------- ### Aggregate Operators: Count and percentiles by status code Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This snippet demonstrates using multiple aggregate operators ('count', 'p50', 'p90') with a 'by' clause to calculate the total count and specific percentiles (50th and 90th) of 'response_ms' grouped by 'status_code'. ```agrind * | json | count, p50(response_ms), p90(response_ms) by status_code ``` -------------------------------- ### Angle Grinder Parse Regex Operator Source: https://context7.com/rcoh/angle-grinder/llms.txt Introduces the `parse regex` operator, which utilizes named capture groups for data extraction with full regular expression support. This allows for more complex and precise pattern matching in log data. ```bash # Example using named capture groups (syntax not provided in prompt, assuming standard regex usage) # agrind '* | parse regex "(?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"' ``` -------------------------------- ### Filter Rows with 'where' Command Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'where' command filters rows based on a boolean expression. The expression can involve field comparisons, literal values, and logical operators. The '!' operator can negate sub-expressions. It handles comparisons with non-existent keys. ```agrind * | json | where status_code >= 400 ``` ```agrind * | json | where user_id_a == user_id_b ``` ```agrind * | json | where url != "/hostname" ``` -------------------------------- ### Parse with Named Capture Groups using Regex Source: https://context7.com/rcoh/angle-grinder/llms.txt The 'parse regex' operator extracts data from input strings using regular expressions with named capture groups. This allows for structured data extraction into fields like 'name', 'ip', 'timestamp', 'level', and 'message'. It can also be used with the 'from' and 'nodrop' options for more specific parsing. ```bash echo 'Hello, World!' | agrind '* | parse regex "Hello, (?P\w+)"' agrind '* | parse regex "(?P\d+\.\d+\.\d+\.\d+)"' agrind '* | parse regex "(?P\d{4}-\d{2}-\d{2}) (?P\w+) (?P.*)"' agrind '* | json | parse regex "user_(?P\d+)" from username nodrop' ``` -------------------------------- ### Parse Network Packet Data and Render with Custom Format Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Parses network packet information (source, destination, length) from live pcap data and renders it using a custom Rust-style format string. This allows for flexible and readable output of network traffic details. ```bash tail -f live_pcap | agrind --format '{src} => {dst} | length={length}' '* | parse "* > *" as src, dest | parse "length *" as length' ``` -------------------------------- ### Calculate Percentiles with 'percentile' Operator Source: https://context7.com/rcoh/angle-grinder/llms.txt The 'percentile' operator calculates specified percentiles for numeric fields. It uses the 'pXX()' syntax, where XX represents the percentile (e.g., p50 for median, p95 for 95th percentile). ```bash agrind '* | json | p50(response_time)' agrind '* | json | p95(duration) by service' ``` -------------------------------- ### Parse Text Patterns with 'parse' Command Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'parse' command extracts text matching specified patterns into variables. Lines not matching the pattern are dropped unless 'nodrop' is specified. The 'noconvert' option preserves parsed fields as strings. It can operate on raw text or a specified field. ```agrind * | parse "[status_code=*]" as status_code ``` -------------------------------- ### Create Computed Fields with 'Field Expression' Operator Source: https://context7.com/rcoh/angle-grinder/llms.txt The 'Field Expression' operator enables the creation of new fields based on calculations, string manipulations, conditional logic, and date/time functions. It supports mathematical operations, string functions like 'toLowerCase', 'concat', 'length', 'substring', conditional logic with 'if()', null/empty checks, and date parsing. ```bash agrind '* | json | value * 100 as percentage' agrind '* | json | bytes / 1024 as kilobytes' agrind '* | json | (end_time - start_time) as duration' agrind '* | json | toLowerCase(method) as method_lower' agrind '* | json | concat(first_name, " ", last_name) as full_name' agrind '* | json | length(message) as msg_length' agrind '* | json | substring(url, 0, 50) as short_url' agrind '* | json | if(status == 200, bytes, 0) as success_bytes' agrind '* | json | if(contains(url, "api"), "api", "web") as request_type' agrind '* | json | if(isNull(user_id), "anonymous", user_id) as user' agrind '* | json | if(isEmpty(email), "no-email", email) as contact' agrind '* | json | if(isBlank(comment), "no-comment", comment) as feedback' agrind '* | json | parseDate(timestamp) as ts | now() - ts as age' agrind '* | json | parseHex(color_code) as color_int' agrind '* | json | num(string_value) * 2 as doubled' ``` -------------------------------- ### Extract Logfmt Data with 'logfmt' Command Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'logfmt' command extracts logfmt-serialized rows into fields. Rows that are not valid logfmt are dropped. It can optionally process data from a specified field. Logfmt is a format commonly used by Heroku and Splunk. ```agrind * | logfmt ``` ```agrind * | json | logfmt from nested_key | fields some ``` -------------------------------- ### Aggregate Operator: Sum of numeric records by action Source: https://github.com/rcoh/angle-grinder/blob/main/README.md This snippet shows the 'sum' aggregate operator used to calculate the total of the 'num_records' column, grouped by the 'action' column. Non-numeric values in 'num_records' are ignored. ```agrind * | json | sum(num_records) by action ``` -------------------------------- ### Angle Grinder JSON Parsing Source: https://context7.com/rcoh/angle-grinder/llms.txt Details the `json` operator for parsing JSON-serialized log data. It covers basic parsing, accessing nested fields, array indexing, parsing JSON from a specific field, and counting distinct nested values. Invalid JSON rows are dropped. ```bash # Basic JSON parsing echo '{"status": 200, "duration": 45}' | agrind '* | json | count by status' # Nested JSON access echo '{"server": {"host": "web-01", "port": 8080}}' | agrind '* | json | count by server.host' # Array indexing (supports negative indices) echo '{"tags": ["api", "v2", "prod"]}' | agrind '* | json | count by tags[0]' # Parse JSON from a specific field agrind '* | parse "INFO *" as payload | json from payload | count by level' # Count distinct nested values echo '{"meta": {"region": "us-east"}}' | agrind '* | json | count_distinct(meta.region)' ``` -------------------------------- ### Count Status Codes by URL Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Counts the occurrences of each status code from JSON logs, grouped by URL. This helps in understanding the distribution of different HTTP status codes for each endpoint. ```bash tail -F my_json_logs | agrind '* | json | count status_code by url' ``` -------------------------------- ### Filter Rows with 'where' Operator Source: https://context7.com/rcoh/angle-grinder/llms.txt The 'where' operator filters input rows based on specified boolean conditions. It supports numeric comparisons (>=, >, <=, <), string comparisons (==, !=), field comparisons, negation (!), and combined conditions using 'and' and 'or'. ```bash agrind '* | json | where status_code >= 400' agrind '* | json | where response_time > 1000' agrind '* | json | where url != "/health"' agrind '* | json | where method == "POST"' agrind '* | json | where user_id_a == user_id_b' agrind '* | json | where !(status_code == 200)' agrind '* | json | where status_code >= 400 and response_time > 500' ``` -------------------------------- ### Basic Angle Grinder Query: Count JSON Logs by Level Source: https://github.com/rcoh/angle-grinder/blob/main/README.md A simple angle-grinder query that parses JSON logs and counts the occurrences of each log level. It demonstrates the basic structure of an angle-grinder command. ```bash agrind '* | json | count by log_level' ``` -------------------------------- ### Extract JSON Data with 'json' Command Source: https://github.com/rcoh/angle-grinder/blob/main/README.md The 'json' command extracts JSON-serialized rows into fields. Rows that are not valid JSON are dropped. It supports nested JSON structures and allows accessing nested values using dot notation and array indexing, including negative indexing. ```agrind * | json ``` ```agrind * | parse "INFO *" as js | json from js ``` ```agrind * | json | count_distinct(nested_key.this) ``` -------------------------------- ### Compute Running Total of a Field Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Computes the running total of a given field. Note that this function does not support grouping and calculates the total across all processed data. ```agrind * | json | total(num_requests) as tot_requests ``` -------------------------------- ### Calculate Percentile of a Column Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Calculates the XXth percentile of a specified column. This function is useful for understanding data distribution and identifying outliers. ```agrind * | json | p50(response_time), p90(response_time) by endpoint_url, status_code ``` -------------------------------- ### Count Distinct Values of a Column Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Counts the number of distinct values in a specified column. Be cautious as this operation is not memory-fixed and can consume significant resources with large datasets. ```agrind * | json | count_distinct(ip_address) ``` -------------------------------- ### Truncate Timestamp to a Duration Source: https://github.com/rcoh/angle-grinder/blob/main/README.md Truncates a timestamp to a specified duration, allowing data to be partitioned into time slices. The duration can be specified in various units (ns, us, ms, s, m, h, d, w). ```agrind * | json | timeslice(parseDate(ts)) 5m ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.