### Example Druid Query Output Source: https://plywood.imply.io/index This is an example of the JSON output from a basic Plywood query to Druid, showing aggregated metrics. ```json [ { "70": 70, "TotalAdded": 32553107, "Count": 113240 } ] ``` -------------------------------- ### Querying Data with PlyQL Source: https://plywood.imply.io/plyql A basic example demonstrating how to query data from a table using PlyQL. ```plyql QUANTILE(delta_hist WHERE cityName = "San Francisco", 0.95) as P95 FROM wikipedia; ``` -------------------------------- ### Example Druid Query Output with Split Source: https://plywood.imply.io/index This is an example of the JSON output from a Plywood query to Druid that includes a split, showing aggregated data and a nested list of top pages. ```json [ { "TotalAdded": 32553107, "Count": 113240 } ] [ { "TotalAdded": 97393743, "Count": 389319, "Pages": [ { "Page": "Jeremy Corbyn", "Count": 314 }, { "Page": "User:Cyde/List of candidates for speedy deletion/Subpage", "Count": 255 }, { "Page": "Wikipedia:Administrators' noticeboard/Incidents", "Count": 228 }, { "Page": "Wikipedia:Vandalismusmeldung", "Count": 186 }, { "Page": "Total Drama Presents: The Ridonculous Race", "Count": 160 }, { "Page": "Wikipedia:Administrator intervention against vandalism", "Count": 145 } ] } ] ``` -------------------------------- ### Create a Simple Plywood Query Source: https://plywood.imply.io/index Demonstrates creating a Plywood query by starting with an empty dataset, applying literal values, and creating expressions using string parsing and method chaining. The `compute()` method resolves the query. ```javascript var ex0 = ply() // Create an empty singleton dataset literal [{}] // 1 is converted into a literal .apply("one", 1) // The string "$one + 1" is parsed into an expression .apply("two", "$one + 1") // The method chaining approach is used to make an expression .apply("four", $("two").multiply(2)) ``` -------------------------------- ### Plywood Query Result Example Source: https://plywood.imply.io/index Illustrates the expected nested data structure output after computing a Plywood query. ```json [ { one: 1, two: 2, four: 4 } ] ``` -------------------------------- ### Nested Queries in PlyQL Source: https://plywood.imply.io/plyql An advanced example showcasing nested queries in PlyQL, where datasets are treated as datatypes for nesting. ```plyql plyql -h 192.168.60.100:8082 -i P1Y -q ' SELECT page as Page, COUNT() as cnt, ( SELECT SUM(added) as TotalAdded GROUP BY TIME_BUCKET(__time, PT1H, "Etc/UTC") LIMIT 3 -- only get the first 3 hours to keep this example output small ) as "ByTime" FROM wikipedia GROUP BY page ORDER BY cnt DESC LIMIT 5; ' ``` -------------------------------- ### Construct Basic Druid Query Source: https://plywood.imply.io/index Build a Plywood query to fetch data from Druid. This example filters by time and channel, calculates the total count and sum of 'added' attribute, and includes a constant value from the context. ```javascript var ex = ply() // Define the external in scope with a filter on time and language .apply("wiki", $('wiki').filter($("time").in({ start: new Date("2015-09-12T00:00:00Z"), end: new Date("2015-09-13T00:00:00Z") }).and($('channel').is('en'))) ) // Calculate count .apply('Count', $('wiki').count()) // Calculate the sum of the `added` attribute .apply('TotalAdded', '$wiki.sum($added)'); // Refer to the seventy defined in the context .apply('70', $('seventy')); ``` -------------------------------- ### Numeric Bucketing with NUMBER_BUCKET Source: https://plywood.imply.io/plyql Buckets numeric data into specified sizes with an offset. Buckets are inclusive of the start and exclusive of the end. ```plyql NUMBER_BUCKET($revenue, 5, 1) ``` -------------------------------- ### Substring Extraction with SUBSTR Source: https://plywood.imply.io/plyql Extracts a substring of a specified length from a string, starting at a given position. ```plyql SUBSTR(_str_, _pos_, _len_) ``` -------------------------------- ### Get String Length Source: https://plywood.imply.io/expressions Returns the number of characters in the string operand. ```javascript var ex = $(\'str\').length(); ex.compute({ str: \'morning\' }).then(console.log); // => 7 ``` -------------------------------- ### Show Tables with PlyQL Source: https://plywood.imply.io/plyql Use this command to list all available data sources in Druid. Specify the broker host and port using the -h option. ```bash plyql -h 192.168.60.100:8082 -q 'SHOW TABLES' ``` -------------------------------- ### Extract Substring Source: https://plywood.imply.io/expressions Extracts a portion of a string operand based on a starting position and length. ```javascript var ex = $(\'str\').substr(1, 5); ex.compute({ str: \'Hello World\' }).then(console.log); // => \'ello \' ``` -------------------------------- ### Describe Data Source Schema with PlyQL Source: https://plywood.imply.io/plyql Use this command to view the column definitions and types for a specific Druid data source. Replace 'wikipedia' with your data source name. ```bash plyql -h 192.168.60.100:8082 -q 'DESCRIBE wikipedia' ``` -------------------------------- ### Configure a Druid External Source: https://plywood.imply.io/design-overview Set up a Druid external for Plywood to connect to a Druid database. Requires specifying the engine, datasource, time attribute, and a requester. ```javascript External.fromJS({ engine: 'druid', source: 'wikipedia', // The datasource name in Druid timeAttribute: 'time', // Druid's anonymous time attribute will be called 'time' requester: druidRequester // a thing that knows how to make Druid requests }) ``` -------------------------------- ### Define Sample Dataset Source: https://plywood.imply.io/expressions Defines a sample dataset using Dataset.fromJS for testing or demonstration purposes. ```javascript var someDataset = Dataset.fromJS([ { cut: \'Good\', price: 400, time: new Date(\'2015-10-01T10:20:30Z\') } { cut: \'Good\', price: 300, time: new Date(\'2015-10-02T10:20:30Z\') } { cut: \'Great\', price: 124, time: null } { cut: \'Wow\', price: 160, time: new Date(\'2015-10-04T10:20:30Z\') } { cut: \'Wow\', price: 100, time: new Date(\'2015-10-05T10:20:30Z\') } ]); ``` -------------------------------- ### Execute and Log Basic Query Results Source: https://plywood.imply.io/index Compute the Plywood query using the defined context and log the results to the console in a formatted JSON string. The '.done()' method ensures the query is executed. ```javascript ex.compute(context).then(function(data) { // Log the data while converting it to a readable standard console.log(JSON.stringify(data.toJS(), null, 2)); }).done(); ``` -------------------------------- ### Import Plywood and Core Functions Source: https://plywood.imply.io/index Import necessary functions from the Plywood library. `ply()` creates a dataset literal, and `$` creates a Reference Expression. ```javascript var plywood = require('plywood'); var ply = plywood.ply; var $ = plywood.$; ``` -------------------------------- ### Build a Plywood Expression Source: https://plywood.imply.io/design-overview Use this to construct data queries with Plywood's expression language. It supports operations like apply, split, sort, and limit. ```javascript var ex = ply() .apply('Count', $('wiki').count()) .apply('TotalAdded', '$wiki.sum($added)') .apply('Pages', $('wiki').split('$page', 'Page') .apply('Count', $('wiki').count()) .sort('$Count', 'descending') .limit(6) ); ``` -------------------------------- ### Configure Druid Requester Source: https://plywood.imply.io/index Initialize the Druid requester factory with the host address of your Druid instance. This object will handle communication with Druid. ```javascript var druidRequester = druidRequesterFactory({ host: '192.168.60.100:8082' // Where ever your Druid may be }); ``` -------------------------------- ### Multi-dimensional GROUP BY in PlyQL Source: https://plywood.imply.io/plyql Demonstrates multi-dimensional GROUP BY operations in PlyQL, useful for complex aggregations. ```plyql plyql -h 192.168.60.100:8082 -i P1Y -q ' SELECT TIME_BUCKET(__time, PT1H, "Etc/UTC") as Hour, page as PageName, SUM(added) as TotalAdded FROM wikipedia GROUP BY 1, 2 ORDER BY TotalAdded DESC LIMIT 3; ' ``` -------------------------------- ### Representing a Dataset Source: https://plywood.imply.io/datatypes Use Dataset.fromJS to create a dataset, which is an abstract representation of a table or data source. Each datum is a collection of attributes. ```javascript Dataset.fromJS([ { x: 1, y: "USA" } { x: 2, y: "UK" } ]) ``` -------------------------------- ### PLYQL String and Utility Functions Source: https://plywood.imply.io/plyql Functions for string manipulation, lookup, null handling, and custom transformations. ```APIDOC ## String and Utility Functions ### `SUBSTR(str, pos, len)` Returns a substring of `str` starting at `pos` with length `len`. ### `CONCAT(str1, str2, ...)` Concatenates one or more string arguments. ### `EXTRACT(str, regexp)` Returns the first matching group from applying `regexp` to `str`. ### `LOOKUP(str, lookup-namespace)` Returns the value for `str` within the specified `lookup-namespace`. ### `IFNULL(expr1, expr2)` Returns `expr1` if it is not null, otherwise returns `expr2`. ### `FALLBACK(expr1, expr2)` Synonym for `IFNULL(expr1, expr2)`. ### `OVERLAP(expr1, expr2)` Checks if `expr1` and `expr2` overlap. ### `CUSTOM_TRANSFORM(expr1, custom_name)` Applies a specified custom transformation to an expression. ``` -------------------------------- ### Define Plywood Execution Context Source: https://plywood.imply.io/index Set up an execution context map for Plywood queries. This context allows you to reference Plywood External objects (like 'wikiDataset') and other values (like 'seventy') by name within your queries. ```javascript var context = { wiki: wikiDataset, seventy: 70 }; ``` -------------------------------- ### Construct Druid Query with Split and Sort Source: https://plywood.imply.io/index Create a more complex Plywood query that includes a split on the 'page' attribute, calculates counts per page, and limits the results to the top 6 pages sorted by count. ```javascript var ex = ply() .apply("wiki", $('wiki').filter($("time").in({ start: new Date("2015-09-12T00:00:00Z"), end: new Date("2015-09-13T00:00:00Z") })) ) .apply('Count', $('wiki').count()) .apply('TotalAdded', '$wiki.sum($added)') .apply('Pages', $('wiki').split('$page', 'Page') .apply('Count', $('wiki').count()) .sort('$Count', 'descending') .limit(6) ); ``` -------------------------------- ### Time Bucketing with PlyQL Source: https://plywood.imply.io/plyql This query uses the TIME_BUCKET function to aggregate data into 6-hour intervals (PT6H) and calculates the sum of the 'added' column for each bucket. The results include the time range for each bucket. ```bash plyql -h 192.168.60.100:8082 -i P1Y -q ' SELECT SUM(added) as TotalAdded FROM wikipedia GROUP BY TIME_BUCKET(__time, PT6H, "Etc/UTC"); ' ``` -------------------------------- ### Exponentiation with POW and POWER Source: https://plywood.imply.io/plyql Calculates the value of the first expression raised to the power of the second expression. POW and POWER are synonyms. ```plyql POW(_expr1_, _expr2_) ``` ```plyql POWER(_expr1_, _expr2_) ``` -------------------------------- ### Creating Time Ranges with TIME_RANGE Source: https://plywood.imply.io/plyql Creates a time range from a given time point and a point shifted by a specified duration and step in a given timezone. Supports negative steps. ```plyql TIME_RANGE(time, 'P1D', -2, 'America/Los_Angeles') ``` -------------------------------- ### Import Druid Dependencies Source: https://plywood.imply.io/index Import necessary modules for Plywood and Druid integration. 'External' is the Plywood class for external data sources, and 'druidRequesterFactory' is specific to Druid. ```javascript var External = plywood.External; var druidRequesterFactory = require('plywood-druid-requester').druidRequesterFactory; ``` -------------------------------- ### Lookup Values with LOOKUP Source: https://plywood.imply.io/plyql Retrieves a value from a specified lookup namespace using a given string key. ```plyql LOOKUP(_str_, _lookup-namespace_) ``` -------------------------------- ### select Source: https://plywood.imply.io/expressions Selects only the provided attributes from the dataset. ```APIDOC ## select(...attributes: string[]) ### Description Select only the provided attributes in the dataset. ### Parameters #### Path Parameters - **attributes** (string[]) - Required - A list of attribute names to select. ### Request Example ```javascript var ex = $('data').select('cut', 'time'); ex.compute({ data: someDataset }).then(console.log); ``` ### Response #### Success Response (200) - **Dataset** - The dataset with only the selected attributes. #### Response Example ```javascript Dataset.fromJS([ { cut: 'Good', time: new Date('2015-10-01T10:20:30Z') }, { cut: 'Good', time: new Date('2015-10-02T10:20:30Z') }, { cut: 'Great', time: null } ]) ``` ``` -------------------------------- ### Add Operands Source: https://plywood.imply.io/expressions Use the add() method to sum the operand with the provided arguments. This is equivalent to parsing `operand + arguments`. ```javascript var ex = $(\'x\').add('$y', 1); ex.compute({ x: 10, y: 2 }).then(console.log); // => 13 ``` -------------------------------- ### Define Druid External Data Source Source: https://plywood.imply.io/index Create an 'External' Plywood object representing a Druid datasource. Specify the engine, source name, time attribute, and any Druid-specific context. ```javascript var wikiDataset = External.fromJS({ engine: 'druid', source: 'wikipedia', // The datasource name in Druid timeAttribute: 'time', // Druid's anonymous time attribute will be called 'time', context: { timeout: 10000 // The Druid context } }, druidRequester); ``` -------------------------------- ### PLYQL Operators Source: https://plywood.imply.io/plyql A comprehensive list of operators supported in PLYQL for various comparisons, logical operations, and pattern matching. ```APIDOC ## PLYQL Operators ### Arithmetic Operators - `+`: Addition operator - `-`: Minus operator / Unary negation - `*`: Multiplication operator - `/`: Division operator ### Comparison Operators - `=, IS`: Equal operator - `!=, <>`: Not equal operator - `>=`: Greater than or equal operator - `>`: Greater than operator - `<=`: Less than or equal operator - `<`: Less than operator ### Range Operators - `BETWEEN ... AND ...`: Checks if a value is within a range. - `NOT BETWEEN ... AND ...`: Checks if a value is not within a range. ### Pattern Matching Operators - `LIKE, CONTAINS`: Simple pattern matching. - `NOT LIKE, CONTAINS`: Negation of simple pattern matching. - `REGEXP`: Pattern matching using regular expressions. - `NOT REGEXP`: Negation of REGEXP. ### Logical Operators - `NOT, !`: Negates a value. - `AND`: Logical AND. - `OR`: Logical OR. ``` -------------------------------- ### Time Bucketing with TIME_BUCKET Source: https://plywood.imply.io/plyql Buckets time data into specified durations within a given timezone. Useful for time-series analysis. ```plyql TIME_BUCKET(time, 'P1D', 'America/Los_Angeles') ``` -------------------------------- ### Flooring Time with TIME_FLOOR Source: https://plywood.imply.io/plyql Floors time data to the nearest specified duration within a given timezone. Useful for aligning time data to specific intervals. ```plyql TIME_FLOOR(time, 'P1D', 'America/Los_Angeles') ``` -------------------------------- ### Handling Null Values with IFNULL and FALLBACK Source: https://plywood.imply.io/plyql Provides a fallback expression if the primary expression evaluates to null. IFNULL and FALLBACK are synonyms. ```plyql IFNULL(_expr1_, _expr2_) ``` ```plyql FALLBACK(_expr1_, _expr2_) ``` -------------------------------- ### apply Source: https://plywood.imply.io/expressions Applies a given expression to each row of the dataset and saves the result in a new field. ```APIDOC ## _operand_.**apply**(name: string, ex: any) ### Description Apply the given expression to every datum in the dataset saving the result as `name`. ### Parameters #### Path Parameters - **name** (string) - Required - The name of the new field to store the result. - **ex** (any) - Required - The expression to apply to each datum. ### Request Example ```javascript var someDataset = Dataset.fromJS([ { cut: 'Good', price: 400, time: new Date('2015-10-01T10:20:30Z') }, { cut: 'Good', price: 300, time: new Date('2015-10-02T10:20:30Z') }, { cut: 'Great', price: 124, time: null }, { cut: 'Wow', price: 160, time: new Date('2015-10-04T10:20:30Z') }, { cut: 'Wow', price: 100, time: new Date('2015-10-05T10:20:30Z') } ]); var ex = $("data").apply('DoublePrice', '$price * 2'); ex.compute({ data: someDataset }).then(console.log); // => // Dataset.fromJS([ // { cut: 'Good', price: 400, DoublePrice: 800, time: new Date('2015-10-01T10:20:30Z') }, // { cut: 'Good', price: 300, DoublePrice: 600, time: new Date('2015-10-02T10:20:30Z') }, // { cut: 'Great', price: 124, DoublePrice: 248, time: null }, // { cut: 'Wow', price: 160, DoublePrice: 320, time: new Date('2015-10-04T10:20:30Z') }, // { cut: 'Wow', price: 100, DoublePrice: 200, time: new Date('2015-10-05T10:20:30Z') } // ]) ``` ``` -------------------------------- ### Group and Order Results with PlyQL Source: https://plywood.imply.io/plyql This query groups results by the 'page' column, counts occurrences, and orders them by count in descending order, limiting to the top 5. A time filter is required to prevent computationally expensive queries. ```bash plyql -h 192.168.60.100:8082 -q ' SELECT page as pg, COUNT() as cnt FROM wikipedia GROUP BY page ORDER BY cnt DESC LIMIT 5; ' ``` -------------------------------- ### String Concatenation with CONCAT Source: https://plywood.imply.io/plyql Concatenates two or more strings together. Can accept a variable number of string arguments. ```plyql CONCAT(_str1_, _str2_, ...) ``` -------------------------------- ### Group and Order with Interval Filter using PlyQL Source: https://plywood.imply.io/plyql This command uses the --interval (-i) option to automatically filter data for the last year (P1Y). This simplifies queries where you need recent data without specifying exact timestamps. ```bash plyql -h 192.168.60.100:8082 -i P1Y -q ' SELECT page as pg, COUNT() as cnt FROM wikipedia GROUP BY page ORDER BY cnt DESC LIMIT 5; ' ``` -------------------------------- ### Group and Order with Time Filter using PlyQL Source: https://plywood.imply.io/plyql This query is similar to the previous one but includes a specific time range filter. This is crucial for performance on large datasets. The time filter ensures that only data within the specified range is considered. ```bash plyql -h 192.168.60.100:8082 -q ' SELECT page as pg, COUNT() as cnt FROM wikipedia WHERE "2015-09-12T00:00:00" <= __time AND __time < "2015-09-13T00:00:00" GROUP BY page ORDER BY cnt DESC LIMIT 5; ' ``` -------------------------------- ### lookup Source: https://plywood.imply.io/expressions Performs a lookup within a specified namespace. ```APIDOC ## lookup ### Description Performs a lookup within the specified namespace. ### Method operand.lookup(lookup: string) ``` -------------------------------- ### Minimum Value with MIN Source: https://plywood.imply.io/plyql Finds the minimum value among all values for a given expression. ```plyql MIN(_expr_) ``` -------------------------------- ### PLYQL Numeric and Time Functions Source: https://plywood.imply.io/plyql Functions for bucketing numeric data, and manipulating time-based data including bucketing, extracting parts, flooring, shifting, and creating ranges. ```APIDOC ## Numeric and Time Functions ### `NUMBER_BUCKET(operand, size, offset)` Buckets numeric data into buckets of a specified `size` with a given `offset`. Buckets are open-closed (start <= x < end). Example: `NUMBER_BUCKET($revenue, 5, 1)` ### `TIME_BUCKET(operand, duration, timezone)` Buckets time data into buckets of a specified `duration` in the given `timezone`. Example: `TIME_BUCKET(time, 'P1D', 'America/Los_Angeles')` ### `TIME_PART(operand, part, timezone)` Extracts specific parts of a time value based on repeating buckets in the given `timezone`. Possible `part` values include: - `SECOND_OF_MINUTE`, `SECOND_OF_HOUR`, `SECOND_OF_DAY`, `SECOND_OF_WEEK`, `SECOND_OF_MONTH`, `SECOND_OF_YEAR` - `MINUTE_OF_HOUR`, `MINUTE_OF_DAY`, `MINUTE_OF_WEEK`, `MINUTE_OF_MONTH`, `MINUTE_OF_YEAR` - `HOUR_OF_DAY`, `HOUR_OF_WEEK`, `HOUR_OF_MONTH`, `HOUR_OF_YEAR` - `DAY_OF_WEEK`, `DAY_OF_MONTH`, `DAY_OF_YEAR` - `WEEK_OF_MONTH`, `WEEK_OF_YEAR` - `MONTH_OF_YEAR` - `YEAR` Example: `TIME_PART(time, 'DAY_OF_YEAR', 'America/Los_Angeles')` ### `TIME_FLOOR(operand, duration, timezone)` Floors a time value to the nearest `duration` in the given `timezone`. Example: `TIME_FLOOR(time, 'P1D', 'America/Los_Angeles')` ### `TIME_SHIFT(operand, duration, step, timezone)` Shifts a time value forwards by `duration` * `step` in the given `timezone`. `step` can be negative. Example: `TIME_SHIFT(time, 'P1D', -2, 'America/Los_Angeles')` ### `TIME_RANGE(operand, duration, step, timezone)` Creates a range from a time value and a point `duration` * `step` away in the given `timezone`. `step` can be negative. Example: `TIME_RANGE(time, 'P1D', -2, 'America/Los_Angeles')` ``` -------------------------------- ### Power Operation Source: https://plywood.imply.io/expressions Use the power() method to raise the operand to a specified power. This is equivalent to parsing `operand ^ power`. ```javascript var ex = $(\'x\').power(0.5); ex.compute({ x: 4 }).then(console.log); // => 2 ``` -------------------------------- ### Split Dataset by Category Source: https://plywood.imply.io/expressions Use `split` to divide a dataset into multiple subsets based on the unique values of a specified column. Each subset is named. ```javascript var someDataset = Dataset.fromJS([ { cut: 'Good', price: 400, time: new Date('2015-10-01T10:20:30Z') }, { cut: 'Good', price: 300, time: new Date('2015-10-02T10:20:30Z') }, { cut: 'Great', price: 124, time: null }, { cut: 'Wow', price: 160, time: new Date('2015-10-04T10:20:30Z') }, { cut: 'Wow', price: 100, time: new Date('2015-10-05T10:20:30Z') } ]); var ex = $(\'data\').split('$cut', 'Cut'); ex.compute({ data: someDataset }).then(console.log); // => Dataset.fromJS([ { Cut: 'Good', data: [ { cut: 'Good', price: 400, time: new Date('2015-10-01T10:20:30Z') }, { cut: 'Good', price: 300, time: new Date('2015-10-02T10:20:30Z') } ] }, { Cut: 'Great', data: [ { cut: 'Great', price: 124, time: null } ] }, { Cut: 'Wow', data: [ { cut: 'Wow', price: 160, time: new Date('2015-10-04T10:20:30Z') }, { cut: 'Wow', price: 100, time: new Date('2015-10-05T10:20:30Z') } ] } ]) ``` -------------------------------- ### Representing a Set of Strings Source: https://plywood.imply.io/datatypes Use Set.fromJS to create a set of distinct string elements. The elements within a set must all have the same type. ```javascript Set.fromJS(['USA', 'UK', 'Japan']) ``` -------------------------------- ### Time Bucketing Source: https://plywood.imply.io/expressions Buckets a time operand to the nearest duration within a specified timezone, creating a TimeRange. ```javascript var ex = $(\'time\').timeBucket(\'P1D\'); ``` -------------------------------- ### Calculate Quantile of Expression Source: https://plywood.imply.io/expressions Computes a specified quantile for a given expression in the dataset. The 0.5 quantile is the median. Does not require sorted data. ```javascript var ex = $(\'data\').quantile($(\'price\'), 0.95); ex.compute({ data: someDataset }).then(console.log); // => 167.343 ``` -------------------------------- ### Representing a Time Range Source: https://plywood.imply.io/datatypes Use TimeRange.fromJS to create a time range object. This is often the result of bucketing. ```javascript TimeRange.fromJS({ start: new Date('2015-02-24T18:00:00'), end: new Date('2015-02-24T19:00:00') }) ``` -------------------------------- ### Reciprocate Operand Source: https://plywood.imply.io/expressions Use the reciprocate() method to calculate the reciprocal of the operand. This is equivalent to parsing `1 / operand`. ```javascript var ex = $(\'x\').reciprocate(); ex.compute({ x: 10 }).then(console.log); // => 0.1 ``` -------------------------------- ### Apply Transformation to Dataset Column Source: https://plywood.imply.io/expressions Use `apply` to create a new column in the dataset by applying an expression to each row. This is useful for feature engineering. ```javascript var someDataset = Dataset.fromJS([ { cut: 'Good', price: 400, time: new Date('2015-10-01T10:20:30Z') }, { cut: 'Good', price: 300, time: new Date('2015-10-02T10:20:30Z') }, { cut: 'Great', price: 124, time: null }, { cut: 'Wow', price: 160, time: new Date('2015-10-04T10:20:30Z') }, { cut: 'Wow', price: 100, time: new Date('2015-10-05T10:20:30Z') } ]); var ex = $(\'data\').apply('DoublePrice', '$price * 2'); ex.compute({ data: someDataset }).then(console.log); // => Dataset.fromJS([ { cut: 'Good', price: 400, DoublePrice: 800, time: new Date('2015-10-01T10:20:30Z') }, { cut: 'Good', price: 300, DoublePrice: 600, time: new Date('2015-10-02T10:20:30Z') }, { cut: 'Great', price: 124, DoublePrice: 248, time: null }, { cut: 'Wow', price: 160, DoublePrice: 320, time: new Date('2015-10-04T10:20:30Z') }, { cut: 'Wow', price: 100, DoublePrice: 200, time: new Date('2015-10-05T10:20:30Z') } ]) ``` -------------------------------- ### Predefined Literal Expressions Source: https://plywood.imply.io/expressions Plywood provides static members on the Expression class for common literal values like NULL, ZERO, ONE, FALSE, TRUE, and EMPTY_STRING. ```javascript Expression.NULL.equals(r(null)); Expression.ZERO.equals(r(0)); Expression.ONE.equals(r(1)); Expression.FALSE.equals(r(false)); Expression.TRUE.equals(r(true)); Expression.EMPTY_STRING.equals(r('')); ``` -------------------------------- ### timeBucket Source: https://plywood.imply.io/expressions Buckets a time operand to the nearest duration within a specified timezone. ```APIDOC ## timeBucket ### Description Buckets the operand time to the nearest `duration` within the given `timezone`. Creates a TimeRange of size `duration`. ### Method operand.timeBucket(duration: any, timezone?: string) ### Example ```javascript var ex = $(\'time\').timeBucket(\'P1D\'); ``` ``` -------------------------------- ### numberBucket Source: https://plywood.imply.io/expressions Buckets a numeric operand into defined ranges based on size and offset. ```APIDOC ## numberBucket ### Description Buckets the numeric operand to buckets defined by the `size` and `offset`. ### Method operand.numberBucket(size: number, offset: number = 0) ### Example ```javascript var ex = $(\'x\').numberBucket(5); ex.compute({ x: 7 }).then(console.log); // => [5, 10) ``` ``` -------------------------------- ### Divide Operands Source: https://plywood.imply.io/expressions Use the divide() method to divide the operand by the provided arguments. This is equivalent to parsing `operand / arguments`. ```javascript var ex = $(\'x\').divide('$y'); ex.compute({ x: 10, y: 2 }).then(console.log); // => 5 ``` -------------------------------- ### Maximum Value with MAX Source: https://plywood.imply.io/plyql Finds the maximum value among all values for a given expression. ```plyql MAX(_expr_) ``` -------------------------------- ### match Source: https://plywood.imply.io/expressions Checks if the operand matches the provided regular expression string. ```APIDOC ## match ### Description Checks whether the operand matches the given RegExp that is provided as a string. ### Method operand.match(re: string) ### Example ```javascript var ex = $(\'str\').match(\'^Hell.*d\ ); ex.compute({ str: \'Hello World\' }).then(console.log); // => true ``` ``` -------------------------------- ### and Source: https://plywood.imply.io/expressions Performs a boolean AND operation between the operand and other expressions. ```APIDOC ## and ### Description Performs a boolean AND operation on the operand and the given expressions. Writing `$(\'x\').and($(\'y\'))` is the same as parsing `$x and $y` ### Method operand.and(...exs: any[]) ### Example ```javascript var ex = $(\'x\').and($(\'y\')); ex.compute({ x: true, y: false }).then(console.log); // => false ``` ``` -------------------------------- ### Row Count with COUNT Source: https://plywood.imply.io/plyql Counts the number of rows. If an expression is provided, it counts rows where the expression is not null. ```plyql COUNT(*) ``` ```plyql COUNT(_expr?_) ``` -------------------------------- ### PLYQL Aggregation Functions Source: https://plywood.imply.io/plyql Functions for performing aggregations such as count, sum, minimum, maximum, average, and quantile. ```APIDOC ## Aggregation Functions ### `COUNT(expr?)` Returns the count of rows. If an expression is provided, returns the count of rows where the expression is not null. `COUNT(*)` counts all rows. ### `COUNT(DISTINCT expr), COUNT_DISTINCT(expr)` Returns the count of distinct values for the given expression. ### `SUM(expr)` Returns the sum of all values for the given expression. ### `MIN(expr)` Returns the minimum value for the given expression. ### `MAX(expr)` Returns the maximum value for the given expression. ### `AVG(expr)` Returns the average of all values for the given expression. ### `QUANTILE(expr, quantile)` Returns the upper `quantile` of all values for the given expression. ``` -------------------------------- ### Query Maximum Time with PlyQL Source: https://plywood.imply.io/plyql This query retrieves the timestamp of the latest event in the specified data source. It uses the MAX aggregation function. ```bash plyql -h 192.168.60.100:8082 -q 'SELECT MAX(__time) AS maxTime FROM wikipedia' ``` -------------------------------- ### Summation with SUM Source: https://plywood.imply.io/plyql Calculates the sum of all values for a given numeric expression. ```plyql SUM(_expr_) ``` -------------------------------- ### Applying Custom Transformations with CUSTOM_TRANSFORM Source: https://plywood.imply.io/plyql Applies a user-defined custom transformation to an expression. ```plyql CUSTOM_TRANSFORM(_expr1_, _custom_name_) ``` -------------------------------- ### length Source: https://plywood.imply.io/expressions Returns the length of the string operand. ```APIDOC ## length ### Description Returns the length of the string ### Method operand.length() ### Example ```javascript var ex = $(\'str\').length(); ex.compute({ str: \'morning\' }).then(console.log); // => 7 ``` ``` -------------------------------- ### Create Reference Expression Source: https://plywood.imply.io/expressions Use $('name') to create an expression that references a value by its name. This is equivalent to parsing `$name`. ```javascript var ex = $(\'x\'); ex.compute({ x: 10 }).then(console.log); // => 10 ``` -------------------------------- ### PLYQL Mathematical Functions Source: https://plywood.imply.io/plyql Mathematical functions for calculating absolute values, powers, square roots, and exponential values. ```APIDOC ## Mathematical Functions ### `ABS(expr)` Returns the absolute value of `expr`. ### `POW(expr1, expr2)` Returns `expr1` raised to the power of `expr2`. ### `POWER(expr1, expr2)` Synonym for `POW(expr1, expr2)`. ### `SQRT(expr)` Returns the square root of `expr`. ### `EXP(expr)` Returns e (base of natural logarithms) raised to the power of `expr`. ``` -------------------------------- ### Representing a Number Range Source: https://plywood.imply.io/datatypes Use NumberRange.fromJS to create a number range object. This is often the result of bucketing. ```javascript NumberRange.fromJS({ start: 4, end: 7.5 }) ``` -------------------------------- ### fallback Source: https://plywood.imply.io/expressions Provides a fallback value if the operand is null. ```APIDOC ## fallback ### Description Returns value of given expression if operand is null. Writing `$(\'str\').fallback(r(\'hello\'))` is the same as parsing `$str === null ? \'hello\' : $str` ### Method operand.fallback(...exs: typeof operand) ### Example ```javascript var ex = $(\'str\').extract("([0-9]+\\.[0-9]+\\.[0-9]+)").fallback(\'missing\'); ex.compute({ str: \'kafka-0.7.2\' }).then(console.log); // => \'0.7.2\' ex.compute({ str: \'Web 2.0\' }).then(console.log); // => \'missing\' ``` ``` -------------------------------- ### cardinality Source: https://plywood.imply.io/expressions Returns the number of elements in a set. ```APIDOC ## cardinality ### Description Returns the cardinality of the set ### Method operand.cardinality() ### Example ```javascript var ex = $(\'colors\').length(); ex.compute({ colors: [\'red\', \'orange\', \'green\', \'blue\'] }).then(console.log); // => 4 ``` ``` -------------------------------- ### Limit Dataset Size Source: https://plywood.imply.io/expressions Limits the number of records in a dataset to a specified positive integer. Useful for pagination or sampling. ```javascript var ex = $(\'data\').limit(3); ex.compute({ data: someDataset }).then(console.log); ``` -------------------------------- ### collect Source: https://plywood.imply.io/expressions Computes the set of values in a specified column. ```APIDOC ## collect(ex: any) ### Description Computes the set of values in a certain column. ### Parameters #### Path Parameters - **ex** (any) - Required - The expression representing the column to collect values from. ### Request Example ```javascript var ex = $('data').collect($('cut')); ex.compute({ data: someDataset }).then(console.log); // => Set('Good', 'Great', 'Wow') ``` ### Response #### Success Response (200) - **Set** - A set containing the distinct values from the specified column. ``` -------------------------------- ### timeRange Source: https://plywood.imply.io/expressions Constructs a range with the operand time as one end point and the other endpoint determined by shifting the provided time by `duration` _`step` within the given `timezone`. Creates a TimeRange of size `duration` _ `step`. ```APIDOC ## _operand_.**timeRange**(duration: any, step: number, timezone?: string) ### Description Constructs a range with the operand time as one end point and the other endpoint determined by shifting the provided time by `duration` _`step` within the given `timezone`. Creates a TimeRange of size `duration` _ `step`. ### Parameters #### Path Parameters - **duration** (any) - Required - The duration to define the range size. - **step** (number) - Required - The multiplier for the duration to determine the range size. - **timezone** (string) - Optional - The timezone to use for the range calculation. ### Request Example ```javascript var ex = $("time").timeRange("P1D", -2); ``` ``` -------------------------------- ### String Match with RegExp Source: https://plywood.imply.io/expressions Checks if a string operand matches a given regular expression provided as a string. ```javascript var ex = $(\'str\').match(\'^Hell.*d\n\n\n\n\'); ex.compute({ str: \'Hello World\' }).then(console.log); // => true ``` -------------------------------- ### Negate Operand Source: https://plywood.imply.io/expressions Use the negate() method to apply a negative sign to the operand. This is equivalent to parsing `-operand`. ```javascript var ex = $(\'x\').negate(); ex.compute({ x: 10 }).then(console.log); // => -10 ``` -------------------------------- ### Time Part Extraction with PlyQL Source: https://plywood.imply.io/plyql This query extracts the hour of the day from the __time column using TIME_PART and calculates the total 'added' for each hour. The results are ordered by TotalAdded in descending order, limited to the top 3 hours. ```bash plyql -h 192.168.60.100:8082 -i P1Y -q ' SELECT TIME_PART(__time, HOUR_OF_DAY, "Etc/UTC") as HourOfDay, SUM(added) as TotalAdded FROM wikipedia GROUP BY 1 ORDER BY TotalAdded DESC LIMIT 3; ' ``` -------------------------------- ### Exponential Function with EXP Source: https://plywood.imply.io/plyql Calculates the value of 'e' (Euler's number) raised to the power of the given expression. ```plyql EXP(_expr_) ``` -------------------------------- ### Average Value with AVG Source: https://plywood.imply.io/plyql Calculates the average of all values for a given numeric expression. ```plyql AVG(_expr_) ``` -------------------------------- ### Extracting Groups with EXTRACT Source: https://plywood.imply.io/plyql Extracts the first matching group from a string based on a regular expression. ```plyql EXTRACT(_str_, _regexp_) ``` -------------------------------- ### Multiply Operands Source: https://plywood.imply.io/expressions Use the multiply() method to multiply the operand by the provided arguments. This is equivalent to parsing `operand * arguments`. ```javascript var ex = $(\'x\').multiply('$y', 3); ex.compute({ x: 10, y: 2 }).then(console.log); // => 60 ``` -------------------------------- ### timeFloor Source: https://plywood.imply.io/expressions Floors the operand time to the nearest `duration` within the given `timezone`. ```APIDOC ## _operand_.**timeFloor**(duration: any, timezone?: string) ### Description Floors the operand time to the nearest `duration` within the given `timezone`. ### Parameters #### Path Parameters - **duration** (any) - Required - The duration to floor the time to. - **timezone** (string) - Optional - The timezone to use for flooring. ### Request Example ```javascript var ex = $("time").timeFloor("P1D"); ``` ``` -------------------------------- ### indexOf Source: https://plywood.imply.io/expressions Returns the 0-based index of a substring within the operand. ```APIDOC ## indexOf ### Description Returns the 0 based index of the substring in the operand. ### Method operand.indexOf(substr: string) ### Example ```javascript var ex = r(\'hello\').indexOf(\'e\'); ex.compute().then(console.log); // => 1 ``` ``` -------------------------------- ### Construct Time Range Source: https://plywood.imply.io/expressions Use `timeRange` to create a time range where one endpoint is the operand time and the other is determined by shifting it. The range size is duration * step. ```javascript var ex = $(\'time\').timeRange(\'P1D\', -2); ``` -------------------------------- ### Shifting Time with TIME_SHIFT Source: https://plywood.imply.io/plyql Shifts time data forwards or backwards by a specified duration and step in a given timezone. Supports negative steps for backward shifting. ```plyql TIME_SHIFT(time, 'P1D', -2, 'America/Los_Angeles') ``` -------------------------------- ### min Source: https://plywood.imply.io/expressions Computes the minimum value of the given expression in the operand dataset. ```APIDOC ## min(ex: any) ### Description Computes the min of the given expression in the operand dataset. ### Parameters #### Path Parameters - **ex** (any) - Required - The expression to find the minimum of. ### Request Example ```javascript var ex = $('data').min($('price')); ex.compute({ data: someDataset }).then(console.log); // => 100 ``` ### Response #### Success Response (200) - **number** - The minimum value of the expression. ``` -------------------------------- ### Create Literal Value Expression Source: https://plywood.imply.io/expressions Use r(value) to create an expression that represents a literal value. This is useful for constants or initial values. ```javascript var ex = r('Hello World'); ex.compute().then(console.log); // => 'Hello World' ``` -------------------------------- ### split Source: https://plywood.imply.io/expressions Splits a dataset into multiple datasets based on the values of a specified expression. ```APIDOC ## _operand_.**split**(splits: any, name?: string, dataName?: string) ### Description Split the data based on the given expression. ### Parameters #### Path Parameters - **splits** (any) - Required - The expression to split the data by. - **name** (string) - Optional - The name of the new field that will contain the split value. - **dataName** (string) - Optional - The name of the field that will contain the split data. ### Request Example ```javascript var someDataset = Dataset.fromJS([ { cut: 'Good', price: 400, time: new Date('2015-10-01T10:20:30Z') }, { cut: 'Good', price: 300, time: new Date('2015-10-02T10:20:30Z') }, { cut: 'Great', price: 124, time: null }, { cut: 'Wow', price: 160, time: new Date('2015-10-04T10:20:30Z') }, { cut: 'Wow', price: 100, time: new Date('2015-10-05T10:20:30Z') } ]); var ex = $("data").split("$cut", "Cut"); ex.compute({ data: someDataset }).then(console.log); // => // Dataset.fromJS([ // { // Cut: 'Good', // data: [ // { cut: 'Good', price: 400, time: new Date('2015-10-01T10:20:30Z') }, // { cut: 'Good', price: 300, time: new Date('2015-10-02T10:20:30Z') } // ] // }, // { // Cut: 'Great', // data: [ // { cut: 'Great', price: 124, time: null } // ] // }, // { // Cut: 'Wow', // data: [ // { cut: 'Wow', price: 160, time: new Date('2015-10-04T10:20:30Z') }, // { cut: 'Wow', price: 100, time: new Date('2015-10-05T10:20:30Z') } // ] // } // ]) ``` ``` -------------------------------- ### Select Specific Attributes Source: https://plywood.imply.io/expressions Filters the dataset to include only the specified attributes. Useful for reducing data size or focusing on relevant fields. ```javascript var ex = $(\'data\').select(\'cut\', \'time\'); ex.compute({ data: someDataset }).then(console.log); ``` -------------------------------- ### Subtract Operands Source: https://plywood.imply.io/expressions Use the subtract() method to subtract the provided arguments from the operand. This is equivalent to parsing `operand - arguments`. ```javascript var ex = $(\'x\').subtract('$y', 1); ex.compute({ x: 10, y: 2 }).then(console.log); // => 7 ``` -------------------------------- ### or Source: https://plywood.imply.io/expressions Performs a boolean OR operation between the operand and other expressions. ```APIDOC ## or ### Description Performs a boolean OR operation on the operand and the given expressions. Writing `$(\'x\').or($(\'y\'))` is the same as parsing `$x or $y` ### Method operand.or(...exs: any[]) ### Example ```javascript var ex = $(\'x\').or($(\'y\')); ex.compute({ x: true, y: false }).then(console.log); // => true ``` ```