### Project Setup and Installation Source: https://github.com/viacominc/data-point/blob/master/CONTRIBUTING.md Clones the Data-Point repository and installs dependencies using Yarn. Assumes Yarn is already installed. ```bash git clone git@github.com:ViacomInc/data-point.git cd data-point yarn install ``` -------------------------------- ### Install bench-trial Source: https://github.com/viacominc/data-point/blob/master/packages/bench-trial/README.md Installs the bench-trial command-line tool globally using npm. ```bash npm install -g bench-trial ``` -------------------------------- ### Quick Start: Create a DataPoint Express Service Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-express/README.md A basic example demonstrating how to create a DataPoint Express service. It defines a simple 'hello-world' entity and maps it to an Express route. It also exposes a DataPoint inspector. ```javascript const express = require('express') const Service = require('data-point-express') const app = new express() Service.create({ // add DataPoint entities entities: { 'entry:hello-world': () => 'Hello World!!' } }) .then((service) => { // expose DataPoint inspector app.use( '/api/inspect', service.inspector() ) // maps route: /api/hello-world to // entityId: entry:hello-world app.get( '/api/hello-world', service.mapTo('entry:hello-world') ) // start Express server app.listen(3000, (err) => { console.log('DataPoint service ready!') }) }) ``` -------------------------------- ### Install data-point-cache Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-cache/README.md Installs the data-point-cache package using npm. ```bash $ npm install data-point-cache ``` -------------------------------- ### Hello World Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md A basic example demonstrating how to use DataPoint with a function reducer to transform a string input. It shows the creation of a DataPoint instance, defining a reducer, and resolving the reducer with an input value. ```javascript const DataPoint = require('data-point') // create DataPoint instance const dataPoint = DataPoint.create() // function reducer that concatenates // accumulator.value with 'World' const reducer = (input) => { return input + ' World' } // applies reducer to input dataPoint .resolve(reducer, 'Hello') .then((output) => { // 'Hello World' console.log(output) }) ``` -------------------------------- ### Install DataPoint Cache Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-service/README.md Installs the data-point-cache package using npm. This is the first step to using the DataPoint factory. ```bash $ npm install data-point-cache ``` -------------------------------- ### Install DataPoint Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Installs the DataPoint library using npm. Requires Node.js v10 LTS or higher. ```bash npm install --save data-point ``` -------------------------------- ### Lerna Publish Output Example Source: https://github.com/viacominc/data-point/wiki/Updating-and-publishing-to-data‐point An example of the output generated by the `lerna publish` command, showing detected changes and prompting for confirmation. ```bash yarn run v1.22.22 $ /Users/timothymahon/projects/data-point/node_modules/.bin/lerna publish --conventional-commits prepatch lerna notice cli v3.16.2 lerna info versioning independent lerna info publish rooted leaf detected, skipping synthetic root lifecycles lerna info Looking for changed packages since data-point-cache@4.1.1-alpha.0 Changes: - data-point-express: 5.0.5-alpha.0 => 5.0.6-alpha.0 - data-point-service: 4.2.8-alpha.0 => 4.2.9-alpha.0 ? Are you sure you want to publish these packages? (ynH) ``` -------------------------------- ### Install DataPoint Express Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-express/README.md Installs the necessary packages for DataPoint Express, including express, data-point, and data-point-express. These are peer dependencies. ```bash npm install --save express data-point data-point-express ``` -------------------------------- ### Example data-point-codemods run Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-codemods/README.md Demonstrates an example execution of the data-point-codemods tool, showing the interactive prompts for version selection and the command to run codemods for a specific version range. ```bash $ data-point-codemods src/data/ ? What version of data-point are you currently using? older than 2.0.0 ? What version of data-point are you moving to? 2.0.0 (latest) For similar projects, you may want to run the following command: data-point-codemods --from 0.0.0 --to 2.0.0 "src/data/" ``` -------------------------------- ### Request.url Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to set the URL for a request entity and retrieve data from a remote API. It shows a practical example of making a GET request and processing the JSON response. ```javascript dataPoint.addEntities({ 'request:getLuke': { url: 'https://swapi.co/api/people/1/' } }) dataPoint.resolve('request:getLuke', {}) .then(output => { /* output -> { name: 'Luke Skywalker', height: '172', ... } */ }) ``` -------------------------------- ### Example: Creating a DataPoint Service with Multiple Entities Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-express/README.md An example showcasing the creation of a DataPoint Service with multiple entities, including a greeting entity that uses route parameters. It demonstrates setting up Express routes for these entities. ```javascript const express = require('express') const Service = require('data-point-express') const app = new express() Service.create({ // add DataPoint entities entities: { 'entry:HelloWorld': (input, acc) => 'Hello World!!', 'entry:Greet': (input, acc) => `Hello ${acc.locals.params.person}!!` } }) .then(service => { // create Express routes app.get('/api/hello-world', service.mapTo('entry:HelloWorld')) app.get('/api/greet/:person', service.mapTo('entry:Greet')) app.listen(3000, (err) => { if(err) { throw err } console.info('DataPoint service ready!') }) }) .catch(error => { console.info('Failed to Create Service') console.error(error) process.exit(1) }) ``` -------------------------------- ### Basic Usage Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-cache/README.md Demonstrates how to create a cache instance, set a key-value pair with a TTL, and retrieve the value. ```javascript const Cache = require('data-point-cache') Cache.create() .then(cache => { return cache.set('myKey', 'foo', '20m') .then(() => { return cache.get('myKey') }) }) .then(result => { console.log('foo') ) ``` -------------------------------- ### Reducer Entity Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md An example showcasing the usage of a Reducer entity. It defines helper functions for transformation and applies them to a nested data structure. ```javascript const input = { a: { b: { c: [1, 2, 3] } } } const getMax = (input) => { return Math.max.apply(null, input) } const multiplyBy = (number) => (input) => { return input * number } dataPoint.addEntities({ 'reducer:foo': ['$a.b.c', getMax, multiplyBy(10)] }) dataPoint .resolve('reducer:foo', input) .then((output) => { assert.strictEqual(output, 30) }) ``` -------------------------------- ### Express Example with DataPoint Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-service/README.md Provides a complete example of integrating DataPoint with an Express.js application. It sets up a simple API endpoint that resolves data using a DataPoint entity. ```javascript const express = require("express"); const DataPoint = require("data-point"); const DataPointService = require("data-point-service"); function server(dataPoint) { const app = express(); app.get("/api/hello-world", (req, res) => { dataPoint.resolve(`entry:HelloWorld`, req.query).then(output => { res.send(output); }); }); app.listen(3000, function() { console.log("listening on port 3000!"); }); } function createService() { return DataPointService.create({ DataPoint, entities: { "reducer:HelloWorld": (input, acc) => "Hello World!!" } }).then(service => { return service.dataPoint; }); } createService().then(server); ``` -------------------------------- ### Parallel Reducer Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Illustrates the use of the parallel reducer to resolve an array of reducers, returning an array of their outputs. ```javascript const reducer = DataPoint.parallel([ '$a', ['$b', (input) => input + 2] // list reducer ]) const input = { a: 1, b: 2 } dataPoint.resolve(reducer, input) // => [1, 4] ``` -------------------------------- ### Entity Reducer Execution Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to use entity reducers to resolve data paths and pipe the output through a custom reducer function for transformation. ```javascript const input = { a: { b: 'Hello World' } } const toUpperCase = (input) => { return input.toUpperCase() } dataPoint.addEntities({ 'reducer:getGreeting': '$a.b', 'reducer:toUpperCase': toUpperCase, }) // resolve `reducer:getGreeting`, // pipe value to `reducer:toUpperCase` dataPoint .resolve(['reducer:getGreeting | reducer:toUpperCase'], input) .then((output) => { assert.strictEqual(output, 'HELLO WORLD') }) ``` -------------------------------- ### Entity Reducer Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Provides an example of using an Entity Instance Reducer to apply a Request entity and a Model entity to resolve data. It fetches person data from an API and transforms it into a specific model structure. ```javascript const { Model, Request } = require('data-point') const PersonRequest = Request('PersonRequest', { url: 'https://swapi.co/api/people/{value}' }) const PersonModel = Model('PersonModel', { value: { name: '$name', birthYear: '$birth_year' } }) dataPoint .resolve([PersonRequest, PersonModel], 1) .then((output) => { // assert.deepStrictEqual(output, { // name: 'Luke Skywalker', // birthYear: '19BBY' // }) }) ``` -------------------------------- ### Install data-point-codemods Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-codemods/README.md Globally install the data-point-codemods package using npm. This makes the `data-point-codemods` binary available in your system's PATH. ```bash npm install -g data-point-codemods ``` -------------------------------- ### Cache API Documentation Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-cache/README.md Details the methods available for the cache instance, including creation, setting, and getting cache entries. ```APIDOC Cache.create Description: Create a cache instance. Signature: Cache.create({ localTTL: Number, redis: Object }):Promise Parameters: localTTL (Number, optional): Value in Milliseconds of in memory TTL, by default is set to 2000 (2 seconds). redis (Object, optional): Value passed to the ioredis constructor. cache.set Description: Adds a new entry to the cache. Signature: cache.set(key:String, value:Any, ttl:String):Promise Parameters: key (String, yes): Key related to this entry. value (Any, yes): Value to be stored, this value will be stringified when stored in redis. ttl (String, optional): Time To Live of the entry. Defaults to 20 minutes. cache.get Description: Gets an entry from the cache. It checks Redis, then the in-memory cache, and re-stores in-memory if found in Redis. Signature: cache.get(key:String):Promise Parameters: key (String, yes): Key related to this entry. Returns: A promise that resolves to the cached value or undefined if the key does not exist. ``` -------------------------------- ### Get help for data-point-codemods Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-codemods/README.md Display the help information for the data-point-codemods command-line interface, listing available options and usage instructions. ```bash data-point-codemods --help ``` -------------------------------- ### Constant Reducer Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how the constant reducer returns a given value and how reducers are not evaluated when defined inside constants. ```javascript const input = { a: 1, b: 2 } const reducer = { a: '$a', b: DataPoint.constant({ a: '$a', b: 3 }) } dataPoint .resolve(reducer, input) .then(output => { // { // a: 1, // b: { // a: '$a', // b: 3 // } // } } }) ``` ```javascript const input = { b: 1 } // object reducer that contains a path reducer ('$a') let reducer = { a: '$b' } dataPoint.resolve(reducer, input) // => { a: 1 } // both the object and the path will be treated as // constants instead of being used to create reducers reducer = DataPoint.constant({ a: '$b' }) dataPoint.resolve(reducer, input) // => { a: '$b' } ``` -------------------------------- ### Fetching Remote Services with Data-Point Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md This example demonstrates how to fetch and aggregate data from multiple remote services using Data-Point. It defines schemas, requests, and models to interact with the SWAPI service to retrieve planet and resident information. ```javascript const DataPoint = require('data-point') // create DataPoint instance const dataPoint = DataPoint.create() const { Request, Model, Schema, map } = DataPoint // schema to verify data input const PlanetSchema = Schema('PlanetSchema', { schema: { type: 'object', properties: { planetId: { $id: '/properties/planet', type: 'integer' } } } }) // remote service request const PlanetRequest = Request('Planet', { // {value.planetId} injects the // value from the accumulator // creates: https://swapi.co/api/planets/1/ url: 'https://swapi.co/api/planets/{value.planetId}' }) const ResidentRequest = Request('Resident', { // check input is string inputType: 'string', url: '{value}' }) // model entity to resolve a Planet const ResidentModel = Model('Resident', { inputType: 'string', value: [ // hit request:Resident ResidentRequest, // extract data { name: '$name', gender: '$gender', birthYear: '$birth_year' } ] }) // model entity to resolve a Planet const PlanetModel = Model('Planet', { inputType: PlanetSchema, value: [ // hit request:Planet data source PlanetRequest, // map result to an object reducer { // map name key name: '$name', population: '$population', // residents is an array of urls // eg. https://swapi.co/api/people/1/ // where each url gets mapped // to a model:Resident residents: ['$residents', map(ResidentModel)] } ] }) const input = { planetId: 1 } dataPoint.resolve(PlanetModel, input) .then((output) => { console.log(output) /* output -> { name: 'Tatooine', population: 200000, residents: [ { name: 'Luke Skywalker', gender: 'male', birthYear: '19BBY' }, { name: 'C-3PO', gender: 'n/a', birthYear: '112BBY' }, { name: 'Darth Vader', gender: 'male', birthYear: '41.9BBY' }, ... ] } */ }) ``` -------------------------------- ### Hash.value Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to use the 'value' property of a Hash entity to resolve the accumulator's value to a specific hash structure. ```javascript const input = { a: { b: { c: 'Hello', d: ' World!!' } } } dataPoint.addEntities({ 'hash:helloWorld': { value: '$a.b' } }) dataPoint .resolve('hash:helloWorld', input) .then((output) => { assert.deepStrictEqual(output, { c: 'Hello', d: ' World!!' }) }) ``` -------------------------------- ### Hash Multiple Reducers Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates adding multiple reducers to a Hash spec, combining functionalities like pickKeys, mapKeys, addValues, and addKeys in a single entity definition. ```javascript const toUpperCase = (input) => { return input.toUpperCase() } dataPoint.addEntities({ 'entry:orgInfo': { value: 'request:getOrgInfo | hash:OrgInfo' }, 'request:getOrgInfo': { url: 'https://api.github.com/orgs/{value.org}', options: () => ({ headers: { 'User-Agent': 'DataPoint' } }) }, 'hash:OrgInfo': { pickKeys: ['repos_url', 'name'], mapKeys: { reposUrl: '$repos_url', orgName: '$name', }, addValues: { info: 'This is a test' }, addKeys: { orgName: [`$orgName`, toUpperCase] } } }) const expectedResult = { reposUrl: 'https://api.github.com/orgs/nodejs/repos', orgName: 'NODE.JS FOUNDATION', info: 'This is a test' } dataPoint .resolve('entry:orgInfo', { org: 'nodejs' }) .then((output) => { assert.deepStrictEqual(output, expectedResult) }) ``` -------------------------------- ### Entity Reducer Array Mapping Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Illustrates how to use the array mapping feature of entity reducers to apply a transformation to each element of an input array. ```javascript const input = { a: [ 'Hello World', 'Hello Laia', 'Hello Darek', 'Hello Italy', ] } const toUpperCase = (input) => { return input.toUpperCase() } dataPoint.addEntities({ 'reducer:toUpperCase': toUpperCase }) dataPoint .resolve(['$a | reducer:toUpperCase[]'], input) .then((output) => { assert.strictEqual(output[0], 'HELLO WORLD') assert.strictEqual(output[1], 'HELLO LAIA') assert.strictEqual(output[2], 'HELLO DAREK') assert.strictEqual(output[3], 'HELLO ITALY') }) ``` -------------------------------- ### Hash.mapKeys Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Illustrates the use of the 'mapKeys' property to transform keys and values within a Hash entity, including mapping to existing properties and generating new values with functions. ```javascript const _ = require('lodash') dataPoint.addEntities({ 'hash:mapKeys': { mapKeys: { // map to acc.value.name name: '$name', // uses a list reducer to // map to acc.value.name // and generate a string with // a function reducer url: [ '$name', input => { return `https://github.com/ViacomInc/${_.kebabCase(input)}` } ] } } }) const input = { name: 'DataPoint' } dataPoint.resolve('hash:mapKeys', input).then(output => { assert.deepStrictEqual(output, { name: 'DataPoint', url: 'https://github.com/ViacomInc/data-point' }) }) ``` -------------------------------- ### withDefault Reducer Examples Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Shows how the withDefault reducer adds a default value for null, undefined, NaN, or empty string resolutions, including using a function for object defaults. ```javascript const input = { a: undefined } // adds a default to a path reducer const r1 = DataPoint.withDefault('$a', 50) dataPoint.resolve(r1, input) // => 50 ``` ```javascript // passing a function is useful when the default value is // an object, because it returns a new object every time const r2 = withDefault('$a', () => { return { b: 1 } }) dataPoint.resolve(r2, input) // => { b: 1 } ``` -------------------------------- ### Test Object Reducer in DataPoint Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/docs/best-practices.md Provides an example of testing an Object Reducer in DataPoint. This involves creating a DataPoint instance and resolving the reducer with sample input data. ```javascript const DataPoint = require('data-point') function getMonth(rawDate) { const date = new Date(rawDate) return date.getMonth() + 1 } const Card = { id: '$uid', title: '$shortTitle', monthCreated: ['$dateCreated', getMonth] } describe('Card Object Reducer', () => { it('should create Card Object', async () => { const dataPoint = DataPoint.create() const input = { uid: 123, shortTitle: 'Some Title', dateCreated: '2018-12-04T03:24:00' } const result = await dataPoint.resolve(Card, input) expect(result).toEqual({ id: 123, title: 'Some Title', monthCreated: 12 }) }) }) ``` -------------------------------- ### Model Entity: Using the `before` property Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Illustrates the use of the `before` property in a Model entity to preprocess input data. This example ensures the input is always an array before further processing. ```javascript const toArray = (input) => { return Array.isArray(input) ? input : [input] } dataPoint.addEntities({ 'model:foo': { before: toArray, value: '$' } }) dataPoint .resolve('model:foo', 100) .then((output) => { assert.deepStrictEqual(output, [100]) }) ``` -------------------------------- ### Tracing DataPoint Transformations Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Enables tracing of Data-Point transformations by setting `trace: true` in the options object. This generates a JSON file detailing reducer execution, including start times, durations, and reducer information. ```javascript const DataPoint = require('data-point') const mocks = require('./async-example.mocks') // mock request calls mocks() const { Model, Request } = DataPoint const PersonRequest = Request('PersonRequest', { url: 'https://swapi.co/api/people/{value}/' }) const PersonModel = Model('PersonModel', { value: { name: '$name', birthYear: '$birth_year' } }) const options = { trace: true } const dataPoint = DataPoint.create() dataPoint .transform([PersonRequest, PersonModel], 1, options) .then((output) => { // a file with the name data-point-trace-.json will // be created. }) ``` -------------------------------- ### Collection.compose with filter and map Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Illustrates using Collection.compose with 'filter' to select forked repositories and 'map' to transform them into a summary format. This example highlights chaining modifiers and using a Hash entity for picking specific keys. ```javascript const isEqualTo = (match) => (input) => { return input === match } dataPoint.addEntities({ 'request:repos': { url: 'https://api.github.com/orgs/nodejs/repos', options: () => ({ headers: { 'User-Agent': 'request' } }) }, 'hash:repositorySummary': { pickKeys: ['id', 'name', 'homepage', 'description'] }, 'collection:forkedReposSummary': { compose: [ { filter: '$fork'}, { map: 'hash:repositorySummary' } ] } }; dataPoint .resolve('request:repos | collection:forkedReposSummary') .then((output) => { console.log(output) /* [ { "id": 28619960, "name": "build-container-sync", "homepage": null, "description": null }, { "id": 30464379, "name": "nodejs-es", "homepage": "", "description": "Localización y traducción de io.js a Español" } ] */ }) ``` -------------------------------- ### Instance Entity Creation and Usage Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to create an Instance Entity using a factory like Model and then resolve it. ```javascript const DataPoint = require('data-point') const { Model } = DataPoint const dataPoint = DataPoint.create() const HelloWorld = Model('HelloWorld', { value: input => ({ hello: 'world' }) }) // to reference it we use the actual entity instance dataPoint.resolve(HelloWorld, {}) .then(value => { console.assert(value, { hello: 'world' }) }) ``` -------------------------------- ### Request Options with Constants and Query Parameters Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Illustrates configuring request options, including setting headers with constants and defining query parameters that reference input values. ```javascript const DataPoint = require('data-point') const dataPoint = DataPoint.create() dataPoint.addEntities({ 'request:searchPeople': { url: 'https://swapi.co/api/people', // options is a Reducer, but values // at any level can be wrapped as // constants (or just wrap the whole // object if all the values are static) options: { 'content-type': DataPoint.constant('application/json') qs: { // get path `searchTerm` from input // to dataPoint.resolve search: '$searchTerm' } } } }) const input = { searchTerm: 'r2' } // the second parameter to transform is the input value dataPoint .resolve('request:searchPeople', input) .then(output => { assert.strictEqual(output.results[0].name, 'R2-D2') }) ``` -------------------------------- ### Registering and Resolving a HelloWorld Entity Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to register a 'model:HelloWorld' entity with a simple value transformation and then resolve it using the DataPoint instance. This showcases the basic entity creation and execution flow. ```javascript const DataPoint = require('data-point') dataPoint = DataPoint.create({ entities: { 'model:HelloWorld', { value: input => ({ hello: 'world' }) } } }) // to reference it we use a string with its registered id dataPoint.resolve('model:HelloWorld', {}) .then(value => { console.assert(value, { hello: 'world' }) }) ``` -------------------------------- ### Clone Data-Point Repository Source: https://github.com/viacominc/data-point/wiki/Updating-and-publishing-to-data‐point Clones the main data-point repository from GitHub to your local environment. ```bash git clone git@github.com:ViacomInc/data-point.git ``` -------------------------------- ### Run Benchmarks Source: https://github.com/viacominc/data-point/blob/master/packages/bench-trial/README.md Executes benchmark tests using bench-trial, specifying the benchmark file and the number of iterations. ```bash bench-trial benchmarks/map-helper-vs-entity.js -i 5 ``` -------------------------------- ### Hash.addValues Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Adds hard-coded values to the Hash value. This allows for the direct inclusion of specific data points into the result. ```javascript dataPoint.addEntities({ 'hash:addValues': { addValues: { foo: 'value', bar: true, obj: { a: 'a' } } } }) // keys came out intact const expectedResult = { foo: 'value', bar: true, obj: { a: 'a' } } dataPoint .resolve('hash:addValues') .then((output) => { assert.deepStrictEqual(output, expectedResult) }) ``` -------------------------------- ### Create DataPoint Service Object Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-service/README.md Demonstrates how to create a new DataPoint Service Object using the factory. It shows how to pass options, including entities, and how to use the created service to transform data. ```javascript const options = { entities: { "reducer:foo": (input, acc) => "bar" } }; factory .create(options) .then(service => { return service.dataPoint.transform("reducer:foo"); }) .then(output => { console.log(output); // bar }); ``` -------------------------------- ### Checkout Version Tag and Create Branch Source: https://github.com/viacominc/data-point/wiki/Updating-and-publishing-to-data‐point Checks out a specific version tag from the repository and creates a new branch for making updates. Replace `{version-tag-to-update}` with the release tag and `{name-of-fix-branch}` with a descriptive branch name. ```bash git checkout -tag tags/{version-tag-to-update} -b {name-of-fix-branch} ``` -------------------------------- ### Running Unit Tests Source: https://github.com/viacominc/data-point/blob/master/CONTRIBUTING.md Commands to execute unit tests for the Data-Point project. Includes options for running with coverage and in watch mode. ```bash # runs all unit tests with test coverage yarn run test # runs all unit tests with test coverage and with watch mode yarn run watch ``` -------------------------------- ### Hash.pickKeys Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Picks a specified list of keys from a Hash value, returning only those keys and their corresponding values. Keys not included in the list are omitted from the result. ```javascript dataPoint.addEntities({ 'hash:pickKeys': { pickKeys: ['url'] } }) const input = { name: 'DataPoint', url: 'https://github.com/ViacomInc/data-point' } dataPoint.resolve('hash:pickKeys', input).then(output => { // notice how name is no longer // in the object assert.deepStrictEqual(output, { url: 'https://github.com/ViacomInc/data-point' }) }) ``` -------------------------------- ### DataPoint Factory Options Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-service/README.md Details the structure of the `options` object passed to `factory.create`. It specifies settings for cache, including whether it's required and Redis-specific configurations. ```APIDOC factory.create(options):Promise Properties of the `options` argument: | option | type | description | | :------------------- | :-------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **cache** | `Object` | cache specific settings | | **cache.isRequired** | `Boolean` | false by default, if true it will throw an error | | **cache.redis** | `Object` | **redis** settings passed to the [ioredis](https://github.com/luin/ioredis/blob/master/API.md#new-redisport-host-options) constructor. For key prefixing, set [keyPrefix](https://github.com/luin/ioredis#transparent-key-prefixing) through `cache.redis.keyPrefix`; if the value is not provided then [os.hostname()](https://nodejs.org/api/os.html#os_os_hostname) will be used. | ``` -------------------------------- ### Hash.omitKeys Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Omits specified keys from a Hash value, allowing other keys to pass through unchanged. This is useful for selectively removing certain data points. ```javascript dataPoint.addEntities({ 'hash:omitKeys': { omitKeys: ['name'] } }) // notice how name is no longer in the object const expectedResult = { url: 'https://github.com/ViacomInc/data-point' } const input = { name: 'DataPoint', url: 'https://github.com/ViacomInc/data-point' } dataPoint.resolve('hash:omitKeys', input).then(output => { assert.deepStrictEqual(output, expectedResult) }) ``` -------------------------------- ### bench-trial CLI API Source: https://github.com/viacominc/data-point/blob/master/packages/bench-trial/README.md Defines the command-line interface for bench-trial, including file input and optional flags for iterations and skipping tests. ```APIDOC bench-trial [-i ] [-s] Parameters: : The path to the benchmark file. -i, --iterations : Specifies the number of iterations for the benchmarks. Defaults to 10 if not provided. -s, --skip-tests: If provided, skips the assertion tests before running benchmarks. ``` -------------------------------- ### DataPoint API: create Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Creates a DataPoint instance. This method can optionally accept an options object to configure values, entities, and entity types upon creation. ```APIDOC DataPoint.create([options]) Arguments: options (Object, optional): Configuration object for the instance. values (Object): Hash with values exposed to every Reducer. entities (Object): Application's defined entities. entityTypes (Object): Custom Entity Types. Returns: DataPoint instance. Example 1: Create a DataPoint object without configuring options ```js const DataPoint = require('data-point') const dataPoint = DataPoint.create() ``` Example 2: Create the DataPoint object and set options argument ```js const DataPoint = require('data-point') const dataPoint = DataPoint .create({ values: { foo: 'bar' }, entities: { 'reducer:HelloWorld': (input) => { return `hello ${input}!!` } } }) ``` ``` -------------------------------- ### Model Entity: Using the `after` property Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Shows how to use the `after` property in a Model entity to perform actions after the main value transformation. This example checks if the resolved value is an array. ```javascript const toArray = (input) => { return Array.isArray(input) ? input : [input] } dataPoint.addEntities({ 'model:foo': { value: '$a.b', after: isArray() } }) const input = { a: { b: [3, 15] } } dataPoint .resolve('model:foo', input) .then((output) => { assert.deepStrictEqual(output, [3, 15]) }) ``` -------------------------------- ### Conditional Entity Execution Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to conditionally execute an entity reducer based on the accumulator's value, preventing execution if the value is false, null, or undefined. ```javascript const people = [ { name: 'Luke Skywalker', swapiId: '1' }, { name: 'Yoda', swapiId: null } ] dataPoint.addEntities({ 'request:getPerson': { url: 'https://swapi.co/api/people/{value}' }, 'reducer:getPerson': { name: '$name', // request:getPerson will only // be executed if swapiId is // not false, null or undefined birthYear: '$swapiId | ?request:getPerson | $birth_year' } }) dataPoint .resolve('reducer:getPerson[]', people) .then((output) => { assert.deepStrictEqual(output, [ { name: 'Luke Skywalker', birthYear: '19BBY' }, { name: 'Yoda', birthYear: undefined } ]) }) ``` -------------------------------- ### Reducer Helpers Import Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Shows how to import various reducer helper factory functions from the 'data-point' library. ```javascript const { assign, constant, filter, find, map, parallel, withDefault } = require('data-point') ``` -------------------------------- ### Filter Collection by Truthiness of a Property Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Filters a collection of GitHub repositories to include only those that are forks. This example demonstrates filtering based on the truthiness of the `fork` property using a shorthand syntax. ```javascript dataPoint.addEntities({ 'request:getOrgRepositories': { url: 'https://api.github.com/orgs/nodejs/repos', options: () => ({ headers: { 'User-Agent': 'request' } }) }, 'collection:getRepositoryUrl': { filter: '$fork' } }) dataPoint .resolve(['request:getOrgRepositories', 'collection:getRepositoryUrl']) .then((output) => { console.log(output) /* [ { "id": 28619960, "name": "build-container-sync", "full_name": "nodejs/build-container-sync", ... "fork": true }, { "id": 30464379, "name": "nodejs-es", "full_name": "nodejs/nodejs-es", ... "fork": true } ] */ }) ``` -------------------------------- ### Hash.addKeys Example Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Adds new keys to a Hash value or overrides existing ones. It functions as an extend operation, appending new key-value pairs to the existing accumulator value. ```javascript dataPoint.addEntities({ 'hash:addKeys': { addKeys: { nameLowerCase: ['$name', input => input.toLowerCase()], url: () => 'https://github.com/ViacomInc/data-point' } } }) const input = { name: 'DataPoint' } dataPoint.resolve('hash:addKeys', input).then(output => { assert.deepStrictEqual(output, { name: 'DataPoint', nameLowerCase: 'datapoint', url: 'https://github.com/ViacomInc/data-point' }) }) ``` -------------------------------- ### Create Instance Entity in DataPoint Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/docs/best-practices.md Shows how to create an instance entity using DataPoint's Model factory. Instance entities are stand-alone reducers that can be exported and used directly. ```javascript const DataPoint = require('data-point') const Card = DataPoint.Model('Card', { value: { id: '$uid', title: '$shortTitle', date: localizeDate('lastModified') } }) const dp = DataPoint.create() dp.resolve(Card, input) .then(result => { console.dir(result) }) ``` -------------------------------- ### Model Entity: Using the `value` property Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to use the `value` property within a Model entity to transform input data. This example extracts a nested array and applies transformations. ```javascript const input = { a: { b: { c: [1, 2, 3] } } } const getMax = (input) => { return Math.max.apply(null, input) } const multiplyBy = (number) => (input) => { return input * number } dataPoint.addEntities({ 'model:foo': { value: ['$a.b.c', getMax, multiplyBy(10)] } }) dataPoint .resolve('model:foo', input) .then((output) => { assert.strictEqual(output, 30) }) ``` -------------------------------- ### Handling Entry Errors in DataPoint Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Demonstrates how to handle errors within an entity using the 'error' transform. This example shows how to catch type check errors and resolve to a default value. ```js dataPoint.addEntities({ 'model:getArray': { // points to a NON Array value value: '$a.b', outputType: 'isArray', error: (error) => { // prints out the error // message generated by // isArray type check console.log(error.message) console.log('Value is invalid, resolving to empty array') // passing a value will stop // the propagation of the error return [] } } }) const input = { a: { b: 'foo' } } dataPoint .resolve('model:getArray', input) .then((output) => { assert.deepStrictEqual(output, []) }) ``` -------------------------------- ### Log Accumulator URL with DataPoint Reducer Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-express/README.md An example of a DataPoint Reducer that logs the `acc.locals.url` to the console, demonstrating how to access request-specific information within an entity's execution flow. ```javascript const reducer = (input, acc) => { console.log('url that originated this call:', acc.locals.url) return input } ``` -------------------------------- ### Pick Object Keys with Ramda Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/docs/best-practices.md Shows how to pick object keys using Ramda's `R.pick` function, which natively supports currying. This makes it suitable for direct use as a reducer, simplifying the data transformation pipeline. ```js const R = require('ramda') const pickPersonFields = R.pick([ 'name', 'height', 'mass' ]) dataPoint.resolve([PeopleRequest, pickPersonFields], 10) .then((result) => { }) ``` -------------------------------- ### Fetching Latest Tag for Each Repository Source: https://github.com/viacominc/data-point/blob/master/packages/data-point/README.md Fetches repositories and then makes a subsequent request for each repository to get its latest tag information. This showcases chaining requests and using the mapped value as a parameter for another request. ```javascript dataPoint.addEntities({ 'request:getOrgRepositories': { url: 'https://api.github.com/orgs/nodejs/repos', options: () => ({ headers: { 'User-Agent': 'request' } }) }, 'request:getLatestTag': { // here we are injecting the current acc.value // that was passed to the request url: 'https://api.github.com/repos/nodejs/{value}/tags', options: () => ({ headers: { 'User-Agent': 'request' } }) }, 'collection:getRepositoryLatestTag': { // magic!! here we are telling it to map each // repository.name to a request:getLatestTag, and return the entire source map: '$name | request:getLatestTag' } }) dataPoint.resolve('request:getOrgRepositories | collection:getRepositoryLatestTag', {}).then((output) => { console.log(output) /* [ [ // repo { "name": "v2.7.1", "zipball_url": "https://api.github.com/repos/nodejs/http-parser/zipball/v2.7.1", "tarball_url": "https://api.github.com/repos/nodejs/http-parser/tarball/v2.7.1", "commit": {...} ... }, ... ], [ // repo { "name": "works", "zipball_url": "https://api.github.com/repos/nodejs/node-v0.x-archive/zipball/works", "tarball_url": "https://api.github.com/repos/nodejs/node-v0.x-archive/tarball/works", "commit": {...} ... }, ... ], ... // more repos ] */ }) ``` -------------------------------- ### Route Middleware Options Source: https://github.com/viacominc/data-point/blob/master/packages/data-point-express/README.md Illustrates different ways to define middleware for routes, including standard Express functions, DataPoint entity IDs, and a mix of both. ```APIDOC Middleware as a Function: middleware: (req, res) => res.send('hello ${req.params.person}!') Middleware as an Entity Id: middleware: 'entry:HelloWorld' Mixed middleware (entity ID must be last): middleware: [requireAuthentication, 'entry:HelloWorld'] ``` ```javascript // Middleware as a Function Example: { helloWorld: { path: '/hello/:person', priority: 100, middleware: (req, res) => res.send('hello ${req.params.person}!') } } // Middleware as an Entity Id Example: // data point entities: { 'entry:HelloWorld': { value: (input, acc) => `hello ${acc.locals.params.person}!` } } // routes { helloWorld: { path: '/hello/:person', priority: 100, middleware: 'entry:HelloWorld' } } // Mixed middleware Example: // data point entities: { 'entry:HelloWorld': { value: (input, acc) => `hello ${acc.locals.params.person}!` } } // routes { helloWorldProtected: { path: '/hello/:person', priority: 100, middleware: [requireAuthentication, 'entry:HelloWorld'] }, badRoute: { path: '/hello/:person', priority: 100, middleware: ['entry:HelloWorld', requireAuthentication] // Not allowed } } ```