### Setup and Build RudderStack Transformer Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md These commands are used to set up the RudderStack transformer repository on a local machine. It involves cloning the repository, running a setup script, cleaning and building the service, and finally starting the server. ```bash # Clone the repository git clone # Setup the repository npm run setup # Build the service npm run build:clean # Start the server npm start ``` -------------------------------- ### Build and Run RudderTransformer Locally Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Commands to set up, build, and start the RudderTransformer service on a local machine. This involves cloning the repository, running setup scripts, and building the service before starting it. ```bash git clone https://github.com/rudderlabs/rudder-transformer npm run setup npm run build:clean npm start ``` -------------------------------- ### Native Installation Setup Script (npm) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/README.md Commands to set up and build the RudderStack Transformer service using npm. Requires Node.js and npm. ```bash npm run setup npm run build:clean npm start ``` -------------------------------- ### Run RudderStack Transformer Locally using npm Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md This snippet outlines the necessary npm commands to set up, build, and start the RudderStack transformer service locally. It requires Node.js version 20 and uses `npm ci` for dependency installation. ```bash nvm use v20 npm ci npm run build:start ``` -------------------------------- ### Run Transformer Locally (npm & nvm) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Commands to set up and run the Rudder Transformer locally. This involves selecting a Node.js version using nvm, installing dependencies with npm ci, and then building and starting the transformer. ```bash nvm use v20 npm ci npm run build:start ``` -------------------------------- ### E2E Test Setup and Execution (make) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/README.md Commands to set up, run, and destroy the environment for End-to-End tests using make. Includes specific commands for ARM processors. ```bash make setup make setup-arm make test make destroy ``` -------------------------------- ### CCPA Data Processing Options Example Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/facebook_conversions/README.md Provides an example of how to structure the context object to enable CCPA compliance by including the `dataProcessingOptions` array. ```javascript { "context": { "dataProcessingOptions": ["LDU", 1, 1000] } } ``` -------------------------------- ### My Destination Event Processing Example Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Illustrates the structure of an event payload for 'My Destination', including user context, device information, and destination-specific configurations. This data is processed by the Rudder Transformer. ```json { "message": { "channel": "web", "context": { "browser": { "url": "https://www.estore.com/best-seller/1", "user_agent": "Dalvik/2.1.0 (Linux; U; Android 9; Android SDK built for x86 Build/PSR1.180720.075)", "initial_referrer": "https://www.google.com/search" }, "mobile": { "app_name": "RSPM", "app_version": "1.9.0" }, "device": { "manufacturer": "Nokia", "model": "N2023" }, "location": { "ip_address": "192.0.2.0", "latitude": 44.56, "longitude": 54.46, "city": "NY", "region": "Atlas", "country": "USA" } }, "session": { "id": "s001" }, "user": { "uid": "u001" } }, "JSON_ARRAY": {}, "XML": {}, "FORM": {} }, "version": "1", "type": "REST", "method": "POST", "endpoint": "https://api.mydestination.com/events", "headers": { "authorization": "Bearer dummyMyDestinationAPIKey", "content-type": "application/json" }, "params": {}, "files": {}, "userId": "" } ``` -------------------------------- ### Install Dependencies for create-trackingid.js Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/personalize/scripts/README.md Installs the necessary npm packages, `aws-sdk` and `readline-sync`, required for the `create-trackingid.js` script to function. ```bash npm install aws-sdk npm install readline-sync ``` -------------------------------- ### Track Subscription Event (JavaScript) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/cdk/v2/destinations/bluecore/README.md Provides examples of tracking subscription-related events, such as opt-in and unsubscribe, using `rudderanalytics.track`. These map to 'optin' events in Bluecore, with details on consent and source. ```javascript // Opt-in example rudderanalytics.track('subscription_event', { channelConsents: { email: true, }, source: 'newsletter_signup', campaign: 'welcome_series', }); // Unsubscribe example rudderanalytics.track('subscription_event', { channelConsents: { email: false, }, reason: 'user_request', source: 'email_footer', }); ``` ```json { "event": "optin", "properties": { "token": "your_namespace", "distinct_id": "user@example.com", "email": "user@example.com", "source": "newsletter_signup", "campaign": "welcome_series" } } ``` -------------------------------- ### Automated Test Case Structure Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Provides an example of how to structure automated test cases for Rudder Transformer integrations, specifying input event payload, expected output, and integration details. This follows patterns used in other integrations. ```typescript /** test/sources/slack/data.ts **/ export const data = [ /** Test 1 - Testing standard event response **/ { "name": "slack", // Replace with your integration name "description": "Team joined event", // Replace with your event description "module": "source", // Replace with your integration type - destination or source "version": "v0", // Replace with your integration approach - v0 or v1 "input": { "request": { "body": "...replaceThisWithYourEventPayload" "method": "POST", "headers": { "Content-Type": "application/json", }, }, "pathSuffix": "", }, "output": { "response": { "status": 200, "body": [ { "output": { "batch": "...replaceThisWithYourTransformedEventOutput" }, }, ], }, }, }, /** Test 2 - Testing custom event response **/ { "name": "slack", // Replace with your integration name "description": "Webhook url verificatin event (not a standard RudderStack event, returns a custom response)", // Replace with your event description "module": "source", // Replace with your integration type - destination or source "version": "v0", // Replace with your integration approach - v0 or v1 "input": { "request": { "body": "...replaceThisWithYourEventPayload" "method": "POST", "headers": { "Content-Type": "application/json", }, }, "pathSuffix": "", }, "output": { "response": { "status": 200, "body": [ { "outputToSource": { "body": "eyJjaGFsb2UiOiIzZVpicncxYUIxMEZFTUFHQVpkNEZ5RlEifQ==", // Replace this with the Base64 encoding of the response body your integrations sends "contentType": "application/json", // Replace this with the content type your integration sends }, "statusCode": 200, // Replace this with the custom response status your integration sends for this event }, ], }, }, }, ]; ``` -------------------------------- ### Example RETL Event for Split.io Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/splitio/docs/retl.md Illustrates a warehouse record transformed into a Split.io 'track' event. This example demonstrates the structure and data mapping required for sending event data to Split.io via RETL. ```javascript // Warehouse record transformed to Split.io track event { "type": "track", "event": "feature_used", "properties": { "feature_name": "advanced_analytics", "user_plan": "premium" }, "context": { "traits": { "email": "user@example.com" } } } ``` -------------------------------- ### My Destination Identify Event Input Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Provides an example of an input payload for an 'identify' event targeting 'My Destination'. It includes user identifiers and associated traits like company and address. ```json { "message": { "userId": "dummy-user001", "channel": "web", "context": { "traits": { "company": "Initech", "address": { "country": "USA", "state": "CA", "street": "101 dummy street" }, "email": "dummyuser@domain.com" } } } } ``` -------------------------------- ### Rudder Transformer Destination Integration File Structure Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Illustrates the standard directory and file organization for a new destination integration within the Rudder Transformer project. This structure ensures modularity and maintainability. ```text src/v0/destinations/**my_destination** (follow the snake_case naming convention) - Your integration code for `my_destination` src/v0/destinations/**my_destination/transform.ts** - Main transformation logic with router and processor functions src/v0/destinations/**my_destination/types.ts** - TypeScript types with Zod schemas for validation src/v0/destinations/**my_destination/config.ts** - Configuration constants and endpoints src/v0/destinations/**my_destination/utils.ts** - Utility functions (optional, only if needed) src/v0/destinations/**my_destination/utils.test.ts** - Unit tests for utilities (optional, only if needed) ``` -------------------------------- ### Dynamic Configuration Resolution Example Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/README.md Illustrates how the RudderTest destination resolves endpoint and API key using dynamic templates based on event traits and context. It shows the input event, the destination configuration with templates, and the resulting resolved configuration and headers. ```javascript // Destination config with dynamic templates { "endpoint": "{{ event.context.endpoint || 'https://default.endpoint.com' }}", "apiKey": "{{ event.traits.appId || 'default-api-key' }}" } // Event with template values { "type": "record", "context": { "endpoint": "https://custom.endpoint.com" }, "traits": { "appId": "my-app-123" } } // Results in resolved config: // endpoint: "https://custom.endpoint.com" // apiKey: "my-app-123" // Headers: { "X-API-Key": "my-app-123" } ``` -------------------------------- ### Manual Testing Endpoint (HTTP Request) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Instructions for manually testing the destination integration using an API request client. It specifies the HTTP method, endpoint, request body format, and required headers. ```http POST /v1/destinations/my_destination Body - An array of event data object received from the source i.e. `[{ …eventData }]` Headers - `Content-Type: application/json` ``` -------------------------------- ### Run Integration Tests (npm) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Execute tests for a specific integration using npm. This command allows targeting a particular destination for testing purposes. Ensure your destination is added to the `INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE` list in `component.test.ts` for the tests to be included. ```bash npm run test:ts -- component --destination=my_destination ``` -------------------------------- ### Amplitude Track Event Payload Example (JSON) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/am/docs/businesslogic.md Example JSON payload for sending a Track event to Amplitude. Contains API key, event details like event type, user ID, device ID, event properties, timestamp, and insert ID. Demonstrates mapping of event name and properties. ```json { "api_key": "YOUR_API_KEY", "events": [ { "event_type": "Product Viewed", "user_id": "user123", "device_id": "device456", "event_properties": { "product_id": "prod123", "product_name": "Example Product", "price": 99.99, "currency": "USD" }, "time": 1609459200000, "insert_id": "message123" } ] } ``` -------------------------------- ### Timestamp Conversion Example Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/google_adwords_offline_conversions/README.md Demonstrates the conversion of an ISO 8601 timestamp to the format required by Google Ads for offline conversions. ```javascript // Example: 2019-10-14T11:15:18.299Z -> 2019-10-14 16:10:29+0530 // Actual conversion logic not provided, this is illustrative. ``` -------------------------------- ### Generate RudderStack UI Configurations Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md This command generates RudderStack UI configuration files using a Python script. It requires a placeholder JSON file as input, which should be adjusted to your integration's requirements. The command should be run from the root directory of the `rudder-integrations-config` repository. ```bash python3 scripts/configGenerator.py ``` -------------------------------- ### JavaScript Configuration Objects for MyDestination Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Defines constants and configuration objects for the MyDestination integration, including API base URL, endpoints, HTTP methods, content types, and event-specific configurations for identify and track events. The `as const` assertion is used for immutability and type inference. ```javascript import { getMappingConfig } from '../../util'; // Destination type identifier - used for logging and error tracking const destType = 'MY_DESTINATION'; // API configuration - defines the base URL, endpoints, and HTTP methods // This makes it easy to change API endpoints or add new ones const API_CONFIG = { BASE_URL: 'https://api.mydestination.com', // Base URL for all API calls ENDPOINTS: { USERS: '/users', // Endpoint for user identification EVENTS: '/events' // Endpoint for event tracking }, METHODS: { POST: 'POST' // HTTP method for sending data }, HEADERS: { CONTENT_TYPE_JSON: 'application/json' // Content type for JSON payloads } } as const; // 'as const' makes the object immutable and provides better type inference // Simplified endpoint configuration - combines base URL with specific endpoints // This creates complete URLs for each event type const ENDPOINT_CONFIG = { IDENTIFY: { url: `${API_CONFIG.BASE_URL}${API_CONFIG.ENDPOINTS.USERS}`, // Full URL for identify events method: API_CONFIG.METHODS.POST, // HTTP method for identify events contentType: API_CONFIG.HEADERS.CONTENT_TYPE_JSON // Content type for identify events }, TRACK: { url: `${API_CONFIG.BASE_URL}${API_CONFIG.ENDPOINTS.EVENTS}`, // Full URL for track events method: API_CONFIG.METHODS.POST, // HTTP method for track events contentType: API_CONFIG.HEADERS.CONTENT_TYPE_JSON // Content type for track events } } as const; // 'as const' makes the object immutable and provides better type inference // Configuration categories - defines the mapping configuration files for each event type // This tells the system which JSON files to use for field mapping const ConfigCategory = { IDENTIFY: { name: 'MyDestinationIdentifyConfig', }, TRACK: { name: 'MyDestinationTrackConfig', }, }; // Load mapping configuration from JSON files // This reads the mapping rules from the data/ folder const MappingConfig = getMappingConfig(ConfigCategory, __dirname); // Export all configuration for use in other files export { destType, ConfigCategory, MappingConfig, API_CONFIG, ENDPOINT_CONFIG }; ``` -------------------------------- ### Implementing Mock Functions After Migration Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/test/scripts/README.md Provides an example of how to correctly implement mock functions after the migration process. This involves defining the mock using a mock adapter, including request matching based on body and headers, and specifying the reply. ```typescript mockFns: (mockAdapter: MockAdapter) => { mockAdapter .onPost( 'https://api.example.com', { asymmetricMatch: (actual) => { return isMatch(actual, { // Expected request body pattern }); }, }, { asymmetricMatch: (actual) => { return isMatch(actual, { // Expected headers pattern }); }, }, ) .reply(200, { success: true }); }; ``` -------------------------------- ### Configuration Settings Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/customerio/README.md Details the required and optional configuration settings for integrating with Customer.io. ```APIDOC ## Configuration Settings ### Description This section outlines the necessary configuration parameters for setting up the Customer.io destination. ### Required Settings - **API Key** (string, Required): Your Customer.io API key for authentication. Used with Site ID for Basic Authentication. - **Site ID** (string, Required): Your Customer.io workspace identifier. Used as the username in Basic Authentication. - **Data Center** (string, Optional, Default: `US`): Specifies the Customer.io data center (`US` or `EU`). Affects the API endpoint used (`track.customer.io` or `track-eu.customer.io`). ### Optional Settings - **Device Token Event Name** (string, Optional): The name of the event that triggers device token registration. - **Send Page Name in SDK** (boolean, Optional): Controls whether page names are sent when using the web SDK. ``` -------------------------------- ### Docker Compose for RudderStack Transformer Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/README.md Instructions for running the RudderStack Transformer service using Docker Compose. Assumes the repository has been cloned. ```bash docker-compose up transformer ``` -------------------------------- ### Manual Recovery: Verify Release Creation Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/docs/RELEASE_PROCESS.md Command to view a specific GitHub release using the GitHub CLI to confirm successful creation. ```bash gh release view v1.102.0 ``` -------------------------------- ### Troubleshooting: Check GitHub CLI Authentication Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/docs/RELEASE_PROCESS.md Command to verify the current authentication status of the GitHub CLI. ```bash gh auth status ``` -------------------------------- ### Manual Release Creation with GitHub CLI (Custom Notes) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/docs/RELEASE_PROCESS.md This snippet illustrates creating a GitHub release using the GitHub CLI, specifying a file containing custom release notes. ```bash gh release create v1.102.0 --title "v1.102.0" --notes-file release-notes.md --latest ``` -------------------------------- ### Example Identify Event for MoEngage Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/moengage/docs/retl.md An example of an 'identify' event structure that can be sent to MoEngage for user profile updates, typically processed through a custom ETL or event stream transformation. ```javascript { "type": "identify", "userId": "user123", "traits": { "email": "user@example.com", "name": "John Doe", "plan": "premium" } } ``` -------------------------------- ### Manual Release Creation with NPM Script Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/docs/RELEASE_PROCESS.md This snippet shows how to create a GitHub release using the project's modern NPM script after checking out a specific tag. ```bash git checkout v1.102.0 npm run release:github:modern ``` -------------------------------- ### Braze Identify Event - Custom Alias Example Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/braze/docs/businesslogic.md An example demonstrating how a custom alias can be configured within the integrations object for Braze, allowing for alternative user identification beyond the default rudder_id. ```json { "integrations": { "braze": { "alias": { "alias_name": "custom-id-123", "alias_label": "custom_label" } } } } ``` -------------------------------- ### Example RETL Event for Criteo Audience Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/criteo_audience/docs/retl.md An example of a transformed warehouse record in the 'audiencelist' event format for Criteo's Audience API. It includes 'add' and 'remove' operations for user identifiers. ```javascript // Warehouse record transformed to Criteo audiencelist event { "type": "audiencelist", "properties": { "listData": { "add": [ { "email": "user1@example.com" }, { "email": "user2@example.com" } ], "remove": [ { "email": "user3@example.com" } ] } } } ``` -------------------------------- ### Manual Release Creation with GitHub CLI (Auto Notes) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/docs/RELEASE_PROCESS.md This snippet demonstrates creating a GitHub release using the GitHub CLI, automatically generating release notes based on commit history. ```bash gh release create v1.102.0 --title "v1.102.0" --generate-notes --latest ``` -------------------------------- ### Manual Recovery: Create Release Using Project Script Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/docs/RELEASE_PROCESS.md Steps for manually creating a release using the project's dedicated Node.js script after checking out a tag. ```bash git checkout v1.102.0 node scripts/create-github-release.js ``` -------------------------------- ### Example RETL Event for Braze Identify Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/braze/docs/retl.md This JSON object represents an example RETL event transformed from a warehouse record into a Braze identify event. It includes the 'context.mappedToDestination' flag and pre-formatted traits. ```json // Warehouse record transformed to Braze identify event { "type": "identify", "userId": "user123", "traits": { "email": "user@example.com", "first_name": "John", "last_name": "Doe", "custom_attribute": "value" }, "context": { "mappedToDestination": true, "externalId": [ { "id": "external_user_123", "type": "brazeExternalId" } ] } } ``` -------------------------------- ### Running Integration Tests for Bluecore Destination Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/cdk/v2/destinations/bluecore/README.md Command to execute integration (component) tests for the Bluecore destination using npm. ```bash npm run test:ts -- component --destination=bluecore ``` -------------------------------- ### Example RETL Event for SFMC Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/sfmc/docs/retl.md This example JSON object represents a warehouse record transformed into an SFMC event for RETL processing. It includes 'type', 'traits', and 'context' with 'mappedToDestination' and 'externalId' details. ```javascript // Warehouse record transformed to SFMC event { "type": "identify", "traits": { "email": "user@example.com", "firstName": "John", "lastName": "Doe", "plan": "premium" }, "context": { "mappedToDestination": true, "externalId": [ { "type": "SFMC-data extension", "id": "user@example.com", "identifierType": "Contact Key" } ] } } ``` -------------------------------- ### Running Unit Tests for Bluecore Destination Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/cdk/v2/destinations/bluecore/README.md Command to execute unit tests for the Bluecore destination using npm. ```bash npm run test:ts -- unit --destination=bluecore ``` -------------------------------- ### Example RETL Event for Google Ads Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/google_adwords_offline_conversions/docs/retl.md An example of a warehouse record transformed into a Google Ads conversion event format. It includes essential properties like order ID, revenue, currency, and a Google Click ID (GCLID). ```javascript // Warehouse record transformed to Google Ads conversion event { "type": "track", "event": "Order Completed", "properties": { "order_id": "12345", "revenue": 99.99, "currency": "USD", "gclid": "abc123def456" }, "context": { "traits": { "email": "user@example.com", "phone": "+1234567890" } } } ``` -------------------------------- ### SFMC Data Extensions API PUT Request Example Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/sfmc/docs/retl.md This example illustrates a PUT request to the Salesforce Marketing Cloud Data Extensions API for updating rows. It shows the endpoint structure and the JSON body containing trait values. ```http // PUT request to Data Extension PUT https://{subdomain}.rest.marketingcloudapis.com/hub/v1/dataevents/key:{externalKey}/rows/{identifierType}:{destinationExternalId} { "values": { ...message.traits } } ``` -------------------------------- ### Customer.io API Endpoints Overview Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/customerio/README.md This table lists the Customer.io API endpoints used by the destination, detailing the event types supported, batching capabilities, and a brief description of each endpoint's function. ```markdown | Endpoint | Event Types | Batch Support | Description | |---|---|---|---| | `/api/v1/customers/:id` | Identify | No | Identify/update user profiles | | `/api/v1/customers/:id/events` | Track, Page, Screen | No | Send events for identified users | | `/api/v1/events` | Track, Page, Screen (Anonymous) | No | Send anonymous events | | `/api/v1/merge_customers` | Alias | No | Merge user profiles | | `/api/v2/batch` | Group | Yes (1000 events, 500KB max) | Batch operations with objects | | `/api/v1/customers/:id/devices` | Device Registration | No | Device token management | | `/api/v1/customers/:id/devices/:device_id` | Device Deletion | No | Device token removal | ``` -------------------------------- ### Amplitude Identify Event Payload Example (JSON) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/am/docs/businesslogic.md Example JSON payload for sending an Identify event to Amplitude. Includes API key, event details like event type ($identify), user ID, device ID, user properties, timestamp, and insert ID for deduplication. ```json { "api_key": "YOUR_API_KEY", "events": [ { "event_type": "$identify", "user_id": "user123", "device_id": "device456", "user_properties": { "name": "John Doe", "email": "john@example.com", "plan": "premium" }, "time": 1609459200000, "insert_id": "message123" } ] } ``` -------------------------------- ### Data Replay Feasibility Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/customerio/README.md Discusses the feasibility and implications of replaying missing or already delivered data in Customer.io, highlighting potential data inconsistencies and duplicate events. ```APIDOC ## Data Replay Feasibility ### Description Evaluates the feasibility of replaying data in Customer.io, emphasizing the risks of data inconsistencies and duplicates. ### Missing Data Replay - **Not Recommended**: Replaying missing data for any event type is not advised due to the strict event ordering requirements, which can lead to data inconsistencies. ### Already Delivered Data Replay - **Not Feasible**: Customer.io treats events as unique based on timestamp and content. Replaying events will create duplicates, affecting analytics and potentially triggering campaigns multiple times. - **Track/Page/Screen Events**: Replaying creates duplicate occurrences. - **Identify Events**: Replaying can overwrite current attributes with older data, causing data regression. - **Group Events**: Replaying may result in duplicate object relationships or overwritten states. ``` -------------------------------- ### Multiplexing: Store Conversions API Calls Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/google_adwords_offline_conversions/README.md Illustrates the sequence of API calls required to complete a single store conversion upload using multiplexing. ```bash POST /offlineUserDataJobs:create POST /offlineUserDataJobs/{jobId}:addOperations POST /offlineUserDataJobs/{jobId}:run ``` -------------------------------- ### Event Mapping and E-commerce Support Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/af/README.md Explains how specific events are mapped to AppsFlyer configurations and how multi-product data is handled for e-commerce events. ```APIDOC ## Event Mapping and Configuration ### Predefined Event Mappings - `Order Completed` maps to Purchase configuration. - `Product Added to Wishlist` maps to Cart/Wishlist configuration. - `Wishlist Product Added to Cart` maps to Cart/Wishlist configuration. - `Checkout Started` maps to Cart/Wishlist configuration. - `Product Removed` maps to Default configuration. - `Product Searched` maps to Search configuration. - `Product Viewed` maps to Content View configuration. ### Multi-Product Support For e-commerce events, the destination automatically handles the `products` array, extracting and mapping: - `product_id` to `af_content_id` - `quantity` to `af_quantity` - `price` to `af_price` ``` -------------------------------- ### Custom Variables - Implementation Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/google_adwords_offline_conversions/docs/businesslogic.md Provides code examples for implementing custom variables. ```APIDOC ### Implementation ```javascript // Example of custom variable mapping in destination configuration customVariables: [ { from: "campaign_id", to: "campaign_identifier" }, { from: "product_category", to: "category" } ] // Runtime processing example const customVariableValue = properties[customVariableKey]; const customVariableResourceName = conversionCustomVariable[customVariableName]; resultantCustomVariables.push({ conversionCustomVariable: customVariableResourceName, value: String(customVariableValue) }); ``` ``` -------------------------------- ### Data Type Validations Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/google_adwords_offline_conversions/docs/businesslogic.md Provides information on data type validations and examples. ```APIDOC ## Data Type Validations ### Description Validations applied to different data types to ensure data integrity. | Field Type | Validation | Example | |------------|------------|---------| | DateTime | ISO 8601 format | `2019-10-14T11:15:18.299Z` | | Number | Positive numeric value | `123.45` | | Integer | Whole number | `5` | | Currency | 3-letter ISO code | `USD` | | Email | Valid email format | `user@example.com` | | Phone | E.164 format (for hashing) | `+1234567890` | ``` -------------------------------- ### Troubleshooting: Verify Git Tag Existence Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/docs/RELEASE_PROCESS.md Command to check if a specific Git tag already exists in the local repository. ```bash git tag -l | grep v1.102.0 ``` -------------------------------- ### Manual Testing Endpoint Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Endpoint for manual testing of the destination integration. It accepts an array of event data objects. ```APIDOC ## POST /v1/destinations/my_destination ### Description Endpoint for manual testing of the destination integration. Accepts an array of event data objects. ### Method POST ### Endpoint /v1/destinations/my_destination ### Headers - **Content-Type**: application/json ### Request Body - **eventData** (array) - Required - An array of event data objects received from the source. ### Request Example ```json [ { "userId": "user123", "event": "Product Viewed", "properties": { "product_id": "prod456", "product_name": "Example Widget" } } ] ``` ### Response #### Success Response (200) - **output** (string) - Details about the integration's output for the given events. #### Response Example ```json { "output": "Events processed successfully." } ``` ``` -------------------------------- ### Identify User Call (JavaScript) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/cdk/v2/destinations/bluecore/README.md Example of using the `rudderanalytics.identify` method in JavaScript to send user profile data to Bluecore. This typically maps to a 'customer_patch' event. ```javascript rudderanalytics.identify('user123', { email: 'user@example.com', firstName: 'John', lastName: 'Doe', phone: '+1234567890', age: 30, address: { street: '123 Main St', city: 'San Francisco', state: 'CA', zip: '94105', }, }); ``` ```json { "event": "customer_patch", "properties": { "token": "your_namespace", "distinct_id": "user@example.com", "customer": { "email": "user@example.com", "first_name": "John", "last_name": "Doe", "phone": "+1234567890", "age": 30, "address": { "street": "123 Main St", "city": "San Francisco", "state": "CA", "zip": "94105" } } } } ``` -------------------------------- ### Identify User API Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md This endpoint is used to identify a user and send user traits to My Destination. It follows the structure defined in `MyDestinationIdentifyConfig.json`. ```APIDOC ## POST /users ### Description Endpoint for user identification. Used to send user traits to My Destination. ### Method POST ### Endpoint /users ### Request Body - **userId** (string) - Required - The unique identifier for the user. - **email** (string) - Optional - The email address of the user. - **traits** (object) - Optional - An object containing various traits of the user (e.g., name, age, etc.). - **context** (object) - Optional - Contextual information about the event, including device and OS information. ### Request Example ```json { "userId": "user123", "email": "user@example.com", "traits": { "name": "John Doe", "age": 30 }, "context": { "ip": "192.168.1.1", "userAgent": "Mozilla/5.0" } } ``` ### Response #### Success Response (200) - **status** (string) - Indicates the success of the operation. #### Response Example ```json { "status": "success" } ``` ``` -------------------------------- ### Customer.io API Versions Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/customerio/README.md Provides information on the current API versions used by Customer.io (v1 and v2) and their respective endpoints, noting the lack of versioned releases and deprecation schedules. ```APIDOC ## Version Information ### Description Details the API versions utilized by Customer.io, including endpoints for v1 (Track API) and v2 (Batch API), and clarifies their deprecation and stability policies. ### Current API Versions Customer.io uses versioned APIs without formal deprecation schedules: - **Track API v1**: For identify, track, alias, and device management operations. - Endpoints: `/api/v1/customers/:id`, `/api/v1/customers/:id/events`, `/api/v1/events`, `/api/v1/merge_customers`, `/api/v1/customers/:id/devices` - **Track API v2**: For batch operations (group events). - Endpoints: `/api/v2/entity`, `/api/v2/batch` ### Deprecation Information - **No Deprecation Timeline**: Customer.io prioritizes backward compatibility and does not announce deprecation dates. - **Version Stability**: Both v1 and v2 APIs are stable and actively maintained. - **Migration Path**: No migration is necessary as both versions are supported indefinitely. ``` -------------------------------- ### TypeScript for Masked Secrets Pattern Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/README.md Example TypeScript code demonstrating the `maskedSecrets.ts` pattern used for configuring test secrets. It includes generating placeholder secrets and constructing authentication headers. ```typescript import path from 'path'; import { base64Convertor } from '@rudderstack/integrations-lib'; // Valid secrets for successful API calls - derived from directory name // This creates unique but predictable values for testing export const secret1 = path.basename(__dirname) + 1; export const secretStaging1 = `stg_` + path.basename(__dirname) + 1; // Invalid secrets for testing error scenarios export const secretInvalid = path.basename(__dirname) + 'invalid'; // Auth headers constructed from secrets export const authHeader1 = `Basic ${base64Convertor(secret1 + ':')}`; export const authHeaderStaging1 = `Basic ${base64Convertor(secretStaging1 + ':')}`; export const authHeaderInvalid = `Basic ${base64Convertor(secretInvalid + ':')}`; ``` -------------------------------- ### My Destination Track Event Input Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Shows the input payload for a 'track' event when processing for 'My Destination'. It includes message details like channel, context, properties, and destination configuration. ```json { "message": { "channel": "web", "context": { "device": { "manufacturer": "Nokia", "model": "N2023" }, "userAgent": "Dalvik/2.1.0 (Linux; U; Android 9; Android SDK built for x86 Build/PSR1.180720.075)" }, "integrations": { "All": true }, "properties": { "userId": "u001", "latitude": 44.56, "longitude": 54.46, "region": "Atlas", "city": "NY", "country": "USA" }, "originalTimestamp": "2020-01-09T10:01:53.558Z", "type": "track", "sentAt": "2020-01-09T10:02:03.257Z" }, "destination": { "ID": "1pYpzzvcn7AQ2W9GGIAZSsN6Mfq", "Name": "My Destination", "DestinationDefinition": { "Config": {} }, "Config": { "apiKey": "dummyMyDestinationAPIKey" }, "Enabled": true, "Transformations": [] }, "metadata": { "destinationId": "destId", "workspaceId": "wspId" } } ``` -------------------------------- ### Braze REST API Documentation Links Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/braze/README.md Provides links to official Braze REST API documentation for various endpoints. ```APIDOC ## Braze REST API Documentation Links - **Braze REST API Overview**: [https://www.braze.com/docs/api/basics/](https://www.braze.com/docs/api/basics/) - **User Track API**: [https://www.braze.com/docs/api/endpoints/user_data/post_user_track/](https://www.braze.com/docs/api/endpoints/user_data/post_user_track/) - **User Identify API**: [https://www.braze.com/docs/api/endpoints/user_data/post_user_identify/](https://www.braze.com/docs/api/endpoints/user_data/post_user_identify/) - **User Delete API**: [https://www.braze.com/docs/api/endpoints/user_data/post_user_delete/](https://www.braze.com/docs/api/endpoints/user_data/post_user_delete/) - **Subscription Group API**: [https://www.braze.com/docs/api/endpoints/subscription_groups/](https://www.braze.com/docs/api/endpoints/subscription_groups/) ``` -------------------------------- ### Track Event API Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md This endpoint is used for tracking user events and their associated properties within My Destination. It maps RudderStack track event fields to destination fields as defined in `MyDestinationTrackConfig.json`. ```APIDOC ## POST /events ### Description Endpoint for event tracking. Used to send event data and properties to My Destination. ### Method POST ### Endpoint /events ### Request Body - **userId** (string) - Required - The unique identifier for the user associated with the event. - **event** (string) - Required - The name of the event being tracked. - **properties** (object) - Optional - An object containing properties associated with the event. - **originalTimestamp** (string) - Optional - The timestamp when the event originally occurred. ### Request Example ```json { "userId": "user123", "event": "Product Viewed", "properties": { "product_id": "prod456", "product_name": "Example Widget" }, "originalTimestamp": "2023-10-27T10:00:00Z" } ``` ### Response #### Success Response (200) - **status** (string) - Indicates the success of the operation. #### Response Example ```json { "status": "success" } ``` ``` -------------------------------- ### Process Batch Events (TypeScript) Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Processes multiple events in a batch. This function is designed for efficiency when handling a large number of events. It utilizes a utility function to manage the batch processing. ```typescript /** * Process batch events * * This function is called for batch event processing. * It uses the utility function to handle multiple events efficiently. * * @param inputs - Array of router requests for batch processing * @returns Promise resolving to array of processed responses */ export const processRouterDest = async ( inputs: MyDestinationRouterRequest[] ): Promise => { const respList = await simpleProcessRouterDest(inputs, process, reqMetadata); return respList; }; ``` -------------------------------- ### Handle Common API Responses Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Creates a standardized HTTP response object for API requests. It configures the endpoint, method, headers, and payload for a given request. Dependencies include 'defaultRequestConfig'. ```typescript const responseHandler = ( endpoint: string, method: string, headers: MyDestinationHeaders, payload: MyDestinationPayload, ): MyDestinationProcessorResponse => { // Create a new response object using the utility const response = defaultRequestConfig(); // Set the endpoint URL response.endpoint = endpoint; // Set the HTTP method response.method = method; // Set the headers (Authorization, Content-Type, etc.) response.headers = headers; // Set the request body as JSON response.body.JSON = payload; return response; }; ``` -------------------------------- ### Integration Functionalities Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/customerio/README.md Details supported message types, batching capabilities, and user identity resolution behavior. ```APIDOC ## Integration Functionalities ### Description This section describes the various integration functionalities supported by the Customer.io destination, including message types, batching, and user identity handling. ### Supported Message Types - Identify - Track - Page - Screen - Group - Alias ### Batching Support - **Supported**: Yes (Limited to Group events). - **Batch Limits**: Up to 1000 Group events per batch, with a maximum payload size of 500KB. ### User Identity Resolution Behavior - **Alias Handling**: Customer.io merges profiles using the `Alias` event. It merges data from `previousId` (secondary) to `userId` (primary) and then deletes the secondary profile. Both email and ID identifiers are supported for merge operations. ### Anonymous User Profiles - **Anonymous Event Tracking**: Anonymous Track, Page, or Screen events (with `anonymousId` but no `userId`) are sent to the anonymous events endpoint. These events can be associated with identified users later if the `anonymousId` matches. ### Proxy Delivery and User Deletion - **Proxy Delivery**: Not supported. - **User Deletion**: Not supported via the suppression API. ### OAuth Support - **Supported**: No. The destination uses API Key authentication (Basic Auth). ``` -------------------------------- ### Standard Event Response Structure Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/CONTRIBUTING.md Defines the expected HTTP 200 OK response structure when only transforming an incoming event and handing it over to RudderStack for delivery. This is the standard case for event processing. ```javascript [ { "output": { "batch": [ { "...transformedEventData" } ] } } ] ``` -------------------------------- ### Event Ordering and Replayability Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/facebook_conversions/README.md Explains the requirements and recommendations for event ordering and replayability with the Facebook Conversions API. ```APIDOC ## Event Ordering and Replayability ### Event Ordering - **Required**: No specific ordering requirements - **Recommendation**: Send events in chronological order when possible - **Impact**: Out-of-order events may affect attribution accuracy ### Event Replayability - **Supported**: Yes, events can be replayed - **Deduplication**: Facebook handles deduplication using event_id and event_time - **Best Practice**: Include unique event_id for each event to prevent duplicates - **Considerations**: Replaying events may affect attribution and optimization ``` -------------------------------- ### Run RudderTest Component Tests Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/README.md This command initiates the component tests specifically for the rudder_test destination using npm. It's a quick way to validate the destination's logic during development. ```bash npm run test:ts:component:rudder_test ``` -------------------------------- ### TikTok Multiplexing: Events to Standard Mapping Example Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/tiktok_ads/README.md This code snippet illustrates how a single event can be mapped to multiple TikTok standard events, resulting in separate API calls for each mapped event. This is configured via the `eventsToStandard` setting. ```javascript standardEventsMap[event].forEach((eventName) => { responseList.push(...); }); ``` -------------------------------- ### Multiplexing Support Source: https://github.com/rudderlabs/rudder-transformer/blob/develop/src/v0/destinations/customerio/README.md Clarifies that the Customer.io destination does not support multiplexing, meaning it does not generate multiple output events from a single input event, and outlines the API calls made for each event type. ```APIDOC ## Multiplexing ### Description The Customer.io destination does not support multiplexing, ensuring that each input event results in a single API call to Customer.io. ### Supported No ### Multiplexing Scenarios All Customer.io event types translate to single API calls: 1. **Identify Events**: `POST /api/v1/customers/:id` 2. **Track/Page/Screen Events**: `POST /api/v1/customers/:id/events` or `POST /api/v1/events` (for anonymous events) 3. **Group Events**: `POST /api/v2/batch` (uses batching, not multiplexing) 4. **Alias Events**: `POST /api/v1/merge_customers` 5. **Device Events**: `POST /api/v1/customers/:id/devices` or specific device deletion endpoint. ```