### Install tap-mixpanel Virtual Environment Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This command installs the tap-mixpanel and sets up a virtual environment for the project. It's the first step in the quick start guide. ```bash make venv ``` -------------------------------- ### Configure tap-mixpanel `state.json` Example Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Example `state.json` file for tap-mixpanel, used to manage synchronization state and bookmarks. It helps in resuming interrupted syncs by tracking the last synced object and bookmark for each stream. ```json { "currently_syncing": "engage", "bookmarks": { "export": "2019-09-27T22:34:39.000000Z", "funnels": "2019-09-28T15:30:26.000000Z", "revenue": "2019-09-28T18:23:53Z" } } ``` -------------------------------- ### Bash Command Line Execution for tap-mixpanel Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Provides examples of how to execute the tap-mixpanel command-line interface. This includes creating a configuration file, running discovery mode to generate a catalog, setting up initial state, and performing sync operations, both to standard output and piped to a target. ```bash # Create config.json cat > config.json <", "project_timezone": "US/Pacific", "date_window_size": "30", "attribution_window": "5", "select_properties_by_default": "true", "denest_properties": "true", "export_events": ["Purchase", "Signup"] } EOF # Run discovery mode tap-mixpanel --config config.json --discover > catalog.json # Select streams in catalog.json (set "selected": true in metadata) # Create initial state cat > state.json < output.jsonl # Extract final state from output tail -1 output.jsonl > state.json # Subsequent runs resume from bookmark tap-mixpanel --config config.json --catalog catalog.json --state state.json | target-jsonl ``` -------------------------------- ### Configure tap-mixpanel `config.json` Example Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Example `config.json` file for the tap-mixpanel, specifying essential parameters like API secrets, date windows, and property handling. This file is crucial for authenticating with Mixpanel and defining data extraction behavior. ```json { "api_secret": "YOUR_API_SECRET", "date_window_size": "30", "attribution_window": "5", "project_timezone": "US/Pacific", "select_properties_by_default": "true", "denest_properties": "true", "start_date": "2019-01-01T00:00:00Z", "user_agent": "tap-mixpanel " } ``` -------------------------------- ### Run Linter (Bash) Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Runs the linter to check for code style and potential errors. This command requires the virtual environment and Python test dependencies to be installed. ```bash make pylint ``` -------------------------------- ### Run Unit Tests (Bash) Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Executes the unit tests for the project. This command assumes that the virtual environment and test dependencies have already been set up. ```bash make unit_test ``` -------------------------------- ### Run tap-mixpanel in Discovery Mode Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Command to run the tap-mixpanel in discovery mode, which generates a `catalog.json` file. This file is used to select specific objects and fields for data integration. ```bash tap-mixpanel --config config.json --discover > catalog.json ``` -------------------------------- ### Python: Generate Mixpanel Singer Catalog in Discovery Mode Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Illustrates how to run tap-mixpanel in discovery mode to generate a Singer catalog with schema definitions for all available Mixpanel streams. It utilizes the `discover` function from the `tap_mixpanel.discover` module and outputs the catalog to a 'catalog.json' file. Configuration options for properties and denesting are included. ```python import json from tap_mixpanel import main from tap_mixpanel.client import MixpanelClient from tap_mixpanel.discover import discover config = { 'api_secret': 'YOUR_API_SECRET', 'user_agent': 'tap-mixpanel ', 'start_date': '2024-01-01T00:00:00Z', 'project_timezone': 'US/Pacific', 'date_window_size': '30', 'attribution_window': '5', 'select_properties_by_default': 'true', 'denest_properties': 'true' } with MixpanelClient( api_secret=config['api_secret'], username=None, password=None, project_id=None, user_agent=config['user_agent'] ) as client: # Discover schemas and metadata catalog = discover( client=client, properties_flag=config.get('select_properties_by_default'), denest_properties=config.get('denest_properties', 'true') ) # Output catalog to file with open('catalog.json', 'w') as f: json.dump(catalog.to_dict(), f, indent=2) print(f"Discovered {len(catalog.streams)} streams") for stream in catalog.streams: print(f" - {stream.stream}: {len(stream.schema.to_dict()['properties'])} properties") ``` -------------------------------- ### Run tap-mixpanel in Sync Mode Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Command to execute the tap-mixpanel in sync mode, utilizing the provided `config.json` and `catalog.json`. This initiates the data extraction process, writing messages to standard output according to the Singer specification. ```bash > tap-mixpanel --config tap_config.json --catalog catalog.json ``` -------------------------------- ### Python: Initialize MixpanelClient for API Authentication Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Demonstrates how to initialize the MixpanelClient for making authenticated requests to Mixpanel APIs. Supports both API Secret and Username/Password authentication methods. The client automatically verifies access and region support upon initialization. It returns True upon successful connection. ```python from tap_mixpanel.client import MixpanelClient # API Secret authentication (recommended) with MixpanelClient( api_secret='YOUR_API_SECRET', username=None, password=None, project_id=None, user_agent='tap-mixpanel ', server='EU' # Optional: 'EU', 'IN', or None for US ) as client: # Client automatically verifies access on initialization # Returns True if connection successful data = client.request( method='GET', path='engage', params={'page_size': 100}, endpoint='engage' ) print(f"Retrieved {len(data.get('results', []))} records") # Username/Password authentication with MixpanelClient( api_secret=None, username='service_account@company.com', password='service_password', project_id=12345, user_agent='tap-mixpanel ' ) as client: response = client.request(method='GET', path='cohorts/list', endpoint='cohorts') ``` -------------------------------- ### Execute Incremental Sync with State Management in Python Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Performs incremental data extraction with bookmark-based state tracking for resumable data pipelines using MixpanelClient and the sync function. It requires configuration, a catalog, and optionally a previous state file for resuming. Outputs Singer messages to stdout. ```python import json from tap_mixpanel.client import MixpanelClient from tap_mixpanel.sync import sync from singer.catalog import Catalog config = { 'api_secret': 'YOUR_API_SECRET', 'user_agent': 'tap-mixpanel ', 'start_date': '2024-01-01T00:00:00Z', 'project_timezone': 'US/Pacific', 'date_window_size': '30', 'attribution_window': '5', 'select_properties_by_default': 'true', 'denest_properties': 'true', 'export_events': ['Page View', 'Button Click'] } # Load catalog with selected streams with open('catalog.json', 'r') as f: catalog_dict = json.load(f) catalog = Catalog.from_dict(catalog_dict) # Load previous state (if resuming) try: with open('state.json', 'r') as f: state = json.load(f) except FileNotFoundError: state = {} with MixpanelClient( api_secret=config['api_secret'], username=None, password=None, project_id=None, user_agent=config['user_agent'] ) as client: # Execute sync - outputs Singer messages to stdout sync( client=client, config=config, catalog=catalog, state=state, start_date=config['start_date'] ) # State is automatically updated during sync via singer.write_state() # Final state written to stdout as last message ``` -------------------------------- ### Pipe tap-mixpanel Output to `target-json` Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This command chains the output of `tap-mixpanel` to `target-json` to load the extracted data into JSON files, useful for verification. The output is directed to a `state.json` file. ```bash > tap-mixpanel --config tap_config.json --catalog catalog.json | target-json > state.json ``` -------------------------------- ### Python Mixpanel API Error Handling with Backoff Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Demonstrates how to handle API errors from the Mixpanel client, including rate limits and server errors, with automatic retries using exponential backoff. It shows how to instantiate the client and handle specific exceptions like unauthorized access and rate limiting. ```python from tap_mixpanel.client import ( MixpanelClient, MixpanelRateLimitsError, MixpanelUnauthorizedError, Server5xxError ) import backoff # Client has built-in backoff decorators # @backoff.on_exception(backoff.expo, (Server5xxError, MixpanelRateLimitsError), max_tries=5) with MixpanelClient( api_secret='YOUR_API_SECRET', user_agent='tap-mixpanel ' ) as client: try: data = client.request( method='GET', path='engage', params={'page_size': 1000}, endpoint='engage' ) print(f"Success: {len(data.get('results', []))} records") except MixpanelUnauthorizedError as e: print(f"Authentication failed: {e}") # Check api_secret in config except MixpanelRateLimitsError as e: print(f"Rate limit hit: {e}") # Automatic retry with exponential backoff except Server5xxError as e: print(f"Server error: {e}") # Automatic retry up to 7 times (BACKOFF_MAX_TRIES_REQUEST) except Exception as e: print(f"Unexpected error: {e}") ``` -------------------------------- ### Mixpanel Revenue API Stream Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This stream fetches revenue data from Mixpanel's engage endpoint. It supports incremental loading based on date. ```APIDOC ## GET /api/2.0/engage/revenue ### Description Fetches revenue data from Mixpanel's engage endpoint. ### Method GET ### Endpoint https://mixpanel.com/api/2.0/engage/revenue ### Parameters #### Query Parameters - **from_date** (string) - Required - The start date for the revenue data. - **to_date** (string) - Required - The end date for the revenue data. - **unit** (string) - Optional - The time unit for the revenue data (e.g., 'day'). ### Request Example ```json { "from_date": "2023-01-01", "to_date": "2023-01-31", "unit": "day" } ``` ### Response #### Success Response (200) - **results** (array) - An array containing revenue data for each date. - **date** (string) - The date of the revenue data. - **revenue** (number) - The total revenue for the given date. #### Response Example ```json { "results": [ { "date": "2023-01-15", "revenue": 150.75 } ] } ``` ``` -------------------------------- ### Dry Run with Stitch Import API (Bash) Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Executes a dry run of the tap-mixpanel data extraction to the Stitch Import API. This command pipes the output of tap-mixpanel to target-stitch, saving the final state to state.json. It requires configuration files for both tap and target. ```bash > tap-mixpanel --config tap_config.json --catalog catalog.json | target-stitch --config target_config.json --dry-run > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json ``` -------------------------------- ### Sync Specific Endpoint with Date Windowing in Python Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Syncs data from a specific endpoint (e.g., 'export') with configurable date windowing for large datasets. It requires a MixpanelClient, a catalog defining the stream, and configuration parameters like `date_window_size` and `attribution_window`. Outputs the total number of records synced and the final bookmark. ```python from tap_mixpanel.client import MixpanelClient from tap_mixpanel.sync import sync_endpoint from tap_mixpanel.streams import STREAMS from singer.catalog import Catalog, CatalogEntry, Schema config = { 'api_secret': 'YOUR_API_SECRET', 'user_agent': 'tap-mixpanel ', 'project_timezone': 'US/Pacific', 'date_window_size': '7', # Process 7 days at a time 'attribution_window': '5' } # Create minimal catalog for export stream catalog = Catalog([]) export_schema = Schema.from_dict({ 'type': 'object', 'properties': { 'event': {'type': 'string'}, 'time': {'type': 'string', 'format': 'date-time'}, 'distinct_id': {'type': 'string'} } }) catalog.streams.append(CatalogEntry( stream='export', tap_stream_id='export', schema=export_schema, key_properties=['mp_reserved_insert_id'], metadata=[] )) state = {'bookmarks': {}} with MixpanelClient( api_secret=config['api_secret'], username=None, password=None, project_id=None, user_agent=config['user_agent'] ) as client: total_records = sync_endpoint( client=client, catalog=catalog, state=state, start_date='2024-10-01T00:00:00Z', stream_name='export', path='export', endpoint_config=STREAMS['export'], bookmark_field='time', project_timezone=config['project_timezone'], days_interval=int(config['date_window_size']), attribution_window=int(config['attribution_window']), export_events=['Purchase', 'Signup'], denest_properties_flag='true' ) print(f"Synced {total_records} total records") print(f"Final bookmark: {state['bookmarks'].get('export')}") ``` -------------------------------- ### Python: Stream Mixpanel Export API Events with JSONL Parsing Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Shows how to stream raw event data from the Mixpanel Export API using the MixpanelClient. This method supports automatic JSONL parsing and date range filtering for memory-efficient data extraction. It handles potential exceptions during the export process. ```python from tap_mixpanel.client import MixpanelClient from datetime import datetime, timedelta with MixpanelClient(api_secret='YOUR_API_SECRET', user_agent='tap-mixpanel ') as client: # Export endpoint returns a generator for memory-efficient streaming from_date = (datetime.now() - timedelta(days=7)).strftime('%Y-%m-%d') to_date = datetime.now().strftime('%Y-%m-%d') params = f"from_date={from_date}&to_date={to_date}" try: events = client.request_export( method='GET', path='export', params=params, endpoint='export' ) # Process streaming records count = 0 for event_record in events: if event_record: print(f"Event: {event_record.get('event')}, Time: {event_record.get('time')}") count += 1 print(f"Total events streamed: {count}") except Exception as err: print(f"Export failed: {err}") ``` -------------------------------- ### Mixpanel Cohorts API Stream Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This stream lists all available cohorts from Mixpanel. It performs a full table scan on each load. ```APIDOC ## GET /api/2.0/cohorts/list ### Description Lists all available cohorts from Mixpanel. ### Method GET ### Endpoint https://mixpanel.com/api/2.0/cohorts/list ### Parameters None ### Request Example ```json { } ``` ### Response #### Success Response (200) - **id** (string) - The unique identifier for the cohort. - **name** (string) - The name of the cohort. #### Response Example ```json { "id": "cohort123", "name": "Power Users" } ``` ``` -------------------------------- ### Mixpanel API Authentication Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md Details on how to authenticate with the Mixpanel API using Basic Authorization with an encoded API secret. ```APIDOC ## Authentication ### Description The Mixpanel API uses Basic Authorization with the `api_secret` from the tap configuration, encoded in Base64. ### Method HTTP Header ### Header Format `Authorization: Basic ` ### Example If your `api_secret` is `your_secret_key`: 1. Encode `your_secret_key` to Base64: `your_secret_key` -> `eW91cl9zZWNyZXRfa2V5` 2. Use it in the header: `Authorization: Basic eW91cl9zZWNyZXRfa2V5` ``` -------------------------------- ### Configure tap-mixpanel `config.json` for Specific Events Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md An addition to the `config.json` that allows specifying a list of events to export from the Mixpanel Raw export API. This is useful for targeting specific data points. ```bash "export_events": ["event_one", "event_two"] ``` -------------------------------- ### Mixpanel Annotations API Stream Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This stream retrieves annotations from Mixpanel. It performs a full table scan on each load. ```APIDOC ## GET /api/2.0/annotations ### Description Retrieves annotations from the Mixpanel Annotations API. ### Method GET ### Endpoint https://mixpanel.com/api/2.0/annotations ### Parameters #### Query Parameters - **from_date** (string) - Required - The start date for the annotations. - **to_date** (string) - Required - The end date for the annotations. ### Request Example ```json { "from_date": "2023-01-01", "to_date": "2023-01-31" } ``` ### Response #### Success Response (200) - **date** (string) - The date of the annotation. - **description** (string) - The description of the annotation. #### Response Example ```json { "date": "2023-01-10", "description": "New feature launch" } ``` ``` -------------------------------- ### Transform Mixpanel Records with Property Denesting in Python Source: https://context7.com/hotgluexyz/tap-mixpanel/llms.txt Transforms raw Mixpanel records by flattening nested properties and converting reserved fields using `transform_record` and `denest_properties`. It handles both export and engage event types, converting timestamps to UTC and prefixing reserved fields. Requires the record data, stream name, project timezone, and a flag for denesting. ```python from tap_mixpanel.transform import transform_record, denest_properties # Export event transformation export_record = { 'event': 'Page View', 'time': 1640995200, # Unix timestamp in project timezone 'properties': { '$browser': 'Chrome', '$city': 'San Francisco', 'user_id': 'abc123', 'plan_type': 'premium' } } transformed = transform_record( record=export_record, stream_name='export', project_timezone='US/Pacific', denest_properties_flag='true' ) print(transformed) # Output: # { # 'event': 'Page View', # 'time': '2022-01-01T00:00:00.000000Z', # Converted to UTC # 'mp_reserved_browser': 'Chrome', # $ prefix converted # 'mp_reserved_city': 'San Francisco', # 'user_id': 'abc123', # 'plan_type': 'premium' # } # Engage profile transformation engage_record = { '$distinct_id': 'user_123', '$properties': { '$email': 'user@example.com', '$name': 'John Doe', 'signup_date': '2024-01-15' } } transformed_engage = transform_record( record=engage_record, stream_name='engage', project_timezone='UTC', denest_properties_flag='true' ) print(transformed_engage) # Output: # { # 'distinct_id': 'user_123', # 'mp_reserved_email': 'user@example.com', # 'mp_reserved_name': 'John Doe', # 'signup_date': '2024-01-15' # } ``` -------------------------------- ### Mixpanel Engage API Stream (People/Users) Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This stream extracts people/user data from the Mixpanel Engage API. It performs a full table scan on each load. ```APIDOC ## GET /api/2.0/engage ### Description Extracts people/user data from the Mixpanel Engage API. ### Method GET ### Endpoint https://mixpanel.com/api/2.0/engage ### Parameters #### Query Parameters - **from_date** (string) - Required - The start date for the data export. - **to_date** (string) - Required - The end date for the data export. ### Request Example ```json { "from_date": "2023-01-01", "to_date": "2023-01-31" } ``` ### Response #### Success Response (200) - **distinct_id** (string) - The unique identifier for the user. - **properties** (object) - Properties associated with the user. #### Response Example ```json { "distinct_id": "user123", "properties": { "$email": "user@example.com", "last_seen": "2023-01-15T10:30:00Z" } } ``` ``` -------------------------------- ### Mixpanel Export API Stream Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This stream pulls raw event data from the Mixpanel Event Export API. It supports incremental loading and allows filtering by specific events. ```APIDOC ## GET /api/2.0/export ### Description Pulls raw event data from the Mixpanel Event Export API. ### Method GET ### Endpoint https://data.mixpanel.com/api/2.0/export ### Parameters #### Query Parameters - **from_date** (string) - Required - The start date for the data export. - **to_date** (string) - Required - The end date for the data export. - **export_events** (array of strings) - Optional - A list of specific events to export. ### Request Example ```json { "from_date": "2023-01-01", "to_date": "2023-01-31", "export_events": ["sign_up", "purchase"] } ``` ### Response #### Success Response (200) - **event** (string) - The name of the event. - **time** (string) - The timestamp of the event. - **distinct_id** (string) - The unique identifier for the user. - **properties** (object) - Custom properties associated with the event. #### Response Example ```json { "event": "purchase", "time": "2023-01-15T10:30:00Z", "distinct_id": "user123", "properties": { "amount": 50, "currency": "USD" } } ``` ``` -------------------------------- ### Mixpanel Cohort Members API Stream Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This stream retrieves members of specific cohorts from Mixpanel. It queries the engage endpoint for each cohort. ```APIDOC ## GET /api/2.0/engage (filtered by cohort) ### Description Retrieves members of specific cohorts from Mixpanel by querying the engage endpoint with a cohort filter. ### Method GET ### Endpoint https://mixpanel.com/api/2.0/engage ### Parameters #### Query Parameters - **filter_by_cohort** (string) - Required - The ID of the cohort to filter members by. ### Request Example ```json { "filter_by_cohort": "cohort123" } ``` ### Response #### Success Response (200) - **distinct_id** (string) - The unique identifier for the user within the specified cohort. #### Response Example ```json { "distinct_id": "user456" } ``` ``` -------------------------------- ### Mixpanel Funnels API Stream Source: https://github.com/hotgluexyz/tap-mixpanel/blob/master/README.md This stream retrieves funnel data from Mixpanel, combining data from both the export and funnels endpoints. It supports incremental loading based on date. ```APIDOC ## GET /api/2.0/export & GET /api/2.0/funnels ### Description Retrieves funnel data from Mixpanel, combining data from the export and funnels endpoints. ### Method GET ### Endpoint - Endpoint 1: https://data.mixpanel.com/api/2.0/export - Endpoint 2: https://mixpanel.com/api/2.0/funnels ### Parameters #### Query Parameters - **funnel_id** (string) - Required - The ID of the funnel to retrieve data for. - **from_date** (string) - Required - The start date for the data export. - **to_date** (string) - Required - The end date for the data export. - **unit** (string) - Optional - The time unit for the funnel data (e.g., 'day'). ### Request Example ```json { "funnel_id": "12345", "from_date": "2023-01-01", "to_date": "2023-01-31", "unit": "day" } ``` ### Response #### Success Response (200) - **results** (array) - An array containing funnel data for each date. - **date** (string) - The date of the funnel data. - **funnel_id** (string) - The ID of the funnel. - **steps** (array) - Details about each step in the funnel. #### Response Example ```json { "results": [ { "date": "2023-01-15", "funnel_id": "12345", "steps": [ { "name": "Step 1", "count": 100 }, { "name": "Step 2", "count": 50 } ] } ] } ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.