### Install dependencies Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/windsurf/fda_vet_tutorial/fda_vet_tutorial.md Commands to set up a virtual environment and install requirements. ```bash python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install fivetran-connector-sdk pip install -r requirements.txt ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/update_and_delete/delete_example/README.md Use this command to create a new project structure based on the delete example template. ```bash fivetran init --template examples/common_patterns_for_connectors/update_and_delete/delete_example ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/update_and_delete/update_example/README.md Use this command to scaffold a new project using the update example template. ```bash fivetran init --template examples/common_patterns_for_connectors/update_and_delete/update_example ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/priority_first_sync_for_high_volume_initial_syncs/README.md Run this command to scaffold a new project using the priority-first sync template. ```bash fivetran init --template examples/common_patterns_for_connectors/priority_first_sync_for_high_volume_initial_syncs ``` -------------------------------- ### Example GitHub API GET Request for Repositories Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/github/README.md This is an example of a GET request to retrieve repositories for an organization, with a page size of 100 records. This is the first step in the pagination flow. ```http GET /orgs/{org}/repos?per_page=100 ``` -------------------------------- ### Example Connector Development Workflow Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/ai_agents/vscode_with_github_copilot/README.md Use descriptive comments to guide GitHub Copilot in generating connector code, including imports, schema definition, and table structures. ```python # Start with this comment to get Copilot suggestions # Create a connector for that fetches [data type] # Copilot will suggest imports from fivetran_connector_sdk import Connector, Logging as log, Operations as op import json # Continue with descriptive comments for better suggestions # Define schema function with primary key for [table_name] def schema(configuration: dict): # Copilot will suggest the schema structure return [ {"table": "your_table", "primary_key": ["id"]} ] ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/incremental_sync_strategies/keyset_pagination/README.md Use this command to scaffold a new project using the keyset pagination template. ```bash fivetran init --template examples/common_patterns_for_connectors/incremental_sync_strategies/keyset_pagination ``` -------------------------------- ### GitHub App Configuration Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/github/README.md Add these items to your connector's configuration.json file for GitHub App authentication. Ensure you have your App ID, private key, organization name, and Installation ID. ```json { "app_id": "YOUR_APP_ID", "private_key": "-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----", "organization": "YOUR_GITHUB_ORGANIZATION", "installation_id": "YOUR_INSTALLATION_ID" } ``` -------------------------------- ### Initialize Project Directory Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/pokeapi_tutorial/pokeapi_connector_tutorial.md Commands to create and navigate into the project directory. ```bash mkdir pokemon-connector cd pokemon-connector ``` -------------------------------- ### Install Dependencies Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/cursor/fda_food_tutorial/fda_food_connector/README.md Installs the necessary Python dependencies for the connector. Ensure you have Python 3.10-3.13 and the Fivetran Connector SDK installed. ```bash pip install -r requirements.txt ``` -------------------------------- ### Initialize Project Directory Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/fda_drug_tutorial/fda_drug_tutorial.md Commands to create and navigate to the project folder. ```bash mkdir fda-drug-connector cd fda-drug-connector ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/hashes/README.md Command to bootstrap a new project using the hashed primary key template. ```bash fivetran init --template examples/common_patterns_for_connectors/hashes ``` -------------------------------- ### Install Connector Dependencies Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/fda_drug_tutorial/fda_drug_connector/README.md Install the necessary Python dependencies for the Fivetran Connector SDK. Ensure you have Python 3.10-3.13 installed. ```bash pip install -r requirements.txt ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/tracking_tables/README.md Use the CLI to scaffold a new project using the table tracking template. ```bash fivetran init --template examples/common_patterns_for_connectors/tracking_tables ``` -------------------------------- ### Install MySQL Driver Dependencies Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/importing_external_drivers/README.md Shell script to install MySQL database development files required by the MySQLdb package. It updates system packages and installs default-libmysqlclient-dev. The script can accept a configuration.json file for specific installation parameters. ```bash #!/bin/bash # Update package lists apt-get update # Install MySQL client development libraries apt-get install -y default-libmysqlclient-dev # Optionally, you can pass configuration parameters if needed # For example, if your installation script needed specific settings: # if [ -f "$1" ]; then # source "$1" # # Use variables from the sourced file # fi echo "MySQL client development libraries installed successfully." ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/supabase/README.md Run this command to scaffold a new project using the Supabase template. ```bash fivetran init --template connectors/supabase ``` -------------------------------- ### Install Dependencies Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/fda_drug_tutorial/fda_drug_tutorial.md Commands to install the Fivetran SDK and project requirements. ```bash pip install fivetran-connector-sdk pip install -r requirements.txt ``` -------------------------------- ### Install Dependencies - Bash Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/cursor/fda_food_tutorial/fda_food_tutorial.md Command to install all Python packages listed in the requirements.txt file. ```bash pip install -r requirements.txt ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/pagination/keyset/README.md Command to scaffold a new project using the keyset pagination template. ```bash fivetran init --template examples/common_patterns_for_connectors/pagination/keyset ``` -------------------------------- ### Install Fivetran Connector SDK Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/FAQ.md Use this command to install the SDK package in your local Python environment. ```bash pip install fivetran-connector-sdk ``` -------------------------------- ### CloudAMQP Connection URL Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/rabbitmq/README.md Example JSON configuration for connecting to RabbitMQ via CloudAMQP using AMQPS for secure communication. ```json { "connection_url": "amqps://username:password@host.cloudamqp.com/vhost" } ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/sam_gov/README.md Use this command to scaffold a new project using the SAM.gov template. ```bash fivetran init --template connectors/sam_gov ``` -------------------------------- ### Install Black Formatter Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/CONTRIBUTING.md Install the Black Python package to format your Python code. This is necessary for checking and fixing formatting issues locally. ```bash pip install black ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/snipeitapp/README.md Use this command to set up the project structure and configuration files for a new connector project. ```bash fivetran init --template connectors/snipeitapp ``` -------------------------------- ### Install Flake8 for Linting Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/CONTRIBUTING.md Install the Flake8 Python package to check for potential errors, stylistic issues, and PEP8 compliance in your Python code. ```bash pip install flake8 ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/partech/README.md Use this command to scaffold a new project using the Partech template. ```bash fivetran init --template connectors/partech ``` -------------------------------- ### Install VS Code Extensions Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/ai_agents/vscode_with_github_copilot/README.md Install essential VS Code extensions for Python development and GitHub Copilot using the command line. ```bash code --install-extension ms-python.python code --install-extension GitHub.copilot code --install-extension eamodio.gitlens code --install-extension usernamehw.errorlens ``` -------------------------------- ### DragonflyDB Connector Key Pattern Examples Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/dragonfly_db/README.md Examples of key patterns that can be used to filter which keys are synced from DragonflyDB. The wildcard '*' matches all keys. ```text * ``` ```text cache:* ``` ```text session:* ``` ```text user:*:profile ``` ```text rate_limit:* ``` -------------------------------- ### Set Up New Connector Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/windsurf/fda_vet_tutorial/fda_vet_tutorial.md Commands to create a new directory for your connector, navigate into it, and set up a Python virtual environment. ```bash mkdir fda-vet-connector && cd fda-vet-connector python -m venv venv source venv/bin/activate # Windows: venv\Scripts\activate ``` -------------------------------- ### Install Fivetran Connector SDK Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/CLAUDE_README.md Ensure you have the required tools and install the Fivetran Connector SDK using pip. Python version 3.10-3.13 is recommended. ```bash python --version # Should be 3.10-3.13 pip install fivetran-connector-sdk ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/quickstart_examples/base_64_encoding_decoding/README.md Use this command to scaffold a new project using the base64 encoding/decoding template. ```bash fivetran init --template examples/quickstart_examples/base_64_encoding_decoding ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/zigpoll/README.md Use this command to scaffold a new project using the Zigpoll template. ```bash fivetran init --template connectors/zigpoll ``` -------------------------------- ### Connector Configuration File Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/quickstart_examples/configuration/README.md Example of a configuration file containing a base64 encoded encryption key. Ensure this file is not checked into version control. ```json { "my_key": "EKlFpH8sZmdhhZ9lGhezgMTwAw3_Y2e7wbco7Gxt3SA=" } ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/sensource/README.md Use this command to scaffold a new project structure using the SenSource template. ```bash fivetran init --template connectors/sensource ``` -------------------------------- ### Self-hosted RabbitMQ Connection URL Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/rabbitmq/README.md Example JSON configuration for connecting to a self-hosted RabbitMQ instance with custom credentials and AMQPS for secure communication. ```json { "connection_url": "amqps://myuser:mypassword@rabbitmq.example.com:5671/production" } ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/sql_server/README.md Use this command to scaffold a new project structure using the SQL Server template. ```bash fivetran init --template connectors/sql_server ``` -------------------------------- ### IBM Informix Connector Schema Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/ibm/ibm_informix_using_ibm_db/README.md An example schema definition for the IBM Informix connector, specifying the table and its primary key for data synchronization. ```json { "table": "sample_table", "primary_key": ["id"] } ``` -------------------------------- ### Local RabbitMQ Connection URL Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/rabbitmq/README.md Example JSON configuration for connecting to a local RabbitMQ instance using default credentials and the AMQP protocol. ```json { "connection_url": "amqp://guest:guest@localhost:5672/" } ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/prefect/README.md Use this command to scaffold a new project structure using the Prefect template. ```bash fivetran init --template connectors/prefect ``` -------------------------------- ### Example Tobacco Problem Report JSON Response Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/vscode/fda_tobacco_tutorial/notes.txt This is an example of the JSON response structure for a tobacco problem report query, showing metadata and result details. ```json { "meta": { "disclaimer": "Do not rely on openFDA to make decisions regarding medical care. While we make every effort to ensure that data is accurate, you should assume all results are unvalidated. We may limit or otherwise restrict your access to the API in line with our Terms of Service.", "terms": "https://open.fda.gov/terms/", "license": "https://open.fda.gov/license/", "last_updated": "2025-05-02", "results": { "skip": 0, "limit": 1, "total": 1265 } }, "results": [ { "date_submitted": "04/04/2019", "nonuser_affected": "No", "reported_health_problems": [ "Seizures" ], "number_tobacco_products": 1, "report_id": 797, "number_health_problems": 1, "reported_product_problems": [ "No information provided" ], "tobacco_products": [ "Electronic cigarette, electronic nicotine or vaping product (E-cigarette, e-cigars, e-hookahs, e-pipes, vape pens, hookah pens, and personal vaporizers; E-liquids, e-juice or vape juice)" ], "number_product_problems": 0 } ] } ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/gpg_private_keys/README.md Command to scaffold a new project using the GPG private keys template. ```bash fivetran init --template examples/common_patterns_for_connectors/gpg_private_keys ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/apache_druid/using_pydruid/README.md Run this command to scaffold a new project using the Apache Druid template. ```bash fivetran init --template connectors/apache_druid/using_pydruid ``` -------------------------------- ### Requirements File Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/sensor_tower/README.md The `requirements.txt` file lists additional Python libraries. For this connector, `requests` and `dateutil` are examples, but avoid declaring pre-installed Fivetran SDK packages. ```text requests dateutil ``` -------------------------------- ### Example GitHub API Link Header for Pagination Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/github/README.md This is an example of a Link header returned by the GitHub API, indicating the URL for the next page of results. This is used to implement cursor-based pagination. ```http ; rel="next" ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/suitedash/README.md Use this command to scaffold a new project structure using the SuiteDash template. ```bash fivetran init --template connectors/suitedash ``` -------------------------------- ### Connector Configuration File Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/owasp_api_vulns/README.md Example of the configuration.json file for the OWASP API Vulnerabilities connector. It includes required and optional parameters for API key, sync behavior, logging, and CWE filtering. ```json { "api_key": "", "force_full_sync": "", "write_temp_files": "", "logging_level": "", "cwe_ids": "" } ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/quest_db/README.md Use this command to set up the project structure and configuration files for a new QuestDB connector project. ```bash fivetran init --template connectors/quest_db ``` -------------------------------- ### Fivetran SDK Checkpointing State Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/agents/ft-csdk-generate.md Provides an example of a state dictionary used for incremental syncs in the Fivetran Connector SDK. It includes cursor values and table-specific cursors, demonstrating how to implement checkpoint logic. ```python state = { "cursor": "2024-03-20T10:00:00Z", "offset": 100, "table_cursors": { "table1": "2024-03-20T10:00:00Z", "table2": "2024-03-20T09:00:00Z" } } op.checkpoint(state=state) ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/authentication/api_key/README.md Command to scaffold a new project using the API key authentication template. ```bash fivetran init --template examples/common_patterns_for_connectors/authentication/api_key ``` -------------------------------- ### Start Mock API with Rate Limiting Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/backoff/README.md Starts the mock source API with rate limiting enabled and a capacity of 1. This is useful for testing and observing the connector's backoff and retry behavior under constrained conditions. ```bash playground start --rate-limit --capacity 1 ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/schema_from_database/README.md Use the Fivetran CLI to scaffold a new project based on the schema-from-database template. ```bash fivetran init --template examples/common_patterns_for_connectors/schema_from_database ``` -------------------------------- ### Flatten Nested Dictionary Example Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/oura_ring/README.md This function flattens nested dictionaries, useful for handling nested contributor objects from the Oura API. For example, `{"contributors": {"deep_sleep": 99}}` becomes `{"contributors_deep_sleep": 99}`. ```python def flatten_dict(d, parent_key='', sep='_'): items = [] for k, v in d.items(): new_key = parent_key + sep + k if parent_key else k if isinstance(v, dict): items.extend(flatten_dict(v, new_key, sep=sep).items()) else: items.append((new_key, v)) return dict(items) ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/pagination/next_page_url/README.md Use the Fivetran CLI to scaffold a new project using the next-page URL pagination template. ```bash fivetran init --template examples/common_patterns_for_connectors/pagination/next_page_url ``` -------------------------------- ### Example API Response Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/fda_drug_tutorial/notes.txt A sample JSON response structure returned by the openFDA API. ```json { "meta": { "disclaimer": "Do not rely on openFDA to make decisions regarding medical care. While we make every effort to ensure that data is accurate, you should assume all results are unvalidated. We may limit or otherwise restrict your access to the API in line with our Terms of Service.", "terms": "https://open.fda.gov/terms/", "license": "https://open.fda.gov/license/", "last_updated": "2025-06-09", "results": { "skip": 0, "limit": 1, "total": 112092 } }, "results": [ { "product_ndc": "80136-063", "generic_name": "Loperamide HCl", "labeler_name": "Northeast Pharma", "brand_name": "Anti-Diarrheal", "active_ingredients": [ { "name": "LOPERAMIDE HYDROCHLORIDE", "strength": "2 mg/1" } ], "finished": true, "packaging": [ { "package_ndc": "80136-063-11", "description": "4 BLISTER PACK in 1 CARTON (80136-063-11) / 6 TABLET in 1 BLISTER PACK", "marketing_start_date": "20210122", "sample": false } ], "listing_expiration_date": "20261231", "openfda": { "manufacturer_name": [ ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/pagination/offset_based/README.md Use the Fivetran CLI to scaffold a new project based on the offset-based pagination template. ```bash fivetran init --template examples/common_patterns_for_connectors/pagination/offset_based ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/quickstart_examples/hello/README.md Use this command to create a new Fivetran Connector SDK project based on the 'hello' quickstart template. It sets up the project structure and configuration files. ```bash fivetran init --template examples/quickstart_examples/hello ``` -------------------------------- ### Check Fivetran SDK Version Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/tutorials/claude/CLAUDE.md Displays the currently installed version of the Fivetran Connector SDK. ```bash fivetran version ``` -------------------------------- ### Define destination table structure Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/veeva_vault/basic_auth/README.md Example of the table schema definition for the document__v object type. ```json { "table": "document__v", "primary_key": ["id"], "columns": "inferred from type_fields" } ``` -------------------------------- ### Initialize a new Connector SDK project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/connectors/apache_druid/using_sql/README.md Use this command to scaffold a new project using the Apache Druid template. ```bash fivetran init --template connectors/apache_druid/using_sql ``` -------------------------------- ### Configure Python Environment Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/all_things_ai/ai_agents/vscode_with_github_copilot/README.md Set up a virtual environment and install the Fivetran Connector SDK using pip. ```bash python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install fivetran_connector_sdk ``` -------------------------------- ### Initialize Connector SDK Project Source: https://github.com/fivetran/fivetran_connector_sdk/blob/main/examples/common_patterns_for_connectors/authentication/session_token/README.md Command to scaffold a new project using the session token authentication template. ```bash fivetran init --template examples/common_patterns_for_connectors/authentication/session_token ```