### Install flowctl on Linux

Source: https://docs.estuary.dev/guides/get-started-with-flowctl

Downloads the flowctl binary for Linux x86-64, makes it executable, and adds it to the system's PATH.

```shell
sudo curl -o /usr/local/bin/flowctl -L 'https://github.com/estuary/flow/releases/latest/download/flowctl-x86_64-linux' && sudo chmod +x /usr/local/bin/flowctl  
```

--------------------------------

### Install flowctl on macOS

Source: https://docs.estuary.dev/guides/get-started-with-flowctl

Downloads the flowctl binary for macOS (supporting Intel and M1 chips), makes it executable, and adds it to the system's PATH.

```shell
sudo curl -o /usr/local/bin/flowctl -L 'https://github.com/estuary/flow/releases/latest/download/flowctl-multiarch-macos' && sudo chmod +x /usr/local/bin/flowctl  
```

--------------------------------

### Install flowctl with Homebrew on macOS

Source: https://docs.estuary.dev/guides/get-started-with-flowctl

Installs flowctl on macOS using the Homebrew package manager by tapping the Estuary repository and installing the flowctl formula.

```shell
brew tap estuary/flowctl  
brew install flowctl  
```

--------------------------------

### PostgreSQL Connection String Example

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/neon-postgres

An example of a PostgreSQL connection string used to provide database credentials and connection details to Estuary Flow. This string format is essential for establishing a connection to a Neon PostgreSQL database.

```sql
postgres://cdc_role:AbC123dEf@ep-cool-darkness-123456.us-east-2.aws.neon.tech/dbname?sslmode=require
```

--------------------------------

### Estuary SSO Setup and Details

Source: https://docs.estuary.dev/getting-started/sso-setup

Information required for initial SSO setup with Estuary and the specific SAML 2.0 details for integration with identity providers.

```APIDOC
## Estuary SSO Integration

### Description
Estuary supports Single Sign-On (SSO) for user authentication, allowing you to log in using your existing identity provider. This section outlines the information needed from your end for initial setup and provides Estuary's SAML 2.0 details for configuring your identity provider.

### Initial SSO Setup
To enable SSO for your organization, contact Estuary support and provide the following:

1.  **Metadata URL or Metadata XML File**: The metadata URL is preferred. If unavailable, a metadata XML file can be provided.
2.  **Company Domain**: The domain(s) associated with your company's email addresses.

### Estuary SAML 2.0 Details
Estuary utilizes Supabase for SSO authentication via SAML 2.0. The following details are required to configure your identity provider:

| Name              | Value                                                                  |
| ----------------- | ---------------------------------------------------------------------- |
| `EntityID`      | `https://eyrcnmuzzyriypdajwdk.supabase.co/auth/v1/sso/saml/metadata`    |
| Metadata URL      | `https://eyrcnmuzzyriypdajwdk.supabase.co/auth/v1/sso/saml/metadata`    |
| Metadata URL (download) | `https://eyrcnmuzzyriypdajwdk.supabase.co/auth/v1/sso/saml/metadata?download=true` |
| ACS URL           | `https://eyrcnmuzzyriypdajwdk.supabase.co/auth/v1/sso/saml/acs`         |
| SLO URL           | `https://eyrcnmuzzyriypdajwdk.supabase.co/auth/v1/sso/slo`              |
| `NameID`          | Required: `emailAddress` or `persistent`                               |

**Alternative Names**: 
*   Metadata URL is also known as 'Audience URL' or 'SP Entity ID'.
*   ACS URL is also referred to as 'Single Sign-on URL' in Okta.

**Attribute Requirements**:
*   The email address of the user is **required**. It can be returned using the `email` key or other supported attribute names by Supabase.
*   Optionally, you can return a `displayName` attribute containing the user's full name.
```

--------------------------------

### Install flowctl on Linux

Source: https://docs.estuary.dev/concepts/flowctl

Installs the flowctl binary on Linux systems by downloading it from GitHub, making it executable, and adding it to the system's PATH.

```shell
sudo curl -o /usr/local/bin/flowctl -L 'https://github.com/estuary/flow/releases/latest/download/flowctl-x86_64-linux' && sudo chmod +x /usr/local/bin/flowctl
```

--------------------------------

### SFTP File Capture Order Example

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/sftp

Illustrates the file capturing order for the SFTP connector. The first example shows default behavior (modification time), and the second shows behavior when 'Ascending Keys' is enabled (lexical order).

```text
/root/
  a/
  b/data.csv
  c/
```

--------------------------------

### Start PostgreSQL and Data Generator Services

Source: https://docs.estuary.dev/getting-started/tutorials/postgresql_cdc_to_snowflake

Starts PostgreSQL and the fake data generator service using Docker Compose. The '-d' flag can be used to run the containers in the background.

```bash
docker compose up
```

--------------------------------

### Shopify Connector Configuration (YAML)

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/shopify-native

This YAML configuration defines the setup for the Shopify connector, specifying the Docker image, authentication credentials, start date, store ID, and data resource bindings for capturing product data.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-shopify-native:dev
        config:
            advanced:
                window_size: P30D
            credentials:
                credentials_title: Private App Credentials
                access_token: <secret>
            start_date: "2025-01-16T12:00:00Z"
            store: <store ID>
    bindings:
      - resource:
          name: products
        target: ${PREFIX}/products

```

--------------------------------

### Backfills and Performance Considerations

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/neon-postgres

Explanation of the default backfill behavior and how to skip backfills for large tables.

```APIDOC
## Backfills and Performance Considerations

By default, the PostgreSQL capture connector performs a _backfill_ upon initiation, capturing the current state of targeted tables before transitioning to ongoing change event capture. This ensures a complete initial data view.

For very large tables, skipping the backfill might be desirable. This can be configured on a per-table basis using the `skip_backfills` property.
```

--------------------------------

### Publish Specifications with Flowctl

Source: https://docs.estuary.dev/concepts/flowctl

Provides an example of publishing specifications using the `flowctl catalog publish` command.

```bash
flowctl catalog publish
```

--------------------------------

### Install Bytewax and Kafka Client

Source: https://docs.estuary.dev/reference/Connectors/materialization-connectors/Dekaf/bytewax

Installs the necessary Python libraries for Bytewax and Kafka integration. This is the first step to setting up a Bytewax dataflow.

```bash
pip install bytewax kafka-python  
```

--------------------------------

### Install flowctl on Mac

Source: https://docs.estuary.dev/concepts/flowctl

Installs the flowctl binary on macOS systems by downloading it from GitHub, making it executable, and adding it to the system's PATH.

```shell
sudo curl -o /usr/local/bin/flowctl -L 'https://github.com/estuary/flow/releases/latest/download/flowctl-multiarch-macos' && sudo chmod +x /usr/local/bin/flowctl
```

--------------------------------

### Create Postgres Source Connector

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/neon-postgres

Steps to create a new Neon PostgreSQL source connector in the Estuary Flow web UI.

```APIDOC
## Create a Postgres Source Connector in Estuary Flow

1. In the Estuary Flow web UI, select **Sources** from the left navigation bar and click **New Capture**.
2. In the connector catalog, choose **Neon PostgreSQL** and click **Connect**.
3. Enter the connection details for your Neon database. These details can be found in your Neon connection string.
   - **Name**: Name of the Capture connector
   - **Server Address**: The host or host:port of the Neon database (e.g., `ep-cool-darkness-123456.us-east-2.aws.neon.tech:5432`)
   - **User**: The database user for authentication (e.g., `cdc_role`)
   - **Password**: The password for the database user (e.g., `AbC123dEf`)
   - **Database**: The logical database name to capture from (e.g., `dbname`)
4. Click **Next**. Estuary Flow will scan the source database for replicable tables. Select tables and optionally rename destination streams.
5. Click **Save and Publish** to provision the connector and start the backfill process.
```

--------------------------------

### Sample Configuration

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/zendesk-support-native

An example of how to configure the Zendesk Support connector with API token authentication.

```APIDOC
## Sample Configuration

This sample demonstrates a basic configuration for the Zendesk Support connector using API token authentication to capture ticket data.

```yaml
captures:
  ${PREFIX}/tickets:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-zendesk-support-native:dev
        config:
            advanced:
              incremental_export_page_size: 1000
            credentials:
              credentials: Email & API Token
              username: user@domain.com
              password: <secret>
            start_date: "2025-01-30T00:00:00Z"
            subdomain: my_subdomain
    bindings:
      - resource:
          name: tickets
        target: ${PREFIX}/tickets
```
```

--------------------------------

### Sample Capture Definition

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/neon-postgres

A minimal YAML configuration for a Postgres source capture definition in Estuary Flow.

```APIDOC
### Sample Capture Definition

```yaml
captures:  
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: "ghcr.io/estuary/source-postgres:dev"
        config:
          address: "localhost:5432"
          database: "postgres"
          user: "flow_capture"
          password: "secret"
    bindings:
      - resource:
          stream: ${TABLE_NAME}
          namespace: ${TABLE_NAMESPACE}
          syncMode: incremental
        target: ${PREFIX}/${COLLECTION_NAME}
```

This sample defines a basic capture for a single table. More complex configurations can include additional bindings for multiple tables.
```

--------------------------------

### Sample Capture Definition

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL

A minimal example of a capture definition for a PostgreSQL source.

```APIDOC
## Sample Capture Definition

A minimal capture definition will look like the following:

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: "ghcr.io/estuary/source-postgres:dev"
        config:
          address: "localhost:5432"
          database: "postgres"
          user: "flow_capture"
          password: "secret"
    bindings:
      - resource:
          stream: ${TABLE_NAME}
          namespace: ${TABLE_NAMESPACE}
          syncMode: incremental
        target: ${PREFIX}/${COLLECTION_NAME}
```

Your capture definition will likely be more complex, with additional bindings for each table in the source database.
```

--------------------------------

### Postgres Source Connector Configuration Properties

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/neon-postgres

Detailed properties for configuring the Postgres source connector, including endpoint and binding configurations.

```APIDOC
## Configuration

Connectors can be configured via the Flow web app or by directly editing the catalog specification file.

### Properties

#### Endpoint

| Property Name | Title | Description | Type | Required/Default |
|---|---|---|---|---|
| `/address` | Address | The host or host:port at which the database can be reached. | string | Required |
| `/database` | Database | Logical database name to capture from. | string | Required, `"postgres"` |
| `/user` | User | The database user to authenticate as. | string | Required, `"flow_capture"` |
| `/password` | Password | Password for the specified database user. | string | Required |
| `/advanced` | Advanced Options | Options for advanced users. | object |  |
| `/advanced/backfill_chunk_size` | Backfill Chunk Size | Number of rows to fetch per backfill query. | integer | `4096` |
| `/advanced/publicationName` | Publication Name | Name of the PostgreSQL publication to replicate from. | string | `"flow_publication"` |
| `/advanced/skip_backfills` | Skip Backfills | Comma-separated list of tables to skip backfilling. | string |   |
| `/advanced/slotName` | Slot Name | Name of the PostgreSQL replication slot to replicate from. | string | `"flow_slot"` |
| `/advanced/watermarksTable` | Watermarks Table | Table for watermark writes during backfills (e.g., `"public.flow_watermarks"`). | string | `"public.flow_watermarks"` |
| `/advanced/sslmode` | SSL Mode | Overrides SSL connection behavior. | string |   |

#### Bindings

| Property Name | Title | Description | Type | Required/Default |
|---|---|---|---|---|
| `/namespace` | Namespace | The namespace/schema of the table. | string | Required |
| `/stream` | Stream | Table name. | string | Required |
| `/syncMode` | Sync mode | Connection method. Must be `incremental`. | string | Required, `incremental` |
```

--------------------------------

### Table Bindings

Source: https://docs.estuary.dev/reference/Connectors/materialization-connectors/apache-iceberg

Configuration for binding tables, specifying table names and optional alternative namespaces.

```APIDOC
## Table Bindings

### Binding Parameters
- **`/table`** (string): Table name. Required.
- **`/namespace`** (string): Alternative namespace for this table. Optional.
```

--------------------------------

### Sample Materialization Configuration

Source: https://docs.estuary.dev/reference/Connectors/materialization-connectors/google-gcs-parquet

Example configuration for setting up a materialization using the GCS Parquet connector.

```APIDOC
## Sample Materialization Configuration

```yaml
materializations:
  ${PREFIX}/${mat_name}:
    endpoint:
      connector:
        image: "ghcr.io/estuary/materialize-gcs-parquet:dev"
        config:
          bucket: "your-gcs-bucket-name"
          credentialsJson: "<your-service-account-key-json>"
          uploadInterval: "5m" # Optional, uses default if omitted
    bindings:
      - resource:
          path: "${COLLECTION_NAME}" # The collection to materialize
        source: "${PREFIX}/${COLLECTION_NAME}" # Source path for the materialization
```

**Note:** Replace `${PREFIX}`, `${mat_name}`, `your-gcs-bucket-name`, `<your-service-account-key-json>`, and `${COLLECTION_NAME}` with your specific values.
```

--------------------------------

### Create Real-Time View

Source: https://docs.estuary.dev/reference/Connectors/materialization-connectors/Dekaf/materialize

SQL example for creating a materialized view to analyze data changes from the Materialize source.

```APIDOC
## Create Real-Time View

### Description
Creates a real-time materialized view in Materialize to analyze incoming data from the `materialize_source`.

### Method
SQL

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```sql
CREATE MATERIALIZED VIEW my_view AS  
SELECT *  
FROM materialize_source;
```

### Response
#### Success Response (200)
Materialized view creation confirmation.

#### Response Example
```
(1 row)
```
```

--------------------------------

### Chargebee Native Connector Configuration Sample

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/chargebee-native

This is a sample configuration for the Chargebee Native connector, demonstrating how to set up credentials, site details, start date, product catalog version, and resource bindings for data capture.

```yaml
captures:  
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-chargebee-native:dev
        config:
          credentials:
            credentials_title: API Key
            api_key: <secret>
          site: mycompany
          start_date: "2024-01-01T00:00:00Z"
          product_catalog: "2.0"
    bindings:
      - resource:
          name: customers
        target: ${PREFIX}/customers
      - resource:
          name: subscriptions
        target: ${PREFIX}/subscriptions
```

--------------------------------

### Front Connector Configuration Sample

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/front

This YAML configuration sample demonstrates how to set up the Front source connector. It specifies the Docker image, API credentials, replication start date, and maps various Front resources (channels, contacts, conversations, etc.) to Flow collections.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-front:dev
        config:
          credentials:
            access_token: <secret>
          start_date: "2024-11-15T00:00:00Z"
    bindings:
      - resource:
          name: channels
        target: ${PREFIX}/channels
      - resource:
          name: contacts
        target: ${PREFIX}/contacts
      - resource:
          name: conversations
        target: ${PREFIX}/conversations
      - resource:
          name: events
        target: ${PREFIX}/events
      - resource:
          name: inboxes
        target: ${PREFIX}/inboxes
      - resource:
          name: tags
        target: ${PREFIX}/tags
      - resource:
          name: teammates
        target: ${PREFIX}/teammates
      - resource:
          name: teams
        target: ${PREFIX}/teams

```

--------------------------------

### Reuse Common Ingest Fixtures with YAML Anchors

Source: https://docs.estuary.dev/concepts/tests

This example illustrates reusing common test fixtures, specifically ingest steps, using YAML anchors and references. It defines a setup with common documents under an anchor named 'mySetup' and then reuses this setup in subsequent tests using a reference.

```yaml
tests:
  acmeCo/tests/one:
    - ingest: &mySetup
        collection: acmeCo/collection
        documents:
          - {id: 1, ...}
          - {id: 2, ...}
          ...
    - verify: ...

  acmeCo/tests/two:
    - ingest: *mySetup
    - verify: ...
```

--------------------------------

### Authenticate flowctl CLI session

Source: https://docs.estuary.dev/guides/get-started-with-flowctl

Logs into the flowctl CLI using an access token obtained from the Estuary web app. This command initiates a browser-based authentication flow.

```shell
flowctl auth login  
```

--------------------------------

### AWS IAM Policy for S3 Tables Access

Source: https://docs.estuary.dev/reference/Connectors/materialization-connectors/apache-iceberg

An example AWS IAM policy granting necessary permissions for S3 Tables access for both IAM users and EMR execution roles.

```APIDOC
## AWS IAM Policy for S3 Tables Access

### Description
This policy grants the necessary permissions for AWS IAM users and EMR execution roles to interact with S3 Tables, including reading and writing data and metadata.

### Method
IAM Policy Definition

### Endpoint
N/A (IAM Policy)

### Parameters
N/A

### Request Example
N/A

### Response
#### Success Response (200)
N/A (Policy document)

#### Response Example
```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3TablesAccess",
            "Effect": "Allow",
            "Action": [
                "s3tables:GetTableBucket",
                "s3tables:ListNamespaces",
                "s3tables:CreateNamespace",
                "s3tables:ListTables",
                "s3tables:CreateTable",
                "s3tables:PutTableData",
                "s3tables:GetTableData",
                "s3tables:UpdateTableMetadataLocation",
                "s3tables:GetTableMetadataLocation",
                "s3tables:DeleteTable"
            ],
            "Resource": [
                "arn:aws:s3tables:<region>:<aws-account-id>:bucket/<bucket>",
                "arn:aws:s3tables:<region>:<aws-account-id>:bucket/<bucket>/*"
            ]
        }
    ]
}
```
```

--------------------------------

### Gainsight NXT Configuration Sample

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/gainsight-nxt

Provides a sample configuration snippet for the Gainsight NXT connector, illustrating how to set up capture endpoints and bindings.

```APIDOC
## Gainsight NXT Configuration Sample

This sample demonstrates how to configure the Gainsight NXT connector, including specifying the Docker image, credentials, domain, start date, and resource bindings.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-gainsight-nxt:dev
        config:
          credentials:
            credentials_title: Private App Credentials
            api_token: <secret>
          domain: yourdomain
          start_date: "2024-01-01T00:00:00Z"
    bindings:
      - resource:
          name: companies
        target: ${PREFIX}/companies
      - resource:
          name: users
        target: ${PREFIX}/users
```
```

--------------------------------

### AWS IAM Trust Policy for Estuary

Source: https://docs.estuary.dev/private-byoc/byoc-deployments/aws-byoc

This JSON policy allows Estuary's data-plane-ops user to assume the IAM role. It requires a unique ExternalId for security. The ARN and ExternalId must be provided to Estuary.

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::770785070253:user/data-plane-ops"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "replace-with-a-random-uuid"
        }
      }
    }
  ]
}
```

--------------------------------

### Custom Integration using cURL

Source: https://docs.estuary.dev/reference/openmetrics-api

Access Estuary's OpenMetrics API directly using `curl`. This example demonstrates how to make a GET request to the metrics endpoint, including the necessary `Authorization` header with a bearer token.

```bash
curl --location 'https://agent-api-1084703453822.us-central1.run.app/api/v1/metrics/PREFIX/' \
--header 'Authorization: Bearer REFRESH_TOKEN'
```

--------------------------------

### Klaviyo Source Connector Configuration Sample

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/klaviyo-native

This sample demonstrates the configuration for the Klaviyo source connector, specifying the Docker image, advanced settings like window size, credentials, start date, and data resource bindings.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-klaviyo-native:dev
        config:
            advanced:
              window_size: P10D
            credentials:
              credentials: API Key
              access_token: <secret>
            start_date: "2025-08-14T00:00:00Z"
    bindings:
      - resource:
          name: events
        target: ${PREFIX}/events

```

--------------------------------

### Configure Fast Sync for Time of Day (Example)

Source: https://docs.estuary.dev/reference/materialization-sync-schedule

Sets up a fast sync period between specific hours of the day. Requires Sync Frequency, Timezone, Fast Sync Start Time, and Fast Sync Stop Time.

```text
Sync Frequency : 0s
Timezone : America/New_York
Fast Sync Start Time : 09:00
Fast Sync Stop Time : 17:00
```

--------------------------------

### Front Source Connector Sample Configuration

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/front

A sample catalog specification demonstrating how to configure the Front source connector to capture data from multiple Front resources.

```APIDOC
## Sample Configuration

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-front:dev
        config:
          credentials:
            access_token: <secret>
          start_date: "2024-11-15T00:00:00Z"
    bindings:
      - resource:
          name: channels
        target: ${PREFIX}/channels
      - resource:
          name: contacts
        target: ${PREFIX}/contacts
      - resource:
          name: conversations
        target: ${PREFIX}/conversations
      - resource:
          name: events
        target: ${PREFIX}/events
      - resource:
          name: inboxes
        target: ${PREFIX}/inboxes
      - resource:
          name: tags
        target: ${PREFIX}/tags
      - resource:
          name: teammates
        target: ${PREFIX}/teammates
      - resource:
          name: teams
        target: ${PREFIX}/teams
```
```

--------------------------------

### IAM Policy for S3 Table Bucket Access

Source: https://docs.estuary.dev/reference/Connectors/materialization-connectors/apache-iceberg

An example AWS IAM policy granting necessary permissions to interact with S3 Table Buckets. This policy is crucial for both the IAM user configured for catalog authentication and the EMR execution role.

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3TablesAccess",
            "Effect": "Allow",
            "Action": [
                "s3tables:GetTableBucket",
                "s3tables:ListNamespaces",
                "s3tables:CreateNamespace",
                "s3tables:ListTables",
                "s3tables:CreateTable",
                "s3tables:PutTableData",
                "s3tables:GetTableData",
                "s3tables:UpdateTableMetadataLocation",
                "s3tables:GetTableMetadataLocation",
                "s3tables:DeleteTable"
            ],
            "Resource": [
                "arn:aws:s3tables:<region>:<aws-account-id>:bucket/<bucket>",
                "arn:aws:s3tables:<region>:<aws-account-id>:bucket/<bucket>/*"
            ]
        }
    ]
}
```

--------------------------------

### Set MySQL Timezone to IANA Timezone

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/MySQL

This example demonstrates setting the MySQL server's time zone using an IANA timezone name. This method automatically handles daylight saving time adjustments. It requires that the MySQL server has the necessary time zone tables installed.

```sql
SET GLOBAL time_zone = 'America/New_York';
```

--------------------------------

### Google Ads Source Configuration Example (YAML)

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/google-ads

A sample YAML configuration for the Google Ads source connector, illustrating manual authentication parameters, customer IDs, date ranges, and custom GAQL queries.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-google-ads:dev
        config:
          conversion_window_days: 7
          credentials:
            client_id: {secret_client_ID}
            client_secret: {secret_secret}
            developer_token: {access_token}
            refresh_token: {refresh_token}
          customer_id: 0123456789, 1234567890
          login_customer_id: 0987654321
          end_date: 2022-01-01
          start_date: 2020-01-01
          custom_queries:
            - query:
                SELECT
                  campaign.id,
                  campaign.name,
                  campaign.status
                FROM campaign
              table_name: campaigns_custom
              primary_key: campaign.id,campaign.name
    bindings:
      - resource:
          stream: campaign
          syncMode: incremental
        target: ${PREFIX}/campaign
      {...}

```

--------------------------------

### Monday.com Connector Configuration Sample

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/monday

A sample catalog specification file demonstrating how to configure the Monday.com source connector. It includes endpoint details, authentication using an API token, and defines bindings for various data resources like boards, items, users, teams, and tags.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-monday:dev
        config:
          credentials:
            credentials_title: Default Monday Credentials
            api_token: <secret>
          advanced:
            limit: 10
    bindings:
      - resource:
          name: teams
        target: ${PREFIX}/teams
      - resource:
          name: users
        target: ${PREFIX}/users
      - resource:
          name: tags
        target: ${PREFIX}/tags
      - resource:
          name: boards
        target: ${PREFIX}/boards
      - resource:
          name: items
        target: ${PREFIX}/items

```

--------------------------------

### Docker Compose for PostgreSQL and Data Generator

Source: https://docs.estuary.dev/getting-started/tutorials/postgresql_cdc_to_snowflake

Defines services for a PostgreSQL database and a mock data generator using Docker. It configures PostgreSQL with logical replication enabled and sets up the data generator to populate the products table. Ensures PostgreSQL is healthy before starting the data generator.

```yaml
services:
  postgres:
    image: postgres:latest
    container_name: postgres_cdc
    hostname: postgres_cdc
    restart: unless-stopped
    user: postgres
    environment:
      POSTGRES_USER: postgres
      POSTGRES_DB: postgres
      POSTGRES_PASSWORD: postgres
    command:
      - "postgres"
      - "-c"
      - "wal_level=logical"
    healthcheck:
      test: ["CMD-SHELL", "sh -c 'pg_isready -U flow_capture -d postgres'"]
      interval: 5s
      timeout: 10s
      retries: 120
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    ports:
      - "5432:5432"

  datagen:
    image: materialize/datagen
    container_name: datagen
    restart: unless-stopped
    environment:
      POSTGRES_HOST: postgres_cdc
      POSTGRES_PORT: 5432
      POSTGRES_DB: postgres
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    entrypoint:
      "datagen -s /app/schemas/products.sql -n 10000 -f postgres -w 1000"
    depends_on:
      postgres:
        condition: service_healthy

    volumes:
      - ./schemas/products.sql:/app/schemas/products.sql

```

--------------------------------

### Google Analytics UA Connector Configuration Example

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/google-analytics

Illustrates the configuration properties for the Google Analytics UA source connector, including authentication details (service account key), custom reports, start date, view ID, and window in days. This configuration is used for capturing data from Google Universal Analytics views.

```json
{
  "credentials": {
    "auth_type": "Service",
    "credentials_json": "-----BEGIN PRIVATE KEY-----\nYOUR_PRIVATE_KEY\n-----END PRIVATE KEY-----\n"
  },
  "custom_reports": "[]",
  "start_date": "2023-01-01",
  "view_id": "YOUR_VIEW_ID",
  "window_in_days": 30,
  "stream": "Website overview",
  "syncMode": "incremental"
}
```

--------------------------------

### Install flowctl on Mac with Homebrew

Source: https://docs.estuary.dev/concepts/flowctl

Installs flowctl on macOS using the Homebrew package manager by tapping the estuary/flowctl repository and then installing the flowctl package.

```shell
brew tap estuary/flowctl
brew install flowctl
```

--------------------------------

### PostgreSQL Setup - Create User and Grant Permissions

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/Supabase

Steps to create a dedicated user with necessary permissions for the PostgreSQL connector. This includes creating the user, assigning read privileges (for PostgreSQL v14+ or earlier versions), and setting up a watermarks table.

```APIDOC
## PostgreSQL Setup

### 1. Create User and Password

```sql
CREATE USER flow_capture WITH PASSWORD 'secret' REPLICATION;
```

### 2. Assign Roles and Permissions

#### For PostgreSQL v14 and later:
```sql
GRANT pg_read_all_data TO flow_capture;
```

#### For PostgreSQL versions earlier than v14:
```sql
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES to flow_capture;
GRANT SELECT ON ALL TABLES IN SCHEMA public, <other_schema> TO flow_capture;
GRANT SELECT ON ALL TABLES IN SCHEMA information_schema, pg_catalog TO flow_capture;
```
*Replace `<other_schema>` with your actual schema names.*

### 3. Create Watermarks Table, Grant Privileges, and Create Publication:

```sql
CREATE TABLE IF NOT EXISTS public.flow_watermarks (slot TEXT PRIMARY KEY, watermark TEXT);
GRANT ALL PRIVILEGES ON TABLE public.flow_watermarks TO flow_capture;
CREATE PUBLICATION flow_publication;
ALTER PUBLICATION flow_publication SET (publish_via_partition_root = true);
ALTER PUBLICATION flow_publication ADD TABLE public.flow_watermarks, <other_tables>;
```
*Replace `<other_tables>` with the tables you wish to capture.*

### 4. Set WAL Level to Logical:

```sql
ALTER SYSTEM SET wal_level = logical;
```

### 5. Restart PostgreSQL

*Restart your PostgreSQL instance for the WAL level changes to take effect.*
```

--------------------------------

### Create and Configure PostgreSQL Publication

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/neon-postgres

This set of SQL commands creates a publication for logical replication, sets the `publish_via_partition_root` option, and adds specific tables to the publication. It also includes creating a watermark table and granting privileges.

```sql
CREATE TABLE IF NOT EXISTS public.flow_watermarks (slot TEXT PRIMARY KEY, watermark TEXT);  
GRANT ALL PRIVILEGES ON TABLE public.flow_watermarks TO flow_capture;  
CREATE PUBLICATION flow_publication;  
ALTER PUBLICATION flow_publication SET (publish_via_partition_root = true);  
ALTER PUBLICATION flow_publication ADD TABLE public.flow_watermarks, <other_tables>;  

```

--------------------------------

### BigQuery Connector Query Template Example

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/bigquery-ga4-native

This SQL template is used by the BigQuery connector to select data from a table. It dynamically includes a WHERE clause based on the cursor column if a previous cursor value exists, ensuring incremental data retrieval. The ORDER BY clause ensures data is processed in the correct sequence for cursor tracking.

```sql
SELECT * FROM $table WHERE $cursorName > $lastCursorValue ORDER BY $cursorName
```

--------------------------------

### Add SSO Users to Your Tenant

Source: https://docs.estuary.dev/getting-started/sso-setup

Instructions for adding new users to your Estuary tenant via SSO, utilizing Just-In-Time account creation.

```APIDOC
## Adding SSO Users to Estuary Tenant

### Description
Estuary supports Just-In-Time (JIT) account creation for new users logging in via SSO. This process involves users authenticating through your identity provider and then being added to your Estuary tenant upon their first login. Tenant administrators can also invite users manually.

### New User Flow
1.  A new user enters their email on the Estuary SSO login page.
2.  Estuary redirects the user to the identity provider for authentication.
3.  Upon successful authentication, the user is redirected back to Estuary.
4.  The user must accept Estuary's terms of service.
5.  The user should **not** create a new tenant; they will be associated with an existing one.

### Tenant Admin Flow (Inviting Users)
1.  The tenant admin logs into the Estuary dashboard and navigates to the **Admin** tab.
2.  Under 'Account Access', locate the **Organization Membership** section.
3.  Click the **Add Users** button.
4.  Configure the invitation: specify a prefix (e.g., tenant name), capability (admin or read access), and type (multi- or single-use).
5.  Create an invite link. This link can be unique per user or a shared multi-use link.
6.  Securely share the invite link with the new user.

Upon following the invite link, the new user will be added to the tenant with the assigned permissions. JIT provisioning ensures that user accounts are generated automatically upon their initial SSO login.
```

--------------------------------

### API Method Example

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/aircall

An example of an API method call to retrieve numbers from Aircall.

```APIDOC
## API Method Example

### Description
An example of an API method call for Aircall, demonstrating how to retrieve numbers.

### Method
GET

### Endpoint
https://api.aircall.io/v1/numbers
```

--------------------------------

### Encrypted config.yaml example

Source: https://docs.estuary.dev/concepts/flowctl

An example of a configuration file after being encrypted by sops, showing the ENC wrapper for values and sops metadata.

```yaml
host: ENC[AES256_GCM,data:K/clly65pThTg2U=,iv:1bNmY8wjtjHFBcXLR1KFcsNMGVXRl5LGTdREUZIgcEU=,tag:5GKcguVPihXXDIM7HHuNnA==,type:str]
password: ENC[AES256_GCM,data:IDDY+fl0/gAcsH+6tjRdww+G,iv:Ye8st7zJ9wsMRMs6BoAyWlaJeNc9qeNjkkjo6BPp/tE=,tag:EPS9Unkdg4eAFICGujlTfQ==,type:str]
user: ENC[AES256_GCM,data:w+F7MMwQhw==,iv:amHhNCJWAJnJaGujZgjhzVzUzAeSchEpUpBau7RVeCg=,tag:62HguhnnSDqJdKdwYnj7mQ==,type:str]
sops:
  # Some items omitted for brevity:
  gcp_kms:
    - resource_id: projects/your-project-id/locations/us-central1/keyRings/your-ring/cryptoKeys/your-key-name
      created_at: "2022-01-05T15:49:45Z"
      enc: CiQAW8BC2GDYWrJTp3ikVGkTI2XaZc6F4p/d/PCBlczCz8BZiUISSQCnySJKIptagFkIl01uiBQp056c
lastmodified: "2022-01-05T15:49:45Z"
version: 3.7.1
```

--------------------------------

### Sample Estuary Kafka Connector Configuration

Source: https://docs.estuary.dev/reference/Connectors/materialization-connectors/apache-kafka

A sample YAML configuration for the Estuary Kafka connector, demonstrating how to set up bootstrap servers, authentication, schema registry, and topic bindings.

```yaml
materializations:
  ${PREFIX}/${mat_name}:
    endpoint:
      connector:
          bootstrap_servers: server1:9092,server2:9092
          tls: system_certificates
          credentials:
            auth_type: UserPassword
            mechanism: SCRAM-SHA-512
            username: bruce.wayne
            password: definitely-not-batman
          schema_registry:
            endpoint: https://schema.registry.com
            username: schemaregistry.username
            password: schemaregistry.password
    bindings:
      - resource:
          topic: ${TOPIC_NAME}
        source: ${PREFIX}/${COLLECTION_NAME}

```

--------------------------------

### PostgreSQL Initialization for CDC

Source: https://docs.estuary.dev/getting-started/tutorials/postgresql_cdc_to_snowflake

Configures the PostgreSQL database for CDC streaming. It creates a user with replication privileges, grants read access, defines the products table, sets up a watermark table for Flow, and creates a publication that includes the products and watermark tables.

```sql
CREATE USER flow_capture WITH PASSWORD 'secret' REPLICATION;

GRANT pg_read_all_data TO flow_capture;

CREATE TABLE products (
  "id" int PRIMARY KEY,
  "name" varchar COMMENT 'faker.internet.userName()',
  "merchant_id" int NOT NULL COMMENT 'faker.datatype.number()',
  "price" int COMMENT 'faker.datatype.number()',
  "status" varchar COMMENT 'faker.datatype.boolean()',
  "created_at" timestamp DEFAULT (now())
);

CREATE TABLE IF NOT EXISTS public.flow_watermarks (slot TEXT PRIMARY KEY, watermark TEXT);
GRANT ALL PRIVILEGES ON TABLE public.flow_watermarks TO flow_capture;
CREATE PUBLICATION flow_publication;
ALTER PUBLICATION flow_publication SET (publish_via_partition_root = true);
ALTER PUBLICATION flow_publication ADD TABLE public.flow_watermarks, public.products;

```

--------------------------------

### BigQuery Connector Setup

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/bigquery-ga4-native

Steps to set up the BigQuery connector in Estuary Flow.

```APIDOC
## BigQuery Connector Setup

### Description
This section outlines the process of configuring the BigQuery connector within the Estuary Flow platform.

### Method
N/A (UI-driven setup)

### Endpoint
N/A

### Parameters
#### Request Body
- **Project ID** (string) - Required - The GCP project ID for the project containing the source BigQuery dataset.
- **Dataset** (string) - Required - The BigQuery dataset to discover tables within.
- **Credentials JSON** (string) - Required - The contents of your Service Account Key JSON file.
- **Poll Interval** (string) - Optional (Default: "24h") - How often to poll bindings (may be overridden for a specific binding).

#### Bindings
- **Name** (string) - Required - A name which uniquely identifies this binding.
- **Cursor** (array) - Optional (Default: []) - The column name(s) which should be used as the incremental capture cursor.
- **Template** (string) - Required - The query (template) which will be executed every polling interval.
- **Poll Interval** (string) - Optional - Override the global polling interval for this binding.

### Request Example
N/A

### Response
#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### Example Aircall API Method Call

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/aircall

Demonstrates an example API method call to retrieve numbers from the Aircall API.

```http
GET https://api.aircall.io/v1/numbers
```

--------------------------------

### Asana Connector Configuration Sample

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/asana

Provides a sample configuration for the Asana source connector in Estuary Flow. This example shows how to set up credentials using a Personal Access Token and define data bindings for capturing 'attachments' with a full refresh sync mode.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-asana:dev
        config:
          credentials:
            option_title: PAT Credentials
            personal_access_token: <secret>
    bindings:
      - resource:
          stream: attachments
          syncMode: full_refresh
        target: ${PREFIX}/attachments
      {...}
```

--------------------------------

### Google Play Connector Sample Configuration

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/google-play

This configuration snippet demonstrates how to set up the Google Play source connector, including specifying the Docker image, bucket, service account credentials, start date, and data resource bindings for crashes with a specific interval.

```yaml
captures:  
  ${PREFIX}/${CAPTURE_NAME}:  
    endpoint:  
      connector:  
        image: ghcr.io/estuary/source-google-play:dev  
          config:  
            bucket: pubsite_prod_0123456_my_bucket  
            credentials:  
                credentials_title: Google Service Account  
                service_account: <secret>  
            start_date: "2025-08-05T00:00:00Z"  
      bindings:  
        - resource:  
            name: crashes  
            interval: PT1H  
          target: ${PREFIX}/crashes  

```

--------------------------------

### YouTube Analytics Connector Sample Configuration

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/youtube-analytics

Provides a sample configuration for setting up the YouTube Analytics connector, including image, credentials, and bindings.

```APIDOC
## Sample Configuration

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-youtube-analytics:dev
        config:
          credentials:
            auth_type: OAuth
    bindings:
      - resource:
          stream: channel_annotations_a1
          syncMode: incremental
        target: ${PREFIX}/channel_annotations_a1
      # ... other bindings
```
```

--------------------------------

### SQL Lambda Example

Source: https://docs.estuary.dev/concepts/derivations

An example of a lambda function defined as a SQL query within a derivation's transformation. This SQL would be executed by the SQLite connector.

```sql
SELECT $foo, $bar;
```

--------------------------------

### Mixpanel Connector Configuration Sample

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/mixpanel

A sample configuration for the Estuary Mixpanel source connector. This snippet demonstrates how to set up the connector with authentication, project details, date ranges, and advanced settings, as well as defining bindings for specific Mixpanel resources like annotations.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-mixpanel:dev
        config:
          credentials:
            auth_type: ServiceAccount
            secret: {secret}
            username: {your_username}
          project_id: 1234567
          attribution_window: 5
          project_timezone: US/Pacific
          start_date: 2017-01-25T00:00:00Z
          end_date: 2019-01-25T00:00:00Z
          region: US
          date_window_size: 30
          advanced:
            page_size: 10000
            minimal_cohort_members_properties: true
    bindings:
      - resource:
          stream: annotations
          syncMode: full_refresh
        target: ${PREFIX}/annotations
      {...}
```

--------------------------------

### Criteo Connector Configuration Sample (YAML)

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/criteo

A sample YAML configuration for the Criteo connector, demonstrating how to set up the Docker image, authentication credentials, advertiser IDs, start date, and data stream bindings.

```yaml
captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-criteo:dev
        config:
          client_id: {secret}
          client_secret: {secret}
          advertiser_ids:
            - "12345"
            - "67890"
          start_date: 2025-01-01T00:00:00.000Z
    bindings:
      - resource:
          stream: advertisers
          syncMode: full_refresh
        target: ${PREFIX}/advertisers
      {...}
```

--------------------------------

### NetSuite Integration Naming Convention Example

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/netsuite-suiteanalytics

Provides an example of how to name a NetSuite integration for use with Estuary Flow. Proper naming aids in organization and identification.

```text
estuary-rest-integration
```

--------------------------------

### NetSuite Role Naming Convention Example

Source: https://docs.estuary.dev/reference/Connectors/capture-connectors/netsuite-suiteanalytics

Shows an example of naming a custom role in NetSuite, specifically for integration with Estuary Flow. This helps in managing permissions and access.

```text
estuary-integration-role
```