### Setup Allium Integration Command

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/allium.md

Sample command to ingest data from Allium into a DuckDB database using API key and query ID. Query parameters like interval start and end are passed separately.

```sh
ingestr ingest \
  --source-uri 'allium://?api_key=your_api_key' \
  --source-table 'query:your_query_id' \
  --interval-start '2025-02-01' \
  --interval-end '2025-02-02' \
  --dest-uri duckdb:///allium.duckdb \
  --dest-table 'allium.query_results'
```

--------------------------------

### Ingest Granola Notes with Interval Start

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/granola.md

Example command for incremental ingestion of Granola notes, specifying a start interval. This ensures only updated notes are fetched.

```sh
ingestr ingest \
  --source-uri 'granola://?api_key=your-api-key' \
  --source-table 'notes' \
  --dest-uri duckdb:///granola.duckdb \
  --dest-table 'main.notes' \
  --interval-start '2026-01-01T00:00:00Z'
```

--------------------------------

### Start Local CrateDB Instance

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/cratedb.md

Starts a single-node CrateDB instance using Docker for evaluation purposes.

```shell
docker run --rm -it --name=cratedb \
  --publish=4200:4200 --publish=5432:5432 \
  --env=CRATE_HEAP_SIZE=2g crate/crate:nightly \
  -Cdiscovery.type=single-node
```

--------------------------------

### Ingest Trustpilot Reviews into DuckDB

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/trustpilot.md

Example command to ingest Trustpilot reviews into a DuckDB database. Replace the example IDs and keys with your own credentials.

```bash
ingestr ingest --source-uri 'trustpilot://123?api_key=key_abc' --source-table 'reviews' --dest-uri duckdb:///trustpilot.duckdb --dest-table 'dest.reviews'
```

--------------------------------

### Install Ingestr with Curl

Source: https://github.com/bruin-data/ingestr/blob/main/docs/index.md

Use this command to download and execute the install script for ingestr.

```sh
curl -LsSf https://getbruin.com/install/ingestr | sh
```

--------------------------------

### Postgres Connection URI Example

Source: https://github.com/bruin-data/ingestr/blob/main/docs/getting-started/core-concepts.md

This is an example of a connection URI for a Postgres database. It includes credentials, host, port, database name, and query parameters.

```plaintext
postgresql://admin:admin@localhost:8837/web?sslmode=disable
```

--------------------------------

### Install DuckDB using pip

Source: https://github.com/bruin-data/ingestr/blob/main/docs/tutorials/load-personio-duckdb.md

Installs the DuckDB database system using pip. Ensure pip is available in your environment.

```bash
pip install duckdb
```

--------------------------------

### Azure SQL Server token authentication example

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/mssql.md

This script demonstrates how to ingest data from Azure SQL Server using an access token generated by the Azure CLI for authentication. Ensure you have the Azure CLI installed and are logged in.

```sh
USER=$(az account show --query user.name -o tsv)
TOKEN=$(az account get-access-token --resource https://database.windows.net/ --query accessToken -o tsv)
ingestr ingest \
    --source-uri "mssql://$USER:$TOKEN@<server>.database.windows.net/<database>?Authentication=ActiveDirectoryAccessToken" \
    --source-table "dbo.example" \
    --dest-uri "duckdb:///example.db" \
    --dest-table "dbo.example" \

```

--------------------------------

### CrateDB Localhost Source Connection

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/cratedb.md

Example URI for connecting to a local CrateDB instance as a source.

```plaintext
cratedb://crate@localhost:5432?sslmode=disable
```

--------------------------------

### Ingestr Example: Trino as Source

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/trino.md

Demonstrates how to use ingestr to ingest data from a Trino source into a DuckDB destination.

```bash
ingestr ingest \
    --source-uri 'trino://admin@localhost:8080/iceberg' \
    --source-table 'default.source_table' \
    --dest-uri 'duckdb:///output.db' \
    --dest-table 'main.destination_table'
```

--------------------------------

### Ingestr Example: Trino with Authentication

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/trino.md

Illustrates ingestr usage with a Trino destination requiring username and password authentication.

```bash
ingestr ingest \
    --source-uri 'mysql://user:pass@localhost:3306/sourcedb' \
    --source-table 'orders' \
    --dest-uri 'trino://user:password@trino-server:8443/iceberg' \
    --dest-table 'sales.orders'
```

--------------------------------

### Manifold URI Parameters Examples

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/manifold.md

Examples of how to use URI parameters to filter data from Manifold.

```plaintext
manifold://?userId=<user-id>
```

```plaintext
manifold://?username=<username>
```

```plaintext
manifold://?market_id=<market-id>
```

```plaintext
manifold://?contract_slug=<market-slug>
```

```plaintext
manifold://?groupId=<group-id>
```

```plaintext
manifold://?term=bitcoin
```

--------------------------------

### CrateDB Cloud Source Connection

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/cratedb.md

Example URI for connecting to CrateDB Cloud as a source.

```plaintext
cratedb://admin:<PASSWORD>@<CLUSTERNAME>.eks1.eu-west-1.aws.cratedb.net:5432?sslmode=require
```

--------------------------------

### Start the ingestr server

Source: https://github.com/bruin-data/ingestr/blob/main/docs/commands/server.md

Starts the ingestr web UI. By default, it listens on port 8080.

```bash
ingestr server
```

--------------------------------

### GDPR Compliance Example

Source: https://github.com/bruin-data/ingestr/blob/main/docs/getting-started/data-masking.md

Demonstrates a full ingestr ingest command for GDPR compliance, applying various masks to sensitive fields.

```bash
ingestr ingest \
  --source-uri "postgres://prod_user:pass@prod.db/customers" \
  --source-table "customer_data" \
  --dest-uri "postgres://dev_user:pass@dev.db/customers" \
  --dest-table "customer_data" \
  --mask "email:hash" \
  --mask "phone:phone" \
  --mask "name:partial:1" \
  --mask "address:redact" \
  --mask "ip_address:hash" \
  --mask "birth_date:year_only"
```

--------------------------------

### Configure ingestr server with flags

Source: https://github.com/bruin-data/ingestr/blob/main/docs/commands/server.md

Starts the ingestr server with custom configurations for port, credentials file, logs directory, and database path.

```bash
ingestr server \
  --port 3000 \
  --creds-file ~/.ingestr/creds.json \
  --logs-dir ~/.ingestr/logs \
  --db ~/.ingestr/ingestr.db
```

--------------------------------

### Ingestr Example: Trino as Destination

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/trino.md

Shows how to use ingestr to load data from a PostgreSQL source into a Trino destination.

```bash
ingestr ingest \
    --source-uri 'postgresql://user:pass@localhost:5432/sourcedb' \
    --source-table 'public.customers' \
    --dest-uri 'trino://admin@localhost:8080/hive' \
    --dest-table 'default.customers'
```

--------------------------------

### Ingest from Couchbase Capella (Cloud)

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/couchbase.md

Example ingestr command for connecting to Couchbase Capella, requiring SSL and a specific host prefix.

```bash
ingestr ingest \
  --source-uri "couchbase://username:password@cb.xxx.cloud.couchbase.com?ssl=true" \
  --source-table "travel-sample.inventory.airport" \
  --dest-uri "duckdb:///airports.db" \
  --dest-table "main.airports"
```

--------------------------------

### Ingest Manifold Market Probabilities for Multiple Markets

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/manifold.md

Example of ingesting 'market_probabilities' for multiple markets by using repeated 'ids' URI parameters.

```bash
ingestr ingest \
  --source-uri 'manifold://?ids=<market-id-1>&ids=<market-id-2>' \
  --source-table market_probabilities \
  --dest-uri 'duckdb:///manifold.duckdb' \
  --dest-table manifold.market_probabilities
```

--------------------------------

### Ingest Monday.com Boards to DuckDB

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/monday.md

Example command to ingest all boards from Monday.com into a DuckDB database. Ensure you replace 'fake_token' with your actual API token.

```sh
ingestr ingest \
  --source-uri "monday://?api_token=fake_token" \
  --source-table "boards" \
  --dest-uri "duckdb://./monday.db" \
  --dest-table "public.boards"
```

--------------------------------

### Load Data from Cassandra using ingestr

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/cassandra.md

Example command to ingest data from a Cassandra table into another destination.

```sh
ingestr ingest \
  --source-uri "cassandra://localhost:9042/analytics?disable_initial_host_lookup=true" \
  --source-table "events" \
  --dest-uri "duckdb:///analytics.duckdb" \
  --dest-table "events"
```

--------------------------------

### E-commerce Data Sharing Example

Source: https://github.com/bruin-data/ingestr/blob/main/docs/getting-started/data-masking.md

Illustrates an ingestr command for sharing e-commerce data with a partner, applying masks to customer and order details.

```bash
ingestr ingest \
  --source-uri "postgres://internal.db/ecommerce" \
  --source-table "orders" \
  --dest-uri "s3://partner-bucket/data.parquet" \
  --dest-table "shared_orders" \
  --mask "customer_email:email" \
  --mask "shipping_address:first_letter" \
  --mask "order_value:round:10" \
  --mask "customer_name:partial:2"
```

--------------------------------

### Configure ingestr server with environment variables

Source: https://github.com/bruin-data/ingestr/blob/main/docs/commands/server.md

Sets environment variables to configure the ingestr server and then starts it. This method allows for persistent configuration across sessions.

```bash
export INGESTR_PORT=3000
export INGESTR_CREDS_FILE=~/.ingestr/creds.json
export INGESTR_LOGS_DIR=~/.ingestr/logs
export INGESTR_DB=~/.ingestr/ingestr.db

ingestr server
```

--------------------------------

### Load Data into Cassandra using ingestr

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/cassandra.md

Example command to load data into a Cassandra table from a source, specifying a primary key for table creation.

```sh
ingestr ingest \
  --source-uri "postgres://user:pass@localhost:5432/app" \
  --source-table "public.events" \
  --dest-uri "cassandra://localhost:9042/analytics?disable_initial_host_lookup=true" \
  --dest-table "events" \
  --incremental-strategy "merge" \
  --primary-key "id"
```

--------------------------------

### Incremental Loading with Date Ranges

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/primer.md

Command to perform incremental loading from Primer by specifying start and end dates. This filters payments within the given interval.

```sh
ingestr ingest \
  --source-uri "primer://?api_key=your_api_key_here" \
  --source-table "payments" \
  --dest-uri "duckdb:///primer.duckdb" \
  --dest-table "dest.payments" \
  --interval-start "2024-01-01" \
  --interval-end "2024-01-31"
```

--------------------------------

### Ingesting Data to DuckLake Source

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/duckdb.md

Example of configuring ingestr to read from a DuckLake source. Requires specifying catalog and storage types and paths.

```bash
ingestr ingest \
  --source-uri="ducklake://?catalog_type=...&storage_type=..." \
  --source-table="schema.table" \
  --dest-uri="postgres://..." \
  --dest-table="schema.table"
```

--------------------------------

### License Initialization

Source: https://github.com/bruin-data/ingestr/blob/main/THIRD_PARTY_LICENSES.txt

A simple Go function to create and return a new License object.

```Go
// NewLicense returns a License.
func NewLicense() *License {
	r := &License{}

	return r
}
```

--------------------------------

### Install Ingestr with Pip

Source: https://github.com/bruin-data/ingestr/blob/main/docs/index.md

Install the ingestr package using pip for Python environments.

```sh
pip install ingestr
```

--------------------------------

### Ingest Data for Multiple Apps

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/appstore.md

Extend the previous example to ingest data for multiple apps by adding additional `app_id` query parameters to the source URI.

```sh
ingestr ingest \
    --source-uri "appstore://app_id=12345&app_id=67890&key_path=api.key&key_id=key_0&issuer_id=issue_0" \
    --source-table "app-downloads-detailed" \
    --dest-uri "duckdb:///analytics.db" \
    --dest-table "public.app_downloads"
```

--------------------------------

### Build and Run ingestr with Arguments

Source: https://github.com/bruin-data/ingestr/blob/main/CLAUDE.md

Builds the application and then runs it with specified arguments for data ingestion. This is a convenient way to test a full ingestion pipeline.

```bash
make run ARGS="ingest --source-uri=postgres://... --dest-uri=sqlite://... --source-table=users"
```

--------------------------------

### Ingest All Notion Databases

Source: https://github.com/bruin-data/ingestr/blob/main/docs/getting-started/migration-to-v1.md

Use '*' as the source table to fetch all accessible Notion databases. Ensure you have the correct source URI.

```bash
ingestr ingest --use-gong --source-uri=notion://... --source-table='*' --dest-uri=...
```

--------------------------------

### Install ingestr with ODBC support

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/mssql.md

Install the ingestr package with the necessary add-on for ODBC support, which is required for Microsoft SQL Server.

```bash
pip install ingestr[odbc]
```

--------------------------------

### Vue.js App Setup and State Management

Source: https://github.com/bruin-data/ingestr/blob/main/pkg/webui/dist/index.html

Initializes the Vue.js application and sets up reactive state variables for managing UI elements, connections, runs, and job status.

```javascript
const { createApp, ref, computed, watch, nextTick, onMounted } = Vue;
createApp({
  setup() {
    const activeTab = ref('newrun');
    const connectors = ref([]);
    const credentials = ref([]);
    const runs = ref([]);
    // New connection form
    const newConnConnector = ref('postgres');
    const newConnName = ref('');
    const newConnFields = ref({});
    // Run form
    const selectedSourceCred = ref('');
    const selectedDestCred = ref('');
    const sourceFields = ref({});
    const destFields = ref({});
    const sourceTable = ref('');
    const destTable = ref('');
    const strategy = ref('replace');
    const primaryKeys = ref('');
    const incrementalKey = ref('');
    // Job state
    const isRunning = ref(false);
    const jobId = ref(null);
    const jobStatus = ref(null);
    const logs = ref([]);
    const logContainer = ref(null);
    // History state
    const expandedRunId = ref(null);
    const selectedRunLogs = ref([]);
    const totalRuns = ref(0);
    const runsOffset = ref(0);
    const runsPerPage = ref(20);

    const currentNewConnConnector = computed(() => connectors.value.find(c => c.id === newConnConnector.value));
    const sourceConnectors = computed(() => connectors.value.filter(c => c.isSource));
    const destConnectors = computed(() => connectors.value.filter(c => c.isDestination));
    const sourceCredentials = computed(() => {
      const ids = new Set(sourceConnectors.value.map(c => c.id));
      return credentials.value.filter(c => ids.has(c.connectorId));
    });
    const destCredentials = computed(() => {
      const ids = new Set(destConnectors.value.map(c => c.id));
      return credentials.value.filter(c => ids.has(c.connectorId));
    });
    const canRun = computed(() => sourceTable.value.trim() !== '' && selectedSourceCred.value && selectedDestCred.value);
    const statusClass = computed(() => ({
      'bg-amber-500/20 text-amber-400': jobStatus.value === 'running',
      'bg-green-500/20 text-green-400': jobStatus.value === 'completed',
      'bg-red-500/20 text-red-400': jobStatus.value === 'failed'
    }));

    const getConnectorName = (connectorId) => {
      const conn = connectors.value.find(c => c.id === connectorId);
      return conn ? conn.name : connectorId;
    };

    async function loadData() {
      const [connRes, credRes] = await Promise.all([
        fetch('/api/connectors').then(r => r.json()),
        fetch('/api/credentials').then(r => r.json())
      ]);
      connectors.value = connRes;
      credentials.value = credRes || [];
      resetNewConnFields();
    }

    async function loadRuns() {
      const res = await fetch(`/api/runs?limit=${runsPerPage.value}&offset=${runsOffset.value}`).then(r => r.json());
      runs.value = res.runs || [];
      totalRuns.value = res.total || 0;
    }

    function prevPage() {
      if (runsOffset.value > 0) {
        runsOffset.value = Math.max(0, runsOffset.value - runsPerPage.value);
        loadRuns();
      }
    }

    function nextPage() {
      if (runsOffset.value + runsPerPage.value < totalRuns.value) {
        runsOffset.value += runsPerPage.value;
        loadRuns();
      }
    }

    function resetNewConnFields() {
      newConnFields.value = {};
      const conn = currentNewConnConnector.value;
      if (conn) {
        conn.fields.forEach(f => {
          newConnFields.value[f.name] = f.default || '';
        });
      }
    }

    async function saveNewConnection() {
      if (!newConnName.value.trim()) {
        alert('Please enter a connection name');
        return;
      }
      const cred = {
        name: newConnName.value,
        connectorId: newConnConnector.value,
        fields: { ...newConnFields.value }
      };
      await fetch('/api/credentials', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json'
        },
        body: JSON.stringify(cred)
      });
      newConnName.value = '';
      resetNewConnFields();
      await loadData();
      alert('Connection saved!');
    }

    async function deleteConnection(id) {
      if (!confirm('Delete this connection?')) return;
      await fet
```

--------------------------------

### Install ingestr with IBM Db2 Add-on

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/db2.md

Install the ingestr package along with the necessary ibm-db add-on to enable IBM Db2 support. This command should be run in your terminal.

```bash
pip install ingestr[ibm-db]
```

--------------------------------

### Ingest Manifold Search Markets

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/manifold.md

Example of ingesting 'search_markets' data from Manifold, filtered by term and sorted by newest.

```bash
ingestr ingest \
  --source-uri 'manifold://?term=bitcoin&sort=newest' \
  --source-table search_markets \
  --dest-uri 'duckdb:///manifold.duckdb' \
  --dest-table manifold.search_markets
```

--------------------------------

### Ingest with Azure SQL (Default Environment Credentials)

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/azuresql.md

Example of ingesting data from Azure SQL using default environment credentials (Azure CLI, managed identity, etc.). Uses the ActiveDirectoryDefault fedauth workflow.

```bash
ingestr ingest \
    --source-uri "azuresql://myserver.database.windows.net/mydb?fedauth=ActiveDirectoryDefault" \
    --source-table "dbo.users" \
    --dest-uri "duckdb:///local.db" \
    --dest-table "main.users"
```

--------------------------------

### Format and Lint Code

Source: https://github.com/bruin-data/ingestr/blob/main/CLAUDE.md

Applies code formatting and linting to the project. 'make format' runs gci, gofumpt, go vet, and golangci-lint, while 'make lint' runs linters only.

```bash
make format
```

```bash
make lint
```

--------------------------------

### Trino Basic Authentication URI

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/trino.md

Example URI for Trino connections using basic username and password authentication.

```plaintext
trino://user:password@localhost:8080/catalog
```

--------------------------------

### Ingest with Azure SQL (SQL Authentication)

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/azuresql.md

Example of ingesting data from Azure SQL using standard SQL authentication. Ensure SQL_USER and SQL_PASSWORD environment variables are set.

```bash
ingestr ingest \
    --source-uri "azuresql://$SQL_USER:$SQL_PASSWORD@myserver.database.windows.net/mydb" \
    --source-table "dbo.users" \
    --dest-uri "duckdb:///local.db" \
    --dest-table "main.users"
```

--------------------------------

### Trino No Authentication URI

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/trino.md

Example URI for Trino connections without authentication, suitable for development or testing environments.

```plaintext
trino://user@localhost:8080/catalog
```

--------------------------------

### Basic HTTP Ingestion Example

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/http.md

Use this command to ingest a CSV file from a public HTTP URL into a DuckDB database. Ensure the source file is publicly accessible.

```bash
ingestr ingest \
    --source-uri "https://example.com/data.csv" \
    --source-table "data" \
    --dest-uri "duckdb:///local.db" \
    --dest-table "my_table"
```

--------------------------------

### Ingest with URL-Encoded Password

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/couchbase.md

Example showing how to URL-encode passwords with special characters for use in the ingestr CLI connection URI.

```bash
ingestr ingest \
  --source-uri "couchbase://admin:MyPass%40123%21@localhost" \
  --source-table "mybucket.myscope.mycollection" \
  --dest-uri "duckdb:///output.db" \
  --dest-table "main.couchbase_data"
```

--------------------------------

### Ingest Data from Customer.io to DuckDB

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/customerio.md

Sample command to ingest data from Customer.io broadcasts into a DuckDB database. Replace 'your_api_key' with your actual API key.

```sh
ingestr ingest \
  --source-uri 'customerio://?api_key=your_api_key&region=us' \
  --source-table 'broadcasts' \
  --dest-uri duckdb:///customerio.duckdb \
  --dest-table 'customerio.broadcasts'
```

--------------------------------

### Run Unit Tests

Source: https://github.com/bruin-data/ingestr/blob/main/CLAUDE.md

Executes all unit tests with race detection enabled. For unit tests only, use 'go test -short ./...'.

```bash
make test
```

```bash
go test -short ./...
```

--------------------------------

### Extracting Base ID and Table ID from URL

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/airtable.md

Example URL demonstrating how to identify the Base ID and Table ID for Airtable.

```plaintext
https://airtable.com/appve10kl227BIT4GV/tblOUnZVLFWbemTP1/viw3qtF76bRQC3wKx/rec9khXgeTotgCQ62?blocks=hide
```

--------------------------------

### GAQL Query: Get All Campaigns with Status

Source: https://github.com/bruin-data/ingestr/blob/main/docs/supported-sources/google-ads.md

This GAQL query retrieves the ID, name, status, and advertising channel type for all campaigns.

```sql
--source-table "gaql_query:SELECT campaign.id, campaign.name, campaign.status, campaign.advertising_channel_type FROM campaign"
```

--------------------------------

### Tokenize with Sequential Integers

Source: https://github.com/bruin-data/ingestr/blob/main/docs/getting-started/data-masking.md

Use 'sequential' to replace values with integers starting from 1. Good for simple anonymization and reducing data size.

```bash
--mask "account_number:sequential"
```

--------------------------------

### Healthcare Data Masking Example

Source: https://github.com/bruin-data/ingestr/blob/main/docs/getting-started/data-masking.md

Shows an ingestr ingest command for healthcare data, anonymizing patient information and shifting admission dates.

```bash
ingestr ingest \
  --source-uri "mysql://user:pass@hospital.db/patients" \
  --source-table "patient_records" \
  --dest-uri "bigquery://project/dataset" \
  --dest-table "patient_analytics" \
  --mask "patient_id:uuid" \
  --mask "ssn:redact" \
  --mask "diagnosis_notes:redact" \
  --mask "admission_date:date_shift:7" \
  --mask "age:round:5"
```