Meltano

https://docs.meltano.com
Admin
Meltano is an open-source platform for building and running data pipelines, offering a hub of...
Tokens:314,778
Snippets:2,556
Trust Score:9.6
Update:2 months ago
Show doc for...
Tokens:
Raw
### Installation output

Source: https://docs.meltano.com/tutorials/custom-extractor

Example output showing the installation process of a custom extractor.

```text
Added extractor 'tap-jsonplaceholder' to your Meltano project  
  
2024-01-01T00:25:40.604941Z [info     ] Installing extractor 'tap-jsonplaceholder'...  
2024-01-01T00:25:53.152127Z [info     ] Installed extractor 'tap-jsonplaceholder'  
  
```

--------------------------------

### Comprehensive install usage examples

Source: https://docs.meltano.com/reference/command-line-interface

Various ways to invoke the install command including schedules, parallelism, and force flags.

```bash
# Install all plugins  
meltano install  
  
# Install specific plugins (recommended - automatically detects type)  
meltano install tap-github target-postgres  
meltano install tap-gitlab  
  
# Install plugins by type  
meltano install --plugin-type=extractor tap-gitlab  
meltano install --plugin-type=extractors  # Install all extractors  
meltano install --plugin-type=extractor tap-gitlab tap-adwords  
  
# Install plugins for a specific schedule  
meltano install --schedule=<schedule_name>  
  
# Install with additional options  
meltano install --parallelism=16  
meltano install --clean  
  
meltano install --force
```

--------------------------------

### Install Meltano Project

Source: https://docs.meltano.com/contribute/tests

Installs the necessary Meltano project dependencies. This is a prerequisite for following the example.

```shell
meltano install
```

--------------------------------

### Install all plugins

Source: https://docs.meltano.com/reference/command-line-interface

Installs all dependencies defined in the meltano.yml file.

```bash
meltano install
```

--------------------------------

### Install Meltano with uv

Source: https://docs.meltano.com/guide/production

Use `uv` to install the Meltano package from PyPI. This is recommended for production environments. Consider adding `meltano` to your `requirements.txt` to automate installation.

```bash
uv pip install meltano
```

```bash
uv pip install -r requirements.txt
```

--------------------------------

### Install custom state backend

Source: https://docs.meltano.com/guide/custom-state-backend

Install the custom backend package alongside Meltano using uv.

```bash
uv tool install --with git+https://github.com/your-username/my-state-manager.git meltano  
```

--------------------------------

### Install uv Package Manager

Source: https://docs.meltano.com/guide/complete_tutorial

Install the uv package manager, a prerequisite for installing Meltano locally. Refer to the provided URL for Windows-specific instructions.

```bash
# See https://docs.astral.sh/uv/getting-started/installation/ for Windows instructions
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Create or Update Datacomponent Examples

Source: https://docs.meltano.com/reference/cloud/api/resources/datacomponents

Examples showing how to perform a PUT request to create or update a datacomponent.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/workspaces/ca7e1251-b0ed-4b63-85d3-519cd5211e3e/datacomponents/b8364152-7bfa-49f4-9d07-39868bb53330' -i -X PUT \
    -H 'Content-Type: application/json' \
    -d '{
  "name" : "extractors/tap-matatika-sit",
  "dataPlugin" : "extractors/tap-matatika-sit"
}'
```

```python
import requests

url = "https://app.meltano.com:443/api/workspaces/ca7e1251-b0ed-4b63-85d3-519cd5211e3e/datacomponents/b8364152-7bfa-49f4-9d07-39868bb53330"

data = {
  "name" : "extractors/tap-matatika-sit",
  "dataPlugin" : "extractors/tap-matatika-sit"
}
headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("PUT", url, headers=headers, data=data)

print(response.text.encode('utf8'))
```

--------------------------------

### Complete Orchestration Script Example

Source: https://docs.meltano.com/contribute/tests

This is a consolidated example of the commands used in the migration process, generated by the integration framework.

```shell
meltano install
meltano el --state-id=my-custom-id tap-gitlab target-postgres
meltano state copy my-custom-id tap-gitlab-to-target-postgres
meltano job add my-new-job --task="tap-gitlab target-postgres"
meltano run my-new-job
```

--------------------------------

### Install specific plugins or types

Source: https://docs.meltano.com/reference/command-line-interface

Install plugins by name or by specifying the plugin type.

```bash
# Install specific plugins by name (automatically detects type)  
meltano install tap-github target-postgres  
  
# Install plugins by type  
meltano install --plugin-type=extractor tap-gitlab
```

```bash
# Using dash syntax  
meltano install - tap-github target-postgres  
  
# Using positional type argument  
meltano install extractor tap-gitlab  
meltano install <plugin_type> <plugin_name>
```

--------------------------------

### Schedule Management Examples

Source: https://docs.meltano.com/reference/command-line-interface

Practical examples for adding, running, and updating schedules including dry-runs and legacy EL tasks.

```bash
# Add a new schedule named "gitlab-sync" to run the job named "gitlab-to-mysql" every day  
meltano schedule add gitlab-sync --job gitlab-to-mysql --interval "@daily"  
  
# Perform a dry-run of the schedule named "gitlab-sync"  
# Behind the scenes, this will execute a `meltano run --dry-run gitlab-sync`  
meltano schedule run gitlab-sync --dry-run  
  
# Update the schedule named "gitlab-sync" to run the job named "gitlab-to-postgres" instead of "gitlab-to-mysql"  
meltano schedule set gitlab-sync --job gitlab-to-postgres  
# Update the schedule named "gitlab-sync" to run weekly instead of daily  
meltano schedule set gitlab-sync --interval "@weekly"  
  
# Add a legacy EL based schedule named "gitlab-to-jsonl" to run every minute  
# This specifies that the following command is to be run every minute:  
# meltano el tap-gitlab target-jsonl --state-id=gitlab-to-jsonl  
meltano schedule add gitlab-to-jsonl --extractor tap-gitlab --loader target-jsonl --interval="* * * * *"  
# Update the schedule named "gitlab-to-jsonl" to use target-csv instead of target-jsonl  
meltano schedule set gitlab-to-jsonl --loader target-csv
```

--------------------------------

### Install Meltano project dependencies

Source: https://docs.meltano.com/tutorials/custom-extractor

Run this command from the project root to install configured plugins.

```bash
meltano install  
```

--------------------------------

### Install and run pipeline plugins

Source: https://docs.meltano.com/meltano-cloud/importing-data

Install the required extractor and loader plugins, then execute the pipeline using the Meltano CLI.

```bash
meltano install extractor tap-example
```

```bash
meltano install loader target-example
```

```bash
meltano run tap-example target-example
```

--------------------------------

### Install Meltano with Postgres extra

Source: https://docs.meltano.com/guide/v3-migration

Install Meltano with the `psycopg2` extra if you are already using Postgres as a backend.

```bash
pipx install "meltano[psycopg2]"
```

--------------------------------

### Check Meltano Installation

Source: https://docs.meltano.com/guide/installation-guide

Verify that Meltano has been successfully installed by checking its version.

```bash
meltano --version
```

--------------------------------

### Create or Update Datastore Examples

Source: https://docs.meltano.com/reference/cloud/api/resources/datastores

Examples of how to perform a PUT request to create or update a datastore using cURL and Python.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/workspaces/9a3c2e67-0817-466d-8704-05ab4ae7a91d/datastores/51c91fe8-78ca-4d9e-a004-1e751bb6b4f8' -i -X PUT \
    -H 'Content-Type: application/json' \
    -d '{
  "name" : "Test datastore",
  "dataPlugin" : "loaders/target-postgres",
  "properties" : {
    "password" : "16RkpH4y_feU7ou4S_UwPCk0l7",
    "host" : "db.meltano.internal",
    "user" : "kqhtunu",
    "default_target_schema" : "analytics",
    "dbname" : "kqhtunu"
  }
}'
```

```python
import requests

url = "https://app.meltano.com:443/api/workspaces/9a3c2e67-0817-466d-8704-05ab4ae7a91d/datastores/51c91fe8-78ca-4d9e-a004-1e751bb6b4f8"

data = {
  "name" : "Test datastore",
  "dataPlugin" : "loaders/target-postgres",
  "properties" : {
    "password" : "16RkpH4y_feU7ou4S_UwPCk0l7",
    "host" : "db.meltano.internal",
    "user" : "kqhtunu",
    "default_target_schema" : "analytics",
    "dbname" : "kqhtunu"
  }
}
headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("PUT", url, headers=headers, data=data)

print(response.text.encode('utf8'))
```

--------------------------------

### Install Meltano using uv

Source: https://docs.meltano.com/guide/installation-guide

Install Meltano using the uv package manager, which places it in an isolated virtual environment.

```bash
uv tool install meltano
```

--------------------------------

### Verify Tap Installation

Source: https://docs.meltano.com/tutorials/custom-extractor

Use uvx to verify that the tap is correctly installed and accessible from either a GitHub repository or PyPI.

```bash
uvx --from git+https://github.com/myusername/tap-my-custom-source.git tap-my-custom-source --help  
  
# Or from PyPI, if available  
uvx tap-my-custom-source --help
```

--------------------------------

### Create or update profile examples

Source: https://docs.meltano.com/reference/cloud/api/resources/profiles

Examples of how to perform a PUT request to create or update a profile using cURL and Python.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/profiles/auth0%7C6a75ddb619d58d80410361c9' -i -X PUT \
    -H 'Content-Type: application/json' \
    -d '{
  "name" : "SIT-Generated Profile [2026-08-07T13:29:27.274367971]",
  "phone" : "20462242290",
  "email" : "sit+8em95w9c@meltano.com"
}'
```

```python
import requests

url = "https://app.meltano.com:443/api/profiles/auth0%7C6a75ddb619d58d80410361c9"

data = {
  "name" : "SIT-Generated Profile [2026-08-07T13:29:27.274367971]",
  "phone" : "20462242290",
  "email" : "sit+8em95w9c@meltano.com"
}
headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("PUT", url, headers=headers, data=data)

print(response.text.encode('utf8'))
```

--------------------------------

### Pipeline Configuration Example

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Example of a pipeline configuration object defining data components and specific properties for an extractor.

```json
{
  "name" : "github",
  "dataComponents" : [ "tap-github", "Warehouse" ],
  "properties" : {
    "tap-github.repositories" : "[\"Matatika/matatika-ce\"]",
    "tap-github.auth_token" : "ghp_***",
    "tap-github._select" : "[\"commits.sha\"]",
    "tap-github._metadata" : "{\"commits\":{\"replication-method\":\"FULL_TABLE\",\"replication-key\":\"commit_timestamp\"}}"
  }
}
```

--------------------------------

### Initialise a pipeline in a workspace

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Use these examples to initialize a new pipeline within a specified workspace.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines' -i -X POST \
    -H 'Content-Type: application/json'
```

```python
import requests

url = "https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("POST", url, headers=headers)

print(response.text.encode('utf8'))
```

--------------------------------

### Initialize Meltano Project Examples

Source: https://docs.meltano.com/reference/command-line-interface

Various ways to initialize a project, including interactive mode, directory specification, and usage statistics configuration.

```bash
# Initialize a new Meltano project interactively  
meltano init  
  
# Initialize a new Meltano project in the  
# "demo-project" directory, and...  
# - share anonymous usage data with the Meltano team  
#   to help them gauge interest in Meltano and its  
#   features and drive development time accordingly:  
meltano init demo-project  
# - OR don't share anything with the Meltano team  
#   about this specific project:  
meltano init demo-project --no-usage-stats  
# - OR don't share anything with the Meltano team  
#   about any project I initialize ever:  
SHELLRC=~/.$(basename $SHELL)rc # ~/.bashrc, ~/.zshrc, etc  
echo "export MELTANO_SEND_ANONYMOUS_USAGE_STATS=0" >> $SHELLRC  
meltano init demo-project # --no-usage-stats is implied  
  
# Initialize a new Meltano project in the current working directory  
meltano init .  
```

--------------------------------

### Initialize local directory and environment file

Source: https://docs.meltano.com/meltano-cloud/importing-data

Navigate to the project directory and create a new .env file for configuration.

```bash
cd your-workspace
touch .env
```

--------------------------------

### Verify a pipeline

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Examples for verifying a pipeline configuration using cURL and Python.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/pipelines/06e1072d-1d38-40fd-b8fa-3db0fa91d75d/verification' -i -X POST \
    -H 'Content-Type: application/json'
```

```python
import requests

url = "https://app.meltano.com:443/api/pipelines/06e1072d-1d38-40fd-b8fa-3db0fa91d75d/verification"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("POST", url, headers=headers)

print(response.text.encode('utf8'))
```

--------------------------------

### Initialize and Run Meltano with Structured Logging

Source: https://docs.meltano.com/changelog/2025-q4-updates-and-changes

Demonstrates initializing a project, adding plugins, selecting streams, and running a pipeline to observe structured log output.

```bash
/projects $ meltano init try-meltano-4  
/projects $ cd try-meltano-4  
/projects/try-meltano-4 $ meltano add tap-fedidb target-csv  
/projects/try-meltano-4 $ meltano select tap-fedidb # select all streams and properties  
/projects/try-meltano-4 $ meltano select tap-fedidb --exclude servers # but exclude the servers stream  
/projects/try-meltano-4 $ meltano run tap-fedidb target-csv  
2025-10-17T18:38:59.967561Z [info     ] Environment 'dev' is active  
2025-10-17T18:39:00.114214Z [info     ] Running job with name 'dev:tap-fedidb-to-target-csv' and run ID '0199f377-d1cd-73b5-8cf3-21501d1df1b1'  
2025-10-17T18:39:00.172674Z [info     ] Using systemdb state backend  
2025-10-17T18:39:00.387700Z [info     ] tap-fedidb v0.5.0, Meltano SDK v0.50.2 name=tap-fedidb  
2025-10-17T18:39:00.387986Z [info     ] Skipping parse of env var settings... name=tap-fedidb  
2025-10-17T18:39:00.388546Z [info     ] Beginning full_table sync of 'popular_accounts' name=tap-fedidb  
2025-10-17T18:39:00.404188Z [info     ] target-csv v0.2.1, Meltano SDK v0.50.2 name=target-csv  
2025-10-17T18:39:00.404340Z [info     ] Skipping parse of env var settings... name=target-csv  
2025-10-17T18:39:00.407237Z [info     ] Writing to destination file '/projects/try-meltano-4/popular_accounts.csv'... name=target-csv  
2025-10-17T18:39:00.655493Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.262111 name=tap-fedidb  
2025-10-17T18:39:00.687525Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb  
2025-10-17T18:39:00.687787Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.2988448143005371 name=tap-fedidb  
2025-10-17T18:39:00.687899Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=tap-fedidb  
2025-10-17T18:39:00.687992Z [info     ] Skipping deselected stream 'servers'. name=tap-fedidb  
2025-10-17T18:39:00.688076Z [info     ] Beginning full_table sync of 'software' name=tap-fedidb  
2025-10-17T18:39:00.697430Z [info     ] Writing to destination file '/projects/try-meltano-4/software.csv'... name=target-csv  
2025-10-17T18:39:00.920552Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.229912 name=tap-fedidb  
2025-10-17T18:39:00.944498Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb  
2025-10-17T18:39:00.944842Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.25652623176574707 name=tap-fedidb  
2025-10-17T18:39:00.945004Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=tap-fedidb  
2025-10-17T18:39:00.975334Z [info     ] Reader 'target-csv' completed processing 203 lines of input (2 schemas, 199 records, 0 batch manifests, 2 state messages, 0 activate version messages). name=target-csv  
2025-10-17T18:39:00.981157Z [info     ] Appending 136 records to file... name=target-csv  
2025-10-17T18:39:00.981449Z [info     ] Appending 63 records to file... name=target-csv  
2025-10-17T18:39:00.984365Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0030138492584228516 name=target-csv  
2025-10-17T18:39:00.984991Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0040929317474365234 name=target-csv  
2025-10-17T18:39:00.993061Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=target-csv  
2025-10-17T18:39:00.993263Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=target-csv  
2025-10-17T18:39:00.999113Z [info     ] Using systemdb state backend  
2025-10-17T18:39:01.002207Z [info     ] Incremental state has been updated at 2025-10-17 18:39:01.002179+00:00.  
2025-10-17T18:39:01.028876Z [info     ] Block run completed  
2025-10-17T18:39:01.029042Z [info     ] Run completed
```

--------------------------------

### Get a Specific Run

Source: https://docs.meltano.com/contribute/api

Example of an HTTP GET request to retrieve details of a specific run.

```http
# get a specific run  
GET meltano/core/v2/envs/prod/jobs/tap-gitlab-target-jsonl/runs/ff643ba2  
Response: {A run}  

```

--------------------------------

### Data Plugin API Response Examples

Source: https://docs.meltano.com/reference/cloud/api/resources/dataplugins

Examples of JSON objects representing various data plugins including extractors and file-based plugins.

```json
"pipUrl" : "git+https://github.com/Matatika/analyze-solarvista@v0.5.0",  
      "repo" : "https://github.com/Matatika/analyze-solarvista",  
      "capabilities" : [ ],  
      "select" : [ ],  
      "update" : {  
        "*.yml" : "true"  
      },  
      "vars" : { },  
      "settings" : [ ],  
      "variants" : [ ],  
      "commands" : { },  
      "matatikaHidden" : false,  
      "requires" : [ {  
        "id" : "4f3acdb4-898b-4ddf-a70f-1141f7b73129",  
        "pluginType" : "TRANSFORM",  
        "name" : "dbt-solarvista",  
        "namespace" : "tap_solarvista",  
        "variant" : "matatika",  
        "hidden" : false,  
        "pipUrl" : "https://github.com/Matatika/dbt-tap-solarvista@v0.4.0",  
        "repo" : "https://github.com/Matatika/dbt-tap-solarvista",  
        "capabilities" : [ ],  
        "select" : [ ],  
        "update" : { },  
        "vars" : {  
          "schema" : "{{ env_var('DBT_SOURCE_SCHEMA') }}"  
        },  
        "settings" : [ ],  
        "variants" : [ ],  
        "commands" : { },  
        "matatikaHidden" : false,  
        "requires" : [ ],  
        "fullDescription" : ""  
      } ],  
      "fullDescription" : "Instant insights on revenue, projects, work items, and engineer performance from Solarvista Live.",  
      "_links" : {  
        "self" : {  
          "href" : "https://app.meltano.com/api/dataplugins/dbf87b80-6eb6-483a-90bb-b7a8c094fb3a"  
        }  
      }  
    }, {  
      "id" : "ffd26c88-aa25-4e04-913c-8dd0b22762d1",  
      "pluginType" : "FILE",  
      "name" : "analyze-trello",  
      "namespace" : "tap_trello",  
      "variant" : "matatika",  
      "label" : "Trello Insights",  
      "description" : "Instant insights on members, cards, boards, and actions from Trello.",  
      "hidden" : false,  
      "pipUrl" : "git+https://github.com/Matatika/analyze-trello@v0.5.0",  
      "repo" : "https://github.com/Matatika/analyze-trello",  
      "capabilities" : [ ],  
      "select" : [ ],  
      "update" : {  
        "*.yml" : "true"  
      },  
      "vars" : { },  
      "settings" : [ ],  
      "variants" : [ ],  
      "commands" : { },  
      "matatikaHidden" : false,  
      "requires" : [ {  
        "id" : "512c097b-df0e-4437-ba9a-3374557a30d9",  
        "pluginType" : "TRANSFORM",  
        "name" : "dbt-tap-trello",  
        "namespace" : "tap_trello",  
        "variant" : "matatika",  
        "hidden" : false,  
        "pipUrl" : "https://github.com/Matatika/dbt-tap-trello@v0.5.0",  
        "repo" : "https://github.com/Matatika/dbt-tap-trello",  
        "capabilities" : [ ],  
        "select" : [ ],  
        "update" : { },  
        "vars" : {  
          "schema" : "{{ env_var('DBT_SOURCE_SCHEMA') }}"  
        },  
        "settings" : [ ],  
        "variants" : [ ],  
        "commands" : { },  
        "matatikaHidden" : false,  
        "requires" : [ ],  
        "fullDescription" : ""  
      } ],  
      "fullDescription" : "Instant insights on members, cards, boards, and actions from Trello.",  
      "_links" : {  
        "self" : {  
          "href" : "https://app.meltano.com/api/dataplugins/ffd26c88-aa25-4e04-913c-8dd0b22762d1"  
        }  
      }  
    }, {  
      "id" : "c5c84dde-1880-494d-95c4-7c71f43528f5",  
      "pluginType" : "EXTRACTOR",  
      "name" : "tap-aftership",  
      "namespace" : "tap_aftership",  
      "variant" : "harrystech",  
      "label" : "AfterShip",  
      "description" : "AfterShip is a shipment tracking platform for online retailers and customers.\n\nAfterShip allows online retailers to track and manage their shipments across multiple carriers and provides customers with real-time updates on the status of their orders. The platform integrates with over 700 carriers worldwide and offers features such as branded tracking pages, delivery notifications, and analytics to help businesses improve their shipping performance. AfterShip also offers a mobile app for customers to track their packages on-the-go.",  
      "logoUrl" : "/assets/logos/extractors/aftership.png",  
      "hidden" : false,  
      "docs" : "https://www.matatika.com/data-details/tap-aftership/",  
      "pipUrl" : "git+https://github.com/harrystech/tap-aftership.git",  
      "repo" : "https://github.com/harrystech/tap-aftership",  
      "capabilities" : [ "CATALOG", "ABOUT", "STATE", "DISCOVER", "STREAM_MAPS", "SCHEMA_FLATTENING" ],  
      "select" : [ ],  
      "update" : { },  
      "vars" : { },  
      "settings" : [ {  
        "name" : "api_key",  
        "aliases" : [ ],  
        "label" : "Api Key",  
        "options" : [ ],  
        "kind" : "STRING",  
        "description" : "A unique identifier used to authenticate and authorize API requests.",  
        "hidden" : false,  
        "sensitive" : true,  
        "configured" : false,  
        "protected" : false  
      }, {  
        "name" : "start_date",
```

--------------------------------

### Install Adapter-Specific dbt Transformer

Source: https://docs.meltano.com/guide/v2-migration

Use this command to install an adapter-specific dbt transformer, for example, for Snowflake. Ensure you have discovered and installed your chosen adapter first.

```bash
# install adapter-specific dbt, e.g. for snowflake  
meltano add dbt-snowflake  
```

--------------------------------

### Create or update a pipeline in a workspace

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Use these examples to create or update an existing pipeline.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines/82317a3b-5ae9-461f-ab9b-8fc4007a0183' -i -X PUT \
    -H 'Content-Type: application/json' \
    -d '{
  "label" : "SIT-generated pipeline [2026-08-07T13:29:14.687250809]",
  "dataComponents" : [ "extractors/tap-google-analytics", "Warehouse", "dbt" ],
  "schedule" : "0 0 0 25 12 ?",
  "properties" : {
    "tap-google-analytics.start_date" : "2026-07-07T13:29:14.698851399Z",
    "tap-google-analytics.property_id" : "1234567890"
  }
}'
```

```python
import requests

url = "https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines/82317a3b-5ae9-461f-ab9b-8fc4007a0183"

data = {
  "label" : "SIT-generated pipeline [2026-08-07T13:29:14.687250809]",
  "dataComponents" : [ "extractors/tap-google-analytics", "Warehouse", "dbt" ],
  "schedule" : "0 0 0 25 12 ?",
  "properties" : {
    "tap-google-analytics.start_date" : "2026-07-07T13:29:14.698851399Z",
    "tap-google-analytics.property_id" : "1234567890"
  }
}
headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("PUT", url, headers=headers, data=data)

print(response.text.encode('utf8'))
```

--------------------------------

### View pipeline metrics

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Examples for retrieving pipeline metrics using cURL and Python.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/pipelines/06e1072d-1d38-40fd-b8fa-3db0fa91d75d/metrics' -i -X GET
```

```python
import requests

url = "https://app.meltano.com:443/api/pipelines/06e1072d-1d38-40fd-b8fa-3db0fa91d75d/metrics"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))
```

--------------------------------

### Retrieve all pipelines in a workspace

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Use these examples to fetch a list of configured pipelines for a specific workspace ID.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines' -i -X GET
```

```python
import requests

url = "https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))
```

--------------------------------

### List All Runs

Source: https://docs.meltano.com/contribute/api

Example of an HTTP GET request to list all runs for a specific job.

```http
# list all runs  
GET meltano/core/v2/envs/prod/jobs/tap-gitlab-target-jsonl/runs  
Response: {[the runs]}  

```

--------------------------------

### Initialize Meltano Project Format

Source: https://docs.meltano.com/reference/command-line-interface

Syntax for creating a new Meltano project directory.

```bash
# Format  
meltano init [project_directory] [--no-usage-stats] [--force]  
```

--------------------------------

### GET /api/dataplugins

Source: https://docs.meltano.com/reference/cloud/api/resources/dataplugins

Retrieves a collection of all data plugins supported by Meltano Cloud.

```APIDOC
## GET /api/dataplugins

### Description
Returns all dataplugins supported by Meltano Cloud.

### Method
GET

### Endpoint
/api/dataplugins

### Response
#### Success Response (200)
- **_embedded** (object) - Dataplugin collection with HAL links.

#### Response Example
{
  "_embedded": {
    "dataplugins": [
      {
        "id": "1149bda6-c93f-4db6-a22c-f95afd60d575",
        "pluginType": "FILE",
        "name": "analyze-sit",
        "namespace": "tap_matatika_sit",
        "variant": "matatika",
        "hidden": false,
        "pipUrl": "git+https://github.com/Matatika/analyze-sit.git",
        "repo": "https://github.com/Matatika/analyze-sit",
        "capabilities": [],
        "select": [],
        "update": {
          "analyze/datasets/tap-matatika-sit/user-ages.yml": "true",
          "analyze/datasets/tap-matatika-sit/user-genders.yml": "true"
        },
        "vars": {},
        "settings": [],
        "variants": [],
        "commands": {},
        "matatikaHidden": true,
        "requires": [],
        "fullDescription": "",
        "_links": {
          "self": {
            "href": "https://app.meltano.com/api/dataplugins/1149bda6-c93f-4db6-a22c-f95afd60d575"
          }
        }
      }
    ]
  }
}
```

--------------------------------

### Add Airflow DAG Generator to Existing Installation

Source: https://docs.meltano.com/guide/orchestration

Add the Meltano DAG generator as a file bundle to your project without installing the Airflow orchestrator plugin. This allows integration with an existing Airflow setup.

```bash
meltano add files files-airflow  
```

--------------------------------

### Initialize and Set Up Meltano Project

Source: https://docs.meltano.com/contribute/api

Commands to activate a virtual environment, initialize a new Meltano project, and navigate into the project directory.

```bash
# Activate your poetry created virtual environment if needed.  
# If you manage your virtualenv activation through other means you can omit this command.  
poetry shell  

# Create a new Meltano project  
meltano init $PROJECT_DIRECTORY  

# Change directory into your newly created project  
cd $PROJECT_DIRECTORY  

```

--------------------------------

### GET /api/workspaces/{workspace-id}/dataplugins

Source: https://docs.meltano.com/reference/cloud/api/resources/dataplugins

Returns all dataplugins available to the specified workspace.

```APIDOC
## GET /api/workspaces/{workspace-id}/dataplugins

### Description
Returns all dataplugins available to the workspace {workspace-id}.

### Method
GET

### Endpoint
/api/workspaces/{workspace-id}/dataplugins

### Parameters
#### Path Parameters
- **workspace-id** (string) - Required - The unique identifier of the workspace.

### Response
#### Success Response (200)
- **_embedded** (object) - Dataplugin collection with HAL links.
```

--------------------------------

### Example .env configuration

Source: https://docs.meltano.com/meltano-cloud/importing-data

Populate the .env file with the environment configuration copied from the Meltano app.

```text
TAP_EXAMPLE_CLIENT_ID=your-client-id
TAP_EXAMPLE_CLIENT_SECRET=your-client-secret
TAP_EXAMPLE_START_DATE=2022-01-01T00:00
TARGET_EXAMPLE_HOST=example.host.com
TARGET_EXAMPLE_PORT=1234
TARGET_EXAMPLE_DB=your-database
TARGET_EXAMPLE_SCHEMA=your-schema
TARGET_EXAMPLE_USERNAME=your-username
TARGET_EXAMPLE_PASSWORD=your-password
```

--------------------------------

### Invitation collection response

Source: https://docs.meltano.com/reference/cloud/api/resources/invitations

Example JSON response body for a successful GET request to the invitations endpoint.

```json
{
  "_embedded" : {
    "invitations" : [ {
      "id" : "47e54624-be37-4886-aa98-8b791f86e8bc",
      "created" : "2026-08-07T13:22:28.750538",
      "lastModified" : "2026-08-07T13:22:28.750539",
      "status" : "PENDING",
      "email" : "sit+test@meltano.com",
      "creator" : {
        "id" : "auth0|6a21dc7aa1db2e036a222942",
        "name" : "sit+prod@meltano.com",
        "handle" : "@sit+prod"
      },
      "workspace" : {
        "id" : "49a8b876-d44e-4aca-86ba-f9213ea40883",
        "created" : "2026-08-07T13:22:22.760153",
        "lastModified" : "2026-08-07T13:22:22.760154",
        "alias" : "ylooops",
        "name" : "Test Workspace [2026-08-07T13:22:22.754833342]",
        "domains" : [ ],
        "repositoryUrl" : "DISABLED",
        "deploymentSecret" : "pvzcuxpbltlavgamwkbzzrfnjliavr",
        "appProperties" : { },
        "status" : "READY",
        "managed" : true,
        "settings" : [ ]
      },
      "_links" : {
        "delete invitation" : {
          "href" : "https://app.meltano.com/api/invitations/47e54624-be37-4886-aa98-8b791f86e8bc",
          "type" : "DELETE"
        },
        "withdraw invitation" : {
          "href" : "https://app.meltano.com/api/invitations/47e54624-be37-4886-aa98-8b791f86e8bc/revoked",
          "type" : "PUT"
        }
      }
    }, {
      "id" : "4bae8f91-2b74-4ef7-9919-854507cbb8cb",
      "created" : "2026-08-07T13:22:33.422861",
      "lastModified" : "2026-08-07T13:22:33.422863",
      "status" : "PENDING",
      "email" : "sit+invited1@meltano.com",
      "creator" : {
        "id" : "auth0|6a21dc7aa1db2e036a222942",
        "name" : "sit+prod@meltano.com",
        "handle" : "@sit+prod"
      },
      "workspace" : {
        "id" : "49a8b876-d44e-4aca-86ba-f9213ea40883",
        "created" : "2026-08-07T13:22:22.760153",
        "lastModified" : "2026-08-07T13:22:22.760154",
        "alias" : "ylooops",
        "name" : "Test Workspace [2026-08-07T13:22:22.754833342]",
        "domains" : [ ],
        "repositoryUrl" : "DISABLED",
        "deploymentSecret" : "pvzcuxpbltlavgamwkbzzrfnjliavr",
        "appProperties" : { },
        "status" : "READY",
        "managed" : true,
        "settings" : [ ]
      },
      "_links" : {
        "delete invitation" : {
          "href" : "https://app.meltano.com/api/invitations/4bae8f91-2b74-4ef7-9919-854507cbb8cb",
          "type" : "DELETE"
        },
        "withdraw invitation" : {
          "href" : "https://app.meltano.com/api/invitations/4bae8f91-2b74-4ef7-9919-854507cbb8cb/revoked",
          "type" : "PUT"
        }
      }
    }, {
      "id" : "249ea87d-8bef-4612-8da2-35fe571ad880",
      "created" : "2026-08-07T13:22:33.488649",
      "lastModified" : "2026-08-07T13:22:33.488652",
      "status" : "PENDING",
      "email" : "sit+invited2@meltano.com",
      "creator" : {
        "id" : "auth0|6a21dc7aa1db2e036a222942",
        "name" : "sit+prod@meltano.com",
        "handle" : "@sit+prod"
      },
      "workspace" : {
        "id" : "49a8b876-d44e-4aca-86ba-f9213ea40883",
        "created" : "2026-08-07T13:22:22.760153",
        "lastModified" : "2026-08-07T13:22:22.760154",
        "alias" : "ylooops",
        "name" : "Test Workspace [2026-08-07T13:22:22.754833342]",
        "domains" : [ ],
        "repositoryUrl" : "DISABLED",
        "deploymentSecret" : "pvzcuxpbltlavgamwkbzzrfnjliavr",
        "appProperties" : { },
        "status" : "READY",
        "managed" : true,
        "settings" : [ ]
      },
      "_links" : {
        "delete invitation" : {
          "href" : "https://app.meltano.com/api/invitations/249ea87d-8bef-4612-8da2-35fe571ad880",
          "type" : "DELETE"
        },
        "withdraw invitation" : {
          "href" : "https://app.meltano.com/api/invitations/249ea87d-8bef-4612-8da2-35fe571ad880/revoked",
          "type" : "PUT"
        }
      }
    } ]
  }
```

--------------------------------

### Delete a pipeline

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Examples for deleting a pipeline using cURL and Python.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/pipelines/5822298c-bde6-4bd3-856e-dd9287ced988' -i -X DELETE
```

```python
import requests

url = "https://app.meltano.com:443/api/pipelines/5822298c-bde6-4bd3-856e-dd9287ced988"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("DELETE", url, headers=headers)

print(response.text.encode('utf8'))
```

--------------------------------

### Retrieve all supported dataplugins

Source: https://docs.meltano.com/reference/cloud/api/resources/dataplugins

Use these examples to fetch the list of all dataplugins supported by Meltano Cloud. Ensure the ACCESS_TOKEN variable is set before execution.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/dataplugins' -i -X GET
```

```python
import requests

url = "https://app.meltano.com:443/api/dataplugins"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))
```

--------------------------------

### Validate a pipeline configuration via API

Source: https://docs.meltano.com/reference/cloud/api/resources/pipelines

Examples for validating a pipeline configuration using cURL and Python.

```bash
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines/validation' -i -X POST \
    -H 'Content-Type: application/json' \
    -d '{
  "label" : "SIT-generated pipeline [2026-08-07T13:29:15.316303400] (updated)",
  "dataComponents" : [ "tap-google-analytics", "Warehouse", "dbt" ]
}'
```

```python
import requests

url = "https://app.meltano.com:443/api/workspaces/ada6c990-3e3c-4940-b2fc-9646ed6c5327/pipelines/validation"

data = {
  "label" : "SIT-generated pipeline [2026-08-07T13:29:15.316303400] (updated)",
  "dataComponents" : [ "tap-google-analytics", "Warehouse", "dbt" ]
}
headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("POST", url, headers=headers, data=data)

print(response.text.encode('utf8'))
```
Tokens:
Raw