### Install Meltano Project

Source: https://docs.meltano.com/contribute/tests

Installs the necessary Meltano project dependencies. This is a prerequisite for following the example.

```shell
meltano install
```

--------------------------------

### Install Meltano with uv

Source: https://docs.meltano.com/guide/production

Use `uv` to install the Meltano package from PyPI. This is recommended for production environments. Consider adding `meltano` to your `requirements.txt` to automate installation.

```bash
uv pip install meltano
```

```bash
uv pip install -r requirements.txt
```

--------------------------------

### Install plugins with options

Source: https://docs.meltano.com/reference/command-line-interface

Use the `install` command to add plugins to your Meltano project. Options like `--schedule`, `--parallelism`, `--clean`, and `--force` allow for customized installation behavior.

```bash
meltano install --schedule=<schedule_name>
```

```bash
meltano install --parallelism=16
```

```bash
meltano install --clean
```

```bash
meltano install --force
```

--------------------------------

### Install Meltano with uv

Source: https://docs.meltano.com/getting-started/installation

Use the uv package manager to install Meltano in an isolated virtual environment. This is recommended for a clean setup.

```bash
$ uv tool install meltano  
---> 100%  
Installed 1 executable: meltano  

```

--------------------------------

### Install uv Package Manager

Source: https://docs.meltano.com/guide/complete_tutorial

Install the uv package manager, a prerequisite for installing Meltano locally. Refer to the provided URL for Windows-specific instructions.

```bash
# See https://docs.astral.sh/uv/getting-started/installation/ for Windows instructions
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Complete Orchestration Script Example

Source: https://docs.meltano.com/contribute/tests

This is a consolidated example of the commands used in the migration process, generated by the integration framework.

```shell
meltano install
meltano el --state-id=my-custom-id tap-gitlab target-postgres
meltano state copy my-custom-id tap-gitlab-to-target-postgres
meltano job add my-new-job --task="tap-gitlab target-postgres"
meltano run my-new-job
```

--------------------------------

### Manage Meltano UI with Docker Compose

Source: https://docs.meltano.com/reference/ui

Instructions for managing the Meltano UI application lifecycle using Docker Compose. Ensure Docker Compose is installed, add the `docker-compose` file bundle, and then start the `meltano-ui` service.

```bash
# For these examples to work, ensure that  
# Docker Compose has been installed  
docker-compose --version  
  
# Add Docker Compose files to your project  
meltano add files files-docker-compose  
  
# Start the `meltano-ui` service in the background  
docker-compose up -d  
```

--------------------------------

### Install Meltano with Postgres extra

Source: https://docs.meltano.com/guide/v3-migration

Install Meltano with the `psycopg2` extra if you are already using Postgres as a backend.

```bash
pipx install "meltano[psycopg2]"
```

--------------------------------

### Install Meltano with Postgres component using uv

Source: https://docs.meltano.com/guide/advanced-topics

Use this command to install Meltano with the `postgres` optional component using `uv`.

```bash
uv tool install --from "meltano[postgres]" meltano  

```

--------------------------------

### Install pipx Package Manager (Linux/macOS)

Source: https://docs.meltano.com/getting-started

Install the pipx package manager, which is recommended for installing Meltano. Ensure your path is updated by sourcing your bashrc.

```bash
python3 -m pip install --user pipx
python3 -m pipx ensurepath
source ~/.bashrc
```

--------------------------------

### Start Meltano UI Server

Source: https://docs.meltano.com/reference/ui

Use this command to start the Meltano UI web server during development. The UI will be available at http://localhost:5000 unless configured otherwise.

```bash
meltano ui  
```

--------------------------------

### Meltano Job CLI Usage Examples

Source: https://docs.meltano.com/contribute/cli

Provides examples for managing Meltano jobs, including listing, adding with single or multiple tasks, and removing jobs. Links to detailed documentation.

```bash
Usage: meltano job [OPTIONS] COMMAND [ARGS]...  
  
  Manage jobs.  
  
  Example usage:  

      # This help  
      meltano job --help  
      # List all jobs in JSON format  
      meltano job list --format json  
      # List a named job  
      meltano job list [JOB_NAME]  

      # Create a new job with a single task representing a single run command.  
      meltano job add NAME --tasks 'tap mapper target command:arg1'  

      # Create a new job with multiple tasks each representing a run command.  
      # The list of tasks must be yaml formatted and consist of a list of strings, list of string lists, or mix of both.  
      meltano job add NAME --tasks '["tap mapper target", "tap2 target2", ...]'  
      meltano job add NAME --tasks '[["tap target dbt:run", "tap2 target2", ...], ...]'  

      # Remove a named job  
      meltano job remove NAME  

 Read more at https://docs.meltano.com/reference/command-line-interface#jobs  
  
Options:  
  --database-uri TEXT  System database URI.  
  --help               Show this message and exit.  
  
Commands:  
  add     Add a new job with tasks.  
  list    List job(s).  
  remove  Remove a job.  
  set     Update an existing jobs tasks  
```

--------------------------------

### Install Meltano with Snowflake Extension using uv

Source: https://docs.meltano.com/concepts/state_backends

Install Meltano and the Snowflake state backend extension using the `uv` package manager.

```bash
uv tool install --with meltano-state-backend-snowflake meltano  
```

--------------------------------

### Install a Mapper Plugin

Source: https://docs.meltano.com/concepts/plugins

Use `meltano add` to install a mapper plugin like any other Meltano plugin. This command installs the specified mapper and provides a link for more information.

```bash
$ meltano add mapper transform-field  
2024-01-01T00:25:40.604941Z [info     ] Installing mapper 'transform-field'  
2024-01-01T00:25:53.152127Z [info     ] Installed mapper 'transform-field'  
  
To learn more about mapper 'transform-field', visit https://github.com/transferwise/pipelinewise-transform-field  

```

--------------------------------

### Install Specific Plugins by Name

Source: https://docs.meltano.com/reference/command-line-interface

Installs specific plugins by their names. Meltano automatically detects the plugin type. This is the recommended way to install plugins.

```bash
meltano install tap-github target-postgres
```

```bash
meltano install tap-gitlab
```

--------------------------------

### Install Meltano and Target

Source: https://docs.meltano.com/tutorials/custom-extractor

Install the custom tap and a target loader using Meltano commands. This prepares the project for data replication.

```bash
meltano install

```

```bash
meltano add target-jsonl

```

--------------------------------

### Install Meltano using uv

Source: https://docs.meltano.com/guide/installation-guide

Install Meltano using the uv package manager, which places it in an isolated virtual environment.

```bash
uv tool install meltano
```

--------------------------------

### Install Plugins with Dash Syntax

Source: https://docs.meltano.com/reference/command-line-interface

Installs specific plugins using dash syntax for plugin names.

```bash
meltano install - tap-github target-postgres
```

--------------------------------

### Install Meltano with Postgres component using pipx

Source: https://docs.meltano.com/guide/advanced-topics

Use this command to install Meltano with the `postgres` optional component using `pipx`.

```bash
pipx install "meltano[postgres]"  

```

--------------------------------

### Example: Add Custom Loader Plugin

Source: https://docs.meltano.com/guide/plugin-management

An example of adding a custom loader plugin named `target-bigquery` to your Meltano project. Note the `--custom` flag is repeated in the source example.

```bash
meltano add --custom loader target-bigquery--custom
```

--------------------------------

### Initialize dbt Project

Source: https://docs.meltano.com/getting-started/part3

Run this command to populate the dbt project scaffold. Ensure dbt-postgres is installed.

```bash
$ meltano invoke dbt-postgres:initialize  
2024-09-22T07:36:52.985090Z [info     ] Environment 'dev' is active  
creating dbt profiles directory path=PosixPath('/[...]/my-meltano-project/transform/profiles/postgres')  
dbt initialized                dbt_ext_type=postgres dbt_profiles_dir=PosixPath('/[...]/my-meltano-project/transform/profiles/postgres') dbt_project_dir=PosixPath('/[...]/my-meltano-project/transform')  

```

--------------------------------

### Check Git Installation

Source: https://docs.meltano.com/guide/installation-guide

Verify if Git is installed on your system, as many Meltano plugins require it.

```bash
git --version
```

--------------------------------

### Verify Custom Tap Installation

Source: https://docs.meltano.com/tutorials/custom-extractor

Verify your custom tap installation using 'uvx' with either a Git URL or from PyPI.

```bash
uvx --from git+https://github.com/myusername/tap-my-custom-source.git tap-my-custom-source --help
```

```bash
uvx tap-my-custom-source --help
```

--------------------------------

### Install Meltano with Snowflake Extension using pipx

Source: https://docs.meltano.com/concepts/state_backends

Install Meltano and inject the Snowflake state backend extension using `pipx`.

```bash
pipx install meltano  
pipx inject meltano 'meltano-state-backend-snowflake'  
```

--------------------------------

### Meltano.yml Configuration Example

Source: https://docs.meltano.com/guide/complete_tutorial

Example of how non-sensitive plugin configuration is stored in `meltano.yml`.

```yaml
environments:
- name: dev
  config:
    plugins:
      extractors:
        - name: tap-gitlab
          config:
            projects: meltano/meltano meltano/tap-gitlab
            start_date: "2024-10-01T00:00:00Z"
```

--------------------------------

### Install Plugins by Type

Source: https://docs.meltano.com/reference/command-line-interface

Installs plugins of a specific type. You can specify the plugin type using the `--plugin-type` flag.

```bash
meltano install --plugin-type=extractor tap-gitlab
```

```bash
meltano install --plugin-type=extractors  # Install all extractors
```

```bash
meltano install --plugin-type=extractor tap-gitlab tap-adwords
```

--------------------------------

### Meltano Add Output for dbt-postgres Installation

Source: https://docs.meltano.com/getting-started/part3

This output log details the process of adding the `dbt-postgres` utility to your Meltano project. It shows environment activation, installation progress, and confirmation of the utility being successfully installed.

```log
2024-09-22T11:32:35.601357Z [info     ] Environment 'dev' is active  
Added utility 'dbt-postgres' to your Meltano project  
...  
2024-01-01T00:25:40.604941Z [info     ] Installing utility 'dbt-postgres'  
---> 100%  
...  
Installed utility 'dbt-postgres'  
2024-01-01T00:25:53.152127Z [info     ] Installed utility 'dbt-postgres'  
2024-01-01T00:25:53.152894Z [info     ] Installed 1/1 plugins  
```

--------------------------------

### Install Plugin from Custom Fork

Source: https://docs.meltano.com/guide/plugin-management

Reinstall a plugin after updating its `pip_url` to point to a custom fork using `meltano install`.

```bash
meltano install [--plugin-type=<type>] <name>

# For example:
meltano install tap-gitlab

```

--------------------------------

### Current Plugin Configuration Output

Source: https://docs.meltano.com/guide/complete_tutorial

Example output showing the current configuration settings for a plugin.

```json
{
  "api_url": "https://gitlab.com",
  "private_token": "my_private_token",
  "groups": "",
  "projects": "meltano/meltano meltano/tap-gitlab",
  "ultimate_license": false,
  "fetch_merge_request_commits": false,
  "fetch_pipelines_extended": false,
  "start_date": "2024-03-01T00:00:00Z"
}
```

--------------------------------

### Install Classic dbt Transformer

Source: https://docs.meltano.com/guide/transformation

Install the generic dbt transformer plugin. This is an older method but still functional.

```bash
meltano add transformer dbt
```

--------------------------------

### YAML Logging Configuration Example

Source: https://docs.meltano.com/guide/logging

An example of a `logging.yaml` file demonstrating various formatter configurations for Meltano's logging system. This includes default Python formatting, colored console output with all keys, plain text console output without locals, console output with locals, key-value format, and JSON format.

```yaml
version: 1  
disable_existing_loggers: false  
  
formatters:  
default: # use a format similar to default generic python logging format  
  format: "[%(asctime)s] [%(process)d|%(threadName)10s|%(name)s] [%(levelname)s] %(message)s"  
structured_colored:  
  (): meltano.core.logging.console_log_formatter  
  colors: true  
structured_colored_all_keys: # log format with colored output showing all log keys  
  (): meltano.core.logging.console_log_formatter  
  colors: true  
  all_keys: true # displays all log keys instead of just the default set  
structured_plain_no_locals: # log format for structured plain text logs without colored output and without local variables  
  (): meltano.core.logging.console_log_formatter  
  colors: false # also disables `rich` traceback formatting  
  show_locals: false # disables local variable logging in tracebacks (which be very verbose and leak sensitive data)  
structured_locals: # log format for structured plain text logs WITH local variables  
  (): meltano.core.logging.console_log_formatter  
  colors: true # also enables traceback formatting with `rich`  
  show_locals: true # enables local variable logging in tracebacks (can be very verbose and leak sensitive data)  
  max_frames: 5 # maximum number of frames to show in tracebacks (default: 2)  
key_value: # log format for traditional key=value style logs  
  (): meltano.core.logging.key_value_formatter  
  sort_keys: false  
json: # log format for json formatted logs  
  (): meltano.core.logging.json_formatter  
  callsite_parameters: true # adds `pathname`, `lineno`, `func_name` and `process` to each log entry  
  dict_tracebacks: false # removes the `exception` object that is added to each log entry  
  show_locals: true # enables local variable logging in tracebacks  
  
handlers:  

```

--------------------------------

### Install Meltano CLI

Source: https://docs.meltano.com/guide/complete_tutorial

Install the Meltano CLI package from PyPI using the uv tool. You can specify a desired Python version using the --python argument.

```bash
uv tool install meltano
```

```bash
uv tool install --python <desired Python version> meltano
```

--------------------------------

### Test PostgreSQL Loader Installation

Source: https://docs.meltano.com/getting-started/part2

Tests if the target-postgres loader was installed correctly by invoking its help command. This command displays usage information and available options for the loader.

```bash
meltano invoke target-postgres --help

```

```bash
$ meltano invoke target-postgres --help  
Usage: target-postgres [OPTIONS]  
  
  Execute the Singer target.  
  
Options:  
  --input FILENAME          A path to read messages from instead of from  
                            standard in.  
  --config TEXT             Configuration file location or 'ENV' to use  
                            environment variables.  
  --format [json|markdown]  Specify output style for --about  
  --about                   Display package metadata and settings.  
  --version                 Display the package version.  
  --help                    Show this message and exit.  

```

--------------------------------

### Run a Basic Pipeline

Source: https://docs.meltano.com/guide/troubleshooting

Example of running a pipeline with specified extractor, loader, transform, and state ID.

```bash
meltano el tap-gitlab target-postgres --transform=run --state-id=gitlab-to-postgres
```

--------------------------------

### Install dbt Transformer Plugin

Source: https://docs.meltano.com/guide/transformation

Install dbt as a transformer plugin. This method is recommended if you prefer using the `elt` command.

```bash
meltano add transformer dbt-<adapter-name>

# For example:

meltano add transformer dbt-snowflake
```

--------------------------------

### Initialize and Run Meltano Project

Source: https://docs.meltano.com/changelog

Demonstrates initializing a new Meltano project, adding plugins, selecting streams, and running a data extraction and loading job. Observe the structured log output.

```bash
/projects $ meltano init try-meltano-4  
/projects $ cd try-meltano-4  
/projects/try-meltano-4 $ meltano add tap-fedidb target-csv  
/projects/try-meltano-4 $ meltano select tap-fedidb # select all streams and properties  
/projects/try-meltano-4 $ meltano select tap-fedidb --exclude servers # but exclude the servers stream  
/projects/try-meltano-4 $ meltano run tap-fedidb target-csv  
2025-10-17T18:38:59.967561Z [info     ] Environment 'dev' is active  
2025-10-17T18:39:00.114214Z [info     ] Running job with name 'dev:tap-fedidb-to-target-csv' and run ID '0199f377-d1cd-73b5-8cf3-21501d1df1b1'  
2025-10-17T18:39:00.172674Z [info     ] Using systemdb state backend  
2025-10-17T18:39:00.387700Z [info     ] tap-fedidb v0.5.0, Meltano SDK v0.50.2 name=tap-fedidb  
2025-10-17T18:39:00.387986Z [info     ] Skipping parse of env var settings... name=tap-fedidb  
2025-10-17T18:39:00.388546Z [info     ] Beginning full_table sync of 'popular_accounts' name=tap-fedidb  
2025-10-17T18:39:00.404188Z [info     ] target-csv v0.2.1, Meltano SDK v0.50.2 name=target-csv  
2025-10-17T18:39:00.404340Z [info     ] Skipping parse of env var settings... name=target-csv  
2025-10-17T18:39:00.407237Z [info     ] Writing to destination file '/projects/try-meltano-4/popular_accounts.csv'... name=target-csv  
2025-10-17T18:39:00.655493Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.262111 name=tap-fedidb  
2025-10-17T18:39:00.687525Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb  
2025-10-17T18:39:00.687787Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.2988448143005371 name=tap-fedidb  
2025-10-17T18:39:00.687899Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=tap-fedidb  
2025-10-17T18:39:00.687992Z [info     ] Skipping deselected stream 'servers'. name=tap-fedidb  
2025-10-17T18:39:00.688076Z [info     ] Beginning full_table sync of 'software' name=tap-fedidb  
2025-10-17T18:39:00.697430Z [info     ] Writing to destination file '/projects/try-meltano-4/software.csv'... name=target-csv  
2025-10-17T18:39:00.920552Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.229912 name=tap-fedidb  
2025-10-17T18:39:00.944498Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb  
2025-10-17T18:39:00.944842Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.25652623176574707 name=tap-fedidb  
2025-10-17T18:39:00.945004Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=tap-fedidb  
2025-10-17T18:39:00.975334Z [info     ] Reader 'target-csv' completed processing 203 lines of input (2 schemas, 199 records, 0 batch manifests, 2 state messages, 0 activate version messages). name=target-csv  
2025-10-17T18:39:00.981157Z [info     ] Appending 136 records to file... name=target-csv  
2025-10-17T18:39:00.981449Z [info     ] Appending 63 records to file... name=target-csv  
2025-10-17T18:39:00.984365Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0030138492584228516 name=target-csv  
2025-10-17T18:39:00.984991Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0040929317474365234 name=target-csv  
2025-10-17T18:39:00.993061Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=target-csv  
2025-10-17T18:39:00.993263Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=target-csv  
2025-10-17T18:39:00.999113Z [info     ] Using systemdb state backend  
2025-10-17T18:39:01.002207Z [info     ] Incremental state has been updated at 2025-10-17 18:39:01.002179+00:00.  
2025-10-17T18:39:01.028876Z [info     ] Block run completed  
2025-10-17T18:39:01.029042Z [info     ] Run completed  

```

--------------------------------

### Initialize and Set Up Meltano Project

Source: https://docs.meltano.com/contribute/api

Commands to activate a virtual environment, initialize a new Meltano project, and navigate into the project directory.

```bash
# Activate your poetry created virtual environment if needed.  
# If you manage your virtualenv activation through other means you can omit this command.  
poetry shell  

# Create a new Meltano project  
meltano init $PROJECT_DIRECTORY  

# Change directory into your newly created project  
cd $PROJECT_DIRECTORY  

```

--------------------------------

### Get a Specific Run

Source: https://docs.meltano.com/contribute/api

Example of an HTTP GET request to retrieve details of a specific run.

```http
# get a specific run  
GET meltano/core/v2/envs/prod/jobs/tap-gitlab-target-jsonl/runs/ff643ba2  
Response: {A run}  

```

--------------------------------

### Initialize and Run Meltano Project with Structured Logging

Source: https://docs.meltano.com/changelog/2025-q4-updates-and-changes

Demonstrates the steps to initialize a new Meltano project, add plugins, select streams, and run a pipeline, showcasing the structured log output. Ensure plugins advertise the `structured_logging` capability for full compatibility.

```bash
/projects $ meltano init try-meltano-4  
/projects $ cd try-meltano-4  
/projects/try-meltano-4 $ meltano add tap-fedidb target-csv  
/projects/try-meltano-4 $ meltano select tap-fedidb # select all streams and properties  
/projects/try-meltano-4 $ meltano select tap-fedidb --exclude servers # but exclude the servers stream  
/projects/try-meltano-4 $ meltano run tap-fedidb target-csv  
2025-10-17T18:38:59.967561Z [info     ] Environment 'dev' is active  
2025-10-17T18:39:00.114214Z [info     ] Running job with name 'dev:tap-fedidb-to-target-csv' and run ID '0199f377-d1cd-73b5-8cf3-21501d1df1b1'  
2025-10-17T18:39:00.172674Z [info     ] Using systemdb state backend  
2025-10-17T18:39:00.387700Z [info     ] tap-fedidb v0.5.0, Meltano SDK v0.50.2 name=tap-fedidb  
2025-10-17T18:39:00.387986Z [info     ] Skipping parse of env var settings... name=tap-fedidb  
2025-10-17T18:39:00.388546Z [info     ] Beginning full_table sync of 'popular_accounts' name=tap-fedidb  
2025-10-17T18:39:00.404188Z [info     ] target-csv v0.2.1, Meltano SDK v0.50.2 name=target-csv  
2025-10-17T18:39:00.404340Z [info     ] Skipping parse of env var settings... name=target-csv  
2025-10-17T18:39:00.407237Z [info     ] Writing to destination file '/projects/try-meltano-4/popular_accounts.csv'... name=target-csv  
2025-10-17T18:39:00.655493Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.262111 name=tap-fedidb  
2025-10-17T18:39:00.687525Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb  
2025-10-17T18:39:00.687787Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.2988448143005371 name=tap-fedidb  
2025-10-17T18:39:00.687899Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=tap-fedidb  
2025-10-17T18:39:00.687992Z [info     ] Skipping deselected stream 'servers'. name=tap-fedidb  
2025-10-17T18:39:00.688076Z [info     ] Beginning full_table sync of 'software' name=tap-fedidb  
2025-10-17T18:39:00.697430Z [info     ] Writing to destination file '/projects/try-meltano-4/software.csv'... name=target-csv  
2025-10-17T18:39:00.920552Z [info     ] METRIC                         metric_name=http_request_duration metric_value=0.229912 name=tap-fedidb  
2025-10-17T18:39:00.944498Z [info     ] METRIC                         metric_name=http_request_count metric_value=1 name=tap-fedidb  
2025-10-17T18:39:00.944842Z [info     ] METRIC                         metric_name=sync_duration metric_value=0.25652623176574707 name=tap-fedidb  
2025-10-17T18:39:00.945004Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=tap-fedidb  
2025-10-17T18:39:00.975334Z [info     ] Reader 'target-csv' completed processing 203 lines of input (2 schemas, 199 records, 0 batch manifests, 2 state messages, 0 activate version messages). name=target-csv  
2025-10-17T18:39:00.981157Z [info     ] Appending 136 records to file... name=target-csv  
2025-10-17T18:39:00.981449Z [info     ] Appending 63 records to file... name=target-csv  
2025-10-17T18:39:00.984365Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0030138492584228516 name=target-csv  
2025-10-17T18:39:00.984991Z [info     ] METRIC                         metric_name=batch_processing_time metric_value=0.0040929317474365234 name=target-csv  
2025-10-17T18:39:00.993061Z [info     ] METRIC                         metric_name=record_count metric_value=136 name=target-csv  
2025-10-17T18:39:00.993263Z [info     ] METRIC                         metric_name=record_count metric_value=63 name=target-csv  
2025-10-17T18:39:00.999113Z [info     ] Using systemdb state backend  
2025-10-17T18:39:01.002207Z [info     ] Incremental state has been updated at 2025-10-17 18:39:01.002179+00:00.  
2025-10-17T18:39:01.028876Z [info     ] Block run completed  
2025-10-17T18:39:01.029042Z [info     ] Run completed  
```

--------------------------------

### Install Adapter-Specific dbt Transformer

Source: https://docs.meltano.com/guide/v2-migration

Use this command to install an adapter-specific dbt transformer, for example, for Snowflake. Ensure you have discovered and installed your chosen adapter first.

```bash
# install adapter-specific dbt, e.g. for snowflake  
meltano add dbt-snowflake  
```

--------------------------------

### Start Airflow Webserver with Meltano

Source: https://docs.meltano.com/guide/orchestration

Invoke the Airflow webserver to access the Airflow UI. This command starts the webserver, allowing you to view and manage Meltano's pipeline DAGs.

```bash
meltano invoke airflow webserver  

```

--------------------------------

### Initialize and Run Meltano Project

Source: https://docs.meltano.com/reference/ui

Commands to initialize a new Meltano project, navigate into it, and start the UI development server. Assumes poetry is used for environment management.

```bash
poetry shell
# Create a new Meltano project
meltano init $PROJECT_DIRECTORY
# Change directory into your newly created project
cd $PROJECT_DIRECTORY
# Start the Meltano API and a production build of Meltano UI that you can ignore
meltano ui
```

```bash
# Open a new terminal tab and go to the directory you cloned meltano into
cd $WHEREVER_YOU_CLONED_MELTANO/src/webapp
# Install frontend infrastructure at the root directory
yarn
# Start local development environment
yarn serve
```

--------------------------------

### Install Custom State Manager with Meltano

Source: https://docs.meltano.com/guide/custom-state-backend

Install your custom state manager package alongside Meltano using pip. This makes the custom backend available for use.

```bash
uv tool install --with git+https://github.com/your-username/my-state-manager.git meltano

```

--------------------------------

### List All Runs

Source: https://docs.meltano.com/contribute/api

Example of an HTTP GET request to list all runs for a specific job.

```http
# list all runs  
GET meltano/core/v2/envs/prod/jobs/tap-gitlab-target-jsonl/runs  
Response: {[the runs]}  

```

--------------------------------

### Meltano Environment Examples

Source: https://docs.meltano.com/reference/command-line-interface

Demonstrates adding a new environment named 'prod', listing existing environments, configuring a plugin within that environment, and removing the environment.

```bash
# Add a new Environment  
meltano environment add prod  
  
# List existing Environments  
meltano environment list  
  
# Add plugin configuration within the new Environment  
meltano --environment=prod config target-postgres set batch_size_rows 50000  
  
# Remove an Environment  
meltano environment remove prod  
```

--------------------------------

### Initialize a New Meltano Project

Source: https://docs.meltano.com/getting-started

Use the `meltano init` command to create a new Meltano project. Replace `<project directory path>` with your desired directory name.

```bash
meltano init <project directory path>
```

```bash
# For example:
meltano init my-meltano-project
```

--------------------------------

### Add Airflow DAG Generator to Existing Installation

Source: https://docs.meltano.com/guide/orchestration

Add the Meltano DAG generator as a file bundle to your project without installing the Airflow orchestrator plugin. This allows integration with an existing Airflow setup.

```bash
meltano add files files-airflow  
```

--------------------------------

### Configure File Bundle Updates with Glob Patterns

Source: https://docs.meltano.com/concepts/plugins

Use quotes for file paths starting with '%' or glob patterns to ensure they are correctly interpreted as YAML. This example matches all `.yml` files for automatic updates.

```yaml
files:

```
  * name: dbt update: '*.yml': true

```

--------------------------------

### Set Up Meltano Development Environment

Source: https://docs.meltano.com/contribute/prerequisites

Clone the Meltano repository, install development tools like Nox and pre-commit, sync dependencies, and activate the virtual environment. This sets up your local environment for development.

```bash
# Clone the Meltano repo  
git clone git@github.com:meltano/meltano.git  
  
# Change directory into the Meltano project  
cd meltano  
  
# Install the, Nox and pre-commit tools  
uv tool install nox  
uv tool install pre-commit  
  
# Install all the dependencies  
uv sync  
  
# Install the pre-commit hook  
pre-commit install --install-hooks  
  
# Obtain a shell in the uv-managed virtual environment  
source .venv/bin/activate  

```

--------------------------------

### Start MailHog for Email Testing

Source: https://docs.meltano.com/reference/ui

Use this Docker command to start a local MailHog instance for testing outgoing emails sent by Meltano. Emails will be available at `http://localhost:8025/`.

```docker
docker run --rm -p 1025:1025 -p 8025:8025 --name mailhog mailhog/mailhog

```

--------------------------------

### Default YAML Configuration

Source: https://docs.meltano.com/guide/user-yaml-config

Example of a user configuration file specifying default YAML formatting settings.

```yaml
yaml:  
  indent: 4  
  block_seq_indent: 2  
  sequence_dash_offset: 2  

```

--------------------------------

### Install Python with uv

Source: https://docs.meltano.com/tutorials/custom-extractor

Installs a specific version of Python using the uv package manager. Ensure uv is installed and version 0.3.0 or higher.

```bash
uv python install 3.13  

```

--------------------------------

### Initialize New Meltano Project with Docker

Source: https://docs.meltano.com/guide/installation-guide

Use this command to create a new Meltano project. Ensure Docker is installed and running. Replace '/your/projects/directory' with your desired path and 'yourprojectname' with your project's name.

```bash
cd /your/projects/directory  

docker run -v "$(pwd)":/projects \
             -w /projects \
             meltano/meltano init yourprojectname  

```

--------------------------------

### Initialize a New Meltano Project

Source: https://docs.meltano.com/tutorials/custom-extractor

Create a new Meltano project in the current directory. You will be prompted for a project name.

```bash
meltano init  
```

--------------------------------

### Install Meltano using pipx

Source: https://docs.meltano.com/guide/installation-guide

Install the Meltano package into its own isolated virtual environment using pipx. This ensures a clean installation without dependency conflicts.

```bash
pipx install meltano
```

--------------------------------

### Start Superset UI

Source: https://docs.meltano.com/guide/analysis

Launch the Superset web interface. It will be accessible at http://localhost:8088 by default. The backend database is stored locally.

```bash
meltano invoke superset:ui
```

--------------------------------

### Install psycopg2 in Meltano Docker image

Source: https://docs.meltano.com/guide/v4-migration

If PostgreSQL support is needed in Meltano Docker images, install psycopg2 explicitly using 'uv pip install psycopg2-binary'.

```dockerfile
FROM meltano/meltano:latest
RUN uv pip install psycopg2-binary

```

--------------------------------

### Install Meltano into a Virtual Environment

Source: https://docs.meltano.com/concepts/python_virtual_environments

Once your virtual environment is activated, install the Meltano package using pip. Note that pipx is generally recommended for installing Python packages.

```bash
pip install meltano  
```

--------------------------------

### Initialize Meltano Project

Source: https://docs.meltano.com/guide/complete_tutorial

Initialize a new Meltano project in a specified directory. This command creates the project structure and the meltano.yml configuration file.

```bash
meltano init <project directory path>

# For example:
meltano init my-meltano-project
```

```bash
# If you're using Docker, don't forget to mount the current working directory:
docker run -v $(pwd):/projects -w /projects meltano/meltano init my-meltano-project
```

--------------------------------

### Add a new plugin without installing

Source: https://docs.meltano.com/reference/command-line-interface

Use this command to add a new plugin to your project without automatically installing it. This is useful for managing dependencies or setting up plugins before installation.

```bash
meltano add tap-shopify --no-install
```

--------------------------------

### Skip Plugin Installation

Source: https://docs.meltano.com/reference/command-line-interface

Use the `--no-install` flag with `meltano add` to add a plugin to your project without automatically installing it. This is useful if you want to manage installations separately.

```bash
meltano add <name> --no-install
```

--------------------------------

### Configure Postgres backend with 'postgres' extra

Source: https://docs.meltano.com/guide/v3-migration

When setting up a Postgres backend for the first time, use the `postgres` extra and the `postgresql+psycopg` URI scheme.

```bash
pipx install "meltano[postgres]"
```

```bash
meltano config meltano set database_uri postgresql+psycopg://<username>:<password>@<host>:<port>/<database>
```

--------------------------------

### Install Meltano with Specific Python Version using uv

Source: https://docs.meltano.com/guide/installation-guide

Install Meltano using uv with a specific Python version, such as 3.13. uv will download the Python version if not already installed.

```bash
uv tool install --python 3.13 meltano
```

--------------------------------

### Initialize a New Meltano Project

Source: https://docs.meltano.com/getting-started/part1

Create a new Meltano project directory. This command sets up the basic project structure, including configuration files and environment initialization.

```bash
meltano init my-meltano-project

```

```text
Created my-new-project
Creating project files...
  my-new-project/
  |-- .meltano
  |-- meltano.yml
  |-- README.md
  |-- requirements.txt
  |-- output/.gitignore
  |-- .gitignore
  |-- extract/.gitkeep
  |-- load/.gitkeep
  |-- transform/.gitkeep
  |-- analyze/.gitkeep
  |-- notebook/.gitkeep
  |-- orchestrate/.gitkeep
Creating system database... Done!
... Project my-new-project has been created!

Meltano Environments initialized with dev, staging, and prod.
To learn more about Environments visit: https://docs.meltano.com/concepts/environments

Next steps:
  cd my-new-project
  Visit https://docs.meltano.com/getting-started#create-your-meltano-project to learn where to go from here.

```

--------------------------------

### API V2 Structure Examples

Source: https://docs.meltano.com/contribute/api

Illustrative examples of API V2 endpoints and their usage.

```APIDOC
## API V2 Examples

### List Runs
```
GET meltano/core/v2/envs/prod/jobs/tap-gitlab-target-jsonl/runs
```
**Response**: `{[the runs]}`

### Get Specific Run
```
GET meltano/core/v2/envs/prod/jobs/tap-gitlab-target-jsonl/runs/ff643ba2
```
**Response**: `{A run}`

### Custom Run Invocation (meltano run equivalent)
```
POST meltano/core/v2/envs/prod/jobs:run
```
**Request Body**: `{"command": "tap mapper target dbt:run", "args"...}`
**Response**: `{The run, with run_id , and "inprogress" status field}`
```

--------------------------------

### Install Specific Plugin

Source: https://docs.meltano.com/guide/plugin-management

Install a specific plugin by its name using `meltano install <name>`. Meltano automatically detects the plugin type. Subsequent calls upgrade the plugin if a newer version is available.

```bash
meltano install tap-gitlab target-postgres
```

--------------------------------

### Load Example Data into Superset

Source: https://docs.meltano.com/guide/analysis

Optionally load sample datasets into Superset to explore its features. This command is equivalent to `superset load_examples`.

```bash
meltano invoke superset:load-examples
```

--------------------------------

### Initialize Meltano Project with Docker

Source: https://docs.meltano.com/getting-started

If using Docker, mount the current working directory to `/projects` and set the working directory to `/projects` when initializing a new Meltano project.

```bash
docker run -v $(pwd):/projects -w /projects meltano/meltano init my-meltano-project
```

--------------------------------

### Start Airflow Webserver

Source: https://docs.meltano.com/getting-started

Invoke the Airflow webserver using Meltano. Add the `-D` flag to run it in the background.

```bash
meltano invoke airflow webserver
```

```bash
# Add `-D` to run the scheduler in the background:  
meltano invoke airflow webserver -D
```

--------------------------------

### Install Meltano with pip

Source: https://docs.meltano.com/getting-started/installation

Install Meltano using pip. It is crucial to install into a clean Python virtual environment to prevent dependency conflicts. The `--require-virtualenv` flag or `PIP_REQUIRE_VIRTUALENV=1` environment variable can enforce this.

```bash
$ pip install --upgrade pip  
Requirement already satisfied.  
$ pip install "meltano"  
---> 100%  
successfully installed meltano  

```

--------------------------------

### Verify Plugin Installation

Source: https://docs.meltano.com/guide/complete_tutorial

Invoke the plugin with the `--help` flag to confirm that Meltano can execute its binary. This is a crucial step after adding a new plugin.

```bash
meltano invoke <plugin> --help
```

```bash
# For example:
meltano invoke target-postgres --help
```

--------------------------------

### Start Airflow Webserver with Meltano

Source: https://docs.meltano.com/guide/complete_tutorial

Launch the Airflow web interface to monitor your DAGs and scheduled pipelines. Use the -D flag to run it in the background.

```bash
meltano invoke airflow webserver

# Add `-D` to run the scheduler in the background:
meltano invoke airflow webserver -D
```

--------------------------------

### Check Python Version

Source: https://docs.meltano.com/getting-started/installation

Verify that you have a supported Python version installed before proceeding with Meltano installation.

```bash
python --version  

```

--------------------------------

### Sample logging configuration file

Source: https://docs.meltano.com/reference/settings

A sample YAML file demonstrating how to configure Python's logging module for Meltano, including formatters and handlers.

```yaml
version: 1  
disable_existing_loggers: false  
  
formatters:  
  default:  
    format: "[%(asctime)s] [%(process)d|%(threadName)10s|%(name)s] [%(levelname)s] %(message)s"  
  structured_plain:  
    (): meltano.core.logging.console_log_formatter  
    colors: False  
  structured_colored:  
    (): meltano.core.logging.console_log_formatter  
    colors: True  
  key_value:  
    (): meltano.core.logging.key_value_formatter  
    sort_keys: False  
  json:  
    (): meltano.core.logging.json_formatter  
  
handlers:  
  console:  
    class: logging.StreamHandler  
    level: DEBUG  
    formatter: structured_colored  
    stream: "ext://sys.stderr"  
  file:  
    class: logging.FileHandler  
    level: INFO  
    filename: /var/log/meltano.log  
    formatter: json  
  
root:  
  level: DEBUG  
  propagate: yes  
  handlers: [console, file]  


```

--------------------------------

### Custom Extractor Installation Log

Source: https://docs.meltano.com/tutorials/custom-extractor

Output indicating the successful installation of a custom extractor into the Meltano project.

```bash
Added extractor 'tap-jsonplaceholder' to your Meltano project  
  
2024-01-01T00:25:40.604941Z [info     ] Installing extractor 'tap-jsonplaceholder'...  
2024-01-01T00:25:53.152127Z [info     ] Installed extractor 'tap-jsonplaceholder'  
  
```

--------------------------------

### Example: View and show job logs

Source: https://docs.meltano.com/reference/command-line-interface

Demonstrates listing recent job runs to identify a log ID, then showing the log content for that specific run.

```bash
# List recent job runs to see available logs  
meltano logs list
```

```bash
# Show log for a specific run (get log_id from list command)  
meltano logs show 550e8400-e29b-41d4-a716-446655440000
```

--------------------------------

### Constrain dbt-postgres Installation

Source: https://docs.meltano.com/getting-started/part3

If installing dbt-postgres requires building psycopg2 from source, constrain the plugin in meltano.yml.

```yaml
utilities:  
  - name: dbt-postgres  
    pip_url: dbt-core dbt-postgres meltano-dbt-ext~=0.4.0  

```

--------------------------------

### Postgres Docker Instance Configuration

Source: https://docs.meltano.com/guide/complete_tutorial

Example command to run a local PostgreSQL Docker instance with specific user, password, and database configurations.

```bash
docker run --name postgres -e POSTGRES_PASSWORD=meltano -e POSTGRES_USER=meltano -e POSTGRES_DB=warehouse -d -p 5432:5432 postgres
```

--------------------------------

### Verify Extractor Installation

Source: https://docs.meltano.com/guide/complete_tutorial

Use `meltano invoke` to verify that the extractor was installed successfully and check its help information.

```bash
meltano invoke <plugin> --help

```

--------------------------------

### Build and Publish a Tap to PyPI

Source: https://docs.meltano.com/tutorials/custom-extractor

Commands to build your tap using 'uv build' and publish it to PyPI using 'uv publish'.

```bash
uv build
```

```bash
uv publish --trusted-publishing=automatic
```

```bash
uv publish --token=<token>
```

--------------------------------

### Navigate to Meltano Project Directory

Source: https://docs.meltano.com/tutorials/custom-extractor

Change the current directory to the newly created Meltano project.

```bash
cd meltano-demo  
```

--------------------------------

### Example: Add Custom Extractor Plugin

Source: https://docs.meltano.com/guide/plugin-management

An example of adding a custom extractor plugin named `tap-covid-19` to your Meltano project.

```bash
meltano add --custom extractor tap-covid-19
```

--------------------------------

### Meltano CLI Help Style: Short and Long Options

Source: https://docs.meltano.com/contribute/cli

Provides examples of how to present short and long options in Meltano CLI documentation, defaulting to long form or showing combined usage.

```bash
meltano --environment=PROD  
```

```bash
meltano [-E/--environment ENVIRONMENT]  
```

--------------------------------

### Invoke plugin without installing

Source: https://docs.meltano.com/reference/command-line-interface

Use `--no-install` with `meltano invoke` to skip the automatic plugin installation step before execution.

```bash
meltano invoke --no-install <plugin>
```

--------------------------------

### Install Plugins with Positional Type Argument

Source: https://docs.meltano.com/reference/command-line-interface

Installs plugins by providing the plugin type as a positional argument, followed by the plugin name.

```bash
meltano install extractor tap-gitlab
```

```bash
meltano install <plugin_type> <plugin_name>
```

--------------------------------

### Meltano CLI Command Structure Example

Source: https://docs.meltano.com/contribute/cli

Illustrates the hierarchical structure of Meltano CLI commands, showing how features are organized into groups and sub-commands with arguments and options.

```bash
meltano job set JOB_NAME --tasks=[<task>...]
```

--------------------------------

### Install Adapter-Specific dbt Utility

Source: https://docs.meltano.com/guide/transformation

Install a dbt adapter as a utility plugin. The plugin type is automatically detected, but can be specified explicitly.

```bash
# install adapter-specific dbt, e.g. for snowflake
# Simplified syntax - plugin type is automatically detected
meltano add dbt-snowflake  # Automatically detected as utility

# Explicit plugin type for disambiguation:
# meltano add --plugin-type utility dbt-snowflake
```