dbt

https://docs.getdbt.com/docs/
Admin
dbt is an industry-standard tool that transforms raw warehouse data into trusted data products by...
Tokens:1,073,072
Snippets:7,825
Trust Score:10
Update:1 month ago
Show doc for...
Tokens:
Raw
### Create virtual environment and install dbt State plugin

Source: https://docs.getdbt.com/docs/deploy/dbt-state-setup?type=platform-signup

Create and activate a virtual environment, then install the dbt State plugin. After installation, run `dbt run` or `dbt build` to start the trial.

```bash
python3 -m venv .venv  
source .venv/bin/activate  

```

```bash
pip install dbt-state  

```

--------------------------------

### Run MetricFlow tutorial

Source: https://docs.getdbt.com/docs/build/metricflow-commands.md

Starts the MetricFlow tutorial for self-hosted users (dbt v1 or v2).

```bash
mf tutorial # For self-hosted users (dbt v1 or v2)
```

--------------------------------

### Install dbt v2 with pip

Source: https://docs.getdbt.com/docs/dbt-versions/dbt-upgrade/upgrading-to-v2.md

Install dbt v2 (Fusion) using pip with the --pre flag to get the pre-release version. This is the primary installation method for upgrading to v2.

```shell
python -m pip install --pre dbt
```

--------------------------------

### Configure and enable OpenAI provider

Source: https://docs.getdbt.com/docs/dbt-ai/wizard-byok.md

Example commands to configure and enable the OpenAI provider. Follow the prompts to enter credentials.

```bash
wizard providers configure openai
wizard providers enable openai
```

--------------------------------

### Wrapping materialization override example

Source: https://docs.getdbt.com/docs/dbt-versions/dbt-upgrade/upgrading-to-v1.7.md

Example of a wrapping materialization in the root project that overrides a built-in materialization from an installed package. This usage is still supported in dbt Core v1.7, while overriding built-in materializations from installed packages without explicit opt-in is deprecated.

```text
{% materialization view, default %}
{{ return(my_cool_package.materialization_view_default()) }}
{% endmaterialization %}
```

--------------------------------

### Install dev locally

Source: https://docs.getdbt.com/docs/local/connect-data-platform/glue-setup.md

Builds and installs the adapter locally for development.

```bash
python3 setup.py build && python3 setup.py install_lib
```

--------------------------------

### Using a macro from a package

Source: https://docs.getdbt.com/docs/build/jinja-macros.md

Example of calling a macro from an installed dbt package.

```sql
select
  field_1,
  field_2,
  field_3,
  field_4,
  field_5,
  count(*)
from my_table
{{ dbt_utils.dimensions(5) }}
```

--------------------------------

### Example output of `wizard providers list`

Source: https://docs.getdbt.com/docs/dbt-ai/wizard-byok.md

Shows the output of the `wizard providers list` command, displaying the currently configured providers and their status.

```bash
➜  jaffle-shop git:(mwong-fusion) wizard providers list
provider              enabled  route      auth         models
dbt                   true     remote     dbt          3
openai                false    local      missing      3
openai_subscription   false    local      missing      1
anthropic             false    local      missing      3
bedrock               true     local      configured   15
azure                 false    local      missing      3
snowflake             false    local      missing      3
gemini                false    local      missing      3
```

--------------------------------

### Install dbt Core 2.0 prerelease with pip

Source: https://docs.getdbt.com/docs/local/install-dbt-v2.md

Installs the dbt Core 2.0 prerelease using pip. Use this during the beta period to get the latest pre-release version.

```shell
python -m pip install --pre dbt-core
```

--------------------------------

### Example profiles.yml configuration

Source: https://docs.getdbt.com/docs/local/profiles.yml.md

Example profiles.yml showing the required structure: a profile name matching dbt_project.yml, a default target, and an outputs block with adapter-specific connection fields. Includes a second profile for multiple projects and comments noting that many configs are adapter-specific.

```yaml
my_project_profile:  # Profile name (matches dbt_project.yml)
  target: dev  # Default target to use
  outputs:
    dev: # Development environment
      type: adapter_type # Required: snowflake, bigquery, databricks, redshift, postgres, etc
      # Connection identifiers (placeholder examples, see adapter-specific pages for supported configs)
      account: abc123  
      database: docs_team 
      schema: dev_schema       
      # Authentication (adapter-specific)
      auth_method: username_password  
      username: username
      password_credentials: password
      # Execution settings (common across adapters)
      threads: 4   # Number of parallel threads

# Multiple profiles (for multiple projects)
my_second_project_profile:
  target: dev
  outputs:
    dev:
      type: snowflake  # Example adapter
      account: account
      user: user
      password: password
      database: database
      schema: schema
      warehouse: warehouse
      threads: 4
```

--------------------------------

### dbt-sail embedded mode profile example

Source: https://docs.getdbt.com/docs/local/connect-data-platform/sail-setup.md

Example profile for embedded mode. Starts a Sail server in-process via pysail; no external server required. Uses schema 'analytics' and 4 threads.

```yaml
my_sail_project:
  target: dev
  outputs:
    dev:
      type: sail
      mode: embedded
      schema: analytics
      threads: 4
```

--------------------------------

### Example config.toml

Source: https://docs.getdbt.com/docs/dbt-ai/wizard-config.md

A complete example of config.toml showing the default model, a trusted project, and an MCP server configuration.

```toml
model = "gpt-4o"

[projects."/Users/you/jaffle-shop"]
trust_level = "trusted"

[mcp_servers.dbt]
command = "DBT_MCP_ENDPOINT"
```

--------------------------------

### Reload PowerShell to use dbt

Source: https://docs.getdbt.com/docs/install-dbt-extension?version=2.0

Starts a new PowerShell process so the updated Path is recognized after installing dbt on Windows.

```powershell
Start-Process powershell  
```

--------------------------------

### Insecure authentication profiles.yml

Source: https://docs.getdbt.com/docs/local/connect-data-platform/hive-setup.md

Example profiles.yml configuration for the insecure authentication method. This method uses no authentication and is only recommended for testing with a local Hive installation.

```yaml
your_profile_name:
  target: dev
  outputs:
    dev:
      type: hive
      host: localhost
      port: PORT # default value: 10000
      schema: SCHEMA_NAME
      
```

--------------------------------

### Install dbt-ibm-netezza adapter

Source: https://docs.getdbt.com/docs/local/connect-data-platform/ibmnetezza-setup.md

Use pip to install the adapter package.

```bash
python -m pip install dbt-ibm-netezza
```

--------------------------------

### Source Code example (SQL)

Source: https://docs.getdbt.com/docs/local/connect-data-platform/glue-setup.md

An incremental Glue model with merge strategy, partition by dt, delta file format, and a macro for incrementality.

```sql
{{ config(
    materialized='incremental',
    incremental_strategy='merge',
    unique_key='user_id',
    partition_by=['dt'],
    file_format='delta'
) }}

with new_events as (

    select * from {{ ref('events') }}

    {% if is_incremental() %}
    where date_day >= date_add(current_date, -1)
    {% endif %}

)

select
    user_id,
    max(date_day) as last_seen,
    current_date() as dt

from events
group by 1
```

--------------------------------

### JSON payload for job.run.started

Source: https://docs.getdbt.com/docs/deploy/webhooks.md

Example webhook payload for a run that has started, including event metadata and run details such as runStatus 'Running' and runStatusCode 3.

```json
{
  "accountId": 1,
  "webhookId": "wsu_12345abcde",
  "eventId": "wev_2L6Z3l8uPedXKPq9D2nWbPIip7Z",
  "timestamp": "2023-01-31T19:28:15.742843678Z",
  "eventType": "job.run.started",
  "webhookName": "test",
  "data": {
    "jobId": "123",
    "jobName": "Daily Job (dbt build)",
    "runId": "12345",
    "environmentId": "1234",
    "environmentName": "Production",
    "dbtVersion": "1.0.0",
    "projectName": "Snowflake Github Demo",
    "projectId": "167194",
    "runStatus": "Running",
    "runStatusCode": 3,
    "runStatusMessage": "None",
    "runReason": "Kicked off from the UI by test@test.com",
    "runStartedAt": "2023-01-31T19:28:07Z"
  }
}
```

--------------------------------

### All environment variables reference

Source: https://docs.getdbt.com/docs/dbt-ai/setup-local-mcp.md

Complete reference of all available environment variables. Most setups only need a subset.

```code
DBT_HOST=cloud.getdbt.com
DBT_PROD_ENV_ID=your-production-environment-id
DBT_DEV_ENV_ID=your-development-environment-id
DBT_USER_ID=your-user-id
DBT_ACCOUNT_ID=your-account-id
DBT_TOKEN=your-service-token
DBT_PROJECT_DIR=/path/to/your/dbt/project
DBT_PATH=/path/to/your/dbt/executable
```

--------------------------------

### OneLogin SAML X.509 Certificate (PEM format)

Source: https://docs.getdbt.com/docs/platform/manage-access/set-up-sso-saml-2.0.md

Example of a PEM-format X.509 certificate that OneLogin provides for the SAML integration. The certificate is required for the dbt setup and must be kept safe.

```text
-----BEGIN CERTIFICATE-----
MIIC8DCCAdigAwIBAgIQSANTIKwxA1221kqhG9w0dbtLabsBAQsFADA0MTIwMAYDVQQD
EylNaWNyb3NvZnQgQXp1cmUgRmVkZXJhdGVkIFNTTyBDZXJ0aWZpY2F0ZTAeFw0yMzEyMjIwMDU1
MDNaFw0yNjEyMjIwMDU1MDNaMDQxMjAwBgNVBAMTKU1pY3Jvc29mdCBBenVyZSBGZWRlcmF0ZWQg
U1NPIENlcnRpZmljYXRlMIIBIjANBgkqhkiG9w0BAEFAAFRANKIEMIIBCgKCAQEAqfXQGc/D8ofK
aXbPXftPotqYLEQtvqMymgvhFuUm+bQ9YSpS1zwNQ9D9hWVmcqis6gO/VFw61e0lFnsOuyx+XMKL
rJjAIsuWORavFqzKFnAz7hsPrDw5lkNZaO4T7tKs+E8N/Qm4kUp5omZv/UjRxN0XaD+o5iJJKPSZ
PBUDo22m+306DE6ZE8wqxT4jTq4g0uXEitD2ZyKaD6WoPRETZELSl5oiCB47Pgn/mpqae9o0Q2aQ
LP9zosNZ07IjKkIfyFKMP7xHwzrl5a60y0rSIYS/edqwEhkpzaz0f8QW5pws668CpZ1AVgfP9TtD
Y1EuxBSDQoY5TLR8++2eH4te0QIDAQABMA0GCSqGSIb3DmAKINgAA4IBAQCEts9ujwaokRGfdtgH
76kGrRHiFVWTyWdcpl1dNDvGhUtCRsTC76qwvCcPnDEFBebVimE0ik4oSwwQJALExriSvxtcNW1b
qvnYduXeZ1CSfwHkHkQLyWBANv8ZCkgtcSWnoHELLOWORLD4aSrAAY2s5hP3ukWdV9zQscUw2b
GwN0/bTxxQgA2NLZzFuHSnkuRX5dbtrunUSPTHMGmFFYBZqZweXZXTcyxp64f3Mtj3g/r/qZ
squyPSq5BrF4ivguYoTcGg4SCPq7fiNRFyBUTTERYU0n46MuPmVC7vXPRQtNRTpJj/b2gGLk
1RcPb1JosS1ct5Mtjs41
-----END CERTIFICATE-----
```

--------------------------------

### Set up Python virtual environment

Source: https://docs.getdbt.com/docs/local/connect-data-platform/glue-setup.md

Creates and activates a virtual environment to isolate package versions and dependencies, then upgrades pip.

```bash
$ sudo yum install git
$ python3 -m venv dbt_venv
$ source dbt_venv/bin/activate
$ python3 -m pip install --upgrade pip
```

--------------------------------

### Use custom incremental strategy from a package (merge_null_safe)

Source: https://docs.getdbt.com/docs/build/incremental-strategy.md

Wraps the `merge_null_safe` custom incremental strategy from the `example` package. Requires installing the package first. The macro delegates to the package's implementation.

```sql
{% macro get_incremental_merge_null_safe_sql(arg_dict) %}
    {% do return(example.get_incremental_merge_null_safe_sql(arg_dict)) %}
{% endmacro %}
```
Tokens:
Raw