### Setup Postgres Database Container

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Use this command to spin up a local Postgres container for development and testing. Ensure Docker is installed.

```shell
make setup-db
```

```shell
docker-compose up --detach postgres
```

--------------------------------

### Install dbt-Core and Dependencies

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Install dbt-core and its associated database adapter dependencies. Use 'make dev' for a simplified setup or pip for direct installation.

```bash
make dev target=[postgres|redshift|...]
```

```bash
python3 -m pip install dbt-core dbt-[postgres|redshift|...]
```

--------------------------------

### Upgrade Pip and Setuptools

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Ensure you have the latest versions of pip and setuptools installed before proceeding with dbt-codegen setup.

```bash
python3 -m pip install --upgrade pip setuptools
```

--------------------------------

### Install Specific dbt-Core Adapter

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Install dbt-core and a specific database adapter, such as PostgreSQL, using either the make command or pip.

```bash
make dev target=postgres
```

```bash
python3 -m pip install dbt-core dbt-postgres
```

--------------------------------

### generate_model_yaml Examples

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_model_yaml.md

Examples demonstrating how to use the generate_model_yaml macro for various scenarios, including single model, multiple models, and with different options.

```APIDOC
## Examples

### Generate YAML for a single model

```jinja2
{{ codegen.generate_model_yaml(
    model_names=['customers']
)
}}
```

### Generate YAML for multiple models

```jinja2
{{ codegen.generate_model_yaml(
    model_names=['customers', 'orders', 'payments']
)
}}
```

### Generate without data types

```jinja2
{{ codegen.generate_model_yaml(
    model_names=['customers'],
    include_data_types=false
)
}}
```

### Generate with upstream descriptions

When upstream descriptions are enabled, column descriptions are inherited from upstream models and sources that feed into the current model. This is useful for maintaining consistent documentation across your DAG.

```jinja2
{{ codegen.generate_model_yaml(
    model_names=['fct_orders'],
    upstream_descriptions=true
)
}}
```

### Using the get_models helper

Use the `get_models` helper macro to dynamically select models by directory and/or prefix:

```jinja2
{% set models = codegen.get_models(directory='marts', prefix='fct_') %}
{{ codegen.generate_model_yaml(
    model_names=models
)
}}
```

Generates YAML for all models in the `marts` directory that start with `fct_`.

### With directory and prefix filtering

```jinja2
{% set staging_models = codegen.get_models(directory='staging', prefix='stg_') %}
{{ codegen.generate_model_yaml(
    model_names=staging_models,
    upstream_descriptions=true,
    include_data_types=true
)
}}
```
```

--------------------------------

### Example Source YAML Configuration

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

This is an example of the YAML output that might be logged to the command line when using the generate_source macro. It defines a source with multiple tables.

```yaml
version: 2

sources:
  - name: raw_jaffle_shop
    database: raw
    schema: raw_jaffle_shop
    tables:
      - name: customers
        description: ""
      - name: orders
        description: ""
      - name: payments
        description: ""
```

--------------------------------

### generate_base_model Macro Usage Examples

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_base_model.md

Examples demonstrating how to use the generate_base_model macro for various configurations, including basic usage, leading commas, materialization, and case-sensitive columns.

```APIDOC
## Examples

### Basic usage: Generate base model SQL

```jinja2
{{ codegen.generate_base_model(
    source_name='raw_data',
    table_name='customers'
) }}
```

### Generate with trailing commas (default)

```jinja2
{{ codegen.generate_base_model(
    source_name='raw_data',
    table_name='customers',
    leading_commas=false
) }}
```

### Generate with leading commas

```jinja2
{{ codegen.generate_base_model(
    source_name='raw_data',
    table_name='customers',
    leading_commas=true
) }}
```

### Generate with materialization config

```jinja2
{{ codegen.generate_base_model(
    source_name='raw_data',
    table_name='customers',
    materialized='table'
) }}
```

### Generate with case-sensitive columns

```jinja2
{{ codegen.generate_base_model(
    source_name='raw_data',
    table_name='Customers',
    case_sensitive_cols=true
) }}
```
```

--------------------------------

### Example Generated Model YAML

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

This is an example of the YAML output generated by the `generate_model_yaml` macro for a 'customers' model. It includes basic structure for model and column definitions.

```yaml
version: 2

models:
  - name: customers
    description: ""
    columns:
      - name: customer_id
        data_type: integer
        description: ""
      - name: customer_name
        data_type: text
        description: ""
```

--------------------------------

### Complete Workflow: Generate and Edit Unit Test

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_unit_test_template.md

This example demonstrates a full workflow: first generating a unit test template and saving it to a file, then manually editing the file to populate it with specific test data.

```bash
dbt run-operation generate_unit_test_template --args '{"model_name": "fct_orders"}' > tests/unit/fct_orders.yml
```

```yaml
unit_tests:
  - name: unit_test_fct_orders
    model: fct_orders

    given:
      - input: ref("stg_customers")
        rows:
          - customer_id: 1
            first_name: John
            last_name: Doe

      - input: ref("stg_orders")
        rows:
          - order_id: 100
            customer_id: 1
            order_date: 2024-01-01

    expect:
      rows:
        - customer_id: 1
          first_name: John
          last_name: Doe
          order_count: 1
          total_amount: 99.99
```

--------------------------------

### Example SQL with Import CTEs

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

This SQL demonstrates the output of the `generate_model_import_ctes` macro. It shows how various model references are structured as import CTEs before the final select statement.

```sql
with customers as (

    select * from {{ ref('stg_customers') }}

),

orders as (

    select * from {{ ref('stg_orders') }}

),

payments as (

    select * from {{ ref('stg_payments') }}

),

customer_orders as (

    select
        customer_id,
        min(order_date) as first_order,
        max(order_date) as most_recent_order,
        count(order_id) as number_of_orders
    from orders
    group by customer_id

),

customer_payments as (

    select
        orders.customer_id,
        sum(amount) as total_amount
    from payments
    left join orders on
         payments.order_id = orders.order_id
    group by orders.customer_id

),

final as (

    select
        customers.customer_id,
        customers.first_name,
        customers.last_name,
        customer_orders.first_order,
        customer_orders.most_recent_order,
        customer_orders.number_of_orders,
        customer_payments.total_amount as customer_lifetime_value
    from customers
    left join customer_orders
        on customers.customer_id = customer_orders.customer_id
    left join customer_payments
        on  customers.customer_id = customer_payments.customer_id

)

select * from final
```

--------------------------------

### Install dbt-codegen Package

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Add the dbt-codegen package and dbt_utils to your packages.yml file to install. Run 'dbt deps' to fetch the packages.

```yaml
packages:
  - package: dbt-labs/codegen
    version: 0.5.0

  - package: dbt-labs/dbt_utils
    version: 1.0.0
```

--------------------------------

### Execute create_base_models via dbt run-operation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/create_base_models.md

This example shows how to execute the create_base_models macro as a dbt run-operation command directly from your local IDE. Ensure you provide the source_name and tables as arguments.

```bash
dbt run-operation codegen.create_base_models --args '{ 
  "source_name": "raw_data",
  "tables": ["customers", "orders", "products"]
}'
```

--------------------------------

### Example Unit Test Template YAML

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

An example of the YAML output from the `generate_unit_test_template` macro. This template provides the basic structure for defining test cases, including inputs and expected results for a model.

```yaml
unit_tests:
  - name: unit_test_order_items
    model: order_items

    given:
      - input: ref("stg_order_items")
        rows:
          - col_a: 
            col_b: 

    expect:
      rows:
        - id: 
```

--------------------------------

### Full dbt Run-Operation Usage with Optional Parameters

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Provides an example of specifying all available optional parameters for a dbt-codegen macro, including schema, column generation, data types, descriptions, and table patterns.

```bash
# Full specification
--args {
  "schema_name": "raw_data",
  "generate_columns": true,
  "include_data_types": true,
  "include_descriptions": true,
  "table_pattern": "dim_%"
}
```

--------------------------------

### Initial Source YAML Output

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

This is the initial YAML output generated by the `generate_source` operation, which serves as a starting point for documentation.

```yaml
version: 2

sources:
  - name: raw_input
    schema: raw_input
    tables:
      - name: customers
        description: ""
        columns:
          - name: id
            data_type: integer
            description: ""
          - name: name
            data_type: text
            description: ""
          - name: email
            data_type: text
            description: ""
```

--------------------------------

### create_base_models Macro Example

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

This macro creates base models from a list of tables within a specified source. It requires `source_name` and accepts a list of `tables` to process.

```jinja2
{{ codegen.create_base_models(
  source_name='required',
  tables=[]
) }}
```

--------------------------------

### generate_base_model Macro as dbt run-operation command

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_base_model.md

Demonstrates how to execute the generate_base_model macro using the `dbt run-operation` command, including examples with and without additional options, and how to output the result to a file.

```APIDOC
### Use as a dbt run-operation command

```bash
dbt run-operation generate_base_model --args '{ "source_name": "raw_data", "table_name": "customers" }'
```

With additional options:

```bash
dbt run-operation generate_base_model --args '{ "source_name": "raw_data", "table_name": "customers", "materialized": "table", "leading_commas": false }'
```

Output to a file:

```bash
dbt --quiet run-operation generate_base_model --args '{ "source_name": "raw_data", "table_name": "customers" }' > models/stg_raw_data_customers.sql
```
```

--------------------------------

### Build and Test New Additions

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

After adding new integration test files, use 'dbt deps' and 'dbt build' to install dependencies and run the specific selection of tests. The '+' operator includes dependencies.

```bash
dbt deps --target {your_target}
dbt build --target {your_target} --select +{your_selection_criteria}
```

--------------------------------

### Original Model with Mixed Sources

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

An example of a dbt model that directly references a raw table and a staging model.

```sql
select
    c.customer_id,
    c.name,
    s.status
from {{ ref('stg_customers') }} c
join raw_db.prod_schema.statuses s
    on c.status_id = s.id
where c.lifetime_value > 1000
```

--------------------------------

### Run generate_base_model with additional options via dbt run-operation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_base_model.md

This example shows how to pass additional parameters like `materialized` and `leading_commas` when running the `generate_base_model` macro via `dbt run-operation`. Ensure all arguments are correctly formatted within the JSON string.

```bash
dbt run-operation generate_base_model --args '{
  "source_name": "raw_data",
  "table_name": "customers",
  "materialized": "table",
  "leading_commas": false
}'
```

--------------------------------

### Example Model YAML with Inherited Descriptions

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

This is an example of a generated model YAML file where column descriptions have been inherited from upstream sources. Columns without upstream descriptions remain empty.

```yaml
models:
  - name: fct_orders
    description: ""
    columns:
      - name: order_id
        data_type: integer
        description: "Unique order identifier"  # From stg_orders
      - name: customer_id
        data_type: integer
        description: "Customer ID"  # From stg_customers
      - name: order_count
        data_type: integer
        description: ""  # New column, no upstream source
```

--------------------------------

### Install dbt-codegen Package

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

Add this package to your packages.yml file to install the dbt-codegen package. Remember to update to the latest version.

```yaml
packages:
  - package: dbt-labs/codegen
    version: X.X.X ## update to latest version here
```

--------------------------------

### dbt run-operation Command

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_model_yaml.md

How to execute the generate_model_yaml macro using the dbt run-operation command, including examples for single and multiple models, and output redirection.

```APIDOC
## Use as a dbt run-operation command

### Single model

```bash
dbt run-operation generate_model_yaml --args '{"model_names": ["customers"]}'
```

### Multiple models

```bash
dbt run-operation generate_model_yaml --args '{ 
  "model_names": ["customers", "orders"],
  "upstream_descriptions": true
}'
```

### Output to a file

```bash
dbt --quiet run-operation generate_model_yaml --args '{ 
  "model_names": ["customers"]
}' > models/schema.yml
```
```

--------------------------------

### generate_base_model Macro Example

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

This macro generates a base model SQL file. It requires `source_name` and `table_name`. Optional parameters include `leading_commas` for style preference and `case_sensitive_cols` for preserving column name casing.

```jinja2
{{ codegen.generate_base_model(
  source_name='required',
  table_name='required',
  leading_commas=False,
  case_sensitive_cols=False,
  materialized=None
) }}
```

--------------------------------

### generate_source Macro Example

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

Use this macro to generate a dbt source definition. All parameters are optional except `schema_name`. It allows customization of column generation, descriptions, data types, and table filtering.

```jinja2
{{ codegen.generate_source(
  schema_name='required',
  database_name=target.database,
  generate_columns=False,
  include_descriptions=False,
  include_data_types=True,
  table_pattern='%',
  exclude='',
  name=schema_name,
  table_names=None,
  include_database=False,
  include_schema=False,
  case_sensitive_databases=False,
  case_sensitive_schemas=False,
  case_sensitive_tables=False,
  case_sensitive_cols=False
) }}
```

--------------------------------

### Filter Models by Prefix for Documentation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Generates schema documentation for models within a directory that match a specific prefix. This allows for targeted documentation generation, for example, only for fact tables.

```jinja2
{% set fact_models = codegen.get_models(directory='marts', prefix='fct_') %}

{{ codegen.generate_model_yaml(
    model_names=fact_models,
    upstream_descriptions=true
) }}
```

--------------------------------

### Generate Base Model with Materialization Config

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_base_model.md

This example shows how to generate base model SQL that includes a dbt `config` block by specifying the `materialized` parameter. The generated SQL will include `{{ config(materialized='...') }}` at the beginning.

```jinja
{{ codegen.generate_base_model(
    source_name='raw_data',
    table_name='customers',
    materialized='table'
) }}
```

--------------------------------

### Activate Virtual Environment

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Reload your virtual environment to apply the installed dependencies. This command activates the environment located in the .venv directory.

```bash
source .venv/bin/activate
```

--------------------------------

### generate_unit_test_template Macro Example

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

Generates a template for dbt unit tests. It requires the `model_name` and has an option `inline_columns` to control the YAML format's compactness.

```jinja2
{{ codegen.generate_unit_test_template(
  model_name='required',
  inline_columns=false
) }}
```

--------------------------------

### Generate Model Documentation YAML

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

After creating models and running `dbt parse`, use `generate_model_yaml()` to generate schema YAML. Paste the output into schema files and add descriptions. This example generates YAML for models in the 'marts' directory with a 'fct_' prefix and includes upstream descriptions.

```jinja2
{% set models = codegen.get_models(directory='marts', prefix='fct_') %}
{{ codegen.generate_model_yaml(
    model_names=models,
    upstream_descriptions=true
) }}
```

--------------------------------

### Customize Base Model SQL

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Example of a customized base model SQL, including transformations like renaming columns, casting data types, and deduplication.

```sql
{{ config(materialized='view') }}

with source as (
    select * from {{ source('raw_input', 'customers') }}
),

renamed as (
    select
        id as customer_id,
        name as customer_name,
        email as customer_email,
        cast(created_at as timestamp) as created_at
    from source
),

deduped as (
    select
        *,
        row_number() over (partition by customer_id order by created_at desc) as rn
    from renamed
)

select * from deduped where rn = 1

```

--------------------------------

### generate_model_yaml Macro Example

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

Use this macro to generate a YAML file for model documentation. It can document specific models, inherit upstream descriptions, and include data types. Set `include_data_types` to `False` for minimal YAML.

```jinja2
{{ codegen.generate_model_yaml(
  model_names=[],
  upstream_descriptions=False,
  include_data_types=True
) }}
```

--------------------------------

### generate_model_import_ctes Macro Example

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

This macro generates SQL code to import CTEs for a given model. It requires the `model_name` and optionally accepts `leading_commas` for code style.

```jinja2
{{ codegen.generate_model_import_ctes(
  model_name='required',
  leading_commas=False
) }}
```

--------------------------------

### Generate Base Model with Trailing Commas

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_base_model.md

This example demonstrates generating base model SQL with trailing commas for column lists, which is the default behavior. Ensure the source and table names are correctly specified.

```jinja
{{ codegen.generate_base_model(
    source_name='raw_data',
    table_name='customers',
    leading_commas=false
) }}
```

--------------------------------

### Install dbt_utils Dependency

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

Ensure dbt_utils is installed in your packages.yml file. Specify the package and its version.

```yaml
packages:
  - package: dbt-labs/dbt_utils
    version: 1.0.0  # or latest
```

--------------------------------

### Set Up Python Virtual Environment

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Create and activate a Python virtual environment in the root of the dbt-codegen repository. This is recommended for managing dependencies.

```shell
python3 -m venv .venv
source .venv/bin/activate
```

--------------------------------

### Adapter Dispatch Pattern in Jinja

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Demonstrates how to use `adapter.dispatch()` for supporting multiple database adapters in dbt macros. This pattern allows for default implementations and adapter-specific overrides.

```jinja
{% macro generate_source(...) %}
    {{ return(adapter.dispatch('generate_source', 'codegen')(...)) }}
{% endmacro %}

{% macro default__generate_source(...) %}
    {# Default implementation for all adapters #}
{% endmacro %}

{% macro bigquery__generate_source(...) %}
    {# BigQuery-specific override #}
{% endmacro %}
```

--------------------------------

### generate_model_import_ctes with Trailing Commas

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_model_import_ctes.md

This example shows the output of the macro when trailing commas are used (default behavior). It generates CTEs for referenced models.

```sql
with stg_customers as (
    select * from {{ ref('stg_customers') }}
),

int_customer_orders as (
    select * from {{ ref('int_customer_orders') }}
),

final as (
    select
        c.customer_id,
        c.first_name,
        o.order_count,
    from stg_customers c
    join int_customer_orders o
        on c.customer_id = o.customer_id
)

select * from final
```

--------------------------------

### Generate Documentation for Models in a Directory

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Generates schema documentation for all models found within a specified directory. This command outputs the documentation to a YAML file.

```bash
dbt run-operation generate_mart_documentation --args '{ 
  "directory": "marts"
}' > models/marts/schema.yml
```

--------------------------------

### Development and Testing Workflow

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

A simplified workflow for development and testing, involving setting up the development environment and running tests. This is an alternative to the more specific 'dbt build' command.

```bash
make dev target={your_target}
make test target={your_target}
```

--------------------------------

### Run Specific Integration Tests Locally

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Run integration tests locally for a specific database adapter, such as PostgreSQL, using either the make command or a shell script.

```bash
make test target=postgres
```

```bash
./run_test.sh postgres
```

--------------------------------

### Generate Source YAML

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Use this command to generate the initial source YAML file for a given schema, including columns, data types, and descriptions.

```bash
dbt run-operation codegen.generate_source --args '{ 
  "schema_name": "raw_input",
  "generate_columns": true,
  "include_descriptions": true,
  "include_data_types": true
}' > models/staging/_sources.yml
```

--------------------------------

### Generate Source with Multi-Database Configuration

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Generate source configurations including the database name for multi-database environments. This ensures correct database context is captured.

```bash
dbt run-operation codegen.generate_source --args '{ 
  "schema_name": "raw_data", 
  "database_name": "analytics_dev", 
  "include_database": true, 
  "generate_columns": true
}'
```

--------------------------------

### Get Model Dependencies

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/helper-macros.md

Retrieves a list of unique node IDs for upstream dependencies (models and sources) of a specified model. Useful for understanding data lineage.

```jinja
{% macro get_model_dependencies(model_name) %}
```

```jinja
{% set deps = codegen.get_model_dependencies('fct_orders') %}
{% for dep in deps %}
  {{ dep }}
{% endfor %}
```

--------------------------------

### Create Unit Test Templates

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Generate a unit test template structure for a model by running `generate_unit_test_template`. Edit the generated YAML file to add test data and then run `dbt test`.

```bash
dbt run-operation generate_unit_test_template --args '{
  "model_name": "fct_orders",
  "inline_columns": false
}' > tests/unit/fct_orders.yml
```

--------------------------------

### Get Tables in Schema

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/helper-macros.md

Retrieves a sorted list of table names from a specified schema. Useful for dynamically generating source definitions or understanding schema contents.

```jinja
{% macro get_tables_in_schema(
  schema_name,
  database_name=target.database,
  table_pattern='%',
  exclude=''
) %}
```

```jinja
{% set tables = codegen.get_tables_in_schema('raw_data') %}
{% for table in tables %}
  - {{ table }}
{% endfor %}
```

--------------------------------

### Using List Parameters in dbt Run-Operation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Illustrates the correct way to pass a list of model names to dbt run-operation.

```bash
# Correct
--args '{"model_names": ["customers", "orders"]}'
```

--------------------------------

### Get dbt Models by Directory and Prefix

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/helper-macros.md

Filters dbt models based on a directory path and/or a name prefix. Useful for organizing and selecting models for specific tasks.

```jinja
{% macro get_models(
  directory=None,
  prefix=None
) %}
```

```jinja
{# Get all staging models #}
{% set stg_models = codegen.get_models(prefix='stg_') %}
```

```jinja
{# Get all fact tables in marts directory #}
{% set facts = codegen.get_models(directory='marts', prefix='fct_') %}
```

```jinja
{# Get all models in staging directory #}
{% set all_staging = codegen.get_models(directory='staging') %}
```

--------------------------------

### Create Base Models using Bash Integration

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Use bash integration to automatically create base model files for a given source and tables. This command is for local CLI use only.

```bash
dbt run-operation codegen.create_base_models --args '{ 
  "source_name": "raw_data", 
  "tables": ["customers", "orders", "products", "payments"]
}'
```

--------------------------------

### Generate Source with Multiple Arguments

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

Use dictionary syntax to pass multiple arguments to the generate_source macro, including schema, database, and a list of table names.

```bash
$ dbt run-operation generate_source --args '{"schema_name": "jaffle_shop", "database_name": "raw", "table_names":["table_1", "table_2"]}'
```

--------------------------------

### Printing Macro Output with Jinja

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Demonstrates how to print the combined output of dbt-codegen macros to the console using the `print()` function and `joined` variable within Jinja.

```jinja
{{ print(joined) }}
{% do return(joined) %}
```

--------------------------------

### Generate Unit Test Template for a Model

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_unit_test_template.md

Use this Jinja macro to generate a basic unit test template for a specified dbt model. This is the starting point for creating comprehensive unit tests.

```jinja2
{{ codegen.generate_unit_test_template(model_name='fct_orders') }}
```

--------------------------------

### Unit Test YAML Structure for Incremental Models

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

This is an example of the YAML output generated for an incremental model unit test. It specifies inputs, overrides, and expected results, including the use of `this` for the incremental table.

```yaml
unit_tests:
  - name: unit_test_incremental_events
    model: incremental_events
    overrides:
      macros:
        is_incremental: true

    given:
      - input: ref("stg_events")
        rows:
          - event_id: 1
            event_date: 2024-01-01

      - input: this
        rows:
          - event_id: 0
            event_date: 2023-12-31

    expect:
      rows:
        - event_id: 1
          event_date: 2024-01-01
        - event_id: 0
          event_date: 2023-12-31
```

--------------------------------

### Refactor Models with Import CTEs

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Transform existing models to use import CTEs at the top by running `generate_model_import_ctes`. Review the output SQL and replace the original model.

```bash
dbt run-operation generate_model_import_ctes --args '{
  "model_name": "fct_orders",
  "leading_commas": false
}' > refactored_fct_orders.sql
```

--------------------------------

### Generate Base Model Commands for Multiple Tables

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/create_base_models.md

Use this macro to generate bash commands for creating multiple base model files at once. It requires the dbt source name and a list of table names.

```jinja
{{ codegen.create_base_models(
    source_name='raw_data',
    tables=['customers', 'orders', 'products']
) }}
```

--------------------------------

### Run Source Generation Operation with Output Redirection

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/configuration.md

Execute the `generate_source` macro as an operation using the local dbt CLI and redirect the output to a file.

```bash
dbt --quiet run-operation generate_source --args '{"schema_name": "raw_data"}' > output.yml
```

--------------------------------

### Get Resource Object from Unique ID

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/helper-macros.md

Use this macro to fetch the complete resource object for a given unique ID. It parses the ID to determine the resource type and looks it up in the appropriate manifest graph structure.

```jinja
{% set resource = codegen.get_resource_from_unique_id('model.my_project.customers') %}
Name: {{ resource.name }}
Type: {{ resource.resource_type }}
Path: {{ resource.path }}

{% set source_resource = codegen.get_resource_from_unique_id('source.my_project.raw.orders') %}
Source: {{ source_resource.source_name }}
Table: {{ source_resource.identifier }}
```

--------------------------------

### Run Tox Supported Tests

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Execute integration tests using tox, simulating the environment used in GitHub workflows. This command targets the PostgreSQL adapter.

```bash
make test_tox target=postgres
```

--------------------------------

### Minimal dbt Run-Operation Usage with Optional Parameters

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Shows the most basic way to invoke a dbt-codegen macro by only specifying the required schema name.

```bash
# Minimal usage
--args '{"schema_name": "raw_data"}'
```

--------------------------------

### generate_unit_test_template Macro

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_unit_test_template.md

Generates a YAML template for dbt unit tests. It automatically discovers upstream dependencies (sources and ref models) and creates a template with `given` inputs and `expect` outputs, making it easier to start writing unit tests for your models.

```APIDOC
## Macro: generate_unit_test_template

### Description
Generates a YAML template for dbt unit tests. Automatically discovers upstream dependencies (sources and ref models) and creates a template with `given` inputs and `expect` outputs, making it easier to start writing unit tests for your models.

### Signature
```sql
{% macro generate_unit_test_template(
  model_name,
  inline_columns=false
) %}
```

### Parameters
#### Path Parameters
- **model_name** (string) - Required - The name of the model to generate unit test template for
- **inline_columns** (boolean) - Optional - Whether to format column definitions on a single line (vs multiline). Defaults to `false`.

### Return Type
`string` — The generated YAML test template is printed to the command line and returned as a string.

### Output Format
The macro generates YAML in the following structure:
```yaml
unit_tests:
  - name: unit_test_{model_name}
    model: {model_name}
    [overrides:
      macros:
        is_incremental: true]  # Only for incremental models
    
    given:  # Input fixtures for each upstream dependency
      - input: ref("upstream_model")
        rows:
          - column_1: 
            column_2:
      
      [- input: source("source_name", "table_name")
        rows:
          - column_1:
            column_2:]  # Optional for incremental models
      
      [- input: this
        rows:
          - column_1:
            column_2:]  # Only for incremental models
    
    expect:  # Expected output
      rows:
        - output_column_1:
          output_column_2:
```
```

--------------------------------

### Auto-Create Base Model Files

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Use `create_base_models` command to automatically create model files for multiple tables. This command is for local CLI use only. It logs bash commands that can be executed to create the files.

```bash
dbt run-operation codegen.create_base_models --args '{
  "source_name": "raw_data",
  "tables": ["customers", "orders", "products"]
}'
```

```bash
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" customers && \
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" orders && \
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" products
```

--------------------------------

### Output Unit Test Template to a File

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_unit_test_template.md

This command generates the unit test template and redirects its output directly to a YAML file, streamlining the test creation process.

```bash
dbt --quiet run-operation generate_unit_test_template --args '{
  "model_name": "fct_orders"
}' > tests/unit/fct_orders.yml
```

--------------------------------

### Generate Unit Test Template for Incremental Models

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Use this command to generate a unit test template for incremental models. It includes configurations for `is_incremental` override and `this` input.

```bash
dbt run-operation codegen.generate_unit_test_template --args '{
  "model_name": "incremental_events",
  "inline_columns": false
}'
```

--------------------------------

### Generate Source with Case-Sensitive Columns (BigQuery)

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Generate source configurations for BigQuery, specifying case-sensitive column names. This command helps in accurately reflecting BigQuery's schema.

```bash
dbt run-operation codegen.generate_source --args '{ 
  "schema_name": "raw_data", 
  "generate_columns": true, 
  "case_sensitive_cols": true, 
  "case_sensitive_tables": false
}'
```

--------------------------------

### base_model_creation Macro

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

A helper macro that orchestrates the creation of base models.

```APIDOC
## base_model_creation Macro

### Description
A helper macro that orchestrates the creation of base models. It is typically called by other macros or operations.

### Method
Macro (dbt run-operation or Jinja rendering)

### Endpoint
N/A (dbt macro)

### Parameters
#### Arguments
- **schema_name** (string) - Required - The name of the schema containing the tables.
- **table_names** (list of strings) - Optional (default=None) - Specific table names to process.
- **source_name** (string) - Optional (default=None) - The name of the source.
- **base_model_relation_name** (string) - Optional (default=None) - The desired name for the base model relation.
- **generate_columns** (boolean) - Optional (default=False) - Whether to generate column definitions.
- **include_descriptions** (boolean) - Optional (default=False) - Whether to include description placeholders.
- **include_data_types** (boolean) - Optional (default=True) - Whether to include data types.
- **database_name** (string) - Optional (default=target.database) - The database name.
- **schema_name_override** (string) - Optional (default=None) - An override for the schema name.
- **alias** (string) - Optional (default=None) - An alias for the source table.
- **quoting** (dict) - Optional (default=None) - Quoting configurations.
- **quote_columns** (boolean) - Optional (default=False) - Whether to quote column names.
- **quote_identifiers** (boolean) - Optional (default=False) - Whether to quote identifiers.

### Request Example
#### Jinja Usage:
```jinja
{{ codegen.base_model_creation(schema_name='raw_data', table_names=['users', 'events'] ) }}
```

#### Operation Usage:
```bash
$ dbt run-operation base_model_creation --args '{"schema_name": "raw_data", "table_names": ["users", "events"]}'
```

### Response
#### Success Response (Output to console or file)
- **SQL Strings** (list of strings) - A list of generated SQL code for each base model.

#### Response Example (SQL Output for multiple models)
```sql
-- base model for users
select
    *
from {{ source('your_source_name', 'users') }}

-- base model for events
select
    *
from {{ source('your_source_name', 'events') }}
```
```

--------------------------------

### Generate Source with Columns

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

Include column names in the generated source by setting 'generate_columns' to true. This requires the 'schema_name' argument.

```bash
$ dbt run-operation generate_source --args '{"schema_name": "jaffle_shop", "generate_columns": true}'
```

--------------------------------

### Execute Template Generation via dbt run-operation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_unit_test_template.md

This command executes the `generate_unit_test_template` macro directly using `dbt run-operation`. It's a convenient way to generate templates without needing to modify Jinja files.

```bash
dbt run-operation generate_unit_test_template --args '{"model_name": "fct_orders"}'
```

--------------------------------

### Execute Base Model Creation Commands

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Copy and paste these commands into your terminal to generate the base model SQL files. These commands are generated by the dbt-codegen operation.

```bash
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" customers && 
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" orders && 
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" products && 
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" payments
```

--------------------------------

### Generate Model YAML by Directory and Prefix

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

This Jinja code snippet demonstrates how to dynamically select models from a specific directory with a given prefix and then generate their corresponding YAML configurations.

```jinja2
{# analysis/generate_staging_docs.sql #}

{% set models = codegen.get_models(
  directory='staging',
  prefix='stg_'
) %}

{{ codegen.generate_model_yaml(
  model_names=models,
  upstream_descriptions=true
) }}
```

--------------------------------

### Bash Script for Creating Base Models

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/create_base_models.md

This is the format of the bash commands generated by the create_base_models macro. Each command executes a bash script to create a single base model file. Copy and execute these commands in your terminal.

```bash
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" customers && 
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" orders && 
source dbt_packages/codegen/bash_scripts/base_model_creation.sh "raw_data" products
```

--------------------------------

### Run generate_base_model Operation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/README.md

Execute the generate_base_model macro as a dbt operation, providing the source name and table name as arguments.

```bash
$ dbt run-operation generate_base_model --args '{"source_name": "raw_jaffle_shop", "table_name": "customers"}'
```

--------------------------------

### Detect Operating System (Mac/Linux vs. Windows)

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/helper-macros.md

Determines if the dbt project is running on a Mac/Linux or Windows environment. This is useful for setting the correct path separator for file operations.

```jinja
{% if codegen.is_os_mac_or_linux() %}
  {# Mac/Linux path separator #}
  {% set sep = "/" %}
{% else %}
  {# Windows path separator #}
  {% set sep = "\\" %}
{% endif %}

```

--------------------------------

### Run Circle CI Integration Tests Locally

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/integration_tests/README.md

Execute all integration tests on your local machine in a manner consistent with how they are run in Circle CI. Specify the target database adapter.

```bash
make test target=[postgres|redshift|...]
```

```bash
./run_test.sh [postgres|redshift|...]
```

--------------------------------

### Find Models by Directory

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Retrieves a list of model names from a specified directory. This is a prerequisite for generating documentation for models within that directory.

```jinja2
{# models/marts/generate_mart_documentation.sql #} 

{% set mart_models = codegen.get_models(directory='marts') %}

{{ codegen.generate_model_yaml(
    model_names=mart_models,
    upstream_descriptions=true,
    include_data_types=true
) }}
```

--------------------------------

### Run as dbt run-operation Command with Multiple Arguments

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_source.md

Executes the generate_source macro as a dbt run-operation command with multiple arguments for advanced configuration.

```bash
dbt run-operation generate_source --args '{
  "schema_name": "raw_data",
  "generate_columns": true,
  "include_data_types": true,
  "include_descriptions": true
}'
```

--------------------------------

### Generate YAML using get_models helper

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_model_yaml.md

Dynamically selects models based on directory and prefix using the `get_models` helper, then generates YAML for them.

```jinja
{% set models = codegen.get_models(directory='marts', prefix='fct_') %}
{{ codegen.generate_model_yaml(
    model_names=models
) }}
```

--------------------------------

### Generate Base Model SQL (Quiet)

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

This command generates the base SQL model and redirects the output to a specified file, using the `--quiet` flag to suppress logs.

```bash
dbt --quiet run-operation codegen.generate_base_model --args '{ 
  "source_name": "raw_input",
  "table_name": "customers",
  "materialized": "view"
}' > models/staging/stg_customers.sql
```

--------------------------------

### Run generate_model_yaml for multiple models with upstream descriptions via dbt run-operation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_model_yaml.md

Executes the `generate_model_yaml` macro using `dbt run-operation` for multiple models, enabling upstream description inheritance.

```bash
dbt run-operation generate_model_yaml --args '{
  "model_names": ["customers", "orders"],
  "upstream_descriptions": true
}'
```

--------------------------------

### Using Boolean Flags in dbt Run-Operation

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/README.md

Demonstrates the correct syntax for passing boolean flags to dbt run-operation commands.

```bash
# Correct
--args '{"generate_columns": true}'
```

--------------------------------

### Generate Model YAML with Upstream Descriptions

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/usage-patterns.md

Use this command to generate a model YAML file for a specified model, automatically populating column descriptions by looking at its upstream dependencies.

```bash
dbt run-operation codegen.generate_model_yaml --args '{ 
  "model_names": ["fct_orders"],
  "upstream_descriptions": true
}'
```

--------------------------------

### Basic Usage: Generate Source for a Schema

Source: https://github.com/dbt-labs/dbt-codegen/blob/main/_autodocs/api-reference/generate_source.md

Generates basic YAML source definitions for all tables in the specified schema.

```jinja
{{ codegen.generate_source('raw_data') }}
```