DBT StarRocks

https://github.com/starrocks/dbt-starrocks
Admin
A dbt adapter plugin that enables dbt to work with StarRocks database for data transformation and...
Tokens:38,147
Snippets:299
Trust Score:7.7
License:Apache-2.0
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### profiles.yml example for StarRocksCredentials

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Example `profiles.yml` configuration that is parsed into `StarRocksCredentials`. Shows a typical dev target with async query settings and polling parameters.

```yaml
# profiles.yml — parsed into StarRocksCredentials
my_profile:
  target: dev
  outputs:
    dev:
      type: starrocks
      host: localhost
      port: 9030
      schema: analytics
      username: root
      password: ''
      is_async: true
      async_query_timeout: 3600
      poll_interval: 5
      poll_max_delay: 60
      poll_factor: 1.5
```

--------------------------------

### Install dbt-starrocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/entrypoint.md

Installs the dbt-starrocks adapter plugin from PyPI.

```bash
pip install dbt-starrocks
```

--------------------------------

### Install dev requirements and run StarRocks Docker image

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Install development dependencies and start a StarRocks Docker container with port 9030 exposed for testing.

```bash
pip install -r dev_requirements.txt
docker run -p 9030:9030 starrocks/allin1-ubuntu:3.5-latest
```

--------------------------------

### profiles.yml example for dbt-starrocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/configuration.md

Example profiles.yml configuration for dbt-starrocks. Sets connection credentials and async submission options (is_async, async_query_timeout, poll_interval, poll_max_delay, poll_factor).

```yaml
starrocks:
  target: dev
  outputs:
    dev:
      type: starrocks
      host: localhost
      port: 9030
      catalog: default_catalog
      schema: analytics
      username: root
      password: ''
      is_async: true
      async_query_timeout: 3600
      poll_interval: 5
      poll_max_delay: 60
      poll_factor: 1.5
```

--------------------------------

### Usage example with rendering

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Shows a dbt model using starrocks__concat, starrocks__dateadd, and starrocks__datediff, followed by the rendered SQL output.

```sql
-- models/example.sql
select
    {{ starrocks__concat(['first_name', 'last_name']) }} as full_name,
    {{ starrocks__dateadd('day', 1, 'current_date()') }} as tomorrow,
    {{ starrocks__datediff('current_date()', "date_add('2024-01-01', interval 30 day)", 'day') }} as days_since
```

```sql
select
    concat(first_name, last_name) as full_name,
    date_add(current_date(), interval 1 day) as tomorrow,
    date_diff('day', date_add('2024-01-01', interval 30 day), current_date()) as days_since
```

--------------------------------

### dbt run config examples

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Example dbt run configs for dbt-starrocks showing various materialization strategies, table types, partitioning, and incremental options. Includes view, table, incremental, and materialized_view configurations with different parameters.

```jinja
{{ config(materialized='view') }}
{{ config(materialized='view', on_view_exists='replace') }}
{{ config(materialized='table', engine='OLAP', buckets=32, distributed_by=['id']) }}
{{ config(materialized='table', indexs=[{ 'columns': 'idx_column' }]) }}
{{ config(materialized='table', partition_by=['date_trunc("day", first_order)'], partition_type='Expr') }}
{{ config(materialized='table', table_type='PRIMARY', keys=['customer_id'], order_by=['first_name', 'last_name'] }}
{{ config(materialized='incremental', table_type='PRIMARY', engine='OLAP', buckets=32, distributed_by=['id']) }}
{{ config(materialized='incremental', partition_by=['my_partition_key'], partition_type='Expr', incremental_strategy='dynamic_overwrite') }}
{{ config(materialized='incremental', partition_by=['my_partition_key'], partition_type='Expr', incremental_strategy='microbatch', event_time='report_day', begin='2025-01-01', lookback=1, batch_size='day') }}
{{ config(materialized='incremental', partition_by=['my_partition_key'], partition_type='Expr', incremental_strategy='microbatch', event_time='report_day', begin='2025-01-01', lookback=1, batch_size='day', microbatch_use_dynamic_overwrite=true) }}
{{ config(materialized='materialized_view') }}
{{ config(materialized='materialized_view', properties={"storage_medium":"SSD"}) }}
{{ config(materialized='materialized_view', refresh_method="ASYNC START('2022-09-01 10:00:00') EVERY (interval 1 day)") }}
```

--------------------------------

### Sample dbt profile for StarRocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Example profile that exercises the open() retry path; if the target schema does not exist, dbt creates it automatically. The version field skips the current_version() query.

```yaml
outputs:
  dev:
    type: starrocks
    host: localhost
    port: 9030
    catalog: default_catalog
    schema: analytics
    username: root
    password: ''
    version: 3.1.0   # skips the current_version() query
    use_pure: "true"
```

--------------------------------

### profiles.yml with async ETL settings

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Example `profiles.yml` configuration enabling async ETL submission and custom polling parameters. `is_async: true` enables automatic submission of suitable ETL tasks, and `async_query_timeout` controls the query timeout for submitted tasks.

```yaml
my_profile:
  target: dev
  outputs:
    dev:
      type: starrocks
      host: host
      port: 9030
      schema: schema
      username: username
      password: password
      is_async: true
      async_query_timeout: 3600 # 1 hour
      poll_interval: 5
      poll_max_delay: 60
      poll_factor: 1.5
```

--------------------------------

### External catalog gating example

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Shows how to conditionally qualify information_schema with the external catalog when not using an internal catalog.

```jinja
{% if not starrocks__is_internal_catalog() %}
    -- external catalog: qualify information_schema with the catalog
    select * from `{{ starrocks__external_catalog(database) }}`.information_schema.tables
{% endif %}
```

--------------------------------

### Install dbt-starrocks via pip

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Install the dbt-starrocks plugin using pip. Requires StarRocks version 2.5.0 or higher; version 3.4.x is recommended. StarRocks versions 2.4 and below are no longer supported.

```shell
$ pip install dbt-starrocks
```

--------------------------------

### profiles.yml entry for dbt-starrocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Example profile configuration for the dbt-starrocks adapter. Defines the target 'dev' with connection details such as host, port, catalog, schema, and credentials. Required options include type, host, port, schema, username, and password; catalog is optional.

```yaml
starrocks:
  target: dev
  outputs:
    dev:
      type: starrocks
      host: localhost
      port: 9030
      catalog: default_catalog
      schema: analytics
      username: your_starrocks_username
      password: your_starrocks_password

```

--------------------------------

### starrocks__partition_by usage with Expr partition

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-adapters.md

Shows how to use the `starrocks__partition_by` macro to render a partition clause with an Expr partition type. The example configures a table materialization with a date-trunc expression and calls the macro directly, producing `PARTITION BY date_trunc('day', order_ts)`.

```sql
-- Partition a table with a date-trunc expression (Expr partition)
{{ config(materialized='table', partition_by=["date_trunc('day', order_ts)"], partition_type='Expr') }}

{{ starrocks__partition_by('Expr', ["date_trunc('day', order_ts)"], none) }}
-- -> PARTITION BY date_trunc('day', order_ts)
```

--------------------------------

### Invalid incremental strategy error example

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

Shows an invalid configuration that triggers a compile-time error. The 'microbatch' incremental strategy requires the 'event_time' configuration to be set.

```sql
{{ config(materialized='incremental', incremental_strategy='microbatch') }}
-- raises: The 'microbatch' incremental strategy requires the 'event_time' configuration to be set.
```

--------------------------------

### Using StarRocksColumn typification helpers in a model

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/column.md

Conceptual Jinja example showing how to use the typification helpers (is_integer, is_string, is_float) and the quoted property within a dbt model. It iterates over columns from a relation and demonstrates the rendered output of col.quoted.

```jinja
-- Within a model, via the adapter's Python path (conceptual)
{%- set cols = adapter.get_columns_in_relation(ref('events')) -%}
{%- for col in cols -%}
    {%- if col.is_integer() %}
        -- int-family column: {{ col.quoted }} renders as `my_col`
    {%- elif col.is_string() %}
        -- string-family column
    {%- elif col.is_float() %}
        -- float-family column
    {%- endif -%}
{%- endfor -%}
```

--------------------------------

### StarRocksColumn __repr__ output

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/column.md

Example of the __repr__ output for a StarRocksColumn instance, which appears in dbt's debug logs when column lists are printed.

```text
<StarRocksColumn name (varchar)>
```

--------------------------------

### StarRocksConfig AdapterConfig class

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/types.md

Typed AdapterConfig subclass for StarRocks table configuration. All fields default to None, meaning 'unset' and macros apply their own defaults. Installed as StarRocksAdapter.AdapterSpecificConfigs and used by OLAP/external table macros.

```python
class StarRocksConfig(AdapterConfig):
    engine: Optional[str] = None
    table_type: Optional[str] = None  # DUPLICATE/PRIMARY/UNIQUE/AGGREGATE
    keys: Optional[List[str]] = None
    partition_by: Optional[List[str]] = None
    partition_by_init: Optional[List[str]] = None
    distributed_by: Optional[List[str]] = None
    buckets: Optional[int] = None
    properties: Optional[Dict[str, str]] = None
    microbatch_use_dynamic_overwrite: Optional[bool] = None
```

--------------------------------

### starrocks__datediff macro definition

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Defines the StarRocks datediff macro. Note the argument order: StarRocks' date_diff(unit, start, end) is emitted with second_date as start and first_date as end.

```jinja
{% macro starrocks__datediff(first_date, second_date, datepart) -%}

    date_diff(
        '{{ datepart }}',
        {{ second_date }},
        {{ first_date }}
        )

{%- endmacro %}
```

--------------------------------

### starrocks__datediff

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Macro that returns a date_diff expression for StarRocks, with arguments ordered as date_diff(unit, start, end).

```APIDOC
## `starrocks__datediff`

### Description
Returns a `date_diff('<datepart>', <second_date>, <first_date>)` expression. Note the argument order: StarRocks' `date_diff(unit, start, end)` is emitted with `second_date` as start and `first_date` as end.

### Parameters
- **first_date** (any) - Required - Later date expression
- **second_date** (any) - Required - Earlier date expression
- **datepart** (string) - Required - Date part (e.g., 'day', 'month')

### Example
```sql
{{ starrocks__datediff('current_date()', "date_add('2024-01-01', interval 30 day)", 'day') }}
```

Renders:
```sql
date_diff('day', date_add('2024-01-01', interval 30 day), current_date())
```
```

--------------------------------

### dbt seed column typing with string_type

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/column.md

Example of how StarRocksColumn.string_type is used during dbt seed to emit varchar(n) for truncated strings. Shows the macro call and the rendered create table statement with a varchar(255) column.

```jinja
-- dbt seed, seeds/sample.csv with column "name"
{{ starrocks__create_csv_table(model, agate_table) }}
-- renders: create table `dbt`.`sample` ( `name` varchar(255) ) ENGINE = OLAP ...
```

--------------------------------

### Configure StarRocks table model with version-gated macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Shows how to configure a StarRocks table model with PRIMARY table type, keys, distribution, and buckets. Also demonstrates the version-gated macro `adapter.is_before_version('3.1.0')` to enforce `distributed_by` for older versions.

```sql
-- models/orders.sql
{{ config(materialized='table',
          table_type='PRIMARY', keys=['id'],
          distributed_by=['id'], buckets=32) }}

{% if adapter.is_before_version('3.1.0') %}
    -- requires distributed_by: enforced by the macro layer
{% endif %}

select id, amount from source_orders
```

--------------------------------

### add_begin_query method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Adds an empty begin query since StarRocks transactions are not begun explicitly.

```python
def add_begin_query(self):
    return self.add_query("", auto_begin=False)
```

--------------------------------

### Version helpers (is_before_version, current_version)

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Version helper methods exposed to Jinja via the @available decorator. is_before_version returns True when the parsed server version tuple is numerically below the given version, False when no live connection exists. current_version returns a "major.minor.patch" string of conn.handle.server_version, or 'UNKNOWN' when no connection or version is (999, 999, 999).

```python
@available
def is_before_version(self, version: str) -> bool:
    conn = self.connections.get_if_exists()
    if conn:
        server_version = conn.handle.server_version
        server_version_tuple = tuple(server_version)
        version_detail_tuple = tuple(
            int(part) for part in version.split(".") if part.isdigit())
        if version_detail_tuple > server_version_tuple:
            return True
    return False

@available
def current_version(self):
    conn = self.connections.get_if_exists()
    if conn:
        server_version = conn.handle.server_version
        if server_version != (999, 999, 999):
            return "{}.{}.{}".format(server_version[0], server_version[1], server_version[2])
    return 'UNKNOWN'
```

--------------------------------

### Adapter Version Helpers and Incremental Strategies

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Version helper methods exposed to Jinja macros via the @available decorator, and the valid incremental strategies method.

```APIDOC
## is_before_version

### Description
Determines whether the current server version is numerically below a given version string. Returns False when no live connection exists.

### Method
N/A (Python method)

### Endpoint
N/A

### Parameters
- **version** (str) - Required - The version string to compare against, e.g., "2.5.0".

### Returns
- **bool** - True when the parsed server version tuple is numerically below `version`; False when no live connection exists.

### Example
```python
adapter.is_before_version("2.5.0")
```
```

```APIDOC
## current_version

### Description
Returns the current server version as a "major.minor.patch" string. Returns 'UNKNOWN' when no connection exists or the version is (999, 999, 999).

### Method
N/A (Python method)

### Endpoint
N/A

### Parameters
None

### Returns
- **str** - The server version string in "major.minor.patch" format, or 'UNKNOWN'.

### Example
```python
adapter.current_version()
```
```

```APIDOC
## valid_incremental_strategies

### Description
Returns the list of incremental strategies supported by the adapter.

### Method
N/A (Python method)

### Endpoint
N/A

### Parameters
None

### Returns
- **list** - A list of strings: ["default", "insert_overwrite", "dynamic_overwrite", "microbatch"].

### Example
```python
adapter.valid_incremental_strategies()
```
```

--------------------------------

### valid_incremental_strategies

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Returns the list of incremental strategies supported by the adapter: default, insert_overwrite, dynamic_overwrite, and microbatch.

```python
@override
def valid_incremental_strategies(self):
    return ["default", "insert_overwrite", "dynamic_overwrite", "microbatch"]
```

--------------------------------

### Configure external catalog source in sources.yml

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/configuration.md

Configure an external catalog source in sources.yml by folding the catalog into the schema. Reference it as {{ source('external_example', 'hive_table_name') }}. The catalog itself is created server-side with CREATE EXTERNAL CATALOG, not by the adapter.

```yaml
sources:
  - name: external_example
    schema: hive_catalog.hive_db
    tables:
      - name: hive_table_name

```

--------------------------------

### Submit task with query_timeout hint

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Shows the SQL template used to submit a task with a configurable query timeout. The `async_query_timeout` value from `profiles.yml` is injected into the `query_timeout` hint.

```sql
submit /*+set_var(query_timeout={async_query_timeout})*/ task ...
```

--------------------------------

### Catalog introspection methods (get_catalog, _catalog_filter_table, _get_one_catalog, _catalog_filter_schemas)

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Catalog introspection methods. get_catalog(manifest, used_schemas) runs catalog queries concurrently (executor(self.config), tpe.submit_connected), returning (catalogs, exceptions). It raises dbt.exceptions.CompilationError when more than one database is present in the schema map. _get_one_catalog likewise raises CompilationError when len(schemas) != 1. Internal helper _catalog_filter_schemas matches rows where (table_database, table_schema.lower()) is in the used-schema set with table_database pinned to None.

```python
def get_catalog(self, manifest, used_schemas):
    ...

@classmethod
def _catalog_filter_table(
    cls, table: "agate.Table", used_schemas: FrozenSet[Tuple[str, str]]
) -> agate.Table:
    ...

def _get_one_catalog(
    self,
    information_schema: InformationSchema,
    schemas: Set[str],
    used_schemas: FrozenSet[Tuple[str, str]],
) -> agate.Table:
    ...

def _catalog_filter_schemas(used_schemas: FrozenSet[Tuple[str, str]]) -> Callable[[agate.Row], bool]:
    ...
```

--------------------------------

### Table materialization with OLAP engine and primary key

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

Creates a table materialization with an OLAP engine, primary key type, range partitioning, bucketing, and replication properties. Use for large fact tables that require efficient querying and data management.

```sql
-- models/orders.sql — partitioned, bucketed OLAP table with primary key
{{ config(materialized='table',
          engine='OLAP', table_type='PRIMARY', keys=['id'],
          partition_by=['dt'], partition_type='RANGE', partition_by_init=["PARTITION p0 VALUES LESS THAN ('2024-01-01')"],
          distributed_by=['id'], buckets=32,
          properties={"replication_num": "1"}) }}
select id, dt, amount from raw_orders
```

--------------------------------

### get_response class method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Builds an AdapterResponse from the cursor, always returning code 'SUCCESS' with rows_affected from cursor.rowcount.

```python
@classmethod
def get_response(cls, cursor) -> AdapterResponse:
```

--------------------------------

### Run dbt model with async mode

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Runs the dbt model with async mode enabled. The CREATE TABLE ... SELECT / INSERT statements are wrapped in a submit task and polled.

```bash
dbt run --select orders     # CREATE TABLE ... SELECT / INSERT statements are wrapped in submit task and polled
```

--------------------------------

### Run dbt debug and dbt run with starrocks profile

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/entrypoint.md

Commands to validate the connection and run models using the starrocks profile.

```bash
dbt debug --profile starrocks   # validates the connection
dbt run --profile starrocks     # runs models
```

--------------------------------

### Incremental dynamic overwrite with expression partitioning

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

Uses the dynamic_overwrite incremental strategy with expression partitioning, available on StarRocks 3.4 and later. Overwrites only the affected partitions based on the partition_by column.

```sql
-- models/daily_orders.sql — dynamic overwrite on StarRocks >= 3.4
{{ config(materialized='incremental', incremental_strategy='dynamic_overwrite',
          partition_by=['dt'], partition_type='Expr') }}
select dt, sum(amount) as amount from raw_orders group by dt
```

--------------------------------

### Define AdapterPlugin for dbt-starrocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/entrypoint.md

Defines the AdapterPlugin instance that dbt auto-discovers at project load. It wires the StarRocksAdapter, StarRocksCredentials, and the bundled include package path.

```python
from dbt.adapters.starrocks.connections import StarRocksCredentials
from dbt.adapters.starrocks.impl import StarRocksAdapter

from dbt.adapters.base import AdapterPlugin
from dbt.include import starrocks


Plugin = AdapterPlugin(
    adapter=StarRocksAdapter,
    credentials=StarRocksCredentials,
    include_path=starrocks.PACKAGE_PATH)
```

--------------------------------

### Configure model to build in external catalog

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Sets catalog and database in a model's config to build it inside an external catalog. Both are required together; catalog defaults to the profile's catalog. The table name comes from the model's file name or alias, not from database.

```jinja
{{ config(materialized='table', catalog='iceberg_catalog', database='iceberg_db') }}
```

```jinja
{{ config(materialized='incremental', incremental_strategy='dynamic_overwrite',
          catalog='iceberg_catalog', database='iceberg_db',
          partition_by=['day(`modified_at`)']) }}
```

--------------------------------

### starrocks__catalog_for macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Returns the catalog for the current model: model-level config('catalog') takes precedence, otherwise uses target.catalog (defaults to 'default_catalog').

```jinja
{% macro starrocks__catalog_for() -%}
  {{ return(config.get('catalog', target.catalog)) }}
{%- endmacro %}
```

--------------------------------

### starrocks__snapshot_hash_arguments macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

Builds an `md5(concat_ws('|', coalesce(cast({arg} as char), '') ...))` expression over the argument list.

```jinja
{% macro starrocks__snapshot_hash_arguments(args) -%}
```

--------------------------------

### Run functional tests with pytest

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Run the functional test suite for the dbt-starrocks adapter.

```bash
python3 -m pytest tests/functional
```

--------------------------------

### open class method signature

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Class method that opens a mysql.connector connection, sets connection.state to 'open', and handles retries and version parsing.

```python
@classmethod
def open(cls, connection):
```

--------------------------------

### render method for relation string

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/relation.md

Renders the relation as a string. When database is set, produces a fully qualified catalog.schema.table form; otherwise delegates to the base render. Raises DbtRuntimeError if both include_policy.database and include_policy.schema are true.

```python
def render(self):
    if self.database is not None:
        return "{catalog}.{database}.{table}".format(
            catalog=self.quoted(self.database),
            database=self.quoted(self.schema),
            table=self.quoted(self.identifier)
        )
    if self.include_policy.database and self.include_policy.schema:
        raise DbtRuntimeError(
            "Got a StarRocks relation with schema and database set to include, but only one can be set"
        )
    return super().render()
```

--------------------------------

### starrocks__snapshot_string_as_time macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

Wraps a timestamp string in `str_to_date('...', '%Y-%m-%d %T')`.

```jinja
{% macro starrocks__snapshot_string_as_time(timestamp) -%}
    {%- set result = "str_to_date('" ~ timestamp ~ "', '%Y-%m-%d %T')" -%}
```

--------------------------------

### get_credentials class method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Identity pass-through for credentials.

```python
@classmethod
def get_credentials(cls, credentials):
    return credentials
```

--------------------------------

### starrocks__generate_database_name macro definition

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Defines the StarRocks generate_database_name macro. Always returns None because StarRocks has no separate database concept (database ≡ schema).

```jinja
{% macro starrocks__generate_database_name(custom_database_name=none, node=none) -%}
  {% do return(None) %}
{%- endmacro %}
```

--------------------------------

### profiles.yml error case: database must equal schema

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Error case showing that `database` must match `schema`; setting a different value raises `DbtRuntimeError`.

```yaml
outputs:
  dev:
    type: starrocks
    host: localhost
    schema: analytics
    database: other_db   # raises DbtRuntimeError: database must equal schema
```

--------------------------------

### StarRocksAdapter class definition

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Defines the StarRocksAdapter class, setting the connection manager, relation, config, and column implementations, plus a class-level dictionary for tracking in-flight async ETL tasks.

```python
class StarRocksAdapter(SQLAdapter):
    ConnectionManager = StarRocksConnectionManager
    Relation = StarRocksRelation
    AdapterSpecificConfigs = StarRocksConfig
    Column = StarRocksColumn

    _running_tasks: Dict[str, str] = {}
```

--------------------------------

### StarRocks adapter module-level exports

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Module-level exports from the StarRocks adapter implementation. Defines the SQLQueryResult type alias, the default catalog name, SQL templates for async task submission and polling, and an internal helper for external catalog detection.

```python
SQLQueryResult: TypeAlias = Tuple[AdapterResponse, "agate.Table"]

DEFAULT_CATALOG = "default_catalog"

SUBMIT_TASK_TEMPLATE = "submit /*+set_var(query_timeout={timeout})*/ task {task_id} as {sql}"
POLL_TASK_TEMPLATE = "select * from information_schema.task_runs where task_name = '{task_id}'"

def _external_catalog(database: Optional[str]) -> Optional[str]: ...
```

--------------------------------

### check_schema_exists method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Runs starrocks__list_schemas and returns whether schema is among the first column values. The starrocks__check_schema_exists macro is a no-op; this Python method is the real implementation.

```python
def check_schema_exists(self, database, schema):
    results = self.execute_macro(
        LIST_SCHEMAS_MACRO_NAME, kwargs={"database": database}
    )
    exists = True if schema in [row[0] for row in results] else False
    return exists
```

--------------------------------

### snapshot_check_all_get_existing_columns macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

Compares `get_columns_in_query(node['compiled_sql'])` against the existing target's columns; returns `[column_added, intersection]` where `column_added` is true when a query column is missing from the target.

```jinja
{% macro snapshot_check_all_get_existing_columns(node, target_exists) -%}
```

--------------------------------

### Configure profiles.yml for StarRocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/entrypoint.md

Minimal profiles.yml configuration for the starrocks profile type, used by dbt to connect to StarRocks via the MySQL protocol.

```yaml
# profiles.yml
starrocks:
  target: dev
  outputs:
    dev:
      type: starrocks
      host: localhost
      port: 9030
      catalog: default_catalog
      schema: analytics
      username: your_starrocks_username
      password: your_starrocks_password
```

--------------------------------

### starrocks__snapshot_merge_sql_update macro and SQL

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

StarRocks-style multi-table update for snapshot merges. The macro signature is followed by the SQL it generates.

```jinja
{% macro starrocks__snapshot_merge_sql_update(target, source, insert_cols) -%}
```

```sql
update {target}, (select dbt_scd_id, dbt_change_type, dbt_valid_to from {source}) as DBT_INTERNAL_SOURCE
set {target}.dbt_valid_to = DBT_INTERNAL_SOURCE.dbt_valid_to
where DBT_INTERNAL_SOURCE.dbt_scd_id = {target}.dbt_scd_id
and DBT_INTERNAL_SOURCE.dbt_change_type = 'update'
and {target}.dbt_valid_to is null
```

--------------------------------

### Define external source in sources.yaml

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Declares a dbt source named external_example that maps to the hive_catalog.hive_db schema and exposes the hive_table_name table. Use this source with the source() macro in models.

```yaml
sources:
  - name: external_example
    schema: hive_catalog.hive_db
    tables:
      - name: hive_table_name
```

--------------------------------

### StarRocksCredentials __post_init__ validation

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Validation hook that raises `dbt_common.exceptions.DbtRuntimeError` when `database` is set and differs from `schema`, because StarRocks treats them as the same thing.

```python
def __post_init__(self):
    # starrocks classifies database and schema as the same thing
    if (
        self.database is not None and
        self.database != self.schema
    ):
        raise dbt_common.exceptions.DbtRuntimeError(
            f"    schema: {self.schema} \n"
            f"    database: {self.database} \n"
            f"On StarRocks, database must be omitted or have the same value as"
            f" schema."
        )
```

--------------------------------

### cancel_open_connections method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Logs 'Canceling N running StarRocks tasks...', calls _cancel_task(task_id) for every entry in _running_tasks (logging each), then delegates to super().cancel_open_connections(). Called when dbt is interrupted so orphan submit task jobs are dropped.

```python
def cancel_open_connections(self):
```

--------------------------------

### starrocks__snapshot_merge_sql_insert macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-materializations.md

Generates an `insert into {target} ({cols}) select DBT_INTERNAL_SOURCE.{col} ... from {source} as DBT_INTERNAL_SOURCE where DBT_INTERNAL_SOURCE.dbt_change_type = 'insert'` statement.

```jinja
{% macro starrocks__snapshot_merge_sql_insert(target, source, insert_cols) -%}
```

--------------------------------

### list_relations_without_caching method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Runs starrocks__list_relations_without_caching. Each result row must have exactly 4 values (database, name, schema, type_info); otherwise raises dbt.exceptions.DbtRuntimeError(f"Invalid value from 'show table extended ...', got {len(row)} values, expected 4"). Relations are built with database=_external_catalog(_database).

```python
def list_relations_without_caching(
    self, schema_relation: StarRocksRelation
) -> List[StarRocksRelation]:
```

--------------------------------

### init_type_map method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/relation.md

Populates the module-level type_map with exact column types from a DESC table. Expects rows where row[0] is the column name and row[1] is the type.

```python
def init_type_map(self, desc_table):
    for row in desc_table:
        # name -> type
        type_map[row[0]] = row[1]
```

--------------------------------

### dbt debug command with error output

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Command to run dbt debug with a profile; a wrong password produces a DbtDatabaseError/ConnectionError.

```bash
dbt debug --profile my_profile
# 01:00:00  Database error while trying to connect: ... (mysql.connector errors -> ConnectionError)
```

--------------------------------

### Set PACKAGE_PATH in dbt.include.starrocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/entrypoint.md

Sets PACKAGE_PATH to the directory of the include package, which contains dbt_project.yml, profile_template.yml, sample_profiles.yml, and the macros directory.

```python
import os
PACKAGE_PATH = os.path.dirname(__file__)
```

--------------------------------

### Enable async mode in profiles.yml

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Enables async mode (submit + poll) via `profiles.yml` without model changes. Sets `is_async: true` and configures polling parameters: `async_query_timeout`, `poll_interval`, `poll_max_delay`, and `poll_factor`.

```yaml
outputs:
  dev:
    type: starrocks
    host: localhost
    schema: analytics
    is_async: true
    async_query_timeout: 3600
    poll_interval: 5
    poll_max_delay: 60
    poll_factor: 1.5
```

--------------------------------

### Changelog entry template

Source: https://github.com/starrocks/dbt-starrocks/blob/main/CONTRIBUTING.md

Template for adding a changelog entry under the [Unreleased] section. Place your change under the appropriate category (e.g., Fixed, Added).

```markdown
## [Unreleased]

### Fixed
- Your PR here!

### Added

```

--------------------------------

### _parse_version: parse current_version() into (major, minor, patch)

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Parses the string returned by `select current_version()` into `(major, minor, patch)`. Two independent checks: if the value contains `-`, the part before the hyphen is used; if it contains a space, the part before the space is used (the space branch wins when both are present). Falls back to `(999, 999, 999)` when the resulting segment is not three dot-separated parts.

```python
def _parse_version(result):
    default_version = (999, 999, 999)
    ...
    return default_version
```

--------------------------------

### Create Hive external catalog in StarRocks

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Creates an external catalog named hive_catalog pointing to a Hive metastore. Required before dbt can read from the catalog. Replace the thrift URI with your metastore address.

```mysql
CREATE EXTERNAL CATALOG `hive_catalog`
PROPERTIES (
    "hive.metastore.uris"  =  "thrift://127.0.0.1:8087",
    "type"="hive"
);
```

--------------------------------

### starrocks__generate_database_name

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Macro that always returns None, as StarRocks has no separate database concept (database ≡ schema).

```APIDOC
## `starrocks__generate_database_name`

### Description
Always returns `None`: StarRocks has no separate database concept (`database` ≡ `schema`), and internal-catalog relations carry `database = None`, which the adapter's include policy and `_external_catalog` rely on.

### Parameters
- **custom_database_name** (any) - Optional - dbt custom-database value (ignored)
- **node** (any) - Optional - Model node (ignored)

### Returns
`None`
```

--------------------------------

### dbt seed properties complete configuration

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Complete YAML configuration for dbt seed properties in dbt-starrocks. Covers materialized, engine, keys, table_type, distributed_by, buckets, indexs, partition_by, partition_by_init, order_by, partition_type, properties, refresh_method, on_view_exists, and incremental options. Includes comments for supported values and constraints.

```yaml
models:
  materialized: table                   // table, view, materialized_view or incremental
  engine: 'OLAP'
  keys: ['id', 'name', 'some_date']
  table_type: 'PRIMARY'                 // PRIMARY or DUPLICATE or UNIQUE
  distributed_by: ['id']
  buckets: 3                            // leave empty for auto bucketing
  indexs=[{ 'columns': 'idx_column' }]
  partition_by: ['some_date']
  partition_by_init: ["PARTITION p1 VALUES [('1971-01-01 00:00:00'), ('1991-01-01 00:00:00')),PARTITION p1972 VALUES [('1991-01-01 00:00:00'), ('1999-01-01 00:00:00'))"]
  // RANGE, LIST, or Expr partition types should be used in conjunction with partition_by configuration
  // Expr partition type requires an expression (e.g., date_trunc) specified in partition_by
  order_by: ['some_column']             // only for PRIMARY table_type
  partition_type: 'RANGE'               // RANGE or LIST or Expr Need to be used in combination with partition_by configuration
  properties: {"replication_num":"1", "in_memory": "true"}
  refresh_method: 'async'               // only for materialized view default manual
  on_view_exists: 'replace'             // only for view: use CREATE OR REPLACE VIEW instead of DROP + CREATE

  // For 'materialized=incremental' in version >= 3.4
  incremental_strategy: 'dynamic_overwrite' // Supported values: ['default', 'insert_overwrite', 'dynamic_overwrite']

  // For 'materialized=incremental' and 'incremental_strategy=microbatch'
  event_time: 'some_timestamp_column'     // The column name of the event time
  begin: '2025-01-01'                     // The start time of the incremental data
  lookback: 1                             // The lookback time of the each incremental run
  batch_size: 'day'                       // The batch size. Supported values ['year', 'month', 'day', 'hour']
  microbatch_use_dynamic_overwrite: true  // Whether to use dynamic_overwrite in version >= 3.4
```

--------------------------------

### Catch validation errors with StarRocksCredentials

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/errors.md

Use this pattern in Python integration tests to catch database/schema mismatch validation errors. Import dbt_common.exceptions and dbt.exceptions, then wrap StarRocksCredentials construction in a try/except for DbtRuntimeError.

```python
import dbt_common.exceptions
import dbt.exceptions
from dbt.adapters.starrocks.connections import StarRocksCredentials

# Validation error — database/schema mismatch
try:
    StarRocksCredentials(database="a", schema="b")
except dbt_common.exceptions.DbtRuntimeError:
    pass
```

--------------------------------

### External catalog write config for Iceberg model

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/relation.md

Configures a dbt model to write to an external Iceberg catalog using the `catalog` and `database` config options. The relation renders in three-part form (`` `iceberg_catalog`.`iceberg_db`.`iceberg_orders` ``) via `render()`.

```sql
-- models/iceberg_orders.sql — external catalog write (config in Jinja)
{{ config(materialized='table', catalog='iceberg_catalog', database='iceberg_db') }}

select * from {{ source('external_example', 'hive_table_name') }}
```

--------------------------------

### _is_submittable_etl static method signature

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Signature for the static method that checks if SQL is submittable as an async ETL task. It normalizes whitespace and lower-cases the SQL, then matches against patterns for CREATE TABLE ... SELECT, INSERT INTO/OVERWRITE, and CACHE SELECT.

```python
@staticmethod
def _is_submittable_etl(sql: str) -> bool:
```

--------------------------------

### exception_handler context manager

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Context manager that wraps statement execution, handling mysql.connector.DatabaseError and other exceptions with rollback and appropriate dbt error wrapping.

```python
@contextmanager
def exception_handler(self, sql):
```

--------------------------------

### StarRocksCredentials dataclass definition

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Dataclass that holds the `profiles.yml` connection block for a StarRocks output. Every key under an `outputs.<name>` entry is passed as a keyword argument to this constructor; unknown keys are accepted and become attributes because `__init__(**kwargs)` sets them dynamically.

```python
@dataclass
class StarRocksCredentials(Credentials):
    host: Optional[str] = None
    port: Optional[int] = None
    catalog: Optional[str] = 'default_catalog'
    database: Optional[str] = None
    schema: Optional[str] = None
    username: Optional[str] = None
    password: Optional[str] = None
    charset: Optional[str] = None
    version: Optional[str] = None
    use_pure: Optional[str] = None
    is_async: Optional[bool] = False
    async_query_timeout: Optional[int] = 300
    poll_interval: Optional[int] = 1
    poll_max_delay: Optional[int] = 600
    poll_factor: Optional[float] = 2.0
    auth_plugin: Optional[str] = ''

    def __init__(self, **kwargs):
        for k, v in kwargs.items():
            setattr(self, k, v)
```

--------------------------------

### Reference external source with source() macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/README.md

Returns the fully qualified name for the external table so it can be referenced in dbt models. The macro arguments match the source and table names defined in sources.yaml.

```jinja
{{ source('external_example', 'hive_table_name') }}
```

--------------------------------

### _execute_async_task method signature

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Signature for the method that submits an async task using SUBMIT_TASK_TEMPLATE with a fresh uuid4 task id and async_query_timeout from credentials, then polls for completion. The _running_tasks entry is always removed in a finally block.

```python
def _execute_async_task(self, sql: str, auto_begin: bool = False, fetch: bool = False, limit: Optional[int] = None) -> SQLQueryResult:
```

--------------------------------

### Type conversion class methods (date_function, convert_datetime_type, convert_text_type, convert_number_type)

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Type conversion class methods for StarRocks. date_function returns "current_date()" for dbt run timestamps. convert_datetime_type always returns "datetime", convert_text_type always returns "string", and convert_number_type returns "double" when the column's max decimal precision is non-zero, else "bigint" to avoid the Postgres float8 alias StarRocks rejects.

```python
@classmethod
def date_function(cls) -> str:
    return "current_date()"

@classmethod
def convert_datetime_type(cls, agate_table: agate.Table, col_idx: int) -> str:
    return "datetime"

@classmethod
def convert_text_type(cls, agate_table: agate.Table, col_idx: int) -> str:
    return "string"

@classmethod
def convert_number_type(cls, agate_table: agate.Table, col_idx: int) -> str:
    decimals = agate_table.aggregate(agate.MaxPrecision(col_idx))
    return "double" if decimals else "bigint"
```

--------------------------------

### Render relation with conflicting include policies

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/relation.md

Shows the failure when a relation has both `include_policy.database` and `include_policy.schema` set to true. `render()` raises a `DbtRuntimeError` with the message "Got a StarRocks relation with schema and database set to include...".

```jinja
-- renders raise: DbtRuntimeError "Got a StarRocks relation with schema and database set to include..."
{{ relation.render() }}   # when relation.include_policy.database and relation.include_policy.schema are both true
```

--------------------------------

### starrocks__concat macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-utils.md

Defines the StarRocks adapter macro for the dbt concat built-in. It joins the provided fields with commas and wraps them in a concat() call. The fields parameter is an iterable of SQL expressions.

```jinja
{% macro starrocks__concat(fields) -%}
    concat({{ fields|join(', ') }})
{%- endmacro %}
```

--------------------------------

### get_relation method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Nulls out the database component for internal-catalog relations (they are addressed as schema.identifier); external-catalog names survive so relations can be looked up by catalog.

```python
def get_relation(self, database: Optional[str], schema: str, identifier: str):
    if (
        _external_catalog(database) is None
        and not self.Relation.get_default_include_policy().database
    ):
        database = None
    return super().get_relation(database, schema, identifier)
```

--------------------------------

### _poll_for_complete_task method signature

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Signature for the method that polls information_schema.task_runs until the task finishes. It handles SUCCESS, FAILED, MERGED, and unknown states, and uses _compute_poll_delay for retries.

```python
def _poll_for_complete_task(self, task_id: str) -> SQLQueryResult:
```

--------------------------------

### StarRocksConnectionManager class definition with TYPE_CODE_TO_NAME mapping

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/connections.md

Defines the StarRocksConnectionManager class with TYPE='starrocks' and a mapping from MySQL protocol field-type codes to StarRocks SQL type names.

```python
class StarRocksConnectionManager(SQLConnectionManager):
    TYPE = 'starrocks'
    TYPE_CODE_TO_NAME = {
        FieldType.DECIMAL: "decimal",
        FieldType.NEWDECIMAL: "decimal",
        FieldType.TINY: "tinyint",
        FieldType.SHORT: "smallint",
        FieldType.LONG: "int",
        FieldType.INT24: "int",
        FieldType.LONGLONG: "bigint",
        FieldType.FLOAT: "float",
        FieldType.DOUBLE: "double",
        FieldType.DATE: "date",
        FieldType.DATETIME: "datetime",
        FieldType.TIMESTAMP: "datetime",
        FieldType.TIME: "varchar",
        FieldType.YEAR: "smallint",
        FieldType.VARCHAR: "varchar",
        FieldType.VAR_STRING: "varchar",
        FieldType.STRING: "varchar",
        FieldType.BLOB: "varbinary",
        FieldType.BIT: "tinyint",
        FieldType.JSON: "json",
    }
```

--------------------------------

### _compute_poll_delay static method

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/adapter.md

Computes the delay in seconds before the next poll, using exponential backoff capped at poll_max_delay. Defaults: poll_interval=1, poll_factor=2.0, poll_max_delay=600.

```python
@staticmethod
def _compute_poll_delay(attempt: int, poll_interval: int = 1, poll_factor: float = 2.0, poll_max_delay: int = 600) -> float:
    return min(poll_max_delay, poll_interval * (poll_factor ** attempt))
```

--------------------------------

### starrocks__current_timestamp macro

Source: https://github.com/starrocks/dbt-starrocks/blob/main/_autodocs/api-reference/macros-adapters.md

Macro that returns the SQL string `current_timestamp()` for freshness checks. It is dispatched by dbt and can be overridden in model or package code.

```jinja
{% macro starrocks__current_timestamp() -%}
  current_timestamp()
{%- endmacro %}
```
Tokens:
Raw