DataOS Vulcan

https://tmdc-io.github.io/vulcan-book
Admin
DataOS Vulcan is a complete stack for building data products, providing visibility, built-in...
Tokens:268,361
Snippets:2,378
Trust Score:8.3
Update:8 months ago
Show doc for...
Tokens:
Raw
### Set Up Vulcan Infrastructure Services

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Initializes and starts the core infrastructure services for Vulcan, including a PostgreSQL statestore and MinIO object storage. This setup is crucial for Vulcan's operation.

```bash
make setup
```

```bash
docker network create vulcan
docker compose -f docker/docker-compose.infra.yml up -d
docker compose -f docker/docker-compose.warehouse.yml up -d
```

--------------------------------

### Start Vulcan API Services

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Starts the Vulcan API server and the transpiler service, which are necessary for querying the semantic model and converting queries to SQL. These services are typically run after infrastructure setup.

```bash
make vulcan-up
```

```bash
docker compose -f docker/docker-compose.vulcan.yml up -d
```

--------------------------------

### Verify Docker Installation (Shell)

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

This snippet checks if Docker and Docker Compose are installed and running on your system by displaying their version numbers. It's a crucial first step to ensure your environment is ready for Vulcan project setup.

```shell
docker --version
docker compose version

```

--------------------------------

### Navigate to Vulcan Project Directory

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Changes the current directory to the 'vulcan-project' folder. This is the first step after extracting the Vulcan download.

```bash
cd vulcan-project
```

--------------------------------

### Verify Vulcan Project Setup and Connection

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Checks the configuration and database connection status of your Vulcan project. It displays connection status, component counts, and project configuration details.

```bash
vulcan info
```

--------------------------------

### Run Vulcan Setup Script on Windows

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Executes the setup script for Vulcan on a Windows environment. This script performs the necessary setup steps, similar to the 'make setup' command on Unix-like systems.

```batch
setup.bat
```

--------------------------------

### Initialize a New Vulcan Project

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Initializes a new Vulcan project with a default structure and specified SQL engine. This command sets up directories for models, seeds, audits, tests, macros, checks, and semantics.

```bash
vulcan init
```

--------------------------------

### View Vulcan Service Logs

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Command to view the logs for Vulcan's infrastructure services. This is useful for diagnosing issues when services fail to start or encounter errors.

```shell
docker compose -f docker/docker-compose.infra.yml logs
```

--------------------------------

### Vulcan Plan Output Example (CLI)

Source: https://tmdc-io.github.io/vulcan-book/guides/incremental_by_time

Example output from the `vulcan plan` command, showing that a new model `sales.weekly_sales` has been added and requires backfilling. It details the date range for backfilling and prompts for confirmation.

```bash
======================================================================
Successfully Ran 2 tests against postgres
----------------------------------------------------------------------

Differences from the `prod` environment:

Models:
└── Added:
    └── sales.weekly_sales

Models needing backfill (missing dates):
└── sales.weekly_sales: 2025-01-01 - 2025-01-15

Apply - Backfill Tables [y/n]: y

```

--------------------------------

### Query Vulcan Data Using the Semantic Layer

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Transpiles a semantic layer query into executable SQL. This allows for business-friendly querying of data models through measures and dimensions.

```bash
vulcan transpile --format sql "SELECT MEASURE(measure_name) FROM model"
```

--------------------------------

### Expanded SQL Query from Advanced @STAR Example

Source: https://tmdc-io.github.io/vulcan-book/components/advanced-features/macros/built_in

Presents the SQL query after macro expansion for the advanced @STAR example. It shows how different prefixes are applied based on the @STAR calls and how explicit columns are included.

```sql
SELECT
  CAST("bar"."a" AS TEXT) AS "ab_pre_a",
  CAST("bar"."b" AS TEXT) AS "ab_pre_b",
  CAST("bar"."d" AS INT) AS "d_pre_d",
  my_column
FROM foo AS bar
```

--------------------------------

### Check Running Vulcan Services

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Commands to list the status of services managed by Docker Compose for Vulcan's infrastructure and warehouse components. This helps verify if services are running as expected.

```shell
docker compose -f docker/docker-compose.infra.yml ps
docker compose -f docker/docker-compose.warehouse.yml ps
```

--------------------------------

### Generate SQL Examples for Documentation

Source: https://tmdc-io.github.io/vulcan-book/guides/transpiling_semantics_q=

Creates SQL examples by transpiling semantic queries. This is useful for documentation and training, providing both the easy-to-understand semantic query and the actual SQL that will be executed.

```bash
# Create SQL reference from semantic queries
vulcan transpile --format sql "SELECT MEASURE(total_arr) FROM subscriptions WHERE subscriptions.status = 'active'"
```

--------------------------------

### Incremental Time Range Model with Lookback and Audits

Source: https://tmdc-io.github.io/vulcan-book/components/model/model_kinds

This SQL example defines an INCREMENTAL_BY_TIME_RANGE model with advanced configurations. It includes a `lookback` parameter to handle late-arriving data, a `start` date to prevent backfilling, a cron schedule for automated runs, and defines primary key audits for data validation.

```sql
MODEL (
  name demo.incrementals_demo,
  kind INCREMENTAL_BY_TIME_RANGE (
    -- How does this model kind behave?

    --   DELETE by time range, then INSERT
    time_column transaction_date,

    -- How do I handle late-arriving data?

    --   Handle late-arriving events for the past 2 (2*1) days based on cron

    --   interval. Each time it runs, it will process today, yesterday, and

    --   the day before yesterday.
    lookback 2,
  ),

  -- Don't backfill data before this date
  start '2024-10-25',

  -- What schedule should I run these at?

  --   Daily at Midnight UTC
  cron '@daily',

  -- Good documentation for the primary key
  grain transaction_id,

  -- How do I test this data?

  --   Validate that the `transaction_id` primary key values are both unique

  --   and non-null. Data audit tests only run for the processed intervals,

  --   not for the entire table.

  -- audits (

  --   UNIQUE_VALUES(columns = (transaction_id)),

  --   NOT_NULL(columns = (transaction_id))

  -- )
);

WITH sales_data AS (
  SELECT
    transaction_id,
    product_id,
    customer_id,
    transaction_amount,
    -- How do I account for UTC vs. PST (California baby) timestamps?

    --   Make sure all time columns are in UTC and convert them to PST in the

    --   presentation layer downstream.
    transaction_timestamp,
    payment_method,
    currency
  FROM vulcan-public-demo.tcloud_raw_data.sales  -- Source A: sales data
  -- How do I make this run fast and only process the necessary intervals?

  --   Use our date macros that will automatically run the necessary intervals.

  --   Because Vulcan manages state, it will know what needs to run each time

  --   you invoke `vulcan run`.
  WHERE transaction_timestamp BETWEEN @start_dt AND @end_dt
),

product_usage AS (
  SELECT
    product_id,
    customer_id,
    last_usage_date,
    usage_count,
    feature_utilization_score,
    user_segment
  FROM vulcan-public-demo.tcloud_raw_data.product_usage  -- Source B
  -- Include usage data from the 30 days before the interval
  WHERE last_usage_date BETWEEN DATE_SUB(@start_dt, INTERVAL 30 DAY) AND @end_dt
)

SELECT
  s.transaction_id,
  s.product_id,
  s.customer_id,
  s.transaction_amount,
  -- Extract the date from the timestamp to partition by day
  DATE(s.transaction_timestamp) as transaction_date,
  -- Convert timestamp to PST using a SQL function in the presentation layer for end users
  DATETIME(s.transaction_timestamp, 'America/Los_Angeles') as transaction_timestamp_pst,
  s.payment_method,
  s.currency,
  -- Product usage metrics
  p.last_usage_date,
  p.usage_count,
  p.feature_utilization_score,
  p.user_segment,
  -- Derived metrics
  CASE
    WHEN p.usage_count > 100 AND p.feature_utilization_score > 0.8 THEN 'Power User'
    WHEN p.usage_count > 50 THEN 'Regular User'
    WHEN p.usage_count IS NULL THEN 'New User'
    ELSE 'Light User'
  END as user_type,
  -- Time since last usage
  DATE_DIFF(s.transaction_timestamp, p.last_usage_date, DAY) as days_since_last_usage
FROM sales_data s
LEFT JOIN product_usage p
  ON s.product_id = p.product_id
  AND s.customer_id = p.customer_id

```

--------------------------------

### Model Defaults Configuration Example (YAML)

Source: https://tmdc-io.github.io/vulcan-book/configurations/overview

An example of the `model_defaults` section in a Vulcan configuration file. This section is mandatory and specifies default settings for all models, such as SQL dialect, owner, start date, and cron schedule.

```yaml
model_defaults:
  dialect: postgres     # Required
  owner: data-team
  start: 2024-01-01
  cron: '@daily'

```

--------------------------------

### Create and Apply a Vulcan Data Plan

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Generates and applies a data plan for your Vulcan models. This command validates models, creates database objects, calculates backfill intervals, and prompts for plan application.

```bash
vulcan plan
```

--------------------------------

### Set Start Date for Backfilling in Vulcan

Source: https://tmdc-io.github.io/vulcan-book/guides/incremental_by_time

Illustrates how to set a start date for Vulcan models, specifying the beginning of the data range for backfilling operations.

```sql
start '2025-01-01'  -- Start processing from this date
```

--------------------------------

### Example: Incremental Unmanaged Model for Append-Only Data in SQL

Source: https://tmdc-io.github.io/vulcan-book/components/model/model_kinds

Illustrates the setup for an INCREMENTAL_UNMANAGED model, designed for append-only scenarios like event logs or audit trails. This model simply appends query results to the table without deduplication or management.

```sql
MODEL (
  name vulcan_demo.incremental_unmanaged,
  kind INCREMENTAL_UNMANAGED,
  cron '@daily',
  start '2025-01-01',
  grains (shipment_id)
);

/* Append-only shipment event log */
SELECT
  s.shipment_id,
  s.order_id,
  s.shipped_date,
  s.carrier,
  o.customer_id,
  c.name AS customer_name,
  o.order_date,
  (s.shipped_date - o.order_date::DATE)::INT AS days_to_ship,
  CURRENT_TIMESTAMP AS shipment_event_timestamp
FROM vulcan_demo.shipments AS s
JOIN vulcan_demo.orders AS o ON s.order_id = o.order_id
JOIN vulcan_demo.customers AS c ON o.customer_id = c.customer_id
ORDER BY s.shipped_date DESC

```

--------------------------------

### Package Custom Materialization using setup.py (Python)

Source: https://tmdc-io.github.io/vulcan-book/components/advanced-features/custom_materializations

This Python code snippet illustrates how to define setuptools entrypoints in a 'setup.py' file to package a custom Vulcan materialization. This method enables Vulcan to automatically discover and load the materialization when the Python package is installed in the environment.

```python
setup(
    ...,
    entry_points={
        "vulcan.materializations": [
            "my_materialization = my_package.my_materialization:CustomFullMaterialization",
        ],
    },
)

```

--------------------------------

### Example: Incremental by Partition Model with CTE in SQL

Source: https://tmdc-io.github.io/vulcan-book/components/model/model_kinds

Demonstrates how to limit partition updates in an INCREMENTAL_BY_PARTITION model using a Common Table Expression (CTE). This pattern avoids full table restatements by specifying which partitions to update based on upstream data.

```sql
MODEL (
  name demo.incremental_by_partition_demo,
  kind INCREMENTAL_BY_PARTITION,
  partitioned_by user_segment
);

-- This is the source of truth for what partitions need to be updated and will join to the product usage data

-- This could be an INCREMENTAL_BY_TIME_RANGE model that reads in the user_segment values last updated in the past 30 days to reduce scope

-- Use this strategy to reduce full restatements
WITH partitions_to_update AS (
  SELECT DISTINCT
    user_segment
  FROM demo.incremental_by_time_range_demo  -- upstream table tracking which user segments to update
  WHERE last_updated_at BETWEEN DATE_SUB(@start_dt, INTERVAL 30 DAY) AND @end_dt
),

product_usage AS (
  SELECT
    product_id,
    customer_id,
    last_usage_date,
    usage_count,
    feature_utilization_score,
    user_segment
  FROM vulcan-public-demo.tcloud_raw_data.product_usage
  WHERE user_segment IN (SELECT user_segment FROM partitions_to_update) -- partition filter applied here
)

SELECT
  product_id,
  customer_id,
  last_usage_date,
  usage_count,
  feature_utilization_score,
  user_segment,
  CASE
    WHEN usage_count > 100 AND feature_utilization_score > 0.7 THEN 'Power User'
    WHEN usage_count > 50 THEN 'Regular User'
    WHEN usage_count IS NULL THEN 'New User'
    ELSE 'Light User'
  END as user_type
FROM product_usage

```

--------------------------------

### Stop All Vulcan Services (Windows)

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Batch script to stop all running Vulcan services on a Windows system. This is a convenient way to shut down all components of the Vulcan environment.

```batch
stop-all.bat           # Stop all services
```

--------------------------------

### Query Vulcan Models with SQL

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Executes a SQL query against your Vulcan models and returns the results as a pandas DataFrame. This is useful for direct data exploration and validation.

```bash
vulcan fetchdf "select * from schema.model_name"
```

--------------------------------

### Generate SQL Examples for Documentation with Vulcan Transpile

Source: https://tmdc-io.github.io/vulcan-book/guides/transpiling_semantics_q=

Leverage the `vulcan transpile` command to generate SQL examples for documenting common query patterns. This ensures that your documentation includes accurate and up-to-date SQL representations of your semantic queries, making it easier for others to understand and use them.

```shell
# Generate SQL examples for documentation
vulcan transpile --format sql "SELECT MEASURE(total_arr) FROM subscriptions WHERE subscriptions.status = 'active'"
```

--------------------------------

### Check and Create Docker Network (Mac/Linux/Windows)

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Commands to check if the 'vulcan' Docker network exists and create it if it doesn't. This network is essential for inter-service communication within the Vulcan environment.

```shell
# Check if the network exists:
docker network ls | grep vulcan

# If it doesn't exist, create it:
docker network create vulcan
```

--------------------------------

### Example VIEW Model Creation SQL (BigQuery)

Source: https://tmdc-io.github.io/vulcan-book/components/model/model_kinds

This SQL sequence illustrates how Vulcan creates a versioned VIEW in BigQuery. It includes the SQL to create the physical view with a version fingerprint and a virtual view in a development schema that points to the physical view.

```sql
MODEL (
  name demo.example_view,
  kind VIEW,
  cron '@daily'
);

SELECT
  'hello there' as a_column

```

```sql
CREATE OR REPLACE VIEW `vulcan-public-demo`.`vulcan__demo`.`demo__example_view__1024042926`
(`a_column`) AS SELECT 'hello there' AS `a_column`

```

```sql
CREATE SCHEMA IF NOT EXISTS `vulcan-public-demo`.`demo__dev`

```

```sql
CREATE OR REPLACE VIEW `vulcan-public-demo`.`demo__dev`.`example_view` AS
SELECT * FROM `vulcan-public-demo`.`vulcan__demo`.`demo__example_view__1024042926`

```

--------------------------------

### Stop Individual Vulcan API Service (Windows)

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Batch script to stop only the Vulcan API services on a Windows system. This allows for targeted shutdown of specific service components.

```batch
vulcan-down.bat        # Stop only Vulcan API services
```

--------------------------------

### SQL Pre-statements for Model Setup

Source: https://tmdc-io.github.io/vulcan-book/components/model/statements

Demonstrates how to use SQL pre-statements to set up a model. This includes creating a table for anomaly tracking and setting session variables using Jinja templating. It also shows the main query structure.

```sql
MODEL (
  name analytics.orders,
  kind INCREMENTAL_BY_TIME_RANGE (
    time_column order_date
  ),
  start '2020-01-01',
  cron '@daily'
);

/* Pre-statement: Create table for anomaly tracking */
CREATE TABLE IF NOT EXISTS analytics._orders_anomalies (
  anomaly_id BIGINT GENERATED ALWAYS AS IDENTITY,
  order_id VARCHAR,
  anomaly_type VARCHAR,
  details VARCHAR,
  captured_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

/* Pre-statement: Set session variables using Jinja */
JINJA_STATEMENT_BEGIN;
{% if start_date is none or end_date is none %}
  SET start_date = DATE '{{ start }}';
  SET end_date = CURRENT_DATE;
{% endif %}
JINJA_END;

/* Main model query */
SELECT
  order_id::VARCHAR AS order_id,
  order_date::DATE AS order_date,
  customer_id::VARCHAR AS customer_id,
  total_amount::FLOAT AS total_amount
FROM demo.raw_data.orders
WHERE
  order_date BETWEEN @start_date AND @end_date;
```

--------------------------------

### Configure Vulcan CLI Alias for Databricks

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the Databricks engine. This facilitates interaction with Vulcan when using Databricks.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-databricks:0.228.1.6 vulcan"
```

--------------------------------

### Minimal Full-Refresh Materialization (Python)

Source: https://tmdc-io.github.io/vulcan-book/components/advanced-features/custom_materializations

Provides a minimal Python example for a custom materialization that performs a full refresh of a table. It utilizes the `replace_query` method of the adapter to achieve this.

```python
from vulcan import CustomMaterialization
from vulcan import Model
import typing as t

class CustomFullMaterialization(CustomMaterialization):
    NAME = "my_custom_full"

    def insert(
        self,
        table_name: str,
        query_or_df: t.Any,
        model: Model,
        is_first_insert: bool,
        render_kwargs: t.Dict[str, t.Any],
        **kwargs: t.Any,
    ) -> None:
        self.adapter.replace_query(table_name, query_or_df)

```

--------------------------------

### Stop and Clean Up All Vulcan Services (Windows)

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Batch script to stop all Vulcan services and remove associated volumes on Windows. This action will delete all data. Use 'clean.bat' for a complete cleanup.

```batch
clean.bat              # Stop and remove volumes (this will delete all data)
```

--------------------------------

### SQL Example: INCREMENTAL_BY_UNIQUE_KEY Model Execution Plan

Source: https://tmdc-io.github.io/vulcan-book/components/model/model_kinds

Illustrates the SQL generated by Vulcan for an INCREMENTAL_BY_UNIQUE_KEY model, including the CREATE TABLE statement with a versioned table name and a validation query. This shows how Vulcan translates the model definition into executable SQL for data warehousing.

```sql
MODEL (
  name demo.incremental_by_unique_key_example,
  kind INCREMENTAL_BY_UNIQUE_KEY (
    unique_key id
  ),
  start '2020-01-01',
  cron '@daily',
);

SELECT
  id,
  item_id,
  event_date
FROM demo.seed_model
WHERE
  event_date BETWEEN @start_date AND @end_date


```

```sql
CREATE TABLE IF NOT EXISTS `vulcan-public-demo`.`vulcan__demo`.`demo__incremental_by_unique_key_example__1161945221` (`id` INT64, `item_id` INT64, `event_date` DATE)

```

```sql
SELECT `seed_model`.`id` AS `id`, `seed_model`.`item_id` AS `item_id`, `seed_model`.`event_date` AS `event_date`

```

--------------------------------

### Configure Vulcan CLI Alias for Spark

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the Spark engine. This is used for interacting with Vulcan when Spark is the processing engine.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-spark:0.228.1.6 vulcan"
```

--------------------------------

### Configure Vulcan CLI Alias for Redshift

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the Redshift engine. This is used for interacting with Vulcan when Redshift is the data warehouse.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-redshift:0.228.1.6 vulcan"
```

--------------------------------

### Transpile Semantic SQL Query using CLI

Source: https://tmdc-io.github.io/vulcan-book/guides/transpiling_semantics_q=

Demonstrates the basic command-line usage for transpiling a Semantic SQL query into native SQL format. It shows the `vulcan transpile` command with the `--format sql` flag and a sample query.

```bash
vulcan transpile --format sql "SELECT MEASURE(total_users) FROM users"

```

--------------------------------

### Configure Vulcan CLI Alias for MySQL

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the MySQL engine. This allows for interaction with Vulcan when MySQL is the selected database.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-mysql:0.228.1.6 vulcan"
```

--------------------------------

### Package Custom Materialization using pyproject.toml (TOML)

Source: https://tmdc-io.github.io/vulcan-book/components/advanced-features/custom_materializations

This TOML snippet shows how to configure a Python package's 'pyproject.toml' file to expose a custom Vulcan materialization via setuptools entrypoints. This allows Vulcan to discover and load the materialization automatically when the package is installed.

```toml
[project.entry-points."vulcan.materializations"]
my_materialization = "my_package.my_materialization:CustomFullMaterialization"

```

--------------------------------

### Configure Vulcan CLI Alias for Postgres

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, specifically configured to use a Postgres engine. This allows interaction with Vulcan for development.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-postgres:0.228.1.6 vulcan"
```

--------------------------------

### Configure Vulcan CLI Alias for Trino

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the Trino engine. This allows interaction with Vulcan when Trino is used as the query engine.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-trino:0.228.1.6 vulcan"
```

--------------------------------

### Initialize a New Vulcan Project

Source: https://tmdc-io.github.io/vulcan-book/cli-commands/cli

Initializes a new Vulcan project with a specified template (dbt, dlt, default, empty) or an empty project. It sets up the basic project structure and configuration files. For the 'dlt' template, additional options for specifying the DLT pipeline and its directory are available.

```bash
Usage: vulcan init [OPTIONS] [ENGINE]

  Create a new Vulcan repository.

Options:
  -t, --template TEXT  Project template. Supported values: dbt, dlt, default,
                       empty.
  --dlt-pipeline TEXT  DLT pipeline for which to generate a Vulcan project.
                       Use alongside template: dlt
  --dlt-path TEXT      The directory where the DLT pipeline resides. Use
                       alongside template: dlt
  --help               Show this message and exit.

Example
$ vulcan init postgres
```

--------------------------------

### Configure Vulcan CLI Alias for Snowflake

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the Snowflake engine. This enables interaction with Vulcan when using Snowflake as the data source.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-snowflake:0.228.1.6 vulcan"
```

--------------------------------

### Custom Insert Logic with Logging (Python)

Source: https://tmdc-io.github.io/vulcan-book/components/advanced-features/custom_materializations

Demonstrates a custom materialization strategy in Python that includes detailed logging for insert operations. It handles both SQL query execution and DataFrame insertion, with conditional logic for first-time table creation.

```python
import typing as t
from sqlalchemy import text
from vulcan import CustomMaterialization
from vulcan import Model

class SimpleCustomMaterialization(CustomMaterialization):
    """Simple custom materialization - demonstrates custom insert logic"""

    NAME = "simple_custom"

    def insert(
        self,
        table_name: str,
        query_or_df: t.Union[str, t.Any],
        model: Model,
        is_first_insert: bool,
        render_kwargs: t.Dict[str, t.Any],
        **kwargs: t.Any,
    ) -> None:
        """Custom insert logic for tables"""

        print(f"Custom materialization: Processing table {table_name}")
        print(f"Model: {model.name}")
        print(f"Is first insert: {is_first_insert}")

        if is_first_insert:
            print("Creating table for the first time")
            # Create the table normally using the adapter
            self.adapter.create_table(
                table_name,
                columns=model.columns_to_types,
                target_columns_to_types=model.columns_to_types,
                partitioned_by=model.partitioned_by,
            )

        # Insert data with custom logic
        if isinstance(query_or_df, str):
            print("Executing SQL query")
            # Execute the query - Vulcan provides the INSERT INTO ... SELECT query
            self.adapter.execute(text(query_or_df))
        else:
            print("Inserting DataFrame")
            # Insert DataFrame normally - useful for Python models that return DataFrames
            self.adapter.insert_append(table_name, query_or_df)

        print(f"Custom materialization completed for {table_name}")

```

--------------------------------

### Configure Vulcan CLI Alias for Fabric

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the Fabric engine. This is used for interacting with Vulcan when Fabric is the chosen data engine.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-fabric:0.228.1.6 vulcan"
```

--------------------------------

### vulcan init

Source: https://tmdc-io.github.io/vulcan-book/cli-commands/cli

Initializes a new Vulcan project with optional templates like dbt or DLT, or an empty project.

```APIDOC
## Initialize a New Vulcan Project

### Description
Initializes a new Vulcan project, setting up the basic project structure and configuration files. Supports various templates (dbt, DLT, default, empty).

### Method
CLI Command

### Endpoint
vulcan init [OPTIONS] [ENGINE]

### Parameters
#### Path Parameters
- **ENGINE** (TEXT) - Optional - The database engine to use for the project.

#### Query Parameters
- **-t, --template** (TEXT) - Optional - Project template. Supported values: dbt, dlt, default, empty.
- **--dlt-pipeline** (TEXT) - Optional - DLT pipeline name to generate a Vulcan project for (use with template: dlt).
- **--dlt-path** (TEXT) - Optional - Directory where the DLT pipeline resides (use with template: dlt).

### Request Example
```
$ vulcan init postgres
```

### Response
#### Success Response
- Project structure and configuration files are created.

#### Response Example
```
Project initialized successfully.
```
```

--------------------------------

### Configure Vulcan CLI Alias for BigQuery

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Creates a temporary Docker alias to access the Vulcan CLI, configured for the BigQuery engine. This enables interaction with Vulcan using BigQuery as the data source.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-bigquery:0.228.1.6 vulcan"
```

--------------------------------

### Stop All Vulcan Services (Mac/Linux)

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker_q=

Commands to stop all running Vulcan services and related infrastructure on Mac or Linux systems. This includes stopping all services or performing a clean stop that removes volumes and data.

```bash
make all-down
or
docker compose -f docker/docker-compose.infra.yml down -v
docker compose -f docker/docker-compose.warehouse.yml down -v
```

```bash
make all-clean
# Stop and remove volumes (this will delete all data)
```

--------------------------------

### Create Weekly Sales Model File (Shell)

Source: https://tmdc-io.github.io/vulcan-book/guides/models_q=

Command to create a new SQL file for defining a weekly sales aggregation model within the Vulcan project's models directory.

```shell
touch models/sales/weekly_sales.sql

```

--------------------------------

### Stop and Clean Up All Vulcan Services (Mac/Linux)

Source: https://tmdc-io.github.io/vulcan-book/guides/get-started/docker

Command to stop all Vulcan services and remove associated volumes. Warning: This action will delete all data stored in the volumes. Use 'make all-clean' for a comprehensive cleanup.

```shell
make all-clean      # Stop and remove volumes (this will delete all data)
```
Tokens:
Raw