### Docker Quickstart Guide

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/FILE_REFERENCES.md

This section provides instructions for getting started with Vulcan using Docker. It's a common entry point for new users.

```markdown
### `getting_started/docker.md`

Linked from 7 file(s):

- **index.md** (Line 104): `quickstart guide`
- **guides/models.md** (Line 9): `Created your project`
- **guides/plan.md** (Line 131): `Docker Quickstart`
- **concepts-old/macros/vulcan_macros.md** (Line 267): `Vulcan quickstart guide`
- **getting_started/cli.md** (Line 26): `Docker Quickstart`
- **getting_started/cli.md** (Line 64): `Docker Quickstart`
- **getting_started/index.md** (Line 15): `Start with Docker Quickstart`
```

--------------------------------

### Run Setup Script - Windows Batch

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-window/vulcan-project/README.md

Executes the setup script to create a Docker network and start essential infrastructure services like statestore, MinIO, and the warehouse database.

```batch
setup.bat
```

--------------------------------

### Select Default Project Type - Bash

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

Example of user input during the `vulcan init` process, selecting the default option to create an example project with models and files.

```bash
1
```

--------------------------------

### Vulcan Project Configuration Example (YAML)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

Illustrates the `config.yaml` file for a Vulcan project, demonstrating how to configure gateway connections, default model settings, and linter rules. This example specifically shows DuckDB integration.

```yaml
# --- Gateway Connection ---
gateways:
  duckdb:
    connection:
      # For more information on configuring the connection to your execution engine, visit:
      # https://vulcan.readthedocs.io/en/stable/reference/configuration/#connection
      # https://vulcan.readthedocs.io/en/stable/integrations/engines/duckdb/#connection-options
      #
      type: duckdb               # <-- DuckDB engine
      database: db.db
      # concurrent_tasks: 1
      # register_comments: True  # <-- Optional setting `register_comments` has a default value of True
      # pre_ping: False
      # pretty_sql: False
      # catalogs:
      # extensions:
      # connector_config:
      # secrets:
      # token:

default_gateway: duckdb

# --- Model Defaults ---
# https://vulcan.readthedocs.io/en/stable/reference/model_configuration/#model-defaults

model_defaults:
  dialect: duckdb                # <-- Models written in DuckDB SQL dialect by default
  start: 2025-06-12 # Start date for backfill history
  cron: '@daily'    # Run models daily at 12am UTC (can override per model)

# --- Linting Rules ---
# Enforce standards for your team
# https://vulcan.readthedocs.io/en/stable/guides/linter/

linter:
  enabled: true
  rules:
    - ambiguousorinvalidcolumn
    - invalidselectstarexpansion
```

--------------------------------

### Start Vulcan Infrastructure with Make

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-mac/vulcan-project/README.md

This command initiates all necessary infrastructure services for Vulcan, including the statestore, MinIO object storage, and the warehouse database. It ensures all dependencies are running before Vulcan services are started.

```bash
make setup
```

--------------------------------

### Project Setup and Management Commands (Bash)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/README.md

A collection of make commands for managing the Vulcan project. These include initial setup, starting a development server, building the static site, and deploying to GitHub Pages. Ensure 'make' is installed and the Makefile is present in the project root.

```bash
make setup    # First time setup
make serve    # Start development server
make build    # Build static site
make deploy   # Deploy to GitHub Pages
```

```bash
make deploy
```

--------------------------------

### Start Vulcan API Services - Windows Batch

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-window/vulcan-project/README.md

A script specifically for starting the Vulcan API services without necessarily starting all other infrastructure components.

```batch
start-vulcan-api.bat
```

--------------------------------

### Start Vulcan Services with Make

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/examples/exhaustive_model/README.md

This command brings up the services defined in the Docker Compose files, specifically for the Vulcan application. It ensures all necessary services are running and connected.

```bash
make vulcan-up
```

--------------------------------

### Setup Infrastructure Services with Make

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/book/vulcan-lifecycle.md

Initializes the necessary infrastructure services for Vulcan, including the statestore, MinIO for artifact storage, and the data warehouse. This step ensures the environment is ready for project development.

```bash
make setup
# Creates: statestore, MinIO, warehouse database
```

--------------------------------

### Example Incremental Model Configuration (SQL)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/incremental_by_time.md

Demonstrates the configuration for an incremental data model named 'sales.daily_sales'. It includes setting the model kind to INCREMENTAL_BY_TIME_RANGE, specifying the time column, defining the start date, and setting the cron schedule.

```sql
MODEL (
  name sales.daily_sales,
    kind INCREMENTAL_BY_TIME_RANGE (
    time_column order_date
  ),
  start '2025-01-01',
  cron '@daily',
  grain order_date,
  description 'Daily sales summary with order counts and revenue',
  column_descriptions (
    order_date = 'Date of the sales',
    total_orders = 'Total number of orders for the day',
    total_revenue = 'Total revenue for the day',
    last_order_id = 'Last order ID processed for the day'
  ),
  assertions (
    unique_values(columns := (order_date)),
    not_null(columns := (order_date, total_orders, total_revenue)),
    positive_values(column := total_orders),
    positive_values(column := total_revenue)
  )
);

SELECT
  CAST(order_date AS TIMESTAMP)::TIMESTAMP AS order_date,
  COUNT(order_id)::INTEGER AS total_orders,
  SUM(total_amount)::FLOAT AS total_revenue,
  MAX(order_id)::VARCHAR AS last_order_id
FROM raw.raw_orders
WHERE order_date BETWEEN @start_ds AND @end_ds  -- ADD THIS
GROUP BY order_date
ORDER BY order_date
```

--------------------------------

### Example INCREMENTAL_BY_TIME_RANGE Model with Lookback and WHERE Clause

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/components/model/model_kinds.md

This comprehensive SQL example illustrates an INCREMENTAL_BY_TIME_RANGE model setup. It includes `time_column`, `lookback` for handling late-arriving data, `start` date, `cron` schedule, `grain` for primary key definition, and data `audits`. The model's query uses `WHERE` clauses with Vulcan's date macros (`@start_dt`, `@end_dt`) to filter upstream data efficiently.

```sql
MODEL (
  name demo.incrementals_demo,
  kind INCREMENTAL_BY_TIME_RANGE (
    -- How does this model kind behave?

    --   DELETE by time range, then INSERT
    time_column transaction_date,

    -- How do I handle late-arriving data?

    --   Handle late-arriving events for the past 2 (2*1) days based on cron

    --   interval. Each time it runs, it will process today, yesterday, and

    --   the day before yesterday.
    lookback 2,
  ),

  -- Don't backfill data before this date
  start '2024-10-25',

  -- What schedule should I run these at?

  --   Daily at Midnight UTC
  cron '@daily',

  -- Good documentation for the primary key
  grain transaction_id,

  -- How do I test this data?

  --   Validate that the `transaction_id` primary key values are both unique

  --   and non-null. Data audit tests only run for the processed intervals,

  --   not for the entire table.

  -- audits (

  --   UNIQUE_VALUES(columns = (transaction_id)),

  --   NOT_NULL(columns = (transaction_id))

  -- )
);

WITH sales_data AS (
  SELECT
    transaction_id,
    product_id,
    customer_id,
    transaction_amount,
    -- How do I account for UTC vs. PST (California baby) timestamps?

    --   Make sure all time columns are in UTC and convert them to PST in the

    --   presentation layer downstream.
    transaction_timestamp,
    payment_method,
    currency
  FROM vulcan-public-demo.tcloud_raw_data.sales  -- Source A: sales data
  -- How do I make this run fast and only process the necessary intervals?

  --   Use our date macros that will automatically run the necessary intervals.

  --   Because Vulcan manages state, it will know what needs to run each time

  --   you invoke `vulcan run`.
  WHERE transaction_timestamp BETWEEN @start_dt AND @end_dt
),

product_usage AS (
  SELECT
    product_id,
    customer_id,
    last_usage_date,
    usage_count,
    feature_utilization_score,
    user_segment
  FROM vulcan-public-demo.tcloud_raw_data.product_usage  -- Source B
  -- Include usage data from the 30 days before the interval
  WHERE last_usage_date BETWEEN DATE_SUB(@start_dt, INTERVAL 30 DAY) AND @end_dt
)

SELECT
  s.transaction_id,
  s.product_id,
  s.customer_id,
  s.transaction_amount,
  -- Extract the date from the timestamp to partition by day
  DATE(s.transaction_timestamp) as transaction_date,
  -- Convert timestamp to PST using a SQL function in the presentation layer for end users
  DATETIME(s.transaction_timestamp, 'America/Los_Angeles') as transaction_timestamp_pst,
  s.payment_method,
  s.currency,
  -- Product usage metrics
  p.last_usage_date,
  p.usage_count,
  p.feature_utilization_score,
  p.user_segment,
  -- Derived metrics
  CASE
    WHEN p.usage_count > 100 AND p.feature_utilization_score > 0.8 THEN 'Power User'
    WHEN p.usage_count > 50 THEN 'Regular User'
    WHEN p.usage_count IS NULL THEN 'New User'
    ELSE 'Light User'
  END as user_type,
  -- Time since last usage

```

--------------------------------

### Vulcan Configuration Example

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-mac/vulcan-project/README.md

This YAML snippet illustrates the configuration required for a Vulcan project when using Docker. It specifies project metadata, gateway connections to the warehouse and statestore databases, and default model settings, including linter rules.

```yaml
# Project metadata
name: orders360
tenant: sales
description: Daily sales analytics pipeline

# Gateway Connection
gateways:
default:
   connection:
      type: postgres
      host: warehouse
      port: 5432
      database: warehouse
      user: vulcan
      password: vulcan
  state_connection:
      type: postgres
      host: statestore
      port: 5432
      database: statestore
      user: vulcan
      password: vulcan

default_gateway: default

# Model Defaults (required)
model_defaults:
 dialect: postgres
 start: 2024-01-01
 cron: '@daily'

# Linting Rules
linter:
 enabled: true
 rules:
    - ambiguousorinvalidcolumn

    - invalidselectstarexpansion

```

--------------------------------

### Transpiler Usage Examples (Bash)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Demonstrates how to use the Vulcan transpiler CLI to convert natural language queries into SQL or JSON formats. Examples cover simple measure queries, queries with dimensions and filters, and complex JSON query structures.

```bash
# Simple measure query
vulcan transpile --format sql "SELECT MEASURE(total_users) FROM users"

# Query with dimensions
vulcan transpile --format sql "SELECT users.plan_type, MEASURE(total_users) FROM users GROUP BY users.plan_type"

# Query with filters
vulcan transpile --format sql "SELECT MEASURE(total_arr) FROM subscriptions WHERE subscriptions.status = 'active'"

# Complex JSON query
vulcan transpile --format json '{
  "query": {
    "measures": ["subscriptions.total_arr"],
    "dimensions": ["subscriptions.plan_type", "users.industry"],
    "filters": [{
      "member": "subscriptions.status",
      "operator": "equals",
      "values": ["active"]
    }],
    "timeDimensions": [{
      "dimension": "subscriptions.start_date",
      "dateRange": ["2024-01-01", "2024-12-31"],
      "granularity": "month"
    }],
    "order": {"subscriptions.total_arr": "desc"},
    "limit": 100
  }
}'
```

--------------------------------

### Manage Vulcan Services with Make

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-mac/vulcan-project/README.md

This section lists various make commands for managing Vulcan's Dockerized services. These commands simplify tasks such as starting, stopping, and cleaning up all related Docker containers and volumes.

```makefile
make setup
make vulcan-up
make network
make infra
make warehouse
make all-down
make all-clean
```

--------------------------------

### Start Infrastructure Services (Mac/Linux)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Starts the essential infrastructure services for Vulcan, including statestore, MinIO, and the warehouse database. These services are required for Vulcan's operation. This command uses Docker Compose for orchestration.

```bash
make setup
# OR manually:
docker network create vulcan
docker compose -f docker/docker-compose.infra.yml up -d
docker compose -f docker/docker-compose.warehouse.yml up -d
```

--------------------------------

### Start Docker Shell - Bash

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

Commands to enter the Vulcan Docker shell, essential for users running Vulcan via Docker. This ensures all subsequent Vulcan commands are executed within the correct Docker environment.

```bash
make vulcan-shell
```

```bash
docker compose -f docker/docker-compose.vulcan.yml run --rm vulcan-shell
```

--------------------------------

### Transpile SQL Query with Vulcan

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/examples/exhaustive_model/README.md

This command uses the Vulcan CLI to transpile a given SQL query into another format, specified here as 'sql'. It's useful for query transformation or validation.

```bash
vulcan transpile --format sql "select measure(total_order_lines) from orders"
```

--------------------------------

### Configure and Use Vulcan CLI

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Details on setting up the Vulcan CLI alias for Mac/Linux and Windows, and examples of common CLI commands for API-like operations such as fetching project info, rendering SQL, executing queries, and transpiling semantic queries.

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-postgres:0.228.2 vulcan"
```

```cmd
vulcan.bat
```

```bash
# Get project information
vulcan info

# Render model SQL
vulcan render model_name

# Execute SQL query
vulcan fetchdf "SELECT * FROM schema.model_name"

# Transpile semantic queries
vulcan transpile --format sql "SELECT MEASURE(total_users) FROM users"
vulcan transpile --format json '{"query": {"measures": ["users.total_users"]}}'

# Start API server
vulcan api --host 0.0.0.0 --port 8000

# Manage GraphQL service
vulcan graphql up
vulcan graphql down
```

--------------------------------

### Create Test Data with Vulcan

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/examples/exhaustive_model/README.md

This command utilizes the Vulcan CLI to create test data for a specified table ('ANALYTICS.ORDERS') based on a provided SQL query. It's designed for populating testing environments.

```bash
vulcan create_test ANALYTICS.ORDERS --query DEMO.RAW_DATA.ORDERS "select ORDER_ID ,ORDER_DATE ,CUSTOMER_ID ,PRODUCT_ID ,QUANTITY ,UNIT_PRICE ,DISCOUNT ,TAX ,SHIPPING_COST ,TOTAL_AMOUNT FROM DEMO.RAW_DATA.ORDERS
WHERE ORDER_DATE  BETWEEN '2025-01-01' and '2025-01-15'"
```

--------------------------------

### Vulcan Plan Output: Backfill with Start Date and Preview (Bash)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/references/plans.md

This bash output demonstrates running `vulcan plan dev --start 2024-09-24`. Because the specified start date is after the model's defined start date ('2024-09-23'), the `start_end_model` is marked as `(preview)`. The backfill range for `incremental_model` is also adjusted to the new start date.

```bash
❯ vulcan plan dev --start 2024-09-24
======================================================================
Successfully Ran 1 tests against duckdb
----------------------------------------------------------------------
New environment `dev` will be created from `prod`

Differences from the `prod` environment:

Models:
├── Directly Modified:
│   ├── vulcan_example__dev.start_end_model
│   └── vulcan_example__dev.incremental_model
└── Indirectly Modified:
    └── vulcan_example__dev.full_model

[...model diff omitted...]

Directly Modified: vulcan_example__dev.start_end_model (Non-breaking)
Models needing backfill (missing dates):
├── vulcan_example__dev.incremental_model: 2024-09-24 - 2024-09-26
└── vulcan_example__dev.start_end_model: 2024-09-24 - 2024-09-26 (preview)
Enter the backfill end date (eg. '1 month ago', '2020-01-01') or blank to backfill up until '2024-09-27 00:00:00':
```

--------------------------------

### Configure Model Defaults

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_errors_reference.md

Example of how to add the required 'model_defaults' section to 'config.yaml' to resolve 'model defaults missing' errors. It specifies the dialect, start date, and cron schedule.

```yaml
model_defaults:
  dialect: postgres  # or your SQL engine
  start: 2024-01-01
  cron: '@daily'
```

--------------------------------

### Initialize Production Environment with Vulcan

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/plan.md

This snippet demonstrates the initial setup of a production environment using Vulcan. It involves checking project status and creating the first plan, which includes detecting all models and preparing them for initialization.

```bash
vulcan info
```

```bash
vulcan plan
```

```bash
y
```

--------------------------------

### Vulcan Project Initialization and Next Steps (Bash)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

Provides the initial message displayed after a Vulcan project is ready, outlining essential next steps for configuration and execution. It includes commands for planning and explaining plans, along with links to documentation and support channels.

```bash
Your Vulcan project is ready!

Next steps:
- Update your gateway connection settings (e.g., username/password) in the project configuration file:
    /vulcan-example/config.yaml
- Run command in CLI: vulcan plan
- (Optional) Explain a plan: vulcan plan --explain

Quickstart guide:
https://vulcan.readthedocs.io/en/stable/quickstart/cli/

Need help?
- Docs:   https://vulcan.readthedocs.io
- Slack:  https://www.tobikodata.com/slack
- GitHub: https://github.com/TobikoData/vulcan/issues
```

--------------------------------

### Run Vulcan Project Setup (Windows Batch)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/docker.md

Executes the setup script for Vulcan projects on Windows. This script initializes essential services like statestore (PostgreSQL), minio (Object Storage), and minio-init. Requires Docker Desktop to be running and user to be logged into RubikLabs.

```batch
cd vulcan-project
setup.bat
```

--------------------------------

### Start API Services (Mac/Linux)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Starts the Vulcan API services, including the REST API server and the transpiler service. These are essential for interacting with Vulcan programmatically and for query translation.

```bash
make vulcan-up
# OR manually:
docker compose -f docker/docker-compose.vulcan.yml up -d
```

--------------------------------

### Create Project Directory - Bash

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

Creates a new directory for your Vulcan project. This is the first step in setting up a new Vulcan project locally.

```bash
mkdir vulcan-example
```

--------------------------------

### Package Custom Materialization using Setuptools Entrypoints (TOML)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/components/advanced-features/custom_materializations.md

This TOML snippet defines a project's entry points for Vulcan materializations using setuptools. It specifies a key 'my_materialization' that maps to the Python path of the custom materialization class.

```toml
[project.entry-points."vulcan.materializations"]
my_materialization = "my_package.my_materialization:CustomFullMaterialization"
```

--------------------------------

### Start Vulcan GraphQL Service

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Starts the Vulcan GraphQL service using the CLI. The `--no-detach` option can be used to run the Docker Compose in the foreground. The service is typically available at http://localhost:4000/graphql.

```bash
vulcan graphql up
```

--------------------------------

### Start Vulcan REST API Server

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Starts the Vulcan REST API server with configurable host and port. It can also enable auto-reloading and specify the number of worker processes. The server is accessible via HTTP.

```bash
vulcan api --host 0.0.0.0 --port 8000
```

--------------------------------

### Run Vulcan Local Setup (Mac/Linux)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/docker.md

Executes the local setup for Vulcan using Make or Docker Compose. This command initiates essential services like statestore (PostgreSQL) and object storage (MinIO), which are critical for Vulcan's operation. Ensure Docker Desktop is running and you are logged into RubikLabs before execution.

```bash
cd vulcan-project
make setup
or
docker network create vulcan
docker compose -f docker/docker-compose.infra.yml up -d
docker compose -f docker/docker-compose.warehouse.yml up -d
```

--------------------------------

### Run Vulcan CLI Commands - Windows Batch

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-window/vulcan-project/README.md

A wrapper script to execute Vulcan CLI commands, including initialization and plan creation. It simplifies interacting with the Vulcan services.

```batch
vulcan.bat
```

```batch
vulcan.bat init
```

```batch
vulcan.bat plan
```

--------------------------------

### Semantic SQL Query Format Example

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

An example illustrating the structure and syntax of a semantic SQL query that can be used with the Vulcan Transpiler. It includes common clauses like SELECT, FROM, CROSS JOIN, WHERE, GROUP BY, ORDER BY, LIMIT, and OFFSET.

```sql
SELECT 
  alias.dimension_name,
  MEASURE(alias.measure_name)
FROM alias
CROSS JOIN other_alias
WHERE alias.dimension_name = 'value'
  AND segment_name = true
GROUP BY alias.dimension_name
ORDER BY MEASURE(alias.measure_name)
LIMIT 100
OFFSET 0
```

--------------------------------

### Install and Use Vulcan Python API

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Instructions for installing the Vulcan Python package and using the VulcanContext object for programmatic access to data pipelines. This includes initializing the context, accessing models, executing queries, and interacting with the semantic layer.

```bash
pip install vulcan
```

```python
from vulcan import VulcanContext

# Initialize Vulcan context
ctx = VulcanContext()

# Access models
models = ctx.models()

# Execute queries
result = ctx.fetchdf("SELECT * FROM schema.model_name")

# Access semantic layer
# (Specific methods depend on Vulcan version and implementation)
```

```python
from vulcan import VulcanContext

ctx = VulcanContext()

# Get project information
info = ctx.info()

# Render a model's SQL
sql = ctx.render("model_name")

# Evaluate a model
df = ctx.evaluate("model_name")

# Run a plan
ctx.plan()

# Execute a run
ctx.run()
```

--------------------------------

### Initialize Vulcan Project

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-mac/vulcan-project/README.md

This command initializes a new Vulcan project. It typically generates necessary configuration files and sets up the project structure, preparing it for data pipeline development.

```bash
vulcan init
```

--------------------------------

### Troubleshoot Vulcan API Connection Refused Errors

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_errors_reference.md

Guides users through resolving 'connection refused' errors when interacting with the Vulcan API. Includes steps to verify service status, check logs, manage Docker networks, and start services.

```bash
# Verify services are running
docker ps

# Check API container logs
docker logs vulcan-api

# Ensure Docker network exists
docker network create vulcan

# Start API services
make vulcan-up
# OR
docker compose -f docker/docker-compose.vulcan.yml up -d

# Verify API is accessible
curl http://localhost:8000/health
```

--------------------------------

### Navigate to Project Directory - Bash

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

Changes the current working directory to the newly created Vulcan project directory. This is necessary before running initialization commands.

```bash
cd vulcan-example
```

--------------------------------

### Verify Vulcan CLI Access

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Tests if the Vulcan CLI alias is correctly configured and accessible. This command should display the help information for the Vulcan CLI if the setup is successful.

```bash
vulcan --help
```

--------------------------------

### Package Custom Materialization using Setuptools Entrypoints (Python)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/components/advanced-features/custom_materializations.md

This Python snippet shows how to configure setuptools for packaging custom materializations. The 'entry_points' argument in the 'setup' function registers the custom materialization under the 'vulcan.materializations' group.

```python
setup(
    ...,
    entry_points={
        "vulcan.materializations": [
            "my_materialization = my_package.my_materialization:CustomFullMaterialization",
        ],
    },
)
```

--------------------------------

### Stop All Services with Make

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/examples/exhaustive_model/README.md

This command stops and removes all services managed by Docker Compose in the project, ensuring a clean shutdown of the entire environment.

```bash
make all-down
```

--------------------------------

### Stop Vulcan Services with Make

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/examples/exhaustive_model/README.md

This command stops and removes the containers, networks, and volumes associated with the Vulcan services defined in the Docker Compose files.

```bash
make vulcan-down
```

--------------------------------

### SQL FULL Model Example with Cron and Grain

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/components/model/model_kinds.md

This SQL example demonstrates a FULL model with a cron schedule for daily execution and a specified grain. It calculates the number of orders per item. The subsequent SQL statements show how Vulcan creates a versioned table in the physical layer and a view in the virtual layer for this model.

```sql
MODEL (
  name demo.full_model_example,
  kind FULL,
  cron '@daily',
  grain item_id,
);

SELECT
  item_id,
  COUNT(DISTINCT id) AS num_orders
FROM demo.incremental_model
GROUP BY
  item_id

```

```sql
CREATE TABLE IF NOT EXISTS `vulcan-public-demo`.`vulcan__demo`.`demo__full_model_example__2345651858` (`item_id` INT64, `num_orders` INT64)

```

```sql
SELECT `incremental_model`.`item_id` AS `item_id`, COUNT(DISTINCT `incremental_model`.`id`) AS `num_orders`
FROM `vulcan-public-demo`.`vulcan__demo`.`demo__incremental_model__89556012` AS `incremental_model`
WHERE FALSE
GROUP BY `incremental_model`.`item_id` LIMIT 0

```

```sql
CREATE OR REPLACE TABLE `vulcan-public-demo`.`vulcan__demo`.`demo__full_model_example__2345651858` AS
SELECT CAST(`item_id` AS INT64) AS `item_id`, CAST(`num_orders` AS INT64) AS `num_orders`
FROM (SELECT `incremental_model`.`item_id` AS `item_id`, COUNT(DISTINCT `incremental_model`.`id`) AS `num_orders`
FROM `vulcan-public-demo`.`vulcan__demo`.`demo__incremental_model__89556012` AS `incremental_model`
GROUP BY `incremental_model`.`item_id`) AS `_subquery`

```

```sql
CREATE SCHEMA IF NOT EXISTS `vulcan-public-demo`.`demo__dev`

```

```sql
CREATE OR REPLACE VIEW `vulcan-public-demo`.`demo__dev`.`full_model_example` AS
SELECT * FROM `vulcan-public-demo`.`vulcan__demo`.`demo__full_model_example__2345651858`

```

--------------------------------

### Consume Vulcan REST API with cURL

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Example of how to query the semantic layer using the Vulcan REST API with cURL. It demonstrates a POST request with a JSON payload specifying measures and dimensions.

```bash
# Example: Query semantic layer
curl -X POST http://localhost:8000/api/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "query": {
      "measures": ["users.total_users"],
      "dimensions": ["users.plan_type"]
    }
  }'
```

--------------------------------

### Plan and Apply Production Updates with Vulcan CLI

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

This snippet demonstrates how to use the `vulcan plan` command to prepare and apply changes to the production environment. It shows the output of the planning phase, including detected model differences, and the prompt for applying the virtual update. The output confirms a virtual layer update without physical layer changes.

```bash
$ vulcan plan
======================================================================
Successfully Ran 1 tests against duckdb
----------------------------------------------------------------------

Differences from the `prod` environment:

Models:
├── Directly Modified:
│   └── vulcan_example.incremental_model
└── Indirectly Modified:
    └── vulcan_example.full_model

---

+++

@@ -14,6 +14,7 @@
 
 SELECT
   id,
   item_id,
+  'z' AS new_column,
   event_date
 FROM vulcan_example.seed_model
 WHERE

Directly Modified: vulcan_example.incremental_model (Non-breaking)
└── Indirectly Modified Children:
    └── vulcan_example.full_model (Indirect Non-breaking)
Apply - Virtual Update [y/n]: y

SKIP: No physical layer updates to perform

SKIP: No model batches to execute

Updating virtual layer  ━━━━━━━━━━━━━━━━━━━━━━━━ 100.0% • 2/2 • 0:00:00

✔ Virtual layer updated

```

--------------------------------

### Model Defaults Configuration Example

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/configurations/overview.md

Illustrates the required `model_defaults` section in a Vulcan configuration file, specifying the SQL dialect and optional parameters like owner, start date, and cron schedule for models.

```yaml
model_defaults:
  dialect: postgres     # Required
  owner: data-team
  start: 2024-01-01
  cron: '@daily'

```

--------------------------------

### Detailed Vulcan Plan Explanation Output

Source: https://github.com/tmdc-io/vulcan-book/blob/main/extra/getting_started/cli.md

This output illustrates the detailed explanation provided by the 'vulcan plan --explain' command. It breaks down the plan into three main stages: validating SQL and creating physical layer objects, backfilling models and running audits, and updating the virtual layer. Each stage lists the affected models and specific actions to be performed.

```bash
Explained plan
├── Validate SQL and create physical layer tables and views if they do not exist
│   ├── vulcan_example.seed_model -> db.vulcan__vulcan_example.vulcan_example__seed_model__2185867172
│   │   ├── Dry run model query without inserting results
│   │   └── Create table if it doesn't exist
│   ├── vulcan_example.full_model -> db.vulcan__vulcan_example.vulcan_example__full_model__2278521865
│   │   ├── Dry run model query without inserting results
│   │   └── Create table if it doesn't exist
│   └── vulcan_example.incremental_model -> db.vulcan__vulcan_example.vulcan_example__incremental_model__1880815781
│       ├── Dry run model query without inserting results
│       └── Create table if it doesn't exist
├── Backfill models by running their queries and run standalone audits
│   ├── vulcan_example.seed_model -> db.vulcan__vulcan_example.vulcan_example__seed_model__2185867172
│   │   └── Fully refresh table
│   ├── vulcan_example.full_model -> db.vulcan__vulcan_example.vulcan_example__full_model__2278521865
│   │   ├── Fully refresh table
│   │   └── Run 'assert_positive_order_ids' audit
│   └── vulcan_example.incremental_model -> db.vulcan__vulcan_example.vulcan_example__incremental_model__1880815781
│       └── Fully refresh table
└── Update the virtual layer for environment 'prod'
    └── Create or update views in the virtual layer to point at new physical tables and views
        ├── vulcan_example.full_model -> db.vulcan__vulcan_example.vulcan_example__full_model__2278521865
        ├── vulcan_example.seed_model -> db.vulcan__vulcan_example.vulcan_example__seed_model__2185867172
        └── vulcan_example.incremental_model -> db.vulcan__vulcan_example.vulcan_example__incremental_model__1880815781

```

--------------------------------

### Consume Vulcan GraphQL API with cURL

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_apis_guide.md

Example of how to query the Vulcan GraphQL API using cURL. It demonstrates a POST request with a JSON payload containing a GraphQL query to retrieve model names.

```bash
curl -X POST http://localhost:4000/graphql \
  -H "Content-Type: application/json" \
  -d '{ 
    "query": "query { models { name } }" 
  }'
```

--------------------------------

### SQL: Example INCREMENTAL_BY_UNIQUE_KEY Model Creation (BigQuery)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/components/model/model_kinds.md

This SQL sequence demonstrates the creation of an INCREMENTAL_BY_UNIQUE_KEY model in BigQuery. It shows the MODEL DDL, the SELECT statement for data retrieval, and the resulting `CREATE TABLE` statement generated by Vulcan, including the version fingerprint in the table name.

```sql
MODEL (
  name demo.incremental_by_unique_key_example,
  kind INCREMENTAL_BY_UNIQUE_KEY (
    unique_key id
  ),
  start '2020-01-01',
  cron '@daily',
);

SELECT
  id,
  item_id,
  event_date
FROM demo.seed_model
WHERE
  event_date BETWEEN @start_date AND @end_date

```

```sql
CREATE TABLE IF NOT EXISTS `vulcan-public-demo`.`vulcan__demo`.`demo__incremental_by_unique_key_example__1161945221` (`id` INT64, `item_id` INT64, `event_date` DATE)

```

--------------------------------

### Threshold Selection Example (YAML)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/components/checks/checks.md

Provides a two-step approach for selecting thresholds for data quality checks. It suggests starting with a conservative, wide range and then adjusting based on observed data over a monitoring period.

```yaml
# Step 1: Start with wide range
checks:
  analytics.orders:
    completeness:
      - row_count > 100:
          name: sufficient_orders_v1

# Step 2: Monitor for 30 days, see actual range: 5000-10000
```

--------------------------------

### Create and Apply Vulcan Plan

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/guides/get-started/zip-mac/vulcan-project/README.md

This command generates and applies a Vulcan plan, which defines data transformations and pipeline logic. It's a crucial step in building and deploying data analytics workflows with Vulcan.

```bash
vulcan plan
```

--------------------------------

### Start Vulcan Services

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/cursor/vulcan_errors_reference.md

Commands to ensure Vulcan API and related services are running. This involves setting up infrastructure and bringing the API service online. It also includes checking running Docker containers.

```bash
make setup
make vulcan-up
docker ps
```

--------------------------------

### Example SQL sequence for creating a VIEW model (BigQuery)

Source: https://github.com/tmdc-io/vulcan-book/blob/main/docs/components/model/model_kinds.md

This SQL demonstrates the sequence of commands Vulcan executes to create a versioned view in BigQuery. It includes creating the physical view with a version fingerprint and a virtual layer view pointing to it, along with schema creation.

```sql
MODEL (
  name demo.example_view,
  kind VIEW,
  cron '@daily'
);

SELECT
  'hello there' as a_column
```

```sql
CREATE OR REPLACE VIEW `vulcan-public-demo`.`vulcan__demo`.`demo__example_view__1024042926`
(`a_column`) AS SELECT 'hello there' AS `a_column`
```

```sql
CREATE SCHEMA IF NOT EXISTS `vulcan-public-demo`.`demo__dev`
```

```sql
CREATE OR REPLACE VIEW `vulcan-public-demo`.`demo__dev`.`example_view` AS
SELECT * FROM `vulcan-public-demo`.`vulcan__demo`.`demo__example_view__1024042926`
```

--------------------------------

### Vulcan CLI Commands Reference

Source: https://github.com/tmdc-io/vulcan-book/blob/main/supporting-docs/book/vulcan-lifecycle.md

A reference table of key Vulcan CLI commands, their purpose, and the phase they belong to. This includes commands for setup, initialization, development, testing, planning, running, querying, and debugging.

```bash
| Phase | Command | Purpose |
|-------|---------|---------|
| **Setup** | `make setup` | Start infrastructure |
| **Setup** | `alias vulcan=...` | Configure CLI |
| **Init** | `vulcan init` | Create project |
| **Init** | `vulcan info` | Verify setup |
| **Develop** | `vulcan lint` | Check code quality |
| **Test** | `vulcan test` | Run unit tests |
| **Plan** | `vulcan plan` | Create & apply changes |
| **Run** | `vulcan run` | Process new data |
| **Query** | `vulcan fetchdf` | Execute SQL queries |
| **Semantic** | `vulcan transpile` | Convert semantic to SQL |
| **API** | `make vulcan-up` | Start API services |
| **Debug** | `vulcan render` | See generated SQL |
```