### Local Development Setup with Docker Compose Source: https://github.com/astronomer/astronomer-cosmos/blob/main/AGENTS.md Start the local development environment using Docker Compose. ```bash docker compose -f dev/docker-compose.yaml up -d --build ``` -------------------------------- ### Local Development Setup with Virtual Environment Source: https://github.com/astronomer/astronomer-cosmos/blob/main/AGENTS.md Set up a local development environment using a virtual environment, installing test dependencies and making the package editable. Sets necessary environment variables for Airflow. ```bash python3 -m venv env && source env/bin/activate pip install "'.[test]'" pip install -e ".[dbt-postgres,dbt-bigquery]" export AIRFLOW_HOME=$(pwd)/dev/ export AIRFLOW__CORE__LOAD_EXAMPLES=false airflow standalone ``` -------------------------------- ### Basic Cosmos DAG with Local Execution Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/airflow-worker/local-execution-mode.md Example of setting up a basic Cosmos DAG using local execution mode. Ensure dbt dependencies are installed by setting 'install_deps' to True and use 'full_refresh' for supported dbt commands. ```python basic_cosmos_dag = DbtDag( # dbt/cosmos-specific parameters project_config=ProjectConfig(DBT_PROJECT_PATH), profile_config=profile_config, operator_args={ "install_deps": True, # install any necessary dependencies before running any dbt command "full_refresh": True, # used only in dbt commands that support this flag }, # normal dag parameters schedule="@daily", start_date=datetime(2023, 1, 1), catchup=False, dag_id="basic_cosmos_dag", default_args={"retries": 0}, ) ``` -------------------------------- ### Set up Python Virtual Environment and Install Dependencies Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/kubernetes.md Create a Python virtual environment, activate it, and install Astronomer Cosmos along with the necessary Kubernetes and dbt-postgres providers. ```bash python -m venv venv source venv/bin/activate pip install --upgrade pip pip install "astronomer-cosmos[dbt-postgres]" apache-airflow-providers-cncf-kubernetes ``` -------------------------------- ### Install pre-commit Hooks Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Install the pre-commit framework to automatically run checks before each commit. ```bash pre-commit install ``` -------------------------------- ### Start Airflow Locally with Astro CLI Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/astro-cli-quickstart.md Start your Airflow instance locally using the Astro CLI. The '--verbosity debug' flag provides detailed logs during the setup process. ```bash astro dev start --verbosity debug ``` -------------------------------- ### Install Google Cloud Provider for Cosmos Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/gcp-gke.md Install the necessary package to enable Google Cloud integration with Cosmos. ```bash pip install "astronomer-cosmos[google]" ``` -------------------------------- ### Install astronomer-cosmos Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/gcc.md Add the astronomer-cosmos package to your project's requirements.txt file to install the library. ```text astronomer-cosmos ``` -------------------------------- ### Install Apache Airflow and Astronomer Cosmos Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/docker.md Install Airflow and the Cosmos package with dbt-postgres support. Ensure pip is up-to-date before installation. ```bash python -m venv venv source venv/bin/activate pip install --upgrade pip pip install apache-airflow pip install "astronomer-cosmos[dbt-postgres]" ``` -------------------------------- ### Create MWAA Startup Script for dbt Installation Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/mwaa.md Use this shell script in MWAA's startup configuration to create a virtual environment and install your dbt adapter. Replace `` with the specific dbt package you need. ```shell #!/bin/sh export DBT_VENV_PATH="${AIRFLOW_HOME}/dbt_venv" python3 -m venv "${DBT_VENV_PATH}" ${DBT_VENV_PATH}/bin/pip install ``` -------------------------------- ### Install Cosmos and dbt-sqlite Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/oss-quickstart.md Install the astronomer-cosmos package and dbt-sqlite into your virtual environment. Airflow is automatically installed as a dependency. ```bash pip install astronomer-cosmos dbt-sqlite ``` -------------------------------- ### Clone Cosmos Example Repository Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/docker.md Clone the example repository to access Dockerfiles and dbt project structures needed for building the Docker image. ```bash git clone https://github.com/astronomer/cosmos-example.git cd cosmos-example ``` -------------------------------- ### Set Up Local Development Environment with Python venv Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Install Airflow and astronomer-cosmos within a Python virtual environment. This method requires setting Airflow home and disabling example DAGs. ```bash python3 -m venv env && source env/bin/activate pip3 install "apache-airflow[cncf.kubernetes,openlineage]" pip3 install -e ".[dbt-postgres,dbt-databricks]" ``` ```bash export AIRFLOW_HOME=$(pwd)/dev/ export AIRFLOW__CORE__LOAD_EXAMPLES=false ``` ```bash airflow standalone ``` -------------------------------- ### Install Hatch with Homebrew Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Use this command to install Hatch on macOS if you don't have it already. ```bash brew install hatch ``` -------------------------------- ### Install Airflow and Cosmos Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/aws-container-run-job.md Install Apache Airflow and Astronomer Cosmos with AWS support. The `aiobotocore` package is optional and required for deferred tasks. ```bash python3 -m venv venv source venv/bin/activate python3 -m pip install --upgrade pip pip install apache-airflow pip install "astronomer-cosmos[amazon]" pip install "aiobotocore[boto3]" ``` -------------------------------- ### Define a Simple Cosmos DAG Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/astro-cli-quickstart.md Define a simple DAG using Cosmos, specifying project configuration, profile configuration, and execution configuration. This example sets up a DAG with a daily schedule and a start date. ```python simple_dag = DbtDag( # dbt/cosmos-specific parameters project_config=ProjectConfig(jaffle_shop_path), profile_config=airflow_db, # The execution_config matches the dbt execution virtual environment defined in the Dockerfile execution_config=venv_execution_config, # normal dag parameters schedule="@daily", start_date=datetime(2023, 1, 1), catchup=False, dag_id="simple_dag", tags=["simple"], ) ``` -------------------------------- ### Install Airflow and Cosmos with Azure support Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/azure-container-instance.md Installs Apache Airflow and the Astronomer Cosmos library with specific support for dbt-Postgres and Azure Container Instance. ```bash python -m venv venv source venv/bin/activate pip install --upgrade pip pip install apache-airflow pip install "astronomer-cosmos[dbt-postgres,azure-container-instance]" ``` -------------------------------- ### Clone Example Cosmos Repository Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/kubernetes.md Clone the Astronomer Cosmos example repository to access sample Airflow DAGs and dbt projects. ```bash git clone https://github.com/astronomer/cosmos-example.git cd cosmos-example/ ``` -------------------------------- ### Install Airflow and Astronomer Cosmos Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/gcp-cloud-run-job.md Install Airflow and the Astronomer Cosmos package with dbt-bigquery and gcp-cloud-run-job providers. ```bash python3 -m venv venv source venv/bin/activate python3 -m pip install --upgrade pip pip install apache-airflow pip install "astronomer-cosmos[dbt-bigquery,gcp-cloud-run-job]" ``` -------------------------------- ### Install dbt in a Virtual Environment Dockerfile Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/open-source.md Use this Dockerfile snippet to install dbt and its adapter into a virtual environment. Replace `` with your specific dbt adapter package. ```docker FROM my-image:latest # install dbt into a virtual environment RUN python -m venv dbt_venv && source dbt_venv/bin/activate && \ pip install --no-cache-dir && deactivate ``` -------------------------------- ### Start Astro Development Environment Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/airflow-worker/async-execution-mode.md Launch your local Airflow environment using the Astro CLI. This command starts the webserver, scheduler, and triggerer, which is essential for deferrable operators. ```bash astro dev start ``` -------------------------------- ### Get Last Month Start Date (dbt_date) Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Returns the start date of the prior month. Timezone can be overridden. ```sql {{ dbt_date.last_month() }} as last_month_start_date ``` ```sql {{ dbt_date.last_month(tz="America/New_York") }} as last_month_start_date ``` -------------------------------- ### Get Day of Week Number with dbt_date Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Extracts the day of the week as a number, starting with 1. Defaults to ISO week (Monday start). ```sql {{ dbt_date.day_of_week("'2022-03-06'") }} as day_of_week_iso ``` ```sql {{ dbt_date.day_of_week("'2022-03-06'", isoweek=False) }} as day_of_week ``` -------------------------------- ### Create Project Directory Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/oss-quickstart.md Create and navigate into your demo project directory. ```bash mkdir oss-quickstart cd oss-quickstart ``` -------------------------------- ### Serve Cosmos Documentation Locally Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Build and serve the project's documentation locally for review. Requires Hatch to be set up. ```bash hatch run docs:serve ``` -------------------------------- ### Get Last Week Start Date (dbt_date) Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Provides the start date of the previous week (non-ISO standard). This function wraps `n_weeks_ago(1)`. Timezone can be overridden. ```sql {{ dbt_date.last_week() }} as last_week_start_date ``` ```sql {{ dbt_date.last_week(tz="America/New_York)) }} as last_week_start_date ``` -------------------------------- ### Serve Documentation Statically Source: https://github.com/astronomer/astronomer-cosmos/blob/main/AGENTS.md Serve the static build of the documentation on http://127.0.0.1:8000 without live reload. ```bash hatch run docs:serve-no-reload # static build served on http://127.0.0.1:8000 ``` -------------------------------- ### Build Documentation Source: https://github.com/astronomer/astronomer-cosmos/blob/main/AGENTS.md Build the project's HTML documentation into the docs/_build directory. ```bash hatch run docs:build # build HTML into docs/_build ``` -------------------------------- ### Get Next Month Start Date Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Returns the first day of the next month. Optionally specify a timezone. ```sql {{ dbt_date.next_month() }} as next_month_start_date ``` ```sql {{ dbt_date.next_month(tz="America/New_York") }} as next_month_start_date ``` -------------------------------- ### Get Next Week Start Date Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Returns the first day of the next week (non-ISO). Optionally specify a timezone. ```sql {{ dbt_date.next_week() }} as next_week_start_date ``` ```sql {{ dbt_date.next_week(tz="America/New_York") }} as next_week_start_date ``` -------------------------------- ### next_week Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Convenience function to get the start date of the next week (non-ISO). Wraps `n_weeks_away(1, tz)`. Optionally accepts a timezone argument. ```APIDOC ## next_week ### Description Convenience function to get the start date of the next week (non-ISO). ### Parameters #### Query Parameters - **tz** (string) - Optional - The timezone to use for the date calculation. Defaults to None. ### Usage ```sql {{ dbt_date.next_week() }} ``` ### Example with timezone ```sql {{ dbt_date.next_week(tz="America/New_York") }} ``` ``` -------------------------------- ### Set Up and Run Cosmos Integration Tests Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Environment variables and Hatch commands to set up and run integration tests, including database connection details. ```bash export AIRFLOW_HOME=`pwd` export AIRFLOW_CONN_AIRFLOW_DB=postgres://postgres:postgres@0.0.0.0:5432/postgres export DATABRICKS_HOST='' export DATABRICKS_TOKEN='' export DATABRICKS_WAREHOUSE_ID='' export DATABRICKS_CLUSTER_ID='' export POSTGRES_PORT=5432 export POSTGRES_SCHEMA=public export POSTGRES_DB=postgres export POSTGRES_PASSWORD=postgres export POSTGRES_USER=postgres export POSTGRES_HOST=localhost hatch run tests.py3.11-2.10-1.9:test-cov:test-integration-setup hatch run tests.py3.11-2.10-1.9:test-cov:test-integration ``` -------------------------------- ### last_week Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Convenience function to get the start date of last week (non-ISO). Wraps `n_weeks_ago(1, tz)`. Optionally overrides the default timezone. ```APIDOC ## last_week ### Description Convenience function to get the start date of last week (non-ISO). Wraps `n_weeks_ago(1, tz)`. Optionally overrides the default timezone. ### Parameters - **tz** (string) - Optional - The timezone to use for date calculations. Defaults to the system's local timezone. ### Usage Examples ```sql {{ dbt_date.last_week() }} as last_week_start_date {{ dbt_date.last_week(tz="America/New_York")) }} as last_week_start_date ``` ``` -------------------------------- ### Create and Activate Virtual Environment Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/oss-quickstart.md Set up and activate a Python virtual environment for your project. ```bash python3 -m venv venv source venv/bin/activate ``` -------------------------------- ### Serve Documentation Locally (No Reload) Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing-docs.md Use this command to serve the documentation locally without auto-reloading, which can help in identifying build errors more clearly. ```bash hatch run docs:serve-no-reload ``` -------------------------------- ### Serve Documentation with Live Reload Source: https://github.com/astronomer/astronomer-cosmos/blob/main/AGENTS.md Serve the documentation using sphinx-autobuild for live reloading during development. ```bash hatch run docs:serve # sphinx-autobuild with live reload ``` -------------------------------- ### Clone Repository and Navigate Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Clone the astronomer-cosmos repository and navigate into the directory. This is the initial step for both local development methods. ```bash git clone https://github.com/astronomer/astronomer-cosmos.git cd astronomer-cosmos/ ``` -------------------------------- ### Example Jaffle Shop DAG with Watcher Kubernetes Execution Mode Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.md This Python script defines an Airflow DAG that utilizes Cosmos to run a dbt project in Kubernetes pods. It configures dbt project paths, Kubernetes secrets for database credentials, and specifies the execution mode. This DAG is designed for environments where dbt is not installed on the Airflow worker. ```python import os from pathlib import Path from airflow.providers.cncf.kubernetes.secret import Secret from pendulum import datetime from cosmos import DbtDag from cosmos.config import ( ExecutionConfig, ProfileConfig, ProjectConfig, RenderConfig, ) from cosmos.constants import ExecutionMode, LoadMode DEFAULT_DBT_ROOT_PATH = Path(__file__).resolve().parent / "dbt" DBT_ROOT_PATH = Path(os.getenv("DBT_ROOT_PATH", DEFAULT_DBT_ROOT_PATH)) AIRFLOW_DBT_PROJECT_DIR = DBT_ROOT_PATH / "jaffle_shop" K8S_PROJECT_DIR = "dags/dbt/jaffle_shop" KBS_DBT_PROFILES_YAML_FILEPATH = Path(K8S_PROJECT_DIR) / "profiles.yml" DBT_IMAGE = "dbt-jaffle-shop:1.0.0" project_seeds = [{"project": "jaffle_shop", "seeds": ["raw_customers", "raw_payments", "raw_orders"]}] postgres_password_secret = Secret( deploy_type="env", deploy_target="POSTGRES_PASSWORD", secret="postgres-secrets", key="password", ) postgres_host_secret = Secret( deploy_type="env", deploy_target="POSTGRES_HOST", secret="postgres-secrets", key="host", ) operator_args = { "deferrable": False, "image": DBT_IMAGE, "get_logs": True, "is_delete_operator_pod": False, "log_events_on_failure": True, "secrets": [postgres_password_secret, postgres_host_secret], "env_vars": { "POSTGRES_DB": "postgres", "POSTGRES_SCHEMA": "public", "POSTGRES_USER": "postgres", "POSTGRES_PORT": "5432", }, "retry": 0, } profile_config = ProfileConfig( profile_name="default", target_name="dev", profiles_yml_filepath=KBS_DBT_PROFILES_YAML_FILEPATH ) project_config = ProjectConfig( project_name="jaffle_shop", manifest_path=AIRFLOW_DBT_PROJECT_DIR / "target/manifest.json", ) render_config = RenderConfig(load_method=LoadMode.DBT_MANIFEST) # Currently airflow dags test ignores priority_weight and weight_rule, for this reason, we're setting the following in the CI only: if os.getenv("CI"): operator_args["trigger_rule"] = "all_success" dag = DbtDag( dag_id="jaffle_shop_watcher_kubernetes", start_date=datetime(2022, 11, 27), doc_md=__doc__, catchup=False, # Cosmos-specific parameters: project_config=project_config, profile_config=profile_config, render_config=render_config, execution_config=ExecutionConfig( execution_mode=ExecutionMode.WATCHER_KUBERNETES, dbt_project_path=K8S_PROJECT_DIR, ), operator_args=operator_args, ) ``` -------------------------------- ### Install Compatible Kubernetes Provider Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.md Ensure you have a compatible version of the apache-airflow-providers-cncf-kubernetes installed. This command installs a version greater than 10.7.0. ```bash pip install "apache-airflow-providers-cncf-kubernetes>10.7.0" ``` -------------------------------- ### Set Up Local Development Environment with Docker Compose Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Build and launch the development environment using Docker Compose. This method requires creating specific directories and setting ownership for Linux users. ```bash mkdir -p dev/dags dev/logs dev/plugins sudo chown 50000:50000 -R dev/dags dev/logs dev/plugins ``` ```bash docker compose -f dev/docker-compose.yaml up -d --build ``` -------------------------------- ### Calculate ISO Week Start Date with dbt_date Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Computes the date of the week's start using ISO format (Monday start). Optionally override the default timezone. ```sql {{ dbt_date.iso_week_start("date_col") }} as iso_week_start_date ``` ```sql {{ dbt_date.iso_week_start("date_col", tz="America/New_York") }} as iso_week_start_date ``` -------------------------------- ### Install dbt Fusion in Astro Dockerfile Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/dbt_setup/dbt-fusion.md Add these lines to your Astro project's Dockerfile to install the dbt Fusion package. This involves downloading and running the dbt Fusion installation script. ```dockerfile USER root RUN apt install -y curl RUN curl -fsSL https://public.cdn.getdbt.com/fs/install/install.sh | sh -s -- --update ``` -------------------------------- ### Build the Cosmos Project Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/contributing.md Compile the project for distribution. This command is used after setting up Hatch. ```bash hatch build ``` -------------------------------- ### Import and Initialize StandardSQLServerAuthProfileMapping Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/reference/profiles/StandardSQLServerAuth.md Import the StandardSQLServerAuthProfileMapping class and initialize it with a connection ID and optional profile arguments. ```python from cosmos.profiles import StandardSQLServerAuthProfileMapping profile = StandardSQLServerAuthProfileMapping( conn_id = 'my_generic_connection', profile_args = { ... }, ) ``` -------------------------------- ### Initialize dbt_project.yml Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/getting_started/oss-quickstart.md Create the dbt project configuration file. This file defines the project's name, version, profile, and model paths. ```bash touch dbt_project/micro_project/dbt_project.yml ``` ```yaml name: 'micro_project' version: '1.0' profile: 'micro_project' model-paths: ["models"] ``` -------------------------------- ### Initialize MysqlUserPasswordProfileMapping Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/reference/profiles/MysqlUserPassword.md Import and initialize the MysqlUserPasswordProfileMapping with a connection ID and optional profile arguments. ```python from cosmos.profiles import MysqlUserPasswordProfileMapping profile = MysqlUserPasswordProfileMapping( conn_id = 'my_mysql_connection', profile_args = { ... }, ) ``` -------------------------------- ### Get Current Timestamp Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Gets the current timestamp based on the specified local timezone. Defaults to 'America/Los_Angeles'. ```sql {{ dbt_date.now() }} ``` ```sql {{ dbt_date.now("America/New_York") }} ``` -------------------------------- ### Import and Instantiate ExasolUserPasswordProfileMapping Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/reference/profiles/ExasolUserPassword.md Import the `ExasolUserPasswordProfileMapping` class from `cosmos.profiles` and instantiate it with a connection ID and optional profile arguments. ```python from cosmos.profiles import ExasolUserPasswordProfileMapping profile = ExasolUserPasswordProfileMapping( conn_id = 'my_exasol_connection', profile_args = { ... }, ) ``` -------------------------------- ### Create Kind Kubernetes Cluster Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/kubernetes.md Use Kind to set up a local Kubernetes cluster for testing and development. ```bash kind create cluster ``` -------------------------------- ### Install Cosmos 1.11.0a1 Alpha Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/dbt_setup/dbt-fusion.md Install the pre-release version of astronomer-cosmos that includes initial support for dbt Fusion. ```bash pip install astronomer-cosmos --pre ``` -------------------------------- ### Get Today's Date Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Gets the current date based on the local timezone. Optionally specify a timezone. ```sql {{ dbt_date.today() }} ``` ```sql {{ dbt_date.today("America/New_York") }} ``` -------------------------------- ### Install Shared dbt Package Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/multi_project/multi-project.md Install the shared dbt package in other projects using either a Git repository or a local path for development. ```yaml packages: # From git repository - git: "https://github.com/your-org/company-shared-macros.git" revision: v1.0.0 # Or from local path (for development) - local: ../shared_macros ``` -------------------------------- ### Import and Initialize TrinoLDAPProfileMapping Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/reference/profiles/TrinoLDAP.md Import the TrinoLDAPProfileMapping class and initialize it with an Airflow connection ID and optional profile arguments. ```python from cosmos.profiles import TrinoLDAPProfileMapping profile = TrinoLDAPProfileMapping( conn_id = 'my_trino_connection', profile_args = { ... }, ) ``` -------------------------------- ### Get Tomorrow's Date Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Gets tomorrow's date based on the local date. Optionally specify a timezone or an anchor date. ```sql {{ dbt_date.tomorrow() }} ``` ```sql {{ dbt_date.tomorrow(tz="America/New_York") }} as date_tomorrow ``` ```sql {{ dbt_date.tomorrow(date="date_col", tz="America/New_York") }} as date_tomorrow ``` -------------------------------- ### Install dbt Dependencies and List Models Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/faq.md Run these commands before deployment to generate dbt artifacts that Cosmos can reuse. This can be done during container image build or via deployment tools. ```bash dbt deps dbt ls ``` -------------------------------- ### Install Kubernetes Provider for Deferrable Mode Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.md To enable deferrable mode for the producer node, install a compatible version of the Airflow Kubernetes provider (>=10.12.2). ```bash pip install "apache-airflow-providers-cncf-kubernetes>=10.12.2" ``` -------------------------------- ### Compute Week Start Date Source: https://github.com/astronomer/astronomer-cosmos/blob/main/dev/dags/dbt/simple/dbt_packages/dbt_date/README.md Computes the week starting date using US format (Sunday). Defaults to today if no date is specified. Timezone can be overridden. ```sql {{ dbt_date.week_start() }} as week_start ``` ```sql {{ dbt_date.week_start("date_col") }} as week_start ``` ```sql {{ dbt_date.week_start("date_col", tz="America/New_York") }} as week_start ``` -------------------------------- ### Import and Initialize PostgresUserPasswordProfileMapping Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/reference/profiles/PostgresUserPassword.md Import the PostgresUserPasswordProfileMapping and initialize it with an Airflow connection ID and optional profile arguments. ```python from cosmos.profiles import PostgresUserPasswordProfileMapping profile = PostgresUserPasswordProfileMapping( conn_id = 'my_postgres_connection', profile_args = { ... }, ) ``` -------------------------------- ### Import and Instantiate SnowflakeEncryptedPrivateKeyFilePemProfileMapping Source: https://github.com/astronomer/astronomer-cosmos/blob/main/docs/reference/profiles/SnowflakeEncryptedPrivateKeyFilePem.md Import the mapping class from cosmos.profiles and instantiate it with your Airflow connection ID and any additional profile arguments. ```python from cosmos.profiles import SnowflakeEncryptedPrivateKeyFilePemProfileMapping profile = SnowflakeEncryptedPrivateKeyFilePemProfileMapping( conn_id = 'my_snowflake_connection', profile_args = { ... }, ) ```