### Install Documentation Dependencies Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Install the necessary dependencies for building and previewing the documentation using `uv`. ```bash uv sync --group=docs ``` -------------------------------- ### Install Playwright and Chromium Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/assets/brand/scripts/README.md Install Playwright and its Chromium browser for rendering PNG assets. ```bash pip install playwright && playwright install chromium ``` -------------------------------- ### Install dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Installs the dbt-bouncer project using the provided Makefile. Ensure Python 3.11+ and uv are installed. ```bash make install ``` -------------------------------- ### Install dbt-Bouncer and prek Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Installs dbt-Bouncer, its dependencies, and prek within a virtual environment. Changes to local source code are reflected immediately. ```shell make install uv run prek install ``` -------------------------------- ### Run dbt-Bouncer with example configuration Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Executes the dbt-Bouncer script using a provided example configuration file. Ensure your virtualenv is activated and the script path is correct. ```shell uv run dbt-bouncer --config-file dbt-bouncer-example.yml ``` -------------------------------- ### dbt-bouncer YAML Configuration Example Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md Example configuration for dbt-bouncer in YAML format. Specify artifact directories and manifest checks. ```yaml # [Optional] Directory where the dbt artifacts exists, generally the `target` directory inside a dbt project. Defaults to `$DBT_PROJECT_DIR/target` if $DBT_PROJECT_DIR is set, else `./target`. dbt_artifacts_dir: target manifest_checks: - name: check_macro_name_matches_file_name - name: check_model_names include: ^models/staging model_name_pattern: ^stg_ ``` -------------------------------- ### Install dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/README.md Install the dbt-bouncer package using pip. This is the first step to using the tool. ```shell pip install dbt-bouncer ``` -------------------------------- ### dbt-bouncer pyproject.toml Configuration Example Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md Example configuration for dbt-bouncer within a `pyproject.toml` file. Uses the `[tool.dbt-bouncer]` section. ```toml [tool.dbt-bouncer] # [Optional] Directory where the dbt artifacts exists, generally the `target` directory inside a dbt project. Defaults to `$DBT_PROJECT_DIR/target` if $DBT_PROJECT_DIR is set, else `./target`. dbt_artifacts_dir = "target" [[tool.dbt-bouncer.manifest_checks]] name = "check_macro_name_matches_file_name" [[tool.dbt-bouncer.manifest_checks]] name = "check_model_names" include = "^models/staging" model_name_pattern = "^stg_" ``` -------------------------------- ### dbt-bouncer TOML Configuration Example Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md Example configuration for dbt-bouncer in TOML format. Similar to YAML, it defines artifact directories and manifest checks. ```toml # [Optional] Directory where the dbt artifacts exists, generally the `target` directory inside a dbt project. Defaults to `$DBT_PROJECT_DIR/target` if $DBT_PROJECT_DIR is set, else `./target`. dbt_artifacts_dir = "target" [[manifest_checks]] name = "check_macro_name_matches_file_name" [[manifest_checks]] name = "check_model_names" include = "^models/staging" model_name_pattern = "^stg_" ``` -------------------------------- ### dbt-bouncer verbose output example Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Example of verbose output from dbt-bouncer, showing detailed debug messages including the configuration file path and source. ```shell INFO: Running dbt-bouncer (0.0.0)... DEBUG: config_file=PosixPath('dbt-bouncer-example.yml') DEBUG: config_file_source='COMMANDLINE' DEBUG: Config file passed via command line: dbt-bouncer-example.yml DEBUG: Loading config from /home/user/dbt-bouncer/dbt-bouncer-example.yml... INFO: Loaded config from dbt-bouncer-example.yml... ``` -------------------------------- ### Run dbt-bouncer with a config file Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Execute dbt-bouncer, specifying the path to your configuration file. This is a common way to run the tool after installation. ```shell dbt-bouncer --config-file ``` -------------------------------- ### dbt-bouncer basic execution output Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Example output when running dbt-bouncer with a configuration file. Shows the loading of configuration and validation status. ```shell Running dbt-bouncer (X.X.X)... Loaded config from dbt-bouncer-example.yml... Validating conf... ``` -------------------------------- ### dbt-bouncer CLI Feature Examples Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/migration/v3.md Showcases new CLI features for listing checks in JSON format and performing dry runs. ```bash dbt-bouncer list --output-format json ``` ```bash dbt-bouncer run --dry-run ``` -------------------------------- ### Migrating Custom Checks: v2.x Class-Based Check Example Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/migration/v3.md Provides an example of a custom check class in dbt-bouncer v2.x, demonstrating subclassing BaseCheck and implementing the execute method. ```python from dbt_bouncer.checks.common import BaseCheck class CheckModelAccess(BaseCheck): access: Literal["private", "protected", "public"] model: DbtBouncerModelBase | None = Field(default=None) name: Literal["check_model_access"] def execute(self) -> None: assert self.model is not None if self.model.access.value != self.access: raise DbtBouncerFailedCheckError( f"Model `{self.model.name}` has access `{self.model.access.value}`, expected `{self.access}`." ) ``` -------------------------------- ### Skipping dbt-bouncer Checks for a Source Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Example of using the 'meta' config in a dbt source to skip specific dbt-bouncer checks. ```yaml version: 2 sources: - name: source_system tables: - name: source_1 config: meta: dbt-bouncer: skip_checks: - check_source_description_populated - check_source_has_meta_keys - check_source_has_tags - check_source_names reason: We recommend documenting why these checks are being skipped. ``` -------------------------------- ### Run dbt-bouncer using uvx Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Execute dbt-bouncer as a standalone Python executable using uvx. This is useful for environments without a persistent Python installation. ```shell uvx dbt-bouncer --config-file ``` -------------------------------- ### dbt-bouncer artifact summary table Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Example output displaying a summary table of parsed dbt artifacts after running dbt-bouncer. It categorizes and counts resources like exposures, macros, and nodes. ```shell Parsed artifacts for 'dbt_bouncer_test_project' ╭──────────────────┬─────────────────┬───────╮ │ Artifact │ Category │ Count │ ├──────────────────┼─────────────────┼───────┤ │ manifest.json │ Exposures │ 2 │ │ │ Macros │ 3 │ │ │ Nodes │ 12 │ │ │ Seeds │ 3 │ │ │ Semantic Models │ 1 │ │ │ Snapshots │ 2 │ │ │ Sources │ 4 │ │ │ Tests │ 36 │ │ │ Unit Tests │ 3 │ │ catalog.json │ Nodes │ 13 │ │ │ Sources │ 0 │ │ run_results.json │ Results │ 51 │ ╰──────────────────┴─────────────────┴───────╯ ``` -------------------------------- ### Example dry run output for dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md This is an example of the output generated when using the --dry-run flag with the dbt-bouncer run command, showing checks that would execute. ```text ╭─ Dry run — checks that would execute ─────────────────────╮ │ Check name │ Resource type │ Count │ │ CheckModelNamePattern │ model │ 1234 │ │ CheckModelDescriptionPopulated │ model │ 1234 │ │ CheckSourceDescriptionPopulated│ source │ 56 │ ╰────────────────────────────────────────────────────────────╯ Dry run complete. 2524 check(s) would run. ``` -------------------------------- ### Example valid config output for dbt-bouncer validate Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md This is the output displayed when the dbt-bouncer validate command successfully verifies a configuration file. ```text Config file is valid! ``` -------------------------------- ### Example invalid config output for dbt-bouncer validate Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md This is an example of the output from dbt-bouncer validate when issues are found in the configuration file, indicating line numbers and error types. ```text Found 2 issue(s) in config file: Line 1: Check is missing required 'name' field Line 3: YAML syntax error: ... ``` -------------------------------- ### Run dbt-Bouncer with Docker Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Execute dbt-Bouncer using Docker for isolated and reproducible runs without requiring a local Python installation. Mount your project directory to the container and specify the configuration file path. ```shell docker run --rm \ --volume "$PWD":/app \ ghcr.io/godatadriven/dbt-bouncer:vX.X.X \ --config-file /app/ ``` -------------------------------- ### dbt-bouncer Configuration for Monorepo Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Configure dbt-bouncer from the root directory of a monorepo using `include` and `exclude` directives. This example targets the 'dbt-project' directory. ```yaml dbt_artifacts_dir: dbt-project/target include: ^dbt-project manifest_checks: - name: check_exposure_based_on_non_public_models ``` -------------------------------- ### Skipping dbt-bouncer Checks for a Model Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Example of using the 'meta' config in a dbt model to skip specific dbt-bouncer checks. ```yaml models: - name: my_model config: meta: dbt-bouncer: skip_checks: - check_model_description_populated - check_model_has_meta_keys reason: We recommend documenting why these checks are being skipped. ``` -------------------------------- ### Python Model Manifest Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Enforce conventions on Python models using manifest checks. This example checks for specific meta keys and discourages memory-intensive operations like .to_pandas(). ```yaml manifest_checks: - name: check_model_has_meta_keys keys: - maturity - name: check_model_description_populated - name: check_model_code_does_not_contain_regexp_pattern regexp_pattern: .*.to_pandas (\) # Discourage memory-intensive operations ``` -------------------------------- ### Regenerate PNGs with Playwright Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/assets/brand/scripts/README.md Use this script to regenerate PNG assets. It leverages Playwright for pixel-perfect rendering, starting local HTTP servers, generating HTML templates, and rendering pages at specific viewport dimensions. ```bash python render_pngs.py ``` -------------------------------- ### dbt-bouncer CI Pipeline Configuration Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Configure dbt-bouncer checks for a CI pipeline, including catalog, manifest, and run results checks. This setup targets specific directories and execution times. ```yaml catalog_checks: - name: check_column_description_populated include: ^models/marts manifest_checks: - name: check_model_directories include: ^models permitted_sub_directories: - intermediate - marts - staging - utilities run_results_checks: - name: check_run_results_max_execution_time max_execution_time_seconds: 10 ``` -------------------------------- ### Preview Documentation Locally Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Run the Zensical development server to preview the documentation site locally. The server rebuilds on file changes. ```bash uv run zensical serve ``` -------------------------------- ### Build Static Documentation Site Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Generate the static HTML files for the documentation site using Zensical. The output is written to the `site/` directory. ```bash uv run zensical build ``` -------------------------------- ### Initialize dbt-bouncer Configuration Source: https://github.com/godatadriven/dbt-bouncer/blob/main/README.md Create a dbt-bouncer.yml configuration file using the 'dbt-bouncer init' command. This sets up the conventions for your project. ```shell dbt-bouncer init ``` -------------------------------- ### Equivalent dbt-bouncer CLI Commands Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/migration/v3.md Demonstrates equivalent commands for running checks in dbt-bouncer, showing the new explicit 'run' subcommand. ```bash # These are equivalent: dbt-bouncer --config-file dbt-bouncer.yml dbt-bouncer run --config-file dbt-bouncer.yml ``` -------------------------------- ### Run dbt-bouncer performance benchmarks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Executes performance benchmarks using bencher and hyperfine tools. ```bash make test-perf ``` -------------------------------- ### Create Python Virtual Environment with uv Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Use this command to create a new Python virtual environment in the root of the dbt-bouncer repository. This is recommended for local development. ```shell uv venv ``` -------------------------------- ### Run Unit Tests Command Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Execute unit tests to ensure the correctness of your checks. This command runs tests located in the `tests/unit/checks/` directory. ```bash make test-unit ``` -------------------------------- ### Run specific pytest tests Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Executes specific unit tests or groups of tests using pytest directly. Ensure your virtualenv is active and dev dependencies are installed. ```shell # run all unit tests in a file uv run pytest ./tests/unit/checks/catalog/test_columns.py # run a specific unit test uv run pytest ./tests/unit/checks/catalog/test_columns.py::test_check_columns_are_documented_in_public_models ``` -------------------------------- ### CI Pipeline with dbt Cloud Artifact Download Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md This YAML configuration demonstrates how to set up a GitHub Actions CI pipeline to download dbt Cloud artifacts and then run dbt-bouncer. Ensure you have the necessary secrets configured in your GitHub repository. ```yaml name: CI pipeline on: pull_request: branches: - main jobs: download-artifacts: runs-on: ubuntu-latest permissions: pull-requests: write steps: - name: Checkout uses: actions/checkout@v6 - name: Download dbt artifacts uses: pgoslatara/dbt-cloud-download-artifacts-action@v1 with: commit-sha: ${{ github.event.pull_request.head.sha }} dbt-cloud-api-token: ${{ secrets.DBT_CLOUD_API_TOKEN }} - name: Run dbt-bouncer uses: godatadriven/dbt-bouncer@vX.X ``` -------------------------------- ### Show all failures with dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Use the --show-all-failures flag to print all failures to the console, even when an output file is specified. ```bash dbt-bouncer run --show-all-failures ``` -------------------------------- ### Register Custom Checks via Entry Points Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/migration/v3.md Register your custom check package using the `dbt_bouncer.checks` entry point in your `pyproject.toml` file. ```toml # pyproject.toml of your custom checks package [project.entry-points."dbt_bouncer.checks"] my_checks = "my_package.checks" ``` -------------------------------- ### Configure Include/Exclude Patterns for dbt-Bouncer Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md Use 'include' and 'exclude' arguments with regular expressions to filter which resources a check applies to. These patterns are based on the original file paths. ```yaml # Specify `include` at the check level only manifest_checks: - name: check_model_names include: ^models/staging model_name_pattern: ^stg_ ``` ```yaml # Specify `include` at the check and global levels include: ^models/marts manifest_checks: - name: check_model_names include: ^models/staging model_name_pattern: ^stg_ ``` ```yaml # Specify `include` at the global level only include: ^models/staging manifest_checks: - name: check_model_names model_name_pattern: ^stg_ ``` -------------------------------- ### Custom Check Python File Example Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Defines a custom check function using the `@check` decorator to validate model naming conventions against a regex pattern. This snippet requires the `re` module and `check`, `fail` from `dbt_bouncer.check_decorator`. ```python import re from dbt_bouncer.check_decorator import check, fail @check def check_model_naming_convention(model, *, model_name_pattern: str = "^(stg|int|fct|dim)_"): """Model names must match the supplied regex.""" if not re.match(model_name_pattern, str(model.name)): fail( f"`{model.unique_id}` does not match the required pattern " f"`{model_name_pattern}`." ) ``` -------------------------------- ### Manifest Check: Snapshots Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_snapshots.md This section covers the manifest checks for snapshots. These checks require the `manifest.json` file to be present. ```APIDOC ## Manifest Check: Snapshots ### Description This check verifies snapshot configurations within the dbt project's manifest. ### Method N/A (This is a check, not an API endpoint) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response #### Success Response (200) N/A (This is a check, not an API endpoint) #### Response Example N/A ### Notes Requires `manifest.json` to be present. ``` -------------------------------- ### Define a dbt-Bouncer check using the decorator API Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Example of adding a new check to dbt-Bouncer using the recommended decorator API. The `@check` decorator infers check name, iteration targets, and parameters from the function signature. ```python @check def my_new_check(model, param1, *, param2): """This is my new check.""" # ... implementation ... ``` -------------------------------- ### Running dbt-bouncer with Centralized Config Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Command to execute dbt-bouncer using a configuration file located within a git submodule. ```shell dbt-bouncer --config-file dbt-bouncer-config/dbt-bouncer.yml ``` -------------------------------- ### Register External Checks via Entry Points Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md External packages can register checks by defining an entry point in their `pyproject.toml`. dbt-bouncer discovers these checks automatically. ```toml # pyproject.toml of your plugin package [project.entry-points."dbt_bouncer.checks"] my_checks = "my_package.checks" ``` -------------------------------- ### Lineage API Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/sources/lineage.md The lineage API checks require the manifest.json file to be present. ```APIDOC ## Lineage API ### Description This API provides lineage information and requires the `manifest.json` file to be available. ### Method GET ### Endpoint /manifest/sources/lineage ### Parameters #### Query Parameters - **manifest_json** (file) - Required - The manifest.json file. ### Response #### Success Response (200) - **lineage_data** (object) - The lineage information. #### Response Example ```json { "lineage_data": { "nodes": {}, "links": [] } } ``` ``` -------------------------------- ### Generate manifest.json Source: https://github.com/godatadriven/dbt-bouncer/blob/main/README.md If a manifest.json file is not present, run 'dbt parse' to generate it. This file is required by dbt-bouncer. ```shell dbt parse ``` -------------------------------- ### Run dbt-bouncer pre-commit hooks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Runs all pre-commit hooks for the project. Note that this command is specific and not the standard 'pre-commit run'. ```bash prek run --all-files ``` -------------------------------- ### Run dbt-Bouncer tests in dev container Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Alternative command for running all tests, specifically intended for use within development containers. ```shell make test-dev-container ``` -------------------------------- ### Run all dbt-bouncer tests Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Executes all unit and integration tests for the dbt-bouncer project. ```bash make test ``` -------------------------------- ### Run dbt-bouncer Manifest Checks in CI Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Execute only manifest checks in a CI environment by running `dbt parse` first, then `dbt-bouncer --only manifest_checks`. ```shell dbt-bouncer --only manifest_checks ``` -------------------------------- ### dbt-Bouncer Path Normalization on Linux/Mac Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md On Linux and Mac, paths like `models/staging/crm/model_1.sql` are used directly for 'exclude' and 'include' arguments in dbt-bouncer. ```shell models/staging/crm/model_1.sql ``` -------------------------------- ### Build and run dbt-bouncer end-to-end validation Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Performs an end-to-end validation of the dbt-bouncer project. ```bash make build-and-run-dbt-bouncer ``` -------------------------------- ### Run dbt-bouncer Catalog Checks in CI Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Execute only catalog checks in a CI environment by running `dbt run --empty` to materialize models, then `dbt-bouncer --only catalog_checks`. ```shell dbt-bouncer --only catalog_checks ``` -------------------------------- ### List Available dbt-bouncer Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md The `list` command displays all available dbt-bouncer checks, categorized for clarity. Use `--output-format json` for machine-readable output. ```bash dbt-bouncer list ``` ```bash dbt-bouncer list --output-format json ``` -------------------------------- ### Set output format for dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Control the output format using --output-format. Options include csv, json, junit, sarif, and tap. Defaults to json. ```bash dbt-bouncer run --output-format json ``` ```bash dbt-bouncer run --output-format junit --output-file results.xml ``` ```bash dbt-bouncer run --output-format sarif --output-file results.sarif ``` -------------------------------- ### Run dbt-bouncer checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Execute dbt-bouncer checks against your dbt project using a specified configuration file. This is the primary command for running checks. ```bash dbt-bouncer run --config-file dbt-bouncer.yml ``` -------------------------------- ### Loader Requirements Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/sources/loader.md The loader functionality requires the `manifest.json` file to be present in the project directory. ```APIDOC ## Loader !!! note The below checks require `manifest.json` to be present. ::: manifest.sources.loader ``` -------------------------------- ### Specify custom config file for dbt-bouncer run Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Use the --config-file option to specify a custom location for the dbt-bouncer YAML configuration file when running checks. ```bash dbt-bouncer run --config-file config/checks.yml ``` -------------------------------- ### Configure dbt-bouncer local pre-commit hook Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Set up dbt-bouncer as a local pre-commit hook. This requires dbt-bouncer to be available in your environment and runs it as a system command. ```yaml - repo: local hooks: - id: dbt-bouncer name: dbt-bouncer entry: dbt-bouncer # --config-file language: system pass_filenames: false always_run: true ``` -------------------------------- ### Specify output file for dbt-bouncer results Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Use the --output-file option to specify the location where check metadata will be saved. Results are written to stdout if not provided. ```bash dbt-bouncer run --output-file results/check-results.json ``` -------------------------------- ### dbt-Bouncer Path Normalization on Windows Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md dbt-bouncer normalizes Windows paths (e.g., `models\\staging\\crm\\model_1.sql`) to the Linux/Mac format (e.g., `models/staging/crm/model_1.sql`) for 'exclude' and 'include' arguments. ```shell models\\staging\\crm\\model_1.sql ``` -------------------------------- ### Catalog Sources Check Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/catalog/check_catalog_sources.md This check verifies catalog sources and requires both catalog.json and manifest.json to be present. ```APIDOC ## Catalog Sources Check ### Description This check verifies catalog sources and requires both `catalog.json` and `manifest.json` to be present. ### Method N/A (This is a check, not an API endpoint) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### Adding dbt-bouncer Config as Git Submodule Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Command to add a centralized dbt-bouncer configuration repository as a git submodule. ```shell git submodule add git@github.com:/dbt-bouncer-config.git ``` -------------------------------- ### Neovim YAML Language Server Schema Comment Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md Add a schema comment to the top of your `dbt-bouncer.yml` file for Neovim with yaml-language-server. ```yaml # yaml-language-server: $schema=https://raw.githubusercontent.com/godatadriven/dbt-bouncer/main/schema.json ``` -------------------------------- ### Regenerate test fixtures for dbt versions Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Regenerates test fixtures for dbt versions 1.10, 1.11, and 1.12. ```bash make build-artifacts ``` -------------------------------- ### Running Tests with manifest.json Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/models/tests.md The tests provided by dbt-bouncer require the manifest.json file to be present in the dbt project. This file contains metadata about your dbt project. ```APIDOC ## Tests ### Description This section outlines the requirements and usage for running tests within the dbt-bouncer project. Specifically, it highlights the dependency on the `manifest.json` file. ### Prerequisites - `manifest.json`: This file must be present in your dbt project's root directory. It is generated by dbt during a `dbt compile` or `dbt run` command. ### Usage To utilize the tests provided by `manifest.models.tests`, ensure that `manifest.json` is available. The exact command or method to run these tests will depend on your dbt project setup and how dbt-bouncer is integrated. ``` -------------------------------- ### Run dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/README.md Execute dbt-bouncer to validate your dbt project against configured conventions. The output shows a summary of parsed artifacts and any failed checks. ```text $ dbt-bouncer run Running dbt-bouncer (X.X.X)... Loaded config from dbt-bouncer-example.yml... Validating conf... Parsed artifacts for 'dbt_bouncer_test_project' ╭──────────────────┬─────────────────┬───────╮ │ Artifact │ Category │ Count │ ├──────────────────┼─────────────────┼───────┤ │ manifest.json │ Exposures │ 2 │ │ │ Macros │ 3 │ │ │ Nodes │ 12 │ │ │ Seeds │ 3 │ │ │ Semantic Models │ 1 │ │ │ Snapshots │ 2 │ │ │ Sources │ 4 │ │ │ Tests │ 36 │ │ │ Unit Tests │ 3 │ │ catalog.json │ Nodes │ 13 │ │ │ Sources │ 0 │ │ run_results.json │ Results │ 51 │ ╰──────────────────┴─────────────────┴───────╯ Assembled 463 checks, running... Running checks... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% `dbt-bouncer` failed. Please see below for more details or run `dbt-bouncer` with the `-v` flag. Failed checks ╭────────────────────────────────────┬────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ Check name │ Severity │ Failure message │ ├────────────────────────────────────┼────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤ │ check_model_names:48:orders │ ERROR │ Models in the staging layer should always start with "stg_". - `orders` does not match the supplied regex `^stg_`. │ ╰────────────────────────────────────┴────────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ Done. SUCCESS=462 WARN=0 ERROR=1 ``` -------------------------------- ### Manifest Check: Seeds Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_seeds.md This section covers the manifest checks for seeds. These checks require the `manifest.json` file to be present. ```APIDOC ## Manifest Check: Seeds ### Description This check verifies the integrity and structure of seed files within your dbt project's manifest. !!! note The below checks require `manifest.json` to be present. ### Method N/A (This is a check, not an API endpoint) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A (This describes a check, not a request/response) ## manifest.check_seeds ### Description This specific check, `manifest.check_seeds`, is part of the manifest checks for seeds. It performs detailed validation on seed-related information within the `manifest.json` file. ### Method N/A (This is a check, not an API endpoint) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### Run dbt-bouncer in verbose mode Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Execute dbt-bouncer with verbose logging enabled using the -v flag. This provides detailed debug information about the configuration loading process. ```shell dbt-bouncer --config-file -v ``` -------------------------------- ### Write Tests for Context-Dependent Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md For checks that depend on context (e.g., model documentation coverage), use `ctx_*` keyword arguments with `check_passes`/`check_fails` to provide the necessary context. ```python from dbt_bouncer.testing import check_fails, check_passes # Resource keyword arguments (model={...}) are auto-merged with sensible defaults check_passes("check_model_names", model={"name": "stg_orders"}, model_name_pattern="^stg_") check_fails("check_model_names", model={"name": "fct_orders"}, model_name_pattern="^stg_") # For context-dependent checks, use ctx_* keyword arguments: check_passes("check_model_documentation_coverage", min_model_documentation_coverage_pct=100, ctx_models=[{"description": "desc", "name": "m1", "unique_id": "model.pkg.m1"}]) ``` -------------------------------- ### Specify custom config file for dbt-bouncer validate Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Use the --config-file option to specify a custom location for the dbt-bouncer YAML configuration file when validating. ```bash dbt-bouncer validate --config-file config/checks.yml ``` -------------------------------- ### Manifest Check: Unit Tests Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_unit_tests.md This check verifies unit tests within the dbt manifest.json file. It requires the manifest.json to be present in the project. ```APIDOC ## Manifest Check: Unit Tests ### Description This check verifies the presence and integrity of unit tests defined within the `manifest.json` file. ### Method N/A (This is a check performed on a file, not an API endpoint) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response #### Success Response (200) N/A (This is a check, not an API call with a traditional response) #### Response Example N/A !!! note The below checks require `manifest.json` to be present. ::: manifest.check_unit_tests ``` -------------------------------- ### Run Multiple Specific dbt-bouncer Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Specify multiple checks to run by providing a comma-separated list to the `--check` flag. ```shell dbt-bouncer --check check_model_has_unique_test,check_model_description_populated ``` -------------------------------- ### Validate dbt-bouncer configuration file Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Check your dbt-bouncer configuration file for common issues such as YAML syntax errors, missing required fields, or incorrect configuration types. ```bash dbt-bouncer validate --config-file dbt-bouncer.yml ``` -------------------------------- ### Manifest Check: Semantic Models Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_semantic_models.md This check verifies the semantic models within the dbt project's manifest.json file. ```APIDOC ## Manifest Check: Semantic Models ### Description This check verifies the semantic models within the dbt project's manifest.json file. It requires the `manifest.json` to be present. ### Method N/A (This is a check function, not an API endpoint) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response #### Success Response (200) N/A (This is a check function, not an API endpoint) #### Response Example N/A ``` -------------------------------- ### Run only unit tests for dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Executes only the unit tests for the dbt-bouncer project. ```bash make test-unit ``` -------------------------------- ### dbt-bouncer Python API Import Path Changes Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/migration/v3.md Shows how to import ResourceType in dbt-bouncer v3.0.0, contrasting it with the v2.x import path. ```python # v2.x from dbt_bouncer.resource_type import ResourceType # v3.0.0 from dbt_bouncer import ResourceType # or from dbt_bouncer.enums import ResourceType ``` -------------------------------- ### Manifest Checks: Tests Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_tests.md This section covers the manifest checks related to tests. It requires the manifest.json file to be present. ```APIDOC ## Manifest Checks: Tests ### Description This check verifies the test configurations within your dbt project's manifest.json file. It requires the `manifest.json` to be present in the project directory. ### Method N/A (This is a check performed on a local file) ### Endpoint N/A ### Parameters This check operates on the `manifest.json` file. No external parameters are required. ### Request Example N/A ### Response This check typically outputs information about the tests found or any issues detected within the test configurations in the manifest. The exact output format may vary based on the tool's implementation. #### Success Response (200) Indicates that the manifest was processed successfully and tests were analyzed. #### Response Example ```json { "status": "success", "message": "Manifest tests analyzed successfully.", "tests_found": 15 } ``` #### Error Response Indicates an issue during the manifest processing or test analysis. #### Error Response Example ```json { "status": "error", "message": "manifest.json not found or is invalid." } ``` ``` -------------------------------- ### Filter dbt-bouncer checks by category Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/configuration.md Use the --only CLI flag to specify which check categories to run. Provide a comma-separated list of categories. ```shell dbt-bouncer --only manifest_checks ``` ```shell dbt-bouncer --only catalog_checks,manifest_checks ``` -------------------------------- ### Write Unit Tests for Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/CONTRIBUTING.md Use `check_passes` and `check_fails` from `dbt_bouncer.testing` to write unit tests for your checks. These helpers simplify resource mocking and assertion. ```python # tests/unit/checks/manifest/models/test_naming.py from dbt_bouncer.testing import check_fails, check_passes def test_check_model_names_pass(): check_passes("check_model_names", model={"name": "stg_orders"}, model_name_pattern="^stg_") def test_check_model_names_fail(): check_fails("check_model_names", model={"name": "fct_orders"}, model_name_pattern="^stg_") ``` -------------------------------- ### Run dbt-bouncer Run Results Checks in CI Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/faq.md Execute only run results checks in a CI environment after a `dbt build` command, using `dbt-bouncer --only run_results_checks`. ```shell dbt-bouncer --only run_results_checks ``` -------------------------------- ### Run dbt-Bouncer in GitHub Actions Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/getting_started.md Integrate dbt-Bouncer into your GitHub Actions CI pipeline to automatically check your dbt project on pull requests. Ensure you have the necessary permissions and checkout the code before running the action. ```yaml name: CI pipeline on: pull_request: branches: - main jobs: run-dbt-bouncer: permissions: pull-requests: write # Required to write a comment on the PR runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v6 - name: Generate or fetch dbt artifacts run: ... - uses: godatadriven/dbt-bouncer@vX.X with: check: '' # optional, comma-separated check names to run config-file: ./ only: manifest_checks # optional, defaults to running all checks output-file: results.json # optional, default does not save a results file output-format: json # optional, one of: csv, json, junit, sarif, tap. Defaults to json output-only-failures: false # optional, defaults to true send-pr-comment: true # optional, defaults to true show-all-failures: false # optional, defaults to false verbose: false # optional, defaults to false ``` -------------------------------- ### Control dbt-bouncer logging verbosity Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Adjust logging verbosity using the -v or --verbosity option. Specify multiple times for increased verbosity. ```bash dbt-bouncer run -v ``` ```bash dbt-bouncer run -vv ``` ```bash dbt-bouncer run -vvv ``` -------------------------------- ### Directories Check Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/sources/directories.md The directories check verifies directory structures within your dbt project. It requires the manifest.json file to be present. ```APIDOC ## Directories Check ### Description Verifies directory structures within your dbt project. This check requires the `manifest.json` file to be present. ### Method N/A (This is a conceptual check, not an API endpoint) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### Run specific categories with dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Limit checks to specific categories using the --only option. Provide a comma-separated list for multiple categories. ```bash dbt-bouncer run --only manifest_checks ``` ```bash dbt-bouncer run --only catalog_checks,manifest_checks ``` -------------------------------- ### Run only integration tests for dbt-bouncer Source: https://github.com/godatadriven/dbt-bouncer/blob/main/AGENTS.md Executes only the integration tests for the dbt-bouncer project. ```bash make test-integration ``` -------------------------------- ### Manifest Metadata Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_metadata.md These checks verify the metadata within your dbt project's manifest.json file. Ensure manifest.json is available in the project root. ```APIDOC ## Manifest Metadata Checks ### Description Performs checks on the metadata present in the `manifest.json` file. ### Method N/A (This is a check, not an API endpoint) ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example N/A ### Response #### Success Response (200) N/A (This is a check, output is typically console logs or exit codes) #### Response Example N/A ``` -------------------------------- ### Manifest Lineage Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_lineage.md These checks require the manifest.json file to be present. They are used to analyze the lineage of your dbt project. ```APIDOC ## Manifest Lineage Checks ### Description Performs lineage checks on the dbt project using the `manifest.json` file. ### Method N/A (This is a check/tool execution, not a direct API endpoint) ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example N/A ### Response #### Success Response (200) N/A (Output is typically console logs or a status code indicating success/failure of the check) #### Response Example N/A ``` -------------------------------- ### dbt-bouncer Boolean Flag Syntax Change Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/migration/v3.md Illustrates the change in boolean flag syntax from value-based to flag/no-flag for dbt-bouncer CLI commands. ```diff - dbt-bouncer --create-pr-comment-file false + dbt-bouncer --no-create-pr-comment-file ``` -------------------------------- ### Manifest Check: Exposures Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/manifest/check_exposures.md Performs checks on exposures defined in the dbt project's manifest.json file. ```APIDOC ## Manifest Check: Exposures ### Description This check verifies the integrity and correctness of exposures defined within your dbt project's `manifest.json` file. It requires the `manifest.json` to be present in the project directory. ### Method Not Applicable (This is a command-line check) ### Endpoint Not Applicable ### Parameters #### Path Parameters - **manifest.json** (file) - Required - The dbt project's manifest file. ### Request Example ```bash dbt-bouncer manifest check exposures ``` ### Response #### Success Response (0) Indicates that all exposure checks passed successfully. #### Response Example ``` All exposure checks passed. ``` #### Error Response - **1**: Indicates a general failure in the exposure checks. - **2**: Indicates that `manifest.json` was not found. ``` -------------------------------- ### Preview checks with dbt-bouncer dry-run Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/cli.md Use the --dry-run flag to assemble the check list and print a summary of checks that would run without executing them. Exits with code 0. ```bash dbt-bouncer run --dry-run ``` -------------------------------- ### Run Results Checks Source: https://github.com/godatadriven/dbt-bouncer/blob/main/docs/checks/run_results/check_run_results.md Performs checks on the dbt run results. This requires manifest.json and run_results.json. ```APIDOC ## Run Results Checks ### Description Performs checks on the dbt run results. This requires manifest.json and run_results.json. ### Method Not specified, typically a command-line execution. ### Endpoint N/A (Command-line tool) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```bash dbt-bouncer check run_results ``` ### Response #### Success Response (200) Output indicating the status of the checks. #### Response Example ``` Checks completed successfully. ``` ```