### Setup Sample Data Source: https://github.com/sidequery/sidemantic/blob/main/examples/pre_aggregations/README.md Generate 10,000 sample orders using the provided Python script. Navigate to the example directory first. ```bash cd examples/pre_aggregations uv run setup_data.py ``` -------------------------------- ### Run Data Setup Script Source: https://github.com/sidequery/sidemantic/blob/main/examples/motherduck/README.md Execute the Python script to create sample data for the MotherDuck example. This script generates 10,000 orders and 20 customers. ```bash uv run python setup_data.py ``` -------------------------------- ### Development Setup for VS Code Extension Source: https://github.com/sidequery/sidemantic/blob/main/vscode-sidemantic/README.md Install dependencies, compile, and test the VS Code extension locally. Press F5 to launch the Extension Development Host for local testing. ```bash cd vscode-sidemantic bun install bun run compile bun run test ``` -------------------------------- ### Install Extension Targets Source: https://github.com/sidequery/sidemantic/blob/main/sidemantic-duckdb/CMakeLists.txt Installs the extension targets (libraries) to the specified installation directory. ```cmake install( TARGETS ${EXTENSION_NAME} EXPORT "${DUCKDB_EXPORT_SET}" LIBRARY DESTINATION "${INSTALL_LIB_DIR}" ARCHIVE DESTINATION "${INSTALL_LIB_DIR}") ``` -------------------------------- ### Install Python Extension Wheel Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-packaging.md Install the sidemantic-rs Python extension wheel using uv. ```bash uv add sidemantic-rs ``` -------------------------------- ### Connection String Example Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/yaml-schema.md Example of specifying a database connection using a string format for DuckDB. ```yaml connection: "duckdb:///path/to/db.duckdb" ``` -------------------------------- ### Run Query Examples Source: https://github.com/sidequery/sidemantic/blob/main/examples/motherduck/README.md Execute the Python script containing various query examples. This script demonstrates querying with and without pre-aggregations, filtering, and cross-model queries. ```bash uv run python query_examples.py ``` -------------------------------- ### Run Hive Parquet Pre-aggregation Example Source: https://github.com/sidequery/sidemantic/blob/main/examples/hive_parquet/README.md Execute the Python script for an example demonstrating pre-aggregations on Hive-partitioned parquet files. ```bash uv run python examples/hive_parquet/hive_parquet_preagg_example.py ``` -------------------------------- ### Install Rust CLI from Source Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-packaging.md Use this command to install the Rust CLI from a local source checkout before the crate is published. ```bash cargo install --path sidemantic-rs --locked ``` -------------------------------- ### Install Dependencies Source: https://github.com/sidequery/sidemantic/blob/main/examples/notebooks/lookml_multi_entity_duckdb_demo.ipynb Installs necessary Python packages for the demo in a Colab environment. ```python # Colab setup !pip -q install sidemantic duckdb polars pyarrow altair ``` -------------------------------- ### Rust CLI Installation (GitHub Release) Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-roadmap.md Install the Rust CLI tool using a shell script downloaded from GitHub releases. ```bash curl -LsSf https://.../sidemantic-rs/install.sh | sh ``` -------------------------------- ### Install Sidemantic Source: https://github.com/sidequery/sidemantic/blob/main/README.md Installs the Sidemantic package using uv. ```bash uv add sidemantic ``` -------------------------------- ### Install Sidemantic with Notebook Widget and Marimo Source: https://github.com/sidequery/sidemantic/blob/main/README.md Installs Sidemantic with notebook widget support and Marimo, then runs Marimo edit. ```bash uv add "sidemantic[widget]" marimo uv run marimo edit ``` -------------------------------- ### Install Sidemantic with API Server Support Source: https://github.com/sidequery/sidemantic/blob/main/README.md Installs Sidemantic with HTTP API server support using uv. ```bash uv add "sidemantic[api]" ``` -------------------------------- ### Install Playwright Browsers Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-webapp-builder/references/webapp-patterns.md Installs the necessary Chromium browser for Playwright. Run this command once if needed. ```bash bunx --bun -p playwright playwright install chromium ``` -------------------------------- ### Install Sidemantic with Malloy Support Source: https://github.com/sidequery/sidemantic/blob/main/README.md Installs Sidemantic with Malloy integration using uv. ```bash uv add "sidemantic[malloy]" ``` -------------------------------- ### Run Hive Parquet Example Source: https://github.com/sidequery/sidemantic/blob/main/examples/hive_parquet/README.md Execute the Python script for a basic example of reading and querying Hive-partitioned parquet files. ```bash uv run python examples/hive_parquet/hive_parquet_example.py ``` -------------------------------- ### Run SQL Syntax Example Script Source: https://github.com/sidequery/sidemantic/blob/main/README.md Executes a Python script that demonstrates SQL syntax features of Sidemantic. ```bash uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/sql/sql_syntax_example.py ``` -------------------------------- ### Python Package Installation with Rust Extra Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-roadmap.md Install the main Sidemantic Python package with the Rust extra, if publication allows for a combined installation. ```bash uv add "sidemantic[rust]" ``` -------------------------------- ### Rust CLI Installation (Cargo) Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-roadmap.md Install the Rust CLI tool using Cargo, the Rust package manager. ```bash cargo install sidemantic ``` -------------------------------- ### Install Sidemantic with Notebook Widget and JupyterLab Source: https://github.com/sidequery/sidemantic/blob/main/README.md Installs Sidemantic with notebook widget support and JupyterLab, then runs Jupyter Lab. ```bash uv add "sidemantic[widget]" jupyterlab uv run jupyter lab ``` -------------------------------- ### Install and Load Community Extension Source: https://github.com/sidequery/sidemantic/blob/main/docs/duckdb-extension.md Commands to install and load the sidemantic extension once published to the community repository. ```sql INSTALL sidemantic FROM community; LOAD sidemantic; ``` -------------------------------- ### Install Published Rust CLI Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-packaging.md Install the Rust CLI using Cargo after the crate has been published to crates.io. ```bash cargo install sidemantic --version 0.1.0 --locked ``` -------------------------------- ### Install Sidemantic Agent Skill (One-liner) Source: https://github.com/sidequery/sidemantic/blob/main/README.md Installs the Sidemantic agent skill using a curl command to download and execute an installation script. No repository clone is required. ```bash curl -fsSL https://raw.githubusercontent.com/sidequery/sidemantic/main/skills/install.sh | bash ``` -------------------------------- ### Manual Setup for PostgreSQL Integration Tests Source: https://github.com/sidequery/sidemantic/blob/main/tests/db/README.md Manually set up PostgreSQL dependencies and environment variables to run integration tests. Adjust connection details as needed. ```bash # Install postgres dependencies uv sync --extra postgres # Set up Postgres (adjust connection details as needed) export POSTGRES_TEST=1 export POSTGRES_URL="postgres://test:test@localhost:5432/sidemantic_test" export POSTGRES_HOST="localhost" export POSTGRES_PORT="5432" export POSTGRES_DB="sidemantic_test" export POSTGRES_USER="test" export POSTGRES_PASSWORD="test" # Run integration tests only uv run pytest -m integration tests/db/test_postgres_integration.py -v uv run pytest -m integration tests/db/test_postgres_cli_e2e.py -v ``` -------------------------------- ### Start Docker Compose for PostgreSQL Tests Source: https://github.com/sidequery/sidemantic/blob/main/tests/db/README.md Start the PostgreSQL service using Docker Compose and run integration tests. This is the recommended method for PostgreSQL integration testing. ```bash docker compose up test --build --abort-on-container-exit ``` -------------------------------- ### Migrator Analysis Example Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/SKILL.md Demonstrates running the migrator to analyze a list of SQL queries. This generates an initial pass of semantic models. ```python from sidemantic_query.migrator import Migrator migrator = Migrator() queries = ["SELECT SUM(amount) FROM orders", "SELECT COUNT(DISTINCT user_id) FROM users"] models = migrator.analyze_queries(queries) ``` -------------------------------- ### Install ADBC Snowflake Driver Source: https://github.com/sidequery/sidemantic/blob/main/examples/adbc_snowflake/README.md Install the ADBC driver manager and the Snowflake driver using pip. ```bash pip install adbc-driver-manager dbc install snowflake ``` ```bash pip install adbc-driver-snowflake ``` -------------------------------- ### Install Sidemantic and Dependencies Source: https://github.com/sidequery/sidemantic/blob/main/examples/notebooks/sidemantic_sql_duckdb_demo.ipynb Installs the necessary libraries for Sidemantic, DuckDB, Polars, and PyArrow. This is a prerequisite for running the notebook. ```python # Colab setup !pip -q install sidemantic duckdb polars pyarrow ``` -------------------------------- ### Start Sidemantic API Server Source: https://github.com/sidequery/sidemantic/blob/main/README.md Starts the Sidemantic API server with specified models, database, port, and authentication token. ```bash sidemantic api-serve models/ --db data.duckdb --port 4400 --auth-token secret ``` -------------------------------- ### Start PostgreSQL-Compatible Server Source: https://github.com/sidequery/sidemantic/blob/main/examples/ecommerce/README.md Launch a `sidemantic` server compatible with PostgreSQL clients. This allows BI tools to connect and query the semantic layer. ```bash sidemantic serve examples/ecommerce/models \ --db examples/ecommerce/data/ecommerce.db \ --port 5433 ``` -------------------------------- ### Run Docker Container in Demo Mode Source: https://github.com/sidequery/sidemantic/blob/main/examples/docker/README.md Starts the SideSemantic container in a simplified demo mode, exposing the PostgreSQL server. ```bash docker run -p 5433:5433 sidequery/sidemantic --demo ``` -------------------------------- ### Example Semantic Project Configuration Source: https://github.com/sidequery/sidemantic/blob/main/docs/native-format.md This YAML configuration defines a semantic model including parameters, dimensions, metrics, segments, and relationships. ```yaml version: 1 parameters: - name: start_date type: date models: - name: orders table: public.orders primary_key: order_id default_time_dimension: created_at default_grain: day dimensions: - name: order_id type: categorical - name: customer_id type: categorical - name: status type: categorical - name: created_at type: time sql: created_at supported_granularities: [day, week, month, quarter, year] metrics: - name: order_count agg: count - name: total_revenue agg: sum sql: amount - name: average_order_value type: ratio numerator: total_revenue denominator: order_count segments: - name: paid sql: "{model}.status = 'paid'" relationships: - name: customers type: many_to_one foreign_key: customer_id primary_key: customer_id - name: customers table: public.customers primary_key: customer_id dimensions: - name: customer_id type: categorical - name: country type: categorical metrics: - name: revenue_per_customer type: ratio numerator: orders.total_revenue denominator: customers.customer_count ``` -------------------------------- ### Install Sidemantic Agent Skill (npx/bunx) Source: https://github.com/sidequery/sidemantic/blob/main/README.md Installs the Sidemantic agent skill using npx or bunx to add the skill from its GitHub repository. ```bash npx skills add https://github.com/sidequery/sidemantic --skill sidemantic-modeler ``` ```bash bunx skills add https://github.com/sidequery/sidemantic --skill sidemantic-modeler ``` -------------------------------- ### Run Semantic Query Source: https://github.com/sidequery/sidemantic/blob/main/docs/duckdb-extension.md Example of running a semantic query directly through the parser extension. ```sql SEMANTIC SELECT orders.revenue FROM orders; ``` -------------------------------- ### Install Sidemantic CLI with LSP Source: https://github.com/sidequery/sidemantic/blob/main/vscode-sidemantic/README.md Install the Sidemantic CLI with LSP support using pip or uv. This is a prerequisite for the VS Code extension's language server functionality. ```bash uv pip install sidemantic[lsp] # or pip install sidemantic[lsp] ``` -------------------------------- ### Python Package Installation (uv) Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-roadmap.md Install the main Sidemantic Python package and the sidemantic-rs Rust extension using the uv package manager. ```bash uv add sidemantic uv add sidemantic-rs ``` -------------------------------- ### Connection Dictionary Form Example Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/yaml-schema.md Illustrates the dictionary form for specifying connection details, including the 'type' field. ```yaml connection: type: "duckdb" ``` -------------------------------- ### Run Rust CLI from Source Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-packaging.md Execute the Rust CLI binary directly from a checked-out repository without installing it. ```bash cargo run --manifest-path sidemantic-rs/Cargo.toml --bin sidemantic -- validate ./models ``` -------------------------------- ### Connect with PostgreSQL Client Source: https://github.com/sidequery/sidemantic/blob/main/examples/ecommerce/README.md Example command to connect to the running `sidemantic` server using the `psql` client. Assumes the server is running on localhost:5433. ```bash psql -h localhost -p 5433 -U user ``` -------------------------------- ### Run Symmetric Aggregates Example Script Source: https://github.com/sidequery/sidemantic/blob/main/README.md Executes a Python script demonstrating the use of symmetric aggregates in Sidemantic. ```bash uv run https://raw.githubusercontent.com/sidequery/sidemantic/main/examples/features/symmetric_aggregates_example.py ``` -------------------------------- ### Setup DuckDB and Load Data Source: https://github.com/sidequery/sidemantic/blob/main/examples/notebooks/lookml_multi_entity_duckdb_demo.ipynb Creates a temporary directory, connects to a DuckDB database, and populates 'customers', 'products', and 'orders' tables with sample data. It then reads the 'orders' table into a Polars DataFrame. ```python import tempfile from pathlib import Path import duckdb workdir = Path(tempfile.mkdtemp(prefix="sidemantic_lookml_demo_")) db_path = workdir / "demo.duckdb" con = duckdb.connect(str(db_path)) con.execute( """ create table customers as select * from (values (1, 'Ava', 'US'), (2, 'Noah', 'US'), (3, 'Liam', 'CA'), (4, 'Mia', 'GB') ) as t(id, name, region); create table products as select * from (values (10, 'Widget', 'hardware'), (11, 'Gizmo', 'hardware'), (12, 'Cloud', 'software') ) as t(id, name, category); create table orders as select * from (values (100, '2025-01-01', 'completed', 120.0, 1, 10), (101, '2025-01-02', 'completed', 80.0, 2, 11), (102, '2025-01-03', 'returned', 60.0, 3, 12), (103, '2025-01-04', 'completed', 200.0, 1, 12), (104, '2025-01-05', 'pending', 50.0, 4, 10) ) as t(id, order_date, status, amount, customer_id, product_id); """ ) orders_df = con.execute("select * from orders").pl() con.close() orders_df ``` -------------------------------- ### Bake Models into Docker Image Source: https://github.com/sidequery/sidemantic/blob/main/examples/docker/README.md Example Dockerfile snippet showing how to copy local model files into the image during the build process. ```dockerfile FROM sidequery/sidemantic COPY my_models/ /app/models/ ``` -------------------------------- ### Run ClickHouse Integration Tests with Docker Compose Source: https://github.com/sidequery/sidemantic/blob/main/tests/db/README.md Start ClickHouse using Docker Compose and run integration tests. This requires the `clickhouse` extra dependencies. ```bash docker compose up -d clickhouse CLICKHOUSE_TEST=1 uv run --extra clickhouse pytest -m integration tests/db/test_clickhouse_integration.py -v CLICKHOUSE_TEST=1 uv run --extra clickhouse pytest -m integration tests/db/test_clickhouse_cli_e2e.py -v ``` -------------------------------- ### Manual Setup for BigQuery Integration Tests Source: https://github.com/sidequery/sidemantic/blob/main/tests/db/README.md Manually set up BigQuery dependencies and environment variables to run integration tests. Adjust emulator host and project/dataset details as needed. ```bash # Install bigquery dependencies uv sync --extra bigquery # Set up BigQuery emulator (adjust as needed) export BIGQUERY_TEST=1 export BIGQUERY_EMULATOR_HOST=localhost:9050 export BIGQUERY_PROJECT=test-project export BIGQUERY_DATASET=test_dataset # Run integration tests only uv run pytest -m integration tests/db/test_bigquery_integration.py -v uv run pytest -m integration tests/db/test_bigquery_cli_e2e.py -v ``` -------------------------------- ### DuckDB Extension Installation Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-roadmap.md Install and load the Sidemantic DuckDB extension using SQL commands. This is the target installation path. ```sql install sidemantic from community; load sidemantic; ``` -------------------------------- ### Setup DuckDB and Load Data Source: https://github.com/sidequery/sidemantic/blob/main/examples/notebooks/sidemantic_sql_duckdb_demo.ipynb Initializes a DuckDB database in a temporary directory, creates an 'orders' table with sample data, and loads it into a Polars DataFrame. The database connection is then closed. ```python import tempfile from pathlib import Path import duckdb from sidemantic import SemanticLayer workdir = Path(tempfile.mkdtemp(prefix="sidemantic_demo_")) db_path = workdir / "demo.duckdb" con = duckdb.connect(str(db_path)) con.execute( """ create table orders as select * from (values (1, '2025-01-01', 'completed', 120.0, 'us', 101), (2, '2025-01-02', 'completed', 80.0, 'us', 102), (3, '2025-01-03', 'returned', 60.0, 'eu', 101), (4, '2025-01-04', 'completed', 200.0, 'eu', 103), (5, '2025-01-05', 'pending', 50.0, 'us', 104) ) as t(order_id, order_date, status, amount, region, customer_id); """ ) df = con.execute("select * from orders").pl() con.close() df ``` -------------------------------- ### Run HTTP API Server Source: https://github.com/sidequery/sidemantic/blob/main/examples/docker/README.md Starts the container in HTTP API mode, exposing port 4400 and requiring an API token. ```bash docker run -p 4400:4400 \ -v ./models:/app/models \ -e SIDEMANTIC_MODE=api \ -e SIDEMANTIC_API_TOKEN=secret \ sidequery/sidemantic ``` -------------------------------- ### Join Examples Source: https://github.com/sidequery/sidemantic/blob/main/docs/compatibility/malloy.md Illustrates different types of joins supported in Malloy, including one-to-one, one-to-many, and cross joins, with various condition specifications. ```Malloy join_one: target with foreign_key ``` ```Malloy join_many: target on condition ``` ```Malloy join_cross: target ``` ```Malloy join_one: alias is source with fk ``` ```Malloy join_one: alias is source on condition ``` ```Malloy join_one: name is connection.table(...) extend { ... } with fk ``` ```Malloy a = b.a and c = b.c ``` ```Malloy gender = cohort.gender and state = cohort.state ``` -------------------------------- ### Recommend Pre-aggregations from Directory Source: https://github.com/sidequery/sidemantic/blob/main/examples/pre_aggregations/README.md Generate pre-aggregation recommendations by pointing the command to a directory containing multiple SQL files. This is useful for analyzing a larger set of queries. ```bash uvx sidemantic preagg recommend --queries /path/to/queries/ ``` -------------------------------- ### Run Spark Integration Tests with Docker Compose Source: https://github.com/sidequery/sidemantic/blob/main/tests/db/README.md Start Spark using Docker Compose and run integration tests. This requires the `spark` extra dependencies. ```bash docker compose up -d spark SPARK_TEST=1 uv run --extra spark pytest -m integration tests/db/test_spark_integration.py -v SPARK_TEST=1 uv run --extra spark pytest -m integration tests/db/test_spark_cli_e2e.py -v ``` -------------------------------- ### Run Complete Demo Source: https://github.com/sidequery/sidemantic/blob/main/examples/pre_aggregations/README.md Execute the entire pre-aggregation demo script. Ensure the script has execute permissions. ```bash cd examples/pre_aggregations chmod +x demo.sh ./demo.sh ``` -------------------------------- ### Start Integration Test Services with Docker Compose Source: https://github.com/sidequery/sidemantic/blob/main/examples/docker/README.md Command to launch the necessary services (Postgres, BigQuery emulator, Spark, ClickHouse) for integration testing using docker-compose. ```bash docker compose -f examples/docker/docker-compose.yml up ``` -------------------------------- ### Recommend Pre-aggregations from SQL File Source: https://github.com/sidequery/sidemantic/blob/main/examples/pre_aggregations/README.md Use the `uvx sidemantic preagg recommend` command to generate pre-aggregation recommendations from a SQL file. This helps identify potential optimizations for your queries. ```bash uvx sidemantic preagg recommend --queries /path/to/queries.sql ``` -------------------------------- ### Validation Error Example Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-roadmap.md Example of a structured error returned by Rust validation for a missing metric dependency. ```json { "code": "missing_metric_dependency", "message": "Metric orders.net_revenue references missing metric orders.gross_revenue", "model": "orders", "field": "net_revenue", "reference": "orders.gross_revenue" } ``` -------------------------------- ### Rejected Semantic SQL Example Source: https://github.com/sidequery/sidemantic/blob/main/docs/rust-native-runtime-roadmap.md Example of SQL syntax that is explicitly rejected by the semantic SQL subset, such as explicit JOINs. ```sql -- Explicit JOIN syntax is not supported in semantic SQL. Reference fields from related models instead and let Sidemantic infer the join path. ``` -------------------------------- ### Build DuckDB Extension from Source Source: https://github.com/sidequery/sidemantic/blob/main/docs/duckdb-extension.md Steps to clone DuckDB and extension-ci-tools, build the extension, and run tests locally. ```bash cd sidemantic-duckdb rm -rf duckdb extension-ci-tools git clone --depth 1 --branch v1.4.2 https://github.com/duckdb/duckdb.git duckdb git clone --depth 1 --branch v1.4.2 https://github.com/duckdb/extension-ci-tools.git extension-ci-tools make make test ``` -------------------------------- ### Generate Sample Database Source: https://github.com/sidequery/sidemantic/blob/main/examples/ecommerce/README.md Use this command to generate the sample e-commerce database. This script creates a DuckDB database with realistic sample data for customers, products, orders, and order items. ```bash uv run examples/ecommerce/data/create_db.py ``` -------------------------------- ### Serve Static App with Bun Source: https://github.com/sidequery/sidemantic/blob/main/examples/sidemantic_wasm_demo/README.md Use Bun to serve the static application files for the Sidemantic WASM demo on a specified port. ```bash bun examples/sidemantic_wasm_demo/server.ts --port 5174 ``` -------------------------------- ### Cube.js Orders Model Example Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/migration.md This is an example of an 'orders' cube definition in Cube.js YAML format, including dimensions, measures, and joins. ```yaml cubes: - name: orders sql_table: public.orders dimensions: - name: id sql: id type: number primary_key: true - name: status sql: status type: string - name: created_at sql: created_at type: time measures: - name: count type: count - name: revenue sql: amount type: sum - name: completed_revenue sql: amount type: sum filters: - sql: "${CUBE}.status = 'completed'" joins: - name: customers sql: "${CUBE}.customer_id = ${customers.id}" relationship: many_to_one ``` -------------------------------- ### Run Sidemantic PostgreSQL Server Demo Source: https://github.com/sidequery/sidemantic/blob/main/README.md Starts a Sidemantic PostgreSQL wire protocol server in demo mode, allowing connections from BI tools like Tableau and DBeaver. ```bash uvx sidemantic serve --demo --port 5433 ``` -------------------------------- ### Time Dimension Granularity Example Source: https://github.com/sidequery/sidemantic/blob/main/examples/ecommerce/README.md Demonstrates how to query revenue data with different time granularities (month and year) from the orders model. This showcases the support for time dimensions. ```sql SELECT revenue, created_at__month FROM orders ``` ```sql SELECT revenue, created_at__year FROM orders ``` -------------------------------- ### Structured Query Payload Example Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-webapp-builder/SKILL.md Example of a structured query payload. Prefer this format over ad hoc SQL strings for defining query contracts. ```json { "metrics": ["orders.revenue"], "dimensions": ["orders.order_date__day"], "filters": ["orders.status = 'completed'"], "order_by": ["orders.order_date__day"], "limit": 500 } ``` -------------------------------- ### Example Metric Series Query Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-webapp-builder/references/webapp-patterns.md An example of a query structured for metric series, including metrics, a time grain dimension, ordering by time, and a limit. ```json { "metrics": ["auctions.bid_request_cnt"], "dimensions": ["auctions.__time__day"], "filters": [ "auctions.__time >= cast('2025-01-01' as date)", "auctions.device_os = 'iOS'" ], "order_by": ["auctions.__time__day"], "limit": 500 } ``` -------------------------------- ### Generated Rewritten Query Example Source: https://github.com/sidequery/sidemantic/blob/main/examples/migrator/README.md Example of Python code generated to query data using the semantic layer, replacing original raw SQL. ```python # Original query: # SELECT status, SUM(total_amount), COUNT(*) # FROM orders # GROUP BY status result = layer.query( dimensions=['orders.status'], metrics=['orders.count', 'orders.sum_total_amount'] ) ``` -------------------------------- ### Generated Model Definition Example Source: https://github.com/sidequery/sidemantic/blob/main/examples/migrator/README.md Example of a generated YAML model definition for the 'orders' table, including dimensions and metrics derived from SQL queries. ```sql SELECT status, SUM(total_amount), COUNT(*) FROM orders GROUP BY status ``` ```yaml models: - name: orders table: orders description: Auto-generated from query analysis dimensions: - name: status sql: status type: categorical metrics: - name: count agg: count sql: '*' - name: sum_total_amount agg: sum sql: total_amount ``` -------------------------------- ### Install sidemantic with widget support Source: https://github.com/sidequery/sidemantic/blob/main/examples/widget_demo/widget_demo.ipynb Installs the sidemantic package with widget extras, using uv if available, and enables custom widget manager in Google Colab. ```python import os import shutil import subprocess import sys from pathlib import Path def _run(cmd): print(f">> {cmd}") subprocess.check_call(cmd, shell=True) is_colab = "COLAB_RELEASE_TAG" in os.environ or "google.colab" in sys.modules repo_root = Path.cwd() is_repo = (repo_root / "pyproject.toml").exists() and (repo_root / "sidemantic").exists() if is_repo: if shutil.which("uv"): _run('uv pip install -e ".[widget]"') else: _run('pip install -e ".[widget]"') else: if shutil.which("uv"): _run('uv pip install "sidemantic[widget]"') else: _run('pip install "sidemantic[widget]"') if is_colab: from google.colab import output output.enable_custom_widget_manager() ``` -------------------------------- ### Build Sidemantic DuckDB Extension from Source Source: https://github.com/sidequery/sidemantic/blob/main/sidemantic-duckdb/README.md Instructions for cloning the repository, building the Rust library, compiling the DuckDB extension using make, and running tests. ```bash # Clone with submodules git clone --recurse-submodules https://github.com/your-repo/sidemantic-duckdb.git cd sidemantic-duckdb # Build the Rust library first cd ../sidemantic-rs cargo build --release cd ../sidemantic-duckdb # Build the extension make # Run tests make test # Use the extension ./build/release/duckdb ``` -------------------------------- ### Interactive Workbench Source: https://github.com/sidequery/sidemantic/blob/main/examples/ecommerce/README.md Launch an interactive workbench to explore your semantic models. Connects to the specified database for querying. ```bash sidemantic workbench examples/ecommerce/models --db examples/ecommerce/data/ecommerce.db ``` -------------------------------- ### Get Database Tables and Columns Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/generation.md Retrieve table and column metadata from the database adapter. Use `get_tables()` to list all tables and `get_columns(table_name)` to get details for a specific table. ```python layer = SemanticLayer(connection="duckdb:///data.duckdb") tables = layer.adapter.get_tables() # [{"table_name": "orders", "schema": "main"}, ...] columns = layer.adapter.get_columns("orders") # [{"column_name": "id", "data_type": "INTEGER"}, ...] ``` -------------------------------- ### Run MCP Server Source: https://github.com/sidequery/sidemantic/blob/main/examples/docker/README.md Launches the container in MCP server mode, mounting local models. ```bash docker run -v ./models:/app/models -e SIDEMANTIC_MODE=mcp sidequery/sidemantic ``` -------------------------------- ### Configure Cargo Build Arguments Source: https://github.com/sidequery/sidemantic/blob/main/sidemantic-duckdb/CMakeLists.txt Prepares the arguments for the 'cargo build' command, including the manifest path and profile. ```cmake set(SIDEMANTIC_CARGO_BUILD_ARGS build --manifest-path "${SIDEMANTIC_RS_DIR}/Cargo.toml" --lib) if(SIDEMANTIC_CARGO_PROFILE STREQUAL "release") list(APPEND SIDEMANTIC_CARGO_BUILD_ARGS --release) elseif(NOT SIDEMANTIC_CARGO_PROFILE STREQUAL "debug") list(APPEND SIDEMANTIC_CARGO_BUILD_ARGS --profile "${SIDEMANTIC_CARGO_PROFILE}") endif() ``` -------------------------------- ### Get Graph Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-webapp-builder/SKILL.md Retrieves the graph representation of the models. ```APIDOC ## GET /graph ### Description Retrieves the graph representation of the models. ### Method GET ### Endpoint /graph ``` -------------------------------- ### Add Sidemantic and DuckDB, Create Model YAML, and Validate Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/SKILL.md This snippet demonstrates the initial setup for Sidemantic using `uv` and creating a basic `orders.yml` model file. It then validates the model using the CLI. ```bash uv add sidemantic duckdb mkdir -p models cat > models/orders.yml <<'YAML' models: - name: orders table: orders primary_key: order_id dimensions: - name: status type: categorical metrics: - name: revenue agg: sum sql: order_amount - name: order_count agg: count YAML uv run sidemantic validate models/ --verbose uv run sidemantic info models/ uv run sidemantic query models/ -c duckdb:///data.duckdb \ "SELECT revenue, status FROM orders ORDER BY revenue DESC LIMIT 5" ``` -------------------------------- ### Dimension Type Guidance: Time Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/yaml-schema.md Example of a time dimension type, specifying granularity for date/timestamp fields. ```yaml dimensions: - name: "order_timestamp" type: "time" granularity: "hour" ``` -------------------------------- ### Initialize DuckDB and Load Data Source: https://github.com/sidequery/sidemantic/blob/main/examples/widget_demo/widget_demo_simple.ipynb Connects to an in-memory DuckDB database and loads sample auction data from a Parquet file into a table named 'auctions'. ```python import duckdb conn = duckdb.connect(":memory:") conn.execute(""" create table auctions as select * from 'https://sampledata.sidequery.dev/sidemantic-demo/auction_data.parquet' ") conn.execute("SELECT * FROM auctions LIMIT 5").fetchdf() ``` -------------------------------- ### Annotation and Description Examples Source: https://github.com/sidequery/sidemantic/blob/main/docs/compatibility/malloy.md Shows how to add descriptions and metadata tags to Malloy sources, dimensions, and measures. ```Malloy ## Description text ``` ```Malloy # desc: value ``` ```Malloy # description: value ``` ```Malloy # tag_name ``` ```Malloy #@ persist ``` ```Malloy #@ persist name=... ``` -------------------------------- ### Dimension Type Guidance: Numeric Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/yaml-schema.md Example of a numeric dimension type, using SQL for computed buckets or ranges. ```yaml dimensions: - name: "order_value_bucket" type: "numeric" sql: "CASE WHEN amount < 100 THEN 'small' WHEN amount < 500 THEN 'medium' ELSE 'large' END" ``` -------------------------------- ### End-to-End Model Generation and Loading Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/generation.md Demonstrates the full workflow: connecting to a database, collecting queries, analyzing them to generate models, reviewing the output, writing model files, and then loading the refined models into a new SemanticLayer instance for querying. ```python from sidemantic import SemanticLayer, Model, Dimension, Metric, Relationship from sidemantic.loaders import load_from_directory from sidemantic.core.migrator import Migrator # 1. Connect to database layer = SemanticLayer(connection="duckdb:///warehouse.duckdb", auto_register=False) # 2. Collect existing queries queries = [ "SELECT status, SUM(amount) AS revenue FROM orders GROUP BY status", "SELECT DATE_TRUNC('month', created_at) AS month, SUM(amount) FROM orders GROUP BY 1", "SELECT c.region, SUM(o.amount) / COUNT(DISTINCT c.id) AS rev_per_customer " "FROM orders o JOIN customers c ON o.customer_id = c.id GROUP BY 1", ] # 3. Generate models migrator = Migrator(layer, connection=layer.conn) report = migrator.analyze_queries(queries) models = migrator.generate_models(report) graph_metrics = migrator.generate_graph_metrics(report, models) # 4. Review what was generated for name, model_def in models.items(): print(f"\n=== {name} ===") for d in model_def.get("dimensions", []): print(f" dim: {d['name']} ({d['type']})") for m in model_def.get("metrics", []): print(f" metric: {m['name']} ({m.get('agg', m.get('type'))})") # 5. Write to disk and refine migrator.write_model_files(models, "output/models/") migrator.write_graph_metrics_file(graph_metrics, "output/models/") # 6. Load refined models and query layer2 = SemanticLayer(connection="duckdb:///warehouse.duckdb", auto_register=False) load_from_directory(layer2, "output/models/") result = layer2.sql("SELECT revenue, status FROM orders") ``` -------------------------------- ### Bootstrap Semantic Layer from Queries Source: https://github.com/sidequery/sidemantic/blob/main/examples/migrator/README.md Generates model definitions and rewritten queries from raw SQL files. Use this to bootstrap your semantic layer. ```bash cd examples/migrator # Generate models and rewritten queries uv run sidemantic migrator --queries raw_queries/ --generate-models output/ ``` -------------------------------- ### Convert Cube.js to MetricFlow using Adapters Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/migration.md Demonstrates cross-format conversion by parsing Cube.js models and exporting them to MetricFlow format using respective adapters. ```python cube_adapter = CubeAdapter() graph = cube_adapter.parse("cube_models/") mf_adapter = MetricFlowAdapter() mf_adapter.export(graph, "output/metricflow_models.yml") ``` -------------------------------- ### Model YAML File Structure Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/generation.md Example structure of a model written to a YAML file in native parser format. ```yaml models: - name: orders table: orders ... ``` -------------------------------- ### Pre-Aggregation Recommender Workflow Source: https://github.com/sidequery/sidemantic/blob/main/skills/sidemantic-modeler/references/generation.md Analyze query patterns to recommend materialized rollup tables. Initialize the recommender, parse query logs or history, and generate pre-aggregation definitions. ```python from sidemantic.core.preagg_recommender import PreAggregationRecommender recommender = PreAggregationRecommender(min_query_count=10, min_benefit_score=0.0) recommender.parse_query_log(queries) # Or: recommender.fetch_and_parse_query_history(connection, days_back=7) for rec in recommender.get_recommendations(top_n=5): definition = recommender.generate_preagg_definition(rec) print(definition) ``` -------------------------------- ### Define Parameters in YAML Source: https://github.com/sidequery/sidemantic/blob/main/docs/native-format.md Define graph-level values that can be used in templates and filters. This example defines a 'status' parameter. ```yaml parameters: - name: status type: string default_value: paid allowed_values: [paid, refunded, pending] ```