### Setup Development Environment

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to initialize a local development environment by installing all workspace packages and necessary dependencies for testing and linting.

```bash
uv sync --all-packages --group dev
```

```bash
python -m venv .venv
source .venv/bin/activate
pip install -e soda-core -e soda-tests -e soda-postgres
pip install pytest pre-commit pydantic python-dotenv freezegun
```

--------------------------------

### Start Local Trino Instance

Source: https://github.com/sodadata/soda-core/blob/main/soda-trino/local_instance/README.md

Starts the local Trino instance using Docker Compose. Assumes `docker-compose.yml` is present and configured.

```bash
docker compose up
```

--------------------------------

### Configure Data Source and Cloud Authentication

Source: https://context7.com/sodadata/soda-core/llms.txt

Examples for setting up a DuckDB data source connection and configuring Soda Cloud API credentials using environment variables.

```yaml
name: duckdb_ds
type: duckdb
connection:
  path: /path/to/database.duckdb
```

```yaml
api_key_id: ${SODA_CLOUD_API_KEY_ID}
api_key_secret: ${SODA_CLOUD_API_KEY_SECRET}
host: cloud.soda.io
```

--------------------------------

### BigQuery Data Source Configuration (YAML)

Source: https://context7.com/sodadata/soda-core/llms.txt

Example YAML configuration for connecting to Google BigQuery. Requires project ID, dataset, and path to service account credentials.

```yaml
name: bigquery_ds
type: bigquery
connection:
  project_id: my-gcp-project
  dataset: my_dataset
  credentials_path: /path/to/service-account.json
```

--------------------------------

### Install Legacy Soda Core v3 Packages

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to install legacy version 3 packages for specific data sources, ensuring compatibility with older project requirements.

```bash
uv pip install soda-core-postgres~=3.5.0
```

```bash
pip install soda-core-postgres~=3.5.0
```

--------------------------------

### Install Soda Core for Data Sources

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to install the Soda Core v4 package for a specific data source using either UV or pip. Replace the package name with the appropriate connector for your data stack.

```bash
uv pip install soda-postgres
```

```bash
pip install soda-postgres
```

--------------------------------

### PostgreSQL Data Source Configuration (YAML)

Source: https://context7.com/sodadata/soda-core/llms.txt

Example YAML configuration for connecting to a PostgreSQL database. Specifies connection details like host, port, database, user, and password.

```yaml
name: postgres_ds
type: postgres
connection:
  host: localhost
  port: 5432
  database: mydb
  user: myuser
  password: ${POSTGRES_PASSWORD}
  sslmode: prefer
```

--------------------------------

### Snowflake Data Source Configuration (YAML)

Source: https://context7.com/sodadata/soda-core/llms.txt

Example YAML configuration for connecting to a Snowflake data warehouse. Includes account details, user credentials, warehouse, database, schema, and role.

```yaml
name: snowflake_ds
type: snowflake
connection:
  account: xy12345.us-east-1
  user: myuser
  password: ${SNOWFLAKE_PASSWORD}
  warehouse: COMPUTE_WH
  database: MYDB
  schema: PUBLIC
  role: ANALYST
```

--------------------------------

### Configure Data Quality Checks in YAML

Source: https://context7.com/sodadata/soda-core/llms.txt

Examples of defining column-level overrides, freshness checks with various time units and filters, aggregate functions, and failed row expressions for data validation.

```yaml
columns:
  - name: score
    valid_min: 0
    valid_max: 100
    checks:
      - invalid:
          valid_min: 50
          valid_max: 100

# Freshness check
checks:
  - freshness:
      column: updated_at
      threshold:
        must_be_less_than: 24

# Aggregate check
columns:
  - name: order_value
    checks:
      - aggregate:
          function: avg
          threshold:
            must_be_greater_than: 50

# Failed rows check
columns:
  - name: price
    checks:
      - failed_rows:
          expression: |
            "price" < 0 OR "price" > 1000000
          threshold:
            must_be: 0
```

--------------------------------

### Define Data Contract

Source: https://github.com/sodadata/soda-core/blob/main/README.md

An example of a contract.yml file defining dataset-level checks and column-specific validation rules such as missing values and invalid value thresholds.

```yaml
dataset: postgres_ds/db/schema/dataset

checks:
  - schema:
  - row_count:

columns:
  - name: id
    checks:
      - missing:
  - name: name
    checks:
      - missing:
          threshold:
            metric: percent
            must_be_less_than: 10
  - name: size
    checks:
      - invalid:
          valid_values: ['S', 'M', 'L']
```

--------------------------------

### Define Data Contract Structure

Source: https://context7.com/sodadata/soda-core/llms.txt

A comprehensive example of a data contract defining dataset identification, variables, dataset-level checks, and column-specific validation rules.

```yaml
dataset: postgres_ds/mydb/public/customers
filter: |
  "status" = 'active'
variables:
  min_rows:
    default: 100
  date_col:
    default: created_at
checks:
  - schema:
  - row_count:
      threshold:
        must_be_greater_than: ${var.min_rows}
columns:
  - name: id
    data_type: integer
    checks:
      - missing:
      - duplicate:
  - name: email
    data_type: varchar
    checks:
      - missing:
          threshold:
            metric: percent
            must_be_less_than: 5
      - invalid:
          valid_format:
            regex: '^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
            name: email_format
```

--------------------------------

### Implement and Use Custom Data Source

Source: https://context7.com/sodadata/soda-core/llms.txt

Demonstrates how to programmatically create and use a custom data source, specifically a PostgreSQL data source. This involves defining connection properties and creating a data source model, which is then used to instantiate an implementation of the data source. This custom data source can then be used in contract verification functions.

```python
from soda_core.contracts.api.verify_api import verify_contract_locally
from soda_postgres.common.data_sources.postgres_data_source import PostgresDataSourceImpl
from soda_postgres.common.data_sources.postgres_data_source_connection import (
    PostgresDataSource,
    PostgresConnectionPassword
)

# Create data source programmatically
connection_props = PostgresConnectionPassword(
    host="localhost",
    port=5432,
    database="mydb",
    user="myuser",
    password="secret"
)

data_source_model = PostgresDataSource(
    name="my_postgres",
    connection_properties=connection_props
)

data_source = PostgresDataSourceImpl(data_source_model)

# Use in verification
result = verify_contract_locally(
    data_sources=[data_source],
    contract_file_path="contract.yml"
)
```

--------------------------------

### Configure and Test Data Source

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to generate a data source configuration template and verify the connection to the data source.

```bash
soda data-source create -f ds_config.yml
soda data-source test -ds ds_config.yml
```

--------------------------------

### Configure and Test Soda Cloud Connection

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to initialize a Soda Cloud configuration file and test the authentication credentials.

```bash
soda cloud create -f sc_config.yml
soda cloud test -sc sc_config.yml
```

--------------------------------

### Create Data Source Configuration Template (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Generates a YAML configuration file template for connecting to a specified data source. This helps in setting up the connection details for Soda Core.

```bash
soda data-source create -f ds_config.yml
soda data-source create -f ds_config.yml -t snowflake
```

--------------------------------

### Test Data Source Connection (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Validates that the data source configuration successfully connects to the data platform. Includes an option for verbose output for debugging purposes.

```bash
soda data-source test -ds ds_config.yml
soda data-source test -ds ds_config.yml -v
```

--------------------------------

### Execute Local Contract Verification with Python API

Source: https://context7.com/sodadata/soda-core/llms.txt

Demonstrates how to use the verify_contract_locally function to run data quality checks, handle results, and optionally publish to Soda Cloud with variables.

```python
from soda_core.contracts.api.verify_api import verify_contract_locally

result = verify_contract_locally(
    data_source_file_path="ds_config.yml",
    contract_file_path="contract.yml"
)

if result.is_passed:
    print("All checks passed!")
elif result.is_failed:
    print(f"Checks failed: {result.number_of_checks_failed}")
```

--------------------------------

### Data Source Management

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to create and test local data source configurations.

```APIDOC
## CLI: soda data-source create

### Description
Generates a template YAML configuration file for a data source.

### Method
CLI Command

### Parameters
#### Options
- **-f, --file** (string) - Required - Output file path for the data source YAML configuration file.

### Request Example
`soda data-source create -f ds_config.yml`

## CLI: soda data-source test

### Description
Validates the connection to a data source using a provided configuration file.

### Parameters
#### Options
- **-ds, --data-source** (string) - Required - Path to a data source YAML configuration file.
```

--------------------------------

### CLI: Data Source Management

Source: https://context7.com/sodadata/soda-core/llms.txt

Commands to create and validate data source configurations for various platforms like PostgreSQL, Snowflake, and BigQuery.

```APIDOC
## CLI: soda data-source

### Description
Manages data source configuration files and validates connectivity to data platforms.

### Methods
- `create`: Generates a YAML configuration template.
- `test`: Validates the connection using the provided configuration.

### Parameters
- **-f** (string) - Required - File path for the configuration YAML.
- **-t** (string) - Optional - Data source type (e.g., postgres, snowflake, bigquery).
- **-ds** (string) - Required - Path to the data source configuration file.
- **-v** (flag) - Optional - Enable verbose output for debugging.

### Request Example
`soda data-source create -f ds_config.yml -t snowflake`

### Response
- **Success** (0) - Configuration file created or connection successful.
- **Failure** (1) - Connection error or invalid configuration.
```

--------------------------------

### Publish Contract and Results to Soda Cloud

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to publish a local contract to the cloud or verify a contract while simultaneously publishing the results to Soda Cloud.

```bash
soda contract publish -c contract.yml -sc sc_config.yml
soda contract verify -ds ds_config.yml -c contract.yml -sc sc_config.yml -p
```

--------------------------------

### Fetch Contract from Soda Cloud (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Downloads a data contract from Soda Cloud to a local file. Supports fetching contracts for single or multiple datasets.

```bash
soda contract fetch -d postgres_ds/db/schema/dataset -sc sc_config.yml -f contract.yml
soda contract fetch -d ds1/db/schema/table1 -d ds1/db/schema/table2 -sc sc_config.yml -f contract1.yml -f contract2.yml
```

--------------------------------

### Publish Contracts using Builder Pattern

Source: https://context7.com/sodadata/soda-core/llms.txt

Publishes data contracts to Soda Cloud using a fluent builder pattern for advanced configuration. This approach allows chaining methods to specify contract and Soda Cloud configuration files before building and executing the publication. The result object contains details of the published contracts.

```python
from soda_core.contracts.contract_publication import ContractPublication

# Build and execute publication
result = (
    ContractPublication.builder()
    .with_contract_yaml_file("contract.yml")
    .with_soda_cloud_yaml_file("sc_config.yml")
    .build()
    .execute()
)

if not result.has_errors:
    for pub_result in result.items:
        print(f"Published: {pub_result.contract.soda_qualified_dataset_name}")
```

--------------------------------

### Soda Cloud Integration

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to configure and test connections to Soda Cloud.

```APIDOC
## CLI: soda cloud create

### Description
Generates a configuration file for Soda Cloud authentication.

### Parameters
#### Options
- **-sc, --soda-cloud** (string) - Required - Output file path for the Soda Cloud YAML configuration file.

## CLI: soda cloud test

### Description
Tests the connection to Soda Cloud using the provided configuration.

### Parameters
#### Options
- **-sc, --soda-cloud** (string) - Required - Path to a Soda Cloud YAML configuration file.
```

--------------------------------

### Run Tests and Pre-commit Checks

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to execute the test suite using pytest and run pre-commit hooks to ensure code quality before submission.

```bash
uv run pytest soda-tests/
uv run pre-commit run --all-files
```

```bash
pytest soda-tests/
pre-commit run --all-files
```

--------------------------------

### Custom Data Source Implementation

Source: https://context7.com/sodadata/soda-core/llms.txt

Programmatically create and use a custom data source for contract verification.

```APIDOC
## Custom Data Source Implementation

### Description
Programmatically create and use a custom data source for contract verification.

### Method
Not applicable (Python function call)

### Endpoint
Not applicable (Python function call)

### Parameters
#### Path Parameters
- **data_sources** (list) - Required - A list of custom data source implementations.
- **contract_file_path** (string) - Required - Path to the contract YAML file.

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from soda_core.contracts.api.verify_api import verify_contract_locally
from soda_postgres.common.data_sources.postgres_data_source import PostgresDataSourceImpl
from soda_postgres.common.data_sources.postgres_data_source_connection import (
    PostgresDataSource,
    PostgresConnectionPassword
)

connection_props = PostgresConnectionPassword(
    host="localhost",
    port=5432,
    database="mydb",
    user="myuser",
    password="secret"
)

data_source_model = PostgresDataSource(
    name="my_postgres",
    connection_properties=connection_props
)

data_source = PostgresDataSourceImpl(data_source_model)

result = verify_contract_locally(
    data_sources=[data_source],
    contract_file_path="contract.yml"
)
```

### Response
#### Success Response (200)
Returns a `VerificationResult` object indicating the outcome of the contract verification against the custom data source.

#### Response Example
```json
{
  "is_ok": true,
  "errors": []
}
```
```

--------------------------------

### Publish Contract to Soda Cloud (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Uploads a local data contract file to Soda Cloud, establishing it as the central source of truth for data quality definitions.

```bash
soda contract publish -c contract.yml -sc sc_config.yml
```

--------------------------------

### Configure Trino Environment Variables

Source: https://github.com/sodadata/soda-core/blob/main/soda-trino/local_instance/README.md

Sets the necessary environment variables for connecting to the Trino instance, including host, port, catalog, and the JWT token.

```env
TRINO_HOST="localhost"
TRINO_PORT=8443
TRINO_CATALOG="system"
TRINO_JWT_TOKEN="{token}"
```

--------------------------------

### Test Trino Instance Connectivity

Source: https://github.com/sodadata/soda-core/blob/main/soda-trino/local_instance/README.md

Verifies that the Trino instance is running and accessible using `curl`. It authenticates with the default user `soda-test` and password `soda-test`.

```bash
curl -k https://localhost:8443/v1/statement -u soda-test:soda-test -d "SELECT 1"
```

--------------------------------

### POST /verify_contract_locally

Source: https://context7.com/sodadata/soda-core/llms.txt

Executes a data contract verification locally using provided configuration files and optional variables.

```APIDOC
## POST /verify_contract_locally

### Description
Executes a local contract verification programmatically by referencing data source and contract configuration files.

### Method
POST

### Endpoint
/verify_contract_locally

### Parameters
#### Request Body
- **data_source_file_path** (string) - Required - Path to the data source configuration YAML.
- **contract_file_path** (string) - Required - Path to the contract definition YAML.
- **soda_cloud_file_path** (string) - Optional - Path to Soda Cloud configuration for publishing results.
- **variables** (object) - Optional - Key-value pairs for dynamic variable injection.
- **publish** (boolean) - Optional - Whether to publish results to Soda Cloud.

### Request Example
{
  "data_source_file_path": "ds_config.yml",
  "contract_file_path": "contract.yml",
  "variables": {"env": "production"},
  "publish": true
}

### Response
#### Success Response (200)
- **is_passed** (boolean) - Overall status of the verification.
- **contract_verification_results** (array) - Detailed list of results per contract.

#### Response Example
{
  "is_passed": true,
  "number_of_checks_failed": 0
}
```

--------------------------------

### CLI: Contract Management

Source: https://context7.com/sodadata/soda-core/llms.txt

Commands to publish, fetch, and manage contract versions via Soda Cloud.

```APIDOC
## CLI: soda contract/request

### Description
Manages the lifecycle of data contracts, including fetching from and publishing to Soda Cloud.

### Methods
- `contract publish`: Uploads a local contract to Soda Cloud.
- `contract fetch`: Downloads a contract from Soda Cloud.
- `request push`: Pushes a change request proposal to Soda Cloud.

### Parameters
- **-c** (string) - Required - Contract file path.
- **-sc** (string) - Required - Soda Cloud configuration file.
- **-d** (string) - Required - Dataset identifier.
- **-r** (integer) - Required - Request ID.

### Request Example
`soda contract fetch -d postgres_ds/db/schema/dataset -sc sc_config.yml -f contract.yml`
```

--------------------------------

### Verify Contract Locally (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Executes a data contract verification scan against a configured data source. Supports options for verbose logging, publishing to Soda Cloud, using variables, and filtering checks.

```bash
soda contract verify -ds ds_config.yml -c contract.yml
soda contract verify -ds ds_config.yml -c contract.yml -v
soda contract verify -ds ds_config.yml -c contract.yml -sc sc_config.yml -p
soda contract verify -ds ds_config.yml -c contract.yml --set "env=production" --set "threshold=100"
soda contract verify -ds ds_config.yml -c contract.yml -cp "columns.id.checks.missing"
soda contract verify -ds ds_config.yml -c contract.yml -cf "type=missing" -cf "column=id"
```

--------------------------------

### Generate Trino JWT Keys

Source: https://github.com/sodadata/soda-core/blob/main/soda-trino/local_instance/README.md

This script generates the necessary RSA keys for JWT authentication in Trino. It should be run from the `local_instance` directory.

```bash
./generate_keys.sh
```

--------------------------------

### Verify Data Contract

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Command to execute a local verification scan against a defined data contract and data source configuration.

```bash
soda contract verify -ds ds_config.yml -c contract.yml
```

--------------------------------

### Verify Contract Using Soda Agent (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Executes contract verification remotely by leveraging Soda Cloud's agent. Allows specifying dataset identifiers and includes options for publishing results and setting custom timeouts.

```bash
soda contract verify -sc sc_config.yml -d postgres_ds/db/schema/dataset -a
soda contract verify -sc sc_config.yml -d postgres_ds/db/schema/dataset -a -p -btm 120
```

--------------------------------

### ContractVerificationSession

Source: https://context7.com/sodadata/soda-core/llms.txt

Execute contract verification with full control over data sources and configuration.

```APIDOC
## ContractVerificationSession

### Description
Execute contract verification with full control over data sources and configuration.

### Method
Not applicable (Python class method)

### Endpoint
Not applicable (Python class method)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **contract_yaml_sources** (list) - Required - A list of `ContractYamlSource` objects.
- **data_source_yaml_sources** (list) - Required - A list of `DataSourceYamlSource` objects.
- **variables** (dict) - Optional - A dictionary of variables to be used in the contract.
- **only_validate_without_execute** (boolean) - Optional - If true, only validates syntax without executing against data.

### Request Example
```python
from soda_core.contracts.contract_verification import ContractVerificationSession
from soda_core.common.yaml import ContractYamlSource, DataSourceYamlSource

contract_yaml = """
dataset: postgres_ds/mydb/public/orders

checks:
  - row_count:
      threshold:
        must_be_greater_than: 0

columns:
  - name: id
    checks:
      - missing:
      - duplicate:
  - name: amount
    checks:
      - aggregate:
          function: sum
          threshold:
            must_be_greater_than: 0
"""

data_source_yaml = """
name: postgres_ds
type: postgres
connection:
  host: localhost
  port: 5432
  database: mydb
  user: myuser
  password: secret
"""

result = ContractVerificationSession.execute(
    contract_yaml_sources=[ContractYamlSource.from_str(contract_yaml)],
    data_source_yaml_sources=[DataSourceYamlSource.from_str(data_source_yaml)],
    variables={"env": "test"},
    only_validate_without_execute=False
)

print(f"Total checks: {result.number_of_checks}")
print(f"Passed: {result.number_of_checks_passed}")
print(f"Failed: {result.number_of_checks_failed}")

if result.is_failed:
    raise Exception(f"Data quality checks failed:\n{result.get_errors_str()}")
```

### Response
#### Success Response (200)
Returns a `ContractVerificationResult` object containing details about the verification process, including the number of checks, passed checks, failed checks, and any errors.

#### Response Example
```json
{
  "number_of_checks": 5,
  "number_of_checks_passed": 4,
  "number_of_checks_failed": 1,
  "is_failed": true,
  "errors": [
    {
      "message": "Row count must be greater than 0 for dataset "postgres_ds/mydb/public/orders"",
      "check_name": "row_count"
    }
  ]
}
```
```

--------------------------------

### Verify Local Contract on Agent

Source: https://context7.com/sodadata/soda-core/llms.txt

Verifies a data contract against a specified data source using an agent. It takes file paths for the Soda Cloud configuration, contract definition, and data source configuration. The function returns a result object indicating success or failure, with detailed errors if verification fails.

```python
result = verify_contract_on_agent(
    soda_cloud_file_path="sc_config.yml",
    contract_file_path="contract.yml",
    data_source_file_path="ds_config.yml",
    publish=True
)

if result.is_ok:
    print("Verification completed successfully")
else:
    print(f"Errors: {result.get_errors_str()}")
```

--------------------------------

### Run Trino JWT Token Test

Source: https://github.com/sodadata/soda-core/blob/main/soda-trino/local_instance/README.md

Executes the `real_jwt_token` test within `test_trino.py` to confirm successful JWT authentication with the configured Trino instance.

```python
Uncomment and run the `real_jwt_token` test in `test_trino.py`.
```

--------------------------------

### Contract Request Management (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Manages contract change requests within Soda Cloud, including fetching proposals, pushing updates, and transitioning request statuses.

```bash
soda request fetch -sc sc_config.yml -r 123 -f proposal.yml
soda request fetch -sc sc_config.yml -r 123 -p 2 -f proposal.yml
soda request push -sc sc_config.yml -r 123 -f proposal.yml -m "Updated schema checks"
soda request transition -sc sc_config.yml -r 123 -s approved
```

--------------------------------

### ContractPublication Builder Pattern

Source: https://context7.com/sodadata/soda-core/llms.txt

Publishes contracts to Soda Cloud using a builder pattern for advanced configuration.

```APIDOC
## ContractPublication Builder Pattern

### Description
Publishes contracts to Soda Cloud using the builder pattern for advanced configuration.

### Method
Not applicable (Python class method)

### Endpoint
Not applicable (Python class method)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from soda_core.contracts.contract_publication import ContractPublication

result = (
    ContractPublication.builder()
    .with_contract_yaml_file("contract.yml")
    .with_soda_cloud_yaml_file("sc_config.yml")
    .build()
    .execute()
)

if not result.has_errors:
    for pub_result in result.items:
        print(f"Published: {pub_result.contract.soda_qualified_dataset_name}")
```

### Response
#### Success Response (200)
Returns a `ContractPublicationResult` object containing details about the publication process, including any published contracts and logs.

#### Response Example
```json
{
  "items": [
    {
      "contract": {
        "soda_qualified_dataset_name": "my_dataset"
      },
      "logs": {
        "errors": []
      }
    }
  ],
  "has_errors": false
}
```
```

--------------------------------

### Execute Contract Verification with Session

Source: https://context7.com/sodadata/soda-core/llms.txt

Executes contract verification using the ContractVerificationSession, providing fine-grained control over data sources and configurations. This method allows loading contracts and data sources from YAML strings and passing runtime variables. It returns a detailed result object summarizing the verification outcomes.

```python
from soda_core.contracts.contract_verification import ContractVerificationSession
from soda_core.common.yaml import ContractYamlSource, DataSourceYamlSource

# Create contract and data source from YAML strings
contract_yaml = """
dataset: postgres_ds/mydb/public/orders

checks:
  - row_count:
      threshold:
        must_be_greater_than: 0

columns:
  - name: id
    checks:
      - missing:
      - duplicate:
  - name: amount
    checks:
      - aggregate:
          function: sum
          threshold:
            must_be_greater_than: 0
"""

data_source_yaml = """
name: postgres_ds
type: postgres
connection:
  host: localhost
  port: 5432
  database: mydb
  user: myuser
  password: secret
"""

# Execute verification
result = ContractVerificationSession.execute(
    contract_yaml_sources=[ContractYamlSource.from_str(contract_yaml)],
    data_source_yaml_sources=[DataSourceYamlSource.from_str(data_source_yaml)],
    variables={"env": "test"},
    only_validate_without_execute=False
)

# Process results
print(f"Total checks: {result.number_of_checks}")
print(f"Passed: {result.number_of_checks_passed}")
print(f"Failed: {result.number_of_checks_failed}")

if result.is_failed:
    raise Exception(f"Data quality checks failed:\n{result.get_errors_str()}")
```

--------------------------------

### Publish Contract to Soda Cloud

Source: https://context7.com/sodadata/soda-core/llms.txt

Publishes a contract file to Soda Cloud.

```APIDOC
## Publish Contract to Soda Cloud

### Description
Publishes a contract file to Soda Cloud.

### Method
Not applicable (Python function call)

### Endpoint
Not applicable (Python function call)

### Parameters
#### Path Parameters
- **contract_file_path** (string) - Required - Path to the contract YAML file.
- **soda_cloud_file_path** (string) - Required - Path to the Soda Cloud configuration file.

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
result = publish_contract(
    contract_file_path="contract.yml",
    soda_cloud_file_path="sc_config.yml"
)

if result.has_errors:
    print(f"Publication failed: {result.logs.get_errors_str()}")
else:
    print(f"Published {len(result)} contract(s)")
    for pub_result in result:
        print(f"  Dataset: {pub_result.contract.soda_qualified_dataset_name}")
```

### Response
#### Success Response (200)
Indicates successful publication. The `result` object contains details about the published contracts.

#### Response Example
```json
[
  {
    "contract": {
      "soda_qualified_dataset_name": "my_dataset"
    },
    "logs": {
      "errors": []
    }
  }
]
```
```

--------------------------------

### Execute Remote Contract Verification via Soda Agent

Source: https://context7.com/sodadata/soda-core/llms.txt

Shows how to trigger a remote contract verification using the Soda Agent, specifying dataset identifiers and blocking timeouts.

```python
from soda_core.contracts.api.verify_api import verify_contract_on_agent

result = verify_contract_on_agent(
    soda_cloud_file_path="sc_config.yml",
    dataset_identifier="postgres_ds/mydb/public/customers",
    publish=True,
    blocking_timeout_in_minutes=30
)
```

--------------------------------

### Contract Management

Source: https://github.com/sodadata/soda-core/blob/main/README.md

Commands to verify and publish data contracts.

```APIDOC
## CLI: soda contract verify

### Description
Evaluates a data contract against a data source locally.

### Parameters
#### Options
- **-ds, --data-source** (string) - Required - Path to a data source YAML configuration file.
- **-c, --contract** (string) - Required - Path to a data contract YAML configuration file.
- **-sc, --soda-cloud** (string) - Optional - Path to a Soda Cloud YAML configuration file.
- **-p, --publish** (boolean) - Optional - Publish results and contract to Soda Cloud.

## CLI: soda contract publish

### Description
Publishes a local contract to Soda Cloud as the source of truth.

### Parameters
#### Options
- **-c, --contract** (string) - Required - Path to a contract YAML file.
- **-sc, --soda-cloud** (string) - Required - Path to Soda Cloud YAML configuration file.
```

--------------------------------

### Airflow Integration: Verify Data Quality Contracts

Source: https://context7.com/sodadata/soda-core/llms.txt

This Python code demonstrates how to integrate Soda Core contract verification into an Apache Airflow DAG. It defines a callable function to execute contract verification locally using specified data source, contract, and Soda Cloud configuration files, and raises an exception if checks fail. The DAG includes a PythonOperator task to run this verification daily.

```python
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
from soda_core.contracts.api.verify_api import verify_contract_locally

def verify_data_quality(**context):
    """Execute Soda contract verification."""
    result = verify_contract_locally(
        data_source_file_path="/opt/airflow/configs/ds_config.yml",
        contract_file_path="/opt/airflow/contracts/orders_contract.yml",
        soda_cloud_file_path="/opt/airflow/configs/sc_config.yml",
        variables={
            "execution_date": context["ds"],
            "min_rows": 1000
        },
        publish=True
    )

    if result.is_failed:
        raise Exception(
            f"Data quality checks failed: {result.number_of_checks_failed} checks\n"
            f"{result.get_errors_str()}"
        )

    return {
        "checks_total": result.number_of_checks,
        "checks_passed": result.number_of_checks_passed,
        "checks_failed": result.number_of_checks_failed
    }

with DAG(
    dag_id="data_quality_verification",
    schedule_interval="@daily",
    start_date=days_ago(1),
    catchup=False
) as dag:

    verify_task = PythonOperator(
        task_id="verify_orders_contract",
        python_callable=verify_data_quality,
        provide_context=True
    )
```

--------------------------------

### Implement Schema and Row Count Checks

Source: https://context7.com/sodadata/soda-core/llms.txt

Validation rules for ensuring table schemas match expectations and verifying row counts with thresholds and filters.

```yaml
checks:
  - schema:
      allow_extra_columns: true
      allow_other_column_order: true

checks:
  - row_count:
      threshold:
        must_be_between:
          greater_than_or_equal: 100
          less_than_or_equal: 10000
```

--------------------------------

### Generate JWT Token for Trino

Source: https://github.com/sodadata/soda-core/blob/main/soda-trino/local_instance/README.md

This script generates a JWT token that can be used for authentication with the Trino instance. It relies on the keys generated previously.

```bash
./generate_jwt_token.sh
```

--------------------------------

### Test Contract Syntax (Bash)

Source: https://context7.com/sodadata/soda-core/llms.txt

Validates the syntax of a contract YAML file without executing the data quality checks. Useful for ensuring the contract file is correctly formatted.

```bash
soda contract test -c contract.yml
soda contract test -c contract.yml -v
```

--------------------------------

### Execute Remote Contract Verification via Soda Agent

Source: https://github.com/sodadata/soda-core/blob/main/README.md

This command triggers a contract verification process using a configured Soda Agent. It requires a Soda Cloud configuration file and the specific dataset identifier retrieved from the Soda Cloud interface.

```bash
soda contract verify -sc sc_config.yml -d postgres_ds/db/schema/dataset -a
```

--------------------------------

### CLI: Contract Verification

Source: https://context7.com/sodadata/soda-core/llms.txt

Commands to execute data quality checks defined in YAML contracts against connected data sources.

```APIDOC
## CLI: soda contract verify

### Description
Executes data quality verification scans based on a defined contract file.

### Parameters
- **-ds** (string) - Required - Data source configuration file.
- **-c** (string) - Required - Contract YAML file.
- **-sc** (string) - Optional - Soda Cloud configuration file for publishing results.
- **-p** (flag) - Optional - Publish results to Soda Cloud.
- **--set** (string) - Optional - Override variables in the contract.
- **-cp** (string) - Optional - Filter specific checks to run.

### Request Example
`soda contract verify -ds ds_config.yml -c contract.yml -p`

### Response
- **Success** (0) - Verification completed successfully.
- **Failure** (1) - Verification failed due to data quality issues or configuration errors.
```

--------------------------------

### Publish Contract to Soda Cloud

Source: https://context7.com/sodadata/soda-core/llms.txt

Publishes a data contract to Soda Cloud. This function requires the path to the contract file and the Soda Cloud configuration file. It returns a result object that contains logs and details about the publication status, including any errors encountered.

```python
from soda_core.contracts.api.publish_api import publish_contract

# Publish contract to Soda Cloud
result = publish_contract(
    contract_file_path="contract.yml",
    soda_cloud_file_path="sc_config.yml"
)

if result.has_errors:
    print(f"Publication failed: {result.logs.get_errors_str()}")
else:
    print(f"Published {len(result)} contract(s)")
    for pub_result in result:
        print(f"  Dataset: {pub_result.contract.soda_qualified_dataset_name}")
```

--------------------------------

### POST /verify_contract_on_agent

Source: https://context7.com/sodadata/soda-core/llms.txt

Triggers a remote contract verification process managed by a Soda Agent.

```APIDOC
## POST /verify_contract_on_agent

### Description
Executes contract verification remotely using a Soda Agent, typically used for cloud-managed data pipelines.

### Method
POST

### Endpoint
/verify_contract_on_agent

### Parameters
#### Request Body
- **soda_cloud_file_path** (string) - Required - Path to Soda Cloud configuration.
- **dataset_identifier** (string) - Required - The fully qualified identifier for the dataset.
- **publish** (boolean) - Optional - Whether to publish results.
- **blocking_timeout_in_minutes** (integer) - Optional - Timeout for the blocking request.

### Request Example
{
  "soda_cloud_file_path": "sc_config.yml",
  "dataset_identifier": "postgres_ds/mydb/public/customers",
  "publish": true,
  "blocking_timeout_in_minutes": 30
}

### Response
#### Success Response (200)
- **status** (string) - The status of the remote verification job.

#### Response Example
{
  "status": "success"
}
```

--------------------------------

### Verify Contract on Agent

Source: https://context7.com/sodadata/soda-core/llms.txt

Verifies a contract by executing it against data sources configured via agent settings.

```APIDOC
## Verify Contract on Agent

### Description
Verifies a contract by executing it against data sources configured via agent settings.

### Method
Not applicable (Python function call)

### Endpoint
Not applicable (Python function call)

### Parameters
#### Path Parameters
- **soda_cloud_file_path** (string) - Required - Path to the Soda Cloud configuration file.
- **contract_file_path** (string) - Required - Path to the contract YAML file.
- **data_source_file_path** (string) - Required - Path to the data source configuration file.

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
result = verify_contract_on_agent(
    soda_cloud_file_path="sc_config.yml",
    contract_file_path="contract.yml",
    data_source_file_path="ds_config.yml",
    publish=True
)

if result.is_ok:
    print("Verification completed successfully")
else:
    print(f"Errors: {result.get_errors_str()}")
```

### Response
#### Success Response (200)
Indicates successful verification. The `result` object contains details about the verification.

#### Response Example
```json
{
  "is_ok": true,
  "errors": []
}
```
```

--------------------------------

### Validate JWT Token with Trino

Source: https://github.com/sodadata/soda-core/blob/main/soda-trino/local_instance/README.md

Tests the validity of a generated JWT token by sending a query to the Trino instance with the token in the Authorization header.

```bash
curl -k https://localhost:8443/v1/query   -H "Authorization: Bearer {token}"
```

--------------------------------

### Pytest Integration: Validate Data Quality with Soda Core

Source: https://context7.com/sodadata/soda-core/llms.txt

This Python code showcases how to use Soda Core within pytest test suites for data validation. It includes pytest fixtures for configuration file paths and defines test functions to verify data quality, freshness, and schema compliance using `verify_contract_locally`. Assertions are used to check the results of the verification, ensuring data meets defined quality standards.

```python
import pytest
from soda_core.contracts.api.verify_api import verify_contract_locally
from soda_core.contracts.contract_verification import ContractVerificationSessionResult

@pytest.fixture
def data_source_path():
    return "tests/configs/ds_config.yml"

@pytest.fixture
def soda_cloud_path():
    return "tests/configs/sc_config.yml"

def test_customer_data_quality(data_source_path, soda_cloud_path):
    """Verify customer data meets quality standards."""
    result: ContractVerificationSessionResult = verify_contract_locally(
        data_source_file_path=data_source_path,
        contract_file_path="contracts/customers.yml"
    )

    assert not result.has_errors, f"Errors occurred: {result.get_errors_str()}"
    assert result.is_passed, (
        f"Data quality checks failed:\n"
        f"  Total: {result.number_of_checks}\n"
        f"  Failed: {result.number_of_checks_failed}"
    )

def test_orders_freshness(data_source_path, soda_cloud_path):
    """Verify orders data is fresh."""
    result = verify_contract_locally(
        data_source_file_path=data_source_path,
        contract_file_path="contracts/orders_freshness.yml",
        variables={"freshness_hours": 24}
    )

    # Check specific freshness result
    freshness_check = result.contract_verification_results[0].check_results[0]
    assert freshness_check.is_passed, (
        f"Data is stale. Freshness: "
        f"{freshness_check.diagnostic_metric_values.get('freshness_in_hours')}" hours"
    )

@pytest.mark.parametrize("table_name", ["orders", "customers", "products"])
def test_table_schema_compliance(data_source_path, table_name):
    """Verify all tables match their schema contracts."""
    result = verify_contract_locally(
        data_source_file_path=data_source_path,
        contract_file_path=f"contracts/{table_name}_schema.yml"
    )

    assert result.is_passed, f"Schema mismatch for {table_name}"
```

--------------------------------

### Test Contract Syntax

Source: https://context7.com/sodadata/soda-core/llms.txt

Validates the YAML syntax of a contract file without executing it against any data.

```APIDOC
## Test Contract Syntax

### Description
Validates the YAML syntax of a contract file without executing it against any data.

### Method
Not applicable (Python function call)

### Endpoint
Not applicable (Python function call)

### Parameters
#### Path Parameters
- **contract_file_path** (string) - Required - Path to the contract YAML file.

#### Query Parameters
- **variables** (dict) - Optional - A dictionary of variables to be used in the contract.

#### Request Body
None

### Request Example
```python
result = test_contract(
    contract_file_path="contract.yml",
    variables={"threshold": 100}
)

if result.has_errors:
    print("Contract validation errors:")
    for error in result.get_errors():
        print(f"  - {error}")
else:
    print("Contract syntax is valid")
```

### Response
#### Success Response (200)
Indicates successful syntax validation. The `result` object contains any validation errors found.

#### Response Example
```json
{
  "has_errors": false,
  "errors": []
}
```
```

--------------------------------

### Test Contract Syntax

Source: https://context7.com/sodadata/soda-core/llms.txt

Validates the syntax of a contract YAML file without executing it against any data. This is useful for ensuring the contract is well-formed before running actual data quality checks. It accepts the contract file path and optional variables for templated contracts.

```python
from soda_core.contracts.api.test_api import test_contract

# Test contract syntax
result = test_contract(
    contract_file_path="contract.yml",
    variables={"threshold": 100}
)

if result.has_errors:
    print("Contract validation errors:")
    for error in result.get_errors():
        print(f"  - {error}")
else:
    print("Contract syntax is valid")
```

--------------------------------

### Implement Missing and Duplicate Value Checks

Source: https://context7.com/sodadata/soda-core/llms.txt

Column-level validation to detect NULLs or missing values and identify duplicate records using thresholds and custom filters.

```yaml
columns:
  - name: status
    missing_values: ['N/A', 'NULL', '']
    checks:
      - missing:
          threshold:
            must_be_less_than: 50

  - name: order_id
    checks:
      - duplicate:
          filter: |
            "status" = 'completed'
```

--------------------------------

### Implement Invalid Value Checks

Source: https://context7.com/sodadata/soda-core/llms.txt

Validation rules for column values using valid value lists, regex patterns, numeric ranges, and length constraints.

```yaml
columns:
  - name: phone
    valid_format:
      regex: '^\+?[1-9]\d{1,14}$'
      name: e164_phone
    checks:
      - invalid:

  - name: country_code
    checks:
      - invalid:
          valid_min_length: 2
          valid_max_length: 3
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.