### Enter REPL Mode (Example)

Source: https://docs.databricks.com/aws/en/dev-tools/databricks-sql-cli

Example of starting the Databricks SQL CLI in REPL mode for the 'default' database.

```bash
dbsqlcli default
```

--------------------------------

### Setup Instructions

Source: https://docs.databricks.com/aws/en/notebooks/source/monitoring/snapshot-monitor-legacy.html

This section outlines the setup steps required before running the snapshot monitor, including cluster verification, SDK installation, and defining table parameters.

```Markdown
%md
## Setup
* Verify cluster configuration
* Install the Python SKD
* Define catalog, schema and table names
```

--------------------------------

### Database Instance with Catalog Example

Source: https://docs.databricks.com/aws/en/dev-tools/bundles/resources

An example demonstrating how to define a database instance and associate it with a database catalog. This shows the relationship between instance configuration and catalog setup.

```yaml
resources:  
  database_instances:  
    my_instance:  
      name: my-instance  
      capacity: CU_1  
  database_catalogs:  
    my_catalog:  
      database_instance_name: ${resources.database_instances.my_instance.name}  
      name: example_catalog  
      database_name: my_database  
      create_database_if_not_exists: true  

```

--------------------------------

### Run Quickstart Scripts

Source: https://docs.databricks.com/aws/en/generative-ai/agent-framework/author-agent

Execute quickstart scripts to install dependencies and start the agent application locally.

```bash
uv run quickstart
uv run start-app
```

--------------------------------

### Setup and Data Insertion

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-qry-select-where

Creates a 'person' table and inserts sample data for demonstrating WHERE clause examples.

```SQL
> CREATE TABLE person (id INT, name STRING, age INT);
> INSERT INTO person VALUES
    (100, 'John',   30),
    (200, 'Mary', NULL),
    (300, 'Mike',   80),
    (400, 'Dan' ,   50);
```

--------------------------------

### Get input file block start offset

Source: https://docs.databricks.com/aws/en/pyspark/reference/functions/input_file_block_start

This example demonstrates how to use the input_file_block_start function to get the starting offset of the input file block. It reads a text file and selects the input_file_block_start column.

```Python
from pyspark.sql import functions as sf
df = spark.read.text("python/test_support/sql/ages_newlines.csv", lineSep=",")
df.select(sf.input_file_block_start()).show()
```

--------------------------------

### Run Local Agent Setup and Start

Source: https://docs.databricks.com/aws/en/generative-ai/agent-framework/multi-agent-apps

Configure Databricks authentication and start the agent server and chat UI locally. The `quickstart` script sets up authentication and MLflow experiments, while `start-app` launches the server.

```bash
uv run quickstart  
uv run start-app  

```

--------------------------------

### Use Catalog Example

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-names

Shows how to switch to an existing catalog.

```sql
> USE CATALOG hive_metastore;
```

--------------------------------

### Instrumentation Guidance with Autolog

Source: https://docs.databricks.com/aws/en/mlflow3/genai/getting-started/genie-code

Use this code snippet to add tracing to your code with `autolog()`. This is a basic example for getting started with MLflow tracing.

```python
import mlflow

mlflow.autolog()
```

--------------------------------

### SQL Examples with EXECUTE IMMEDIATE and USING

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-parameter-marker

These SQL examples demonstrate using `EXECUTE IMMEDIATE` with `USING` for type casting, view creation, and table creation with default values and comments.

```sql
> EXECUTE IMMEDIATE 'SELECT 1::DECIMAL(?, ?)' USING 6, 4;  
  1.0000  
  
> EXECUTE IMMEDIATE 'CREATE VIEW v(c1 INT) AS SELECT ? AS c1' USING 10;  
> SELECT * FROM v;  
  10  
  
> EXECUTE IMMEDIATE 'CREATE TABLE T(c1 INT DEFAULT ? COMMENT \'This is a \' ?)' USING 17, 'comment';  
```

--------------------------------

### Create MLflow Experiment

Source: https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/concepts/judges/guidelines

This step involves creating an MLflow experiment. Refer to the MLflow setup your environment quickstart guide for detailed instructions.

--------------------------------

### Directory Structure Example

Source: https://docs.databricks.com/aws/en/notebooks/best-practices

Illustrates the expected file and directory structure after setting up notebooks and tests.

```text
├── covid_analysis  
│  └── transforms.py  
├── notebooks  
│  ├── covid_eda_modular  
│  ├── covid_eda_raw (optional)  
│  └── run_unit_tests  
├── requirements.txt  
└── tests  
    ├── testdata.csv  
    └── transforms_test.py  


```

--------------------------------

### Create a database instance

Source: https://docs.databricks.com/aws/en/dev-tools/cli/reference/database-commands

Use this command to provision a new database instance. You can specify the capacity SKU and whether the instance should start in a stopped state. For advanced configurations, use the `--json` option.

```bash
databricks database create-database-instance NAME [flags]
```

```bash
databricks database create-database-instance my-instance --capacity CU_1
```

--------------------------------

### PySpark nth_value Example 1: Get First Value

Source: https://docs.databricks.com/aws/en/pyspark/reference/functions/nth_value

Demonstrates how to retrieve the first value within a window frame using nth_value. Requires DataFrame setup and Window specification.

```python
from pyspark.sql import functions as sf
from pyspark.sql import Window
df = spark.createDataFrame(
    [("a", 1), ("a", 2), ("a", 3), ("b", 8), ("b", 2)], ["c1", "c2"])
df.show()
```

```python
w = Window.partitionBy("c1").orderBy("c2")
df.withColumn("nth_value", sf.nth_value("c2", 1).over(w)).show()
```

--------------------------------

### SQL Example: Creating Views and Listing All Views

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-aux-show-views

This example demonstrates creating various types of views (permanent, temporary, global temporary) and then listing all views in the default schema.

```sql
-- Create views in different schemas, also create global/local temp views.  
> CREATE VIEW sam AS SELECT id, salary FROM employee WHERE name = 'sam';  
> CREATE VIEW sam1 AS SELECT id, salary FROM employee WHERE name = 'sam1';  
> CREATE VIEW suj AS SELECT id, salary FROM employee WHERE name = 'suj';  
> USE SCHEMA usersc;  
> CREATE VIEW user1 AS SELECT id, salary FROM default.employee WHERE name = 'user1';  
> CREATE VIEW user2 AS SELECT id, salary FROM default.employee WHERE name = 'user2';  
> USE SCHEMA default;  
> CREATE TEMP VIEW temp1 AS SELECT 1 AS col1;  
> CREATE TEMP VIEW temp2 AS SELECT 1 AS col1;  
  
-- List all views in default schema  
> SHOW VIEWS;  
  namespace     viewName     isTemporary  
 ------------- ------------ --------------  
  default       sam          false  
  default       sam1         false  
  default       suj          false  
                temp2        true  
  
```

--------------------------------

### Run and Debug Databricks Connect App with Databricks CLI

Source: https://docs.databricks.com/aws/en/dev-tools/databricks-connect/python/tutorial-apps

Use the `databricks apps run-local` command to run and debug your application. This command handles environment setup, dependency installation, and starts the app with a debugger.

```bash
databricks apps run-local --prepare-environment --debug
```

--------------------------------

### Example: Get a specific job using `api get`

Source: https://docs.databricks.com/aws/en/dev-tools/cli/reference/api-commands

This example demonstrates how to retrieve details for a specific job by providing the job ID in a JSON payload to the `api get` command.

```bash
databricks api get /api/2.0/jobs/get --json '{"job_id": 123}'
```

--------------------------------

### Create a group with users

Source: https://docs.databricks.com/aws/en/sql/language-manual/security-create-group

This example demonstrates how to create a group and immediately add specified users as its members.

```sql
-- Create tv_aliens with Alf and Thor as members.  
CREATE GROUP tv_aliens WITH USER `alf@melmak.et`, `thor@asgaard.et`;  

```

--------------------------------

### Multiple Partitioning Hints Example

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-qry-select-hints

Shows how multiple partitioning hints can be specified in a single query. The optimizer picks the leftmost hint when multiple are present.

```sql
-- multiple partitioning hints  
> EXPLAIN EXTENDED SELECT /*+ REPARTITION(100), COALESCE(500), REPARTITION_BY_RANGE(3, c) */ * FROM t;  
```

--------------------------------

### Get Start Point of Linestring with ZM Coordinates

Source: https://docs.databricks.com/aws/en/sql/language-manual/functions/st_startpoint

Shows how to get the starting point of a linestring that has ZM (Z and M) coordinates using st_startpoint and st_geogfromtext.

```SQL
-- Returns first point with ZM coordinates.  
> SELECT st_asewkt(st_startpoint(st_geogfromtext('LINESTRING ZM (1 2 3 4,5 6 7 8)')));  
  SRID=4326;POINT ZM (1 2 3 4)
```

--------------------------------

### Full Configuration Example

Source: https://docs.databricks.com/aws/en/repos/serverless-private-git

Provides a comprehensive example of a Git configuration file, including default settings for SSL verification, CA certificates, HTTP proxy, and custom port, along with specific configurations for multiple remotes.

```json
{
  "default": {
    "sslVerify": true,
    "caCertPath": "/Workspace/my_ca_cert.pem",
    "httpProxy": "https://git-proxy-server.company.com",
    "customHttpPort": "8080"
  },
  "remotes": [
    {
      "urlPrefix": "https://my-private-git.company.com/",
      "caCertPath": "/Workspace/my_ca_cert_2.pem"
    },
    {
      "urlPrefix": "https://another-git-server.com/project.git",
      "sslVerify": false
    }
  ]
}

```

--------------------------------

### Install Faker Package for PySpark Data Source Example

Source: https://docs.databricks.com/aws/en/pyspark/datasources

Install the 'faker' package, which is used in the example to generate fake data for the custom data source.

```python
%pip install faker
```

--------------------------------

### Setup API Utilities for Managed Ingestion Pipelines

Source: https://docs.databricks.com/aws/en/notebooks/source/saas-cdc/wiz-audit-logs-pipeline.html

Imports necessary libraries and defines utility functions for interacting with the Databricks Pipelines API. Includes functions for checking API responses, creating, editing, deleting, listing, getting, and starting pipelines.

```python
import requests
import json
notebook_context = dbutils.notebook.entry_point.getDbutils().notebook().getContext()
api_token = notebook_context.apiToken().get()
workspace_url = notebook_context.apiUrl().get()
api_url = f"{workspace_url}/api/2.0/pipelines"
headers = {"Authorization": "Bearer {}".format(api_token), "Content-Type": "application/json"}
def check_response(response):
if response.status_code == 200:
print("Response from API:\n{}".format(json.dumps(response.json(), indent=2, sort_keys=False)))
else:
print(
f"Failed to retrieve data: error_code={response.status_code}, error_message={response.json().get
('message', response.text)}"
)
def create_pipeline(pipeline_definition: str):
response = requests.post(url=api_url, headers=headers, data=pipeline_definition)
check_response(response)
def edit_pipeline(id: str, pipeline_definition: str):
response = requests.put(url=f"{api_url}/{id}", headers=headers, data=pipeline_definition)
check_response(response)
def delete_pipeline(id: str):
response = requests.delete(url=f"{api_url}/{id}", headers=headers)
check_response(response)
def list_pipeline(filter: str):
body = "" if len(filter) == 0 else f"{{\"filter\": \"{filter}\"}}"
response = requests.get(url=api_url, headers=headers, data=body)
check_response(response)
def get_pipeline(id: str):
response = requests.get(url=f"{api_url}/{id}", headers=headers)
check_response(response)
def start_pipeline(id: str, full_refresh: bool = False):
body = f"
{{
\"full_refresh\": {str(full_refresh).lower()},
\"validate_only\": false,
\"cause\": \"API_CALL\"
}}"
response = requests.post(url=f"{api_url}/{id}/updates", headers=headers, data=body)
check_response(response)
def stop_pipeline(id: str):
print("cannot stop pipeline")
```

--------------------------------

### SQL EXPLAIN - Example

Source: https://docs.databricks.com/aws/en/optimizations/aqe

Demonstrates how to use SQL EXPLAIN to view query plans. Since the query is not executed, the plan shown is always the initial plan.

```sql
EXPLAIN SELECT * FROM my_table
```

--------------------------------

### Example: List available clusters using `api get`

Source: https://docs.databricks.com/aws/en/dev-tools/cli/reference/api-commands

This example shows how to retrieve a list of all available clusters in your Databricks workspace by making a GET request to the clusters list API endpoint.

```bash
databricks api get /api/2.0/clusters/list
```

--------------------------------

### Install Required Packages

Source: https://docs.databricks.com/aws/en/notebooks/source/sgc-examples/sgc-gpt-oss-120b-ddp-fsdp.html

Installs essential Python libraries for distributed training and model fine-tuning. Ensure these packages are installed before proceeding with the training setup.

```python
%pip install "trl==1.1.0"
%pip install "peft==0.19.1"
%pip install "transformers==5.5.4"
%pip install "fsspec==2024.9.0"
%pip install "huggingface_hub==1.11.0"
%pip install "datasets==3.2.0"
%pip install "accelerate==1.13.0"
%restart_python
```

--------------------------------

### Full Example of Postgres Database Configuration

Source: https://docs.databricks.com/aws/en/dev-tools/bundles/resources

Illustrates a complete setup for Postgres databases, including projects, branches, roles, and the database itself. This example shows how to link resources and configure database ownership.

```yaml
resources:  
  postgres_projects:  
    my_project:  
      project_id: test-pg-proj  
      display_name: 'Test Project for Database'  
      pg_version: 16  
      history_retention_duration: '604800s'  
  
  postgres_branches:  
    main:  
      parent: ${resources.postgres_projects.my_project.id}  
      branch_id: main  
      no_expiry: true  
  
  postgres_roles:  
    owner:  
      parent: ${resources.postgres_branches.main.id}  
      role_id: app-owner  
      postgres_role: app_owner  
  
  postgres_databases:  
    my_database:  
      parent: ${resources.postgres_branches.main.id}  
      database_id: my-database  
      postgres_database: app_db  
      # The live API requires `role`. Declare an explicit role so the bundle is  
      # portable across users (the auto-created project-owner role's id is  
      # derived from the creator's identity).  
      role: ${resources.postgres_roles.owner.id}  

```

--------------------------------

### Install PostgreSQL Extensions with Dependencies

Source: https://docs.databricks.com/aws/en/oltp/projects/extensions

Install an extension and its required dependencies automatically by using the `CASCADE` option. This example installs `postgis_topology` and its base `postgis` dependency.

```SQL
CREATE EXTENSION postgis_topology CASCADE;
```

--------------------------------

### Example Bundle Project Structure

Source: https://docs.databricks.com/aws/en/dev-tools/bundles/templates

After initializing a new bundle, the project folder will contain configuration files, resources, and source code.

```text
my_test_bundle  
   ├── databricks.yml  
   ├── resources  
   │  └── my_test_bundle_job.yml  
   └── src  
      └── simple_notebook.ipynb  

```

--------------------------------

### Scala Spark Shell Output Example

Source: https://docs.databricks.com/aws/en/dev-tools/databricks-connect-legacy

This is an example of the output you will see when starting the Scala shell with Databricks Connect.

```text
Setting default log level to "WARN".  
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).  
../../.. ..:..:.. WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
Spark context Web UI available at http://...  
Spark context available as 'sc' (master = local[*], app id = local-...).  
Spark session available as 'spark'.  
Welcome to  
      ____              __  
     / __/__  ___ _____/ /__  
    _ \/ _ \/ _ `/ __/  '_/  
   /___/ .__/,/ _/ /_/\   version 3...  
      /_/  
  
Using Scala version 2... (OpenJDK 64-Bit Server VM, Java 1.8...)  
Type in expressions to have them evaluated.  
Type :help for more information.  
  
scala>  
```

--------------------------------

### Example: Generate App Configuration and Download Files

Source: https://docs.databricks.com/aws/en/dev-tools/cli/bundle-commands

This command generates a new YAML file for the app in the 'resources' folder and downloads the app's code files to the bundle's 'src' folder. It uses the app name 'hello_world'.

```bash
databricks bundle generate app --existing-app-name "hello_world"  
```

--------------------------------

### Install Java Dependencies with Maven

Source: https://docs.databricks.com/aws/en/oltp/projects/external-apps-connect

Installs Java dependencies using Maven, assuming a Maven project setup.

```bash
mvn install  
```

--------------------------------

### Example: Define Database Instance and Catalog

Source: https://docs.databricks.com/aws/en/dev-tools/bundles/resources

This example demonstrates how to define a database instance and a corresponding database catalog. It shows how to reference the database instance name using a variable.

```yaml
resources:
  database_instances:
    my_instance:
      name: my-instance
      capacity: CU_1
  database_catalogs:
    my_catalog:
      database_instance_name: ${resources.database_instances.my_instance.name}
      name: example_catalog
      database_name: my_database
      create_database_if_not_exists: true
```

--------------------------------

### Install Go Dependencies

Source: https://docs.databricks.com/aws/en/oltp/projects/external-apps-connect

Initializes a Go module and installs the Databricks SDK and PostgreSQL driver.

```bash
go mod init myapp  
go get github.com/databricks/databricks-sdk-go  
go get github.com/jackc/pgx/v5  
```

--------------------------------

### Install a specific version with pip

Source: https://docs.databricks.com/aws/en/dev-tools/sdk-python

Install a specific version of the databricks-sdk package, for example, version 0.1.6. Useful for testing or compatibility.

```bash
pip3 install databricks-sdk==0.1.6
```

--------------------------------

### Setuptools Project Structure

Source: https://docs.databricks.com/aws/en/dev-tools/bundles/python-wheel

Example directory structure for a Python package using Setuptools.

```tree
├── src  
│     └── my_package  
│           ├── __init__.py  
│           ├── main.py  
│           └── my_module.py  
└── setup.py  

```

--------------------------------

### Example 2: Negative Start Index

Source: https://docs.databricks.com/aws/en/pyspark/reference/functions/slice

Illustrates slicing an array using a negative start index to count from the end of the array.

```APIDOC
## slice(x, start, length)

### Description
Slicing with negative start index.

### Request Example
```python
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3],), ([4, 5],)], ['x'])
df.select(sf.slice(df.x, -1, 1)).show()
```

### Response Example
```
+---------------+  
|slice(x, -1, 1)|
+---------------+  
|            [3]|
|            [5]|
+---------------+  
```
```

--------------------------------

### Example: Create a standard catalog named 'example'

Source: https://docs.databricks.com/aws/en/catalogs/create-catalog

This is an example of creating a standard catalog with the name 'example'.

```sql
CREATE CATALOG IF NOT EXISTS example;  
```

--------------------------------

### Start Airflow Scheduler

Source: https://docs.databricks.com/aws/en/jobs/how-to/use-airflow-with-jobs

Starts the Airflow scheduler, which is responsible for scheduling DAGs. Run this in a new terminal from the Airflow installation directory.

```bash
pipenv shell
export AIRFLOW_HOME=$(pwd)
airflow scheduler
```

--------------------------------

### Get current user with parentheses

Source: https://docs.databricks.com/aws/en/sql/language-manual/functions/current_user

This example demonstrates how to call the `current_user` function with parentheses to get the current user's identifier.

```sql
SELECT current_user();
```

--------------------------------

### Databricks Bundle Plan Output Example

Source: https://docs.databricks.com/aws/en/dev-tools/cli/bundle-commands

Example output showing the actions that will be performed when planning a bundle deployment.

```text
Building python_artifact...
create jobs.my_bundle_job
create pipelines.my_bundle_pipeline
```

--------------------------------

### Example: Full Postgres Synced Table Setup

Source: https://docs.databricks.com/aws/en/dev-tools/bundles/resources

Demonstrates a comprehensive configuration for a Postgres synced table, including pipeline storage, project, catalog, and table-specific settings. This example shows how to create a new pipeline for the synced table using `new_pipeline_spec`.

```yaml
resources:
  schemas:
    pipeline_storage:
      name: test_pipeline_storage
      catalog_name: main
      comment: 'Pipeline storage for the synced-table test'

  postgres_projects:
    my_project:
      project_id: test-pg-proj
      display_name: 'Test Project for Synced Table'
      pg_version: 17

  postgres_catalogs:
    my_catalog:
      catalog_id: lakebase_test
      branch: ${resources.postgres_projects.my_project.id}/branches/production
      postgres_database: appdb
      create_database_if_missing: true

  postgres_synced_tables:
    my_table:
      synced_table_id: ${resources.postgres_catalogs.my_catalog.catalog_id}.public.trips_synced
      source_table_full_name: main.source_test.trips_source
      primary_key_columns: ['tpep_pickup_datetime']
      scheduling_policy: SNAPSHOT
      postgres_database: appdb
      branch: ${resources.postgres_projects.my_project.id}/branches/production
      create_database_objects_if_missing: true
      new_pipeline_spec:
        storage_catalog: ${resources.schemas.pipeline_storage.catalog_name}
        storage_schema: ${resources.schemas.pipeline_storage.name}

```

--------------------------------

### Get Exterior Ring of an Empty Polygon

Source: https://docs.databricks.com/aws/en/pyspark/reference/functions/st_exteriorring

This example demonstrates how to get the exterior ring of an empty polygon. The result is an empty LineString.

```python
from pyspark.databricks.sql import functions as dbf  
df = spark.createDataFrame([('POLYGON EMPTY', 'POLYGON((0 0,10 0,0 10,0 0))', 'POLYGON ZM ((0 0 111 -11,10 0 222 -22,0 10 333 -33,0 0 444 -44),(1 1 555 -55,4 1 666 -66,1 4 777 -77,1 1 888 -88))')], ['pgn1', 'pgn2', 'pgn3'])  
df.select(dbf.st_asewkt(dbf.st_exteriorring(dbf.st_geomfromtext('pgn1'))).alias('result')).collect()
```

--------------------------------

### Install R Package from GitHub with `renv`

Source: https://docs.databricks.com/aws/en/sparkr/renv

Install an R package directly from a GitHub repository, for example, `eddelbuettel/digest`, using `renv::install`.

```r
renv::install("eddelbuettel/digest")
```

--------------------------------

### Correct Kafka Client Options (With Prefix)

Source: https://docs.databricks.com/aws/en/connect/streaming/kafka/faq

This example demonstrates the correct way to set Kafka client configuration options by including the required `kafka.` prefix.

```Python
.option("kafka.security.protocol", "SASL_SSL") 
.option("kafka.sasl.mechanism", "PLAIN")  

```

--------------------------------

### Create and Drop Procedure Example

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-drop-procedure

Demonstrates how to create a simple SQL procedure and then drop it. It also shows the behavior when attempting to drop a non-existent procedure and how to use IF EXISTS.

```sql
-- Create a procedure `hello`  
> CREATE PROCEDURE hello() SQL SECURITY INVOKER LANGUAGE SQL  
  AS BEGIN  
    SELECT 'hello!';  
  END;  
  
-- Drop the procedure  
> DROP PROCEDURE hello;  
  
-- Try to drop a procedure which is not present  
> DROP PROCEDURE hello;  
Error: ROUTINE_NOT_FOUND  
  
-- Drop a procedure only if it exists  
> DROP PROCEDURE IF EXISTS hello;  
```

--------------------------------

### Example 3: Column Inputs for Start and Length

Source: https://docs.databricks.com/aws/en/pyspark/reference/functions/slice

Shows how to use column inputs for both the start index and length parameters in the slice function.

```APIDOC
## slice(x, start, length)

### Description
Slice function with column inputs for start and length.

### Request Example
```python
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3], 2, 2), ([4, 5], 1, 3)], ['x', 'start', 'length'])
df.select(sf.slice(df.x, df.start, df.length)).show()
```

### Response Example
```
+-----------------------+
|slice(x, start, length)|
+-----------------------+
|                 [2, 3]|
|                 [4, 5]|
+-----------------------+
```
```

--------------------------------

### Start Airflow Webserver

Source: https://docs.databricks.com/aws/en/jobs/how-to/use-airflow-with-jobs

Starts the Airflow web server. Ensure you are in the Airflow installation directory. This command is used to access the Airflow UI.

```bash
pipenv shell
export AIRFLOW_HOME=$(pwd)
airflow webserver
```

--------------------------------

### Example: Configure Postgres Project, Branch, and Endpoint

Source: https://docs.databricks.com/aws/en/dev-tools/bundles/resources

This example demonstrates how to configure a Postgres project, its main branch, and a primary read-write endpoint with autoscaling settings.

```yaml
resources:  
  postgres_projects:  
    my_db:  
      project_id: test-prod-app  
      display_name: 'Production Database'  
      pg_version: 17  
  
  postgres_branches:  
    main:  
      parent: ${resources.postgres_projects.my_db.id}  
      branch_id: main  
      is_protected: false  
      no_expiry: true  
  
  postgres_endpoints:  
    primary:  
      parent: ${resources.postgres_branches.main.id}  
      endpoint_id: primary  
      endpoint_type: ENDPOINT_TYPE_READ_WRITE  
      autoscaling_limit_min_cu: 0.5  
      autoscaling_limit_max_cu: 4  

```

--------------------------------

### Example SQL GET Statement

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-aux-connector-get

Demonstrates how to use the GET statement to download a CSV file from a specified volume path to a local directory.

```sql
> GET '/Volumes/test_catalog/test_schema/test_volume/2023/06/file1.csv' TO '/home/bob/data.csv'  
```

--------------------------------

### Date Partitioned Files Example

Source: https://docs.databricks.com/aws/en/ingestion/cloud-object-storage/auto-loader/directory-listing-mode

Shows examples of files uploaded in a date-partitioned format, including standard and year/month/day partitioning.

```text
// <base-path>/yyyy/MM/dd/HH:mm:ss-randomString
<base-path>/2021/12/01/10:11:23-b1662ecd-e05e-4bb7-a125-ad81f6e859b4.json  
<base-path>/2021/12/01/10:11:23-b9794cf3-3f60-4b8d-ae11-8ea320fad9d1.json  
...  
  
// <base-path>/year=yyyy/month=MM/day=dd/hour=HH/minute=mm/randomString
<base-path>/year=2021/month=12/day=04/hour=08/minute=22/442463e5-f6fe-458a-8f69-a06aa970fc69.csv  
<base-path>/year=2021/month=12/day=04/hour=08/minute=22/8f00988b-46be-4112-808d-6a35aead0d44.csv <- this may be uploaded before the file above as long as processing happens less frequently than a minute  
```

--------------------------------

### Get Fulfillment Metadata

Source: https://docs.databricks.com/aws/en/dev-tools/cli/reference/consumer-fulfillments-commands

Retrieves high-level preview metadata for a listing's installable content. Specify the listing ID to get the information.

```bash
databricks consumer-fulfillments get 12345  
```

--------------------------------

### Set up SSH Connection

Source: https://docs.databricks.com/aws/en/dev-tools/remote-development

Use this command to initiate the SSH connection setup. Replace `<connection-name>` with your desired name.

```bash
databricks ssh setup --name <connection-name>  

```

--------------------------------

### Example Job Get Response

Source: https://docs.databricks.com/aws/en/reference/jobs-2.0-api

This is an example JSON response when retrieving information about a job. It includes job ID, settings, and creation time.

```json
{
  "job_id": 1,
  "settings": {
    "name": "Nightly model training",
    "new_cluster": {
      "spark_version": "7.3.x-scala2.12",
      "node_type_id": "r3.xlarge",
      "aws_attributes": {
        "availability": "ON_DEMAND"
      },
      "num_workers": 10
    },
    "libraries": [
      {
        "jar": "dbfs:/my-jar.jar"
      },
      {
        "maven": {
          "coordinates": "org.jsoup:jsoup:1.7.2"
        }
      }
    ],
    "email_notifications": {
      "on_start": [],
      "on_success": [],
      "on_failure": []
    },
    "webhook_notifications": {
      "on_start": [
        {
          "id": "bf2fbd0a-4a05-4300-98a5-303fc8132233"
        }
      ],
      "on_success": [
        {
          "id": "bf2fbd0a-4a05-4300-98a5-303fc8132233"
        }
      ],
      "on_failure": []
    },
    "notification_settings": {
      "no_alert_for_skipped_runs": false,
      "no_alert_for_canceled_runs": false,
      "alert_on_last_attempt": false
    },
    "timeout_seconds": 100000000,
    "max_retries": 1,
    "schedule": {
      "quartz_cron_expression": "0 15 22 * * ?",
      "timezone_id": "America/Los_Angeles",
      "pause_status": "UNPAUSED"
    },
    "spark_jar_task": {
      "main_class_name": "com.databricks.ComputeModels"
    }
  },
  "created_time": 1457570074236
}

```

--------------------------------

### List All Configuration Parameters with Value and Description

Source: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-aux-conf-mgmt-set

Execute this command to display all configuration parameters, including their current values and a description of their meaning. This is useful for understanding available settings.

```sql
SET -v;
```

--------------------------------

### Log in with a Specific Profile and Configure Cluster Settings

Source: https://docs.databricks.com/aws/en/dev-tools/cli/reference/auth-commands

This example shows how to log in using a pre-defined profile and simultaneously configure cluster settings.

```bash
databricks auth login --profile my-profile --configure-cluster
```