### Example: Install a PyPI Library Source: https://docs.databricks.com/api/gcp/workspace/libraries/install This example shows how to install a Python package from PyPI onto a cluster. Ensure the cluster ID is correct. ```json { "cluster_id": "", "libraries": [ { "pypi": { "package": "simplejson" } } ] } ``` -------------------------------- ### Example: List installed Databricks Labs applications Source: https://docs.databricks.com/aws/en/dev-tools/cli/reference/labs-commands Demonstrates how to list all installed Databricks Labs applications using the Databricks CLI. ```bash databricks labs installed ``` -------------------------------- ### Example JAR Library Installation Source: https://docs.databricks.com/gcp/en/reference/jobs-2.0-api Specifies a JAR library to be installed using its URI. Supports DBFS and GCS URIs. ```json { "jar": "dbfs:/mnt/databricks/library.jar"} ``` ```json { "jar": "gs://my-bucket/library.jar" } ``` -------------------------------- ### Get H3 Cell Index from Coordinates (Yet Another Example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples A further example of `h3_geo_to_h3` to demonstrate its application with various geographic points and resolutions. ```sql SELECT h3_geo_to_h3(41.8781, -87.6298, 5) AS h3_index; ``` -------------------------------- ### Get H3 Cell Index from Coordinates (Final Example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples The last example of `h3_geo_to_h3` for converting coordinates to an H3 index, reinforcing the understanding of its usage. ```sql SELECT h3_geo_to_h3(29.7604, -95.3698, 6) AS h3_index; ``` -------------------------------- ### Get H3 Cell Index from Coordinates (Another Example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Another example of using `h3_geo_to_h3` to convert coordinates to an H3 index, useful for verifying the function's behavior with different inputs. ```sql SELECT h3_geo_to_h3(40.7128, -74.0060, 4) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (yet another coordinate example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts a third set of latitude and longitude coordinates to an H3 index. ```sql SELECT h3_polyfill(41.8781, -87.6298, 5) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (another coordinate example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts another set of latitude and longitude coordinates to an H3 index. ```sql SELECT h3_polyfill(40.7128, -74.0060, 5) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index. ```sql SELECT h3_polyfill(34.0522, -118.2437, 5) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different resolution) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts latitude and longitude to an H3 index at a different resolution. ```sql SELECT h3_polyfill(37.7749, -122.4194, 6) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with another resolution) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts latitude and longitude to an H3 index at another resolution. ```sql SELECT h3_polyfill(37.7749, -122.4194, 8) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with yet another resolution) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts latitude and longitude to an H3 index at yet another resolution. ```sql SELECT h3_polyfill(37.7749, -122.4194, 10) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (another coordinate and resolution example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts another set of latitude and longitude coordinates to an H3 index at a specific resolution. ```sql SELECT h3_polyfill(40.7128, -74.0060, 7) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at a specific resolution. ```sql SELECT h3_polyfill(34.0522, -118.2437, 6) AS h3_index; ``` -------------------------------- ### Full Postgres Project Configuration Example Source: https://docs.databricks.com/aws/en/dev-tools/bundles/resources An example demonstrating the setup of a Postgres project, its main branch, and a read-write compute endpoint. This illustrates how to link these resources together using parent references. ```yaml resources: postgres_projects: my_db: project_id: test-prod-app display_name: 'Production Database' pg_version: 17 postgres_branches: main: parent: ${resources.postgres_projects.my_db.id} branch_id: main is_protected: false no_expiry: true postgres_endpoints: primary: parent: ${resources.postgres_branches.main.id} endpoint_id: primary endpoint_type: ENDPOINT_TYPE_READ_WRITE autoscaling_limit_min_cu: 0.5 autoscaling_limit_max_cu: 4 ``` -------------------------------- ### Run Quickstart Scripts Source: https://docs.databricks.com/aws/en/generative-ai/agent-framework/author-agent Execute quickstart scripts to install dependencies, set up the environment, and start the agent application locally. ```bash uv run quickstart ``` ```bash uv run start-app ``` -------------------------------- ### Get H3 Index from Lat/Lng (yet another coordinate and resolution example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts a third set of latitude and longitude coordinates to an H3 index at a specific resolution. ```sql SELECT h3_polyfill(41.8781, -87.6298, 8) AS h3_index; ``` -------------------------------- ### H3 K-Ring Variation Example Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples This example demonstrates how to configure a hexagon layer with H3 K-ring data, including visualization settings for color, radius, and coverage. ```json { "id": "3bz7bh", "type": "hexagonId", "config": { "dataId": "hex6_flight", "label": "hex_id", "color": [ 255, 251, 152 ], "columns": { "hex_id": "hex_id" }, "isVisible": True, "visConfig": { "opacity": 0.8, "colorRange": { "name": "Global Warming", "type": "sequential", "category": "Uber", "colors": [ "#5A1846", "#900C3F", "#C70039", "#E3611C", "#F1920E", "#FFC300" ] }, "coverage": 1, "enable3d": False, "sizeRange": [ 0, 500 ], "coverageRange": [ 0, 1 ], "elevationScale": 5 }, "hidden": False, "textLabel": [ { "field": null, "color": [ 255, 255, 255 ], "size": 18, "offset": [ 0, 0 ], "anchor": "start", "alignment": "center" } ] }, "visualChannels": { "colorField": null, "colorScale": "quantile", "sizeField": null, "sizeScale": "linear", "coverageField": null, "coverageScale": "linear" } } ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with highest resolution) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts latitude and longitude to an H3 index at the highest resolution. ```sql SELECT h3_polyfill(37.7749, -122.4194, 14) AS h3_index; ``` -------------------------------- ### SQL EXPLAIN Example Source: https://docs.databricks.com/aws/en/optimizations/aqe Demonstrates how to use the SQL EXPLAIN command to view the initial query plan. Note that this does not show AQE's dynamically generated plan as the query is not executed. ```sql EXPLAIN SELECT * FROM my_table; ``` -------------------------------- ### Get the first point of a linestring with ZM coordinates Source: https://docs.databricks.com/aws/en/sql/language-manual/functions/st_startpoint This example demonstrates how to get the starting point of a linestring that has ZM (Z and M) coordinates. The output preserves the ZM information. ```sql -- Returns first point with ZM coordinates. > SELECT st_asewkt(st_startpoint(st_geogfromtext('LINESTRING ZM (1 2 3 4,5 6 7 8)'))); SRID=4326;POINT ZM (1 2 3 4) ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 11) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 11. ```sql SELECT h3_polyfill(40.7128, -74.0060, 11) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 7) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 7. ```sql SELECT h3_polyfill(41.8781, -87.6298, 7) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 11) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 11. ```sql SELECT h3_polyfill(34.0522, -118.2437, 11) AS h3_index; ``` -------------------------------- ### Install from a listing Source: https://docs.databricks.com/api/azure/workspace/consumerinstallations Creates a new consumer installation for a specified Marketplace listing. ```APIDOC ## Install from a listing ### Description Creates a new consumer installation for a specified Marketplace listing. ### Method POST ### Endpoint /api/marketplace/consumer/installations ### Parameters #### Request Body - **listing_id** (string) - Required - The ID of the listing to install. - **workspace_id** (string) - Required - The ID of the Databricks workspace where the listing will be installed. ### Request Example { "listing_id": "listing-abcde", "workspace_id": "workspace-123" } ### Response #### Success Response (201) - **installation_id** (string) - The unique identifier for the newly created installation. - **listing_id** (string) - The ID of the listing. - **workspace_id** (string) - The ID of the workspace. - **status** (string) - The initial status of the installation (e.g., 'PENDING'). #### Response Example { "installation_id": "install-abcde", "listing_id": "listing-abcde", "workspace_id": "workspace-123", "status": "PENDING" } ``` -------------------------------- ### Get Recent StreamingQuery Progress Updates Source: https://docs.databricks.com/aws/en/pyspark/reference/classes/streamingquery/recentProgress This example demonstrates how to start a streaming query and access its recent progress updates using the `recentProgress` attribute. It initializes a streaming DataFrame, writes it to a memory table, starts the query, accesses its progress, and then stops the query. ```Python sdf = spark.readStream.format("rate").load() sq = sdf.writeStream.format('memory').queryName('this_query').start() sq.recentProgress # [...] sq.stop() ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 9) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 9. ```sql SELECT h3_polyfill(41.8781, -87.6298, 9) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 13) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 13. ```sql SELECT h3_polyfill(34.0522, -118.2437, 13) AS h3_index; ``` -------------------------------- ### Get Input File Block Start Offset in PySpark Source: https://docs.databricks.com/aws/en/pyspark/reference/functions/input_file_block_start This example demonstrates how to use the input_file_block_start function on a DataFrame read from a text file. It selects the function's output and displays it. ```Python from pyspark.sql import functions as sf df = spark.read.text("python/test_support/sql/ages_newlines.csv", lineSep=",") df.select(sf.input_file_block_start()).show() ``` -------------------------------- ### Create an Example for a Knowledge Assistant Source: https://docs.databricks.com/api/workspace/knowledgeassistants/createexample Use this API to create a new example for a Knowledge Assistant. Provide the parent resource ID and the example details, including the question and optional guidelines. ```bash curl -X POST \ api/2.1/{parent=knowledge-assistants/*}/examples \ -d '{ "guidelines": [ "string" ], "question": "string" }' ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 9) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 9. ```sql SELECT h3_polyfill(40.7128, -74.0060, 9) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 13) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 13. ```sql SELECT h3_polyfill(37.7749, -122.4194, 13) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and highest resolution) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at the highest resolution. ```sql SELECT h3_polyfill(34.0522, -118.2437, 15) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 11) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 11. ```sql SELECT h3_polyfill(41.8781, -87.6298, 11) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 7) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 7. ```sql SELECT h3_polyfill(34.0522, -118.2437, 7) AS h3_index; ``` -------------------------------- ### Get Last Streaming Query Progress Source: https://docs.databricks.com/aws/en/pyspark/reference/classes/streamingquery/lastProgress This example demonstrates how to start a streaming query and then access its most recent progress update using the `lastProgress` attribute. It's important to stop the query after retrieving the progress. ```python sdf = spark.readStream.format("rate").load() sq = sdf.writeStream.format('memory').queryName('this_query').start() sq.lastProgress sq.stop() ``` -------------------------------- ### Create a Standard Catalog Example (SQL) Source: https://docs.databricks.com/aws/en/catalogs/create-catalog Example of creating a catalog named 'example' using the `CREATE CATALOG` SQL command. ```sql CREATE CATALOG IF NOT EXISTS example; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 13) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 13. ```sql SELECT h3_polyfill(40.7128, -74.0060, 13) AS h3_index; ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 9) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 9. ```sql SELECT h3_polyfill(34.0522, -118.2437, 9) AS h3_index; ``` -------------------------------- ### Delta Live Tables Quickstart (SQL) Source: https://docs.databricks.com/aws/en/ldp/hive-metastore This notebook provides an example Delta Live Tables pipeline in SQL. It demonstrates reading raw JSON, cleaning data with queries and expectations, and performing analysis. ```sql # Delta Live Tables quickstart (SQL) A notebook that provides an example Delta Live Tables pipeline to: * Read raw JSON clickstream data into a table. * Read records from the raw data table and use a Delta Live Tables query and expectations to create a new table with cleaned and prepared data. * Perform an analysis on the prepared data with a Delta Live Tables query. ``` -------------------------------- ### Get Library JAR Allowlist Example Source: https://docs.databricks.com/gcp/en/dev-tools/cli/reference/artifact-allowlists-commands Example of how to get the artifact allowlist specifically for library JAR files. ```bash databricks artifact-allowlists get LIBRARY_JAR ``` -------------------------------- ### Get H3 Index from Lat/Lng (example with different coordinates and resolution 13) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts different latitude and longitude coordinates to an H3 index at resolution 13. ```sql SELECT h3_polyfill(41.8781, -87.6298, 13) AS h3_index; ``` -------------------------------- ### Databricks CLI consumer-providers get example Source: https://docs.databricks.com/aws/en/dev-tools/cli/reference/consumer-providers-commands An example of how to execute the `get` command to retrieve a provider using its ID. ```bash databricks consumer-providers get ``` -------------------------------- ### Get Build Logs Source: https://docs.databricks.com/api/gcp/workspace/servingendpoints/buildlogs Retrieves the build logs for a specific serving endpoint. This includes information about dependencies, environment setup, and installation progress. It also details possible error codes that might be encountered during the build process. ```APIDOC ## GET /api/gcp/workspace/servingendpoints/{endpoint_id}/buildlogs ### Description Retrieves the build logs for a specific serving endpoint. ### Method GET ### Endpoint /api/gcp/workspace/servingendpoints/{endpoint_id}/buildlogs ### Parameters #### Path Parameters - **endpoint_id** (string) - Required - The unique identifier of the serving endpoint. ### Response #### Success Response (200) - **logs** (string) - The build logs content. #### Response Example ```json { "logs": "- conda-forge\ndependencies:\n- python=3.9.5\n- pip<=21.2.4\n- pip:\n- mlflow\n- cloudpickle==2.2.0\nname: mlflow-env\nCollecting package metadata (repodata.json): ...working... done\nSolving environment: ...working... done\nPreparing transaction: ...working... done\nVerifying transaction: ...working... done\nExecuting transaction: ...working... done\nInstalling pip dependencies: ...working... done\n" } ``` ### Error Handling #### Possible error codes: - **401 UNAUTHENTICATED**: Request does not have valid authentication credentials for the operation. - **404 NOT_FOUND**: Operation was performed on a resource that does not exist. - **500 INTERNAL_ERROR**: Internal error. ``` -------------------------------- ### Example: Install Libraries from a Requirements File Source: https://docs.databricks.com/api/gcp/workspace/libraries/install Install Python libraries listed in a requirements.txt file located at a Workspace path. Ensure the file exists and is accessible. ```json { "cluster_id": "", "libraries": [ { "requirements": "/Workspace/path/to/requirements.txt" } ] } ``` -------------------------------- ### Get H3 Index from Lat/Lng (another coordinate and highest resolution example) Source: https://docs.databricks.com/gcp/en/sql/language-manual/sql-ref-h3-geospatial-functions-examples Converts another set of latitude and longitude coordinates to an H3 index at the highest resolution. ```sql SELECT h3_polyfill(40.7128, -74.0060, 15) AS h3_index; ```