### Run Static Token Example Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/examples/tokenFederation/README.md Execute the static token example using environment variables for configuration. This method uses a static access token that remains constant throughout the application's lifecycle. ```bash DATABRICKS_HOST= DATABRICKS_HTTP_PATH= DATABRICKS_TOKEN= npx ts-node staticToken.ts ``` -------------------------------- ### Run M2M Federation Example Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/examples/tokenFederation/README.md Execute the machine-to-machine token federation example with a service principal. This requires a `federationClientId` to identify the service principal to Databricks and uses environment variables for configuration. ```bash DATABRICKS_HOST= DATABRICKS_HTTP_PATH= DATABRICKS_CLIENT_ID= SERVICE_ACCOUNT_TOKEN= npx ts-node m2mFederation.ts ``` -------------------------------- ### Run Token Federation Example Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/examples/tokenFederation/README.md Execute the token federation example using environment variables. This enables automatic exchange of tokens from external identity providers (like Azure AD, Google, Okta) for Databricks-compatible tokens via RFC 8693 token exchange. ```bash DATABRICKS_HOST= DATABRICKS_HTTP_PATH= AZURE_AD_TOKEN= npx ts-node federation.ts ``` -------------------------------- ### Run Custom Token Provider Example Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/examples/tokenFederation/README.md Execute the custom token provider example using environment variables. This involves implementing the `ITokenProvider` interface for complete control over token management, including custom caching, refresh logic, retries, and error handling. ```bash DATABRICKS_HOST= DATABRICKS_HTTP_PATH= OAUTH_SERVER_URL= OAUTH_CLIENT_ID= OAUTH_CLIENT_SECRET= npx ts-node customTokenProvider.ts ``` -------------------------------- ### Install Databricks SQL Driver for Node.js Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Install the `@databricks/sql` package using npm. ```bash npm install @databricks/sql ``` -------------------------------- ### Run External Token Example Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/examples/tokenFederation/README.md Execute the external token example with environment variables. This approach uses a callback function to supply tokens dynamically, suitable for integration with secret managers or vaults. The driver automatically caches tokens. ```bash DATABRICKS_HOST= DATABRICKS_HTTP_PATH= DATABRICKS_TOKEN= npx ts-node externalToken.ts ``` -------------------------------- ### Install Databricks SQL Driver Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/README.md Install the Databricks SQL driver for Node.js using npm. Ensure Node.js 14 or newer is installed. ```bash npm i @databricks/sql ``` -------------------------------- ### Commit Sign-off Example Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md A sample commit message line indicating that the contributor has signed the Developer Certificate of Origin. Use your real name. ```text Signed-off-by: Joe Smith ``` -------------------------------- ### Run End-to-End Tests Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Execute the end-to-end tests for the project. These tests validate the application's behavior from start to finish. ```bash npm run e2e ``` -------------------------------- ### Create DBSQLClient Instance Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Instantiate the `DBSQLClient` to establish a connection. You can use the default client which logs to the console, or provide a custom logger for more control over logging, including writing to a file. ```javascript const { DBSQLClient, DBSQLLogger, LogLevel } = require('@databricks/sql'); // Default client (logs to console at INFO level) const client = new DBSQLClient(); // Client with a custom logger that also writes to a file const logger = new DBSQLLogger({ level: LogLevel.debug, filepath: 'app.log' }); const clientWithLogger = new DBSQLClient({ logger }); // Listen for connection-level events client.on('error', (err) => console.error('Connection error:', err.message)); client.on('close', () => console.log('Connection closed')); ``` -------------------------------- ### Run Unit Tests Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Execute the unit tests for the project. This helps ensure individual components function correctly. ```bash npm test ``` -------------------------------- ### Query Server Information Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Use `session.getInfo(infoType)` with `TGetInfoType` constants to query general server information like DBMS version and server name. Remember to close the session and client when done. ```javascript const { DBSQLClient, thrift } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); const dbmsVersion = await session.getInfo( thrift.TCLIService_types.TGetInfoType.CLI_DBMS_VER ); console.log('DBMS Version:', dbmsVersion.getValue()); const serverName = await session.getInfo( thrift.TCLIService_types.TGetInfoType.CLI_DBMS_NAME ); console.log('Server Name:', serverName.getValue()); await session.close(); await client.close(); ``` -------------------------------- ### Handle Staging Operations with `session.executeStatement` Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Upload, download, and remove files using staging operations with presigned URLs. Ensure `stagingAllowedLocalPath` is configured to a valid directory for security. ```javascript const { DBSQLClient } = require('@databricks/sql'); const path = require('path'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); const localDir = path.join(__dirname, 'uploads'); // Upload a file to DBFS/Volumes via a staging PUT const putOp = await session.executeStatement( "PUT '/local/data.csv' INTO 'dbfs:/tmp/data.csv' OVERWRITE", { stagingAllowedLocalPath: localDir } ); await putOp.close(); // Download a file from DBFS/Volumes via a staging GET const getOp = await session.executeStatement( "GET 'dbfs:/tmp/data.csv' TO '/local/data_copy.csv'", { stagingAllowedLocalPath: localDir } ); await getOp.close(); // Remove a file const removeOp = await session.executeStatement( "REMOVE 'dbfs:/tmp/data.csv'", { stagingAllowedLocalPath: localDir } ); await removeOp.close(); await session.close(); await client.close(); ``` -------------------------------- ### Configure Local End-to-End Tests Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/README.md Set up Databricks SQL connection information for end-to-end tests by creating a `tests/e2e/utils/config.local.js` file. ```javascript { host: '***.databricks.com', path: '/sql/2.0/warehouses/***', token: 'dapi***', database: ['catalog', 'database'], } ``` -------------------------------- ### Build the Project Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Compile the TypeScript source code into distributable JavaScript files. This command should be run before committing or releasing. ```bash npm run build ``` -------------------------------- ### Configure Built-in Winston-based Logger with `DBSQLLogger` Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Configure log level and optional file output using the built-in `DBSQLLogger`. This logger implements `IDBSQLLogger` and can be replaced with a custom implementation. ```javascript const { DBSQLClient, DBSQLLogger, LogLevel } = require('@databricks/sql'); // Logger writing INFO+ to console and file const logger = new DBSQLLogger({ level: LogLevel.info, filepath: 'databricks-sql.log', }); const client = new DBSQLClient({ logger }); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); const op = await session.executeStatement('SELECT "Hello, World!"'); console.table(await op.fetchAll()); await op.close(); // Dynamically increase verbosity at runtime logger.setLevel(LogLevel.debug); const op2 = await session.executeStatement('SELECT current_timestamp()'); console.table(await op2.fetchAll()); await op2.close(); await session.close(); await client.close(); ``` -------------------------------- ### Run Unit Tests Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/README.md Execute all unit tests or specify a particular test file to run using npm test. ```bash npm test npm test -- ``` -------------------------------- ### client.connect() - Default Authentication Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Establishes an HTTPS Thrift connection to a Databricks endpoint using the default 'access-token' authentication. Requires host and path, and optionally accepts token, socket timeout, and proxy configurations. ```APIDOC ## client.connect(options) ### Description Opens an HTTPS Thrift connection. `host`, `path`, and an authentication option are required. Returns `Promise` (the same client instance, enabling chaining). The default `authType` is `'access-token'`; other values are `'databricks-oauth'`, `'token-provider'`, `'external-token'`, `'static-token'`, and `'custom'`. ### Parameters - **options** (object) - Required. Configuration object for the connection. - **host** (string) - Required. The Databricks SQL endpoint host. - **path** (string) - Required. The path to the SQL warehouse. - **token** (string) - Optional. The personal access token for authentication (used when `authType` is `'access-token'`). - **authType** (string) - Optional. The authentication type. Defaults to `'access-token'`. - **socketTimeout** (number) - Optional. Custom socket timeout in milliseconds. Defaults to 15 minutes. - **proxy** (object) - Optional. HTTP proxy configuration. - **protocol** (string) - e.g., 'https' - **host** (string) - Proxy host address. - **port** (number) - Proxy port. - **userAgentEntry** (string) - Optional. A string to be appended to the User-Agent header for request tagging. ### Returns - `Promise` - A promise that resolves with the client instance upon successful connection. ### Request Example ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); // Personal access token (default authType) await client.connect({ host: 'adb-1234567890.1.azuredatabricks.net', path: '/sql/2.0/warehouses/abc123def456', token: process.env.DATABRICKS_TOKEN, // Optional: custom socket timeout (ms), default 15 minutes socketTimeout: 10 * 60 * 1000, // Optional: HTTP proxy proxy: { protocol: 'https', host: 'proxy.corp.net', port: 8080 }, // Optional: tag requests from this client in the User-Agent header userAgentEntry: 'MyApp/1.0', }); console.log('Connected'); ``` ``` -------------------------------- ### Open Databricks SQL Session Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Open a new session with optional configurations for initial catalog, schema, Spark properties, and query tags. Use `client.openSession()` for a minimal session. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); // Minimal session const session = await client.openSession(); // Session with initial catalog/schema, Spark config, and query tags const taggedSession = await client.openSession({ initialCatalog: 'my_catalog', initialSchema: 'my_database', configuration: { ansi_mode: 'false', 'spark.sql.shuffle.partitions': '200', }, queryTags: { team: 'data-engineering', env: 'production', service: 'etl-pipeline', }, }); console.log('Session ID:', taggedSession.id); await taggedSession.close(); await client.close(); ``` -------------------------------- ### Connect and Query Databricks SQL Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/README.md Connect to Databricks SQL using the DBSQLClient, open a session, execute a query, fetch results, and close the client. Requires connection details like host, path, and token. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); client .connect({ host: '********.databricks.com', path: '/sql/2.0/warehouses/****************', token: 'dapi********************************', }) .then(async (client) => { const session = await client.openSession(); const queryOperation = await session.executeStatement('SELECT "Hello, World!"'); const result = await queryOperation.fetchAll(); await queryOperation.close(); console.table(result); await session.close(); await client.close(); }) .catch((error) => { console.log(error); }); ``` -------------------------------- ### Execute SQL Statements with Databricks SQL Node.js Driver Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Demonstrates executing simple and parameterized SQL statements using `session.executeStatement`. Supports named and ordinal parameters, query timeouts, and query tags. ```javascript const { DBSQLClient, DBSQLParameter, DBSQLParameterType } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); // Simple query const op1 = await session.executeStatement('SELECT "Hello, World!" AS greeting'); console.log(await op1.fetchAll()); // [{ greeting: 'Hello, World!' }] await op1.close(); // Named parameterized query const op2 = await session.executeStatement( 'SELECT * FROM orders WHERE status = :status AND total > :minAmount', { namedParameters: { status: 'shipped', minAmount: new DBSQLParameter({ type: DBSQLParameterType.DOUBLE, value: 99.99 }), }, queryTimeout: 30, // seconds (for compute clusters) maxRows: 5000, queryTags: { request_id: 'req-abc-123', feature: 'order-report' }, } ); const orders = await op2.fetchAll(); console.log(`Fetched ${orders.length} orders`); await op2.close(); // Ordinal parameterized query const op3 = await session.executeStatement( 'SELECT * FROM events WHERE event_date = ? AND user_id = ?', { ordinalParameters: [new Date('2024-01-15'), 42], } ); console.log(await op3.fetchAll()); await op3.close(); await session.close(); await client.close(); ``` -------------------------------- ### Connect to Databricks with Databricks OAuth (U2M) Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Connect using Databricks OAuth in a User-to-Machine (U2M) flow, which typically involves opening a browser for user consent. Ensure the `host` and `path` are correctly specified. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); // U2M (user-to-machine): browser opens for user consent await client.connect({ host: 'adb-1234567890.1.azuredatabricks.net', path: '/sql/2.0/warehouses/abc123def456', authType: 'databricks-oauth', }); ``` -------------------------------- ### DBSQLClient Constructor Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Instantiate a new DBSQLClient. The client can be configured with a custom logger. It also acts as an EventEmitter for connection-level events. ```APIDOC ## new DBSQLClient(options?) ### Description The entry point for all driver usage. Accepts an optional `ClientOptions` object with a custom `logger`. The client is an `EventEmitter` and emits `error`, `reconnecting`, `close`, and `timeout` events on the underlying connection. ### Parameters - **options** (ClientOptions) - Optional. An object that can contain a custom `logger`. ### Events - **error**: Emitted when a connection error occurs. - **reconnecting**: Emitted when the client attempts to reconnect. - **close**: Emitted when the connection is closed. - **timeout**: Emitted when a connection times out. ### Request Example ```javascript const { DBSQLClient, DBSQLLogger, LogLevel } = require('@databricks/sql'); // Default client (logs to console at INFO level) const client = new DBSQLClient(); // Client with a custom logger that also writes to a file const logger = new DBSQLLogger({ level: LogLevel.debug, filepath: 'app.log' }); const clientWithLogger = new DBSQLClient({ logger }); // Listen for connection-level events client.on('error', (err) => console.error('Connection error:', err.message)); client.on('close', () => console.log('Connection closed')); ``` ``` -------------------------------- ### Run End-to-End Tests Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/README.md Execute end-to-end tests using the npm run e2e command. Optionally specify a particular test file. ```bash npm run e2e npm run e2e -- ``` -------------------------------- ### client.openSession(request?) — Open a session Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Opens a new Databricks SQL session, optionally allowing configuration of the initial catalog, schema, Spark properties, and session-level query tags. ```APIDOC ## `client.openSession(request?)` — Open a session Returns `Promise`. The optional `OpenSessionRequest` allows setting the initial catalog, schema, Spark configuration properties, and session-level query tags (key-value pairs serialized into `QUERY_TAGS`). ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); // Minimal session const session = await client.openSession(); // Session with initial catalog/schema, Spark config, and query tags const taggedSession = await client.openSession({ initialCatalog: 'my_catalog', initialSchema: 'my_database', configuration: { ansi_mode: 'false', 'spark.sql.shuffle.partitions': '200', }, queryTags: { team: 'data-engineering', env: 'production', service: 'etl-pipeline', }, }); console.log('Session ID:', taggedSession.id); await taggedSession.close(); await client.close(); ``` ``` -------------------------------- ### `session.getInfo(infoType)` Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Queries server information, such as DBMS version and server name, using `TGetInfoType` constants. ```APIDOC ## session.getInfo(infoType) ### Description Returns general information about the data source (DBMS version, server name, etc.) using `TGetInfoType` constants from `thrift.TCLIService_types`. ### Method ```javascript await session.getInfo(infoType); ``` ### Parameters * `infoType` (thrift.TCLIService_types.TGetInfoType) - The type of information to retrieve. ### Response * `TGetInfo` - An object containing the requested information. Use `.getValue()` to extract the actual value. ### Request Example ```javascript const dbmsVersion = await session.getInfo( thrift.TCLIService_types.TGetInfoType.CLI_DBMS_VER ); console.log('DBMS Version:', dbmsVersion.getValue()); const serverName = await session.getInfo( thrift.TCLIService_types.TGetInfoType.CLI_DBMS_NAME ); console.log('Server Name:', serverName.getValue()); ``` ``` -------------------------------- ### Advanced Metadata: Functions, Primary Keys, Cross-References Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Use `session.getFunctions()`, `session.getPrimaryKeys()`, and `session.getCrossReference()` for advanced metadata retrieval. These methods allow filtering by patterns or specifying table relationships. Remember to close operations and sessions. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); async function fetchAll(op) { const rows = await op.fetchAll(); await op.close(); return rows; } // Find built-in functions matching a pattern const funcs = await fetchAll(await session.getFunctions({ functionName: 'to_date%' })); console.log('Functions:', funcs.map(r => r.FUNCTION_NAME)); // Get primary keys of a table const pks = await fetchAll(await session.getPrimaryKeys({ schemaName: 'default', tableName: 'orders', })); console.log('Primary keys:', pks.map(r => r.COLUMN_NAME)); // Get foreign key relationships between two tables const fks = await fetchAll(await session.getCrossReference({ parentCatalogName: 'main', parentSchemaName: 'default', parentTableName: 'customers', foreignCatalogName: 'main', foreignSchemaName: 'default', foreignTableName: 'orders', })); console.log('Foreign keys:', fks); await session.close(); await client.close(); ``` -------------------------------- ### Format Code with Prettier Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Apply code formatting rules defined by Prettier to ensure consistent code style across the project. ```bash npm run prettier ``` -------------------------------- ### Fetch Data in Chunks with `fetchChunk` Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Use `fetchChunk` for manual pagination, retrieving up to `maxRows` per call. Combine with `hasMoreRows()` to control the fetching loop. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); const op = await session.executeStatement('SELECT * FROM events WHERE dt = "2024-01-15"'); let page = 0; while (await op.hasMoreRows()) { const chunk = await op.fetchChunk({ maxRows: 10000 }); console.log(`Page ${++page}: ${chunk.length} rows`); // process chunk... } await op.close(); await session.close(); await client.close(); ``` -------------------------------- ### Connect with Custom ITokenProvider Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Implement the `ITokenProvider` interface for full control over token caching, refresh, retry, and error handling. Pass an instance of your provider to `tokenProvider`. ```typescript import { DBSQLClient } from '@databricks/sql'; import { ITokenProvider, Token } from '@databricks/sql/lib/connection/auth/tokenProvider'; class MyOAuthProvider implements ITokenProvider { async getToken(): Promise { const resp = await fetch('https://auth.example.com/oauth/token', { method: 'POST', headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, body: new URLSearchParams({ grant_type: 'client_credentials', client_id: process.env.CLIENT_ID!, client_secret: process.env.CLIENT_SECRET!, }), }); const data = (await resp.json()) as { access_token: string; expires_in: number }; return new Token(data.access_token, { expiresAt: new Date(Date.now() + data.expires_in * 1000), }); } getName() { return 'MyOAuthProvider'; } } const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST!, path: process.env.DATABRICKS_HTTP_PATH!, authType: 'token-provider', tokenProvider: new MyOAuthProvider(), enableTokenFederation: true, // optional: exchange for Databricks token }); ``` -------------------------------- ### client.connect() - Databricks OAuth Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Connects using Databricks OAuth, supporting both User-to-Machine (U2M) via browser flow and Machine-to-Machine (M2M) via client credentials. ```APIDOC ## client.connect with `authType: 'databricks-oauth'` ### Description Triggers a browser-based U2M flow when `oauthClientSecret` is omitted, or an M2M client-credentials flow when both `oauthClientId` and `oauthClientSecret` are provided. ### Parameters - **options** (object) - Required. Configuration object for the connection. - **host** (string) - Required. The Databricks SQL endpoint host. - **path** (string) - Required. The path to the SQL warehouse. - **authType** (string) - Must be set to `'databricks-oauth'`. - **oauthClientId** (string) - Optional. The OAuth client ID (required for M2M flow). - **oauthClientSecret** (string) - Optional. The OAuth client secret (required for M2M flow). ### Request Example ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); // U2M (user-to-machine): browser opens for user consent await client.connect({ host: 'adb-1234567890.1.azuredatabricks.net', path: '/sql/2.0/warehouses/abc123def456', authType: 'databricks-oauth', }); // M2M (machine-to-machine): service principal await client.connect({ host: 'adb-1234567890.1.azuredatabricks.net', path: '/sql/2.0/warehouses/abc123def456', authType: 'databricks-oauth', oauthClientId: process.env.DATABRICKS_CLIENT_ID, oauthClientSecret: process.env.DATABRICKS_CLIENT_SECRET, }); ``` ``` -------------------------------- ### Connect to Databricks with Personal Access Token Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Establish an HTTPS Thrift connection to a Databricks endpoint using a personal access token. Configure optional parameters like `socketTimeout` and `proxy`. The `userAgentEntry` can be used to tag requests. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); // Personal access token (default authType) await client.connect({ host: 'adb-1234567890.1.azuredatabricks.net', path: '/sql/2.0/warehouses/abc123def456', token: process.env.DATABRICKS_TOKEN, // Optional: custom socket timeout (ms), default 15 minutes socketTimeout: 10 * 60 * 1000, // Optional: HTTP proxy proxy: { protocol: 'https', host: 'proxy.corp.net', port: 8080 }, // Optional: tag requests from this client in the User-Agent header userAgentEntry: 'MyApp/1.0', }); console.log('Connected'); ``` -------------------------------- ### operation.fetchAll Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Fetches all result rows from an operation, looping internally over server pages. ```APIDOC ## operation.fetchAll(options?) ### Description Loops internally over all server pages and returns them as a single flat `Array`. Each object has keys matching the result column names. Optionally tracks query progress via a callback. ### Method `operation.fetchAll` ### Parameters - `options` (object, optional) - Configuration options. - `progress` (boolean, optional) - Request progress updates from the server. - `callback` (function, optional) - A callback function to process progress status updates. ### Request Example ```javascript const op = await session.executeStatement('SELECT id, name, value FROM large_table LIMIT 1000000'); const rows = await op.fetchAll({ progress: true, // request progress updates from the server callback: (statusResp) => { if (statusResp.progressUpdateResponse) { process.stdout.write('\r' + utils.formatProgress(statusResp.progressUpdateResponse)); } }, }); console.log(`\nTotal rows fetched: ${rows.length}`); console.table(rows.slice(0, 5)); // preview first 5 await op.close(); ``` ### Response Returns a `Promise>` which resolves to an array of result rows. Each row is an object with keys corresponding to the column names. ``` -------------------------------- ### Fetch All Result Rows with Progress Tracking Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Fetches all rows from a query result, internally handling pagination. Supports optional progress tracking via a callback function. ```javascript const { DBSQLClient, utils } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); const op = await session.executeStatement('SELECT id, name, value FROM large_table LIMIT 1000000'); const rows = await op.fetchAll({ progress: true, // request progress updates from the server callback: (statusResp) => { if (statusResp.progressUpdateResponse) { process.stdout.write('\r' + utils.formatProgress(statusResp.progressUpdateResponse)); } }, }); console.log(`\nTotal rows fetched: ${rows.length}`); console.table(rows.slice(0, 5)); // preview first 5 await op.close(); await session.close(); await client.close(); ``` -------------------------------- ### Advanced Metadata: `getFunctions`, `getPrimaryKeys`, `getCrossReference` Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Retrieve function definitions, primary key constraints, and foreign key relationships between tables. ```APIDOC ## Advanced Metadata Methods ### Description These methods retrieve function definitions, primary key constraints, and foreign key relationships between tables. ### Methods * `session.getFunctions(options?) * `session.getPrimaryKeys(options?) * `session.getCrossReference(options?) ### Parameters * `options` (object) - Optional. Filter parameters for the respective methods. * `functionName` (string) - Pattern for function names. * `schemaName` (string) - Name of the schema. * `tableName` (string) - Name of the table. * `parentCatalogName` (string) - Catalog name of the parent table. * `parentSchemaName` (string) - Schema name of the parent table. * `parentTableName` (string) - Name of the parent table. * `foreignCatalogName` (string) - Catalog name of the foreign table. * `foreignSchemaName` (string) - Schema name of the foreign table. * `foreignTableName` (string) - Name of the foreign table. ### Response * `IOperation` - An object representing the operation that can be used to fetch rows. ### Request Example ```javascript async function fetchAll(op) { const rows = await op.fetchAll(); await op.close(); return rows; } // Find built-in functions matching a pattern const funcs = await fetchAll(await session.getFunctions({ functionName: 'to_date%' })); console.log('Functions:', funcs.map(r => r.FUNCTION_NAME)); // Get primary keys of a table const pks = await fetchAll(await session.getPrimaryKeys({ schemaName: 'default', tableName: 'orders', })); console.log('Primary keys:', pks.map(r => r.COLUMN_NAME)); // Get foreign key relationships between two tables const fks = await fetchAll(await session.getCrossReference({ parentCatalogName: 'main', parentSchemaName: 'default', parentTableName: 'customers', foreignCatalogName: 'main', foreignSchemaName: 'default', foreignTableName: 'orders', })); console.log('Foreign keys:', fks); ``` ``` -------------------------------- ### Schema Introspection: Catalogs, Schemas, Tables, Columns Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Utilize `session.getCatalogs()`, `session.getSchemas()`, `session.getTables()`, and `session.getColumns()` to retrieve metadata. These methods accept optional filter parameters and return `IOperation` objects. Ensure operations are closed after fetching all rows. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); const session = await client.openSession(); async function fetchAll(op) { const rows = await op.fetchAll(); await op.close(); return rows; } // List all catalogs const catalogs = await fetchAll(await session.getCatalogs()); console.log('Catalogs:', catalogs.map(r => r.TABLE_CAT)); // List schemas in a catalog const schemas = await fetchAll(await session.getSchemas({ catalogName: 'main' })); console.log('Schemas:', schemas.map(r => r.TABLE_SCHEM)); // List tables matching a pattern const tables = await fetchAll(await session.getTables({ catalogName: 'main', schemaName: 'default', tableName: 'orders%', tableTypes: ['TABLE', 'VIEW'], })); console.log('Tables:', tables.map(r => r.TABLE_NAME)); // Describe columns of a specific table const columns = await fetchAll(await session.getColumns({ catalogName: 'main', schemaName: 'default', tableName: 'orders', })); columns.forEach(c => console.log(`${c.COLUMN_NAME}: ${c.TYPE_NAME}`)); await session.close(); await client.close(); ``` -------------------------------- ### Fix Code Formatting with Prettier Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Automatically fix code formatting issues according to Prettier rules. This should be run before submitting changes. ```bash npm run prettier:fix ``` -------------------------------- ### Sign Commits Automatically Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Configure Git to automatically sign your commit messages. Ensure your 'user.name' and 'user.email' Git configurations are set. ```bash git commit -s ``` -------------------------------- ### client.connect with authType: 'token-provider' — Custom ITokenProvider Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Provides full control over token management, including custom caching, refresh, and retry logic, by implementing the `ITokenProvider` interface and passing an instance to the `connect` method. ```APIDOC ## `client.connect` with `authType: 'token-provider'` — Custom `ITokenProvider` For full control—custom caching, refresh, retry, and error handling—implement the `ITokenProvider` interface and pass the instance as `tokenProvider`. ```typescript import { DBSQLClient } from '@databricks/sql'; import { ITokenProvider, Token } from '@databricks/sql/lib/connection/auth/tokenProvider'; class MyOAuthProvider implements ITokenProvider { async getToken(): Promise { const resp = await fetch('https://auth.example.com/oauth/token', { method: 'POST', headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, body: new URLSearchParams({ grant_type: 'client_credentials', client_id: process.env.CLIENT_ID!, client_secret: process.env.CLIENT_SECRET!, }), }); const data = (await resp.json()) as { access_token: string; expires_in: number }; return new Token(data.access_token, { expiresAt: new Date(Date.now() + data.expires_in * 1000), }); } getName() { return 'MyOAuthProvider'; } } const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST!, path: process.env.DATABRICKS_HTTP_PATH!, authType: 'token-provider', tokenProvider: new MyOAuthProvider(), enableTokenFederation: true, // optional: exchange for Databricks token }); ``` ``` -------------------------------- ### Apply Session-level and Statement-level Query Tags Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Tag queries using key-value pairs for session-level inheritance and statement-level overrides. Special characters and null values in tags are handled automatically. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); await client.connect({ host: process.env.DATABRICKS_HOST, path: process.env.DATABRICKS_HTTP_PATH, token: process.env.DATABRICKS_TOKEN, }); // Session-level tags inherited by all statements on this session const session = await client.openSession({ queryTags: { team: 'engineering', env: 'production', driver: 'node' }, }); // Statement A: inherits session tags const opA = await session.executeStatement('SELECT 1 AS inherits_session_tags'); console.log(await opA.fetchAll()); await opA.close(); // Statement B: per-statement tags (override env for this statement only) const opB = await session.executeStatement('SELECT 2 AS has_statement_tags', { queryTags: { env: 'staging', request_id: 'req-xyz-789', feature: 'reporting' }, }); console.log(await opB.fetchAll()); await opB.close(); // Statement C: null values and special-character escaping const opC = await session.executeStatement('SELECT 3 AS special_tags', { queryTags: { path: 'C:\\users\\me', // backslash escaped note: 'hello, world', // comma escaped flag: null, // bare key, no value }, }); console.log(await opC.fetchAll()); await opC.close(); await session.close(); await client.close(); ``` -------------------------------- ### Schema Introspection: `getCatalogs`, `getSchemas`, `getTables`, `getColumns` Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Provides methods for schema introspection, returning metadata about catalogs, schemas, tables, and columns. ```APIDOC ## Schema Introspection Methods ### Description These methods return metadata about catalogs, schemas, tables, and columns as `IOperation` objects. All accept optional filter parameters. ### Methods * `session.getCatalogs()` * `session.getSchemas(options?) * `session.getTables(options?) * `session.getColumns(options?) ### Parameters * `options` (object) - Optional. Filter parameters for `getSchemas`, `getTables`, and `getColumns`. * `catalogName` (string) - Name of the catalog. * `schemaName` (string) - Name of the schema. * `tableName` (string) - Name of the table (supports pattern matching). * `tableTypes` (string[]) - Array of table types to filter by (e.g., ['TABLE', 'VIEW']). ### Response * `IOperation` - An object representing the operation that can be used to fetch rows. ### Request Example ```javascript async function fetchAll(op) { const rows = await op.fetchAll(); await op.close(); return rows; } // List all catalogs const catalogs = await fetchAll(await session.getCatalogs()); console.log('Catalogs:', catalogs.map(r => r.TABLE_CAT)); // List schemas in a catalog const schemas = await fetchAll(await session.getSchemas({ catalogName: 'main' })); console.log('Schemas:', schemas.map(r => r.TABLE_SCHEM)); // List tables matching a pattern const tables = await fetchAll(await session.getTables({ catalogName: 'main', schemaName: 'default', tableName: 'orders%', tableTypes: ['TABLE', 'VIEW'], })); console.log('Tables:', tables.map(r => r.TABLE_NAME)); // Describe columns of a specific table const columns = await fetchAll(await session.getColumns({ catalogName: 'main', schemaName: 'default', tableName: 'orders', })); columns.forEach(c => console.log(`${c.COLUMN_NAME}: ${c.TYPE_NAME}`)); ``` ``` -------------------------------- ### Watch TypeScript Files for Hot Compilation Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Use this command to continuously compile TypeScript files as they are changed. This is useful during development for quick feedback. ```bash npm run watch ``` -------------------------------- ### client.connect with authType: 'external-token' — Dynamic token callback Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Connects to Databricks using an external token, supporting dynamic token fetching via a callback. It can also enable token federation for seamless token exchange with identity providers like Azure AD, Google, or Okta. ```APIDOC ## `client.connect` with `authType: 'external-token'` — Dynamic token callback Accepts an async `getToken` callback invoked whenever a fresh token is needed. Tokens are automatically cached by the driver. Setting `enableTokenFederation: true` activates RFC 8693 exchange so tokens from Azure AD, Google, or Okta are transparently swapped for Databricks tokens. ```typescript import { DBSQLClient } from '@databricks/sql'; async function fetchTokenFromVault(): Promise { // Replace with AWS Secrets Manager / Azure Key Vault / HashiCorp Vault call return process.env.DATABRICKS_TOKEN!; } const client = new DBSQLClient(); // Without federation (token is already a Databricks PAT / AAD token) await client.connect({ host: process.env.DATABRICKS_HOST!, path: process.env.DATABRICKS_HTTP_PATH!, authType: 'external-token', getToken: fetchTokenFromVault, }); // With federation: exchange an Azure AD JWT for a Databricks token automatically await client.connect({ host: process.env.DATABRICKS_HOST!, path: process.env.DATABRICKS_HTTP_PATH!, authType: 'external-token', getToken: async () => process.env.AZURE_AD_TOKEN!, enableTokenFederation: true, }); const session = await client.openSession(); const op = await session.executeStatement('SELECT current_user() AS user'); console.log(await op.fetchAll()); // [{ user: 'service-principal@tenant.com' }] await op.close(); await session.close(); await client.close(); ``` ``` -------------------------------- ### Connect to Databricks with Databricks OAuth (M2M) Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Connect using Databricks OAuth in a Machine-to-Machine (M2M) flow, also known as client-credentials flow. This requires providing both `oauthClientId` and `oauthClientSecret` for service principal authentication. ```javascript const { DBSQLClient } = require('@databricks/sql'); const client = new DBSQLClient(); // M2M (machine-to-machine): service principal await client.connect({ host: 'adb-1234567890.1.azuredatabricks.net', path: '/sql/2.0/warehouses/abc123def456', authType: 'databricks-oauth', oauthClientId: process.env.DATABRICKS_CLIENT_ID, oauthClientSecret: process.env.DATABRICKS_CLIENT_SECRET, }); ``` -------------------------------- ### Lint Code Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Run the linter to check for code style and potential errors. This helps maintain code quality. ```bash npm run lint ``` -------------------------------- ### `operation.getSchema(options?)` Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Fetches the `TTableSchema` describing column names, types, and positions for a result set. This is useful for schema inspection before fetching data. ```APIDOC ## operation.getSchema(options?) ### Description Returns the `TTableSchema` describing column names, types, and positions. Useful for schema inspection before fetching data. ### Method ```javascript await op.getSchema(); ``` ### Parameters * `options` (object) - Optional. Configuration options for fetching the schema. ### Response * `TTableSchema` - An object containing schema information, including a `columns` array. ### Request Example ```javascript const schema = await op.getSchema(); schema?.columns.forEach((col) => { console.log(`${col.columnName}: ${JSON.stringify(col.typeDesc)}`); }); ``` ``` -------------------------------- ### Fetch a single page of results Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Fetches up to a specified number of rows (default 100,000) per call. This method is designed for manual pagination when combined with `hasMoreRows()`. ```APIDOC ## fetchChunk(options?) ### Description Returns up to `maxRows` rows (default 100,000) per call. Combine with `hasMoreRows()` for manual pagination. ### Parameters #### Options - **maxRows** (number) - Optional - The maximum number of rows to fetch in this chunk. ### Request Example ```javascript const chunk = await op.fetchChunk({ maxRows: 10000 }); ``` ### Response #### Success Response - **chunk** (Array) - An array of row objects. ### Usage Example ```javascript let page = 0; while (await op.hasMoreRows()) { const chunk = await op.fetchChunk({ maxRows: 10000 }); console.log(`Page ${++page}: ${chunk.length} rows`); // process chunk... } ``` ``` -------------------------------- ### Connect with Dynamic Token Callback Source: https://context7.com/databricks/databricks-sql-nodejs/llms.txt Use `authType: 'external-token'` with a `getToken` callback for dynamic token retrieval. Enable `enableTokenFederation: true` to automatically exchange external tokens for Databricks tokens. ```typescript import { DBSQLClient } from '@databricks/sql'; async function fetchTokenFromVault(): Promise { // Replace with AWS Secrets Manager / Azure Key Vault / HashiCorp Vault call return process.env.DATABRICKS_TOKEN!; } const client = new DBSQLClient(); // Without federation (token is already a Databricks PAT / AAD token) await client.connect({ host: process.env.DATABRICKS_HOST!, path: process.env.DATABRICKS_HTTP_PATH!, authType: 'external-token', getToken: fetchTokenFromVault, }); // With federation: exchange an Azure AD JWT for a Databricks token automatically await client.connect({ host: process.env.DATABRICKS_HOST!, path: process.env.DATABRICKS_HTTP_PATH!, authType: 'external-token', getToken: async () => process.env.AZURE_AD_TOKEN!, enableTokenFederation: true, }); const session = await client.openSession(); const op = await session.executeStatement('SELECT current_user() AS user'); console.log(await op.fetchAll()); // [{ user: 'service-principal@tenant.com' }] await op.close(); await session.close(); await client.close(); ``` -------------------------------- ### Fix Linting Errors Source: https://github.com/databricks/databricks-sql-nodejs/blob/main/CONTRIBUTING.md Automatically fix linting errors according to the project's linting rules. This should be run before submitting changes. ```bash npm run lint:fix ```