### Example: Adding Circuit Breaker Flag Source: https://github.com/databricks/databricks-sql-go/blob/main/telemetry/ADDING_FEATURE_FLAGS.md A comprehensive example demonstrating the addition of a 'circuit breaker' feature flag, covering all four steps: adding the constant, registering it, creating the checking method, and using it in code. ```go // Step 1: Add constant const ( flagEnableTelemetry = "databricks.partnerplatform.clientConfigsFeatureFlags.enableTelemetryForGoDriver" flagEnableCircuitBreaker = "databricks.partnerplatform.clientConfigsFeatureFlags.enableCircuitBreakerForGoDriver" ) // Step 2: Register for fetching func getAllFeatureFlags() []string { return []string{ flagEnableTelemetry, flagEnableCircuitBreaker, } } // Step 3: Add public method func (c *featureFlagCache) isCircuitBreakerEnabled(ctx context.Context, host string, httpClient *http.Client) (bool, error) { return c.getFeatureFlag(ctx, host, httpClient, flagEnableCircuitBreaker) } // Step 4: Use it if enabled, _ := flagCache.isCircuitBreakerEnabled(ctx, host, httpClient); enabled { // Use circuit breaker } ``` -------------------------------- ### Transaction Start Comparison Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Compares how transactions are initiated in JDBC and Go implementations. The Go implementation uses `db.BeginTx` for explicit transaction control. ```go db.BeginTx(ctx, opts) ``` -------------------------------- ### Example Usage of Feature Flag Check Source: https://github.com/databricks/databricks-sql-go/blob/main/telemetry/ADDING_FEATURE_FLAGS.md Demonstrates how to use the newly added feature flag checking method in your application code. It includes error handling and conditional logic based on the flag's status. ```go // Example usage in your code: flagCache := getFeatureFlagCache() enabled, err := flagCache.isNewFeatureEnabled(ctx, host, httpClient) if err != nil { // Handle error (falls back to false on error with no cache) } if enabled { // Feature is enabled - use new behavior } else { // Feature is disabled - use old behavior } ``` -------------------------------- ### Fetch All Flags API Request Example Source: https://github.com/databricks/databricks-sql-go/blob/main/telemetry/ADDING_FEATURE_FLAGS.md Illustrates the HTTP GET request format used to fetch all feature flags for a given host. Multiple flags are requested in a single API call. ```http GET /api/2.0/feature-flags?flags=flagOne,flagTwo,flagThree ``` -------------------------------- ### Test BeginTx with Read Committed Isolation Level Returns Error Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Confirms that the driver returns an 'ErrUnsupportedIsolation' error when attempting to start a transaction with the Read Committed isolation level (2), indicating that this specific isolation level is not supported. ```go // Call `BeginTx()` with `IsolationLevel = 2` (read committed) // Verify returns `ErrUnsupportedIsolation` // Verify no SQL executed ``` -------------------------------- ### Implement Client-Side Caching for Autocommit State Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md This Go code implements client-side caching of the autocommit boolean within a connection struct. It avoids round-trips by checking the cached state before initiating a transaction and updates the cache after successfully starting or ending a transaction. This approach is chosen for its performance benefits and consistency with JDBC. ```go type conn struct { autoCommit bool // Cached state } func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) { // Check cached state (no round-trip) if !c.autoCommit { return nil, errors.New("transaction already active") } // Start transaction _, _, err := c.runQuery(ctx, "SET AUTOCOMMIT = FALSE", nil) if err != nil { return nil, err } // Update cache c.autoCommit = false return &tx{conn: c}, nil } ``` -------------------------------- ### Begin() Method Wrapper Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md A simple wrapper method for compatibility that delegates to BeginTx with default options, including background context and default isolation level (Serializable). ```go func (c *conn) Begin() (driver.Tx, error) ``` -------------------------------- ### Begin() Method Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Begins a new transaction using default settings. This method is a simple wrapper that delegates to `BeginTx` with a background context and default transaction options. ```APIDOC ## Begin() Method ### Description Begins a new transaction using default settings. This method serves as a compatibility wrapper by delegating the call to `BeginTx`. It utilizes `context.Background()` and an empty `driver.TxOptions` struct, effectively starting a transaction with the default isolation level (Serializable) and no other specific options. ### Method Signature ```go func (c *conn) Begin() (driver.Tx, error) ``` ### Behavior Delegates to `BeginTx(context.Background(), driver.TxOptions{})`. ### Contract - Simple wrapper for compatibility. - Uses default isolation level (Serializable). - Uses background context. ``` -------------------------------- ### BeginTx() Method Implementation Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Initiates a new transaction with specified options. It validates the isolation level, checks the autocommit cache, executes SET AUTOCOMMIT = FALSE, and updates the cache on success. It rejects unsupported isolation levels and returns specific error types for validation or execution failures. ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) ``` -------------------------------- ### BeginTx() Method Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Begins a new transaction with specified options. It validates the isolation level, checks the autocommit cache, and executes `SET AUTOCOMMIT = FALSE`. It returns a `driver.Tx` instance on success or an error if validation or execution fails. ```APIDOC ## BeginTx() Method ### Description Begins a new transaction with specified options, including the isolation level. This method first validates the provided isolation level, checks if autocommit is currently enabled (it must be `true` to start a new transaction), and then executes `SET AUTOCOMMIT = FALSE` to disable it. On success, it returns a `driver.Tx` instance representing the new transaction. Errors are returned for validation failures or if the `SET AUTOCOMMIT` statement execution fails. ### Method Signature ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) ``` ### Behavior 1. Validates the isolation level from `opts`. 2. Checks the `autoCommit` cache (must be `true`). 3. Executes `SET AUTOCOMMIT = FALSE` statement. 4. Updates `conn.autoCommit = false` on success. 5. Returns `&tx{conn: c}` instance. ### Contracts - Only accepts `driver.IsolationLevel`: `LevelDefault` (0) or `LevelSerializable` (8). - Rejects: `LevelReadUncommitted`, `LevelReadCommitted`, `LevelRepeatableRead`. - Returns `DBDriverError` for validation failures (no SQL executed). - Returns `DBExecutionError` for SQL execution failures. - Does NOT update the cache if SQL fails. - Respects context cancellation throughout the operation. - Thread-safe via database/sql's connection locking. ### Error Conditions | Condition | Check | Error Type | SQL Executed | |-----------|-------|------------|--------------| | Unsupported isolation level | Validate opts | DBDriverError | No | | autoCommit == false | Check cache | DBDriverError | No | | ReadOnly transaction | Check opts | DBDriverError | No | | SET AUTOCOMMIT fails | SQL error | DBExecutionError | Yes | | Context cancelled | ctx.Done() | Context error | Partial | ``` -------------------------------- ### Connect using New Connector Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Instantiate a new connector object with functional options for server hostname, port, HTTP path, and access token. This approach provides more explicit configuration. ```go import ( "database/sql" _ "github.com/databricks/databricks-sql-go" ) connector, err := dbsql.NewConnector( dbsql.WithServerHostname(), dbsql.WithPort(), dbsql.WithHTTPPath(), dbsql.WithAccessToken() ) if err != nil { log.Fatal(err) } db := sql.OpenDB(connector) deferr db.Close() ``` -------------------------------- ### Begin Transaction Querying Server State Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md An alternative approach to initiating a transaction by first querying the server for the current autocommit state. This method ensures accuracy but incurs an extra round-trip, impacting performance. ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) { // Query current autocommit state from server rows, _ := c.runQuery(ctx, "SET AUTOCOMMIT", nil) currentState := parseAutoCommitState(rows) if !currentState { return nil, errors.New("transaction already active") } // Proceed to start transaction _, _, err := c.runQuery(ctx, "SET AUTOCOMMIT = FALSE", nil) // ... } ``` -------------------------------- ### Run Unit Tests Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Execute unit tests for the Databricks SQL Go driver. ```bash go test ``` -------------------------------- ### Connection Initialization with AutoCommit Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Shows how the 'autoCommit' cache is initialized to 'true' during connection creation in the Connect method of the connector. This sets the default state for new connections. ```go func (c *connector) Connect(ctx context.Context) (driver.Conn, error) { // ... existing session creation code ... conn := &conn{ id: connId, cfg: c.cfg, client: c.client, session: sessionResp, autoCommit: true, // NEW: Initialize cache } return conn, nil } ``` -------------------------------- ### Component Diagram for databricks-sql-go MST Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Illustrates the interaction between Go's standard database/sql package, the driver interfaces, and the custom connection and transaction implementations within databricks-sql-go, as well as the external Databricks Server. ```mermaid classDiagram class database_sql { <> +Begin() Tx +BeginTx(ctx, opts) Tx } class driver_Tx { <> +Commit() error +Rollback() error } class driver_ConnBeginTx { <> +BeginTx(ctx, opts) Tx } class conn { -string id -Config cfg -TCLIService client -TOpenSessionResp session -bool autoCommit +Begin() driver.Tx +BeginTx(ctx, opts) driver.Tx +ExecContext(ctx, query, args) +QueryContext(ctx, query, args) +ResetSession(ctx) error } class tx { -conn* conn +Commit() error +Rollback() error } class DatabricksServer { <> +ExecuteStatement() +ManagesTransactionState() } database_sql --> driver_ConnBeginTx : uses database_sql --> driver_Tx : uses conn ..|> driver_ConnBeginTx : implements tx ..|> driver_Tx : implements conn --> tx : creates tx --> conn : references conn --> DatabricksServer : SQL commands ``` -------------------------------- ### DSN with Cloud Fetch Enabled Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Enable Cloud Fetch for improved performance with large results and configure the number of download threads. ```go token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?useCloudFetch=true&maxDownloadThreads=3 ``` -------------------------------- ### Test BeginTx with Serializable Isolation Level Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Verifies that the driver supports and successfully initiates transactions with the Serializable isolation level (8). This test ensures compatibility with higher isolation settings. ```go // Call `BeginTx()` with `IsolationLevel = 8` (serializable) // Verify succeeds ``` -------------------------------- ### AutoCommit State Machine for databricks-sql-go Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Visualizes the transitions between different autocommit states within the databricks-sql-go driver, from connection creation through transaction execution, commit, rollback, and resetting to the default autocommit state. ```mermaid stateDiagram-v2 [*] --> AutoCommitTrue: Connection Created AutoCommitTrue --> ValidatingBeginTx: BeginTx() called ValidatingBeginTx --> ExecutingSetAutoCommit: Validation passed ValidatingBeginTx --> AutoCommitTrue: Validation failed (error) ExecutingSetAutoCommit --> TransactionActive: SET AUTOCOMMIT=FALSE succeeds ExecutingSetAutoCommit --> AutoCommitTrue: SET AUTOCOMMIT=FALSE fails (error) TransactionActive --> ExecutingStatements: Execute queries/updates ExecutingStatements --> TransactionActive: Statement succeeds ExecutingStatements --> AbortedState: Statement fails TransactionActive --> ExecutingCommit: Commit() called ExecutingCommit --> ResettingAutoCommit: COMMIT succeeds/fails AbortedState --> ExecutingRollback: Rollback() called AbortedState --> AbortedState: Commit() called (error) AbortedState --> AbortedState: Execute() called (error) TransactionActive --> ExecutingRollback: Rollback() called ExecutingRollback --> ResettingAutoCommit: ROLLBACK completes ResettingAutoCommit --> AutoCommitTrue: SET AUTOCOMMIT=TRUE (always) note right of AutoCommitTrue cache: autoCommit = true Server: autocommit = 1 end note note right of TransactionActive cache: autoCommit = false Server: autocommit = 0 Transaction active end note note right of AbortedState cache: autoCommit = false Server: transaction aborted Only ROLLBACK allowed end note note right of ResettingAutoCommit ALWAYS resets autoCommit=true Even if COMMIT/ROLLBACK fails Uses defer pattern end note ``` -------------------------------- ### Transaction with Timeout using Context Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Demonstrates how to control transaction timeouts by using `context.WithTimeout`. The `BeginTx`, `Commit`, and `Rollback` operations will respect the provided context's deadline. ```go // Transaction with timeout ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) defer cancel() tx, err := db.BeginTx(ctx, nil) // ... transaction operations respect timeout ... err = tx.Commit() ``` -------------------------------- ### Begin Transaction with SET AUTOCOMMIT=FALSE Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Initiates a transaction by setting AUTOCOMMIT to FALSE. If this fails, the connection's autocommit cache is not updated, and an error is returned. ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) { // ... validation ... _, _, err := c.runQuery(ctx, "SET AUTOCOMMIT = FALSE", nil) if err != nil { // DO NOT update cache return nil, dbsqlerrint.NewExecutionError(ctx, dbsqlerrint.ErrTransactionBegin, err, nil) } // Only update on success c.autoCommit = false return &tx{conn: c}, nil } ``` -------------------------------- ### Go: Context Cancellation in BeginTx Scenario Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Shows how context cancellation during `db.BeginTx()` is detected by `runQuery()`. The operation may be cancelled on the server, and the function returns `ctx.Err()`. ```go ctx, cancel := context.WithCancel(context.Background()) cancel() // Cancel immediately tx, err := db.BeginTx(ctx, nil) ``` -------------------------------- ### Error Wrapping Patterns Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Illustrates two patterns for error wrapping: one for validation errors where no SQL is executed, and another for SQL execution errors that includes operation status. ```go // Validation errors - no SQL executed return nil, dbsqlerrint.NewDriverError(ctx, dbsqlerr.ErrTransactionNested, nil) // SQL execution errors - includes operation status return nil, dbsqlerrint.NewExecutionError(ctx, dbsqlerr.ErrTransactionCommit, err, opStatusResp) ``` -------------------------------- ### Proper Aborted State Handling Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Illustrates the correct way to handle an aborted transaction. Instead of attempting to continue or commit, the application should detect the error and explicitly roll back the transaction. ```mermaid sequenceDiagram participant App as Application participant Conn as conn/tx participant Server as Databricks Gateway App->>Conn: tx.Exec("INVALID SQL") Conn->>Server: ExecuteStatement() Server-->>Conn: ERROR Note over Server: Transaction ABORTED Conn-->>App: error Note over App: Detect error, rollback App->>Conn: tx.Rollback() Conn->>Server: ROLLBACK Server-->>Conn: Success (clears aborted state) Conn->>Server: SET AUTOCOMMIT = TRUE Conn->>Conn: autoCommit = true Conn-->>App: nil (success) Note over App: Connection ready for reuse ``` -------------------------------- ### Implicit Transaction Flow Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Illustrates the recommended pattern for implicit transactions where the driver automatically manages `AUTOCOMMIT` settings. This flow is initiated by `db.Begin()` and involves setting `AUTOCOMMIT` to FALSE on the server. ```mermaid sequenceDiagram participant App as Application participant DB as database/sql participant Conn as conn participant Server as Databricks Gateway App->>DB: db.Begin() DB->>Conn: BeginTx(ctx, nil) Note over Conn: Check autoCommit == true Conn->>Conn: Validate: autoCommit must be true Conn->>Server: SET AUTOCOMMIT = FALSE Server-->>Conn: Success Conn->>Conn: autoCommit = false Conn-->>DB: tx{conn} DB-->>App: tx Note over App: Execute statements in transaction App->>DB: tx.Exec("INSERT ...") DB->>Server: ExecuteStatement(INSERT) Server-->>DB: Success DB-->>App: Result App->>DB: tx.Exec("UPDATE ...") DB->>Server: ExecuteStatement(UPDATE) Server-->>DB: Success DB-->>App: Result App->>DB: tx.Commit() DB->>Conn: tx.Commit() Note over Conn: defer: Always reset autoCommit Conn->>Server: COMMIT Server-->>Conn: Success Conn->>Server: SET AUTOCOMMIT = TRUE Server-->>Conn: Success Conn->>Conn: autoCommit = true Conn-->>DB: nil DB-->>App: nil (success) ``` -------------------------------- ### Test BeginTx Sets Autocommit False Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Verifies that calling BeginTx correctly sets the internal autoCommit state to false and executes the 'SET AUTOCOMMIT = FALSE' SQL command. It also ensures that a valid transaction instance is returned. ```go // Verify `autoCommit` changes from `true` to `false` // Verify SQL command `SET AUTOCOMMIT = FALSE` executed // Verify returns `tx` instance ``` -------------------------------- ### Begin Transaction with SET AUTOCOMMIT = FALSE Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Initiates a multi-statement transaction by setting AUTOCOMMIT to FALSE. This method reuses existing context handling, correlation ID propagation, polling logic, and error wrapping patterns. ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) { // ... validation ... // Use existing runQuery for SET AUTOCOMMIT _, _, err := c.runQuery(ctx, "SET AUTOCOMMIT = FALSE", nil) if err != nil { return nil, dbsqlerrint.NewExecutionError(ctx, dbsqlerr.ErrTransactionBegin, err, nil) } c.autoCommit = false return &tx{conn: c}, nil } ``` -------------------------------- ### DSN with Timeout and Max Rows Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Configure query timeout in seconds and maximum rows per network request using DSN parameters. ```go token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?timeout=1000&maxRows=1000 ``` -------------------------------- ### Connect and Query Databricks SQL Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Basic usage for connecting to Databricks SQL and executing a query. Ensure the databricks-sql-go package is imported. ```go import ( "context" "database/sql" _ "github.com/databricks/databricks-sql-go" ) db, err := sql.Open("databricks", "token:********@********.databricks.com:443/sql/1.0/endpoints/********") if err != nil { panic(err) } deferr db.Close() rows, err := db.QueryContext(context.Background(), "SELECT 1") deferr rows.Close() ``` -------------------------------- ### Rollback() Method Implementation Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Implements the Rollback method for transactions. It ensures autocommit is reset using defer, executes the ROLLBACK statement, sets AUTOCOMMIT to TRUE, updates the internal cache, and returns any errors encountered. This method is forgiving and can succeed even without an active transaction. ```go func (t *tx) Rollback() error ``` -------------------------------- ### Commit() Method Implementation Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Implements the Commit method for transactions. It ensures autocommit is reset using defer, executes the COMMIT statement, sets AUTOCOMMIT to TRUE, updates the internal cache, and returns any errors encountered. ```go func (t *tx) Commit() error ``` -------------------------------- ### Test BeginTx with Default Isolation Level Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Checks that calling BeginTx with the default isolation level (0) succeeds without errors, indicating support for the default transaction isolation settings. ```go // Call `BeginTx()` with `IsolationLevel = 0` (default) // Verify succeeds ``` -------------------------------- ### Go: Manual SET AUTOCOMMIT Scenario Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Highlights the issue of manually executing `SET AUTOCOMMIT = 0` which bypasses the driver's internal cache. This can lead to a desynchronized state between the driver and the server, potentially causing nested transaction errors. ```go db.Exec("SET AUTOCOMMIT = 0") // Bypasses driver cache // Cache still shows autoCommit=true tx, err := db.Begin() // Driver thinks it's starting transaction // Actually nested transaction attempt on server ``` -------------------------------- ### Configure Connection Pool Settings in Go Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Configure connection pool settings based on application concurrency needs. This includes setting the maximum number of open connections, idle connections, and connection maximum lifetime. ```go // Configure based on concurrency needs db.SetMaxOpenConns(100) // Max concurrent connections db.SetMaxIdleConns(10) // Keep 10 idle for fast acquisition db.SetConnMaxLifetime(1*time.Hour) // Recycle connections ``` -------------------------------- ### Go: Double Rollback Scenario Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Demonstrates attempting to rollback a transaction twice. The server accepts the second ROLLBACK command as a no-op, and the driver returns nil for both operations, matching JDBC behavior. ```go tx, _ := db.Begin() err1 := tx.Rollback() // Success, autoCommit: false → true err2 := tx.Rollback() // Also succeeds (server allows) ``` -------------------------------- ### DSN with Telemetry Enabled Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Opt-in to telemetry collection for performance and reliability improvements. This respects server-side feature flags. ```go token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?enableTelemetry=true ``` -------------------------------- ### VS Code Lint Settings Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Configure VS Code to use golangci-lint for linting Go code, with a fast check option. ```json { "go.lintTool": "golangci-lint", "go.lintFlags": [ "--fast" ] } ``` -------------------------------- ### Test BeginTx with Read Uncommitted Isolation Level Returns Error Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Ensures that an error is returned when attempting to initiate a transaction with the Read Uncommitted isolation level (1), highlighting the driver's limitations or specific configuration requirements for this isolation level. ```go // Call `BeginTx()` with `IsolationLevel = 1` (read uncommitted) // Verify returns error ``` -------------------------------- ### Go: Connection Pool Reuse Scenario Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Presents a scenario where a connection is returned to the pool with `autoCommit=false` due to an unhandled transaction. The `ResetSession` logic handles this by resetting the connection's state. ```go // Goroutine 1: Starts transaction but crashes tx, _ := db.Begin() // autoCommit: true → false // goroutine exits without Commit/Rollback // connection returned to pool with autoCommit=false // Goroutine 2: Gets same connection from pool tx2, err := db.Begin() // What happens? ``` -------------------------------- ### Test BeginTx Returns Error When Already Active Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Ensures that calling BeginTx when the autoCommit state is already false (indicating an active transaction) returns the expected error (ErrTransactionNested). It also verifies that no SQL commands are executed and the autoCommit state remains unchanged. ```go // Set `autoCommit = false` manually // Call `BeginTx()` // Verify returns `ErrTransactionNested` // Verify no SQL executed // Verify `autoCommit` remains `false` ``` -------------------------------- ### Go: Context Cancellation in Commit Scenario Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Illustrates context cancellation during a `Commit()` operation. The commit proceeds using a background context, unaffected by the original request context's expiration. The `defer` ensures autocommit is reset. ```go ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) tx, _ := db.BeginTx(ctx, nil) time.Sleep(200 * time.Millisecond) err := tx.Commit() // Context already expired ``` -------------------------------- ### Hybrid Autocommit Validation in Go Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md This Go code snippet demonstrates a hybrid approach for managing autocommit state, where the client caches the state but periodically validates it against the server if a configuration option is enabled. This offers a balance between performance and accuracy but introduces more complexity. ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) { // Check cache if !c.autoCommit { return nil, errors.New("transaction already active") } // Optionally validate (if config enabled) if c.cfg.ValidateAutoCommit { c.validateCacheAgainstServer(ctx) } // Proceed with transaction // ... } ``` -------------------------------- ### Construct Validation Error (No SQL Execution) Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Creates a driver error when validation fails before any SQL is executed, such as detecting a nested transaction or unsupported isolation level. ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) { // Check for nested transaction if !c.autoCommit { return nil, dbsqlerrint.NewDriverError(ctx, dbsqlerrint.ErrTransactionNested, nil) } // Check isolation level if opts.Isolation != driver.IsolationLevel(sql.LevelDefault) && opts.Isolation != driver.IsolationLevel(sql.LevelSerializable) { return nil, dbsqlerrint.NewDriverError(ctx, dbsqlerrint.ErrUnsupportedIsolation, nil) } // ... continue with SQL execution ... } ``` -------------------------------- ### Sign Commits with Git Source: https://github.com/databricks/databricks-sql-go/blob/main/CONTRIBUTING.md To automatically sign your commit messages, configure your git user name and email. ```bash git commit -s ``` -------------------------------- ### Aborted State Recovery Flow Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Demonstrates the flow when a transaction enters an aborted state due to an error, such as invalid SQL. Subsequent operations on the same transaction will fail, and a new transaction must be initiated. ```mermaid sequenceDiagram participant App as Application participant Conn as conn/tx participant Server as Databricks Gateway Note over App,Server: Transaction is active (autoCommit=false) App->>Conn: tx.Exec("INVALID SQL") Conn->>Server: ExecuteStatement() Server-->>Conn: ERROR: SQL syntax error Note over Server: Transaction enters
ABORTED state Conn-->>App: DBExecutionError Note over App: Try to continue transaction App->>Conn: tx.Exec("SELECT 1") Conn->>Server: ExecuteStatement() Server-->>Conn: ERROR: transaction aborted Conn-->>App: DBExecutionError Note over App: Try to commit App->>Conn: tx.Commit() Conn->>Server: COMMIT Server-->>Conn: ERROR: cannot commit aborted transaction Note over Conn: defer still resets autoCommit=true Conn->>Server: SET AUTOCOMMIT = TRUE Conn->>Conn: autoCommit = true Conn-->>App: DBExecutionError Note over App: Must begin new transaction App->>Conn: db.Begin() Conn->>Server: SET AUTOCOMMIT = FALSE Server-->>Conn: Success Conn-->>App: new tx ``` -------------------------------- ### Debug Mode: Validate Autocommit Cache Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md In debug mode (FetchAutoCommitFromServer=true), this function queries the server state for AUTOCOMMIT and logs a warning if the cache mismatches the server's state. ```go type Config struct { // ... existing fields ... FetchAutoCommitFromServer bool // Future: debug mode } func (c *conn) validateAutoCommitCache(ctx context.Context) { if !c.cfg.FetchAutoCommitFromServer { return } // Query server state rows, _ := c.runQuery(ctx, "SET AUTOCOMMIT", nil) serverState := parseAutoCommitFromRows(rows) if serverState != c.autoCommit { log.Warn().Msgf( "AutoCommit cache mismatch: cache=%v server=%v", c.autoCommit, serverState) } } ``` -------------------------------- ### Go: Nested Transaction Attempt Scenario Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Illustrates the scenario where a second transaction is attempted before the first one is committed or rolled back. This is detected by checking `conn.autoCommit == false`. ```go tx1, err := db.Begin() // autoCommit: true → false tx2, err := db.Begin() // ERROR: autoCommit already false ``` -------------------------------- ### DSN with Cloud Fetch Disabled Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Disable Cloud Fetch to avoid overhead when handling smaller datasets. ```go token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?useCloudFetch=false ``` -------------------------------- ### Correct Pattern: Concurrent Transactions with Connection Pool Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md The correct pattern for concurrent transactions involves using a connection pool. Each goroutine acquires its own connection from the pool, allowing true parallel execution of transactions. ```go // GOOD: Each goroutine gets own connection db.SetMaxOpenConns(10) // Allow 10 concurrent connections var wg sync.WaitGroup for i := 0; i < 5; i++ { wg.Add(1) go func(id int) { defer wg.Done() // Each goroutine gets own connection from pool tx, err := db.Begin() if err != nil { log.Printf("Transaction %d failed to begin: %v", id, err) return } // Execute transaction operations _, err = tx.Exec("INSERT INTO table VALUES (?)", id) if err != nil { tx.Rollback() return } err = tx.Commit() if err != nil { log.Printf("Transaction %d failed to commit: %v", id, err) } }(i) } wg.Wait() ``` -------------------------------- ### Register Flag for Fetching in featureflag.go Source: https://github.com/databricks/databricks-sql-go/blob/main/telemetry/ADDING_FEATURE_FLAGS.md Add your new flag constant to the `getAllFeatureFlags()` function in `featureflag.go`. This ensures the flag is included when fetching all feature flags. ```go func getAllFeatureFlags() []string { return []string{ flagEnableTelemetry, flagEnableNewFeature, // Add your new flag here } } ``` -------------------------------- ### Tx Struct Definition Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Defines the `tx` struct in the Go driver, which implements the `driver.Tx` interface. It holds a reference to the parent connection and is not thread-safe, relying on `database/sql` for locking. ```go type tx struct { conn *conn } ``` -------------------------------- ### Go: Connection Reset Session Logic Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md This function is called by the database/sql package before reusing a connection from the pool. It ensures that if a connection was left in a non-autocommit state, it is reset to autocommit mode. ```go func (c *conn) ResetSession(ctx context.Context) error { if !c.autoCommit { log.Warn().Msg("databricks: connection has autoCommit=false, resetting") _, _, err := c.runQuery(ctx, "SET AUTOCOMMIT = TRUE", nil) if err != nil { log.Err(err).Msg("databricks: failed to reset autocommit") return driver.ErrBadConn // Mark connection as bad } c.autoCommit = true } return nil } ``` -------------------------------- ### Conn Struct with AutoCommit Cache Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Defines the conn struct, including a new boolean field 'autoCommit' to cache the autocommit state. This field is crucial for managing transaction states without querying the server. ```go type conn struct { id string cfg *config.Config client cli_service.TCLIService session *cli_service.TOpenSessionResp autoCommit bool // NEW: Cache autocommit state } ``` -------------------------------- ### Go: Defer for Guaranteed State Reset in Commit Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Ensures `SET AUTOCOMMIT = TRUE` is always executed after a COMMIT attempt, regardless of the COMMIT outcome. This prevents the connection from remaining in transaction mode. ```go func (t *tx) Commit() error { ctx := driverctx.NewContextWithConnId(context.Background(), t.conn.id) corrId := driverctx.CorrelationIdFromContext(ctx) ctx = driverctx.NewContextWithCorrelationId(ctx, corrId) log := logger.WithContext(t.conn.id, corrId, "") // CRITICAL: defer for guaranteed state reset defer func() { // Always execute SET AUTOCOMMIT = TRUE _, _, resetErr := t.conn.runQuery(ctx, "SET AUTOCOMMIT = TRUE", nil) if resetErr != nil { log.Err(resetErr).Msg("databricks: failed to reset autocommit after commit") } // Always update cache t.conn.autoCommit = true }() // Execute COMMIT _, _, err := t.conn.runQuery(ctx, "COMMIT", nil) if err != nil { log.Err(err).Msg("databricks: commit failed") return dbsqlerrint.NewExecutionError(ctx, dbsqlerrint.ErrTransactionCommit, err, nil) } return nil } ``` -------------------------------- ### Rollback Flow Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Details the sequence for rolling back an active transaction. The `tx.Rollback()` call ensures that any pending changes are discarded and the connection's `AUTOCOMMIT` is reset. ```mermaid sequenceDiagram participant App as Application participant Conn as conn/tx participant Server as Databricks Gateway App->>Conn: tx.Rollback() Note over Conn: defer: Always reset autoCommit Conn->>Server: ROLLBACK Note over Server: Forgiving: succeeds even
if no active transaction Server-->>Conn: Success Conn->>Server: SET AUTOCOMMIT = TRUE Server-->>Conn: Success Conn->>Conn: autoCommit = true Conn-->>App: nil (success) ``` -------------------------------- ### Commit Transaction with Context Propagation Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Commits a transaction using a background context to ensure completion, even if the request context is cancelled. It preserves the correlation ID from the connection context. ```go func (t *tx) Commit() error { // Create background context to avoid cancellation ctx := context.Background() // Add connection ID ctx = driverctx.NewContextWithConnId(ctx, t.conn.id) // Preserve correlation ID from connection context corrId := t.conn.getCorrelationId() // Hypothetical ctx = driverctx.NewContextWithCorrelationId(ctx, corrId) // Use for SQL operations _, _, err := t.conn.runQuery(ctx, "COMMIT", nil) return err } ``` -------------------------------- ### Add Public Method to Check Flag Status Source: https://github.com/databricks/databricks-sql-go/blob/main/telemetry/ADDING_FEATURE_FLAGS.md Implement a public method in `featureflag.go` to check the status of your new feature flag. This method utilizes the `getFeatureFlag` helper function. ```go // isNewFeatureEnabled checks if the new feature is enabled for the host. // Uses cached value if available and not expired. func (c *featureFlagCache) isNewFeatureEnabled(ctx context.Context, host string, httpClient *http.Client) (bool, error) { return c.getFeatureFlag(ctx, host, httpClient, flagEnableNewFeature) } ``` -------------------------------- ### Add New Flag Constant in featureflag.go Source: https://github.com/databricks/databricks-sql-go/blob/main/telemetry/ADDING_FEATURE_FLAGS.md Define a new constant for your feature flag in the `featureflag.go` file. This constant will be used to identify the flag throughout the system. ```go const ( // ... existing constants ... // flagEnableTelemetry controls whether telemetry is enabled for the Go driver flagEnableTelemetry = "databricks.partnerplatform.clientConfigsFeatureFlags.enableTelemetryForGoDriver" // YOUR NEW FLAG - Add it here flagEnableNewFeature = "databricks.partnerplatform.clientConfigsFeatureFlags.enableNewFeatureForGoDriver" ) ``` -------------------------------- ### New Transaction Error Constants Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Defines new error constants for transaction-related operations in the Databricks SQL Go driver. ```go const ( // Transaction errors - NEW ErrTransactionBegin = "failed to begin transaction" ErrTransactionCommit = "failed to commit transaction" ErrTransactionRollback = "failed to rollback transaction" ErrTransactionNested = "transaction already in progress" ErrUnsupportedIsolation = "unsupported transaction isolation level" ) ``` -------------------------------- ### Test Rollback Resets Autocommit to True Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Confirms that rolling back a transaction correctly resets the autoCommit state to true and executes the ROLLBACK command along with SET AUTOCOMMIT = TRUE. This ensures proper cleanup after a rollback operation. ```go // Start transaction (`autoCommit = false`) // Call `tx.Rollback()` // Verify `autoCommit` changes to `true` // Verify SQL commands executed: ROLLBACK, SET AUTOCOMMIT = TRUE ``` -------------------------------- ### Anti-Pattern: Concurrent Transactions on Single Connection Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Sharing a single database connection across multiple goroutines for concurrent transactions is an anti-pattern. This can lead to blocking as goroutines wait for the connection lock, preventing parallel execution. ```go // BAD: Attempting concurrent transactions on single connection db.SetMaxOpenConns(1) // Force single connection go func() { tx1, _ := db.Begin() // Acquires connection lock time.Sleep(1 * time.Second) tx1.Commit() }() go func() { tx2, _ := db.Begin() // BLOCKS waiting for connection lock tx2.Commit() }() ``` -------------------------------- ### Construct SQL Execution Error Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Creates an execution error when a SQL statement fails during transaction operations, such as setting autocommit to false. ```go func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) { _, _, err := c.runQuery(ctx, "SET AUTOCOMMIT = FALSE", nil) if err != nil { return nil, dbsqlerrint.NewExecutionError(ctx, dbsqlerrint.ErrTransactionBegin, err, nil) } // ... success path ... } ``` -------------------------------- ### DSN with Telemetry Disabled Source: https://github.com/databricks/databricks-sql-go/blob/main/README.md Explicitly disable telemetry collection using the DSN. ```go token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?enableTelemetry=false ``` -------------------------------- ### Commit() Method Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Commits the current transaction. It ensures autocommit is reset, executes the COMMIT statement, and updates the internal autocommit cache. It returns an error if the COMMIT operation fails. ```APIDOC ## Commit() Method ### Description Commits the current transaction. This method ensures that the autocommit state is reset, executes the `COMMIT` SQL statement, and updates the driver's internal cache for the autocommit state. It is designed to handle errors gracefully and reset autocommit even if the `COMMIT` statement itself fails. ### Method Signature ```go func (t *tx) Commit() error ``` ### Behavior 1. Uses `defer` to ensure autocommit reset in all cases. 2. Executes `COMMIT` statement via `conn.runQuery()`. 3. Executes `SET AUTOCOMMIT = TRUE` statement. 4. Updates `conn.autoCommit = true` in cache. 5. Returns an error if the `COMMIT` statement fails. ### Contracts - **MUST** reset autocommit even if `COMMIT` SQL fails. - **MUST** update cache even if `SET AUTOCOMMIT` fails. - Uses background context with correlation ID (to avoid cancellation). - Returns `DBExecutionError` wrapping the underlying error. - Idempotent: Can be called multiple times (though not recommended). ### Error Conditions - `COMMIT` statement fails (SQL error, network error). - Transaction is in an aborted state (server rejects `COMMIT`). - Context is cancelled (if using request context). ``` -------------------------------- ### Telemetry Request Export Format Source: https://github.com/databricks/databricks-sql-go/blob/main/telemetry/DESIGN.md This JSON structure represents the telemetry data format sent to the Databricks telemetry service. It includes upload time, items, and a list of protoLogs, where each log entry is a JSON-encoded TelemetryFrontendLog. ```json { "uploadTime": 1234567890000, "items": [], "protoLogs": [ "{\"frontend_log_event_id\":\"20240101120000-a1b2c3d4e5f6g7h8\",\"context\":{...},\"entry\":{\"sql_driver_log\":{\"session_id\":\"...\",\"sql_statement_id\":\"...\",\"operation_latency_ms\":42,\"sql_operation\":{\"chunk_details\":{\"total_chunks_iterated\":3},...}}}" ] } ``` -------------------------------- ### Reset Session Autocommit State Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Resets the autocommit state to TRUE if it was previously set to FALSE, ensuring the connection is clean before being reused from the pool. Logs a warning if manual AUTOCOMMIT commands are detected. ```go func (c *conn) ResetSession(ctx context.Context) error { log := logger.WithContext(c.id, "", "") // NEW: Check and reset autocommit state if !c.autoCommit { log.Warn().Msg("databricks: resetting autocommit in ResetSession") _, _, err := c.runQuery(ctx, "SET AUTOCOMMIT = TRUE", nil) if err != nil { log.Err(err).Msg("databricks: failed to reset autocommit") // Mark connection as bad so it's discarded return driver.ErrBadConn } c.autoCommit = true } return nil } ``` -------------------------------- ### Handle Aborted Transaction in Go Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md When a server aborts a transaction due to an invalid SQL statement, subsequent operations will fail. A rollback is required to clear the aborted state and allow new transactions. ```go tx, _ := db.Begin() _, err := tx.Exec("INVALID SQL") // Server aborts transaction // Try to continue _, err = tx.Exec("SELECT 1") // ERROR from server err = tx.Commit() // ERROR from server // Must rollback err = tx.Rollback() // Success, clears aborted state ``` -------------------------------- ### Rollback() Method Source: https://github.com/databricks/databricks-sql-go/blob/main/DESIGN_MULTI_STATEMENT_TRANSACTIONS.md Rolls back the current transaction. This method ensures autocommit is reset, executes the ROLLBACK statement, and updates the autocommit cache. It is designed to be forgiving and safe to call multiple times. ```APIDOC ## Rollback() Method ### Description Rolls back the current transaction. This method ensures that the autocommit state is reset, executes the `ROLLBACK` SQL statement, and updates the driver's internal cache for the autocommit state. It is designed to be forgiving, succeeding even if there is no active transaction on the server, and is safe to call multiple times. ### Method Signature ```go func (t *tx) Rollback() error ``` ### Behavior 1. Uses `defer` to ensure autocommit reset in all cases. 2. Executes `ROLLBACK` statement via `conn.runQuery()`. 3. Executes `SET AUTOCOMMIT = TRUE` statement. 4. Updates `conn.autoCommit = true` in cache. 5. Returns an error if `ROLLBACK` fails (rare). ### Contracts - **MUST** reset autocommit even if `ROLLBACK` SQL fails. - **MUST** update cache even if `SET AUTOCOMMIT` fails. - "Forgiving": `ROLLBACK` succeeds on the server even without an active transaction. - Uses background context with correlation ID. - Returns `DBExecutionError` wrapping the underlying error. - Idempotent: Safe to call multiple times. ### Error Conditions - Network failure communicating with the server. - Server-side error (extremely rare for `ROLLBACK`). ```