### Go Mod File Example Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md This is an example of what your go.mod file should look like after adding the Databricks SDK dependency. ```go module sample go 1.24 require github.com/databricks/databricks-sdk-go v0.9.0 // Indirect dependencies will go here. ``` -------------------------------- ### Databricks Token Authentication Example Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Demonstrates how to initialize a Databricks workspace client using Databricks token authentication. Requires Host and Token to be provided. ```go package main import ( "bufio" "context" "fmt" "os" "strings" "github.com/databricks/databricks-sdk-go" "github.com/databricks/databricks-sdk-go/config" ) func main() { // Perform Databricks token authentication for a Databricks workspace. w, err := databricks.NewWorkspaceClient(&databricks.Config{ Host: askFor("Host:"), // workspace url ``` -------------------------------- ### Zerolog Text Output Example Source: https://github.com/databricks/databricks-sdk-go/blob/main/examples/zerolog/README.md Illustrates the plain text output format when zerolog is configured for human-readable logging with the Databricks SDK. ```text 8:35AM TRC Loading config via environment global=true 8:35AM TRC Loading config via config-file global=true 8:35AM DBG Loading DEFAULT profile from [...]/.databrickscfg global=true 8:35AM TRC Attempting to configure auth: pat global=false 8:35AM DBG GET /api/2.0/preview/scim/v2/Me < HTTP/2.0 200 OK ... ``` -------------------------------- ### Google ID Authentication Example Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use this snippet to authenticate using Google ID. Ensure you have the necessary GCP credentials configured. ```go w, err := databricks.NewWorkspaceClient(&databricks.Config{ Host: askFor("Host:"), GoogleServiceAccount: askFor("Google Service Account:"), Credentials: config.GoogleDefaultCredentials{}, }) ``` -------------------------------- ### Install Library on Databricks Cluster and Wait Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Installs a specified library (e.g., a Python PyPI package) onto a Databricks cluster and waits for the update to complete. This method follows the convention of long-running operations, ensuring the library is installed before returning. It simplifies cluster library management by wrapping install and check operations. ```go err = w.Libraries.UpdateAndWait(ctx, libraries.Update{ ClusterId: clusterId, Install: []libraries.Library{ { Pypi: &libraries.PythonPyPiLibrary{ Package: "dbl-tempo", }, }, }, }) ``` -------------------------------- ### Zerolog JSON Output Example Source: https://github.com/databricks/databricks-sdk-go/blob/main/examples/zerolog/README.md Shows the typical JSON output format when zerolog is configured for structured logging with the Databricks SDK. ```text {"level":"trace","global":true,"time":"2023-03-10T08:34:56+01:00","message":"Loading config via environment"} {"level":"trace","global":true,"time":"2023-03-10T08:34:56+01:00","message":"Loading config via config-file"} {"level":"debug","global":true,"time":"2023-03-10T08:34:56+01:00","message":"Loading DEFAULT profile from [...]/.databrickscfg"} {"level":"trace","global":false,"time":"2023-03-10T08:34:56+01:00","message":"Attempting to configure auth: pat"} {"level":"debug","global":false,"time":"2023-03-10T08:34:57+01:00","message":"GET /api/2.0/preview/scim/v2/Me\n< HTTP/2.0 200 OK\n..."} ``` -------------------------------- ### Databricks PAT Authentication Example Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Demonstrates how to authenticate using a Personal Access Token (PAT) and retrieve the current user's display name. Requires user input for the PAT. ```go func main() { w, err := databricks.NewWorkspaceClient( "databricks.cloud.databricks.com", // Host config.WithPAT("", "", ""), // PAT ) if err != nil { panic(err) } me, err := w.CurrentUser.Me(context.Background()) if err != nil { panic(err) } fmt.Printf("Hello, my name is %s!\n", me.DisplayName) } ``` ```go func askFor(prompt string) string { var s string r := bufio.NewReader(os.Stdin) for { fmt.Fprint(os.Stdout, prompt+" ") s, _ = r.ReadString('\n') s = strings.TrimSpace(s) if s != "" { break } } return s } ``` -------------------------------- ### Build and Development Commands Source: https://github.com/databricks/databricks-sdk-go/blob/main/CLAUDE.md Common make commands for building, testing, formatting, linting, and managing dependencies for the Databricks SDK for Go. Ensure Go 1.24+ is installed. ```bash make build # go build ./... make test # Unit tests with coverage make fmt # goimports + gofmt make lint # staticcheck ./... make integration # Integration tests (requires cloud env) make vendor # Vendor dependencies make coverage # View HTML coverage report make codegen # Regenerate service/ from OpenAPI specs ``` -------------------------------- ### Text Output Example with slog Source: https://github.com/databricks/databricks-sdk-go/blob/main/examples/slog/README.md This example demonstrates the plain text output format from the slog logger when integrated with the Databricks SDK. It displays similar operational details as the JSON output but in a more human-readable, key-value pair format. ```text time=2023-03-10T08:33:44.219+01:00 level=DEBUG msg="Loading config via environment" global=true time=2023-03-10T08:33:44.219+01:00 level=DEBUG msg="Loading config via config-file" global=true time=2023-03-10T08:33:44.219+01:00 level=DEBUG msg="Loading DEFAULT profile from [...]/.databrickscfg" global=true time=2023-03-10T08:33:44.220+01:00 level=DEBUG msg="Attempting to configure auth: pat" global=false time=2023-03-10T08:33:45.077+01:00 level=DEBUG msg="GET /api/2.0/preview/scim/v2/Me\n< HTTP/2.0 200 OK\n ``` -------------------------------- ### JSON Output Example with slog Source: https://github.com/databricks/databricks-sdk-go/blob/main/examples/slog/README.md This example shows the typical JSON output generated by the slog logger when used with the Databricks SDK. It includes timestamps, log levels, and messages detailing configuration loading and authentication attempts. ```text {"time":"2023-03-10T08:33:11.318122+01:00","level":"DEBUG","msg":"Loading config via environment","global":true} {"time":"2023-03-10T08:33:11.318411+01:00","level":"DEBUG","msg":"Loading config via config-file","global":true} {"time":"2023-03-10T08:33:11.318552+01:00","level":"DEBUG","msg":"Loading DEFAULT profile from [...]/.databrickscfg","global":true} {"time":"2023-03-10T08:33:11.31862+01:00","level":"DEBUG","msg":"Attempting to configure auth: pat","global":false} {"time":"2023-03-10T08:33:12.170168+01:00","level":"DEBUG","msg":"GET /api/2.0/preview/scim/v2/Me\n< HTTP/2.0 200 OK\n ``` -------------------------------- ### Development Build and Test Commands Source: https://github.com/databricks/databricks-sdk-go/blob/main/AGENTS.md Common make targets for building, testing, formatting, linting, and generating code for the Databricks SDK for Go. Ensure Go 1.24+ is installed. ```bash make build # go build ./... make test # Unit tests with coverage make fmt # goimports + gofmt make lint # staticcheck ./... make integration # Integration tests (requires cloud env) make vendor # Vendor dependencies make coverage # View HTML coverage report make codegen # Regenerate service/ from OpenAPI specs ``` -------------------------------- ### Mock Databricks Workspace and Account Clients in Go Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use the SDK's mock implementation to test code interacting with Databricks workspace and account services. This example demonstrates mocking cluster listing and user listing operations. ```go package my_test import ( "context" "testing" "github.com/databricks/databricks-sdk-go/experimental/mocks" "github.com/databricks/databricks-sdk-go/listing" "github.com/databricks/databricks-sdk-go/qa/poll" "github.com/databricks/databricks-sdk-go/service/compute" "github.com/databricks/databricks-sdk-go/service/iam" "github.com/databricks/databricks-sdk-go/service/sql" "github.com/stretchr/testify/mock" ) func TestDatabricksSDK(t *testing.T) { ctx := context.Background() w := mocks.NewMockWorkspaceClient(t) w.GetMockClustersAPI().EXPECT().ListAll( ctx, mock.AnythingOfType("compute.ListClustersRequest"), ).Return( []compute.ClusterDetails{ {ClusterName: "test-cluster-1"}, {ClusterName: "test-cluster-2"}, }, nil) // You can also mock the AccountClient as follows. a := mocks.NewMockAccountClient(t) a.GetMockAccountUsersAPI().EXPECT().ListAll( ctx, mock.AnythingOfType("iam.ListAccountUsersRequest"), ).Return( []iam.User{ {DisplayName: "test-user-1"}, {DisplayName: "test-user-2"}, }, nil) } ``` -------------------------------- ### Retrieve All Paginated Results Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md The `XxxAll` methods abstract away the complexity of API pagination, allowing you to retrieve all results of a certain entity type in a single, high-level interface. This example shows how to list all repositories. ```go all, err := w.Repos.ListAll(ctx, repos.List{}) if err != nil { return fmt.Errorf("list repos: %w", err) } for _, repo := range all { println(repo.Path) } ``` -------------------------------- ### Override .databrickscfg Profile Example Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use this snippet to specify a custom profile from your .databrickscfg file for authentication. This overrides the default 'DEFAULT' profile. ```go w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{ Profile: "MYPROFILE", })) // Now call the Databricks workspace APIs as desired... ``` -------------------------------- ### Mock Iterators and Pollers in Go Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Simplify mocking of listing operations using `listing.SliceIterator` and long-running operations using `poll.Simple`. This example shows how to mock user listings and warehouse edits with polling. ```go func TestDatabricksSDK_helpers(t *testing.T) { // To mock iterators, you can provide the items to iterate over with // *listing.SliceIterator. iterator := listing.SliceIterator[iam.User]([]iam.User{ {DisplayName: "test-user-1"}, {DisplayName: "test-user-2"}, }) a.GetMockAccountUsersAPI().EXPECT().List( ctx, mock.AnythingOfType("iam.ListAccountUsersRequest"), ).Return(&iterator) // To mock Wait* structures, you can stub out the Poll field. getResponse := sql.GetWarehouseResponse{ Id: "abc", } wait := sql.WaitGetWarehouseRunning[struct{}]{ Poll: poll.Simple(getResponse), } w.GetMockWarehousesAPI().EXPECT().Edit(mock.Anything, sql.EditWarehouseRequest{}).Return(&wait, nil) } ``` -------------------------------- ### Get Databricks Object by Name Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md The `GetByName` utility methods simplify retrieving workspace objects by their names, which can be more intuitive than using identifiers. Note that these helpers return an error if duplicate names are detected. ```go repo, err := w.Repos.GetByPath(ctx, path) if err != nil { return err } return w.Repos.Update(ctx, repos.UpdateRepo{ RepoId: repo.Id, Branch: tag, }) ``` -------------------------------- ### Initialize Go Module Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Run this command in your project directory to create a go.mod file and track dependencies. ```bash go mod init sample ``` -------------------------------- ### Set Product in User Agent Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use `WithProduct()` to define the name and version of your product. Subsequent calls overwrite previous ones. ```go useragent.WithProduct("databricks-example-product", "1.2.0") ``` -------------------------------- ### Run Go Application Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Execute your Go program using the `go run` command. This assumes your main file is named `main.go`. ```bash go run main.go ``` -------------------------------- ### Vendor Go Module Dependencies Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md This command copies all of the packages needed to support builds and tests into the `vendor` directory. ```bash go mod vendor ``` -------------------------------- ### Add Partner to User Agent Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use `WithPartner()` to attribute requests to a specific partner. Multiple partners can be registered. ```go useragent.WithPartner("partner-abc") useragent.WithPartner("partner-xyz") ``` -------------------------------- ### Add Extra User Agent Information Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use `WithUserAgentExtra()` to add arbitrary key-value pairs to the User-Agent header. Multiple values per key are allowed. ```go useragent.WithUserAgentExtra("key", "value") ``` -------------------------------- ### Implement Custom Credentials Provider for Databricks SDK Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Create a custom credentials provider by implementing the `Credentials` interface to gain deeper control over authentication. This allows you to define how authentication tokens are generated and applied to requests. ```go type CustomCredentials struct {} func (c *CustomCredentials) Name() string { return "custom" } func (c *CustomCredentials) Configure(ctx context.Context, cfg *config.Config) (func(*http.Request) error, error) { return func(r *http.Request) error { token := "..." r.Header.Set("Authorization", fmt.Sprintf("Bearer %s", token)) return nil }, nil } func main() { w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{ Credentials: &CustomCredentials{}, })) // .. ``` -------------------------------- ### Configure Unified Host for Databricks Account and Workspace Clients Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md This configuration allows both AccountClient and WorkspaceClient to use the same host and profile. It requires 'AccountID' and 'WorkspaceID' to be present, either explicitly set or auto-discovered. ```ini # .databrickscfg [unified] host = https://mycompany.databricks.com account_id = 00000000-0000-0000-0000-000000000000 workspace_id = 1234567890 ``` ```go // Both clients share the same host and profile w, _ := databricks.NewWorkspaceClient(&databricks.Config{Profile: "unified"}) a, _ := databricks.NewAccountClient(&databricks.Config{Profile: "unified"}) // A WorkspaceClient for a different workspace under the same host and account w, _ = databricks.NewWorkspaceClient(&databricks.Config{ Profile: "unified", WorkspaceID: "2345678901", }) ``` -------------------------------- ### List Databricks Clusters in Go Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md This Go program connects to your Databricks workspace and prints the names of all clusters. Ensure you have set up Databricks authentication. ```go package main import ( "context" "github.com/databricks/databricks-sdk-go" "github.com/databricks/databricks-sdk-go/service/compute" ) func main() { w := databricks.Must(databricks.NewWorkspaceClient()) all, err := w.Clusters.ListAll(context.Background(), compute.ListClustersRequest{}) if err != nil { panic(err) } for _, c := range all { println(c.ClusterName) } } ``` -------------------------------- ### Select Databricks Runtime and Node Type Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use selector methods to easily pick the latest Databricks Runtime or the smallest suitable node type based on specified criteria like local disk availability. This simplifies multi-cloud development. ```go // Fetch the list of spark runtime versions. sparkVersions, err := w.Clusters.SparkVersions(ctx) if err != nil { return err } // Select the latest LTS version. latestLTS, err := sparkVersions.Select(clusters.SparkVersionRequest{ Latest: true, LongTermSupport: true, }) if err != nil { return err } // Fetch the list of available node types. nodeTypes, err := w.Clusters.ListNodeTypes(ctx) if err != nil { return err } // Select the smallest node type ID. smallestWithDisk, err := nodeTypes.Smallest(clusters.NodeTypeRequest{ LocalDisk: true, }) if err != nil { return err } // Create the cluster and wait for it to start properly. runningCluster, err := w.Clusters.CreateAndWait(ctx, clusters.CreateCluster{ ClusterName: clusterName, SparkVersion: latestLTS, NodeTypeId: smallestWithDisk, AutoterminationMinutes: 15, NumWorkers: 1, }) ``` -------------------------------- ### Tidy Go Module Dependencies Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Run this command after adding or removing Go packages to ensure your go.mod and go.sum files are consistent. ```bash go mod tidy ``` -------------------------------- ### Upload File to DBFS using io.Reader Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Integrate with standard Go `io` interfaces to upload files to DBFS. The `w.Dbfs.Open` function returns a `dbfs.Handle` compatible with `io.Reader` and `io.Writer`. ```go upload, _ := os.Open("/path/to/local/file.ext") remote, _ := w.Dbfs.Open(ctx, "/path/to/remote/file", dbfs.FileModeWrite|dbfs.FileModeOverwrite) _, _ = io.Copy(remote, upload) _ = remote.Close() ``` -------------------------------- ### Download File from DBFS to io.Writer Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use standard Go `io` interfaces to download files from DBFS. The `w.Dbfs.Open` function returns a `dbfs.Handle` compatible with `io.Reader` and `io.Writer`. ```go download, _ := os.Create("/path/to/local") remote, _ := w.Dbfs.Open(ctx, "/path/to/remote/file", dbfs.FileModeRead) _, _ = io.Copy(download, remote) ``` -------------------------------- ### Execute Command on Databricks Cluster Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Executes a command (e.g., Python script) on a specified Databricks cluster and retrieves the text output. This wrapper simplifies command execution and handles potential failures. The execution timeout is fixed at 20 minutes and cannot be overridden. ```go res := w.CommandExecutor.Execute(ctx, clusterId, "python", "print(1)") if res.Failed() { return fmt.Errorf("command failed: %w", res.Err()) } println(res.Text()) ``` -------------------------------- ### Databricks SDK Default Authentication Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md This snippet shows how to initialize a Databricks workspace client using the default authentication flow, which respects configuration profiles and environment variables. ```go w := databricks.Must(databricks.NewWorkspaceClient()) w./*press TAB for autocompletion*/ ``` -------------------------------- ### Configure Debug Logging Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Override `logger.DefaultLogger` to enable debug-level logging by setting `Level: logger.LevelDebug` in `SimpleLogger`. ```go import "github.com/databricks/databricks-sdk-go/logger" func init() { logger.DefaultLogger = &logger.SimpleLogger{ Level: logger.LevelDebug, } } ``` -------------------------------- ### Add Databricks SDK Dependency Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use this command to add the Databricks SDK for Go package to your project's dependencies. ```bash go mod edit -require github.com/databricks/databricks-sdk-go@latest ``` -------------------------------- ### Upload File to DBFS from Byte Slice Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Conveniently upload a file to DBFS directly from a byte slice using the `w.Dbfs.WriteFile` function. ```go err := w.Dbfs.WriteFile(ctx, "/path/to/remote/file", []byte("Hello world!")) ``` -------------------------------- ### Check for Specific Predefined Errors Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use `errors.Is()` with predefined errors like `databricks.ErrResourceDoesNotExist` to check for specific error conditions. ```go c, err := w.Clusters.GetByClusterId(ctx, "12345") if errors.Is(err, databricks.ErrResourceDoesNotExist) {... } ``` -------------------------------- ### Create Databricks Cluster and Wait for Running State Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Creates a Databricks cluster and waits until it reaches the 'RUNNING' state. A custom timeout of 10 minutes is applied. This method returns cluster information upon successful completion. ```go clusterInfo, err = w.Clusters.CreateAndWait(ctx, clusters.CreateCluster{ ClusterName: "Created cluster", SparkVersion: latestLTS, NodeTypeId: smallestWithDisk, AutoterminationMinutes: 10, NumWorkers: 1, }, retries.Timeout[clusters.ClusterInfo](10*time.Minute)) ``` -------------------------------- ### Enable Debug HTTP Headers in Databricks Config Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md To enable debugging of HTTP headers, set the `DebugHeaders` field to `true` in the `databricks.Config`. This is useful for inspecting sensitive data like access tokens, but should be used with caution. ```go w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{ DebugHeaders: true, })) // Now call the Databricks workspace APIs as desired... ``` -------------------------------- ### Create Databricks Workspace Client with Azure Client Secret Auth Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use this configuration to authenticate to Databricks using Azure Active Directory service principal credentials. Ensure you have the necessary Azure Resource ID, Tenant ID, Client ID, and Client Secret. ```go w, err := databricks.NewWorkspaceClient(&databricks.Config{ Host: askFor("Host:"), AzureResourceID: askFor("Azure Resource ID:"), AzureTenantID: askFor("AAD Tenant ID:"), AzureClientID: askFor("AAD Client ID:"), AzureClientSecret: askFor("AAD Client Secret:"), Credentials: config.AzureClientSecretCredentials{}, }) ``` -------------------------------- ### Automatically Sign Commits with Git Source: https://github.com/databricks/databricks-sdk-go/blob/main/CONTRIBUTING.md Configure your git user name and email to automatically sign commits using the '-s' flag. ```git git commit -s -m "Your commit message" ``` -------------------------------- ### Force Azure CLI Authentication in Databricks SDK for Go Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use this to force a specific authentication method like Azure CLI, bypassing auto-detection. Ensure the 'AuthType' and 'Cloud' fields are correctly set in the Config. ```go // Force Azure CLI authentication — skip all other methods w, err := databricks.NewWorkspaceClient(&databricks.Config{ Host: "https://mycompany.databricks.com", AuthType: "azure-cli", Cloud: "Azure", }) ``` -------------------------------- ### Download File from DBFS into Byte Slice Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Read the content of a DBFS file directly into a byte slice using the `w.Dbfs.ReadFile` function. ```go buf, err := w.Dbfs.ReadFile(ctx, "/path/to/remote/file") ``` -------------------------------- ### Track Intermediate State of Long-Running Operations Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Use the `func(i *retries.Info[Zzz])` option to monitor the state of operations that take time to complete. This is useful for long-running tasks where you need to observe progress. ```go clusterInfo, err = w.Clusters.CreateAndWait(ctx, clusters.CreateCluster{ // ... }, func(i *retries.Info[clusters.ClusterInfo]) { updateIntermediateState(i.Info.StateMessage) }) ``` -------------------------------- ### Sign Off Commit Message Source: https://github.com/databricks/databricks-sdk-go/blob/main/CONTRIBUTING.md Add this line to your commit message to certify compliance with the Developer Certificate of Origin. ```git Signed-off-by: Joe Smith ``` -------------------------------- ### Handle API Errors Source: https://github.com/databricks/databricks-sdk-go/blob/main/README.md Inspect API errors by asserting the error as `apierr.APIError` to access error code, message, status code, and details. ```go _, err := w.Clusters.Create(ctx, compute.CreateCluster{...}) if e, ok := err.(*apierr.APIError); ok { fmt.Printf("Error code: %s\n", e.ErrorCode) fmt.Printf("Error message: %s\n", e.Message) fmt.Printf("Status code: %s\n", e.StatusCode) fmt.Printf("Error details: %v\n", e.Details) } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.