### Multi-Host and Failover Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example demonstrating how to configure the driver for multi-host connections and failover. ```go multi_host.go ``` -------------------------------- ### Client Information Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example showing how to retrieve client information from the ClickHouse server. ```go client_info.go ``` -------------------------------- ### Qubit Data Type Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example showing the handling of the QUBIT data type. ```go qbit.go ``` -------------------------------- ### Query Parameters Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example demonstrating the use of query parameters for safe and efficient queries. ```go query_parameters.go ``` -------------------------------- ### Start, Test, and Stop ClickHouse with Docker Source: https://github.com/clickhouse/clickhouse-go/blob/main/CONTRIBUTING.md Use these make targets to manage a local ClickHouse instance for development and testing. Ensure Docker is installed and running. ```bash make up # start ClickHouse via Docker Compose make test # run all tests with -race make down # stop and remove containers ``` -------------------------------- ### Settings Map Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/7-types-and-errors.md Illustrates how to define server-side settings for queries using a map. ```go settings := clickhouse.Settings{ "max_execution_time": 60, "enable_http_compression": 1, "max_rows_to_read": 1000000, } ``` -------------------------------- ### Install ClickHouse Go Driver Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Install the ClickHouse Go driver using the go get command. ```sh go get github.com/ClickHouse/clickhouse-go/v2 ``` -------------------------------- ### ClientInfo Type Definition and Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/7-types-and-errors.md Defines the structure for client metadata and provides an example of its usage. ```go info := clickhouse.ClientInfo{ Products: []struct{ Name string Version string }{ {Name: "myapp", Version: "1.2.3"}, }, Comment: "batch processing", } ``` -------------------------------- ### Columnar Insert Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example demonstrating how to perform inserts using a columnar format. ```go columnar_insert.go ``` -------------------------------- ### Scan Struct Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of scanning query results directly into Go structs. ```go scan_struct.go ``` -------------------------------- ### JSON Structs Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example demonstrating how to handle JSON data when it's represented as Go structs. ```go json_structs.go ``` -------------------------------- ### HTTP Asynchronous Insert Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Demonstrates asynchronous data insertion using the HTTP protocol. ```go async_http.go ``` -------------------------------- ### Native Asynchronous Insert Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Demonstrates asynchronous data insertion using the native ClickHouse protocol. ```go async_native.go ``` -------------------------------- ### BFloat16 Data Type Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example demonstrating the handling of the BFloat16 data type. ```go bfloat16.go ``` -------------------------------- ### Batch Insert with Structs Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of performing a batch insert where data is provided as Go structs. ```go append_struct.go ``` -------------------------------- ### Combined Context Options Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/4-context-options.md Demonstrates how to build a context with multiple options for a ClickHouse query. ```APIDOC ## Combined Example ### Description Build context with multiple options for ClickHouse operations. ### Code Example ```go package main import ( "context" "fmt" "log" "time" "github.com/ClickHouse/clickhouse-go/v2" ) func main() { conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, }) if err != nil { log.Fatal(err) } defer conn.Close() // Build context with multiple options ctx := clickhouse.Context( context.Background(), clickhouse.WithQueryID("daily-report-123"), clickhouse.WithSettings(clickhouse.Settings{ "max_execution_time": 300, "max_rows_to_read": 100000000, }), clickhouse.WithQuotaKey("analytics-team"), clickhouse.WithProgress(func(p *clickhouse.Progress) { fmt.Printf("Progress: %d rows, %.2f MB\n", p.ReadRows, float64(p.ReadBytes)/1024/1024) }), clickhouse.WithProfileInfo(func(info *clickhouse.ProfileInfo) { fmt.Printf("Execution completed. Total rows: %d\n", info.Rows) }), ) rows, err := conn.Query(ctx, "SELECT * FROM events WHERE date >= ?", "2024-01-01") if err != nil { log.Fatal(err) } defer rows.Close() for rows.Next() { var id int var eventTime time.Time var data string if err := rows.Scan(&id, &eventTime, &data); err != nil { log.Fatal(err) } fmt.Printf("ID: %d, Time: %v, Data: %s\n", id, eventTime, data) } if err := rows.Err(); err != nil { log.Fatal(err) } } ``` ``` -------------------------------- ### Dynamic Data Type Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example showing how to work with dynamic data types in ClickHouse. ```go dynamic.go ``` -------------------------------- ### Geo Data Type Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example demonstrating the handling of geographical data types. ```go geo.go ``` -------------------------------- ### Variant Data Type Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example demonstrating the use of the VARIANT data type. ```go variant.go ``` -------------------------------- ### Ephemeral Columns (Native) Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example showing the use of ephemeral columns with the native ClickHouse protocol. ```go ephemeral_native.go ``` -------------------------------- ### Complete Application with ClickHouse Go Driver Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md This example shows a full application workflow: connecting to ClickHouse, configuring connection pools, creating a table, inserting data in a transaction, and querying data with specific settings. Ensure ClickHouse is running and accessible at the specified address. ```go package main import ( "context" "database/sql" "fmt" "log" "time" "github.com/ClickHouse/clickhouse-go/v2" ) func main() { // Open database db := clickhouse.OpenDB(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, Auth: clickhouse.Auth{ Database: "default", Username: "default", Password: "", }, }) defer db.Close() // Configure pool db.SetMaxOpenConns(10) db.SetMaxIdleConns(5) db.SetConnMaxLifetime(time.Hour) // Create table _, err := db.ExecContext(context.Background(), ` CREATE TABLE IF NOT EXISTS users ( id UInt32, name String, email String, created_at DateTime DEFAULT now() ) ENGINE = MergeTree() ORDER BY id `) if err != nil { log.Fatal(err) } // Insert data via transaction ctx := clickhouse.Context(context.Background(), clickhouse.WithQueryID("batch-insert-001"), ) tx, err := db.BeginTx(ctx, nil) if err != nil { log.Fatal(err) } defer tx.Rollback() stmt, err := tx.PrepareContext(ctx, "INSERT INTO users (id, name, email) VALUES (?, ?, ?)") if err != nil { log.Fatal(err) } defer stmt.Close() for i := 1; i <= 100; i++ { _, err = stmt.ExecContext(ctx, i, fmt.Sprintf("user%d", i), fmt.Sprintf("user%d@example.com", i)) if err != nil { log.Fatal(err) } } if err = tx.Commit(); err != nil { log.Fatal(err) } // Query data queryCtx := clickhouse.Context(context.Background(), clickhouse.WithQueryID("select-query-001"), clickhouse.WithSettings(clickhouse.Settings{ "max_execution_time": 30, }), ) rows, err := db.QueryContext(queryCtx, "SELECT id, name, email FROM users WHERE id > ? LIMIT 10", 50) if err != nil { log.Fatal(err) } defer rows.Close() for rows.Next() { var id int var name, email string if err := rows.Scan(&id, &name, &email); err != nil { log.Fatal(err) } fmt.Printf("ID: %d, Name: %s, Email: %s\n", id, name, email) } if err = rows.Err(); err != nil { log.Fatal(err) } } ``` -------------------------------- ### Native API Batch Insert Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of performing a batch insert using the native ClickHouse API. ```go batch.go ``` -------------------------------- ### Configure Connection Options for Timeout Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/8-connection-management.md Example of how to configure connection options to increase `MaxOpenConns` or `DialTimeout` to mitigate acquire timeout errors. ```go conn, err := clickhouse.Open(&clickhouse.Options{ MaxOpenConns: 50, // Increase from default DialTimeout: 60 * time.Second, // Or increase timeout }) ``` -------------------------------- ### Bind Parameters Example (Deprecated) Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of using bind parameters for queries. Note: This is deprecated in favor of native query parameters. ```go bind.go ``` -------------------------------- ### Ephemeral Columns (HTTP) Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example showing the use of ephemeral columns with the HTTP ClickHouse protocol. ```go ephemeral_http.go ``` -------------------------------- ### PrepareContext Example for Batch Inserts Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Prepares a statement for repeated execution, useful for batch operations. Ensure to close the statement after use. ```go stmt, err := db.PrepareContext(context.Background(), "INSERT INTO users (id, name, email) VALUES (?, ?, ?)") if err != nil { log.Fatal(err) } deffer stmt.Close() for _, user := range users { if _, err := stmt.ExecContext(context.Background(), user.ID, user.Name, user.Email); err != nil { log.Fatal(err) } } ``` -------------------------------- ### TLS Configuration with OpenDB (Default) Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of setting up a secure TLS connection using clickhouse.OpenDB with default secure settings. This creates a minimal tls.Config. ```go conn := clickhouse.OpenDB(&clickhouse.Options{ ... TLS: &tls.Config{ InsecureSkipVerify: false } ... }) ``` -------------------------------- ### SetMaxOpenConns Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Sets the maximum number of concurrently open database connections. This overrides the MaxOpenConns option set during database initialization. ```go db.SetMaxOpenConns(25) ``` -------------------------------- ### Native Protocol DSN Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of a DSN string for connecting to ClickHouse using the native protocol. Includes host, port, database, and various connection timeouts. ```sh clickhouse://username:password@host1:9000,host2:9000/database?dial_timeout=200ms&read_timeout=30s&max_execution_time=60 ``` -------------------------------- ### OpError Handling Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/7-types-and-errors.md Demonstrates how to check for and extract specific `OpError` details when a query fails, providing context about the operation and column involved. ```go rows, err := conn.Query(ctx, "SELECT * FROM t") if err != nil { var opErr *clickhouse.OpError if errors.As(err, &opErr) { fmt.Printf("Failed %s on column %s: %v\n", opErr.Op, opErr.ColumnName, opErr.Err) } } ``` -------------------------------- ### Native API Batch Insert with Connection Release Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of a batch insert using the native API, including releasing the connection. ```go batch_release_connection.go ``` -------------------------------- ### Native API Reference - Open Function Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/COMPLETION_SUMMARY.txt Documentation for the `Open()` function in the native API, including its signature, parameters, return values, and usage examples. ```APIDOC ## Open() ### Description Opens a new ClickHouse connection using the native protocol. ### Signature `func Open(ctx context.Context, cfg *clickhouse.Config, opts ...clickhouse.Option) (clickhouse.Conn, error)` ### Parameters - `ctx` (context.Context) - The context for the operation. - `cfg` (*clickhouse.Config) - Configuration for the ClickHouse connection. - `opts` (...clickhouse.Option) - Optional configuration settings. ### Returns - `clickhouse.Conn` - A connection object if successful. - `error` - An error if the connection fails. ``` -------------------------------- ### PingContext Example for Connectivity Check Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Verifies connectivity to the database using the configured DialTimeout. Handles potential connection errors. ```go if err := db.PingContext(context.Background()); err != nil { log.Fatal("Database unreachable:", err) } fmt.Println("Connected to ClickHouse") ``` -------------------------------- ### QueryContext Example with ClickHouse Options Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Executes a SELECT query with context supporting ClickHouse options like query ID and settings. Ensure to close rows and handle scan errors. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithQueryID("report-123"), clickhouse.WithSettings(clickhouse.Settings{ "max_execution_time": 60, }), ) rows, err := db.QueryContext(ctx, "SELECT id, name FROM users WHERE created_at > ?", "2024-01-01") if err != nil { log.Fatal(err) } deffer rows.Close() for rows.Next() { var id int var name string if err := rows.Scan(&id, &name); err != nil { log.Fatal(err) } fmt.Println(id, name) } ``` -------------------------------- ### Native TCP DSN with Authentication Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/2-options-configuration.md Example of a DSN string for native TCP connections with username, password, host, port, database, and connection options. ```text clickhouse://user:pass@localhost:9000/mydb?dial_timeout=30s&max_open_conns=10 ``` -------------------------------- ### Custom Dial Strategy for Failover Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/8-connection-management.md Implement custom server selection logic for failover or advanced routing. This example demonstrates a weighted server selection strategy to prioritize primary servers and fall back to backups. ```go type weightedServer struct { addr string weight int } dialStrategy := func(ctx context.Context, connID int, opt *Options, dial clickhouse.Dial) (clickhouse.DialResult, error) { servers := []weightedServer{ {"primary:9000", 10}, // Preferred {"backup:9000", 1}, // Fallback } for i := 0; i < len(servers); i++ { // Pick server based on weight idx := (connID + i) % len(servers) result, err := dial(ctx, servers[idx].addr, opt) if err == nil { return result, nil } } return clickhouse.DialResult{}, errors.New("all servers down") } conn, err := clickhouse.Open(&clickhouse.Options{ DialStrategy: dialStrategy, }) ``` -------------------------------- ### Stats Example for Pool Statistics Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Retrieves and prints database connection pool statistics, including open, in-use, and idle connections, as well as wait counts and durations. ```go stats := db.Stats() fmt.Printf("Open: %d, In Use: %d, Idle: %d\n", stats.OpenConnections, stats.InUse, stats.Idle) ``` -------------------------------- ### Multiple Hosts DSN with Round-Robin and Server Setting Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/2-options-configuration.md Example of a DSN string connecting to multiple hosts using round-robin strategy, specifying a ClickHouse server setting. ```text clickhouse://default@host1:9000,host2:9000,host3:9000/default?connection_open_strategy=round_robin&max_execution_time=60 ``` -------------------------------- ### Configure ClickHouse Go Client with Multiple Context Options Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/4-context-options.md Demonstrates building a context with multiple options for advanced client configuration, including query ID, settings, quota key, and progress/profile callbacks. ```go package main import ( "context" "fmt" "log" "time" "github.com/ClickHouse/clickhouse-go/v2" ) func main() { conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, }) if err != nil { log.Fatal(err) } defer conn.Close() // Build context with multiple options ctx := clickhouse.Context( context.Background(), clickhouse.WithQueryID("daily-report-123"), clickhouse.WithSettings(clickhouse.Settings{ "max_execution_time": 300, "max_rows_to_read": 100000000, }), clickhouse.WithQuotaKey("analytics-team"), clickhouse.WithProgress(func(p *clickhouse.Progress) { fmt.Printf("Progress: %d rows, %.2f MB\n", p.ReadRows, float64(p.ReadBytes)/1024/1024) }), clickhouse.WithProfileInfo(func(info *clickhouse.ProfileInfo) { fmt.Printf("Execution completed. Total rows: %d\n", info.Rows) }), ) rows, err := conn.Query(ctx, "SELECT * FROM events WHERE date >= ?", "2024-01-01") if err != nil { log.Fatal(err) } defer rows.Close() for rows.Next() { var id int var eventTime time.Time var data string if err := rows.Scan(&id, &eventTime, &data); err != nil { log.Fatal(err) } fmt.Printf("ID: %d, Time: %v, Data: %s\n", id, eventTime, data) } if err := rows.Err(); err != nil { log.Fatal(err) } } ``` -------------------------------- ### Standard Library API - sql.Open with DSN Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/COMPLETION_SUMMARY.txt Demonstrates how to use Go's standard `sql.Open()` function with a ClickHouse DSN. ```APIDOC ## sql.Open() with DSN ### Description Initializes a ClickHouse connection using the standard `sql.Open` function by providing a Data Source Name (DSN). ### Usage ```go db, err := sql.Open("clickhouse", "clickhouse://user:password@host:port/database?dial_timeout=...&compression=...&format=...") if err != nil { // handle error } // Use db object for standard SQL operations ``` ### Parameters - `driverName` (string) - Should be "clickhouse". - `dataSourceName` (string) - The connection string (DSN) specifying connection details, including host, port, user, password, database, and various options like timeouts and compression. ``` -------------------------------- ### Execute Async Insert (Deprecated vs. Recommended) Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/3-native-api-reference.md Demonstrates the deprecated `AsyncInsert` method and the recommended `Exec` with `WithAsync` context option for asynchronous inserts. ```go err := conn.AsyncInsert(context.Background(), "INSERT INTO events (id, data) VALUES (?, ?)", false, // wait=false 1, "data", ) ``` ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithAsync(false), // wait=false ) err := conn.Exec(ctx, "INSERT INTO events (id, data) VALUES (?, ?)", 1, "data") ``` -------------------------------- ### Configure Basic Authentication Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/2-options-configuration.md Use the Auth struct to provide username, password, and database for basic authentication. ```go type Auth struct { Database string Username string Password string } ``` ```go conn, err := clickhouse.Open(&clickhouse.Options{ Auth: clickhouse.Auth{ Database: "mydb", Username: "myuser", Password: "mypass", }, }) ``` -------------------------------- ### Basic Query and Scan in Go Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/10-examples-and-patterns.md Demonstrates how to establish a connection, execute a basic SELECT query, and scan the results into Go variables. Ensure ClickHouse is running and accessible at the specified address. ```go package main import ( "context" "fmt" "log" "github.com/ClickHouse/clickhouse-go/v2" ) func main() { conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, Auth: clickhouse.Auth{ Database: "default", Username: "default", Password: "", }, }) if err != nil { log.Fatal(err) } defer conn.Close() rows, err := conn.Query(context.Background(), "SELECT * FROM users LIMIT 10") if err != nil { log.Fatal(err) } defer rows.Close() for rows.Next() { var id int var name string if err := rows.Scan(&id, &name); err != nil { log.Fatal(err) } fmt.Printf("ID: %d, Name: %s\n", id, name) } if err := rows.Err(); err != nil { log.Fatal(err) } } ``` -------------------------------- ### Query with Monitoring and Settings in Go Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/10-examples-and-patterns.md Demonstrates how to add query monitoring (progress, profile info) and apply specific ClickHouse settings to a query. Use clickhouse.Context to pass these options. ```go package main import ( "context" "fmt" "log" "github.com/ClickHouse/clickhouse-go/v2" ) func main() { conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, }) if err != nil { log.Fatal(err) } defer conn.Close() ctx := clickhouse.Context(context.Background(), clickhouse.WithQueryID("analysis-2024-01-15"), clickhouse.WithProgress(func(p *clickhouse.Progress) { fmt.Printf("Progress: %d rows (%.2f MB)\n", p.ReadRows, float64(p.ReadBytes)/1024/1024) }), clickhouse.WithProfileInfo(func(info *clickhouse.ProfileInfo) { fmt.Printf("Completed: %d rows, %d bytes, %v ns\n", info.Rows, info.Bytes, info.ElapsedNanoseconds) }), clickhouse.WithSettings(clickhouse.Settings{ "max_execution_time": 300, }), ) rows, err := conn.Query(ctx, "SELECT COUNT(*) FROM events WHERE date >= ?", "2024-01-01") if err != nil { log.Fatal(err) } defer rows.Close() for rows.Next() { var count int rows.Scan(&count) fmt.Printf("Total events: %d\n", count) } } ``` -------------------------------- ### HTTP Protocol DSN Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of a DSN string for connecting to ClickHouse using the HTTP protocol. Useful for proxying traffic or using load balancers. ```sh http://host1:8123,host2:8123/database?dial_timeout=200ms&max_execution_time=60 ``` -------------------------------- ### Batch Insert with Progress Tracking in Go Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/10-examples-and-patterns.md Shows how to efficiently insert a large number of records in batches, with progress updates printed to the console. Configure MaxOpenConns and BlockBufferSize for optimal performance. ```go package main import ( "context" "fmt" "log" "github.com/ClickHouse/clickhouse-go/v2" "time" ) func main() { conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, MaxOpenConns: 20, BlockBufferSize: 10, }) if err != nil { log.Fatal(err) } defer conn.Close() batch, err := conn.PrepareBatch( context.Background(), "INSERT INTO events (id, timestamp, message)", ) if err != nil { log.Fatal(err) } defer batch.Close() for i := 1; i <= 100000; i++ { if err := batch.Append( i, time.Now(), fmt.Sprintf("event_%d", i), ); err != nil { log.Fatal(err) } if i%10000 == 0 { fmt.Printf("Progress: %d rows\n", i) batch.Flush() } } if err := batch.Send(); err != nil { log.Fatal(err) } fmt.Println("All events inserted") } ``` -------------------------------- ### External Tables Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/COMPLETION_SUMMARY.txt Provides guidance and examples on how to use external tables with the ClickHouse Go driver. ```APIDOC ## External Tables This section explains how to define and use external tables within your ClickHouse queries via the Go driver. ### Usage External tables allow you to query data that is not permanently stored in ClickHouse, such as data from files or other sources, by defining them on-the-fly for a specific query. ### Example (Conceptual) ```go // Conceptual example for using external tables // Assuming a function like this exists: // db.Query("SELECT * FROM external_table", clickhouse.WithExternalTable("my_data.csv", "CSV", "col1 String, col2 Int32")) // The actual implementation might involve providing data directly or referencing a file. ``` ``` -------------------------------- ### Run Go Benchmark Tests Source: https://github.com/clickhouse/clickhouse-go/blob/main/CONTRIBUTING.md Execute all benchmark tests within the project using the `-bench=.` flag. The `-benchmem` flag includes memory allocation statistics. ```bash # Run Go benchmark tests go test -bench=. -benchmem ./benchmark/... ``` -------------------------------- ### Configuring ClickHouse Context Options in Go Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/3-native-api-reference.md Shows how to create a context with ClickHouse-specific options like query ID, settings, async mode, and custom logging. This context can then be passed to query execution functions. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithQueryID("my-query-1"), clickhouse.WithSettings(clickhouse.Settings{ "max_execution_time": 60, }), clickhouse.WithAsync(false), clickhouse.WithLogs(func(log *clickhouse.Log) { fmt.Printf("Log: %v\n", log) }), ) rows, err := conn.Query(ctx, "SELECT * FROM large_table") ``` -------------------------------- ### QueryRowContext Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Executes a query expected to return a single row. The error is deferred until Scan() is called. ```go var count int err := db.QueryRowContext(context.Background(), "SELECT COUNT(*) FROM users").Scan(&count) if err != nil { log.Fatal(err) } fmt.Printf("Total users: %d\n", count) ``` -------------------------------- ### Get Batch Columns Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/3-native-api-reference.md Returns the column objects associated with the batch, which define the schema for the data being inserted. ```go func (b Batch) Columns() []column.Interface ``` -------------------------------- ### SetMaxIdleConns Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Sets the maximum number of idle connections in the pool. This overrides the MaxIdleConns option set during database initialization. ```go db.SetMaxIdleConns(10) ``` -------------------------------- ### InOrder Connection Strategy Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/8-connection-management.md Use the default 'InOrder' strategy to always try servers in the order they are listed in Addr. Suitable when one server is preferred. ```go conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{ "primary:9000", "replica1:9000", "replica2:9000", }, ConnOpenStrategy: clickhouse.ConnOpenInOrder, }) ``` -------------------------------- ### SetConnMaxLifetime Example Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Sets the maximum lifetime of a connection in the pool. Connections older than this duration will be closed and replaced. This overrides the ConnMaxLifetime option. ```go db.SetConnMaxLifetime(time.Hour) ``` -------------------------------- ### Async Insert with Server Settings via Go Context Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/9-advanced-features.md Configure async insert parameters by passing settings within the Go context for the connection. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithAsync(false), clickhouse.WithSettings(clickhouse.Settings{ "async_insert": 1, "async_insert_max_data_size": 10485760, "async_insert_busy_timeout_ms": 1000, }), ) ``` -------------------------------- ### Pass Application Metadata with ClientInfo Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/9-advanced-features.md Provide application metadata, such as product names and versions, by using `clickhouse.WithClientInfo`. This information is visible in ClickHouse system tables like `system.query_log`. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithClientInfo(clickhouse.ClientInfo{ Products: []struct{ Name string Version string }{ {Name: "my-app", Version: "1.2.3"}, {Name: "integrations/kafka", Version: "0.1"}, }, Comment: "analytics-batch", }), ) rows, err := conn.Query(ctx, "SELECT * FROM events") ``` ```sql SELECT client_name FROM system.query_log WHERE database = 'default' ``` -------------------------------- ### Get Number of Rows in Batch Buffer Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/3-native-api-reference.md Returns the number of rows currently appended to the batch buffer, which have not yet been sent to the server. ```go func (b Batch) Rows() int ``` -------------------------------- ### OpenDB: Create Standard SQL DB Handle Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Use `OpenDB` to create a `*sql.DB` handle compatible with the standard Go `database/sql` package. It wraps the native ClickHouse protocol, applying all connection pooling and option settings identically. Ensure to close the database connection when done. ```go import ( "database/sql" "github.com/ClickHouse/clickhouse-go/v2" ) db := clickhouse.OpenDB(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, Auth: clickhouse.Auth{ Database: "default", Username: "default", Password: "", }, }) deferr db.Close() var name string er := db.QueryRowContext(context.Background(), "SELECT name FROM users LIMIT 1").Scan(&name) ``` -------------------------------- ### Get Connection Pool Statistics Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/3-native-api-reference.md Retrieves current connection pool statistics without blocking. Useful for monitoring pool pressure. ```go stats := conn.Stats() fmt.Printf("Open: %d/%d, Idle: %d/%d\n", stats.Open, stats.MaxOpenConns, stats.Idle, stats.MaxIdleConns, ) ``` -------------------------------- ### Get Pool Statistics Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/8-connection-management.md Retrieve real-time statistics about the connection pool, including open and idle connections, to monitor pool pressure. ```go stats := conn.Stats() fmt.Printf("Open connections: %d (max %d)\n", stats.Open, stats.MaxOpenConns) fmt.Printf("Idle connections: %d (max %d)\n", stats.Idle, stats.MaxIdleConns) // Detect pool pressure if stats.Open >= stats.MaxOpenConns { log.Println("WARNING: Connection pool exhausted") } ``` -------------------------------- ### HTTP DSN with Compression and Secure Connection Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/2-options-configuration.md Example of a DSN string for HTTP(S) connections with authentication, multiple hosts, compression, and TLS settings. ```text https://user:pass@host1:8443,host2:8443/mydb?compress=zstd&compress_level=3&secure=true&skip_verify=false ``` -------------------------------- ### Multi-Goroutine Queries with Connection Pooling Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/10-examples-and-patterns.md Demonstrates how to use connection pooling to execute multiple queries concurrently across different goroutines. Configure MaxOpenConns and MaxIdleConns in clickhouse.Options for efficient connection management. ```go package main import ( "context" "fmt" "log" "sync" "github.com/ClickHouse/clickhouse-go/v2" ) func main() { conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, MaxOpenConns: 20, MaxIdleConns: 10, }) if err != nil { log.Fatal(err) } defer conn.Close() var wg sync.WaitGroup // Launch 10 concurrent queries for i := 0; i < 10; i++ { wg.Add(1) go func(id int) { defer wg.Done() var count int err := conn.QueryRow(context.Background(), fmt.Sprintf("SELECT COUNT(*) FROM events WHERE id %% 10 = %d", id)). Scan(&count) if err != nil { log.Printf("Query %d failed: %v", id, err) return } fmt.Printf("Query %d: %d rows\n", id, count) }(i) } wg.Wait() fmt.Println("All queries completed") } ``` -------------------------------- ### Inserting Values into Ephemeral Columns Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/9-advanced-features.md Insert values into ephemeral columns, which are computed server-side, by defining them in WithColumnNamesAndTypes. The computed_field is an example of an ephemeral column. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithColumnNamesAndTypes([]clickhouse.ColumnNameAndType{ {Name: "id", Type: "UInt32"}, {Name: "name", Type: "String"}, {Name: "computed_field", Type: "String"}, // Ephemeral }), ) batch, err := conn.PrepareBatch(ctx, "INSERT INTO t") if err != nil { log.Fatal(err) } defer batch.Close() batch.Append(1, "Alice", "computed_value") batch.Send() ``` -------------------------------- ### Create and Query Time/Time64 Table Source: https://github.com/clickhouse/clickhouse-go/blob/main/TYPES.md This snippet demonstrates creating a table with Time and Time64 columns of varying precisions, inserting data using time.Duration, and querying it back. Ensure 'enable_time_time64_type' setting is enabled. ```go package main import ( "context" "fmt" "time" "github.com/ClickHouse/clickhouse-go/v2" ) func main() { conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{"localhost:9000"}, }) if err != nil { panic(err) } defer conn.Close() ctx := clickhouse.Context(context.Background(), clickhouse.WithSettings(clickhouse.Settings{ "enable_time_time64_type": 1, })) // Create table with Time and Time64 columns err = conn.Exec(ctx, ` CREATE TABLE IF NOT EXISTS time_example ( id UInt32, t_time Time, t_time64_3 Time64(3), t_time64_6 Time64(6), t_time64_9 Time64(9), arr_time Array(Time), nullable_time Nullable(Time64(9)) ) ENGINE = MergeTree() ORDER BY id `) if err != nil { panic(err) } // Insert data using time.Duration batch, err := conn.PrepareBatch(ctx, "INSERT INTO time_example") if err != nil { panic(err) } // Time values as time.Duration (duration since midnight) timeValue := 12*time.Hour + 34*time.Minute + 56*time.Second time64Value := 15*time.Hour + 30*time.Minute + 45*time.Second + 123456789*time.Nanosecond timeArray := []time.Duration{ 6 * time.Hour, 12*time.Hour + 30*time.Minute, 18*time.Hour + 45*time.Minute + 30*time.Second, } err = batch.Append( uint32(1), timeValue, time64Value, time64Value, time64Value, timeArray, &time64Value, // nullable value ) if err != nil { panic(err) } // Insert NULL for nullable column err = batch.Append( uint32(2), timeValue, time64Value, time64Value, time64Value, timeArray, nil, // NULL value ) if err != nil { panic(err) } err = batch.Send() if err != nil { panic(err) } // Query data rows, err := conn.Query(ctx, "SELECT id, t_time, t_time64_9, arr_time, nullable_time FROM time_example ORDER BY id") if err != nil { panic(err) } defer rows.Close() for rows.Next() { var ( id uint32 tTime time.Duration tTime64 time.Duration arrTime []time.Duration nullableTime *time.Duration ) if err := rows.Scan(&id, &tTime, &tTime64, &arrTime, &nullableTime); err != nil { panic(err) } fmt.Printf("ID: %d\n", id) fmt.Printf(" Time (second precision): %v\n", tTime) fmt.Printf(" Time64(9) (nanosecond precision): %v\n", tTime64) fmt.Printf(" Array(Time): %v\n", arrTime) if nullableTime != nil { fmt.Printf(" Nullable Time64: %v\n", *nullableTime) } else { fmt.Printf(" Nullable Time64: NULL\n") } } } ``` -------------------------------- ### Get Server Version Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/8-connection-management.md Retrieve the ClickHouse server version, including major, minor, and patch numbers. This can be used to check server capabilities. ```go version, err := conn.ServerVersion() if err != nil { log.Fatal(err) } fmt.Printf("ClickHouse %d.%d.%d\n", version.MajorVersion, version.MinorVersion, version.Patch) ``` -------------------------------- ### PrepareBatch with ReleaseConnection Option Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Use the WithReleaseConnection option with PrepareBatch to return the connection to the pool after preparation. This is useful for creating long-lived batches. ```go // For clickhouse.Conn.PrepareBatch (native interface) // batch, err := conn.PrepareBatch(ctx, "INSERT INTO ...") // defer batch.Close() // batch.Append(...) // batch.Flush() // Sends buffered rows, keeps batch usable // batch.Send() // Flushes remaining rows and finalizes INSERT ``` -------------------------------- ### Add Query Settings Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/README.md Applies specific query settings, such as `max_execution_time`, by creating a context with custom options. This allows fine-grained control over query execution. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithQueryID("report-1"), clickhouse.WithSettings(clickhouse.Settings{ "max_execution_time": 60, }), ) rows, err := conn.Query(ctx, "SELECT * FROM events") ``` -------------------------------- ### Connect using HTTPS with Protocol and TLS Config Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Establish an HTTPS connection by explicitly setting the protocol to clickhouse.HTTP and configuring TLS settings. This provides more granular control over the connection parameters. ```go conn := clickhouse.OpenDB(&clickhouse.Options{ Addr: []string{"127.0.0.1:8443"}, Auth: clickhouse.Auth{ Database: "default", Username: "default", Password: "", }, Protocol: clickhouse.HTTP, }) ``` -------------------------------- ### HTTP Proxy Configuration via DSN Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Example of setting an HTTP proxy directly within the DSN string. The proxy address must be URL encoded. ```sh http://host1:8123,host2:8123/database?dial_timeout=200ms&max_execution_time=60&http_proxy=http%3A%2F%2Fproxy%3A8080 ``` -------------------------------- ### Use Standard Library API Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/README.md Integrates ClickHouse with Go's standard `database/sql` package. This allows using familiar SQL interfaces for interacting with ClickHouse. ```go db := clickhouse.OpenDB(&clickhouse.Options{ Addr: []string{"127.0.0.1:8123"}, }) var count int err := db.QueryRowContext(ctx, "SELECT COUNT(*) FROM users").Scan(&count) ``` -------------------------------- ### Get Server Version Information Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/3-native-api-reference.md Retrieve the ClickHouse server version and protocol capabilities using ServerVersion. This method uses an internal timeout defined by Options.DialTimeout. ```go sv, err := conn.ServerVersion() if err != nil { log.Fatal(err) } fmt.Printf("ClickHouse version: %d.%d.%d\n", sv.MajorVersion, sv.MinorVersion, sv.Patch) ``` -------------------------------- ### Track Query Progress with QueryOption Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/4-context-options.md Implement WithProgress to track query execution progress in real time using a callback function. This is beneficial for displaying progress on long-running queries. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithProgress(func(p *clickhouse.Progress) { fmt.Printf("Read: %d rows, %.2f MB\n", p.ReadRows, float64(p.ReadBytes)/1024/1024) }), ) rows, err := conn.Query(ctx, "SELECT * FROM events LIMIT 1000000") ``` -------------------------------- ### Enable Asynchronous Inserts (Fire and Forget) Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/4-context-options.md Use WithAsync(false) to enable asynchronous insert mode, returning immediately after the server accepts the insert. Errors are only reported if the insert is rejected before queuing. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithAsync(false), // Return immediately ) err := conn.Exec(ctx, "INSERT INTO events (id, message) VALUES (?, ?)", 1, "event data") if err != nil { log.Fatal(err) // Error only if rejected before queueing } ``` -------------------------------- ### Append JSON as String Mode Source: https://github.com/clickhouse/clickhouse-go/blob/main/README.md Appends a raw JSON string to a JSON column. This example demonstrates an error when trying to append a string to a column already in 'object' mode. ```go batch.Append(`{"x":1}`) // error: string in an object-mode column ``` -------------------------------- ### OpenDB Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Creates a standard `*sql.DB` handle compatible with the `database/sql` package, using the native ClickHouse protocol. Connection options can be provided via `*Options`. ```APIDOC ## Function: OpenDB ### Description Creates a standard `*sql.DB` handle compatible with the `database/sql` package, using the native ClickHouse protocol. Connection options can be provided via `*Options`. ### Signature ```go func OpenDB(opt *Options) *sql.DB ``` ### Parameters #### Path Parameters - **opt** (`*Options`) - Optional - Connection options; nil uses all defaults ### Returns - `*sql.DB`: Standard Go SQL database handle ### Example ```go import ( "context" "database/sql" "github.com/ClickHouse/clickhouse-go/v2" ) db := clickhouse.OpenDB(&clickhouse.Options{ Addr: []string{"127.0.0.1:9000"}, Auth: clickhouse.Auth{ Database: "default", Username: "default", Password: "", }, }) dfer db.Close() var name string err := db.QueryRowContext(context.Background(), "SELECT name FROM users LIMIT 1").Scan(&name) ``` ``` -------------------------------- ### Retrieve Query Profile Information Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/4-context-options.md Use `WithProfileInfo` to get detailed query execution statistics like rows scanned and execution time. The callback function is invoked after the query completes. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithProfileInfo(func(info *clickhouse.ProfileInfo) { fmt.Printf("Rows read: %d\n", info.Rows) fmt.Printf("Bytes read: %d\n", info.Bytes) fmt.Printf("Time: %v\n", info.ElapsedNanoseconds) }), ) rows, err := conn.Query(ctx, "SELECT * FROM events WHERE year = 2024") ``` -------------------------------- ### ExecContext Example for INSERT with Async Option Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/5-standard-library-api.md Executes a statement like INSERT, UPDATE, or DELETE. If context contains WithAsync(false), INSERT operations can be fire-and-forget. Note that RowsAffected() is not applicable for ClickHouse. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithAsync(false), // Fire and forget ) result, err := db.ExecContext(ctx, "INSERT INTO logs (message) VALUES (?)", "error occurred") if err != nil { log.Fatal(err) } // Note: RowsAffected() is not applicable for ClickHouse ``` -------------------------------- ### Integrate ClickHouse with GORM ORM Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/10-examples-and-patterns.md Utilize the GORM ORM with the ClickHouse driver for simplified database interactions, including schema migration, creation, and querying. Ensure GORM and the ClickHouse driver are installed. ```go package main import ( "fmt" "log" "gorm.io/driver/clickhouse" "gorm.io/gorm" ) type Event struct { ID uint `gorm:"primaryKey"` Name string Count int } func main() { db, err := gorm.Open( clickhouse.Open("clickhouse://default@127.0.0.1:9000/default"), &gorm.Config{}, ) if err != nil { log.Fatal(err) } // AutoMigrate db.AutoMigrate(&Event{}) // Create db.Create(&Event{Name: "click", Count: 1}) // Query var events []Event db.Where("count > ?", 0).Find(&events) for _, e := range events { fmt.Printf("ID: %d, Name: %s, Count: %d\n", e.ID, e.Name, e.Count) } } ``` -------------------------------- ### Compression Method Constants Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/7-types-and-errors.md Enumerates the supported compression algorithms with their corresponding constant values. ```go const ( CompressionNone = CompressionMethod(0) CompressionLZ4 = CompressionMethod(1) CompressionLZ4HC = CompressionMethod(2) CompressionZSTD = CompressionMethod(3) CompressionGZIP = CompressionMethod(0x95) CompressionDeflate = CompressionMethod(0x96) CompressionBrotli = CompressionMethod(0x97) ) ``` -------------------------------- ### Access Detailed Profile Events for Performance Analysis Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/9-advanced-features.md Use `clickhouse.WithProfileEvents` to get granular profiling events, including event names and values, for in-depth performance analysis. Requires server version 25.11 or later. ```go ctx := clickhouse.Context(context.Background(), clickhouse.WithProfileEvents(func(events []clickhouse.ProfileEvent) { for _, e := range events { if e.Value > 0 { fmt.Printf("%s: %d\n", e.Name, e.Value) } } }), ) rows, err := conn.Query(ctx, "SELECT * FROM large_table") ``` -------------------------------- ### Retry Batch Operations After Error Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/6-batch-operations.md If a batch operation fails, its state becomes unpredictable. Always create a new batch instance for retries. This example demonstrates preparing a new batch and sending data after a previous `Send` failure. ```go batch, err := conn.PrepareBatch(ctx, "INSERT INTO t") if err != nil { log.Fatal(err) } defer batch.Close() if err := batch.Append(1, "data"); err != nil { log.Fatal(err) } if err := batch.Send(); err != nil { log.Printf("Send failed: %v", err) // Create a new batch to retry newBatch, err := conn.PrepareBatch(ctx, "INSERT INTO t") if err != nil { log.Fatal(err) } defer newBatch.Close() newBatch.Append(1, "data") newBatch.Send() } ``` -------------------------------- ### Configure Failover and Load Balancing in Go Source: https://github.com/clickhouse/clickhouse-go/blob/main/_autodocs/README.md Set up multiple ClickHouse addresses for failover and load balancing using a round-robin connection strategy. ```go conn, err := clickhouse.Open(&clickhouse.Options{ Addr: []string{ "replica1:9000", "replica2:9000", "replica3:9000", }, ConnOpenStrategy: clickhouse.ConnOpenRoundRobin, }) ```