### Install go-elasticsearch Client Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Installs the latest version of the Elasticsearch Go client using the go get command. Requires Go version 1.21+. ```shell go get github.com/elastic/go-elasticsearch/v9@latest ``` -------------------------------- ### Complete Elasticsearch Go Client Example Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/installation.md A full example demonstrating how to set up a Go application with the Elasticsearch client. It includes creating a `go.mod` file and a `main.go` file to initialize the client and print its version and info. ```text mkdir my-elasticsearch-app && cd my-elasticsearch-app cat > go.mod <<-"END" module my-elasticsearch-app require github.com/elastic/go-elasticsearch/v9 main END cat > main.go <<-"END" package main import ( "log" "github.com/elastic/go-elasticsearch/v9" ) func main() { es, _ := elasticsearch.NewDefaultClient() log.Println(elasticsearch.Version) log.Println(es.Info()) } END go run main.go ``` -------------------------------- ### Setup Elasticsearch Demo with Docker Compose Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/instrumentation/README.md Commands to run the Elasticsearch demo environment using Docker Compose. This includes starting the Elasticsearch stack and later downing the services. ```Shell docker-compose --file elasticstack.yml up --build ``` ```Shell docker-compose --file elasticstack.yml down --remove-orphans --volumes ``` -------------------------------- ### Search Documents (Go Client) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Executes a search query against an Elasticsearch index using both low-level and fully-typed Go client APIs. Shows a simple match_all query example. ```go query := `{ "query": { "match_all": {} } }` client.Search( client.Search.WithIndex("my_index"), client.Search.WithBody(strings.NewReader(query)), ) ``` ```go typedClient.Search(). Index("my_index"). Request(&search.Request{ Query: &types.Query{MatchAll: &types.MatchAllQuery{}}, }). Do(context.TODO()) ``` -------------------------------- ### Create Index (Go Client) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Shows how to create an Elasticsearch index using both the low-level and fully-typed APIs of the Go client. ```go client.Indices.Create("my_index") ``` ```go typedClient.Indices.Create("my_index").Do(context.TODO()) ``` -------------------------------- ### Benchmark Setup and Dependencies Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/bulk/benchmarks/README.md Commands required to set up the Go environment, download dependencies, install easyjson, and generate necessary auxiliary files for running benchmarks. ```shell go mod download go get -u github.com/mailru/easyjson/... grep '~/go/bin' ~/.profile || echo 'export PATH=$PATH:~/go/bin' >> ~/.profile && source ~/.profile go generate -v ./model ``` -------------------------------- ### Connect to Elasticsearch (Go Client) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Demonstrates how to instantiate the Elasticsearch Go client using Cloud ID and API Key. Supports both low-level and fully-typed API connections. ```go client, err := elasticsearch.NewClient(elasticsearch.Config{ CloudID: "", APIKey: "", }) ``` ```go typedClient, err := elasticsearch.NewTypedClient(elasticsearch.Config{ CloudID: "", APIKey: "", }) ``` -------------------------------- ### Index Document (Go Client) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Demonstrates indexing a document into Elasticsearch using both low-level and fully-typed Go client APIs. Includes JSON marshaling for the document body. ```go document := struct { Name string `json:"name"` }{ "go-elasticsearch", } data, _ := json.Marshal(document) client.Index("my_index", bytes.NewReader(data)) ``` ```go document := struct { Name string `json:"name"` }{ "go-elasticsearch", } typedClient.Index("my_index"). Id("1"). Request(document). Do(context.TODO()) ``` -------------------------------- ### GCP Cloud Functions Elasticsearch Client Setup (Go) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/connecting.md Demonstrates best practices for initializing the Elasticsearch client in a Function-as-a-Service (FaaS) environment, specifically for GCP Cloud Functions. The client is initialized globally in the `init` function to improve performance and enable background operations. ```go package httpexample import ( "github.com/elastic/go-elasticsearch/v9" "log" "net/http" ) var client *elasticsearch.Client func init() { var err error // ... Client configuration (e.g., addresses, auth) cfg := elasticsearch.Config{ Addresses: []string{ "https://localhost:9200", }, } client, err = elasticsearch.NewClient(cfg) if err != nil { log.Fatalf("elasticsearch.NewClient: %v", err) } } func HttpExample(w http.ResponseWriter, r *http.Request) { // ... Client usage with the global 'client' variable // For example: res, err := client.Info() } ``` -------------------------------- ### Run Go Manual TLS Configuration Example Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/security/README.md Executes a Go program that demonstrates manual configuration of TLS settings for the HTTP transport, specifically for adding CA certificates to trust the Elasticsearch cluster. ```bash go run tls_configure_ca.go # Expected Output: # [200 OK] { # ... # } ``` -------------------------------- ### Update Document (Go Client) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Updates an existing document in Elasticsearch, for example, to add a new field, using both low-level and fully-typed Go client APIs. ```go client.Update("my_index", "id", strings.NewReader(`{doc: { language: "Go" }}`)) ``` ```go typedClient.Update("my_index", "id"). Request(&update.Request{ Doc: json.RawMessage(`{ language: "Go" }`), }).Do(context.TODO()) ``` -------------------------------- ### Get Document (Go Client) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Retrieves a document from Elasticsearch by its index and ID using both low-level and fully-typed Go client APIs. ```go client.Get("my_index", "id") ``` ```go typedClient.Get("my_index", "id").Do(context.TODO()) ``` -------------------------------- ### Initialize Typed Elasticsearch Client Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/_getting_started_with_the_api.md Demonstrates how to obtain a new typed Elasticsearch client using the `NewTypedClient` function from the `elasticsearch` package. It requires an `elasticsearch.Config` object for cluster connection details. The client is generated from the elasticsearch-specification. ```go client, err := elasticsearch.NewTypedClient(elasticsearch.Config{ // Proper configuration for your Elasticsearch cluster. }) ``` -------------------------------- ### Run Go TLS Example Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/security/README.md Executes a Go program that utilizes TLS for secure communication with Elasticsearch, demonstrating the client configuration with a provided CA certificate. ```bash go run tls_with_ca.go # Expected Output: # [200 OK] { # ... # } ``` -------------------------------- ### Run Default Go Elasticsearch Loggers Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/logging/README.md Executes a Go program demonstrating the default loggers provided by the elastictransport package. It shows how to run the example and the resulting log output, illustrating various HTTP requests and their status codes. ```bash go run default.go # ... # DELETE http://localhost:9200/test/_doc/1 404 Not Found 8ms # HEAD http://localhost:9200/test/_doc/1 404 Not Found 16ms # POST http://localhost:9200/test/_doc?filter_path=result,_id&pretty=true&refresh=true 201 Created 45ms # GET http://localhost:9200/_search?q=[FAIL 400 Bad Request 21ms # GET http://localhost:9200/test/_search?filter_path=took,hits.hits&pretty=true&size=1 200 OK 11ms # ... ``` -------------------------------- ### Create Index with Mapping Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/examples.md Demonstrates how to create an Elasticsearch index named 'test-index' and define a mapping for the 'price' field as an integer using the Go client's builder pattern. ```go res, err := es.Indices.Create("test-index"). Request(&create.Request{ Mappings: &types.TypeMapping{ Properties: map[string]types.Property{ "price": types.NewIntegerNumberProperty(), }, }, }). Do(nil) ``` -------------------------------- ### Generate Elasticsearch Certificates Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/security/README.md Demonstrates generating TLS certificates for Elasticsearch nodes using the `elasticsearch-certutil` tool via `make` commands. This is a prerequisite for securing communication and starting a cluster with full security. ```bash make certificates make cluster ``` -------------------------------- ### Delete Elasticsearch Index using Go Client Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Demonstrates how to delete one or more Elasticsearch indices using the Go client. It shows examples for both the low-level and the fully-typed API, allowing developers to choose the most suitable method for their needs. ```go client.Indices.Delete([]string{"my_index"}) ``` ```go typedClient.Indices.Delete("my_index").Do(context.TODO()) ``` -------------------------------- ### Run Custom Go Elasticsearch Logger Example Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/logging/README.md Executes a Go program that implements and utilizes a custom logger with the Elasticsearch client. The output demonstrates structured logging with timestamps, log levels, request details, and status codes. ```bash go run custom.go # 1:41PM WRN http://localhost:9200/test/_doc/1 duration=19.001 method=DELETE req_bytes=0 res_bytes=155 status_code=404 # 1:41PM WRN http://localhost:9200/test/_doc/1 duration=5.898 method=HEAD req_bytes=0 res_bytes=0 status_code=404 # 1:41PM INF http://localhost:9200/test/_doc?refresh=true duration=68.841 method=POST req_bytes=21 res_bytes=194 status_code=201 # 1:41PM WRN http://localhost:9200/_search?q=%7BFAIL duration=17.276 method=GET req_bytes=0 res_bytes=765 status_code=400 # 1:41PM INF http://localhost:9200/test/_search?size=1 duration=9.429 method=GET req_bytes=49 res_bytes=280 status_code=200 ``` -------------------------------- ### Perform Sum Aggregation Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/examples.md Shows how to execute a sum aggregation on the 'price' field within a specified index. The example sets the search size to 0 to retrieve only aggregation results, demonstrating the structure for defining aggregations. ```go totalPricesAgg, err := es.Search(). Index("index_name"). Request( &search.Request{ Size: some.Int(0), Aggregations: map[string]types.Aggregations{ "total_prices": { Sum: &types.SumAggregation{ Field: some.String("price"), }, }, }, }, ).Do(context.Background()) ``` -------------------------------- ### Go Bulk Indexing: Low-Level Elasticsearch API Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/bulk/README.md Demonstrates low-level mechanics of Elasticsearch's Bulk API in Go. It covers preparing meta/data pairs, filling a buffer, sending batches, and checking for errors. This example avoids abstractions to show core functionality. ```bash go run default.go -count=100000 -batch=25000 # Bulk: documents [100,000] batch size [25,000] # # → Generated 100,000 articles # → Sending batch [1/4] [2/4] [3/4] [4/4] # ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ # Sucessfuly indexed [100,000] documents in 3.423s (29,214 docs/sec) ``` -------------------------------- ### Install Elasticsearch Go Client (Git Clone) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/installation.md Clone the Elasticsearch Go client repository directly from GitHub. This method allows you to specify a branch, often corresponding to a specific client version. ```text git clone --branch 9.0 https://github.com/elastic/go-elasticsearch.git $GOPATH/src/github ``` -------------------------------- ### Install Elasticsearch Go Client (go.mod) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/installation.md Add the Elasticsearch Go client package to your `go.mod` file for installation. This specifies the version of the client to use, typically aligning with Elasticsearch major versions. ```text require github.com/elastic/go-elasticsearch/v9 9.0 ``` -------------------------------- ### Run Benchmark with Small Dataset Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/bulk/benchmarks/README.md Example command to execute benchmarks using the 'small' dataset with specific configurations, including document count, flush size, sharding, and HTTP transport. ```shell ELASTICSEARCH_URL=http://server:9200 go run benchmarks.go --dataset=small --count=1_000_000 --flush=2MB --shards=5 --replicas=0 --fasthttp=true --easyjson=true ``` -------------------------------- ### Run Benchmark with HTTP Log Event Dataset Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/bulk/benchmarks/README.md Example command to run benchmarks using the 'httplog' dataset, which represents a larger document size, with custom settings for count, flush, and transport. ```shell ELASTICSEARCH_URL=http://server:9200 go run benchmarks.go --dataset=httplog --count=1_000_000 --flush=3MB --shards=5 --replicas=0 --fasthttp=true --easyjson=true ``` -------------------------------- ### Instrument Go Elasticsearch Client with OpenCensus Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/instrumentation/README.md Demonstrates using the `ochttp.Transport` wrapper to auto-instrument Elasticsearch client calls. It includes a simple exporter that prints instrumentation information to the terminal. ```Go package main import ( "context" "fmt" "go.opencensus.io/exporter/stdout" "go.opencensus.io/plugin/ochttp" "go.opencensus.io/trace" "github.com/elastic/go-elasticsearch/v8" ) func main() { // Configure OpenCensus exporter exporter, err := stdout.NewExporter(stdout.Options{ PrettyPrint: true, }) if err != nil { panic(err) } trace.RegisterExporter(exporter) trace.SetDefaultSampler(trace.AlwaysSample()) // Create Elasticsearch client with instrumented transport cfg := elasticsearch.Config{ Addresses: []string{ "http://localhost:9200", }, } es, err := elasticsearch.NewClient(cfg) if err != nil { panic(err) } // Perform a search request res, err := es.Search( es.Search.WithContext(context.Background()), es.Search.WithIndex("my-index"), es.Search.WithBodyJson(map[string]interface{}{ "query": map[string]interface{}{ "match": map[string]interface{}{ "message": "elasticsearch", }, }, }), ) if err != nil { panic(err) } fmt.Println("Search response:", res.String()) // The exporter will print trace information to the terminal } ``` -------------------------------- ### Search Document Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/examples.md Illustrates how to perform a search query, specifically looking for documents where the 'name' field matches 'Foo' within the 'index_name'. It shows building the query using structs and executing it. ```go res, err := es.Search(). Index("index_name"). Request(&search.Request{ Query: &types.Query{ Match: map[string]types.MatchQuery{ "name": {Query: "Foo"}, }, }, }).Do(context.Background()) ``` -------------------------------- ### Index Document (Struct and Raw) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/examples.md Shows two methods for indexing a document: providing a Go struct which is automatically JSON encoded, or using the Raw method with pre-serialized JSON bytes. ```go document := struct { Id int `json:"id"` Name string `json:"name"` Price int `json:"price"` }{ Id: 1, Name: "Foo", Price: 10, } res, err := es.Index("index_name"). Request(document). Do(context.Background()) ``` ```go res, err := es.Index("index_name"). Raw([]byte(`{ "id": 1, "name": "Foo", "price": 10 }`)).Do(context.Background()) ``` -------------------------------- ### Delete Document (Go Client) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/getting-started.md Removes a document from Elasticsearch by its index and ID using both low-level and fully-typed Go client APIs. ```go client.Delete("my_index", "id") ``` ```go typedClient.Delete("my_index", "id").Do(context.TODO()) ``` -------------------------------- ### Retrieve Document Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/examples.md Explains how to retrieve a specific document from an index by providing the index name and the document ID to the Get API. ```go res, err := es.Get("index_name", "doc_id").Do(context.Background()) ``` -------------------------------- ### Run Go Elasticsearch Client Extension Example Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/extension/README.md Demonstrates how to execute the Go program that extends the Elasticsearch client and shows the expected output, including HTTP requests and responses from custom API calls. ```bash go run main.go ``` ```output GET http://localhost:9209/_cat/health 200 OK 25ms « 1555252476 14:34:36 go-elasticsearch green 1 1 0 0 0 0 0 0 - 100.0% GET http://localhost:9209/_cat/example 200 OK 0s « Hello from Cat Example action ``` -------------------------------- ### Deploy Go Cloud Function with gcloud Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/cloudfunction/README.md Command to deploy the Google Cloud Function. It specifies the entry point, runtime, trigger type, memory allocation, and environment variables for the Elasticsearch URL. ```bash go mod vendor gcloud functions deploy clusterstatus \ --entry-point Health \ --runtime go111 \ --trigger-http \ --memory 128MB \ --set-env-vars ELASTICSEARCH_URL=https://...cloud.es.io:9243 ``` -------------------------------- ### Get CA Fingerprint via OpenSSL s_client Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/connecting.md Provides a command to retrieve the root CA fingerprint of a running Elasticsearch instance using `openssl s_client`. This is helpful when the CA certificate file is not directly accessible. ```sh # Replace the values of 'localhost' and '9200' to the # corresponding host and port values for the cluster. openssl s_client -connect localhost:9200 -servername localhost -showcerts /dev/null \ | openssl x509 -fingerprint -sha256 -noout -in /dev/stdin ``` -------------------------------- ### Go Bulk Indexing: esutil.BulkIndexer Helper Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/bulk/README.md Demonstrates using the `esutil.BulkIndexer` helper for efficient, parallel indexing in Go. It allows adding items and flushing batches based on configured thresholds, simplifying bulk operations. ```bash go run indexer.go -count=100000 -flush=1000000 # BulkIndexer: documents [100,000] workers [8] flush [1.0 MB] # # → Generated 100,000 articles # ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ # Sucessfuly indexed [100,000] documents in 1.909s (52,383 docs/sec) ``` ```go indexer, _ := esutil.NewBulkIndexer(esutil.BulkIndexerConfig{}) indexer.Add( context.Background(), esutil.BulkIndexerItem{ Action: "index", Body: strings.NewReader(`{"title":"Test"}`), }) indexer.Close(context.Background()) ``` -------------------------------- ### Go Client Enum Serialization Example Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/conventions.md Illustrates how enum values in the Go Elasticsearch client are serialized to their corresponding API string representations, using refresh options as an example. ```go refresh.True => "true" refresh.False => "false" refresh.Waitfor => "wait_for" ``` -------------------------------- ### Benchmark Configuration Options Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/bulk/benchmarks/README.md Displays the available command-line flags for configuring the benchmark execution, including document count, dataset, flush thresholds, and worker settings. ```shell go run benchmarks.go --help -count int Number of documents to generate (default 100000) -dataset string Dataset to use for indexing (default "small") -debug Enable logging output -easyjson Use mailru/easyjson for JSON decoding -fasthttp Use valyala/fasthttp for HTTP transport -flush value Flush threshold in bytes (default 3MB) -index string Index name (default "test-bulk-benchmarks") -mockserver Measure added, not flushed items -replicas int Number of index replicas (default 0) -runs int Number of runs (default 10) -shards int Number of index shards (default 3) -wait duration Wait duration between runs (default 1s) -warmup int Number of warmup runs (default 3) -workers int Number of indexer workers (default 4) ``` -------------------------------- ### Initialize Default Elasticsearch Client (Go) Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/connecting.md Create an Elasticsearch client instance with default settings using `elasticsearch.NewDefaultClient()`. This function handles common configurations automatically. The client can then be used to make API requests, such as fetching cluster info. ```go es, err := elasticsearch.NewDefaultClient() if err != nil { log.Fatalf("Error creating the client: %s", err) } res, err := es.Info() if err != nil { log.Fatalf("Error getting response: %s", err) } defer res.Body.Close() log.Println(res) ``` -------------------------------- ### Go Elasticsearch Client: Low-level API Usage Source: https://github.com/elastic/go-elasticsearch/blob/main/docs/reference/index.md Demonstrates how to initialize and use the low-level API of the Go Elasticsearch client. It requires the `go-elasticsearch/v9` package and shows a basic `es.Info()` call to retrieve cluster information. ```go package main import ( "log" "github.com/elastic/go-elasticsearch/v9" ) func main() { es, _ := elasticsearch.NewDefaultClient() log.Println(es.Info()) } ``` -------------------------------- ### JSONReader for Structs/Maps Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/encoding/README.md Shows how to use the esutil.JSONReader() helper method to convert Go structs, maps, or other serializable objects into a JSON-wrapped reader. This reader can be conveniently passed to methods like WithBody(). ```go package main import ( "fmt" "github.com/elastic/go-elasticsearch/v8/esutil" ) type Document struct { Title string } func main() { doc := Document{Title: "Test"} // Assuming 'es' is an initialized Elasticsearch client // es.Search(es.Search.WithBody(esutil.NewJSONReader(&doc))) fmt.Println("esutil.JSONReader example placeholder") } ``` -------------------------------- ### Run Local Tests Source: https://github.com/elastic/go-elasticsearch/blob/main/CONTRIBUTING.md Execute the project's test suite locally to ensure code integrity before submitting changes. This command verifies the general functionality of the Go client. ```shell make test ``` -------------------------------- ### Manual Transport Configuration for TLS Source: https://github.com/elastic/go-elasticsearch/blob/main/_examples/security/README.md Illustrates manual configuration of the HTTP transport layer for TLS in the go-elasticsearch client. This involves using the `TLSClientConfig.RootCAs.AppendCertsFromPEM()` method to add CA certificates. ```go // Manual transport configuration // Use (*http.Transport).TLSClientConfig.RootCAs.AppendCertsFromPEM() ``` -------------------------------- ### Commit and Push API Changes Source: https://github.com/elastic/go-elasticsearch/wiki/How-to-make-a-release After generating the Go API, inspect the changes, stage them, commit with a descriptive message, and push to the repository. ```bash git status --short --branch ``` ```bash git add esapi/ ``` ```bash git commit --verbose --message='API: Update the APIs for Elasticsearch 8.x (152ee3ac)' ``` ```bash git push ``` -------------------------------- ### Generate Go API from Elasticsearch Spec Source: https://github.com/elastic/go-elasticsearch/wiki/How-to-make-a-release Steps to generate the Go API client from the Elasticsearch REST API specification. This involves cloning the Elasticsearch repository, checking out a specific version, and running the generation command. ```bash export version="elasticsearch:7.10.0" ``` ```bash $ make cluster-clean cluster-update cluster ``` ```bash $ curl -ks -u elastic:elastic https://localhost:9200 | jq -r '.version.build_hash' ``` ```bash $ curl -s http://localhost:9200 | jq -r '.version.build_hash' ``` ```bash cd tmp/elasticsearch git pull git checkout 152ee3aca4d6f47637ff9062f6a064349026ed8c ``` ```bash make gen-api ```