### Install ecs-logging-go-zap Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Add the package to your go.mod file. ```go require go.elastic.co/ecszap master ``` -------------------------------- ### EpochMicrosTimeEncoder Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/encoder.md Demonstrates how EpochMicrosTimeEncoder would be configured. Note that this encoder is not the default and requires explicit setup via ToZapCoreEncoderConfig(). ```go import ( "time" "go.elastic.co/ecszap" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) config := ecszap.EncoderConfig{ EnableCaller: false, EnableName: false, EnableStackTrace: false, } // Note: EpochMicrosTimeEncoder would be set via ToZapCoreEncoderConfig() // if custom time encoding is desired, but the default RFC3339UTCTimeEncoder // is recommended for ECS compliance ``` -------------------------------- ### Example Usage of Write Method Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Shows an example of logging an error, which triggers ECS field conversion before writing. The output format for errors is also indicated. ```go logger.Error("operation failed", zap.Error(err)) // Error is converted to ECS format before writing: // "error": {"message": "...", "stack_trace": "..."} ``` -------------------------------- ### JSON Configuration for EnableName Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Example of how to configure the EnableName option using JSON. ```json { "enableName": true } ``` -------------------------------- ### Minimal ECS Logger Setup in Go Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Sets up a basic ECS logger with default configurations, writing to standard output. Ensure zap and ecszap are imported. ```go package main import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) func main() { config := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(config, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller()) defer logger.Sync() logger.Info("application started") logger.Error("something went wrong", zap.String("reason", "timeout")) } ``` -------------------------------- ### Create Default EncoderConfig Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Use NewDefaultEncoderConfig to create an EncoderConfig with ECS-recommended defaults. This is the recommended way to start. ```go import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) config := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(config, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller()) ``` -------------------------------- ### Example Usage of With Method Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Demonstrates how to use the `With` method to add an error field, which will be converted to ECS format. ```go logger = logger.With(zap.Error(fmt.Errorf("connection failed"))) // Error is converted to ECS format internally ``` -------------------------------- ### Example Usage of stackTracer Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/types.md Demonstrates how an error wrapped with additional context implements the stackTracer interface, enabling serialization with stack_trace in ECS format. ```go import "github.com/pkg/errors" err := errors.Wrap(baseErr, "additional context") // This error implements stackTracer and will be serialized with stack_trace in ECS format ``` -------------------------------- ### Core Creation Flow Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/types.md Diagram illustrating the step-by-step process of creating a zap.Logger with ECS conversion, starting from EncoderConfig and ending with a wrapped Core. ```text EncoderConfig ↓ ToZapCoreEncoderConfig() → zapcore.EncoderConfig ↓ zapcore.NewJSONEncoder() → zapcore.Encoder ↓ zapcore.NewCore() → zapcore.Core ↓ WrapCore() → wrapped Core (with ECS conversion) ↓ zap.New() → *zap.Logger ``` -------------------------------- ### LowercaseLevelEncoder Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Encodes log level in lowercase, e.g., 'debug', 'info'. ```go func(zapcore.Level, zapcore.PrimitiveArrayEncoder) ``` -------------------------------- ### RFC3339UTCTimeEncoder Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Outputs time in RFC3339 format, suitable for ECS default ISO 8601 UTC. ```go func(time.Time, zapcore.PrimitiveArrayEncoder) ``` -------------------------------- ### Configure Zap Encoder with CallerEncoder Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/types.md Example of how to configure the Zap `EncoderConfig` to enable caller information and specify the `ShortCallerEncoder` for encoding. ```go config := ecszap.EncoderConfig{ EnableCaller: true, EncodeCaller: ecszap.ShortCallerEncoder, } ``` -------------------------------- ### NanosDurationEncoder Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Encodes duration in nanoseconds. ```go func(time.Duration, zapcore.PrimitiveArrayEncoder) ``` -------------------------------- ### ShortCallerEncoder Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Encodes caller information using the relative file path, suitable for the ECS default. ```go func(zapcore.EntryCaller, zapcore.PrimitiveArrayEncoder) ``` -------------------------------- ### Extract Metrics from Logs Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/integration-guide.md Log structured data that can be used to extract metrics. This example logs operation and duration. ```go package metrics import ( "fmt" "go.uber.org/zap" ) func LogAndMetric(logger *zap.Logger, operation string, duration float64) { // Log structured entry logger.Info(operation, zap.Float64("duration_ms", duration), ) // Could push to metrics system // metrics.Histogram("operation_duration", duration, map[string]string{ // "operation": operation, // }) } ``` -------------------------------- ### Duration Encoding in Milliseconds Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Configure the logger to encode durations in milliseconds, suitable for timing-sensitive applications. This example shows how to set up the encoder and log a duration. ```go import ( "time" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) // Milliseconds for timing-sensitive applications config := ecszap.EncoderConfig{ EncodeDuration: zapcore.MillisDurationEncoder, } core := ecszap.NewCore(config, os.Stdout, zap.InfoLevel) logger := zap.New(core) logger.Info("operation complete", zap.Duration("elapsed", 234*time.Millisecond)) // Output: "elapsed": 234 ``` -------------------------------- ### RFC3339UTCTimeEncoder Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/encoder.md Shows how RFC3339UTCTimeEncoder converts any time.Time value to UTC before formatting it as an RFC3339 string with millisecond precision. This is the recommended encoder for ECS compliance. ```go import ( "time" "go.elastic.co/ecszap" ) // Any local time is converted to UTC t := time.Date(2023, 7, 3, 14, 30, 45, 123000000, time.FixedZone("PST", -8*3600)) // Output: "2023-07-03T21:30:45.123Z" (converted to UTC) ``` -------------------------------- ### Wrap Existing Zap Core Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Wrap an existing zap core with ecszap to make it ECS compatible. Useful when integrating with existing logging setups. ```go existingCore := /* ... */ wrappedCore := ecszap.WrapCore(existingCore) logger := zap.New(wrappedCore) ``` -------------------------------- ### Kibana Dashboard Configuration for Application Logs Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/integration-guide.md Example Kibana dashboard configuration for visualizing application logs. Includes panel definitions for log display and error count. ```json { "title": "Application Logs", "panels": [ { "type": "logs", "filters": { "log.logger": "myapp" }, "columns": ["@timestamp", "message", "log.level", "log.origin.file.name"] }, { "type": "metric", "query": "log.level: error", "title": "Error Count" } ] } ``` -------------------------------- ### ECS Error Output Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/types.md This JSON demonstrates the expected output format for an ECS-compliant error, including message, stack trace, and cause. ```json { "error": { "message": "connection failed", "stack_trace": "...", "cause": [ {"message": "timeout"}, {"message": "EOF"} ] } } ``` -------------------------------- ### ECS Output Format Example Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/encoder.md This JSON object demonstrates the standard output format produced by the ECS encoder, including caller information under the 'log.origin' key. ```json { "@timestamp": "2023-07-03T14:30:45.123Z", "log.level": "info", "message": "Application started", "log.origin": { "file.name": "main/main.go", "file.line": 42, "function": "main.main" }, "log.logger": "myapp", "ecs.version": "1.6.0" } ``` -------------------------------- ### Create Default ECS Encoder Config Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/encoder-config.md Use NewDefaultEncoderConfig to get a pre-configured EncoderConfig suitable for ECS logging. This config is then used to create a zap core. ```go import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) config := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(config, os.Stdout, zap.InfoLevel) logger := zap.New(core, zap.AddCaller()) def logger.Sync() logger.Info("using default ECS config") ``` -------------------------------- ### Wrap Existing Zap Core for ECS Compliance Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/index.md Transition an existing zap logger to ECS compliance by wrapping its core. This method allows you to integrate ECS formatting into your current logging setup. ```go // Transition an existing zap logger to ECS compliance existingConfig := zap.NewProductionEncoderConfig() ecsConfig := ecszap.ECSCompatibleEncoderConfig(existingConfig) encoder := zapcore.NewJSONEncoder(ecsConfig) core := zapcore.NewCore(encoder, os.Stdout, zap.InfoLevel) logger := zap.New(ecszap.WrapCore(core)) ``` -------------------------------- ### Log Configuration Initialization with Zap Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/integration-guide.md Log application initialization steps, including environment and version information, and report errors during database validation. This helps in understanding the application's startup process and identifying configuration issues. ```go package config import ( "fmt" "go.uber.org/zap" ) type Config struct { Server ServerConfig Database DatabaseConfig Logger *zap.Logger } func (c *Config) Init() error { c.Logger.Info("initializing application", zap.String("environment", c.Server.Environment), zap.String("version", c.Server.Version), ) if err := c.validateDatabase(); err != nil { c.Logger.Error("database validation failed", zap.Error(err), zap.String("host", c.Database.Host), zap.String("port", fmt.Sprint(c.Database.Port)), ) return err } c.Logger.Info("application initialized successfully") return nil } ``` -------------------------------- ### Transition from existing configurations Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Adapt existing Zap configurations to be ECS compatible. ```go encoderConfig := ecszap.ECSCompatibleEncoderConfig(zap.NewDevelopmentEncoderConfig()) encoder := zapcore.NewJSONEncoder(encoderConfig) core := zapcore.NewCore(encoder, os.Stdout, zap.DebugLevel) logger := zap.New(ecszap.WrapCore(core), zap.AddCaller()) ``` ```go config := zap.NewProductionConfig() config.EncoderConfig = ecszap.ECSCompatibleEncoderConfig(config.EncoderConfig) logger, err := config.Build(ecszap.WrapCoreOption(), zap.AddCaller()) ``` -------------------------------- ### Wrapping Existing Core Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Shows how to wrap an existing Zap core with ECS logging capabilities. ```go zapcore.NewCore(encoder, writer, level) ↓ WrapCore(core) ↓ zap.New(wrappedCore) ``` -------------------------------- ### Set Up Global Application Logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Initializes a global zap logger during application startup. This provides a convenient way to access the logger from anywhere in the application. ```go var globalLogger *zap.Logger func init() { var err error globalLogger, err = createProductionLogger() if err != nil { panic(err) } } func Log() *zap.Logger { return globalLogger } ``` -------------------------------- ### Using zap.Option for Configuration Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Demonstrates configuring an ECS-compatible logger using zap.Option, including custom encoder configurations and wrapping options. ```go zap.NewProductionConfig() ↓ ECSCompatibleEncoderConfig(config.EncoderConfig) ↓ config.Build(WrapCoreOption(), zap.AddCaller()) ``` -------------------------------- ### Log Configuration Loading Events Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Log the process of loading configuration files, including success and failure events. This provides visibility into the application's startup and configuration state. ```go func loadConfig(logger *zap.Logger) (*Config, error) { logger.Info("loading configuration") cfg, err := readConfigFile("config.yaml") if err != nil { logger.Error("failed to load configuration", zap.Error(err)) return nil, err } logger.Info("configuration loaded", zap.String("env", cfg.Environment), zap.String("version", cfg.Version), ) return cfg, nil } ``` -------------------------------- ### Recommended Logger Creation Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Illustrates the recommended sequence of function calls to create a default ECS-compatible logger with Zap. ```go NewDefaultEncoderConfig() ↓ NewCore(config, os.Stdout, zap.DebugLevel) ↓ zap.New(core, zap.AddCaller()) ``` -------------------------------- ### Default ECS JSON log output Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/index.md Example of the default JSON structure produced by the ECS logging encoder. ```json { "log.level": "info", "@timestamp": "2020-09-13T10:48:03.000Z", "message":" some logging info", "ecs.version": "1.6.0" } ``` -------------------------------- ### Printf-Style Logging with Sugar Logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Migrate from standard logging by using printf-style formatting with Sugar logger. ```go sugar := logger.Sugar() sugar.Infof("Server listening on %s:%d", "localhost", 8080) sugar.Warnf("Deprecated API called from %s", callerName) ``` -------------------------------- ### Run Tests with Go Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/README.md Execute all tests in the project using the go test command. ```bash go test ./... ``` -------------------------------- ### Use structured logging Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Add fields and logger names to structured logs. ```go // Add fields and a logger name logger = logger.With(zap.String("custom", "foo")) logger = logger.Named("mylogger") // Use strongly typed Field values logger.Info("some logging info", zap.Int("count", 17), zap.Error(errors.New("boom"))) ``` ```json { "log.level": "info", "@timestamp": "2020-09-13T10:48:03.000Z", "log.logger": "mylogger", "log.origin": { "file.name": "main/main.go", "file.line": 265 }, "message": "some logging info", "ecs.version": "1.6.0", "custom": "foo", "count": 17, "error": { "message":"boom" } } ``` -------------------------------- ### Wrap custom zapcore.Core Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Wrap an existing zapcore.Core to make it ECS compatible. ```go encoderConfig := ecszap.NewDefaultEncoderConfig() encoder := zapcore.NewJSONEncoder(encoderConfig.ToZapCoreEncoderConfig()) syslogCore := newSyslogCore(encoder, level) //create your own loggers core := ecszap.WrapCore(syslogCore) logger := zap.New(core, zap.AddCaller()) ``` -------------------------------- ### Use sugar logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Use the sugar logger for loosely typed logging. ```go sugar := logger.Sugar() sugar.Infow("some logging info", "foo", "bar", "count", 17, ) ``` ```json { "log.level": "info", "@timestamp": "2020-09-13T10:48:03.000Z", "log.logger": "mylogger", "log.origin": { "file.name": "main/main.go", "file.line": 311 }, "message": "some logging info", "ecs.version": "1.6.0", "custom": "foo", "foo": "bar", "count": 17 } ``` -------------------------------- ### Custom Name Encoding with ecszap Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/encoder-config.md Configure the logger name to be included in the output and customize its encoding. This example shows how to convert the logger name to uppercase. ```go import ( "strings" "go.elastic.co/ecszap" "go.uber.org/zap/zapcore" ) config := ecszap.EncoderConfig{ EnableName: true, EncodeName: func(name string, enc zapcore.PrimitiveArrayEncoder) { enc.AppendString(strings.ToUpper(name)) }, } // Logger named "myapp" outputs: "log.logger": "MYAPP" ``` -------------------------------- ### Wrap Core as Zap Option Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Configure zap's production settings and then wrap the core with ecszap's option for ECS compatibility during logger build. ```go config := zap.NewProductionConfig() config.EncoderConfig = ecszap.ECSCompatibleEncoderConfig(config.EncoderConfig) logger, err := config.Build(ecszap.WrapCoreOption()) ``` -------------------------------- ### Example Error Group Serialization Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/types.md Illustrates the expected ECS JSON format for errors implementing the errorGroup interface, showing a 'cause' array for underlying errors. ```json { "message": "primary error", "cause": [ {"message": "cause 1"}, {"message": "cause 2"} ] } ``` -------------------------------- ### Core Creation Functions Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Functions for creating and wrapping zapcore.Core instances. ```APIDOC ## Core Creation ### `NewCore` #### Signature `NewCore(cfg EncoderConfig, ws zapcore.WriteSyncer, enab zapcore.LevelEnabler) zapcore.Core` ### `WrapCore` #### Signature `WrapCore(c zapcore.Core) zapcore.Core` ### `WrapCoreOption` #### Signature `WrapCoreOption() zap.Option` ``` -------------------------------- ### Create ECS Logger (Recommended) Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Create a new ECS logger with default configuration, outputting to stdout at debug level. Includes caller information. ```go config := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(config, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller()) ``` -------------------------------- ### Basic Log Entry Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Log a simple message with a key-value pair. Ensure zap is imported. ```go logger.Info("message", zap.String("key", "value")) ``` -------------------------------- ### Adapt Existing Zap Config for ECS Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Modify an existing Zap configuration to be ECS compatible. This allows leveraging existing Zap setups while ensuring ECS compliance. ```go config := zap.NewProductionConfig() config.EncoderConfig = ecszap.ECSCompatibleEncoderConfig(config.EncoderConfig) logger, _ := config.Build() ``` -------------------------------- ### Log Error with Stack Trace using pkg/errors Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Log an error that includes a stack trace by wrapping it with context. Requires the 'pkg/errors' package. ```go import "github.com/pkg/errors" err := errors.Wrap(baseErr, "context") logger.Error("failed", zap.Error(err)) ``` -------------------------------- ### Custom Name Encoder Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Use EncodeName to provide a custom function for serializing the logger's hierarchical name. This is only used if EnableName is true. The example shows how to convert logger names to uppercase. ```go import ( "strings" "go.elastic.co/ecszap" "go.uber.org/zap/zapcore" ) // Make logger names uppercase config := ecszap.EncoderConfig{ EnableName: true, EncodeName: func(name string, enc zapcore.PrimitiveArrayEncoder) { enc.AppendString(strings.ToUpper(name)) }, } // Named logger "myapp" produces: "log.logger": "MYAPP" ``` -------------------------------- ### Create Default ECS Logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/index.md Use this to create a default ECS-compliant logger with standard configuration. Ensure to defer logger.Sync() for proper log flushing. ```go import ( "go.elastic.co/ecszap" "go.uber.org/zap" "os" ) // Create a default ECS-compliant logger config := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(config, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller()) def logger.Sync() // Ensure logs are flushed // Log messages logger.Info("application started", zap.String("version", "1.0.0")) ``` -------------------------------- ### Logging Errors with Stack Traces Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Utilize `github.com/pkg/errors` to wrap errors and include stack traces in your logs. This is crucial for debugging complex error chains. ```go import "github.com/pkg/errors" func fetchData() error { resp, err := http.Get("http://example.com") if err != nil { return errors.Wrap(err, "http request failed") } return nil } err := fetchData() logger.Error("data fetch failed", zap.Error(err)) ``` -------------------------------- ### Filebeat 7.16+ Filestream Input Configuration Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/integration-guide.md Configure Filebeat 7.16 and later to read logs from a file using the filestream input type. This setup includes ndjson parsing and common processors for enriching log data. ```yaml filebeat.inputs: - type: filestream paths: /var/log/app.log parsers: - ndjson: overwrite_keys: true add_error_key: true expand_keys: true processors: - add_host_metadata: ~ - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~ output.elasticsearch: hosts: ["elasticsearch:9200"] index: "logs-app-%{+yyyy.MM.dd}" ``` -------------------------------- ### Level Encoders (from zapcore) Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Configure how log levels are represented. Options include lowercase, uppercase, or colored output. ```go zapcore.LowercaseLevelEncoder // "debug", "info", "error" ``` ```go zapcore.CapitalLevelEncoder // "DEBUG", "INFO", "ERROR" ``` ```go zapcore.ColorLevelEncoder // ANSI colored (console only) ``` -------------------------------- ### Core Creation and Wrapping Functions Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/README.md Functions for creating and wrapping ECS-compliant cores, along with options for Zap logger configuration. ```APIDOC ## Core Creation and Wrapping Functions ### `NewCore()` **Description**: Creates an ECS-compliant core for Zap logger. ### `WrapCore()` **Description**: Wraps an existing Zap core with ECS conversion capabilities. ### `WrapCoreOption()` **Description**: A Zap option to apply ECS conversion to the logger. ### Core type methods **Description**: Methods available on the Core type for logger customization. - **With**: Adds structured data to the logger. - **Check**: Checks if a log message should be logged. - **Write**: Writes the log record. ``` -------------------------------- ### Create Loggers with Dynamic Log Levels Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Instantiate loggers with different core levels for various components. This allows fine-grained control over logging verbosity per module. ```go debugCore := ecszap.NewCore(config, os.Stdout, zap.DebugLevel) infoCore := ecszap.NewCore(config, os.Stdout, zap.InfoLevel) detailedLogger := zap.New(debugCore) normalLogger := zap.New(infoCore) detailedLogger.Debug("verbose operation details") // Logged normalLogger.Debug("verbose operation details") // Not logged ``` -------------------------------- ### Transition Existing Zap Config to ECS Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Applies ECS compatibility transformations to an existing Zap configuration and builds the logger with ECS wrapping. ```go import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) // Start with an existing production config config := zap.NewProductionConfig() // Apply ECS compatibility transformations config.EncoderConfig = ecszap.ECSCompatibleEncoderConfig(config.EncoderConfig) // Build the logger with ECS wrapping logger, err := config.Build(ecszap.WrapCoreOption(), zap.AddCaller()) if err != nil { panic(err) } defer logger.Sync() logger.Info("transitioned to ECS logging") ``` -------------------------------- ### Sugar Logger for Convenience Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Utilize the Sugar logger for more flexible, printf-style formatting and variadic key-value pairs. This can be more convenient for simple logging tasks. ```go sugar := logger.Sugar() sugar.Infow("structured log", "key", "value", "count", 42) sugar.Infof("formatted %s", "message") ``` -------------------------------- ### Enable Stack Trace in Output Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Set EnableStackTrace to true to include stack traces in the output under the 'log.origin.stack_trace' field. Defaults to true. ```go // Enabled (default) config := ecszap.EncoderConfig{ EnableStackTrace: true, } core := ecszap.NewCore(config, os.Stdout, zap.InfoLevel) logger := zap.New(core, zap.AddStacktrace(zap.ErrorLevel)) ``` ```go // Disabled config := ecszap.EncoderConfig{ EnableStackTrace: false, // No "log.origin.stack_trace" field } ``` -------------------------------- ### Import ECS Zap Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Import the necessary ecszap package to use ECS logging features. ```go import "go.elastic.co/ecszap" ``` -------------------------------- ### Configure ECS Zap Logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Set up a default or customized ECS logger using ecszap. ```go encoderConfig := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(encoderConfig, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller()) ``` ```go encoderConfig := ecszap.EncoderConfig{ EncodeName: customNameEncoder, EncodeLevel: zapcore.CapitalLevelEncoder, EncodeDuration: zapcore.MillisDurationEncoder, EncodeCaller: ecszap.FullCallerEncoder, } core := ecszap.NewCore(encoderConfig, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller()) ``` -------------------------------- ### Log with Multiple Fields Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Log a message with several key-value pairs to provide rich context. This is useful for tracking events like user logins. ```go logger.Info("user login", zap.String("username", "john"), zap.Int("user_id", 123), zap.Bool("success", true), ) ``` -------------------------------- ### Using Strongly-Typed Fields Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Utilize strongly-typed fields for improved performance and type safety in your logs. Ensure necessary imports are included. ```go import "go.uber.org/zap" logger.Info("user login", zap.String("username", "john.doe"), zap.String("ip_address", "192.168.1.100"), zap.Int("user_id", 12345), zap.Bool("success", true), zap.Duration("login_time", 234*time.Millisecond), zap.Float64("score", 98.5), ) ``` -------------------------------- ### core.Check Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Verifies whether the provided entry should be logged by comparing the log level against the configured level. If enabled, the core is added to the returned entry for later writing. ```APIDOC ## core.Check ### Description Verifies whether the provided entry should be logged by comparing the log level against the configured level. If enabled, the core is added to the returned entry for later writing. ### Signature ```go func (c core) Check(ent zapcore.Entry, ce *zapcore.CheckedEntry) *zapcore.CheckedEntry ``` ``` -------------------------------- ### Create Production Logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Configures a zap logger for production use with file rotation and caller information. Use this when initializing your application's primary logger. ```go func createProductionLogger() (*zap.Logger, error) { config := ecszap.NewDefaultEncoderConfig() // Use a file writer with rotation writer, err := getFileWriter("/var/log/app.log") if err != nil { return nil, err } // Production level is usually Info or above core := ecszap.NewCore(config, writer, zap.InfoLevel) // Add caller information and stack traces for errors logger := zap.New( core, zap.AddCaller(), zap.AddStacktrace(zap.ErrorLevel), ) return logger, nil } ``` -------------------------------- ### zapcore.Core Interface Implementation Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-index.md Defines the structure of the zapcore.Core interface, which is implemented by the internal core type for logging operations. ```go interface zapcore.Core { With(fields []zapcore.Field) zapcore.Core Check(ent zapcore.Entry, ce *zapcore.CheckedEntry) *zapcore.CheckedEntry Write(ent zapcore.Entry, fields []zapcore.Field) error Enabled(l zapcore.Level) bool Sync() error } ``` -------------------------------- ### Configure Kubernetes Pod Annotations Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Add these annotations to pods to ensure Filebeat correctly parses logs using hints-based autodiscover. ```yaml annotations: co.elastic.logs/json.overwrite_keys: true <1> co.elastic.logs/json.add_error_key: true <2> co.elastic.logs/json.expand_keys: true <3> ``` -------------------------------- ### Log Database Queries with Zap Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/integration-guide.md Create a database logger that logs query statements, parameters, and execution duration. This is useful for performance monitoring and debugging database interactions. ```go package db import ( "context" "database/sql" "time" "go.uber.org/zap" ) type Logger struct { db *sql.DB logger *zap.Logger } func (l *Logger) QueryRow(ctx context.Context, query string, args ...interface{}) *sql.Row { start := time.Now() row := l.db.QueryRowContext(ctx, query, args...) l.logger.Debug("database_query", zap.String("db.statement", query), zap.Any("db.params", args), zap.Duration("db.duration", time.Since(start)), ) return row } ``` -------------------------------- ### Use a Named Logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Create a logger with a specific name, useful for identifying the source of logs within different application modules. Ensure the logger is configured to output names. ```go authLogger := logger.Named("auth") authLogger.Info("user authenticated") ``` -------------------------------- ### With Method Signature Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Signature for the `With` method, which prepares fields for ECS-compliant logging. ```go func (c core) With(fields []zapcore.Field) zapcore.Core ``` -------------------------------- ### JSON Configuration for Caller Encoder with Go Zap Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Demonstrates how to configure the caller encoder using JSON unmarshaling, allowing 'full' or 'short' string values. ```go import ( "encoding/json" "go.elastic.co/ecszap" ) configJSON := []byte(`{ "enableCaller": true, "callerEncoder": "full" }`) var config ecszap.EncoderConfig json.Unmarshal(configJSON, &config) // config.EncodeCaller is now FullCallerEncoder ``` -------------------------------- ### Log Loosely Typed Fields with Sugar Logger Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Use Sugar logger for convenience when performance is not critical. Pass fields as key-value pairs. ```go sugar := logger.Sugar() sugar.Infow("user registration", "username", "alice", "email", "alice@example.com", "age", 25, ) sugar.Errorw("registration failed", "username", "bob", "error", err, "retry_count", 3, ) ``` -------------------------------- ### Dockerfile for ECS Logging Application Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/integration-guide.md A Dockerfile for building a Go application that utilizes ECS logging. It includes multi-stage builds for efficient image creation. ```dockerfile FROM golang:1.20-alpine AS builder WORKDIR /app COPY . . RUN go build -o app . FROM alpine:latest RUN apk add --no-cache ca-certificates COPY --from=builder /app/app /app/app ENTRYPOINT ["/app/app"] ``` -------------------------------- ### NewCore Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Creates a new zapcore.Core that uses an ECS-conformant JSON encoder. This is the safest and most straightforward way to create an ECS-compatible core, automatically converting error fields to ECS format and injecting the ECS version field. ```APIDOC ## NewCore ### Description Creates a new `zapcore.Core` that uses an ECS-conformant JSON encoder. ### Function Signature ```go func NewCore(cfg EncoderConfig, ws zapcore.WriteSyncer, enab zapcore.LevelEnabler) zapcore.Core ``` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | cfg | [EncoderConfig](#encoderconfig) | Yes | — | Configuration for ECS field encoding and serialization | | ws | zapcore.WriteSyncer | Yes | — | Output destination for log entries (e.g., os.Stdout, file) | | enab | zapcore.LevelEnabler | Yes | — | Level enabler that determines which log levels are written | ### Request Example ```go package main import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) func main() { config := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(config, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller()) defer logger.Sync() logger.Info("server started", zap.String("port", "8080")) } ``` ### Response #### Success Response (200) Returns `zapcore.Core` — A core that can be passed to `zap.New()` to create a logger. #### Response Example None explicitly provided, but the return type is `zapcore.Core`. ``` -------------------------------- ### Configure Docker Container Labels Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Add these labels to containers to ensure Filebeat correctly parses logs using hints-based autodiscover. ```yaml labels: co.elastic.logs/json.overwrite_keys: true <1> co.elastic.logs/json.add_error_key: true <2> co.elastic.logs/json.expand_keys: true <3> ``` -------------------------------- ### Minimal ECS Configuration Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Configures a logger with minimal settings, disabling caller and stack trace information, and logging at info level. ```go import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) config := ecszap.EncoderConfig{ EnableName: false, EnableCaller: false, EnableStackTrace: false, } core := ecszap.NewCore(config, os.Stdout, zap.InfoLevel) logger := zap.New(core) ``` -------------------------------- ### ECS Logger with Stack Traces in Go Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Enables stack traces for log levels error and above. Requires importing the errors package and configuring zap with AddStacktrace. ```go package main import ( "errors" "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) func main() { config := ecszap.NewDefaultEncoderConfig() core := ecszap.NewCore(config, os.Stdout, zap.DebugLevel) logger := zap.New(core, zap.AddCaller(), zap.AddStacktrace(zap.ErrorLevel)) defer logger.Sync() err := doSomething() if err != nil { logger.Error("operation failed", zap.Error(err)) } } func doSomething() error { return errors.New("something went wrong") } ``` -------------------------------- ### Implement HTTP Server Request Logging Middleware Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/usage-patterns.md Add middleware to your HTTP server to log incoming requests and their completion status. This helps in monitoring traffic and debugging issues. ```go import ( "net/http" "time" ) func loggingMiddleware(logger *zap.Logger) func(http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { start := time.Now() requestLogger := logger.With( zap.String("method", r.Method), zap.String("path", r.RequestURI), zap.String("remote_addr", r.RemoteAddr), ) requestLogger.Info("request received") next.ServeHTTP(w, r) duration := time.Since(start) requestLogger.Info("request completed", zap.Duration("duration", duration), ) }) } } ``` -------------------------------- ### Apply ECS Core Wrapping via Zap Option Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Use this zap.Option to integrate ECS formatting into existing zap logger configurations without manual core creation. It can be used with zap.New() or config.Build(). ```go package main import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) func main() { // Build a logger from a production config, wrapping the core for ECS config := zap.NewProductionConfig() config.EncoderConfig = ecszap.ECSCompatibleEncoderConfig(config.EncoderConfig) logger, err := config.Build(ecszap.WrapCoreOption(), zap.AddCaller()) if err != nil { panic(err) } defer logger.Sync() logger.Info("ECS logger built from production config") } ``` -------------------------------- ### Log errors Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/docs/reference/setup.md Log errors with stack traces using zap.Error. ```go err := errors.New("boom") logger.Error("some error", zap.Error(pkgerrors.Wrap(err, "crash"))) ``` ```json { "log.level": "error", "@timestamp": "2020-09-13T10:48:03.000Z", "log.logger": "mylogger", "log.origin": { "file.name": "main/main.go", "file.line": 290 }, "message": "some error", "ecs.version": "1.6.0", "custom": "foo", "error": { "message": "crash: boom", "stack_trace": "\nexample.example\n\t/Users/xyz/example/example.go:50\nruntime.example\n\t/Users/xyz/.gvm/versions/go1.13.8.darwin.amd64/src/runtime/proc.go:203\nruntime.goexit\n\t/Users/xyz/.gvm/versions/go1.13.8.darwin.amd64/src/runtime/asm_amd64.s:1357" } } ``` -------------------------------- ### Direct Zap Core Usage Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Use the ECS logger core directly with zap.New(). ```go zap.New(core) ``` -------------------------------- ### Zap Core with Stack Traces Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/api-reference/core.md Include stack traces for error-level logs when using the ECS logger core with zap.New(). ```go zap.New(core, zap.AddStacktrace()) ``` -------------------------------- ### Configure Caller Encoding with Go Zap Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Control how source file location information is serialized when EnableCaller is true. Choose between short relative paths or full absolute paths. ```go import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) // Short path (default) config := ecszap.EncoderConfig{ EnableCaller: true, EncodeCaller: ecszap.ShortCallerEncoder, } core := ecszap.NewCore(config, os.Stdout, zap.InfoLevel) logger := zap.New(core, zap.AddCaller()) logger.Info("message") // Produces: "log.origin.file.name": "myapp/main.go" ``` ```go import ( "os" "go.elastic.co/ecszap" "go.uber.org/zap" ) // Full path config := ecszap.EncoderConfig{ EnableCaller: true, EncodeCaller: ecszap.FullCallerEncoder, } // Produces: "log.origin.file.name": "/home/user/project/myapp/main.go" ``` -------------------------------- ### Configure Filebeat for JSON Root Keys Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/quick-reference.md Ensure Filebeat correctly parses JSON logs by setting 'json.keys_under_root' to true in its configuration. ```yaml ✗ json.keys_under_root: false ✓ json.keys_under_root: true ``` -------------------------------- ### Enable Caller Information in Output Source: https://github.com/elastic/ecs-logging-go-zap/blob/main/_autodocs/configuration.md Set EnableCaller to true to include source file location (file, line, function) in the output under the 'log.origin' object. Defaults to true. Requires zap.AddCaller() to be enabled on the logger. ```go // Enabled (default) - requires zap.AddCaller() config := ecszap.EncoderConfig{ EnableCaller: true, } core := ecszap.NewCore(config, os.Stdout, zap.InfoLevel) logger := zap.New(core, zap.AddCaller()) // Must add caller option ``` ```go // Disabled config := ecszap.EncoderConfig{ EnableCaller: false, // No "log.origin" field } ```