### Periodic Job Execution with Jitter in Go Source: https://github.com/medunes/go-kata/blob/master/01-context-cancellation-concurrency/14-leak-free-scheduler/README.md Illustrates how to execute a job periodically with a specified interval and jitter. This example demonstrates using time.NewTimer and resetting it to handle the next tick, incorporating random jitter to avoid synchronized executions. It also shows how to check for context cancellation. ```go package main import ( "context" "log/slog" "math/rand" "time" ) func runPeriodicJob(ctx context.Context, interval time.Duration, job func(context.Context) error) { // Seed the random number generator (ideally done once at application start) rand.Seed(time.Now().UnixNano()) ticker := time.NewTicker(interval) defer ticker.Stop() for { select { case <-ctx.Done(): slog.Info("Scheduler stopping") return case <-ticker.C: // Calculate jitter jitterAmount := time.Duration(rand.Intn(int(interval/10))) - interval/20 // ±10% nextTick := interval + jitterAmount // Create a new timer for the next tick with jitter timer := time.NewTimer(nextTick) select { case <-ctx.Done(): timer.Stop() // Ensure timer is stopped if context is cancelled before job runs slog.Info("Scheduler stopping before job execution") return case <-timer.C: start := time.Now() slog.Info("Running job") err := job(ctx) duration := time.Since(start) if err != nil { slog.Error("Job failed", "error", err, "duration", duration) } else { slog.Info("Job completed", "duration", duration) } } } } } ``` -------------------------------- ### Go: Demonstrating and Fixing the Typed Nil Interface Trap Source: https://context7.com/medunes/go-kata/llms.txt This Go code demonstrates the 'typed nil' pitfall where a nil pointer of a specific type is returned as an error, resulting in a non-nil error interface. It shows the incorrect way (`doThingBad`) and the correct way (`doThingGood`) to handle potential nil errors. It also includes an example of safe error extraction using `errors.As` and checks for the extracted error's nilness. ```go package main import ( "errors" "fmt" ) // MyError is a custom error type type MyError struct { Op string Code int } func (e *MyError) Error() string { return fmt.Sprintf("operation %s failed with code %d", e.Op, e.Code) } // BAD: Returns typed nil - caller's `err != nil` check will be TRUE func doThingBad() error { var e *MyError = nil // Typed nil pointer // ... some logic that doesn't set e ... return e // BUG: Returns (*MyError)(nil), not nil interface } // GOOD: Explicitly return nil when there's no error func doThingGood() error { var e *MyError // ... some logic ... if e == nil { return nil // Correct: returns nil interface } return e } // Demonstration of the bug and fix func ExampleTypedNilTrap() { // The trap err := doThingBad() fmt.Printf("doThingBad: err != nil is %v\n", err != nil) // true! fmt.Printf("doThingBad: type=%T, value=%#v\n", err, err) // *main.MyError, (*main.MyError)(nil) // The fix err = doThingGood() fmt.Printf("doThingGood: err != nil is %v\n", err != nil) // false // Safe extraction pattern with errors.As wrappedErr := fmt.Errorf("wrapped: %w", &MyError{Op: "save", Code: 500}) var me *MyError if errors.As(wrappedErr, &me) && me != nil { fmt.Printf("Extracted error: op=%s, code=%d\n", me.Op, me.Code) } } // Output: // doThingBad: err != nil is true // doThingBad: type=*main.MyError, value=(*main.MyError)(nil) // doThingGood: err != nil is false // Extracted error: op=save, code=500 // Key rules to avoid typed-nil bugs: // 1. Always return explicit `nil` when no error occurred // 2. Never return a typed pointer that might be nil as an error interface // 3. Use errors.As for extraction, and check the extracted value is non-nil // 4. In error factories, check `if ptr == nil { return nil }` before returning ``` -------------------------------- ### Implement APIClient with GetJSON in Go Source: https://github.com/medunes/go-kata/blob/master/03-http-middleware/16-http-client-hygiene/README.md Implement an APIClient struct and a GetJSON method in Go. This method should handle HTTP GET requests, decode JSON responses, and manage response bodies according to best practices for timeouts, cancellation, and connection reuse. ```go package main import ( "context" "encoding/json" "fmt" "io" "log/slog" "net/http" "time" ) const maxErrorBodySize = 4096 // Limit to read for error bodies type APIClient struct { client *http.Client logger *slog.Logger } func NewAPIClient(logger *slog.Logger) *APIClient { // Configure transport with timeouts and connection pooling transport := &http.Transport{ MaxIdleConns: 100, IdleConnTimeout: 90 * time.Second, DisableKeepAlives: false, TLSHandshakeTimeout: 10 * time.Second, // Add other transport-level timeouts if necessary } client := &http.Client{ Transport: transport, // Set a sensible overall client timeout Timeout: 30 * time.Second, } return &APIClient{ client: client, logger: logger, } } func (c *APIClient) GetJSON(ctx context.Context, url string, out any) error { startTime := time.Now() req, err := http.NewRequestWithContext(ctx, "GET", url, nil) if err != nil { return fmt.Errorf("failed to create request: %w", err) } resp, err := c.client.Do(req) if err != nil { // Log the error, potentially including context details if available c.logger.Error("HTTP GET request failed", "error", err, "url", url, "duration", time.Since(startTime)) return fmt.Errorf("HTTP GET request failed: %w", err) } defer resp.Body.Close() defer func() { // Drain the body to ensure connection reuse, especially for non-2xx responses if _, err := io.Copy(io.Discard, resp.Body); err != nil { c.logger.Warn("failed to drain response body", "error", err, "url", url, "status", resp.StatusCode) } }() statusOK := resp.StatusCode >= 200 && resp.StatusCode < 300 if !statusOK { // Read up to maxErrorBodySize bytes for error details bodyBytes, readErr := io.ReadAll(io.LimitReader(resp.Body, maxErrorBodySize)) var errorMsg string if readErr != nil { errorMsg = fmt.Sprintf("failed to read error body: %v", readErr) } else { errorMsg = string(bodyBytes) } c.logger.Error("non-2xx response received", "url", url, "status", resp.StatusCode, "body", errorMsg, "duration", time.Since(startTime)) return fmt.Errorf("API returned non-2xx status: %d, body: %s", resp.StatusCode, errorMsg) } // Decode JSON on 2xx responses if err := json.NewDecoder(resp.Body).Decode(out); err != nil { c.logger.Error("failed to decode JSON response", "url", url, "status", resp.StatusCode, "error", err, "duration", time.Since(startTime)) return fmt.Errorf("failed to decode JSON response: %w", err) } c.logger.Info("HTTP GET request successful", "url", url, "status", resp.StatusCode, "duration", time.Since(startTime)) return nil } func main() { // Example Usage: logger := slog.New(slog.NewTextHandler(os.Stdout, nil)) client := NewAPIClient(logger) type User struct { Name string `json:"name"` Age int `json:"age"` } var userData User ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() // Replace with a real API endpoint for testing err := client.GetJSON(ctx, "https://jsonplaceholder.typicode.com/users/1", &userData) if err != nil { logger.Error("Failed to get user data", "error", err) return } logger.Info("User data retrieved", "user", userData) } ``` -------------------------------- ### Go Rate-Limited Fan-Out Client Implementation Source: https://context7.com/medunes/go-kata/llms.txt Provides the core `FanOutClient` struct and its methods (`NewFanOutClient`, `FetchAll`, `fetchUser`) for making concurrent, rate-limited HTTP requests. It utilizes `context` for cancellation, `rate.Limiter` for QPS control, and `semaphore.Weighted` for managing in-flight requests. Dependencies include standard Go libraries and the `golang.org/x/sync` and `golang.org/x/time` packages. ```go package main import ( "context" "fmt" "io" "log/slog" "net/http" "sync" "time" "golang.org/x/sync/semaphore" "golang.org/x/time/rate" ) type FanOutClient struct { client *http.Client limiter *rate.Limiter sem *semaphore.Weighted logger *slog.Logger } func NewFanOutClient(qps float64, burst int, maxInFlight int64) *FanOutClient { return &FanOutClient{ client: &http.Client{ Timeout: 30 * time.Second, Transport: &http.Transport{ MaxIdleConns: 100, MaxIdleConnsPerHost: 10, IdleConnTimeout: 90 * time.Second, }, }, limiter: rate.NewLimiter(rate.Limit(qps), burst), sem: semaphore.NewWeighted(maxInFlight), logger: slog.Default(), } } func (c *FanOutClient) FetchAll(ctx context.Context, userIDs []int) (map[int][]byte, error) { results := make(map[int][]byte) var mu sync.Mutex var firstErr error var errOnce sync.Once ctx, cancel := context.WithCancel(ctx) defer cancel() var wg sync.WaitGroup for _, id := range userIDs { id := id // Capture for goroutine // Acquire semaphore (bounded concurrency) if err := c.sem.Acquire(ctx, 1); err != nil { return results, err } wg.Add(1) go func() { defer wg.Done() defer c.sem.Release(1) // Rate limit if err := c.limiter.Wait(ctx); err != nil { errOnce.Do(func() { firstErr = err; cancel() }) return } start := time.Now() data, err := c.fetchUser(ctx, id) latency := time.Since(start) if err != nil { c.logger.Error("fetch failed", "user_id", id, "latency", latency, "error", err) errOnce.Do(func() { firstErr = err; cancel() }) return } c.logger.Info("fetch success", "user_id", id, "latency", latency) mu.Lock() results[id] = data mu.Unlock() }() } wg.Wait() return results, firstErr } func (c *FanOutClient) fetchUser(ctx context.Context, userID int) ([]byte, error) { url := fmt.Sprintf("https://api.example.com/users/%d/widgets", userID) req, err := http.NewRequestWithContext(ctx, "GET", url, nil) if err != nil { return nil, err } resp, err := c.client.Do(req) if err != nil { return nil, err } defer resp.Body.Close() return io.ReadAll(resp.Body) } ``` -------------------------------- ### Go: Database Backup with Defer Cleanup and Error Preservation Source: https://context7.com/medunes/go-kata/llms.txt This Go function demonstrates a robust backup process for a database. It utilizes defer statements to ensure resources like files and database connections are properly closed, even if errors occur. Named return values and errors.Join are used to aggregate errors from the primary operation and cleanup steps, ensuring no error is lost. ```go package main import ( "context" "errors" "fmt" ) // Mock interfaces for database operations type DB interface { Connect(ctx context.Context) error Close() error Begin(ctx context.Context) (Tx, error) } type Tx interface { Query(ctx context.Context, query string) (Rows, error) Commit() error Rollback() error } type Rows interface { Next() bool Scan(dest ...interface{}) error Close() error } // BackupDatabase demonstrates proper defer cleanup with error preservation func BackupDatabase(ctx context.Context, db DB, filename string) (err error) { // 1. Open output file file, fileErr := openFile(filename) if fileErr != nil { return fmt.Errorf("open file: %w", fileErr) } defer func() { if closeErr := file.Close(); closeErr != nil { err = errors.Join(err, fmt.Errorf("close file: %w", closeErr)) } }() // 2. Connect to database if connErr := db.Connect(ctx); connErr != nil { return fmt.Errorf("connect: %w", connErr) } defer func() { if closeErr := db.Close(); closeErr != nil { err = errors.Join(err, fmt.Errorf("close db: %w", closeErr)) } }() // 3. Begin transaction tx, txErr := db.Begin(ctx) if txErr != nil { return fmt.Errorf("begin tx: %w", txErr) } // Track commit status for rollback decision var committed bool defer func() { if !committed { if rbErr := tx.Rollback(); rbErr != nil { err = errors.Join(err, fmt.Errorf("rollback: %w", rbErr)) } } }() // 4. Stream data rows, queryErr := tx.Query(ctx, "SELECT * FROM data") if queryErr != nil { return fmt.Errorf("query: %w", queryErr) } defer rows.Close() for rows.Next() { var data string if scanErr := rows.Scan(&data); scanErr != nil { return fmt.Errorf("scan: %w", scanErr) } if _, writeErr := file.Write([]byte(data + "\n")); writeErr != nil { return fmt.Errorf("write: %w", writeErr) } } // 5. Commit transaction if commitErr := tx.Commit(); commitErr != nil { return fmt.Errorf("commit: %w", commitErr) } committed = true return nil // Deferred functions may still add errors via errors.Join } // Mock file for example type mockFile struct{} func openFile(name string) (*mockFile, error) { return &mockFile{}, nil } func (f *mockFile) Write(p []byte) (int, error) { return len(p), nil } func (f *mockFile) Close() error { return nil } ``` -------------------------------- ### Idiomatic Resource Cleanup with Defer in Go Source: https://github.com/medunes/go-kata/blob/master/04-errors-semantics/19-defer-cleanup-chain/README.md Demonstrates the idiomatic Go pattern of deferring cleanup operations immediately after resource acquisition. This ensures that resources like files and database transactions are closed or rolled back in the correct LIFO order, even if errors occur during the main operation. It also shows how to preserve errors using `errors.Join`. ```go func BackupDatabase(ctx context.Context, dbURL, filename string) (err error) { file, err := os.Create(filename) if err != nil { return fmt.Errorf("failed to create file: %w", err) } defer func() { closeErr := file.Close() if err == nil { err = closeErr } else if closeErr != nil { err = errors.Join(err, fmt.Errorf("file close failed: %w", closeErr)) } }() db, err := sql.Open("postgres", dbURL) if err != nil { return fmt.Errorf("failed to connect to database: %w", err) } defer func() { dbCloseErr := db.Close() if err == nil { err = dbCloseErr } else if dbCloseErr != nil { err = errors.Join(err, fmt.Errorf("database close failed: %w", dbCloseErr)) } }() ttx, err := db.BeginTx(ctx, nil) if err != nil { return fmt.Errorf("failed to begin transaction: %w", err) } defer func() { if err == nil { if commitErr := ttx.Commit(); commitErr != nil { err = fmt.Errorf("transaction commit failed: %w", commitErr) } } else { if rollbackErr := ttx.Rollback(); rollbackErr != nil { err = errors.Join(err, fmt.Errorf("transaction rollback failed: %w", rollbackErr)) } } }() // Simulate streaming data for i := 0; i < 100; i++ { _, writeErr := file.WriteString(fmt.Sprintf("row %d\n", i)) if writeErr != nil { return fmt.Errorf("failed to write to file: %w", writeErr) } } return nil } ``` -------------------------------- ### Load Configuration Files using fs.FS and fs.WalkDir in Go Source: https://github.com/medunes/go-kata/blob/master/05-filesystems-packaging/13-iofs-config-loader/README.md Implements the `LoadConfigs` function to recursively walk a given root directory within an `fs.FS` and read all files ending with '.conf'. It returns a map where keys are file paths and values are file contents. This function adheres to idiomatic Go practices by accepting `fs.FS` for abstraction and utilizing `fs.WalkDir` and `fs.ReadFile` for file system operations, ensuring testability without touching the real filesystem. ```go package main import ( "io/fs" "path/filepath" ) func LoadConfigs(fsys fs.FS, root string) (map[string][]byte, error) { configs := make(map[string][]byte) err := fs.WalkDir(fsys, root, func(path string, d fs.DirEntry, err error) error { if err != nil { return err } if !d.IsDir() && filepath.Ext(path) == ".conf" { content, readErr := fs.ReadFile(fsys, path) if readErr != nil { return readErr } configs[path] = content } return nil }) if err != nil { return nil, err } return configs, nil } ``` -------------------------------- ### Load Configuration Files with io/fs Abstraction (Go) Source: https://context7.com/medunes/go-kata/llms.txt This Go function, LoadConfigs, reads configuration files with a '.conf' extension from a specified root directory within a given filesystem. It leverages the io/fs interface for flexibility, allowing it to work with various file sources like the actual disk (os.DirFS), embedded files, or in-memory test filesystems (fstest.MapFS). The function returns a map of file paths to their byte content. ```go package main import ( "io/fs" "path/filepath" "testing/fstest" ) // LoadConfigs walks a directory tree and returns all .conf file contents func LoadConfigs(fsys fs.FS, root string) (map[string][]byte, error) { configs := make(map[string][]byte) err := fs.WalkDir(fsys, root, func(path string, d fs.DirEntry, err error) error { if err != nil { return err } if d.IsDir() { return nil } if filepath.Ext(path) != ".conf" { return nil } data, err := fs.ReadFile(fsys, path) if err != nil { return err } configs[path] = data return nil }) return configs, err } // Production usage with os.DirFS: // configs, err := LoadConfigs(os.DirFS("/"), "etc/myapp") // Test usage with fstest.MapFS (no real filesystem needed): func ExampleLoadConfigs() { testFS := fstest.MapFS{ "config/database.conf": &fstest.MapFile{Data: []byte("host=localhost")}, "config/cache.conf": &fstest.MapFile{Data: []byte("redis://127.0.0.1")}, "config/readme.txt": &fstest.MapFile{Data: []byte("ignored")}, } configs, _ := LoadConfigs(testFS, "config") // configs["config/database.conf"] = []byte("host=localhost") // configs["config/cache.conf"] = []byte("redis://127.0.0.1") // readme.txt is ignored (not .conf) } // Embed usage: // //go:embed configs/* // var configFS embed.FS // configs, err := LoadConfigs(configFS, "configs") ``` -------------------------------- ### Normalize HTTP Header Keys with Go Testing (Table-Driven, Parallel, Fuzz) Source: https://context7.com/medunes/go-kata/llms.txt This Go code defines a function NormalizeHeaderKey to canonicalize HTTP header keys, ensuring consistent formatting by capitalizing the first letter of each word separated by hyphens. It includes robust testing using table-driven tests with parallel execution via t.Run and t.Parallel for efficiency. Additionally, it features a fuzz test to discover edge cases and ensure the function's idempotency and correctness under varied inputs. ```go package main import ( "strings" "testing" "unicode" ) // NormalizeHeaderKey canonicalizes HTTP header keys func NormalizeHeaderKey(s string) (string, error) { if s == "" { return "", nil } var result strings.Builder capitalize := true for _, r := range s { if !isValidHeaderChar(r) { return "", &InvalidCharError{Char: r} } if r == '-' { result.WriteRune('-') capitalize = true } else if capitalize { result.WriteRune(unicode.ToUpper(r)) capitalize = false } else { result.WriteRune(unicode.ToLower(r)) } } return result.String(), nil } func isValidHeaderChar(r rune) bool { return (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '-' } type InvalidCharError struct{ Char rune } func (e *InvalidCharError) Error() string { return "invalid character in header key" } // Table-driven test with parallel subtests func TestNormalizeHeaderKey(t *testing.T) { tests := []struct { name string input string want string wantErr bool }{ {"empty", "", "", false}, {"lowercase", "content-type", "Content-Type", false}, {"uppercase", "CONTENT-TYPE", "Content-Type", false}, {"mixed", "X-Custom-Header", "X-Custom-Header", false}, {"invalid_space", "content type", "", true}, {"invalid_unicode", "content-typ\u00e9", "", true}, } for _, tt := range tests { tt := tt // Capture range variable for parallel safety t.Run(tt.name, func(t *testing.T) { t.Parallel() got, err := NormalizeHeaderKey(tt.input) if (err != nil) != tt.wantErr { t.Errorf("NormalizeHeaderKey(%q) error = %v, wantErr %v", tt.input, err, tt.wantErr) return } if got != tt.want { t.Errorf("NormalizeHeaderKey(%q) = %q, want %q", tt.input, got, tt.want) } }) } } // Fuzz test for edge case discovery func FuzzNormalizeHeaderKey(f *testing.F) { // Seed corpus f.Add("content-type") f.Add("X-Custom-Header") f.Add("") f.Fuzz(func(t *testing.T, input string) { result, err := NormalizeHeaderKey(input) if err != nil { return // Invalid input is fine } // Invariant: result should be idempotent result2, err2 := NormalizeHeaderKey(result) if err2 != nil { t.Errorf("idempotence failed: NormalizeHeaderKey(%q) = %q, but NormalizeHeaderKey(%q) errored", input, result, result) } if result != result2 { t.Errorf("not idempotent: NormalizeHeaderKey(%q) = %q, NormalizeHeaderKey(%q) = %q", input, result, result, result2) } }) } // Run with: go test -fuzz=FuzzNormalizeHeaderKey -fuzztime=30s ``` -------------------------------- ### BackupDatabase Function Signature with Named Return Source: https://github.com/medunes/go-kata/blob/master/04-errors-semantics/19-defer-cleanup-chain/README.md Defines the function signature for `BackupDatabase` using a named return parameter `err`. This allows deferred functions to modify the error value directly, facilitating error preservation and composition. ```go func BackupDatabase(ctx context.Context, dbURL, filename string) (err error) ``` -------------------------------- ### Graceful Shutdown HTTP Server in Go Source: https://context7.com/medunes/go-kata/llms.txt An HTTP server in Go designed for graceful shutdown, ensuring in-flight requests are completed, background tasks are stopped, and resources are closed in the correct order upon receiving a SIGTERM signal. It utilizes context for managing shutdown and a sync.WaitGroup for coordinating goroutines. Dependencies include standard packages like 'context', 'log/slog', 'net/http', 'os', 'os/signal', 'sync', 'syscall', and 'time'. ```go package main import ( "context" "log/slog" "net/http" "os" "os/signal" "sync" "syscall" "time" ) type Server struct { httpServer *http.Server workerCount int jobs chan func() // Channel for background jobs wg sync.WaitGroup logger *slog.Logger } func NewServer(addr string, workers int) *Server { s := &Server{ workerCount: workers, jobs: make(chan func(), 100), // Buffered channel for jobs logger: slog.Default(), } mux := http.NewServeMux() mux.HandleFunc("/", s.handleRequest) // Simple request handler s.httpServer = &http.Server{Addr: addr, Handler: mux} return s } func (s *Server) handleRequest(w http.ResponseWriter, r *http.Request) { w.Write([]byte("OK")) } func (s *Server) Start(ctx context.Context) error { // Start worker pool for i := 0; i < s.workerCount; i++ { s.wg.Add(1) go func(id int) { defer s.wg.Done() for { select { case job, ok := <-s.jobs: if !ok { return // Channel closed, exit worker } job() // Execute the background job case <-ctx.Done(): return // Context cancelled, exit worker } } }(i) } // Start cache warmer ticker (example background task) ticker := time.NewTicker(30 * time.Second) s.wg.Add(1) go func() { defer s.wg.Done() defer ticker.Stop() for { select { case <-ticker.C: s.logger.Info("warming cache") case <-ctx.Done(): return } } }() // Start HTTP server in a goroutine go func() { if err := s.httpServer.ListenAndServe(); err != http.ErrServerClosed { s.logger.Error("server error", "error", err) } }() return nil } func (s *Server) Stop(ctx context.Context) error { // 1. Stop accepting new HTTP requests gracefully if err := s.httpServer.Shutdown(ctx); err != nil { s.logger.Error("http shutdown error", "error", err) // Consider returning the error or handling it as appropriate } // 2. Close jobs channel to signal workers to stop processing new jobs close(s.jobs) // 3. Wait for all workers (including cache warmer) to finish with a timeout done := make(chan struct{}) go func() { s.wg.Wait() // Wait for all goroutines managed by wg to complete close(done) }() select { case <-done: s.logger.Info("graceful shutdown complete") case <-ctx.Done(): s.logger.Warn("shutdown timeout, forcing exit") // Optionally, force exit or perform cleanup here if timeout occurs } return nil } // Usage: // ctx, cancel := context.WithCancel(context.Background()) // srv := NewServer(":8080", 4) // srv.Start(ctx) // // sigCh := make(chan os.Signal, 1) // signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGINT) // <-sigCh // Block until a signal is received // // shutdownCtx, _ := context.WithTimeout(context.Background(), 10*time.Second) // srv.Stop(shutdownCtx) ``` -------------------------------- ### Go HTTP Middleware for Reusing Buffers with sync.Pool Source: https://context7.com/medunes/go-kata/llms.txt An HTTP middleware that leverages `sync.Pool` to reuse `bytes.Buffer` instances for auditing request bodies. This significantly reduces memory allocations in performance-critical paths. It limits the amount of data read for auditing and ensures buffers are reset and returned to the pool. ```go package main import ( "bytes" "io" "log/slog" "net/http" "sync" ) var bufferPool = sync.Pool{ New: func() interface{} { return bytes.NewBuffer(make([]byte, 0, 16*1024)) // Pre-allocate 16KB }, } func AuditBody(maxBytes int, next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // Get buffer from pool buf := bufferPool.Get().(*bytes.Buffer) // Ensure cleanup: reset and return to pool defer func() { // Only keep reasonably-sized buffers in pool if buf.Cap() <= maxBytes*2 { buf.Reset() bufferPool.Put(buf) } }() // Read up to maxBytes for audit limited := io.LimitReader(r.Body, int64(maxBytes)) buf.ReadFrom(limited) // Log the captured body slog.Info("request audit", "method", r.Method, "path", r.URL.Path, "body_preview", buf.String(), "body_size", buf.Len(), ) // Reconstruct body: captured bytes + remainder r.Body = io.NopCloser(io.MultiReader(bytes.NewReader(buf.Bytes()), r.Body)) next.ServeHTTP(w, r) }) } // Usage: // mux := http.NewServeMux() // mux.HandleFunc("/api/data", handleData) // handler := AuditBody(16*1024, mux) // http.ListenAndServe(":8080", handler) // // Benchmark shows ~0 allocs/op in steady state due to buffer reuse ``` -------------------------------- ### Context-Aware Retry Policy in Go Source: https://context7.com/medunes/go-kata/llms.txt Implements a retry mechanism that respects context cancellation, avoids `time.Sleep` by reusing timers, and classifies errors to only retry transient failures. It uses exponential backoff with jitter. Dependencies include the standard Go `context`, `errors`, `fmt`, `math/rand`, `net`, and `time` packages. ```go package main import ( "context" "errors" "fmt" "math/rand" "net" "time" ) var ErrTransient = errors.New("transient error") type Retryer struct { MaxAttempts int BaseDelay time.Duration MaxDelay time.Duration Jitter func() time.Duration // Inject for testing } func NewRetryer(maxAttempts int, baseDelay, maxDelay time.Duration) *Retryer { return &Retryer{ MaxAttempts: maxAttempts, BaseDelay: baseDelay, MaxDelay: maxDelay, Jitter: func() time.Duration { return time.Duration(rand.Int63n(int64(baseDelay / 4))) }, } } func (r *Retryer) Do(ctx context.Context, fn func(context.Context) error) error { timer := time.NewTimer(0) if !timer.Stop() { <-timer.C } defer timer.Stop() var lastErr error for attempt := 0; attempt < r.MaxAttempts; attempt++ { if attempt > 0 { delay := r.backoff(attempt) timer.Reset(delay) select { case <-ctx.Done(): return fmt.Errorf("retry aborted after %d attempts: %w", attempt, ctx.Err()) case <-timer.C: } } lastErr = fn(ctx) if lastErr == nil { return nil } if !r.isTransient(lastErr) { return fmt.Errorf("non-retryable error on attempt %d: %w", attempt+1, lastErr) } } return fmt.Errorf("max retries (%d) exceeded: %w", r.MaxAttempts, lastErr) } func (r *Retryer) backoff(attempt int) time.Duration { delay := r.BaseDelay * (1 << attempt) // Exponential: base * 2^attempt if delay > r.MaxDelay { delay = r.MaxDelay } return delay + r.Jitter() } func (r *Retryer) isTransient(err error) bool { if errors.Is(err, ErrTransient) { return true } var netErr net.Error if errors.As(err, &netErr) && netErr.Timeout() { return true } return false } // Usage: // retryer := NewRetryer(3, 100*time.Millisecond, 2*time.Second) // err := retryer.Do(ctx, func(ctx context.Context) error { // return callFlakyService(ctx) // }) ``` -------------------------------- ### Scheduler Structure and Run Method in Go Source: https://github.com/medunes/go-kata/blob/master/01-context-cancellation-concurrency/14-leak-free-scheduler/README.md Defines the Scheduler struct and its Run method, which orchestrates the periodic execution of a job. The Run method accepts a context for cancellation and a job function to be executed. It must handle timer management, concurrency control, and context propagation. ```go package main import ( "context" "log/slog" "time" ) type Scheduler struct { // internal fields to manage timer, etc. } func (s *Scheduler) Run(ctx context.Context, job func(context.Context) error) error { // Implementation details for scheduling the job return nil } ``` -------------------------------- ### Parsing JSON Stream with json.Decoder in Go Source: https://github.com/medunes/go-kata/blob/master/02-performance-allocation/04-zero-allocation-json-parser/README.md Demonstrates how to parse specific fields from a JSON stream using `json.Decoder.Token()`. This approach avoids unmarshaling the entire JSON into memory, reducing allocations and GC pressure. It's suitable for processing large or continuous JSON data. ```go package main import ( "encoding/json" "io" "sync" ) type SensorData struct { SensorID string `json:"sensor_id"` Value float64 `json:"readings"` // Assuming first reading } var decoderPool = sync.Pool{ New: func() interface{} { // Pre-allocate buffer to avoid repeated allocations return json.NewDecoder(&io.LimitedReader{R: nil, N: 0}) }, } func ParseSensorData(r io.Reader) (SensorData, error) { decoder := decoderPool.Get().(*json.Decoder) // Reset the reader for the decoder // This part might need adjustment based on how you manage the underlying reader // For simplicity, assuming a new reader is passed each time or managed externally // A more robust solution would involve resetting the decoder's internal buffer if possible or re-creating it. // For this example, we'll simulate by creating a new decoder if the pool one isn't suitable or if we need to reset. // In a real-world scenario with sync.Pool, you'd typically manage the reader lifecycle outside the pool. // For demonstration, let's assume we can reset the reader or create a new decoder if needed. // A common pattern is to pass the reader to the pool object if the pool object holds the reader. // Here, we'll create a new decoder for clarity on the Token() usage. streamDecoder := json.NewDecoder(r) var data SensorData var err error for { token, err := streamDecoder.Token() if err == io.EOF { break } if err != nil { return SensorData{}, err // Malformed JSON } if token == nil { continue } switch key := token.(type) { case json.Delim: if key == '}' { // End of object, return if we have the data if data.SensorID != "" && data.Value != 0 { return data, nil } } case string: if key == "sensor_id" { token, err = streamDecoder.Token() if err != nil { return SensorData{}, err } if str, ok := token.(string); ok { data.SensorID = str } } else if key == "readings" { token, err = streamDecoder.Token() // Should be '[' if err != nil { return SensorData{}, err } if delim, ok := token.(json.Delim); ok && delim == '[' { // Read the first value from the array token, err = streamDecoder.Token() if err != nil { return SensorData{}, err } if num, ok := token.(float64); ok { data.Value = num } // Consume the rest of the array until ']' to avoid parsing issues for { token, err = streamDecoder.Token() if err != nil { return SensorData{}, err } if delim, ok := token.(json.Delim); ok && delim == ']' { break } } } } } } // If loop finishes without returning, it means EOF was reached before finding required fields if data.SensorID == "" || data.Value == 0 { return SensorData{}, io.ErrUnexpectedEOF } return data, nil } // Note: The sync.Pool usage for json.Decoder is complex due to its internal state. // In practice, managing the reader and decoder lifecycle might be simpler without pooling the decoder itself, // or by pooling a struct that holds the reader and decoder. // The primary goal here is demonstrating the Token() method for streaming. ``` -------------------------------- ### Go Fail-Fast Data Aggregator with errgroup and Functional Options Source: https://context7.com/medunes/go-kata/llms.txt Orchestrates concurrent data fetching from multiple services using `errgroup` for fail-fast cancellation and functional options for clean configuration. When any service fails, all other requests cancel immediately via context propagation. This pattern is useful for aggregating data from various sources efficiently. ```go package main import ( "context" "fmt" "log/slog" "time" "golang.org/x/sync/errgroup" ) // UserAggregator fetches data from multiple services concurrently type UserAggregator struct { timeout time.Duration logger *slog.Logger } // Option configures the aggregator using functional options pattern type Option func(*UserAggregator) func WithTimeout(d time.Duration) Option { return func(a *UserAggregator) { a.timeout = d } } func WithLogger(l *slog.Logger) Option { return func(a *UserAggregator) { a.logger = l } } func NewUserAggregator(opts ...Option) *UserAggregator { a := &UserAggregator{ timeout: 5 * time.Second, logger: slog.Default(), } for _, opt := range opts { opt(a) } return a } // Aggregate fetches profile and orders concurrently, cancels all on first error func (a *UserAggregator) Aggregate(ctx context.Context, userID int) (string, error) { ctx, cancel := context.WithTimeout(ctx, a.timeout) defer cancel() g, ctx := errgroup.WithContext(ctx) var profile, orders string g.Go(func() error { // Simulated profile fetch - replace with real service call select { case <-ctx.Done(): return ctx.Err() case <-time.After(100 * time.Millisecond): profile = "Name: Alice" return nil } }) g.Go(func() error { // Simulated orders fetch - replace with real service call select { case <-ctx.Done(): return ctx.Err() case <-time.After(150 * time.Millisecond): orders = "Orders: 5" return nil } }) if err := g.Wait(); err != nil { a.logger.Error("aggregation failed", "user_id", userID, "error", err) return "", fmt.Errorf("aggregate user %d: %w", userID, err) } return fmt.Sprintf("User: %s | %s", profile, orders), nil } // Usage: // agg := NewUserAggregator(WithTimeout(2*time.Second), WithLogger(slog.Default())) // result, err := agg.Aggregate(context.Background(), 42) // Expected output: "User: Name: Alice | Orders: 5" ```