### Run MultiCache Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/multicache/README.md Commands to start the Redis dependency and execute the Go example application. ```bash # Make sure Redis is running (if you want to test with Redis tier) docker run -d -p 6379:6379 redis:alpine # Run the example go run main.go ``` -------------------------------- ### Run Basic Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/basic/README.md Execute the basic httpcache example from the project root or the examples/basic directory. ```bash go run ./examples/basic/main.go ``` ```bash go run main.go ``` -------------------------------- ### Run Cache Key Headers Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/cachekeyheaders/README.md Command to execute the provided example application. ```bash cd examples/cachekeyheaders go run main.go ``` -------------------------------- ### Run Compression Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/compresscache/README.md Commands to execute the compression example from the project root or the specific directory. ```bash go run ./examples/compresscache/main.go ``` ```bash go run main.go ``` -------------------------------- ### Install PostgreSQL Cache Backend Source: https://github.com/sandrolain/httpcache/blob/master/postgresql/README.md Use go get to add the package to your project. ```bash go get github.com/sandrolain/httpcache/postgresql ``` -------------------------------- ### Install securecache Package Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/securecache/README.md Use go get to install the securecache package. ```bash go get github.com/sandrolain/httpcache/wrapper/securecache ``` -------------------------------- ### Run the Prometheus Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/prometheus/README.md Commands to execute the provided Prometheus integration example. ```bash cd examples/prometheus go run main.go ``` -------------------------------- ### Install MultiCache Package Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/multicache/README.md Use the Go toolchain to fetch the package. ```bash go get github.com/sandrolain/httpcache/wrapper/multicache ``` -------------------------------- ### Run Custom Cache Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/custom-backend/README.md Commands to execute the custom cache backend example from the project root or the specific directory. ```bash go run ./examples/custom-backend/main.go ``` ```bash go run main.go ``` -------------------------------- ### Run LevelDB Cache Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/leveldb/README.md Execute the LevelDB cache example from the project root or the examples/leveldb directory. ```bash go run ./examples/leveldb/main.go ``` ```bash go run main.go ``` -------------------------------- ### Run Redis Cache Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/redis/README.md Executes the Redis cache example from the project root or the examples/redis directory. ```bash go run ./examples/redis/main.go ``` ```bash go run main.go ``` -------------------------------- ### Run In-Memory Storage Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/blobcache/README.md Executes the example using the in-memory storage backend for testing purposes. ```bash cd examples/blobcache go run main.go ``` -------------------------------- ### Install and Run NATS Locally Source: https://github.com/sandrolain/httpcache/blob/master/natskv/README.md Commands to install the NATS server binary and start it with JetStream support. ```bash # Install NATS server go install github.com/nats-io/nats-server/v2@latest # Run with JetStream enabled nats-server -js ``` -------------------------------- ### Start PostgreSQL Server Source: https://github.com/sandrolain/httpcache/blob/master/examples/postgresql/README.md Commands to initialize a PostgreSQL instance using Docker or Homebrew. ```bash # Using Docker docker run --name postgres-cache -e POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres # Or using Homebrew on macOS brew services start postgresql ``` -------------------------------- ### Run SecureCache Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/security-best-practices/README.md Commands to execute the demonstration application with optional custom passphrase configuration. ```bash # Basic usage go run main.go # With custom passphrase CACHE_PASSPHRASE="your-secret-passphrase" go run main.go ``` -------------------------------- ### Execute Go Example Source: https://github.com/sandrolain/httpcache/blob/master/examples/hazelcast/README.md Runs the Go application that utilizes the Hazelcast cache. ```bash go run main.go ``` -------------------------------- ### Initialize BlobCache in Go Source: https://github.com/sandrolain/httpcache/blob/master/examples/blobcache/README.md Example of importing necessary drivers, configuring the cache, and integrating it with an HTTP client. ```go import ( "github.com/sandrolain/httpcache/blobcache" _ "gocloud.dev/blob/s3blob" // For AWS S3 // _ "gocloud.dev/blob/gcsblob" // For Google Cloud Storage // _ "gocloud.dev/blob/azureblob" // For Azure Blob Storage // _ "gocloud.dev/blob/memblob" // For in-memory (testing) ) ctx := context.Background() // Create cache cache, err := blobcache.New(ctx, blobcache.Config{ BucketURL: bucketURL, // Cloud storage URL KeyPrefix: "httpcache/", // Optional prefix for cache keys Timeout: 30 * time.Second, // Operation timeout }) if err != nil { log.Fatal(err) } defer cache.Close() // Use with HTTP client transport := httpcache.NewTransport(cache) client := &http.Client{Transport: transport} ``` -------------------------------- ### Example Output Source: https://github.com/sandrolain/httpcache/blob/master/examples/compresscache/README.md Sample output showing compression statistics for Gzip, Brotli, and Snappy. ```text === Gzip Compression (Level: BestSpeed) === Original size: 15360 bytes Compressed size: 5120 bytes Compression ratio: 0.33 Space savings: 66.67% === Brotli Compression (Level: 6) === Original size: 15360 bytes Compressed size: 4096 bytes Compression ratio: 0.27 Space savings: 73.33% === Snappy Compression === Original size: 15360 bytes Compressed size: 7680 bytes Compression ratio: 0.50 Space savings: 50.00% ``` -------------------------------- ### Example PromQL Queries Source: https://github.com/sandrolain/httpcache/blob/master/docs/monitoring.md Provides example PromQL queries for common monitoring scenarios like cache hit rate, latency, bandwidth saved, and traffic distribution. ```APIDOC ## Example PromQL Queries ### Cache Hit Rate ```promql rate(httpcache_cache_requests_total{result="hit"}[5m]) / rate(httpcache_cache_requests_total{operation="get"}[5m]) * 100 ``` ### P95 Latency ```promql histogram_quantile(0.95, rate(httpcache_cache_operation_duration_seconds_bucket[5m])) ``` ### Bandwidth Saved ```promql httpcache_http_response_size_bytes_total{cache_status="hit"} ``` ### Traffic Distribution ```promql sum by (cache_status) (rate(httpcache_http_requests_total[5m])) ``` ``` -------------------------------- ### Start MongoDB with Docker Source: https://github.com/sandrolain/httpcache/blob/master/examples/mongodb/README.md Run a MongoDB container instance for development purposes. ```bash docker run -d \ --name mongodb \ -p 27017:27017 \ -e MONGO_INITDB_ROOT_USERNAME=root \ -e MONGO_INITDB_ROOT_PASSWORD=password \ mongo:8 ``` -------------------------------- ### Implement basic HTTP caching with compression Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Example showing how to wrap a memory cache with Gzip and use it with an HTTP client. ```go package main import ( "compress/gzip" "fmt" "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/wrapper/compresscache" ) func main() { // Create base cache baseCache := httpcache.NewMemoryCache() // Wrap with gzip compression cache, err := compresscache.NewGzip(compresscache.GzipConfig{ Cache: baseCache, Level: gzip.BestSpeed, }) if err != nil { panic(err) } // Use with HTTP caching transport := httpcache.NewTransport(cache) client := transport.Client() // Make requests - responses are automatically compressed resp, _ := client.Get("https://api.example.com/data") defer resp.Body.Close() // Check compression statistics stats := cache.Stats() fmt.Printf("Compression ratio: %.2f%%\n", stats.SavingsPercent) } ``` -------------------------------- ### Install compresscache package Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Use this command to add the package to your Go project. ```bash go get github.com/sandrolain/httpcache/wrapper/compresscache ``` -------------------------------- ### Configure MinIO for Local Development Source: https://github.com/sandrolain/httpcache/blob/master/examples/blobcache/README.md Commands to start a local MinIO instance, create a bucket, and run the application with S3-compatible settings. ```bash docker run -p 9000:9000 -p 9001:9001 \ -e "MINIO_ROOT_USER=minioadmin" \ -e "MINIO_ROOT_PASSWORD=minioadmin" \ minio/minio server /data --console-address ":9001" ``` ```bash # Using MinIO Client (mc) mc alias set local http://localhost:9000 minioadmin minioadmin mc mb local/http-cache # Or via web console at http://localhost:9001 ``` ```bash export AWS_ACCESS_KEY_ID=minioadmin export AWS_SECRET_ACCESS_KEY=minioadmin export BUCKET_URL="s3://http-cache?endpoint=http://localhost:9000&s3ForcePathStyle=true®ion=us-east-1" go run main.go ``` -------------------------------- ### Configure PostgreSQL Cache with Brotli Source: https://github.com/sandrolain/httpcache/blob/master/examples/compresscache/README.md Example of wrapping a PostgreSQL cache with Brotli compression. ```go pgCache, _ := postgresql.New(postgresql.Config{ ConnectionString: "postgres://...", }) cache, _ := compresscache.NewBrotli(compresscache.BrotliConfig{ Cache: pgCache, Level: 8, // High compression }) ``` -------------------------------- ### Configure Brotli with S3 Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Example of using Brotli compression with an S3 cache backend. ```go cache, err := compresscache.NewBrotli(compresscache.BrotliConfig{ Cache: s3Cache, Level: 8, // High compression }) ``` -------------------------------- ### Basic Usage Example Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Demonstrates how to integrate the compresscache wrapper with the httpcache transport to automatically compress responses. ```APIDOC ## Basic Usage of Compress Cache Wrapper ### Description This example shows how to initialize a base cache, wrap it with a compression algorithm (e.g., Gzip), and then use this compressed cache with the `httpcache.Transport` to automatically handle compression and decompression of HTTP responses. ### Code Example ```go package main import ( "compress/gzip" "fmt" "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/wrapper/compresscache" ) func main() { // 1. Create a base cache implementation (e.g., in-memory) baseCache := httpcache.NewMemoryCache() // 2. Wrap the base cache with a compression algorithm (e.g., Gzip) cache, err := compresscache.NewGzip(compresscache.GzipConfig{ Cache: baseCache, Level: gzip.BestSpeed, }) if err != nil { panic(err) } // 3. Create an httpcache Transport using the compressed cache transport := httpcache.NewTransport(cache) client := transport.Client() // 4. Make HTTP requests. Responses will be automatically compressed/decompressed. resp, err := client.Get("https://api.example.com/data") if err != nil { panic(err) } defer resp.Body.Close() // 5. Access compression statistics (optional) stats := cache.Stats() fmt.Printf("Compression ratio: %.2f%% savings\n", stats.SavingsPercent) } ``` ### Explanation - The `httpcache.NewMemoryCache()` creates a simple in-memory cache. - `compresscache.NewGzip()` wraps the `baseCache` with Gzip compression. - `httpcache.NewTransport(cache)` integrates the compressed cache into the HTTP client. - When `client.Get()` is called, the `httpcache` transport uses the `cache` wrapper. If a response is cacheable, it will be compressed before being stored. If retrieved from the cache, it will be decompressed. - `cache.Stats()` provides metrics on the compression effectiveness. ``` -------------------------------- ### Configure Private vs Public Cache Source: https://github.com/sandrolain/httpcache/blob/master/docs/advanced-features.md Examples demonstrating the difference in caching behavior between private and public cache modes. ```go transport := httpcache.NewMemoryCacheTransport() // transport.IsPublicCache = false // Default client := transport.Client() // Response: Cache-Control: private, max-age=3600 resp, _ := client.Get("https://api.example.com/user/profile") // ✅ Response is cached (private caches can cache private responses) // Second request resp, _ = client.Get("https://api.example.com/user/profile") // Returns from cache (X-From-Cache: 1) ``` ```go transport := httpcache.NewMemoryCacheTransport() transport.IsPublicCache = true // Shared cache mode client := transport.Client() // Response: Cache-Control: private, max-age=3600 resp, _ := client.Get("https://api.example.com/user/profile") // ❌ Response is NOT cached (public caches must not cache private responses) // Second request resp, _ = client.Get("https://api.example.com/user/profile") // Makes a fresh request to the server (not from cache) // Response: Cache-Control: public, max-age=3600 resp, _ = client.Get("https://api.example.com/public/data") // ✅ Response is cached (public caches can cache public responses) ``` -------------------------------- ### Fetch Metrics via Curl Source: https://github.com/sandrolain/httpcache/blob/master/examples/prometheus/README.md Retrieve current metrics from the running example server. ```bash curl http://localhost:9090/metrics ``` -------------------------------- ### Install NATS JetStream Cache Backend Source: https://github.com/sandrolain/httpcache/blob/master/natskv/README.md Use this command to add the package to your Go project. ```bash go get github.com/sandrolain/httpcache/natskv ``` -------------------------------- ### Basic Memory Cache Setup Source: https://github.com/sandrolain/httpcache/blob/master/examples/README.md Initializes a basic memory cache transport and obtains the HTTP client. Suitable for simple caching needs where persistence is not required. ```go transport := httpcache.NewMemoryCacheTransport() client := transport.Client() ``` -------------------------------- ### Configure Redis Cache with Gzip Source: https://github.com/sandrolain/httpcache/blob/master/examples/compresscache/README.md Example of wrapping a Redis cache with Gzip compression. ```go redisCache, _ := redis.New("localhost:6379") cache, _ := compresscache.NewGzip(compresscache.GzipConfig{ Cache: redisCache, Level: gzip.BestSpeed, }) ``` -------------------------------- ### Start Hazelcast Server via Docker Source: https://github.com/sandrolain/httpcache/blob/master/examples/hazelcast/README.md Launches a Hazelcast instance on port 5701 using Docker. ```bash docker run -p 5701:5701 hazelcast/hazelcast:5.5 ``` -------------------------------- ### Create CDN-like Caching Hierarchy Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/multicache/README.md Demonstrates setting up a multi-tier cache hierarchy for a CDN-like architecture. Tiers are ordered from fastest to slowest. This setup involves memory, Redis, and PostgreSQL caches. ```go // Create a CDN-like caching hierarchy edge := httpcache.NewMemoryCache() // Edge cache (fast, small) regional := redis.New("regional:6379") // Regional cache (medium) origin := postgresql.New("origin-db") // Origin cache (persistent) cdn := multicache.New(edge, regional, origin) // First request: Miss in all tiers, fetch from upstream, store everywhere // Second request from same edge: Hit in edge (fastest) // Request from different edge: Miss in edge, hit in regional, promote to edge // Request after edge restart: Miss in edge, hit in regional, promote to edge ``` -------------------------------- ### Configure Per-User Caching Source: https://github.com/sandrolain/httpcache/blob/master/examples/cachekeyheaders/README.md Example of setting up an API client with per-user caching using the Authorization header. ```go // API client with per-user caching transport := httpcache.NewMemoryCacheTransport() transport.CacheKeyHeaders = []string{"Authorization"} client := transport.Client() // Each user gets their own cached responses req1, _ := http.NewRequest("GET", "https://api.example.com/user/profile", nil) req1.Header.Set("Authorization", "Bearer user1_token") resp1, _ := client.Do(req1) req2, _ := http.NewRequest("GET", "https://api.example.com/user/profile", nil) req2.Header.Set("Authorization", "Bearer user2_token") resp2, _ := client.Do(req2) // user1 and user2 have separate cache entries ``` -------------------------------- ### Age Header Validation Examples Source: https://github.com/sandrolain/httpcache/blob/master/docs/how-it-works.md Demonstrates how the system handles malformed or multiple Age headers. ```go // Example: Multiple Age headers // Age: 300 // Age: 600 // Result: Uses 300 (first value), logs warning // Example: Invalid Age header // Age: -100 // Result: Ignores header, logs warning // Example: Non-numeric Age header // Age: invalid // Result: Ignores header, logs warning ``` -------------------------------- ### Configure Cache Memory Size Source: https://github.com/sandrolain/httpcache/blob/master/examples/freecache/README.md Examples of initializing freecache with different memory allocations based on expected entry volume. ```go // Small cache (10MB) - ~10k cached responses cache := freecache.New(10 * 1024 * 1024) // Medium cache (100MB) - ~100k cached responses cache := freecache.New(100 * 1024 * 1024) // Large cache (1GB) - ~1M cached responses cache := freecache.New(1024 * 1024 * 1024) debug.SetGCPercent(10) // Reduce GC frequency for large caches ``` -------------------------------- ### Install httpcache via Go Modules Source: https://github.com/sandrolain/httpcache/blob/master/README.md Command to add the httpcache package to a Go project. ```bash go get github.com/sandrolain/httpcache ``` -------------------------------- ### Multi-Tier Cache Implementation Source: https://github.com/sandrolain/httpcache/blob/master/examples/compresscache/README.md Example of combining an uncompressed memory cache with a compressed Redis cache. ```go // Fast tier: uncompressed memory memCache := httpcache.NewMemoryCache() // Slow tier: compressed Redis redisCache, _ := redis.New("localhost:6379") compressedRedis, _ := compresscache.NewGzip(compresscache.GzipConfig{ Cache: redisCache, Level: gzip.BestSpeed, }) // Combine tiers cache := multicache.New(memCache, compressedRedis) ``` -------------------------------- ### Example Scenario: Differentiating Cache Entries with Authorization Header Source: https://github.com/sandrolain/httpcache/blob/master/docs/advanced-features.md Demonstrates how different Authorization header values result in separate cache entries. Subsequent requests with the same header values will result in cache hits. ```go transport := httpcache.NewMemoryCacheTransport() transport.CacheKeyHeaders = []string{"Authorization"} client := transport.Client() // Request 1: Authorization: Bearer token1 req1, _ := http.NewRequest("GET", "https://api.example.com/user/profile", nil) req1.Header.Set("Authorization", "Bearer token1") resp1, _ := client.Do(req1) // Cache miss, fetches from server io.Copy(io.Discard, resp1.Body) resp1.Body.Close() // Request 2: Authorization: Bearer token2 (different token) req2, _ := http.NewRequest("GET", "https://api.example.com/user/profile", nil) req2.Header.Set("Authorization", "Bearer token2") resp2, _ := client.Do(req2) // Cache miss, fetches from server (different cache entry) io.Copy(io.Discard, resp2.Body) resp2.Body.Close() // Request 3: Authorization: Bearer token1 (same as request 1) req3, _ := http.NewRequest("GET", "https://api.example.com/user/profile", nil) req3.Header.Set("Authorization", "Bearer token1") resp3, _ := client.Do(req3) // Cache hit! Serves cached response from request 1 io.Copy(io.Discard, resp3.Body) resp3.Body.Close() fmt.Println(resp3.Header.Get(httpcache.XFromCache)) // "1" ``` -------------------------------- ### Initialize BlobCache for Cloud Storage (AWS S3 Example) Source: https://github.com/sandrolain/httpcache/blob/master/docs/backends.md Suitable for cloud-native applications, multi-cloud deployments, and serverless functions. Supports AWS S3, Google Cloud Storage, and Azure Blob Storage. Configure credentials via environment variables. ```go import ( "github.com/sandrolain/httpcache/blobcache" _ "gocloud.dev/blob/s3blob" // For AWS S3 // _ "gocloud.dev/blob/gcsblob" // For Google Cloud Storage // _ "gocloud.dev/blob/azureblob" // For Azure Blob Storage ) ctx := context.Background() // AWS S3 cache, _ := blobcache.New(ctx, blobcache.Config{ BucketURL: "s3://my-bucket?region=us-east-1", KeyPrefix: "httpcache/", Timeout: 30 * time.Second, }) // Google Cloud Storage // cache, _ := blobcache.New(ctx, blobcache.Config{ // BucketURL: "gs://my-bucket", // KeyPrefix: "httpcache/", // }) // Azure Blob Storage // cache, _ := blobcache.New(ctx, blobcache.Config{ // BucketURL: "azblob://my-container", // KeyPrefix: "httpcache/", // }) defer cache.(interface{ Close() error }).Close() transport := httpcache.NewTransport(cache) client := &http.Client{Transport: transport} ``` -------------------------------- ### Custom Cache Backend Setup Source: https://github.com/sandrolain/httpcache/blob/master/examples/README.md Integrates a custom cache backend with the httpcache transport. Requires a custom cache implementation that satisfies the cache interface. ```go cache := customcache.New() transport := httpcache.NewTransport(cache) client := &http.Client{Transport: transport} ``` -------------------------------- ### Cache Key Format Examples Source: https://github.com/sandrolain/httpcache/blob/master/docs/advanced-features.md Illustrates the difference in cache key format with and without `CacheKeyHeaders` configured. Headers are appended to the URL, separated by '|', and sorted alphabetically. ```text Without CacheKeyHeaders: http://api.example.com/data ``` ```text With CacheKeyHeaders: http://api.example.com/data|Accept-Language:en|Authorization:Bearer token1 ``` -------------------------------- ### Run NATS Server with JetStream Source: https://github.com/sandrolain/httpcache/blob/master/examples/natskv/README.md Start a NATS server with JetStream enabled using Docker. This is a prerequisite for the example. ```bash docker run -p 4222:4222 nats:latest -js ``` -------------------------------- ### Initialize and Use Freecache Backend Source: https://github.com/sandrolain/httpcache/blob/master/examples/freecache/README.md Demonstrates setting up a 100MB cache, configuring the HTTP transport, and accessing cache statistics. ```go package main import ( "fmt" "io" "net/http" "runtime/debug" "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/freecache" ) func main() { // Create a 100MB cache // For large caches, reduce GC percentage for better performance cache := freecache.New(100 * 1024 * 1024) debug.SetGCPercent(20) // Create HTTP transport with the cache transport := httpcache.NewTransport(cache) client := transport.Client() // Make HTTP requests resp, err := client.Get("https://api.example.com/data") if err != nil { panic(err) } defer resp.Body.Close() // Read response body, err := io.ReadAll(resp.Body) if err != nil { panic(err) } fmt.Printf("Response: %s\n", body) fmt.Printf("From cache: %s\n", resp.Header.Get("X-From-Cache")) // Check cache statistics fmt.Printf("Hit rate: %.2f%%\n", cache.HitRate()*100) fmt.Printf("Entries: %d\n", cache.EntryCount()) fmt.Printf("Evictions: %d\n", cache.EvacuateCount()) } ``` -------------------------------- ### Quick Start: Prometheus Integration Source: https://github.com/sandrolain/httpcache/blob/master/docs/monitoring.md This snippet demonstrates the basic setup for integrating Prometheus metrics with httpcache by wrapping the cache and transport layers. ```APIDOC ## Quick Start: Prometheus Integration This snippet demonstrates the basic setup for integrating Prometheus metrics with httpcache by wrapping the cache and transport layers. ### Code Example ```go import ( "github.com/sandrolain/httpcache" prommetrics "github.com/sandrolain/httpcache/wrapper/metrics/prometheus" ) // Create metrics collector collector := prommetrics.NewCollector() // Wrap your cache cache := httpcache.NewMemoryCache() instrumentedCache := prommetrics.NewInstrumentedCache(cache, "memory", collector) // Wrap your transport transport := httpcache.NewTransport(instrumentedCache) instrumentedTransport := prommetrics.NewInstrumentedTransport(transport, collector) // Use the instrumented client client := instrumentedTransport.Client() // Metrics are automatically exposed via Prometheus default registry // Access them at your /metrics endpoint ``` ``` -------------------------------- ### Initialize Compression Algorithms Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Configure different compression strategies based on throughput or storage requirements. ```go // High-throughput API (prioritize speed) cache, _ := compresscache.NewSnappy(compresscache.SnappyConfig{ Cache: baseCache, }) // Storage-optimized (prioritize space) cache, _ := compresscache.NewBrotli(compresscache.BrotliConfig{ Cache: baseCache, Level: 8, }) // Balanced general purpose cache, _ := compresscache.NewGzip(compresscache.GzipConfig{ Cache: baseCache, Level: gzip.BestSpeed, }) ``` -------------------------------- ### Integration with Existing Prometheus Setup Source: https://github.com/sandrolain/httpcache/blob/master/docs/monitoring.md Integrate httpcache metrics into an existing Prometheus setup by using the default registry, which automatically includes httpcache metrics at the /metrics endpoint. ```go // Your existing Prometheus setup http.Handle("/metrics", promhttp.Handler()) // httpcache metrics use the default registry collector := prommetrics.NewCollector() // All metrics are exposed together at /metrics ``` -------------------------------- ### Quick Start: Instrumenting Cache and Transport Source: https://github.com/sandrolain/httpcache/blob/master/docs/monitoring.md Wrap your existing cache and transport with Prometheus metrics collectors to start monitoring. Metrics are automatically exposed via the default Prometheus registry. ```go import ( "github.com/sandrolain/httpcache" prommetrics "github.com/sandrolain/httpcache/wrapper/metrics/prometheus" ) // Create metrics collector collector := prommetrics.NewCollector() // Wrap your cache cache := httpcache.NewMemoryCache() instrumentedCache := prommetrics.NewInstrumentedCache(cache, "memory", collector) // Wrap your transport transport := httpcache.NewTransport(instrumentedCache) instrumentedTransport := prommetrics.NewInstrumentedTransport(transport, collector) // Use the instrumented client client := instrumentedTransport.Client() // Metrics are automatically exposed via Prometheus default registry // Access them at your /metrics endpoint ``` -------------------------------- ### Install Prometheus Dependencies Source: https://github.com/sandrolain/httpcache/blob/master/examples/prometheus/README.md Required packages for Prometheus metrics collection. ```bash go get github.com/prometheus/client_golang/prometheus go get github.com/prometheus/client_golang/prometheus/promhttp ``` -------------------------------- ### Initialize Snappy Compression Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Create a new Snappy compression wrapper for an existing cache instance. ```go cache, err := compresscache.NewSnappy(compresscache.SnappyConfig{ Cache: memCache, }) ``` -------------------------------- ### Initialize Snappy compression Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Configure a Snappy cache wrapper, which does not support configurable compression levels. ```go cache, err := compresscache.NewSnappy(compresscache.SnappyConfig{ Cache: baseCache, }) ``` -------------------------------- ### Initialize Multi-Tier Cache Source: https://github.com/sandrolain/httpcache/blob/master/docs/advanced-features.md Set up a multi-tier caching system by combining different cache backends like memory, disk, and Redis. The order of tiers is crucial for performance and fallback. ```go import "github.com/sandrolain/httpcache/wrapper/multicache" // Create individual cache tiers memCache := httpcache.NewMemoryCache() // Fast, volatile diskCache := diskcache.New("/tmp/cache") // Medium, persistent redisCache, _ := redis.New("localhost:6379") // Distributed, shared // Combine into multi-tier cache (order matters!) mc := multicache.New(memCache, diskCache, redisCache) transport := httpcache.NewTransport(mc) client := &http.Client{Transport: transport} ``` -------------------------------- ### Initialize Cache with New() Source: https://github.com/sandrolain/httpcache/blob/master/natskv/README.md Recommended approach for most use cases, as it manages the NATS connection internally. ```go package main import ( "context" "time" "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/natskv" ) func main() { ctx := context.Background() // Create cache with automatic connection management cache, err := natskv.New(ctx, natskv.Config{ NATSUrl: "nats://localhost:4222", // Optional, defaults to nats.DefaultURL Bucket: "http-cache", Description: "HTTP response cache", TTL: 24 * time.Hour, // Cache entries expire after 24 hours }) if err != nil { panic(err) } defer cache.(interface{ Close() error }).Close() // Use the cache with httpcache transport := httpcache.NewTransport(cache) client := transport.Client() resp, err := client.Get("https://example.com") // ... } ``` -------------------------------- ### Initialize MultiCache with HTTP Client Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/multicache/README.md Configure a multi-tier cache by passing multiple cache backends to the constructor, then integrate it with an HTTP client. ```go package main import ( "net/http" httpcache "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/diskcache" "github.com/sandrolain/httpcache/redis" "github.com/sandrolain/httpcache/wrapper/multicache" ) func main() { // Create individual cache tiers memCache := httpcache.NewMemoryCache() diskCache := diskcache.New("/tmp/cache") redisCache, _ := redis.New("localhost:6379") // Combine into multi-tier cache (order matters!) mc := multicache.New( memCache, // Tier 1: fastest, checked first diskCache, // Tier 2: medium speed redisCache, // Tier 3: slowest, checked last ) // Use with HTTP caching transport := httpcache.NewTransport(mc) client := &http.Client{Transport: transport} } ``` -------------------------------- ### Compression Performance Settings Source: https://github.com/sandrolain/httpcache/blob/master/examples/compresscache/README.md Recommended configurations for different performance and storage requirements. ```go cache, _ := compresscache.NewSnappy(compresscache.SnappyConfig{ Cache: baseCache, }) ``` ```go cache, _ := compresscache.NewBrotli(compresscache.BrotliConfig{ Cache: baseCache, Level: 8, }) ``` ```go cache, _ := compresscache.NewGzip(compresscache.GzipConfig{ Cache: baseCache, Level: gzip.BestSpeed, }) ``` -------------------------------- ### Initialize Cache with Different Connection Methods Source: https://github.com/sandrolain/httpcache/blob/master/postgresql/README.md Connect to PostgreSQL using automatic pooling, an existing pgxpool, or a single connection. ```go ctx := context.Background() connString := "postgres://user:pass@localhost:5432/mydb?sslmode=disable" cache, err := postgresql.New(ctx, connString, nil) if err != nil { log.Fatal(err) } defer cache.Close() ``` ```go import "github.com/jackc/pgx/v5/pgxpool" pool, err := pgxpool.New(ctx, connString) if err != nil { log.Fatal(err) } config := postgresql.DefaultConfig() cache, err := postgresql.NewWithPool(pool, config) if err != nil { log.Fatal(err) } ``` ```go import "github.com/jackc/pgx/v5" conn, err := pgx.Connect(ctx, connString) if err != nil { log.Fatal(err) } cache, err := postgresql.NewWithConn(conn, nil) if err != nil { log.Fatal(err) } ``` -------------------------------- ### Integrate httpcache with Prometheus Source: https://github.com/sandrolain/httpcache/blob/master/examples/prometheus/README.md Setup the collector using either the default or a custom registry. ```go // Use the default Prometheus registry (shared with your app) collector := prommetrics.NewCollector() // Or use a custom registry customRegistry := prometheus.NewRegistry() collector := prommetrics.NewCollectorWithConfig(prommetrics.CollectorConfig{ Registry: customRegistry, Namespace: "myapp", // Custom namespace instead of "httpcache" }) // Your existing metrics endpoint will include httpcache metrics http.Handle("/metrics", promhttp.Handler()) ``` -------------------------------- ### Initialize PostgreSQL Cache Source: https://github.com/sandrolain/httpcache/blob/master/postgresql/README.md Create a new cache instance and integrate it with an HTTP transport. ```go import ( "context" "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/postgresql" ) ctx := context.Background() // Create cache with default configuration cache, err := postgresql.New(ctx, "postgres://user:pass@localhost/mydb", nil) if err != nil { log.Fatal(err) } defer cache.Close() // Use with HTTP transport transport := httpcache.NewTransport(cache) client := transport.Client() ``` -------------------------------- ### Understand must-understand Directive Source: https://github.com/sandrolain/httpcache/blob/master/docs/how-it-works.md Examples of how the must-understand directive interacts with status codes and cache-control headers. ```http // Server response: HTTP/1.1 200 OK Cache-Control: must-understand, no-store, max-age=3600 Content-Type: application/json // Result: Response IS cached because: // - Status 200 is understood // - must-understand is present // - must-understand overrides no-store for understood status codes ``` ```http // Server response: HTTP/1.1 418 I'm a teapot Cache-Control: must-understand, max-age=3600 Content-Type: text/plain // Result: Response is NOT cached because: // - Status 418 is NOT in the understood list // - must-understand requires understood status for caching ``` -------------------------------- ### Initialize a Cached HTTP Client in Go Source: https://github.com/sandrolain/httpcache/blob/master/README.md Demonstrates creating a memory-backed transport and using it to perform cached HTTP requests. ```go package main import ( "fmt" "io" "net/http" "github.com/sandrolain/httpcache" ) func main() { // Create a cached HTTP client transport := httpcache.NewMemoryCacheTransport() client := transport.Client() // Make requests - second request will be cached! resp, _ := client.Get("https://example.com") io.Copy(io.Discard, resp.Body) resp.Body.Close() // Check if response came from cache if resp.Header.Get(httpcache.XFromCache) == "1" { fmt.Println("Response was cached!") } } ``` -------------------------------- ### Initialize Gzip compression Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/compresscache/README.md Configure a Gzip cache wrapper with a specific compression level. ```go cache, err := compresscache.NewGzip(compresscache.GzipConfig{ Cache: baseCache, Level: gzip.BestSpeed, // -2 to 9 }) ``` -------------------------------- ### Define MongoDB Document Structure Source: https://github.com/sandrolain/httpcache/blob/master/examples/mongodb/README.md Example of the JSON document format stored in the MongoDB collection. ```json { "_id": "cache:http://example.com/api/data", "data": BinData(...), // Response data as binary "createdAt": ISODate("2024-01-01T00:00:00Z") } ``` -------------------------------- ### Define Cache Sizing by Tier Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/multicache/README.md Example of sizing different cache backends based on their performance characteristics. ```go // Tier 1: Small, holds only hot data memCache := httpcache.NewMemoryCache() // ~10-100 MB // Tier 2: Larger, holds warm data diskCache := diskcache.New("/cache") // ~1-10 GB // Tier 3: Large, holds all cacheable data pgCache := postgresql.New("...") // Unlimited ``` -------------------------------- ### Prometheus Metrics Output Format Source: https://github.com/sandrolain/httpcache/blob/master/examples/prometheus/README.md Example output showing the structure of collected cache and HTTP metrics. ```text # HELP httpcache_cache_requests_total Total number of cache operations # TYPE httpcache_cache_requests_total counter httpcache_cache_requests_total{backend="memory",operation="get",result="hit"} 11 httpcache_cache_requests_total{backend="memory",operation="get",result="miss"} 4 httpcache_cache_requests_total{backend="memory",operation="set",result="success"} 4 # HELP httpcache_http_requests_total Total number of HTTP requests # TYPE httpcache_http_requests_total counter httpcache_http_requests_total{cache_status="hit",method="GET"} 11 httpcache_http_requests_total{cache_status="miss",method="GET"} 4 # HELP httpcache_cache_operation_duration_seconds Cache operation duration in seconds # TYPE httpcache_cache_operation_duration_seconds histogram httpcache_cache_operation_duration_seconds_bucket{backend="memory",operation="get",le="0.0001"} 15 ... # HELP httpcache_http_response_size_bytes_total Total size of HTTP responses in bytes # TYPE httpcache_http_response_size_bytes_total counter httpcache_http_response_size_bytes_total{cache_status="hit",method="GET"} 4321 httpcache_http_response_size_bytes_total{cache_status="miss",method="GET"} 4321 ``` -------------------------------- ### Initialize Cache with Connection Pool Source: https://github.com/sandrolain/httpcache/blob/master/examples/postgresql/README.md Using an existing pgxpool for production environments. ```go import ( "github.com/jackc/pgx/v5/pgxpool" "github.com/sandrolain/httpcache/postgresql" ) pool, err := pgxpool.New(ctx, connString) if err != nil { log.Fatal(err) } defer pool.Close() config := postgresql.DefaultConfig() cache, err := postgresql.NewWithPool(pool, config) if err != nil { log.Fatal(err) } defer cache.Close() ``` -------------------------------- ### Implement Three-Tier Cache Strategy Source: https://github.com/sandrolain/httpcache/blob/master/wrapper/multicache/README.md Demonstrates setting, retrieving, and the automatic promotion of data between memory, disk, and PostgreSQL tiers. ```go package main import ( "fmt" "log" httpcache "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/diskcache" "github.com/sandrolain/httpcache/postgresql" "github.com/sandrolain/httpcache/wrapper/multicache" ) func main() { // Tier 1: Memory - Fast, small (10 MB), volatile memCache := httpcache.NewMemoryCache() // Tier 2: Disk - Medium speed, larger (100 MB), survives restarts diskCache := diskcache.New("/var/cache/httpcache") // Tier 3: PostgreSQL - Slower, unlimited, highly persistent, shared pgCache, err := postgresql.New("postgresql://user:pass@localhost/cache") if err != nil { log.Fatal(err) } defer pgCache.Close() // Create multi-tier cache mc := multicache.New(memCache, diskCache, pgCache) // Example: Store and retrieve mc.Set("user:123", []byte(`{"name":"John","email":"john@example.com"}`)) // First Get: Reads from memory (fastest) data, ok := mc.Get("user:123") fmt.Printf("Found in cache: %v\n", ok) // Simulate memory cache eviction (e.g., LRU evicted it) memCache.Delete("user:123") // Second Get: Reads from disk, promotes back to memory data, ok = mc.Get("user:123") fmt.Printf("Found and promoted: %v\n", ok) // Third Get: Reads from memory again (fast) data, ok = mc.Get("user:123") fmt.Printf("Data: %s\n", data) } ``` -------------------------------- ### GitHub Actions CI/CD Integration Source: https://github.com/sandrolain/httpcache/blob/master/postgresql/INTEGRATION_TESTS.md Example step for running integration tests within a GitHub Actions workflow. ```yaml # Example GitHub Actions workflow - name: Run Integration Tests run: go test -v -tags=integration ./postgresql/... ``` -------------------------------- ### Run Redis Docker Container Source: https://github.com/sandrolain/httpcache/blob/master/examples/redis/README.md Starts a Redis server in a Docker container. Ensure Redis is running before proceeding. ```bash docker run -d -p 6379:6379 redis ``` -------------------------------- ### Run Tests and Benchmarks Source: https://github.com/sandrolain/httpcache/blob/master/postgresql/README.md Commands to execute tests and benchmarks against a PostgreSQL instance. ```bash # Set connection string (optional) export POSTGRESQL_TEST_URL="postgres://postgres:postgres@localhost:5432/httpcache_test?sslmode=disable" # Run tests go test -v # Run benchmarks go test -bench=. ``` ```bash # Run integration tests (requires Docker) go test -v -tags=integration # Run specific integration test go test -v -tags=integration -run TestPostgreSQLCacheIntegration ``` -------------------------------- ### Initialize MongoDB Cache Configuration Source: https://github.com/sandrolain/httpcache/blob/master/examples/mongodb/README.md Set up the configuration struct for the MongoDB cache backend. ```go config := mongodb.Config{ URI: "mongodb://localhost:27017", // MongoDB connection URI Database: "httpcache", // Database name Collection: "cache", // Collection name KeyPrefix: "http:", // Optional key prefix Timeout: 10 * time.Second, // Operation timeout TTL: 24 * time.Hour, // Optional TTL for cache entries } ``` -------------------------------- ### Integration with Existing Metrics Source: https://github.com/sandrolain/httpcache/blob/master/docs/monitoring.md Explains how to integrate httpcache metrics with an existing Prometheus setup and how to use custom namespaces or registries. ```APIDOC ## Integration with Existing Metrics If your application already has Prometheus metrics, httpcache metrics are automatically included: ```go // Your existing Prometheus setup http.Handle("/metrics", promhttp.Handler()) // httpcache metrics use the default registry collector := prommetrics.NewCollector() // All metrics are exposed together at /metrics ``` For custom namespaces or registries: ```go customRegistry := prometheus.NewRegistry() collector := prommetrics.NewCollectorWithConfig(prommetrics.CollectorConfig{ Registry: customRegistry, Namespace: "myapp", // Use "myapp_cache_requests_total" instead of "httpcache_..." }) ``` ``` -------------------------------- ### Configure Cloud Storage Backend (BlobCache) Source: https://context7.com/sandrolain/httpcache/llms.txt Sets up a cache using cloud storage services like AWS S3. Ensure necessary environment variables (e.g., AWS credentials) are set. ```go package main import ( "context" "net/http" "time" "github.com/sandrolain/httpcache" "github.com/sandrolain/httpcache/blobcache" _ "gocloud.dev/blob/s3blob" // _ "gocloud.dev/blob/gcsblob" // _ "gocloud.dev/blob/azureblob" ) func main() { ctx := context.Background() // AWS S3 (requires AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env vars) cache, err := blobcache.New(ctx, blobcache.Config{ BucketURL: "s3://my-cache-bucket?region=us-east-1", KeyPrefix: "httpcache/", Timeout: 30 * time.Second, }) if err != nil { panic(err) } defer cache.(interface{ Close() error }).Close() // Google Cloud Storage // cache, _ := blobcache.New(ctx, blobcache.Config{ // BucketURL: "gs://my-cache-bucket", // KeyPrefix: "httpcache/", // }) // Azure Blob Storage // cache, _ := blobcache.New(ctx, blobcache.Config{ // BucketURL: "azblob://my-container", // KeyPrefix: "httpcache/", // }) transport := httpcache.NewTransport(cache) client := &http.Client{Transport: transport} resp, _ := client.Get("https://api.example.com/data") defer resp.Body.Close() } ``` -------------------------------- ### Define Cache Interface Source: https://context7.com/sandrolain/httpcache/llms.txt The core interface that all storage backends must implement to support Get, Set, and Delete operations. ```go // httpcache.Cache is the interface implemented by all backends type Cache interface { Get(key string) (responseBytes []byte, ok bool) Set(key string, responseBytes []byte) Delete(key string) } ``` -------------------------------- ### Run Application with MongoDB Source: https://github.com/sandrolain/httpcache/blob/master/examples/mongodb/README.md Execute the application using different connection configurations. ```bash go run main.go ``` ```bash MONGODB_URI="mongodb://user:pass@localhost:27017/mydb" go run main.go ``` ```bash MONGODB_URI="mongodb://username:password@localhost:27017/?authSource=admin" go run main.go ``` ```bash MONGODB_URI="mongodb+srv://username:password@cluster.mongodb.net/?retryWrites=true&w=majority" go run main.go ``` -------------------------------- ### Checking for Warning Headers Source: https://github.com/sandrolain/httpcache/blob/master/docs/how-it-works.md Example of how to retrieve and log 'Warning' headers from an HTTP response, which inform clients about cache conditions like staleness or revalidation failures. ```go resp, _ := client.Get(url) if warning := resp.Header.Get("Warning"); warning != "" { log.Printf("Cache warning: %s", warning) } ```