### Install doublestar Package Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/bmatcuk/doublestar/README.md Use `go get` to install the doublestar package. This command fetches and installs the package and its dependencies. ```bash go get github.com/bmatcuk/doublestar ``` -------------------------------- ### Create and Run a SOCKS5 Server Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/cloudfoundry/go-socks5/README.md This example demonstrates how to create a new SOCKS5 server with default configuration and start listening for incoming TCP connections on a specified address and port. Ensure the 'socks5' package is imported. ```go // Create a SOCKS5 server conf := &socks5.Config{} server, err := socks5.New(conf) if err != nil { panic(err) } // Create SOCKS5 proxy on localhost port 8000 if err := server.ListenAndServe("tcp", "127.0.0.1:8000"); err != nil { panic(err) } ``` -------------------------------- ### Install the YAML package Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/go.yaml.in/yaml/v3/README.md Use the go get command to add the package to your project. ```bash go get go.yaml.in/yaml/v3 ``` -------------------------------- ### Install Backoff package Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/jpillora/backoff/README.md Command to install the backoff library via go get. ```bash $ go get -v github.com/jpillora/backoff ``` -------------------------------- ### Example output Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/go.yaml.in/yaml/v3/README.md The expected output generated by the provided Go example code. ```text --- t: {Easy! {2 [3 4]}} --- t dump: a: Easy! b: c: 2 d: [3, 4] --- m: map[a:Easy! b:map[c:2 d:[3 4]]] --- m dump: a: Easy! b: c: 2 d: - 3 - 4 ``` -------------------------------- ### Install YAML Package for Go Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/gopkg.in/yaml.v3/README.md Use 'go get' to install the yaml.v3 package. This is the primary method for integrating YAML support into your Go projects. ```bash go get gopkg.in/yaml.v3 ``` -------------------------------- ### Convert Format Strings to Structured Logging Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/go-logr/logr/README.md Examples demonstrating the migration of klog format-string logs to structured logr-style key-value pairs. ```go klog.V(4).Infof("Client is returning errors: code %v, error %v", responseCode, err) ``` ```go logger.Error(err, "client returned an error", "code", responseCode) ``` ```go klog.V(4).Infof("Got a Retry-After %ds response for attempt %d to %v", seconds, retries, url) ``` ```go logger.V(4).Info("got a retry-after response when requesting url", "attempt", retries, "after seconds", seconds, "url", url) ``` ```go log.Printf("unable to reflect over type %T") ``` ```go logger.Info("unable to reflect over type", "type", fmt.Sprintf("%T")) ``` -------------------------------- ### Ginkgo Spec Example Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/onsi/ginkgo/v2/README.md Demonstrates a typical Ginkgo spec structure with BeforeEach, When, Context, It, and assertions using Gomega matchers. Includes SpecTimeout for setting test duration limits. ```go import ( . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ... ) var _ = Describe("Checking books out of the library", Label("library"), func() { var library *libraries.Library var book *books.Book var valjean *users.User BeforeEach(func() { library = libraries.NewClient() book = &books.Book{ Title: "Les Miserables", Author: "Victor Hugo", } valjean = users.NewUser("Jean Valjean") }) When("the library has the book in question", func() { BeforeEach(func(ctx SpecContext) { Expect(library.Store(ctx, book)).To(Succeed()) }) Context("and the book is available", func() { It("lends it to the reader", func(ctx SpecContext) { Expect(valjean.Checkout(ctx, library, "Les Miserables")).To(Succeed()) Expect(valjean.Books()).To(ContainElement(book)) Expect(library.UserWithBook(ctx, book)).To(Equal(valjean)) }, SpecTimeout(time.Second * 5)) }) Context("but the book has already been checked out", func() { var javert *users.User BeforeEach(func(ctx SpecContext) { javert = users.NewUser("Javert") Expect(javert.Checkout(ctx, library, "Les Miserables")).To(Succeed()) }) It("tells the user", func(ctx SpecContext) { err := valjean.Checkout(ctx, library, "Les Miserables") Expect(err).To(MatchError("Les Miserables is currently checked out")) }, SpecTimeout(time.Second * 5)) It("lets the user place a hold and get notified later", func(ctx SpecContext) { Expect(valjean.Hold(ctx, library, "Les Miserables")).To(Succeed()) Expect(valjean.Holds(ctx)).To(ContainElement(book)) By("when Javert returns the book") Expect(javert.Return(ctx, library, book)).To(Succeed()) By("it eventually informs Valjean") notification := "Les Miserables is ready for pick up" Eventually(ctx, valjean.Notifications).Should(ContainElement(notification)) Expect(valjean.Checkout(ctx, library, "Les Miserables")).To(Succeed()) Expect(valjean.Books(ctx)).To(ContainElement(book)) Expect(valjean.Holds(ctx)).To(BeEmpty()) }, SpecTimeout(time.Second * 10)) }) }) When("the library does not have the book in question", func() { It("tells the reader the book is unavailable", func(ctx SpecContext) { err := valjean.Checkout(ctx, library, "Les Miserables") Expect(err).To(MatchError("Les Miserables is not in the library catalog")) }, SpecTimeout(time.Second * 5)) }) }) ``` -------------------------------- ### Install Counterfeiter to GOPATH/bin Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/maxbrunsfeld/counterfeiter/v6/README.md This command installs counterfeiter globally to your `$GOPATH/bin` directory, allowing you to invoke it from your shell even outside of a Go module. The output shows the usage instructions. ```shell go install github.com/maxbrunsfeld/counterfeiter/v6 $ ~/go/bin/counterfeiter USAGE counterfeiter [-generate>] [-o ] [-p] [--fake-name ] [-header ] [] [-] ``` -------------------------------- ### Use backoff with net package Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/jpillora/backoff/README.md Example of using backoff to manage reconnection attempts for network connections. ```go b := &backoff.Backoff{ Max: 5 * time.Minute, } for { conn, err := net.Dial("tcp", "example.com:5309") if err != nil { d := b.Duration() fmt.Printf("%s, reconnecting in %s", err, d) time.Sleep(d) continue } //connected b.Reset() conn.Write([]byte("hello world!")) // ... Read ... Write ... etc conn.Close() //disconnected } ``` -------------------------------- ### Build and Test Commands for Ginkgo Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/onsi/ginkgo/v2/CONTRIBUTING.md Commands to install, test, and verify changes within the Ginkgo project. ```bash make ``` ```bash go install ./... ``` ```bash ginkgo -r -p ``` ```bash go vet ./... ``` ```bash bundle && bundle exec jekyll serve ``` -------------------------------- ### Create HTTP Client with Retry Logic Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Demonstrates creating an HTTP client with automatic retry capabilities. Configure max attempts, delay, and logger for retry behavior. Use NewHTTPClient to wrap the retry client for convenient methods like Get, Post, and GetCustomized. ```go package main import ( "fmt" "io" "log" "net/http" "time" "github.com/cloudfoundry/bosh-utils/httpclient" "github.com/cloudfoundry/bosh-utils/logger" ) func main() { boshLogger := logger.NewLogger(logger.LevelDebug) // Create default HTTP client (skip TLS verification) defaultClient := httpclient.CreateDefaultClientInsecureSkipVerify() // Create retry client - retries up to 3 times with 1 second delay retryClient := httpclient.NewRetryClient( defaultClient, 3, // maxAttempts 1*time.Second, // retryDelay boshLogger, ) // Create network-safe retry client (only retries GET/HEAD on specific errors) safeRetryClient := httpclient.NewNetworkSafeRetryClient( defaultClient, 5, // maxAttempts 2*time.Second, // retryDelay boshLogger, ) // Wrap with HTTPClient for convenient methods httpClient := httpclient.NewHTTPClient(retryClient, boshLogger) // GET request resp, err := httpClient.Get("https://api.example.com/data") if err != nil { log.Fatalf("GET failed: %v", err) } defer resp.Body.Close() body, _ := io.ReadAll(resp.Body) fmt.Printf("Response: %s\n", body) // POST request with payload payload := []byte(`{"name": "test"}`) resp, err = httpClient.Post("https://api.example.com/items", payload) if err != nil { log.Fatalf("POST failed: %v", err) } defer resp.Body.Close() // Customized request with headers resp, err = httpClient.GetCustomized("https://api.example.com/secure", func(req *http.Request) { req.Header.Set("Authorization", "Bearer token123") req.Header.Set("Accept", "application/json") }) if err != nil { log.Fatalf("Customized GET failed: %v", err) } defer resp.Body.Close() // Use safe retry client for idempotent operations _ = safeRetryClient } ``` -------------------------------- ### Create and Wrap Errors with BOSH Errors Package Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Shows how to create simple, formatted, and chained errors using the `bosherr` package. It also demonstrates creating multi-errors and user-facing errors, along with a practical example of wrapping errors in a function. ```go package main import ( "fmt" "io" "os" bosherr "github.com/cloudfoundry/bosh-utils/errors" ) func main() { // Create simple error err := bosherr.Error("connection refused") fmt.Printf("Simple error: %v\n", err) // Create formatted error err = bosherr.Errorf("failed to connect to %s:%d", "localhost", 8080) fmt.Printf("Formatted error: %v\n", err) // Wrap error with context originalErr := io.EOF wrappedErr := bosherr.WrapError(originalErr, "reading configuration file") fmt.Printf("Wrapped error: %v\n", wrappedErr) // Output: "reading configuration file: EOF" // Wrap with formatting wrappedErr = bosherr.WrapErrorf(originalErr, "reading file %s", "/etc/config.yml") fmt.Printf("Wrapped formatted: %v\n", wrappedErr) // Chain multiple errors err1 := bosherr.Error("network timeout") err2 := bosherr.WrapError(err1, "fetching blob") err3 := bosherr.WrapError(err2, "installing package") fmt.Printf("Error chain: %v\n", err3) // Output: "installing package: fetching blob: network timeout" // Create multi-error for collecting multiple failures errs := bosherr.NewMultiError( bosherr.Error("failed to start service A"), bosherr.Error("failed to start service B"), bosherr.Error("failed to start service C"), ) fmt.Printf("Multi-error:\n%v\n", errs) // User-facing errors userErr := bosherr.NewUserError("Invalid deployment manifest") fmt.Printf("User error: %v\n", userErr) // Use in real code if err := doSomething(); err != nil { fmt.Fprintf(os.Stderr, "Operation failed: %v\n", err) } } func doSomething() error { if err := connectToDatabase(); err != nil { return bosherr.WrapError(err, "initializing application") } return nil } func connectToDatabase() error { return bosherr.Error("connection refused") } ``` -------------------------------- ### Create Retryable Blobstore with Digest Verification Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Wraps a base blobstore with digest verification and then with retry logic. Use this to automatically retry Get and Create operations on a blobstore up to a specified number of times, handling transient network or service issues. ```go package main import ( "github.com/cloudfoundry/bosh-utils/blobstore" "github.com/cloudfoundry/bosh-utils/crypto" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/system" ) func main() { boshLogger := logger.NewLogger(logger.LevelDebug) fs := system.NewOsFileSystem(boshLogger) // Create base blobstore (e.g., local or external) baseBlobstore := createBaseBlobstore(fs) // Wrap with digest verification createAlgos := []crypto.Algorithm{crypto.DigestAlgorithmSHA256} verifiableBlobstore := blobstore.NewDigestVerifiableBlobstore(baseBlobstore, fs, createAlgos) // Wrap with retry logic - retries up to 3 times on failure maxRetries := 3 retryableBlobstore := blobstore.NewRetryableBlobstore(verifiableBlobstore, maxRetries, boshLogger) // Use the retryable blobstore - Get/Create will automatically retry digest := crypto.MustParseMultipleDigest("sha256:abc123...") filePath, err := retryableBlobstore.Get("blob-id", digest) if err != nil { // All 3 attempts failed panic(err) } defer retryableBlobstore.CleanUp(filePath) } func createBaseBlobstore(fs system.FileSystem) blobstore.Blobstore { // Implementation omitted for brevity return nil } ``` -------------------------------- ### Initialize and Use BOSH Logger Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Demonstrates creating a logger with a minimum level, using a custom writer, configuring tag-based log levels, and logging messages at different levels. It also shows how to use RFC3339 timestamps, log with details, toggle forced debug mode, handle panics, and parse/convert log levels. ```go package main import ( "os" "github.com/cloudfoundry/bosh-utils/logger" ) func main() { // Create logger with minimum level boshLogger := logger.NewLogger(logger.LevelInfo) // Or create with custom writer fileLogger := logger.NewWriterLogger(logger.LevelDebug, os.Stdout) // Use RFC3339 timestamps instead of legacy format fileLogger.UseRFC3339Timestamps() // Configure per-tag log levels fileLogger.UseTags([]logger.LogTag{ {Name: "httpClient", LogLevel: logger.LevelDebug}, {Name: "blobstore", LogLevel: logger.LevelWarn}, }) // Log at different levels boshLogger.Debug("myComponent", "Debug message: %s", "details") boshLogger.Info("myComponent", "Starting process with %d workers", 4) boshLogger.Warn("myComponent", "Connection slow, retrying...") boshLogger.Error("myComponent", "Failed to connect: %v", "timeout") // Log with details block boshLogger.DebugWithDetails("myComponent", "Request details", "POST /api/v1/data") boshLogger.ErrorWithDetails("myComponent", "Stack trace", "panic at line 42...") // Toggle forced debug mode (shows all debug logs regardless of level) boshLogger.ToggleForcedDebug() // Handle panics gracefully (logs error and exits) defer boshLogger.HandlePanic("main") // Parse log level from string level, err := logger.Levelify("DEBUG") if err != nil { boshLogger.Error("config", "Invalid log level: %v", err) } _ = level // Convert level to string levelStr := logger.AsString(logger.LevelInfo) boshLogger.Info("config", "Current level: %s", levelStr) } ``` -------------------------------- ### Create Local Blobstore in Go Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Shows how to create a local filesystem-based blobstore using BOSH Utils, configuring it to store blobs in a specified directory and perform basic operations like creation, retrieval, and deletion. ```go package main import ( "fmt" "log" "github.com/cloudfoundry/bosh-utils/blobstore" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/system" "github.com/cloudfoundry/bosh-utils/uuid" ) func main() { boshLogger := logger.NewLogger(logger.LevelInfo) fs := system.NewOsFileSystem(boshLogger) uuidGen := uuid.NewGenerator() // Configure local blobstore options options := map[string]interface{}{ "blobstore_path": "/var/vcap/data/blobs", } // Create local blobstore store := blobstore.NewLocalBlobstore(fs, uuidGen, options) // Validate configuration if err := store.Validate(); err != nil { log.Fatalf("Invalid blobstore config: %v", err) } // Store a blob blobID, err := store.Create("/tmp/my-package.tgz") if err != nil { log.Fatalf("Failed to store blob: %v", err) } fmt.Printf("Stored blob with ID: %s\n", blobID) // Retrieve the blob tempFilePath, err := store.Get(blobID) if err != nil { log.Fatalf("Failed to retrieve blob: %v", err) } fmt.Printf("Retrieved blob to temp path: %s\n", tempFilePath) // Clean up temp file when done store.CleanUp(tempFilePath) // Delete blob from store store.Delete(blobID) } ``` -------------------------------- ### Blobstore Interface Usage in Go Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Demonstrates creating and interacting with a local blobstore using the BOSH Utils library, including blob creation, retrieval, and deletion with digest verification and retry mechanisms. ```go package main import ( "fmt" "log" "os" "github.com/cloudfoundry/bosh-utils/blobstore" "github.com/cloudfoundry/bosh-utils/crypto" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/system" ) func main() { // Create dependencies boshLogger := logger.NewLogger(logger.LevelDebug) fs := system.NewOsFileSystem(boshLogger) cmdRunner := system.NewExecCmdRunner(boshLogger) // Create blobstore provider provider := blobstore.NewProvider(fs, cmdRunner, "/tmp/blobstore-config", boshLogger) // Get a local blobstore with digest verification and retry options := map[string]interface{}{ "blobstore_path": "/tmp/my-blobstore", } store, err := provider.Get(blobstore.BlobstoreTypeLocal, options) if err != nil { log.Fatalf("Failed to create blobstore: %v", err) } // Create a blob from a file blobID, digest, err := store.Create("/path/to/source/file.tar.gz") if err != nil { log.Fatalf("Failed to create blob: %v", err) } fmt.Printf("Created blob ID: %s with digest: %s\n", blobID, digest.String()) // Retrieve the blob with digest verification downloadedPath, err := store.Get(blobID, digest) if err != nil { log.Fatalf("Failed to get blob: %v", err) } defer store.CleanUp(downloadedPath) fmt.Printf("Downloaded blob to: %s\n", downloadedPath) // Delete the blob if err := store.Delete(blobID); err != nil { log.Fatalf("Failed to delete blob: %v", err) } } ``` -------------------------------- ### Implement simple backoff Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/jpillora/backoff/README.md Demonstrates basic configuration and usage of the Backoff struct to generate increasing durations. ```go b := &backoff.Backoff{ //These are the defaults Min: 100 * time.Millisecond, Max: 10 * time.Second, Factor: 2, Jitter: false, } fmt.Printf("%s\n", b.Duration()) fmt.Printf("%s\n", b.Duration()) fmt.Printf("%s\n", b.Duration()) fmt.Printf("Reset!\n") b.Reset() fmt.Printf("%s\n", b.Duration()) ``` -------------------------------- ### Validate Version Against Constraint in Go Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/Masterminds/semver/v3/README.md Demonstrates how to create a semver constraint and a version, then validate if the version meets the constraint. It shows how to retrieve error messages if validation fails. ```go c, err := semver.NewConstraint("<= 1.2.3, >= 1.4") if err != nil { // Handle constraint not being parseable. } v, err := semver.NewVersion("1.3") if err != nil { // Handle version not being parseable. } // Validate a version against a constraint. a, msgs := c.Validate(v) // a is false for _, m := range msgs { fmt.Println(m) // Loops over the errors which would read // "1.3 is greater than 1.2.3" // "1.3 is less than 1.4" } ``` -------------------------------- ### Initialize the root logger Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/go-logr/logr/README.md Create a root logger instance using a specific implementation early in the application lifecycle. ```go func main() { // ... other setup code ... // Create the "root" logger. We have chosen the "logimpl" implementation, // which takes some initial parameters and returns a logr.Logger. logger := logimpl.New(param1, param2) // ... other setup code ... ``` -------------------------------- ### Create and Check Version Constraints Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/Masterminds/semver/v3/README.md Instantiate a version constraint and check if a given version satisfies it. Handle potential errors during constraint or version parsing. ```go c, err := semver.NewConstraint(">= 1.2.3") if err != nil { // Handle constraint not being parsable. } v, err := semver.NewVersion("1.3") if err != nil { // Handle version not being parsable. } // Check if the version meets the constraints. The variable a will be true. a := c.Check(v) ``` -------------------------------- ### Parse Command Line Arguments Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/jessevdk/go-flags/README.md Demonstrates defining a complex options struct with various types and parsing arguments using flags.ParseArgs. ```go var opts struct { // Slice of bool will append 'true' each time the option // is encountered (can be set multiple times, like -vvv) Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"` // Example of automatic marshalling to desired type (uint) Offset uint `long:"offset" description:"Offset"` // Example of a callback, called each time the option is found. Call func(string) `short:"c" description:"Call phone number"` // Example of a required flag Name string `short:"n" long:"name" description:"A name" required:"true"` // Example of a flag restricted to a pre-defined set of strings Animal string `long:"animal" choice:"cat" choice:"dog"` // Example of a value name File string `short:"f" long:"file" description:"A file" value-name:"FILE"` // Example of a pointer Ptr *int `short:"p" description:"A pointer to an integer"` // Example of a slice of strings StringSlice []string `short:"s" description:"A slice of strings"` // Example of a slice of pointers PtrSlice []*string `long:"ptrslice" description:"A slice of pointers to string"` // Example of a map IntMap map[string]int `long:"intmap" description:"A map from string to int"` // Example of env variable Thresholds []int `long:"thresholds" default:"1" default:"2" env:"THRESHOLD_VALUES" env-delim:","` } // Callback which will invoke callto: to call a number. // Note that this works just on OS X (and probably only with // Skype) but it shows the idea. opts.Call = func(num string) { cmd := exec.Command("open", "callto:"+num) cmd.Start() cmd.Process.Release() } // Make some fake arguments to parse. args := []string{ "-vv", "--offset=5", "-n", "Me", "--animal", "dog", // anything other than "cat" or "dog" will raise an error "-p", "3", "-s", "hello", "-s", "world", "--ptrslice", "hello", "--ptrslice", "world", "--intmap", "a:1", "--intmap", "b:5", "arg1", "arg2", "arg3", } // Parse flags from `args'. Note that here we use flags.ParseArgs for // the sake of making a working example. Normally, you would simply use // flags.Parse(&opts) which uses os.Args args, err := flags.ParseArgs(&opts, args) if err != nil { panic(err) } fmt.Printf("Verbosity: %v\n", opts.Verbose) fmt.Printf("Offset: %d\n", opts.Offset) fmt.Printf("Name: %s\n", opts.Name) fmt.Printf("Animal: %s\n", opts.Animal) fmt.Printf("Ptr: %d\n", *opts.Ptr) fmt.Printf("StringSlice: %v\n", opts.StringSlice) fmt.Printf("PtrSlice: [%v %v]\n", *opts.PtrSlice[0], *opts.PtrSlice[1]) fmt.Printf("IntMap: [a:%v b:%v]\n", opts.IntMap["a"], opts.IntMap["b"]) fmt.Printf("Remaining args: %s\n", strings.Join(args, " ")) // Output: Verbosity: [true true] // Offset: 5 // Name: Me // Ptr: 3 // StringSlice: [hello world] // PtrSlice: [hello world] // IntMap: [a:1 b:5] // Remaining args: arg1 arg2 arg3 ``` -------------------------------- ### Compress Directory to Tarball with BOSH Utils Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Compresses an entire directory into a tarball using gzip compression. Ensure the source directory exists. The created tarball path is returned and should be cleaned up using `CleanUp`. ```go package main import ( "fmt" "log" "github.com/cloudfoundry/bosh-utils/fileutil" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/system" ) func main() { boshLogger := logger.NewLogger(logger.LevelInfo) fs := system.NewOsFileSystem(boshLogger) cmdRunner := system.NewExecCmdRunner(boshLogger) compressor := fileutil.NewTarballCompressor(cmdRunner, fs) // Compress entire directory to tarball tarballPath, err := compressor.CompressFilesInDir("/path/to/source/dir", fileutil.CompressorOptions{ NoCompression: false, // Use gzip compression }) if err != nil { log.Fatalf("Compression failed: %v", err) } defer compressor.CleanUp(tarballPath) fmt.Printf("Created tarball: %s\n", tarballPath) // Compress specific files tarballPath, err = compressor.CompressSpecificFilesInDir( "/path/to/source/dir", []string{"file1.txt", "subdir/file2.txt"}, fileutil.CompressorOptions{NoCompression: false}, ) if err != nil { log.Fatalf("Compression failed: %v", err) } fmt.Printf("Created tarball with specific files: %s\n", tarballPath) // Decompress tarball err = compressor.DecompressFileToDir("/path/to/archive.tar.gz", "/tmp/extracted", fileutil.CompressorOptions{ SameOwner: false, // Don't preserve original ownership StripComponents: 1, // Strip leading directory component PathInArchive: "", // Extract all contents }) if err != nil { log.Fatalf("Decompression failed: %v", err) } fmt.Println("Extracted to /tmp/extracted") // Check if file is uncompressed tarball if compressor.IsNonCompressedTarball("/path/to/file.tar") { fmt.Println("File is an uncompressed tar archive") } } ``` -------------------------------- ### Perform File Operations with BOSH FileSystem Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Utilize `system.NewOsFileSystem` for platform-independent file operations. Supports writing, reading, directory creation, copying, symlinking, and more. ```go package main import ( "fmt" "log" "os" "path/filepath" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/system" ) func main() { boshLogger := logger.NewLogger(logger.LevelInfo) fs := system.NewOsFileSystem(boshLogger) // Write and read files err := fs.WriteFileString("/tmp/test.txt", "Hello, BOSH!") if err != nil { log.Fatalf("Write failed: %v", err) } content, err := fs.ReadFileString("/tmp/test.txt") if err != nil { log.Fatalf("Read failed: %v", err) } fmt.Printf("Content: %s\n", content) // Create directory structure err = fs.MkdirAll("/tmp/bosh-test/nested/dir", 0755) if err != nil { log.Fatalf("MkdirAll failed: %v", err) } // Copy files err = fs.CopyFile("/tmp/test.txt", "/tmp/bosh-test/test-copy.txt") if err != nil { log.Fatalf("Copy failed: %v", err) } // Create symlink err = fs.Symlink("/tmp/test.txt", "/tmp/bosh-test/test-link.txt") if err != nil { log.Fatalf("Symlink failed: %v", err) } // Follow symlink target, err := fs.ReadAndFollowLink("/tmp/bosh-test/test-link.txt") if err != nil { log.Fatalf("ReadAndFollowLink failed: %v", err) } fmt.Printf("Symlink target: %s\n", target) // Create temp file tempFile, err := fs.TempFile("bosh-test") if err != nil { log.Fatalf("TempFile failed: %v", err) } defer tempFile.Close() fmt.Printf("Temp file: %s\n", tempFile.Name()) // Glob for files matches, err := fs.Glob("/tmp/bosh-test/*.txt") if err != nil { log.Fatalf("Glob failed: %v", err) } fmt.Printf("Matched files: %v\n", matches) // Recursive glob matches, err = fs.RecursiveGlob("/tmp/bosh-test/**/*") if err != nil { log.Fatalf("RecursiveGlob failed: %v", err) } fmt.Printf("All files: %v\n", matches) // Walk directory tree fs.Walk("/tmp/bosh-test", func(path string, info os.FileInfo, err error) error { if err != nil { return err } fmt.Printf(" %s (dir=%v)\n", path, info.IsDir()) return nil }) // Check file existence if fs.FileExists("/tmp/test.txt") { fmt.Println("File exists") } // Clean up fs.RemoveAll("/tmp/bosh-test") } ``` -------------------------------- ### Run Go Generate Command Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/maxbrunsfeld/counterfeiter/v6/README.md Execute this command in your module's root directory or the specific package directory to generate the fake implementations. ```shell go generate ./... ``` -------------------------------- ### Execute External Commands with BOSH Utils Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Use `system.NewExecCmdRunner` to execute commands, capture output, and handle errors. Supports stdin, environment variables, and working directories. ```go package main import ( "fmt" "log" "strings" "time" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/system" ) func main() { boshLogger := logger.NewLogger(logger.LevelInfo) cmdRunner := system.NewExecCmdRunner(boshLogger) // Simple command execution stdout, stderr, exitStatus, err := cmdRunner.RunCommand("ls", "-la", "/tmp") if err != nil { log.Fatalf("Command failed (exit %d): %v\nstderr: %s", exitStatus, err, stderr) } fmt.Printf("Directory listing:\n%s\n", stdout) // Command with stdin input stdout, _, _, err = cmdRunner.RunCommandWithInput("hello world", "cat") if err != nil { log.Fatalf("Cat failed: %v", err) } fmt.Printf("Echo: %s\n", stdout) // Complex command with environment and working directory cmd := system.Command{ Name: "env", Args: []string{}, Env: map[string]string{"MY_VAR": "my_value"}, WorkingDir: "/tmp", Quiet: false, } stdout, stderr, exitStatus, err = cmdRunner.RunComplexCommand(cmd) if err != nil { log.Fatalf("Env command failed: %v", err) } fmt.Printf("Environment includes MY_VAR: %v\n", strings.Contains(stdout, "MY_VAR")) // Async command execution asyncCmd := system.Command{ Name: "sleep", Args: []string{"10"}, } process, err := cmdRunner.RunComplexCommandAsync(asyncCmd) if err != nil { log.Fatalf("Failed to start async command: %v", err) } // Wait for result or terminate early select { case result := <-process.Wait(): fmt.Printf("Process completed with exit status: %d\n", result.ExitStatus) case <-time.After(2 * time.Second): // Terminate with grace period process.TerminateNicely(5 * time.Second) fmt.Println("Process terminated after timeout") } // Check if command exists if cmdRunner.CommandExists("git") { fmt.Println("Git is available") } } ``` -------------------------------- ### Instantiate a Fake Test Double Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/maxbrunsfeld/counterfeiter/v6/README.md Import the generated fakes package and instantiate the fake by taking its address. This fake can then be used to record calls and stub return values. ```go import "my-repo/path/to/foo/foofakes" var fake = &foofakes.FakeMySpecialInterface{} ``` -------------------------------- ### Implement Retry Strategies in Go Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Demonstrates using attempt-based and timeout-based retry strategies to handle operations that may fail. ```go package main import ( "fmt" "time" "code.cloudfoundry.org/clock" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/retrystrategy" ) func main() { boshLogger := logger.NewLogger(logger.LevelDebug) // Create a retryable operation using function wrapper attemptCount := 0 retryable := retrystrategy.NewRetryable(func() (bool, error) { attemptCount++ fmt.Printf("Attempt %d\n", attemptCount) if attemptCount < 3 { return true, fmt.Errorf("temporary failure") } return false, nil // Success, don't retry }) // Attempt-based retry: retry up to N times with fixed delay strategy := retrystrategy.NewAttemptRetryStrategy( 5, // maxAttempts 500*time.Millisecond, // delay between attempts retryable, boshLogger, ) err := strategy.Try() if err != nil { fmt.Printf("All attempts failed: %v\n", err) } else { fmt.Println("Operation succeeded!") } // Timeout-based retry: keep trying until timeout timeService := clock.NewClock() timeoutRetryable := retrystrategy.NewRetryable(func() (bool, error) { // Simulate operation that might fail return true, fmt.Errorf("still failing") }) timeoutStrategy := retrystrategy.NewTimeoutRetryStrategy( 10*time.Second, // total timeout 1*time.Second, // delay between attempts timeoutRetryable, timeService, boshLogger, ) err = timeoutStrategy.Try() fmt.Printf("Timeout strategy result: %v\n", err) } ``` -------------------------------- ### OS Interface and GlobOS/PathMatchOS Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/bmatcuk/doublestar/README.md Explains how to abstract the OS package for testing and interoperability using the OS interface and the GlobOS/PathMatchOS functions. ```APIDOC ## Abstracting the `os` package **doublestar** by default uses the `Open`, `Stat`, and `Lstat`, functions and `PathSeparator` value from the standard library's `os` package. To abstract this, for example to be able to perform tests of Windows paths on Linux, or to interoperate with your own filesystem code, it includes the functions `GlobOS` and `PathMatchOS` which are identical to `Glob` and `PathMatch` except that they operate on an `OS` interface: ```go type OS interface { Lstat(name string) (os.FileInfo, error) Open(name string) (*os.File, error) PathSeparator() rune Stat(name string) (os.FileInfo, error) } ``` `StandardOS` is a value that implements this interface by calling functions in the standard library's `os` package. ``` -------------------------------- ### Define Options Struct Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/jessevdk/go-flags/README.md Use struct field tags to define command line options with short and long names. ```go type Options struct { Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"` } ``` -------------------------------- ### Unmarshal and Marshal YAML Data in Go Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/gopkg.in/yaml.v3/README.md Demonstrates how to unmarshal YAML data into a Go struct and a map, and then marshal them back into YAML format. Ensure struct fields are public for correct unmarshalling. ```Go package main import ( "fmt" "log" "gopkg.in/yaml.v3" ) var data = ` a: Easy! b: c: 2 d: [3, 4] ` // Note: struct fields must be public in order for unmarshal to // correctly populate the data. type T struct { A string B struct { RenamedC int `yaml:"c"` D []int `yaml:",flow"` } } func main() { t := T{} err := yaml.Unmarshal([]byte(data), &t) if err != nil { log.Fatalf("error: %v", err) } fmt.Printf("--- t:\n%v\n\n", t) d, err := yaml.Marshal(&t) if err != nil { log.Fatalf("error: %v", err) } fmt.Printf("--- t dump:\n%s\n\n", string(d)) m := make(map[interface{}]interface{}) err = yaml.Unmarshal([]byte(data), &m) if err != nil { log.Fatalf("error: %v", err) } fmt.Printf("--- m:\n%v\n\n", m) d, err = yaml.Marshal(&m) if err != nil { log.Fatalf("error: %v", err) } fmt.Printf("--- m dump:\n%s\n\n", string(d)) } ``` -------------------------------- ### Add Counterfeiter as a Tool Dependency Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/maxbrunsfeld/counterfeiter/v6/README.md Use this command to establish a tool dependency on counterfeiter within your Go module. ```shell go get -tool github.com/maxbrunsfeld/counterfeiter/v6 ``` -------------------------------- ### Generate Test Double for Local Interface Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/maxbrunsfeld/counterfeiter/v6/README.md Use `go tool counterfeiter` with the package path and interface name to generate a fake implementation. The fake will be written to a `fakes` subdirectory within the package. ```shell $ cat path/to/foo/file.go ``` ```go package foo type MySpecialInterface interface { DoThings(string, uint64) (int, error) } ``` ```shell $ go tool counterfeiter path/to/foo MySpecialInterface Wrote `FakeMySpecialInterface` to `path/to/foo/foofakes/fake_my_special_interface.go` ``` -------------------------------- ### Generate Test Double for Third-Party Interface Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/maxbrunsfeld/counterfeiter/v6/README.md For interfaces defined in external packages, use the alternative syntax `.` with `go tool counterfeiter`. ```shell $ go tool counterfeiter github.com/go-redis/redis.Pipeliner ``` -------------------------------- ### Define system call entry points Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/golang.org/x/sys/unix/README.md Assembly entry points for system call dispatch, implemented per GOOS/GOARCH pair. ```go func Syscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr) func Syscall6(trap, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2, err uintptr) func RawSyscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr) ``` -------------------------------- ### Copy Filtered Files to Temp Directory with BOSH Utils Source: https://context7.com/cloudfoundry/bosh-utils/llms.txt Copies files matching glob patterns from a source directory to a temporary directory. Uses `CleanUp` to remove the temporary directory. Supports copying from multiple source directories with specified prefixes. ```go package main import ( "fmt" "log" "github.com/cloudfoundry/bosh-utils/fileutil" "github.com/cloudfoundry/bosh-utils/logger" "github.com/cloudfoundry/bosh-utils/system" ) func main() { boshLogger := logger.NewLogger(logger.LevelInfo) fs := system.NewOsFileSystem(boshLogger) copier := fileutil.NewGenericCpCopier(fs, boshLogger) // Copy filtered files to temp directory filters := []string{ "*.yml", // All YAML files "config/**/*", // Everything in config directory "src/*.go", // Go source files } tempDir, err := copier.FilteredCopyToTemp("/path/to/project", filters) if err != nil { log.Fatalf("Copy failed: %v", err) } defer copier.CleanUp(tempDir) fmt.Printf("Copied matching files to: %s\n", tempDir) // Copy from multiple source directories with prefixes dirs := []fileutil.DirToCopy{ {Dir: "/path/to/config", Prefix: "config"}, {Dir: "/path/to/scripts", Prefix: "bin"}, } tempDir, err = copier.FilteredMultiCopyToTemp(dirs, []string{"**/*"}) if err != nil { log.Fatalf("Multi-copy failed: %v", err) } defer copier.CleanUp(tempDir) fmt.Printf("Copied to: %s/config and %s/bin\n", tempDir, tempDir) } ``` -------------------------------- ### Create GitHub Release and Fetch Tags Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/onsi/gomega/RELEASING.md This sequence of Git commands is used to finalize a release. It involves committing changes, pushing to the remote repository, creating a release on GitHub, and fetching all tags. ```git git commit -m "vM.m.p" git push gh release create "vM.m.p" git fetch --tags origin master ``` -------------------------------- ### Load Slim-Sprig FuncMap in Go Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/go-task/slim-sprig/v3/README.md Integrate the Slim-Sprig FuncMap into Go's html/template package. Ensure the FuncMap is set before parsing templates. ```go import ( "html/template" "github.com/go-task/slim-sprig" ) // This example illustrates that the FuncMap *must* be set before the // templates themselves are loaded. tpl := template.Must( template.New("base").Funcs(sprig.FuncMap()).ParseGlob("*.html") ) ``` -------------------------------- ### Path-Aware Matching with Custom OS Interface Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/bmatcuk/doublestar/README.md The `PathMatchOS` function performs path-aware matching using a provided `OS` interface. This enables testing or using alternative filesystem implementations. ```go func PathMatchOS(osys OS, pattern, name string) (bool, error) ``` -------------------------------- ### Parse a Semantic Version Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/Masterminds/semver/v3/README.md Use NewVersion to parse a version string, which may include pre-release tags or build metadata. ```go v, err := semver.NewVersion("1.2.3-beta.1+build345") ``` -------------------------------- ### Generate Multiple Fakes with counterfeiter:generate Directive Source: https://github.com/cloudfoundry/bosh-utils/blob/master/vendor/github.com/maxbrunsfeld/counterfeiter/v6/README.md For packages with many interfaces, use the `counterfeiter:generate` directive to consolidate generation commands. This speeds up the `go generate` process. Ensure only one `go:generate go tool counterfeiter -generate` directive exists per package. ```go package foo // You only need **one** of these per package! //go:generate go tool counterfeiter -generate // You will add lots of directives like these in the same package... //counterfeiter:generate . MySpecialInterface type MySpecialInterface interface { DoThings(string, uint64) (int, error) } // Like this... //counterfeiter:generate . MyOtherInterface type MyOtherInterface interface { DoOtherThings(string, uint64) (int, error) } ```