s3gof3r

https://pkg.go.dev/github.com/rlmcpherson/s3gof3r
Admin
s3gof3r provides fast, parallelized, pipelined streaming access to Amazon S3, optimized for...
Tokens:8,814
Snippets:119
Trust Score:9.6
Update:1 month ago
Show doc for...
Tokens:
Raw
### Practical gof3r Usage Examples

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r

Real-world examples demonstrating piping tar archives to S3 and retrieving them.

```bash
$ tar -cf - /foo_dir/ | gof3r put -b my_s3_bucket -k bar_dir/s3_object -m x-amz-meta-custom-metadata:abc123 -m x-amz-server-side-encryption:AES256
$ gof3r get -b my_s3_bucket -k bar_dir/s3_object | tar -x
```

--------------------------------

### Speed Benchmark Examples

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Examples of measuring transfer speeds using the pv utility during S3 upload and download operations.

```bash
  $ gof3r get -b test-bucket -k 8_GB_tar | pv -a | tar -x
  Duration: 53.201632211s
  [ 167MB/s]
  

  $ tar -cf - test_dir/ | pv -a | gof3r put -b test-bucket -k 8_GB_tar
  Duration: 1m16.080800315s
  [ 119MB/s]
```

--------------------------------

### Install s3gof3r

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Commands to install the gof3r CLI tool or the s3gof3r Go package using the go get command.

```bash
$ go get github.com/rlmcpherson/s3gof3r/gof3r
```

```bash
$ go get github.com/rlmcpherson/s3gof3r
```

--------------------------------

### Parse Command Line Arguments

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Comprehensive example demonstrating various data types, callbacks, and parsing logic using flags.ParseArgs.

```go
var opts struct {
	// Slice of bool will append 'true' each time the option
	// is encountered (can be set multiple times, like -vvv)
	Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"`

	// Example of automatic marshalling to desired type (uint)
	Offset uint `long:"offset" description:"Offset"`

	// Example of a callback, called each time the option is found.
	Call func(string) `short:"c" description:"Call phone number"`

	// Example of a required flag
	Name string `short:"n" long:"name" description:"A name" required:"true"`

	// Example of a value name
	File string `short:"f" long:"file" description:"A file" value-name:"FILE"`

	// Example of a pointer
	Ptr *int `short:"p" description:"A pointer to an integer"`

	// Example of a slice of strings
	StringSlice []string `short:"s" description:"A slice of strings"`

	// Example of a slice of pointers
	PtrSlice []*string `long:"ptrslice" description:"A slice of pointers to string"`

	// Example of a map
	IntMap map[string]int `long:"intmap" description:"A map from string to int"`
}

// Callback which will invoke callto:<argument> to call a number.
// Note that this works just on OS X (and probably only with
// Skype) but it shows the idea.
opts.Call = func(num string) {
	cmd := exec.Command("open", "callto:"+num)
	cmd.Start()
	cmd.Process.Release()
}

// Make some fake arguments to parse.
args := []string{
	"-vv",
	"--offset=5",
	"-n", "Me",
	"-p", "3",
	"-s", "hello",
	"-s", "world",
	"--ptrslice", "hello",
	"--ptrslice", "world",
	"--intmap", "a:1",
	"--intmap", "b:5",
	"arg1",
	"arg2",
	"arg3",
}

// Parse flags from `args'. Note that here we use flags.ParseArgs for
// the sake of making a working example. Normally, you would simply use
// flags.Parse(&opts) which uses os.Args
args, err := flags.ParseArgs(&opts, args)

if err != nil {
	panic(err)
	os.Exit(1)
}

fmt.Printf("Verbosity: %v\n", opts.Verbose)
fmt.Printf("Offset: %d\n", opts.Offset)
fmt.Printf("Name: %s\n", opts.Name)
fmt.Printf("Ptr: %d\n", *opts.Ptr)
fmt.Printf("StringSlice: %v\n", opts.StringSlice)
fmt.Printf("PtrSlice: [%v %v]\n", *opts.PtrSlice[0], *opts.PtrSlice[1])
fmt.Printf("IntMap: [a:%v b:%v]\n", opts.IntMap["a"], opts.IntMap["b"])
fmt.Printf("Remaining args: %s\n", strings.Join(args, " "))

// Output: Verbosity: [true true]
// Offset: 5
// Name: Me
// Ptr: 3
// StringSlice: [hello world]
// PtrSlice: [hello world]
// IntMap: [a:1 b:5]
// Remaining args: arg1 arg2 arg3
```

--------------------------------

### INI File Format Example

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Example of the expected INI file structure.

```ini
[Option group name]
option = value
```

--------------------------------

### Example Output

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Displays the expected output format for parsed flags and arguments.

```text
Output:
Verbosity: [true true]
Offset: 5
Name: Me
Ptr: 3
StringSlice: [hello world]
PtrSlice: [hello world]
IntMap: [a:1 b:5]
Filename: hello.go
Args.Id: id
Args.Num: 10
Args.Rest: [remaining1 remaining2]
```

--------------------------------

### Define Options Struct

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Basic example of defining a struct with tags to specify command line options.

```go
type Options struct {
    Verbose []bool `short:"v" long:"verbose" description:"Show verbose debug information"`
}
```

--------------------------------

### Get bucket instance

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Returns a bucket instance initialized with DefaultConfig.

```go
func (s *S3) Bucket(name string) *Bucket
```

--------------------------------

### Get error string

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Returns the error message string for a RespError.

```go
func (e *RespError) Error() string
```

--------------------------------

### Get service region

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Infers and returns the service region from the S3 domain.

```go
func (s *S3) Region() string
```

--------------------------------

### Get S3 object reader

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Provides a reader for downloading data using parallel ranged requests. Call Close on the returned reader to release resources.

```go
func (b *Bucket) GetReader(path string, c *Config) (r io.ReadCloser, h http.Header, err error)
```

--------------------------------

### Get S3 object writer

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Provides a writer for uploading data via multipart requests. Call Close on the returned writer to ensure resources are released.

```go
func (b *Bucket) PutWriter(path string, h http.Header, c *Config) (w io.WriteCloser, err error)
```

--------------------------------

### Create new S3 instance

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Initializes a new S3 service instance.

```go
func New(domain string, keys Keys) *S3
```

--------------------------------

### Create a Default Parser

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Initializes a new parser using the current executable name from os.Args[0].

```go
func NewParser(data interface{}, options Options) *Parser
```

--------------------------------

### Write Help and Man Pages

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Outputs help messages or man pages to a provided io.Writer.

```go
func (p *Parser) WriteHelp(writer io.Writer)
```

```go
func (p *Parser) WriteManPage(wr io.Writer)
```

--------------------------------

### Create a Named Parser

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Initializes a new parser with a specific application name.

```go
func NewNamedParser(appname string, options Options) *Parser
```

--------------------------------

### Command Line Argument Ordering

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Demonstrates the valid and invalid ordering of global options relative to commands.

```text
Valid:   ./app -v add
Invalid: ./app add -v
```

--------------------------------

### Enable Bash Completion

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Shows how to trigger the completion routine by setting the GO_FLAGS_COMPLETION environment variable.

```bash
GO_FLAGS_COMPLETION=1 ./completion-example arg1 arg2 arg3
```

--------------------------------

### Option methods

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Methods for retrieving option metadata and values.

```go
func (option *Option) LongNameWithNamespace() string
```

```go
func (option *Option) String() string
```

```go
func (option *Option) Value() interface{}
```

--------------------------------

### Usage Interface

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Interface for providing custom usage strings in help messages.

```go
type Usage interface {
	// Usage is called for commands to allow customized printing of command
	// usage in the generated help message.
	Usage() string
}
```

--------------------------------

### Stream and Copy Data with gof3r

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r

Basic command-line patterns for streaming data to/from S3 and copying files between local storage and S3 buckets.

```bash
To stream up to S3:
   $  <input_stream> | gof3r put -b <bucket> -k <s3_path>
To stream down from S3:
   $ gof3r get -b <bucket> -k <s3_path> | <output_stream>
To upload a file to S3:
   $ gof3r cp <local_path> s3://<bucket>/<s3_path> -m<http_header1> -m<http_header2>...
To download a file from S3:
   $ gof3r cp s3://<bucket>/<s3_path> <local_path>
```

--------------------------------

### Retrieve subcommands

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Returns a list of subcommands for the current command.

```go
func (c *Command) Commands() []*Command
```

--------------------------------

### Parse command line options

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses command line options with default settings using a pointer to a struct.

```go
func Parse(data interface{}) ([]string, error)
```

--------------------------------

### Bash Completion Script

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

A sample bash function to integrate go-flags completion into a shell environment.

```bash
_completion_example() {
    # All arguments except the first one
    args=("${COMP_WORDS[@]:1:$COMP_CWORD}")

    # Only split on newlines
    local IFS=$'\n'

    # Call completion (note that the first element of COMP_WORDS is
    # the executable itself)
    COMPREPLY=($(GO_FLAGS_COMPLETION=1 ${COMP_WORDS[0]} "${args[@]}"))
    return 0
}

complete -F _completion_example completion-example
```

--------------------------------

### Parse INI configuration

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses command line options with default settings from an INI formatted file.

```go
func IniParse(filename string, data interface{}) error
```

--------------------------------

### NewParser

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Creates a new parser using os.Args[0] as the application name.

```APIDOC
## func NewParser(data interface{}, options Options) *Parser

### Description
Creates a new parser. It uses os.Args[0] as the application name. The provided data is a pointer to a struct representing the default option group, or nil if the default group should not be added.
```

--------------------------------

### Define command-line parsing options

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Constants used to configure the behavior of the command-line parser, such as help flag support, error printing, and argument handling.

```go
const (
	// None indicates no options.
	None Options = 0

	// HelpFlag adds a default Help Options group to the parser containing
	// -h and --help options. When either -h or --help is specified on the
	// command line, the parser will return the special error of type
	// ErrHelp. When PrintErrors is also specified, then the help message
	// will also be automatically printed to os.Stderr.
	HelpFlag = 1 << iota

	// PassDoubleDash passes all arguments after a double dash, --, as
	// remaining command line arguments (i.e. they will not be parsed for
	// flags).
	PassDoubleDash

	// IgnoreUnknown ignores any unknown options and passes them as
	// remaining command line arguments instead of generating an error.
	IgnoreUnknown

	// PrintErrors prints any errors which occurred during parsing to
	// os.Stderr.
	PrintErrors

	// PassAfterNonOption passes all arguments after the first non option
	// as remaining command line arguments. This is equivalent to strict
	// POSIX processing.
	PassAfterNonOption

	// Default is a convenient default set of options which should cover
	// most of the uses of the flags package.
	Default = HelpFlag | PrintErrors | PassDoubleDash
)
```

--------------------------------

### INI Parser Operations

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Methods for initializing and using the INI parser.

```go
func NewIniParser(p *Parser) *IniParser
```

```go
func (i *IniParser) Parse(reader io.Reader) error
```

--------------------------------

### Define map-based command line options

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Use a map type in a struct to capture key:value pairs from command line arguments.

```go
type Options struct {
    AuthorInfo string[string] `short:"a"`
}
```

--------------------------------

### Option struct definition

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Defines the configuration for an individual command-line option.

```go
type Option struct {
	// The description of the option flag. This description is shown
	// automatically in the built-in help.
	Description string

	// The short name of the option (a single character). If not 0, the
	// option flag can be 'activated' using -<ShortName>. Either ShortName
	// or LongName needs to be non-empty.
	ShortName rune

	// The long name of the option. If not "", the option flag can be
	// activated using --<LongName>. Either ShortName or LongName needs
	// to be non-empty.
	LongName string

	// The default value of the option.
	Default []string

	// The optional environment default value key name.
	EnvDefaultKey string

	// The optional delimiter string for EnvDefaultKey values.
	EnvDefaultDelim string

	// If true, specifies that the argument to an option flag is optional.
	// When no argument to the flag is specified on the command line, the
	// value of Default will be set in the field this option represents.
	// This is only valid for non-boolean options.
	OptionalArgument bool

	// The optional value of the option. The optional value is used when
	// the option flag is marked as having an OptionalArgument. This means
	// that when the flag is specified, but no option argument is given,
	// the value of the field this option represents will be set to
	// OptionalValue. This is only valid for non-boolean options.
	OptionalValue []string

	// If true, the option _must_ be specified on the command line. If the
	// option is not specified, the parser will generate an ErrRequired type
	// error.
	Required bool

	// A name for the value of an option shown in the Help as --flag [ValueName]
	ValueName string

	// A mask value to show in the help instead of the default value. This
	// is useful for hiding sensitive information in the help, such as
	// passwords.
	DefaultMask string
	// contains filtered or unexported fields
}
```

--------------------------------

### Retrieve command arguments

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Returns a list of positional arguments associated with the command.

```go
func (c *Command) Args() []*Arg
```

--------------------------------

### Define Command struct

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Represents an application command that can be added to a parser.

```go
type Command struct {
	// Embedded, see Group for more information
	*Group

	// The name by which the command can be invoked
	Name string

	// The active sub command (set by parsing) or nil
	Active *Command

	// Whether subcommands are optional
	SubcommandsOptional bool

	// Aliases for the command
	Aliases []string

	// Whether positional arguments are required
	ArgsRequired bool
	// contains filtered or unexported fields
}
```

--------------------------------

### Generate Man Page

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r

Command to generate the manual page for the gof3r utility.

```bash
gof3r -m
```

--------------------------------

### Parse Command Line Arguments

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses arguments from os.Args or a provided slice, returning non-option arguments.

```go
func (p *Parser) Parse() ([]string, error)
```

```go
func (p *Parser) ParseArgs(args []string) ([]string, error)
```

--------------------------------

### Request instance keys

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Fetches AWS keys from EC2 instance metadata.

```go
func InstanceKeys() (keys Keys, err error)
```

--------------------------------

### NewNamedParser

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Creates a new parser with a specified application name and configuration options.

```APIDOC
## func NewNamedParser(appname string, options Options) *Parser

### Description
Creates a new parser. The appname is used to display the executable name in the built-in help message. Option groups and commands can be added to this parser by using AddGroup and AddCommand.
```

--------------------------------

### func Parse

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses command line options using default settings and os.Args.

```APIDOC
## func Parse(data interface{}) ([]string, error)

### Description
Parse is a convenience function to parse command line options with default settings. The provided data is a pointer to a struct representing the default option group.

### Parameters
- **data** (interface{}) - Required - A pointer to a struct to populate with options.
```

--------------------------------

### Define Config structure

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Contains configuration parameters for s3gof3r, including concurrency settings and MD5 verification options.

```go
type Config struct {
	*http.Client       // http client to use for requests
	Concurrency  int   // number of parts to get or put concurrently
	PartSize     int64 // initial  part size in bytes to use for multipart gets or puts
	NTry         int   // maximum attempts for each part
	Md5Check     bool  // The md5 hash of the object is stored in <bucket>/.md5/<object_key>.md5
	// When true, it is stored on puts and verified on gets
	Scheme    string // url scheme, defaults to 'https'
	PathStyle bool   // use path style bucket addressing instead of virtual host style
}
```

--------------------------------

### Stream Data to and from S3

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Use Unix pipes to stream data directly to or from S3 buckets using the gof3r command-line tool.

```bash
  $ tar -czf - <my_dir/> | gof3r put -b <s3_bucket> -k <s3_object>    
  $ gof3r get -b <s3_bucket> -k <s3_object> | tar -zx
```

--------------------------------

### Define Completion struct

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Contains information about a specific completion item.

```go
type Completion struct {
	// The completed item
	Item string

	// A description of the completed item (optional)
	Description string
}
```

--------------------------------

### Find a subcommand

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Locates a subcommand by name, returning nil if not found.

```go
func (c *Command) Find(name string) *Command
```

--------------------------------

### Parser.Parse

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses command line arguments from os.Args.

```APIDOC
## func (p *Parser) Parse() ([]string, error)

### Description
Parses the command line arguments from os.Args using Parser.ParseArgs.
```

--------------------------------

### Configure AWS Credentials

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r

Set required AWS credentials as environment variables for authentication.

```bash
$ export AWS_ACCESS_KEY_ID=<access_key>
$ export AWS_SECRET_ACCESS_KEY=<secret_key>
```

--------------------------------

### Group Management Methods

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Methods for manipulating option groups, including adding, finding, and listing groups and options.

```go
func (g *Group) AddGroup(shortDescription string, longDescription string, data interface{}) (*Group, error)
```

```go
func (g *Group) Find(shortDescription string) *Group
```

```go
func (g *Group) Groups() []*Group
```

```go
func (g *Group) Options() []*Option
```

--------------------------------

### Define Keys structure

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Holds AWS credentials used for signing HTTP requests.

```go
type Keys struct {
	AccessKey     string
	SecretKey     string
	SecurityToken string
}
```

--------------------------------

### Define INI parsing options

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Constants used to configure how INI files are parsed and written, including default value handling and comment inclusion.

```go
const (
	// IniNone indicates no options.
	IniNone IniOptions = 0

	// IniIncludeDefaults indicates that default values should be written.
	IniIncludeDefaults = 1 << iota

	// IniCommentDefaults indicates that if IniIncludeDefaults is used
	// options with default values are written but commented out.
	IniCommentDefaults

	// IniIncludeComments indicates that comments containing the description
	// of an option should be written.
	IniIncludeComments

	// IniDefault provides a default set of options.
	IniDefault = IniIncludeComments
)
```

--------------------------------

### Add a group to a command

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Adds a new group of options to the command.

```go
func (c *Command) AddGroup(shortDescription string, longDescription string, data interface{}) (*Group, error)
```

--------------------------------

### Define Arg struct

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Represents a positional argument on the command line.

```go
type Arg struct {
	// The name of the positional argument (used in the help)
	Name string

	// A description of the positional argument (used in the help)
	Description string
	// contains filtered or unexported fields
}
```

--------------------------------

### Define Bucket structure

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Represents an S3 bucket with associated configuration and service details.

```go
type Bucket struct {
	*S3
	Name string
	*Config
}
```

--------------------------------

### func (*Command) AddCommand

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Adds a new subcommand to the parser.

```APIDOC
## func (c *Command) AddCommand(command string, shortDescription string, longDescription string, data interface{}) (*Command, error)

### Description
AddCommand adds a new command to the parser with the given name and data. The data needs to be a pointer to a struct from which the fields indicate which options are in the command.

### Parameters
- **command** (string) - Required - The name of the command.
- **shortDescription** (string) - Required - A short description for help output.
- **longDescription** (string) - Required - A long description for help output.
- **data** (interface{}) - Required - A pointer to a struct representing the command's options.
```

--------------------------------

### Define Completer interface

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Interface for providing custom command line argument completion.

```go
type Completer interface {
	// Complete receives a prefix representing a (partial) value
	// for its type and should provide a list of possible valid
	// completions.
	Complete(match string) []Completion
}
```

--------------------------------

### Write INI configuration

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Writes current flag values to an INI format writer or file.

```go
func (i *IniParser) Write(writer io.Writer, options IniOptions)
```

```go
func (i *IniParser) WriteFile(filename string, options IniOptions) error
```

--------------------------------

### Parser configuration types

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Types used for parser configuration and structure.

```go
type Options uint
```

```go
type Parser struct {
	// Embedded, see Command for more information
	*Command
```

--------------------------------

### Define S3 structure

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Contains the endpoint domain and authentication keys for an S3-compatible service.

```go
type S3 struct {
	Domain string // The s3-compatible endpoint. Defaults to "s3.amazonaws.com"
	Keys
}
```

--------------------------------

### Add a command to the parser

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Adds a new command to the parser with a name and associated data struct.

```go
func (c *Command) AddCommand(command string, shortDescription string, longDescription string, data interface{}) (*Command, error)
```

--------------------------------

### Read environment keys

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Retrieves AWS keys from the environment variables.

```go
func EnvKeys() (keys Keys, err error)
```

--------------------------------

### Filename Completion

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Type and method for providing filename completion functionality.

```go
type Filename string
```

```go
func (f *Filename) Complete(match string) []Completion
```

--------------------------------

### Define Commander interface

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Interface for commands to implement custom execution logic.

```go
type Commander interface {
	// Execute will be called for the last active (sub)command. The
	// args argument contains the remaining command line arguments. The
	// error that Execute returns will be eventually passed out of the
	// Parse method of the Parser.
	Execute(args []string) error
}
```

--------------------------------

### IniParser.WriteFile

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Writes the flags as INI format into a file.

```APIDOC
## func (*IniParser) WriteFile

### Description
Writes the flags as ini format into a file. Returns an error if the file could not be opened for writing.

### Signature
func (i *IniParser) WriteFile(filename string, options IniOptions) error

### Parameters
- **filename** (string) - Required - The path to the file to write.
- **options** (IniOptions) - Required - Configuration options for the write operation.
```

--------------------------------

### Error Handling Methods

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Methods for retrieving error messages from the Error type.

```go
func (e *Error) Error() string
```

--------------------------------

### Parser.WriteHelp

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Writes a help message to the provided writer.

```APIDOC
## func (p *Parser) WriteHelp(writer io.Writer)

### Description
Writes a help message containing all the possible options and their descriptions to the provided writer.
```

--------------------------------

### Parse specific arguments

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses a provided list of command line arguments into a struct.

```go
func ParseArgs(data interface{}, args []string) ([]string, error)
```

--------------------------------

### Parse INI file

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses flags from an INI formatted file.

```go
func (i *IniParser) ParseFile(filename string) error
```

--------------------------------

### Parser.WriteManPage

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Writes a man page in groff format.

```APIDOC
## func (p *Parser) WriteManPage(wr io.Writer)

### Description
Writes a basic man page in groff format to the specified writer.
```

--------------------------------

### Parser.ParseArgs

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses provided command line arguments.

```APIDOC
## func (p *Parser) ParseArgs(args []string) ([]string, error)

### Description
Parses the command line arguments according to the option groups that were added to the parser. On successful parsing, the remaining, non-option arguments are returned.
```

--------------------------------

### Default Configuration Definition

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

The DefaultConfig variable defines the standard settings for S3 operations when a custom configuration is not provided.

```go
var DefaultConfig = &Config{
	Concurrency: 10,
	PartSize:    20 * mb,
	NTry:        10,
	Md5Check:    true,
	Scheme:      "https",
	Client:      ClientWithTimeout(clientTimeout),
}
```

--------------------------------

### func IniParse

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses command line options with default settings from an INI formatted file.

```APIDOC
## func IniParse(filename string, data interface{}) error

### Description
IniParse is a convenience function to parse command line options with default settings from an ini formatted file. The provided data is a pointer to a struct representing the default option group.

### Parameters
- **filename** (string) - Required - The path to the INI file.
- **data** (interface{}) - Required - A pointer to a struct to populate with options.
```

--------------------------------

### ClientWithTimeout Function Signature

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Returns an http.Client optimized for high-throughput S3 access with aggressive timeouts and TCP connection deadlines.

```go
func ClientWithTimeout(timeout time.Duration) *http.Client
```

--------------------------------

### func ClientWithTimeout

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Creates an http.Client optimized for high throughput to S3 with aggressive timeout settings.

```APIDOC
## func ClientWithTimeout(timeout time.Duration) *http.Client

### Description
Creates an http.Client optimized for high throughput to S3. It times out more aggressively than the default http client in net/http and sets deadlines on the TCP connection.

### Parameters
- **timeout** (time.Duration) - Required - The duration for the client timeout.
```

--------------------------------

### SetLogger Function Signature

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Configures the internal logging for s3gof3r using the standard library log package. Debug logging is disabled by default.

```go
func SetLogger(out io.Writer, prefix string, flag int, debug bool)
```

--------------------------------

### IniParser.Write

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Writes the current values of all flags to an io.Writer in INI format.

```APIDOC
## func (*IniParser) Write

### Description
Writes the current values of all the flags to an ini format. Typically called after settings have been parsed.

### Signature
func (i *IniParser) Write(writer io.Writer, options IniOptions)

### Parameters
- **writer** (io.Writer) - Required - The destination to write the INI content.
- **options** (IniOptions) - Required - Configuration options for the write operation.
```

--------------------------------

### Godeps directory warning

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps

This text file serves as a warning that the directory is auto-generated and should not be modified.

```text
This directory tree is generated automatically by godep.

Please do not edit.

See https://github.com/tools/godep for more information.
```

--------------------------------

### IniParser.ParseFile

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses flags from an INI formatted file.

```APIDOC
## func (*IniParser) ParseFile

### Description
Parses flags from an ini formatted file. The returned errors can be of the type flags.Error or flags.IniError.

### Signature
func (i *IniParser) ParseFile(filename string) error

### Parameters
- **filename** (string) - Required - The path to the INI file to parse.
```

--------------------------------

### Sign HTTP request

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Signs an HTTP request for S3 authentication.

```go
func (b *Bucket) Sign(req *http.Request)
```

--------------------------------

### func SetLogger

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Configures the internal logging for the s3gof3r package.

```APIDOC
## func SetLogger(out io.Writer, prefix string, flag int, debug bool)

### Description
Wraps the standard library log package to set the internal logging output and format for s3gof3r.

### Parameters
- **out** (io.Writer) - Required - The output destination for logs.
- **prefix** (string) - Required - The prefix for log entries.
- **flag** (int) - Required - The logging flags.
- **debug** (bool) - Required - Whether to enable debug logging.
```

--------------------------------

### INI Error Handling

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Structure and method for handling errors related to INI file parsing.

```go
type IniError struct {
	// The error message.
	Message string

	// The filename of the file in which the error occurred.
	File string

	// The line number at which the error occurred.
	LineNumber uint
}
```

```go
func (x *IniError) Error() string
```

--------------------------------

### func ParseArgs

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Parses command line options from a provided slice of arguments.

```APIDOC
## func ParseArgs(data interface{}, args []string) ([]string, error)

### Description
ParseArgs is a convenience function to parse command line options with default settings using a provided list of arguments.

### Parameters
- **data** (interface{}) - Required - A pointer to a struct to populate with options.
- **args** ([]string) - Required - The list of command line arguments to parse.
```

--------------------------------

### Default Domain Definition

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

DefaultDomain specifies the endpoint for the U.S. S3 service.

```go
var DefaultDomain = "s3.amazonaws.com"
```

--------------------------------

### ErrorType Constants

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Enumeration of possible error types that can occur during flag parsing.

```go
const (
	// ErrUnknown indicates a generic error.
	ErrUnknown ErrorType = iota

	// ErrExpectedArgument indicates that an argument was expected.
	ErrExpectedArgument

	// ErrUnknownFlag indicates an unknown flag.
	ErrUnknownFlag

	// ErrUnknownGroup indicates an unknown group.
	ErrUnknownGroup

	// ErrMarshal indicates a marshalling error while converting values.
	ErrMarshal

	// ErrHelp indicates that the built-in help was shown (the error
	// contains the help message).
	ErrHelp

	// ErrNoArgumentForBool indicates that an argument was given for a
	// boolean flag (which don't not take any arguments).
	ErrNoArgumentForBool

	// ErrRequired indicates that a required flag was not provided.
	ErrRequired

	// ErrShortNameTooLong indicates that a short flag name was specified,
	// longer than one character.
	ErrShortNameTooLong

	// ErrDuplicatedFlag indicates that a short or long flag has been
	// defined more than once
	ErrDuplicatedFlag

	// ErrTag indicates an error while parsing flag tags.
	ErrTag

	// ErrCommandRequired indicates that a command was required but not
	// specified
	ErrCommandRequired

	// ErrUnknownCommand indicates that an unknown command was specified.
	ErrUnknownCommand
)
```

--------------------------------

### Define ErrNotPointerToStruct variable

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Indicates that a provided data container is not a pointer to a struct.

```go
var ErrNotPointerToStruct = errors.New("provided data is not a pointer to struct")
```

--------------------------------

### Marshaler interface

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Interface for types that can marshal themselves to a string representation of a flag.

```go
type Marshaler interface {
	// MarshalFlag marshals a flag value to its string representation.
	MarshalFlag() (string, error)
}
```

--------------------------------

### Delete S3 object

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Removes the key at the specified path. Returns nil if the path does not exist.

```go
func (b *Bucket) Delete(path string) error
```

--------------------------------

### Define Error struct

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Represents an error encountered during parsing.

```go
type Error struct {
	// The type of error
	Type ErrorType
```

--------------------------------

### Define RespError structure

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r

Represents an S3 HTTP error response.

```go
type RespError struct {
	Code       string
	Message    string
	Resource   string
	RequestID  string `xml:"RequestId"`
	StatusCode int
}
```

--------------------------------

### ErrorType Stringer

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Method to convert an ErrorType to its string representation.

```go
func (e ErrorType) String() string
```

--------------------------------

### Unmarshaler Interface

Source: https://pkg.go.dev/github.com/rlmcpherson/s3gof3r/gof3r/Godeps/_workspace/src/github.com/jessevdk/go-flags

Interface for types that handle their own flag unmarshaling.

```go
type Unmarshaler interface {
	// UnmarshalFlag unmarshals a string value representation to the flag
	// value (which therefore needs to be a pointer receiver).
	UnmarshalFlag(value string) error
}
```
Tokens:
Raw