Go Rabin

https://pkg.go.dev/github.com/aclements/go-rabin
Admin
The go-rabin package implements Rabin hashing (aka fingerprinting) and content-defined chunking...
Tokens:2,361
Snippets:46
Trust Score:9.6
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install go-rabin

Source: https://pkg.go.dev/github.com/aclements/go-rabin%40v0.0.0-20170911142644-d0b643ea1a4c

Command to download the go-rabin package.

```bash
go get -d -u github.com/aclements/go-rabin/rabin
```

--------------------------------

### Install cdsplit demo

Source: https://pkg.go.dev/github.com/aclements/go-rabin%40v0.0.0-20170911142644-d0b643ea1a4c

Command to install the cdsplit demo program.

```bash
go get github.com/aclements/go-rabin/cmd/cdsplit
```

--------------------------------

### Get Hash Block Size

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Returns the configured window size or 1 if not configured.

```go
func (h *Hash) BlockSize() int
```

--------------------------------

### Get Hash Size

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Returns the number of bytes required to represent the hash.

```go
func (h *Hash) Size() int
```

--------------------------------

### Initialize New Table

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Creates a table for constructing Rabin hashes with a specific polynomial and window size.

```go
func NewTable(polynomial uint64, window int) *Table
```

--------------------------------

### Initialize New Chunker

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Creates a chunker with specified byte constraints. Requires a table and an io.Reader.

```go
func NewChunker(table *Table, r io.Reader, minBytes, avgBytes, maxBytes int) *Chunker
```

--------------------------------

### Initialize New Hash

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Creates a new Rabin hash instance using the provided table.

```go
func New(table *Table) *Hash
```

--------------------------------

### New(table *Table) *Hash

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Initializes a new Rabin hash instance using the provided pre-computed table.

```APIDOC
## func New

### Description
Returns a new Rabin hash using the polynomial and window size represented by the provided table.
```

--------------------------------

### Format a byte size as a string

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

The String method returns a pretty-printed representation of the byte size using SI prefixes.

```go
func (b Bytes) String() string
```

--------------------------------

### NewTable(polynomial uint64, window int) *Table

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Creates a new Table for constructing Rabin hashes using a specific polynomial and window size.

```APIDOC
## func NewTable

### Description
Returns a Table for constructing Rabin hashes using the specified polynomial and window size.

### Parameters
- **polynomial** (uint64) - The irreducible polynomial over GF(2).
- **window** (int) - The window size for the hash.
```

--------------------------------

### Import rabin package

Source: https://pkg.go.dev/github.com/aclements/go-rabin%40v0.0.0-20170911142644-d0b643ea1a4c

Import statement for including the rabin package in Go projects.

```go
import "github.com/aclements/go-rabin/rabin"
```

--------------------------------

### Write Data to Hash

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Adds data to the running hash, potentially expiring old window bytes.

```go
func (h *Hash) Write(p []byte) (n int, err error)
```

--------------------------------

### Compute 64-bit Hash

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Returns the 64-bit hash of all written bytes.

```go
func (h *Hash) Sum64() uint64
```

--------------------------------

### Parse a byte size string

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

The Set method parses a string into bytes, supporting SI prefixes.

```go
func (b *Bytes) Set(s string) error
```

--------------------------------

### Create a new Bytes flag

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

FlagBytes defines a flag and returns a pointer to the storage location.

```go
func FlagBytes(name string, value Bytes, usage string) *Bytes
```

--------------------------------

### Define Hash Type

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Computes Rabin fingerprints and implements hash.Hash64.

```go
type Hash struct {
	// contains filtered or unexported fields
}
```

--------------------------------

### NewChunker(table *Table, r io.Reader, minBytes, avgBytes, maxBytes int) *Chunker

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Creates a content-defined chunker that divides a byte stream into chunks based on Rabin fingerprints.

```APIDOC
## func NewChunker

### Description
Returns a content-defined chunker for data read from an io.Reader using the Rabin hash defined by the table. The chunks produced will be between minBytes and maxBytes, with an average size of avgBytes.
```

--------------------------------

### Reset Hash State

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Resets the hash instance to its initial state.

```go
func (h *Hash) Reset()
```

--------------------------------

### Define Table Type

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Stores pre-computed tables for Rabin fingerprinting.

```go
type Table struct {
	// contains filtered or unexported fields
}
```

--------------------------------

### Define a Bytes flag variable

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

Use FlagBytesVar to define a flag that stores its value in a provided pointer.

```go
func FlagBytesVar(p *Bytes, name string, value Bytes, usage string)
```

--------------------------------

### Retrieve Next Chunk Length

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Returns the length of the next chunk or an error if the stream ends.

```go
func (c *Chunker) Next() (int, error)
```

--------------------------------

### func FlagBytes

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

Defines a Bytes flag and returns a pointer to the variable where the value will be stored.

```APIDOC
## func FlagBytes(name string, value Bytes, usage string) *Bytes

### Description
Defines a Bytes flag and returns a pointer to the variable where the value of the flag will be stored.

### Parameters
- **name** (string) - Required - The name of the flag.
- **value** (Bytes) - Required - The default value of the flag.
- **usage** (string) - Required - A description of the flag's purpose.

### Returns
- ***Bytes** - A pointer to the Bytes variable storing the flag value.
```

--------------------------------

### Compute Hash Sum

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Appends the current hash to a byte slice.

```go
func (h *Hash) Sum(b []byte) []byte
```

--------------------------------

### Define Rabin Polynomial Constant

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Represents a 64-bit irreducible polynomial over GF(2) used for Rabin hashing.

```go
const Poly64 = 0xbfe6b8a5bf378d83
```

--------------------------------

### Define Discarder Interface

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Interface for skipping bytes in an input stream efficiently.

```go
type Discarder interface {
	// Discard skips the next n bytes, returning the number of
	// bytes discarded.
	//
	// If Discard skips fewer than n bytes, it also returns an
	// error. Discard must not skip beyond the end of the file.
	Discard(n int) (discarded int, err error)
}
```

--------------------------------

### Define Chunker Type

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Performs content-defined chunking to divide byte sequences into chunks.

```go
type Chunker struct {
	// contains filtered or unexported fields
}
```

--------------------------------

### Define the Bytes type

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

The Bytes type represents a byte size and implements the flag.Value interface.

```go
type Bytes int64
```

--------------------------------

### type Bytes

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

Represents a byte size and provides methods for flag parsing and string representation.

```APIDOC
## type Bytes

### Description
Bytes is a byte size type that implements the flag.Value interface.

### Methods
- **Set(s string) error**: Parses the string s into bytes, accepting SI prefixes.
- **String() string**: Returns a pretty-printed string representation of the byte size using an SI prefix.
```

--------------------------------

### func FlagBytesVar

Source: https://pkg.go.dev/github.com/aclements/go-rabin/cmd/internal/cdflags

Defines a Bytes flag and stores the value in the provided pointer.

```APIDOC
## func FlagBytesVar(p *Bytes, name string, value Bytes, usage string)

### Description
Defines a Bytes flag. The argument p points to a Bytes variable in which to store the value of the flag.

### Parameters
- **p** (*Bytes) - Required - Pointer to the Bytes variable to store the flag value.
- **name** (string) - Required - The name of the flag.
- **value** (Bytes) - Required - The default value of the flag.
- **usage** (string) - Required - A description of the flag's purpose.
```

--------------------------------

### Define Rabin Hashing Polynomial

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Mathematical representation of the Rabin hash function polynomial over GF(2).

```text
p(x) = ... + p₂x² + p₁x + p₀   where pₙ ∈ GF(2)
```

--------------------------------

### Reset Chunker State

Source: https://pkg.go.dev/github.com/aclements/go-rabin/rabin

Clears the internal buffer. The underlying reader must be at a chunk boundary.

```go
func (c *Chunker) Reset()
```
Tokens:
Raw