### User Guide for Documentation Navigation

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

A step-by-step guide for users on how to effectively navigate and utilize the documentation.

```markdown
1. **Start with [README.md](README.md)** — Explains the structure
2. **Choose your document** — Based on your task
3. **Follow the links** — Cross-references guide you through related content
4. **Try the examples** — Copy and adapt code patterns
5. **Check source** — Line numbers let you verify in the actual code
```

--------------------------------

### Go Zstd Bulk Compression and Decompression Example

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/bulk.md

Demonstrates how to use the Zstd Bulk Processor to compress and then decompress data using a pre-trained dictionary. The destination buffer is allocated automatically. This is suitable for scenarios involving multiple messages with common patterns.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	dict := []byte("common_prefix_data")
	processor, _ := zstd.NewBulkProcessor(dict, zstd.DefaultCompression)
	
	original := []byte("common_prefix_data_block_1")
	
	// Compress
	compressed, _ := processor.Compress(nil, original)
	fmt.Printf("Compressed from %d to %d bytes\n", len(original), len(compressed))
	
	// Decompress
	decompressed, err := processor.Decompress(nil, compressed)
	if err != nil {
		fmt.Println("Decompression failed:", err)
		return
	}
	
	fmt.Println("Match:", string(original) == string(decompressed))
}
```

--------------------------------

### configuration.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the configuration.md file, including lines, sections, examples, and code blocks.

```markdown
| configuration.md | 275 | 8 | 12 | 30 |
```

--------------------------------

### Example: Handling Decompression Errors

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/types.md

Demonstrates how to handle decompression errors by checking the returned error and printing its type and message. This is useful for debugging and user feedback.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	// Attempt decompression with invalid data
	invalid := []byte{0xFF, 0xFF, 0xFF, 0xFF}
	_, err := zstd.Decompress(nil, invalid)
	
	if err != nil {
		fmt.Println("Error type:", fmt.Sprintf("%T", err))
		fmt.Println("Error message:", err.Error())
	}
}
```

--------------------------------

### Compress Data with Default Level

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/context.md

Compresses multiple data chunks using the default compression level of a reusable context. This example shows how to iterate through data and compress each chunk efficiently.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	ctx := zstd.NewCtx()
	
	chunks := [][]byte{
		[]byte("Chunk 1: Some data"),
		[]byte("Chunk 2: More data"),
		[]byte("Chunk 3: Even more data"),
	}
	
	for i, chunk := range chunks {
		compressed, err := ctx.Compress(nil, chunk)
		if err != nil {
			fmt.Printf("Chunk %d compression failed: %v\n", i, err)
			continue
		}
		fmt.Printf("Chunk %d: %d → %d bytes\n", i, len(chunk), len(compressed))
	}
}
```

--------------------------------

### context.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the context.md file, including lines, sections, examples, and code blocks.

```markdown
| context.md | 359 | 5 | 7 | 18 |
```

--------------------------------

### README.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the README.md file, including lines, sections, examples, and code blocks.

```markdown
| Document | Lines | Sections | Examples | Code Blocks |
|----------|-------|----------|----------|-------------|
| README.md | 317 | 8 | 5 | 12 |
```

--------------------------------

### api-surface.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the api-surface.md file, including lines, sections, examples, and code blocks.

```markdown
| api-surface.md | 464 | 14 | 2 | 25 |
```

--------------------------------

### Zstandard Writer with Dictionary

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Demonstrates how to create a Zstandard writer and reader using a pre-trained dictionary for improved compression of similar messages. The same dictionary must be used for both compression and decompression.

```go
dict := []byte("common patterns in data")
writer := zstd.NewWriterLevelDict(w, level, dict)

reader := zstd.NewReaderDict(r, dict)

processor, _ := zstd.NewBulkProcessor(dict, level)
```

--------------------------------

### Create and Reuse Compression Context

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/context.md

Demonstrates creating a reusable compression context and using it for multiple compress and decompress operations. This is more efficient for bulk operations than creating a new context for each operation.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	// Create a context once
	ctx := zstd.NewCtx()
	
	// Reuse for multiple operations
	data1 := []byte("First data block")
	data2 := []byte("Second data block")
	data3 := []byte("Third data block")
	
	// Compress all with the same context
	compressed1, _ := ctx.Compress(nil, data1)
	compressed2, _ := ctx.Compress(nil, data2)
	compressed3, _ := ctx.Compress(nil, data3)
	
	// Decompress all with the same context

decompressed1, _ := ctx.Decompress(nil, compressed1)
decompressed2, _ := ctx.Decompress(nil, compressed2)
decompressed3, _ := ctx.Decompress(nil, compressed3)
	
	fmt.Println("Verify match:")
	fmt.Println("  Block 1:", string(data1) == string(decompressed1))
	fmt.Println("  Block 2:", string(data2) == string(decompressed2))
	fmt.Println("  Block 3:", string(data3) == string(decompressed3))
}
```

--------------------------------

### errors.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the errors.md file, including lines, sections, examples, and code blocks.

```markdown
| errors.md | 370 | 8 | 8 | 28 |
```

--------------------------------

### types.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the types.md file, including lines, sections, examples, and code blocks.

```markdown
| types.md | 190 | 6 | 1 | 15 |
```

--------------------------------

### Create New BulkProcessor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/bulk.md

Initializes a new BulkProcessor with a provided dictionary and compression level. This is useful for setting up reusable compression contexts for multiple similar data operations.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	// Dictionary containing common patterns
	dictionary := []byte(`{"name":"","type":"","version":"1.0","status":"ok"}`)
	
	processor, err := zstd.NewBulkProcessor(dictionary, zstd.DefaultCompression)
	if err != nil {
		fmt.Println("Failed to create processor:", err)
		return
	}
	
	// Compress multiple similar messages efficiently
	messages := []string{
		`{"name":"alice","type":"user","version":"1.0","status":"ok"}`,
		`{"name":"bob","type":"user","version":"1.0","status":"ok"}`,
		`{"name":"charlie","type":"user","version":"1.0","status":"ok"}`,
	}
	
	for _, msg := range messages {
		compressed, err := processor.Compress(nil, []byte(msg))
		if err != nil {
			fmt.Println("Compression failed:", err)
			continue
		}
		fmt.Printf("Message %d compressed from %d to %d bytes\n", 
			len(msg), len(compressed))
	}
}
```

--------------------------------

### Create Bulk Processor with Dictionary and Level

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Initializes a Zstd BulkProcessor with a specified dictionary and compression level. The level is fixed upon creation.

```go
processor, err := zstd.NewBulkProcessor(dictionary, level)
```

--------------------------------

### bulk.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the bulk.md file, including lines, sections, examples, and code blocks.

```markdown
| bulk.md | 228 | 3 | 6 | 14 |
```

--------------------------------

### Handle Empty Dictionary for BulkProcessor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/errors.md

Shows how to handle the error returned when creating a BulkProcessor with a nil or empty dictionary. A non-empty dictionary is required for pre-computation.

```Go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	// This fails with ErrEmptyDictionary
	_, err := zstd.NewBulkProcessor(nil, zstd.DefaultCompression)
	if err != nil {
		fmt.Println("Error:", err) // Output: Error: Dictionary is empty
	}
}
```

--------------------------------

### stream.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the stream.md file, including lines, sections, examples, and code blocks.

```markdown
| stream.md | 566 | 10 | 12 | 35 |
```

--------------------------------

### compress.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the compress.md file, including lines, sections, examples, and code blocks.

```markdown
| compress.md | 273 | 4 | 7 | 18 |
```

--------------------------------

### Build with External libzstd

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Builds the Go project using an external libzstd library instead of the vendored source. Requires libzstd 1.4.0 or later and pkg-config.

```bash
go build -tags external_libzstd
```

--------------------------------

### Default Build Command

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/INDEX.md

Builds the Go project using default settings, which includes vendored zstd v1.5.7.

```bash
go build
```

--------------------------------

### Create Bulk Processor with Dictionary

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a dictionary-based processor for bulk compression. Returns an error if the dictionary is invalid.

```go
func NewBulkProcessor(dictionary []byte, compressionLevel int) (*BulkProcessor, error)
```

--------------------------------

### REFERENCE.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the REFERENCE.md file, including lines, sections, examples, and code blocks.

```markdown
| REFERENCE.md | 551 | 12 | 25 | 50 |
```

--------------------------------

### Build with external libzstd

Source: https://github.com/datadog/zstd/blob/1.x/README.md

Use the external_libzstd build tag to link against a system-installed libzstd library instead of the vendored version.

```bash
go build -tags external_libzstd
```

--------------------------------

### INDEX.md File Statistics

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Statistics for the INDEX.md file, including lines, sections, examples, and code blocks.

```markdown
| INDEX.md | 321 | 10 | 8 | 20 |
```

--------------------------------

### io.WriteCloser Interface Implementation

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Demonstrates that the Writer type implements the io.WriteCloser interface, combining io.Writer and io.Closer.

```go
type WriteCloser interface {
    io.Writer
    io.Closer
}

func (w *Writer) Write(p []byte) (int, error)
func (w *Writer) Close() error
```

--------------------------------

### Run Go Benchmarks

Source: https://github.com/datadog/zstd/blob/1.x/README.md

Execute benchmarks for the Zstd Go wrapper. Ensure your payload filepath is exported as the PAYLOAD environment variable before running.

```go
go test -bench .

```

--------------------------------

### Compress Data at Different Levels

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/context.md

Demonstrates compressing data using a Zstd context with varying compression levels, from best speed to best compression. Useful for tuning compression performance.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	ctx := zstd.NewCtx()
	data := []byte("Data to compress at different levels")
	
	// Try different compression levels
	for level := zstd.BestSpeed; level <= zstd.BestCompression; level += 5 {
		compressed, err := ctx.CompressLevel(nil, data, level)
		if err != nil {
			fmt.Printf("Level %d failed: %v\n", level, err)
			continue
		}
		fmt.Printf("Level %2d: %4d bytes\n", level, len(compressed))
	}
}
```

--------------------------------

### Reusable Context with Varying Compression Levels

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Utilize a reusable compression context to perform operations with different compression levels. This allows flexibility in balancing compression speed and ratio across multiple operations.

```go
ctx := zstd.NewCtx()

for i, data := range chunks {
	level := zstd.BestSpeed + (i % 10)
	compressed, _ := ctx.CompressLevel(nil, data, level)
}
```

--------------------------------

### Compress and Decompress Data

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/compress.md

Demonstrates basic compression and decompression of byte slices. The Compress function handles buffer allocation automatically. The Decompress function also manages buffer sizing, falling back to streaming if the destination is too small.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	original := []byte("Hello, World! This is a test.")
	
	// Compress
	compressed, err := zstd.Compress(nil, original)
	if err != nil {
		fmt.Println("Compression failed:", err)
		return
	}
	
	// Decompress
	decompressed, err := zstd.Decompress(nil, compressed)
	if err != nil {
		fmt.Println("Decompression failed:", err)
		return
	}
	
	fmt.Println("Original:", string(original))
	fmt.Println("Decompressed:", string(decompressed))
	fmt.Println("Match:", string(original) == string(decompressed))
}
```

--------------------------------

### Compress Data to a File using Zstd

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

This function demonstrates how to read data from an input file, compress it using Zstd, and write the compressed data to an output file.

```go
func compressFile(inputPath, outputPath string) error {
	input, _ := os.ReadFile(inputPath)
	compressed, err := zstd.Compress(nil, input)
	if err != nil {
		return err
	}
	return os.WriteFile(outputPath, compressed, 0644)
}
```

--------------------------------

### Import zstd Package

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Import the necessary zstd package for compression and decompression operations.

```go
import "github.com/DataDog/zstd"
```

--------------------------------

### Decompress Data into Pre-sized Buffer

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/context.md

Demonstrates decompressing data into a pre-allocated buffer of known size using `DecompressInto`. This is more efficient as it avoids reallocations.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	ctx := zstd.NewCtx()
	
	original := []byte("Test data for decompression")
	compressed, _ := ctx.Compress(nil, original)
	
	// Decompress with exact-size buffer

decompressed := make([]byte, len(original))
	written, err := ctx.DecompressInto(decompressed, compressed)
	if err != nil {
		fmt.Println("Decompression failed:", err)
		return
	}
	
	fmt.Printf("Decompressed %d bytes\n", written)
	fmt.Println("Result:", string(decompressed[:written]))
}
```

--------------------------------

### Streaming Compression with Custom Level to File

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Create a streaming writer with a specific compression level for compressing data to a file. This allows control over the compression-decompression trade-off.

```go
writer := zstd.NewWriterLevel(file, zstd.BestCompression)
defer writer.Close()

for chunk := range dataChunks {
	writer.Write(chunk)
}
```

--------------------------------

### Handle Compression Errors

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/errors.md

Demonstrates how to check for and handle errors that may occur during Zstd compression.

```go
compressed, err := zstd.Compress(nil, data)
if err != nil {
	fmt.Printf("Compression failed: %v\n", err)
	return err
}
```

--------------------------------

### NewCtx Constructor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a new reusable context for compression and decompression operations.

```go
func NewCtx() Ctx
```

--------------------------------

### Select Appropriate Zstd Compression Level

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Choose the compression level based on your needs: `BestSpeed` for real-time, `DefaultCompression` for balanced, and `BestCompression` for archiving. Using excessively high levels like 20 can be too slow for most use cases.

```go
// ❌ Wrong for most cases
zstd.CompressLevel(nil, data, 20) // Too slow for most uses

// ✅ Right for different scenarios
zstd.CompressLevel(nil, data, zstd.BestSpeed)          // Real-time, ~1GB/s
zstd.CompressLevel(nil, data, zstd.DefaultCompression) // Balanced, ~100MB/s
zstd.CompressLevel(nil, data, zstd.BestCompression)    // Archive, ~10MB/s
```

--------------------------------

### Handle Bad Dictionary for BulkProcessor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/errors.md

Illustrates handling the error when the zstd C library fails to load the provided dictionary for BulkProcessor creation. This can be due to corruption, invalid size, or memory issues.

```Go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	// Very large or invalid dictionary
	badDict := make([]byte, 1<<20) // 1MB of zeros
	_, err := zstd.NewBulkProcessor(badDict, zstd.DefaultCompression)
	if err != nil {
		fmt.Println("Failed to create bulk processor:", err)
	}
}
```

--------------------------------

### Create Zstd Writer with Default Settings

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/stream.md

Use this function to create a streaming writer with default compression settings. It writes compressed data to an underlying io.Writer.

```go
package main

import (
	"fmt"
	"os"
	"github.com/DataDog/zstd"
)

func main() {
	// Create a writer to compress data to a file
	file, err := os.Create("output.zst")
	if err != nil {
		fmt.Println("Failed to create file:", err)
		return
	}
	defer file.Close()
	
	writer := zstd.NewWriter(file)
	defer writer.Close()
	
	// Write data to be compressed
	data := []byte("Hello, World!")
	n, err := writer.Write(data)
	if err != nil {
		fmt.Println("Write failed:", err)
		return
	}
	fmt.Printf("Wrote %d bytes\n", n)
}
```

--------------------------------

### NewWriter Constructor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a new streaming compression writer with the default compression level.

```go
func NewWriter(w io.Writer) *Writer
```

--------------------------------

### Enabling Multi-Threaded Compression

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Enable multi-threaded compression using `SetNbWorkers` on a streaming writer. This can significantly speed up compression for large data, but requires the library to be built with parallel support.

```go
writer := zstd.NewWriter(file)
defer writer.Close()

// Enable 4 parallel workers
if err := writer.SetNbWorkers(4); err != nil {
	fmt.Println("Parallel support not available")
	// Will use single-threaded fallback
}

// Write becomes asynchronous when > 1 worker
writer.Write(largeData)

// Flush periodically to control memory
writer.Flush()
writer.Write(moreData)
writer.Flush()
```

--------------------------------

### Compression with Different Levels

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Compress data using different compression levels, from fastest to best compression ratio. The default level offers a balance.

```go
// Fast compression
fast, _ := zstd.CompressLevel(nil, data, zstd.BestSpeed)

// Balanced (default)
balanced, _ := zstd.CompressLevel(nil, data, zstd.DefaultCompression)

// Maximum compression
small, _ := zstd.CompressLevel(nil, data, zstd.BestCompression)
```

--------------------------------

### Go Compress Data with Default Level

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/compress.md

Compresses data using the default compression level (5). Supports pre-allocated destination buffers for efficiency.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	src := []byte("Hello, World! Hello, World!")
	
	// Compress without pre-allocated buffer
	compressed, err := zstd.Compress(nil, src)
	if err != nil {
		fmt.Println("Compression failed:", err)
		return
	}
	
	fmt.Printf("Original size: %d bytes\n", len(src))
	fmt.Printf("Compressed size: %d bytes\n", len(compressed))
	
	// Compress with pre-allocated buffer
	dst := make([]byte, zstd.CompressBound(len(src)))
	compressed2, err := zstd.Compress(dst, src)
	if err != nil {
		fmt.Println("Compression failed:", err)
		return
	}
	fmt.Printf("Reused buffer result size: %d bytes\n", len(compressed2))
}
```

--------------------------------

### Pre-allocate Output Buffers for Zstd Compression

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Pre-allocating the output buffer using `CompressBound` before compression can prevent multiple allocations and improve performance.

```go
// ❌ Multiple allocations
compressed, _ := zstd.Compress(nil, data)

// ✅ Single allocation
size := zstd.CompressBound(len(data))
buf := make([]byte, size)
compressed, _ := zstd.Compress(buf, data)
```

--------------------------------

### NewWriterLevel Constructor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a new streaming compression writer with a specified compression level.

```go
func NewWriterLevel(w io.Writer, level int) *Writer
```

--------------------------------

### Compress Data with Default Level using Context

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Performs compression using a Zstd context with the default compression level.

```go
ctx.Compress(nil, data)
```

--------------------------------

### Create Zstd Writer with Dictionary and Custom Level

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Creates a new Zstd writer using a provided dictionary and a custom compression level.

```go
writer := zstd.NewWriterLevelDict(w, 10, dictionary)
```

--------------------------------

### Zstd vs. Czlib Compression Benchmark Results

Source: https://github.com/datadog/zstd/blob/1.x/README.md

Benchmark results comparing Zstd (this wrapper) and czlib for compression and decompression of a 7Mb PDF. Note that czlib may be faster for small payloads due to optimizations.

```text
BenchmarkCompression               5     221056624 ns/op      67.34 MB/s
BenchmarkDecompression           100      18370416 ns/op     810.32 MB/s

BenchmarkFzlibCompress             2     610156603 ns/op      24.40 MB/s
BenchmarkFzlibDecompress          20      81195246 ns/op     183.33 MB/s

```

--------------------------------

### Predefined Compression Levels

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Defines constants for common compression levels. Arbitrary levels between 1 and 20 are also supported.

```go
const (
    BestSpeed          = 1
    BestCompression    = 20
    DefaultCompression = 5
)
```

--------------------------------

### NewReader Constructor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a new streaming decompression reader without a dictionary.

```go
func NewReader(r io.Reader) io.ReadCloser
```

--------------------------------

### NewWriterLevelDict Constructor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a new streaming compression writer with a specified level and an optional dictionary.

```go
func NewWriterLevelDict(w io.Writer, level int, dict []byte) *Writer
```

--------------------------------

### NewReaderDict Constructor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a new streaming decompression reader with an optional dictionary.

```go
func NewReaderDict(r io.Reader, dict []byte) io.ReadCloser
```

--------------------------------

### io.ReadCloser Interface Implementation

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Demonstrates that the reader type implements the io.ReadCloser interface, combining io.Reader and io.Closer. Instances are returned from NewReader() and NewReaderDict().

```go
type ReadCloser interface {
    io.Reader
    io.Closer
}

func (r *reader) Read(p []byte) (int, error)
func (r *reader) Close() error
```

--------------------------------

### Compression and Decompression with Pre-allocated Buffers

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Utilize pre-allocated destination buffers for compression and decompression to potentially improve performance by avoiding reallocations. Ensure the buffer size is adequate for decompression.

```go
// For compression
dst := make([]byte, zstd.CompressBound(len(src)))
compressed, _ := zstd.Compress(dst, src)

// For decompression
dst := make([]byte, decompressedSize)
written, err := zstd.DecompressInto(dst, compressed)
if err != nil {
	// Buffer too small or other error
}
result := dst[:written]
```

--------------------------------

### NewBulkProcessor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/bulk.md

Creates a new BulkProcessor with a pre-trained dictionary. This processor can be reused for multiple compression and decompression operations, significantly improving efficiency for bulk data processing.

```APIDOC
## NewBulkProcessor

### Description
Creates a reusable dictionary processor that digests a dictionary once and reuses it for multiple compression/decompression operations. This is much more efficient than loading the dictionary for each operation, especially when compressing many small messages or blocks. The BulkProcessor is thread-safe for read-only operations (multiple threads can call `Compress()` or `Decompress()` concurrently).

The BulkProcessor automatically frees C resources when garbage-collected.

### Parameters
#### Path Parameters
- None
#### Query Parameters
- None
#### Request Body
- None

### Method
`func NewBulkProcessor(dictionary []byte, compressionLevel int) (*BulkProcessor, error)`

### Parameters
- **dictionary** (`[]byte`) - Required - Pre-trained dictionary bytes. Must not be empty.
- **compressionLevel** (`int`) - Required - Compression level from 1 (BestSpeed) to 20 (BestCompression).

### Return Type
Returns a pointer to a `BulkProcessor` instance and an error if dictionary creation fails.

### Errors
- `ErrEmptyDictionary`: If the dictionary is empty or nil.
- `ErrBadDictionary`: If the dictionary cannot be loaded by the C library.

### Example
```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	// Dictionary containing common patterns
	dictionary := []byte(`{"name":"","type":"","version":"1.0","status":"ok"}`)
	
	processor, err := zstd.NewBulkProcessor(dictionary, zstd.DefaultCompression)
	if err != nil {
		fmt.Println("Failed to create processor:", err)
		return
	}
	
	// Compress multiple similar messages efficiently
	messages := []string{
		`{"name":"alice","type":"user","version":"1.0","status":"ok"}`,
		`{"name":"bob","type":"user","version":"1.0","status":"ok"}`,
		`{"name":"charlie","type":"user","version":"1.0","status":"ok"}`,
	}
	
	for _, msg := range messages {
		compressed, err := processor.Compress(nil, []byte(msg))
		if err != nil {
			fmt.Println("Compression failed:", err)
			continue
		}
		fmt.Printf("Message %d compressed from %d to %d bytes\n", 
			len(msg), len(compressed))
	}
}
```
```

--------------------------------

### Create Zstd Writer with Level and Dictionary

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/stream.md

Use this function to create a streaming writer with a specified compression level and an optional dictionary. Dictionaries can improve compression ratios for similar messages. The dictionary must not be modified during use.

```go
package main

import (
	"bytes"
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	var buf bytes.Buffer
	
	// Dictionary for JSON data
	dict := []byte(`{"key":"value","type":"message","status":"ok"}`)
	
	writer := zstd.NewWriterLevelDict(&buf, zstd.DefaultCompression, dict)
	defer writer.Close()
	
	// Now write similar JSON messages
	json1 := []byte(`{"key":"data1","type":"message","status":"ok"}`)
	json2 := []byte(`{"key":"data2","type":"message","status":"ok"}`)
	
	writer.Write(json1)
	writer.Write(json2)
	
	writer.Close()
	fmt.Printf("Dictionary-compressed size: %d bytes\n", buf.Len())
```

--------------------------------

### Ctx Interface

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/types.md

Interface for reusable compression and decompression contexts. Allows for efficient repeated compressions/decompressions using persistent contexts.

```APIDOC
## Ctx Interface

### Description

`Ctx` defines the interface for compression and decompression operations using persistent C contexts. The concrete implementation is the unexported `ctx` type created by `NewCtx()`.

### Methods

| Method | Signature | Description |
|--------|-----------|-------------|
| Compress | `(dst, src []byte) ([]byte, error)` | Compress with default level |
| CompressLevel | `(dst, src []byte, level int) ([]byte, error)` | Compress with specified level |
| Decompress | `(dst, src []byte) ([]byte, error)` | Decompress with auto-sizing |
| DecompressInto | `(dst, src []byte) (int, error)` | Decompress into pre-sized buffer |
```

--------------------------------

### BulkProcessor Methods

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Methods for the BulkProcessor type, used for dictionary-based compression and decompression.

```go
func (p *BulkProcessor) Compress(dst, src []byte) ([]byte, error)
func (p *BulkProcessor) Decompress(dst, src []byte) ([]byte, error)
```

--------------------------------

### Streaming Compression and Decompression with Dictionary

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Use a dictionary for both streaming compression and decompression. This is beneficial when dealing with repetitive data patterns across multiple writes and reads.

```go
dict := []byte("common prefix data")

// Write with dictionary
writer := zstd.NewWriterLevelDict(file, zstd.DefaultCompression, dict)
defer writer.Close()
writer.Write(data1)
writer.Write(data2)

// Read with same dictionary
reader := zstd.NewReaderDict(file, dict)
defer reader.Close()
result, _ := ioutil.ReadAll(reader)
```

--------------------------------

### Reusable Compression Contexts API

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/README.md

Allows for the creation and reuse of compression contexts for performance optimization, supporting per-operation level variations and single-threaded bulk operations.

```APIDOC
## Reusable Compression Contexts API

### Description
Allows for the creation and reuse of compression contexts for performance optimization, supporting per-operation level variations and single-threaded bulk operations.

### Ctx Interface

- **NewCtx() Ctx**
  - Description: Creates a new reusable compression context.
  - Returns:
    - `Ctx`: A new context instance.

### Ctx Methods

- **Compress()**: Compresses data using the context's current settings.
- **CompressLevel(level int) ([]byte, error)**
  - Description: Compresses data using a specified level for this operation.
  - Parameters:
    - `level` (int): The compression level for this specific operation.
  - Returns:
    - `[]byte`: Compressed data.
    - `error`: An error if compression fails.
- **Decompress()**: Decompresses data using the context's current settings.
- **DecompressInto(dst, src []byte) (int, error)**
  - Description: Decompresses data into a pre-sized buffer using the context's current settings.
  - Parameters:
    - `dst` ([]byte): Destination buffer.
    - `src` ([]byte): Source data to decompress.
  - Returns:
    - `int`: The number of bytes written.
    - `error`: An error if decompression fails.
```

--------------------------------

### Use Dictionary for Similar Messages in Zstd

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

For compressing similar messages, such as JSON, using a dictionary with `NewBulkProcessor` can significantly improve compression speed.

```go
// ❌ Slow: without dictionary
for msg := range jsonMessages {
	zstd.Compress(nil, msg)
}

// ✅ Faster: with dictionary
processor, _ := zstd.NewBulkProcessor(jsonDictionary, zstd.DefaultCompression)
for msg := range jsonMessages {
	processor.Compress(nil, msg)
}
```

--------------------------------

### Create Zstd Writer with Default Compression Level

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Creates a new Zstd writer using the default compression level (5).

```go
writer := zstd.NewWriter(w)
```

--------------------------------

### Ctx Interface Definition

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Interface for reusable compression and decompression contexts, defining methods for compress and decompress operations.

```go
type Ctx interface {
    Compress(dst, src []byte) ([]byte, error)
    CompressLevel(dst, src []byte, level int) ([]byte, error)
    Decompress(dst, src []byte) ([]byte, error)
    DecompressInto(dst, src []byte) (int, error)
}
```

--------------------------------

### Compare Zstd API with go-compress/zstd

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

The Datadog Zstd binding provides a simpler API compared to other libraries like `go-compress/zstd`.

```go
// Zstd: Simpler API
compressed, _ := zstd.Compress(nil, data)

// Other: More verbose
enc, _ := zstd.NewWriter(nil)
compressed := enc.EncodeAll(data, nil)
```

--------------------------------

### Compress Data with BulkProcessor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-reference/bulk.md

Compresses source data using the BulkProcessor's dictionary. It supports both unallocated and pre-allocated destination buffers for flexibility and performance.

```go
package main

import (
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	dict := []byte("common_prefix_data")
	processor, _ := zstd.NewBulkProcessor(dict, zstd.DefaultCompression)
	
	data := []byte("common_prefix_data_block_1")
	
	// Without pre-allocated buffer
	compressed, err := processor.Compress(nil, data)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
	fmt.Printf("Compressed to %d bytes\n", len(compressed))
	
	// With pre-allocated buffer
	dst := make([]byte, zstd.CompressBound(len(data)))
	compressed2, err := processor.Compress(dst, data)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
	fmt.Printf("Reused buffer: %d bytes\n", len(compressed2))
}
```

--------------------------------

### BulkProcessor.Compress

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Compresses data using a pre-loaded dictionary with the BulkProcessor.

```APIDOC
## BulkProcessor.Compress

### Description
Compresses data using a pre-loaded dictionary with the BulkProcessor.

### Method
func (bp *BulkProcessor) Compress(dst, src []byte) ([]byte, error)
```

--------------------------------

### Compress Data with Pre-allocated Buffer

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Compresses source data into a pre-allocated destination buffer using a specified compression level.

```go
dst := make([]byte, zstd.CompressBound(len(src)))
compressed, _ := zstd.CompressLevel(dst, src, level)
```

--------------------------------

### Dictionary-based Compression with BulkProcessor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

Compress multiple similar messages efficiently using a dictionary with BulkProcessor. This is ideal for scenarios like compressing many JSON objects with common fields.

```go
dict := []byte(`{"type":"","id":"","data":""}`)

processor, err := zstd.NewBulkProcessor(dict, zstd.DefaultCompression)
if err != nil {
	log.Fatal(err)
}

// Compress many similar messages efficiently
for message := range jsonMessages {
	compressed, _ := processor.Compress(nil, message)
	store(compressed)
}
```

--------------------------------

### Reusable Compression/Decompression Context

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/INDEX.md

Utilizes a reusable `Ctx` for multiple compression and decompression operations. This can improve performance by avoiding the overhead of context creation for each operation.

```Go
ctx := zstd.NewCtx()
for data := range chunks {
	compressed, _ := ctx.Compress(nil, data)
	decompressed, _ := ctx.Decompress(nil, compressed)
}
```

--------------------------------

### Basic Compression and Decompression

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/INDEX.md

Compresses and decompresses byte slices using default settings. The `Compress` function automatically allocates and resizes the destination buffer. The `Decompress` function also handles buffer allocation.

```Go
src := []byte("Hello, World!")
compressed, err := zstd.Compress(nil, src)
decompressed, err := zstd.Decompress(nil, compressed)
```

--------------------------------

### Simple Compression and Decompression

Source: https://github.com/datadog/zstd/blob/1.x/README.md

Functions for compressing and decompressing byte slices. These methods allow for buffer reuse to minimize allocations.

```go
// Compress compresses the byte array given in src and writes it to dst.
// If you already have a buffer allocated, you can pass it to prevent allocation
// If not, you can pass nil as dst.
// If the buffer is too small, it will be reallocated, resized, and returned by the function
// If dst is nil, this will allocate the worst case size (CompressBound(src))
Compress(dst, src []byte) ([]byte, error)
```

```go
// CompressLevel is the same as Compress but you can pass another compression level
CompressLevel(dst, src []byte, level int) ([]byte, error)
```

```go
// Decompress will decompress your payload into dst.
// If you already have a buffer allocated, you can pass it to prevent allocation
// If not, you can pass nil as dst (allocates a 4*src size as default).
// If the buffer is too small, it will retry 3 times by doubling the dst size
// After max retries, it will switch to the slower stream API to be sure to be able
// to decompress. Currently switches if compression ratio > 4*2**3=32.
Decompress(dst, src []byte) ([]byte, error)
```

--------------------------------

### Fast Real-Time Zstandard Compression

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Configures a Zstandard writer for fast, real-time compression, suitable for streaming data. Ensure data is flushed periodically to guarantee it's sent.

```go
writer := zstd.NewWriterLevel(conn, zstd.BestSpeed)
defer writer.Close()

// Write streaming data
for chunk := range dataChan {
	writer.Write(chunk)
	writer.Flush() // Ensure data is sent
}
```

--------------------------------

### NewWriter

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a streaming compression writer with the default compression level.

```APIDOC
## NewWriter

### Description
Creates a streaming compression writer with the default compression level.

### Method
func NewWriter(w io.Writer) *Writer
```

--------------------------------

### File Dependencies Graph

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

A visual representation of the file dependencies within the documentation structure.

```markdown
```
README.md (start here)
    ├→ INDEX.md (overview)
    ├→ REFERENCE.md (quick start)
    ├→ api-reference/
    │   ├→ compress.md
    │   ├→ stream.md
    │   ├→ bulk.md
    │   └→ context.md
    ├→ api-surface.md (symbol index)
    ├→ types.md
    ├→ errors.md
    └→ configuration.md
```
```

--------------------------------

### Check for Parallel Compression Support

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/errors.md

Demonstrates how to check if parallel compression is available by attempting to set the number of workers and catching the ErrNoParallelSupport error. This error occurs if the zstd library was compiled without multi-threading support.

```Go
package main

import (
	"bytes"
	"fmt"
	"github.com/DataDog/zstd"
)

func main() {
	var buf bytes.Buffer
	writer := zstd.NewWriter(&buf)
	defer writer.Close()
	
	err := writer.SetNbWorkers(4)
	if err == zstd.ErrNoParallelSupport {
		fmt.Println("Parallel compression not available")
	}
}
```

--------------------------------

### Calculate Maximum Compressed Size

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Returns the worst-case compressed size for a given input size.

```go
func CompressBound(srcSize int) int
```

--------------------------------

### Compress Data with Custom Level using Context

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/configuration.md

Performs compression using a Zstd context with a specified custom compression level per operation.

```go
ctx.CompressLevel(nil, data, zstd.BestCompression)
```

--------------------------------

### Compression with Custom Level

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/INDEX.md

Compresses data using a specified compression level. `zstd.BestCompression` offers the highest compression ratio but is slower.

```Go
compressed, err := zstd.CompressLevel(nil, src, zstd.BestCompression)
```

--------------------------------

### NewWriterLevel

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a streaming compression writer with a specified compression level.

```APIDOC
## NewWriterLevel

### Description
Creates a streaming compression writer with a specified compression level.

### Method
func NewWriterLevel(w io.Writer, level int) *Writer
```

--------------------------------

### Documentation Version Information

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/MANIFEST.md

Version details for the documentation, including dates, Zstd version, Go version, and generated file statistics.

```markdown
| Aspect | Value |
|--------|-------|
| Documentation Date | June 2, 2026 |
| Zstd Version | 1.5.7 (facebook/zstd) |
| Go Version | 1.24+ |
| Package | github.com/DataDog/zstd |
| Generated Markdown Files | 11 |
| Total Lines | 3,914 |
| Total Size | ~72 KB |
```

--------------------------------

### ctx Methods

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Methods available on the Ctx interface for managing compression contexts.

```APIDOC
## ctx Methods

### Compress

### Description
Compresses data using the context with default compression settings.

### Method
func (c *ctx) Compress(dst, src []byte) ([]byte, error)

### Parameters
#### Path Parameters
- **dst** ([]byte) - Required - The destination buffer for compressed data.
- **src** ([]byte) - Required - The source data to compress.

### Response
#### Success Response
- **[]byte** - The compressed data.
- **error** (error) - An error if compression fails.

---

## ctx Methods

### CompressLevel

### Description
Compresses data using the context with a specified compression level.

### Method
func (c *ctx) CompressLevel(dst, src []byte, level int) ([]byte, error)

### Parameters
#### Path Parameters
- **dst** ([]byte) - Required - The destination buffer for compressed data.
- **src** ([]byte) - Required - The source data to compress.
- **level** (int) - Required - The compression level to use.

### Response
#### Success Response
- **[]byte** - The compressed data.
- **error** (error) - An error if compression fails.

---

## ctx Methods

### Decompress

### Description
Decompresses data using the context.

### Method
func (c *ctx) Decompress(dst, src []byte) ([]byte, error)

### Parameters
#### Path Parameters
- **dst** ([]byte) - Required - The destination buffer for decompressed data.
- **src** ([]byte) - Required - The source data to decompress.

### Response
#### Success Response
- **[]byte** - The decompressed data.
- **error** (error) - An error if decompression fails.

---

## ctx Methods

### DecompressInto

### Description
Decompresses data into the destination buffer using the context.

### Method
func (c *ctx) DecompressInto(dst, src []byte) (int, error)

### Parameters
#### Path Parameters
- **dst** ([]byte) - Required - The destination buffer for decompressed data.
- **src** ([]byte) - Required - The source data to decompress.

### Response
#### Success Response
- **int** - The number of bytes decompressed.
- **error** (error) - An error if decompression fails.
```

--------------------------------

### BulkProcessor Methods

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Methods available on the BulkProcessor for compressing and decompressing data using a dictionary.

```APIDOC
## BulkProcessor Methods

### Compress

### Description
Compresses the source byte slice into the destination byte slice using the configured dictionary.

### Method
func (p *BulkProcessor) Compress(dst, src []byte) ([]byte, error)

### Parameters
#### Path Parameters
- **dst** ([]byte) - Required - The destination buffer to write the compressed data to.
- **src** ([]byte) - Required - The source byte slice to compress.

### Response
#### Success Response
- **[]byte** - The compressed data written to the destination buffer.
- **error** (error) - An error if compression fails.

---

## BulkProcessor Methods

### Decompress

### Description
Decompresses the source byte slice into the destination byte slice using the configured dictionary.

### Method
func (p *BulkProcessor) Decompress(dst, src []byte) ([]byte, error)

### Parameters
#### Path Parameters
- **dst** ([]byte) - Required - The destination buffer to write the decompressed data to.
- **src** ([]byte) - Required - The source byte slice to decompress.

### Response
#### Success Response
- **[]byte** - The decompressed data written to the destination buffer.
- **error** (error) - An error if decompression fails.
```

--------------------------------

### NewReader

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/api-surface.md

Creates a streaming decompression reader without a dictionary.

```APIDOC
## NewReader

### Description
Creates a streaming decompression reader without a dictionary.

### Method
func NewReader(r io.Reader) io.ReadCloser
```

--------------------------------

### Streaming API

Source: https://github.com/datadog/zstd/blob/1.x/README.md

Streaming interfaces for compression and decompression using io.Reader and io.Writer.

```APIDOC
## NewWriter

### Description
Creates a new Writer for streaming compression.

### Parameters
- **w** (io.Writer) - Required - Underlying writer
- **level** (int) - Optional - Compression level
- **dict** ([]byte) - Optional - Precomputed dictionary

## Writer.Write

### Description
Compresses input data and writes it to the underlying writer.

### Parameters
- **p** ([]byte) - Required - Data to compress

## NewReader

### Description
Returns an io.ReadCloser that decompresses data from the underlying reader.

### Parameters
- **r** (io.Reader) - Required - Underlying reader
- **dict** ([]byte) - Optional - Dictionary for decompression
```

--------------------------------

### NewBulkProcessor

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/README.md

Creates a new zstd bulk processor for efficient handling of many similar messages. This function is fully documented.

```APIDOC
## NewBulkProcessor

### Description
Creates a new zstd bulk processor for efficient handling of many similar messages.

### Method
(Implicitly a function call in Go)

### Return Value
- `*zstd.BulkProcessor` - A new zstd bulk processor.

### Example
```go
bp := zstd.NewBulkProcessor()
// Use bp for bulk compression or decompression operations
```
```

--------------------------------

### Compress All Files in a Directory using Zstd

Source: https://github.com/datadog/zstd/blob/1.x/_autodocs/REFERENCE.md

This function iterates through all files in a given directory, reads their content, compresses them using a provided `BulkProcessor`, and indicates where to store the compressed data.

```go
func compressDir(dir string, processor *zstd.BulkProcessor) error {
	entries, _ := os.ReadDir(dir)
	for _, entry := range entries {
		if !entry.IsDir() {
			data, _ := os.ReadFile(filepath.Join(dir, entry.Name()))
			compressed, err := processor.Compress(nil, data)
			if err != nil {
				return err
			}
			// Store compressed
		}
	}
	return nil
}
```