pyzstd

https://pyzstd.readthedocs.io/en/latest
Admin
pyzstd is a Python library providing Zstandard (zstd) compression support with an API similar to the...
Tokens:37,739
Snippets:344
Trust Score:9.6
Update:5 months ago
Show doc for...
Tokens:
Raw
### ZstdCompressor Usage Examples

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Examples demonstrating how to use the ZstdCompressor for streaming compression.

```APIDOC
c = ZstdCompressor()

# traditional streaming compression
dat1 = c.compress(b'123456')
dat2 = c.compress(b'abcdef')
dat3 = c.flush()

# use .compress() method with mode argument
compressed_dat1 = c.compress(raw_dat1, c.FLUSH_BLOCK)
compressed_dat2 = c.compress(raw_dat2, c.FLUSH_FRAME)
```

--------------------------------

### ZstdCompressor Usage Example

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Demonstrates basic compression and flushing using the ZstdCompressor object. Ensure raw_dat is defined before use.

```python
c = ZstdCompressor(level_or_option=option)
compressed_dat1 = c.compress(raw_dat)
compressed_dat2 = c.flush()
```

--------------------------------

### Streaming Decompression with Limited Output

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Use EndlessZstdDecompressor for streaming decompression when the output size is unknown or potentially very large. This example limits the output buffer to prevent excessive memory usage.

```python
d2 = EndlessZstdDecompressor()

while True:
    if d2.needs_input:
        dat = read_input(2*1024*1024) # read 2 MiB input data
        if not dat: # input stream ends
            if not d2.at_frame_edge:
                raise Exception('data ends in an incomplete frame.')
            break
    else: # maybe there is unconsumed input data
        dat = b''

    chunk = d2.decompress(dat, 10*1024*1024) # limit output buffer to 10 MiB
    write_output(chunk)
```

--------------------------------

### Get Zstandard Frame Information

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Use get_frame_info to extract decompressed size and dictionary ID from a zstd frame header. Ensure the input buffer starts at the frame beginning and contains at least the header.

```python
>>> pyzstd.get_frame_info(compressed_dat[:20])
frame_info(decompressed_size=687379, dictionary_id=1040992268)
```

--------------------------------

### Compression with Strategy and Checksum

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Shows how to configure compression using `CParameter.strategy` and `CParameter.checksumFlag`.

```APIDOC
## Compression with Strategy and Checksum

### Description
Configure compression parameters, including the compression strategy and whether to include a checksum.

### Usage
```python
option = {CParameter.strategy : Strategy.lazy2,
          CParameter.checksumFlag : 1}
compressed_dat = compress(raw_dat, option=option)
```

### Parameters
#### `CParameter.strategy`
- **Type**: `Strategy` (IntEnum)
- **Description**: Used for compression strategies, listed from fastest to strongest. New strategies might be added in the future, only the order (from fast to strong) is guaranteed.

#### `CParameter.checksumFlag`
- **Type**: Integer (0 or 1)
- **Description**: Flag to indicate whether to include a checksum in the compressed data.
```

--------------------------------

### Get Zstandard Compression Level Values

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Access compressionLevel_values to get the default, minimum, and maximum compression levels supported by the underlying zstd library.

```python
>>> pyzstd.compressionLevel_values  # 131072 = 128*1024
values(default=3, min=-131072, max=22)
```

--------------------------------

### Set Advanced Compression Parameters

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Demonstrates setting advanced compression parameters using a dictionary with CParameter enums. Used with the compress() function.

```python
option = {CParameter.compressionLevel : 10,
          CParameter.checksumFlag : 1}

# used with compress() function
compressed_dat = compress(raw_dat, option)
```

--------------------------------

### Accessing Compression Parameter Bounds

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Shows how to retrieve the inclusive lower and upper bounds for compression parameters like compressionLevel, windowLog, and enableLongDistanceMatching.

```python
CParameter.compressionLevel.bounds()
CParameter.windowLog.bounds()
CParameter.enableLongDistanceMatching.bounds()
```

--------------------------------

### Get Zstandard Library Version (String)

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Access the zstd_version variable to retrieve the version of the underlying zstd library as a string.

```python
>>> pyzstd.zstd_version
'1.4.5'
```

--------------------------------

### Get Zstandard Library Version (Tuple)

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Access the zstd_version_info variable to retrieve the version of the underlying zstd library as a tuple of integers.

```python
>>> pyzstd.zstd_version_info
(1, 4, 5)
```

--------------------------------

### Load and Use Zstd Dictionary for Compression/Decompression

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Demonstrates loading a zstd dictionary from a file and using it with `compress` and `decompress` functions. Reusing compressor objects can improve performance when applying the same dictionary multiple times.

```python
import io
from pyzstd import ZstdDict, compress, decompress

# load a zstd dictionary from file
with io.open(dict_path, 'rb') as f:
    file_content = f.read()
zd = ZstdDict(file_content)

# use the dictionary to compress.
# if use a dictionary for compressor multiple times, reusing
# a compressor object is faster, see .as_undigested_dict doc.
compressed_dat = compress(raw_dat, zstd_dict=zd)

# use the dictionary to decompress
decompressed_dat = decompress(compressed_dat, zstd_dict=zd)
```

--------------------------------

### Get Compression Level Bounds

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Retrieves the inclusive lower and upper bounds for the compression level parameter using the bounds() method of the CParameter enum.

```python
>>> CParameter.compressionLevel.bounds()
(-131072, 22)
```

--------------------------------

### Replace compress_stream with pyzstd.open

Source: https://pyzstd.readthedocs.io/en/latest/_sources/deprecated.md.txt

Use `pyzstd.open` for stream compression instead of `compress_stream`. For more control, consider `ZstdCompressor`.

```python
# before
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        compress_stream(ifh, ofh, level_or_option=5)

# after
with io.open(input_file_path, 'rb') as ifh:
    with pyzstd.open(output_file_path, 'w', level_or_option=5) as ofh:
        shutil.copyfileobj(ifh, ofh)
```

```python
# after: more complex alternative
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        compressor = ZstdCompressor(level_or_option=5)
        compressor._set_pledged_input_size(pledged_input_size)  # optional
        while data := ifh.read(read_size):
            ofh.write(compressor.compress(data))
            callback_progress(ifh.tell(), ofh.tell())  # optional
        ofh.write(compressor.flush())
```

--------------------------------

### Initialize ZstdCompressor

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Initialize a ZstdCompressor object. The compression level or advanced parameters can be specified, or a pre-trained dictionary can be provided. Defaults to zstd's default compression level.

```python
c = ZstdCompressor()
```

--------------------------------

### Get Bounds for DParameter.windowLogMax

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Retrieve the lower and upper inclusive bounds for the DParameter.windowLogMax parameter. This helps in understanding the valid range for memory allocation limits.

```python
DParameter.windowLogMax.bounds()
```

--------------------------------

### Initialize ZstdCompressor

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Create a `ZstdCompressor` object for streaming compression. This object can be initialized with a compression level (integer) or a dictionary of advanced compression parameters.

```python
compressor = ZstdCompressor(level_or_option=None, zstd_dict=None)
```

--------------------------------

### Compress and Decompress Tar Archives with Zstd

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Demonstrates how to open and interact with .tar.zst archives for both writing (compression) and reading (decompression) using a custom TarFile implementation.

```python
with CustomTarFile.open('archive.tar.zst', mode='w:zst') as tar:
    # do something

with CustomTarFile.open('archive.tar.zst') as tar:
    # do something
```

--------------------------------

### Get Decompression Parameter Bounds

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

The bounds() method of a DParameter enum member returns the inclusive lower and upper bounds for its valid values. This helps in understanding the acceptable range for specific decompression settings.

```python
>>> DParameter.windowLogMax.bounds()
(10, 31)
```

--------------------------------

### Replace compress_stream with pyzstd.open

Source: https://pyzstd.readthedocs.io/en/latest/deprecated.html

Use `pyzstd.open` for stream compression instead of `compress_stream`. For more control, consider using `ZstdCompressor`.

```python
# before
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        compress_stream(ifh, ofh, level_or_option=5)
```

```python
# after
with io.open(input_file_path, 'rb') as ifh:
    with pyzstd.open(output_file_path, 'w', level_or_option=5) as ofh:
        shutil.copyfileobj(ifh, ofh)
```

```python
# after: more complex alternative
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        compressor = ZstdCompressor(level_or_option=5)
        compressor._set_pledged_input_size(pledged_input_size)  # optional
        while data := ifh.read(read_size):
            ofh.write(compressor.compress(data))
            callback_progress(ifh.tell(), ofh.tell())  # optional
        ofh.write(compressor.flush())
```

--------------------------------

### Get Zstandard Frame Size

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Use get_frame_size to determine the total size of a zstd frame, including header and optional checksum. The input buffer must contain at least one complete frame.

```python
>>> pyzstd.get_frame_size(compressed_dat)
252874
```

--------------------------------

### hashLog

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Sets the size of the initial probe table as a power of 2. Affects compression ratio and speed.

```APIDOC
## hashLog

### Description
Determines the size of the initial probe table, expressed as a power of 2. The resulting memory usage is `1 << (hashLog + 2)` bytes. Larger tables can improve compression ratio for certain strategies and speed for others.

### Parameter Details
- **hashLog**: Size of the initial probe table, as a power of 2. Memory usage is `1 << (hashLog + 2)` bytes. Must be clamped between `ZSTD_HASHLOG_MIN` and `ZSTD_HASHLOG_MAX`. Larger tables improve compression ratio of strategies <= `dfast`, and improve speed of strategies > `dfast`. Special value `0` means “use default hashLog”, then the value is dynamically set.

### Bounds
Refer to `CParameter.hashLog.bounds()` for specific bounds.
```

--------------------------------

### Set Compression Strategy and Checksum Flag

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Configure compression using CParameter.strategy and CParameter.checksumFlag. Strategy.lazy2 is used for a balance between speed and compression ratio, and checksumFlag enables data integrity checks.

```python
option = {CParameter.strategy : Strategy.lazy2,
          CParameter.checksumFlag : 1}
compressed_dat = compress(raw_dat, option)
```

--------------------------------

### Streaming Compression with ZstdCompressor

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Demonstrates traditional streaming compression using ZstdCompressor. Data is fed in chunks, and the compressor manages frame and block flushing.

```python
c = ZstdCompressor()

# traditional streaming compression
dat1 = c.compress(b'123456')
dat2 = c.compress(b'abcdef')
dat3 = c.flush()
```

--------------------------------

### Traditional Streaming Compression with ZstdCompressor

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Demonstrates traditional streaming compression using the ZstdCompressor. Data is fed in chunks using the compress() method, and remaining data is flushed using flush().

```python
# traditional streaming compression
dat1 = c.compress(b'123456')
dat2 = c.compress(b'abcdef')
dat3 = c.flush()
```

--------------------------------

### Decompression with Memory Limit

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Demonstrates how to set a memory allocation limit for decompression using `DParameter.windowLogMax`. This is useful when dealing with frames that might require more memory than desired.

```APIDOC
## Decompression with Memory Limit

### Description
Set a memory allocation limit (in power of 2) for decompression to protect the host from unreasonable memory requirements. If a frame requires more memory than the set value, a `ZstdError` exception is raised.

### Usage
```python
# Set memory allocation limit to 16 MiB (1 << 24)
option = {DParameter.windowLogMax : 24}

# Used with decompress() function
decompressed_dat = decompress(dat, option=option)

# Used with ZstdDecompressor object
d = ZstdDecompressor(option=option)
decompressed_dat = d.decompress(dat)
```

### Parameters
#### `DParameter.windowLogMax`
- **Type**: Integer
- **Description**: Select a size limit (in power of 2) beyond which the streaming API will refuse to allocate memory buffer. Special value `0` means "use default maximum windowLog". By default, accepted window sizes are <= `(1 << 27)` (128 MiB). If a frame requests a window size greater than this value, this parameter needs to be explicitly set.
```

--------------------------------

### Compressing data with a prefix for patching

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Use VER_1 as a prefix to compress VER_2, enabling efficient patching. Ensure windowLog is set appropriately and long distance matching is enabled. The resulting PATCH is smaller than the full VER_2.

```python
# use VER_1 as prefix
v1 = ZstdDict(VER_1, is_raw=True)

# let the window cover the longest version.
# don't forget to clamp windowLog to valid range.
# enable "long distance matching".
windowLog = max(len(VER_1), len(VER_2)).bit_length()
option = {CParameter.windowLog: windowLog,
          CParameter.enableLongDistanceMatching: 1}

# get a small PATCH
PATCH = compress(VER_2, level_or_option=option, zstd_dict=v1.as_prefix)

```

--------------------------------

### CParameter Usage

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Demonstrates how to use the CParameter enum to set advanced compression options. Parameters are provided as a dictionary where keys are CParameter enum members and values are integers.

```APIDOC
## CParameter

### Description
Advanced compression parameters. When using, put the parameters in a `dict` object, the key is a `CParameter` name, the value is a 32-bit signed integer value.

### Example
```python
option = {CParameter.compressionLevel : 10,
          CParameter.checksumFlag : 1}

# used with compress() function
compressed_dat = compress(raw_dat, option)
```
```

--------------------------------

### ZstdFile Initialization

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Initializes a ZstdFile object for reading, writing, or appending. The `filename` argument can be a file path or a file object. Advanced compression parameters can be specified using `level_or_option` and `zstd_dict`. `max_frame_content_size` is crucial for compression modes to control frame size and seeking performance.

```APIDOC
## ZstdFile.__init__

### Description
Initializes a ZstdFile object. For append modes ('a', 'ab'), `filename` must be a file path.

### Parameters
* **filename** (str/bytes/PathLike or file object) - The file to open.
* **mode** (str) - The mode to open the file in (e.g., 'r', 'w', 'a', 'rb', 'wb', 'ab'). Defaults to 'r'.
* **level_or_option** (int or dict) - Compression level or advanced compression options.
* **zstd_dict** (ZstdDict) - A dictionary for compression/decompression.
* **max_frame_content_size** (int) - Maximum size of a compression frame. Used in compression modes.

### Attention
* `max_frame_content_size` is used for compression modes ('w', 'wb', 'a', 'ab', 'x', 'xb').
* When uncompressed data reaches this size, the current frame is automatically closed.
* Default value (1 GiB) is often not optimal; user should set based on data and seeking needs.
* Smaller frame sizes improve seeking speed but reduce compression ratio. Larger sizes do the opposite.
* Avoid very small frame sizes (<1 KiB) as they significantly hurt compression ratio.
* Frames can be manually closed using `f.flush(mode=f.FLUSH_FRAME)`.
```

--------------------------------

### Set Advanced Compression Options

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Demonstrates how to set advanced compression parameters using the CParameter enum. These options can be passed to the compress() function or a ZstdCompressor object.

```python
option = {CParameter.compressionLevel : 10,
          CParameter.checksumFlag : 1}

# used with compress() function
compressed_dat = compress(raw_dat, option)

# used with ZstdCompressor object
c = ZstdCompressor(level_or_option=option)
compressed_dat1 = c.compress(raw_dat)
compressed_dat2 = c.flush()
```

--------------------------------

### Compression Level

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Explains the different compression levels available, including regular, ultra, default, and negative levels.

```APIDOC
## Compression Level

### Description
Compression level is an integer that determines the trade-off between compression speed and ratio.

### Levels
- **Regular Levels**: `1` to `22` (currently). Levels >= 20, labeled _ultra_, should be used with caution as they require more memory.
- **Default Level**: `0` means use the default level, which is currently `3`.
- **Negative Levels**: `-131072` to `-1`. Lower levels mean faster speed but lower compression ratio.

### Advanced Usage
Compression levels map to a set of compression parameters. Setting a compression level may dynamically impact other un-set compression parameters. Manually set parameters will remain.

### Note
`compressionLevel_values` are defined by the underlying zstd library. For advanced users, compression levels are numbers that map to a set of compression parameters. The parameters may be adjusted by the underlying zstd library after gathering information, such as data size, using a dictionary or not. Setting a compression level does not set all other compression parameters to default. Setting this will dynamically impact the compression parameters which have not been manually set, the manually set ones will “stick”.
```

--------------------------------

### compress

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Compresses data using the Zstandard algorithm. It can accept a compression level (int) or advanced parameters (dict). Optionally, a pre-trained dictionary can be provided for better compression ratios.

```APIDOC
## compress(data, level_or_option=None, zstd_dict=None)

### Description
Compress *data*, return the compressed data.

Compressing ``b''`` will get an empty content frame (9 bytes or more).

### Parameters
* **data** (bytes-like object) - Data to be compressed.
* **level_or_option** (int or dict) - When it's an ``int`` object, it represents :ref:`compression level<compression_level>`. When it's a ``dict`` object, it contains :ref:`advanced compression parameters<CParameter>`. The default value ``None`` means to use zstd's default compression level/parameters.
* **zstd_dict** (ZstdDict) - Pre-trained dictionary for compression.

### Returns
* **bytes** - Compressed data

### Examples
```python
# int compression level
compressed_dat = compress(raw_dat, 10)

# dict option, use 6 threads to compress, and append a 4-byte checksum.
option = {
    CParameter.compressionLevel : 10,
    CParameter.nbWorkers : 6,
    CParameter.checksumFlag : 1
}
compressed_dat = compress(raw_dat, option)
```
```

--------------------------------

### open function

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Opens a zstd-compressed file, supporting both binary and text modes.

```APIDOC
## open(filename, mode='r', ...)

### Description
Opens a zstd-compressed file in binary or text mode. This function acts as a convenient wrapper for creating or accessing Zstandard compressed files, similar to Python's built-in `open()` function.

### Parameters
- **filename**: The path to the file to be opened (string, bytes, or path-like object).
- **mode**: The mode in which to open the file. Supports 'r', 'w', 'x', 'a' for reading, writing, exclusive creation, and appending, respectively. Can be combined with 'b' for binary mode (e.g., 'rb', 'wb') or 't' for text mode (e.g., 'rt', 'wt'). Defaults to 'r'.
- **level_or_option**: Compression level or option, used when writing.
- **zstd_dict**: A Zstandard dictionary for compression or decompression.

### Returns
A file-like object (instance of `ZstdFile` or a similar object for text mode) that allows reading from or writing to the compressed file.
```

--------------------------------

### Decompressing data with a prefix for patching

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Decompress a PATCH using the same prefix (VER_1) used during compression. Explicitly set windowLogMax if the required window size exceeds 128MiB. Failure to use the correct prefix will result in a 'Data corruption detected' error.

```python
# use VER_1 as prefix
v1 = ZstdDict(VER_1, is_raw=True)

# allow large window, the actual windowLog is from frame header.
option = {DParameter.windowLogMax: 31}

# get VER_2 from (VER_1 + PATCH)
VER_2 = decompress(PATCH, zstd_dict=v1.as_prefix, option=option)

```

--------------------------------

### ZstdCompressor compress() with Mode Parameter

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Shows how to use the compress() method with explicit mode arguments (FLUSH_BLOCK, FLUSH_FRAME) for more control over data flushing and frame management.

```python
# use .compress() method with mode argument
compressed_dat1 = c.compress(raw_dat1, c.FLUSH_BLOCK)
compressed_dat2 = c.compress(raw_dat2, c.FLUSH_FRAME)
```

--------------------------------

### hashLog

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Determines the size of the initial probe table, affecting compression ratio and speed. Special value 0 uses the default.

```APIDOC
## hashLog

### Description
Size of the initial probe table, as a power of 2, resulting memory usage is ``1 << (hashLog+2)`` bytes.

Must be clamped between ``ZSTD_HASHLOG_MIN`` and ``ZSTD_HASHLOG_MAX``.

Larger tables improve compression ratio of strategies <= :py:attr:`~Strategy.dfast`, and improve speed of strategies > :py:attr:`~Strategy.dfast`.

Special: value ``0`` means "use default hashLog", then the value is dynamically set, see "H" column in `this table <https://github.com/facebook/zstd/blob/release/lib/compress/clevels.h>`_.
```

--------------------------------

### open() Function

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

The open() function provides a convenient way to open zstd-compressed files in either binary or text mode. It mirrors the functionality of bz2.open(), gzip.open(), and lzma.open() from the standard library.

```APIDOC
## open()

### Description
Opens a zstd-compressed file in binary or text mode, returning a file-like object. Similar to standard library open functions for compression.

### Parameters

- **filename** (str | bytes | path-like object | file object): The file to open or wrap.
- **mode** (str): 'r', 'rb', 'w', 'wb', 'x', 'xb', 'a', 'ab' for binary mode; 'rt', 'wt', 'xt', 'at' for text mode. Default is 'rb'.
- **level_or_option** (dict): Decompression options when in reading mode. Does not support integer compression levels.
- **zstd_dict** (zstd_dict): A dictionary for Zstandard dictionary compression/decompression.
- **encoding** (str, optional): The encoding to use for text mode.
- **errors** (str, optional): Error handling scheme for text mode.
- **newline** (str, optional): Controls universal newlines mode for text mode.

### Return Value

- **ZstdFile**: In binary mode.
- **io.TextIOWrapper**: In text mode, wrapping a ZstdFile object.
```

--------------------------------

### Multi-threaded Compression

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Details how to enable multi-threaded compression using the `CParameter.nbWorkers` parameter.

```APIDOC
## Multi-threaded Compression

### Description
Enable multi-threaded compression by setting the `CParameter.nbWorkers` parameter to a value greater than or equal to 1.

### Usage
```python
# Enable multi-threaded compression with 4 workers
option = {CParameter.nbWorkers : 4}
compressed_dat = compress(raw_dat, option=option)
```

### Parameters
#### `CParameter.nbWorkers`
- **Type**: Integer
- **Description**: Set to `>= 1` to enable multi-threaded compression. `1` means "1-thread multi-threaded mode". The threads are spawned by the underlying zstd library, not by the pyzstd module.
```

--------------------------------

### Loading and Using a Zstd Dictionary

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Load a zstd dictionary from a file and use it for compression. Reusing a compressor object is recommended for multiple compressions with the same dictionary for better performance.

```python
# load a zstd dictionary from file
with io.open(dict_path, 'rb') as f:
    file_content = f.read()
zd = ZstdDict(file_content)

# use the dictionary to compress.
# if use a dictionary for compressor multiple times, reusing
# a compressor object is faster, see .as_undigested_dict doc.
compressed_dat = compress(raw_dat, zstd_dict=zd)
```

--------------------------------

### Replace decompress_stream with pyzstd.open

Source: https://pyzstd.readthedocs.io/en/latest/_sources/deprecated.md.txt

Use `pyzstd.open` for stream decompression instead of `decompress_stream`. For more control, consider `EndlessZstdDecompressor`.

```python
# before
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        decompress_stream(ifh, ofh)

# after
with pyzstd.open(input_file_path) as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        shutil.copyfileobj(ifh, ofh)
```

```python
# after: more complex alternative
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        decompressor = EndlessZstdDecompressor()
        while True:
            if decompressor.needs_input:
                data = input_stream.read(read_size)
                if not data:
                    break
            else:
                data = b"")
            ofh.write(decompressor.decompress(data, write_size))
            callback_progress(ifh.tell(), ofh.tell())  # optional
        if not decompressor.at_frame_edge:
            raise ValueError("zstd data ends in an incomplete frame")
```

--------------------------------

### Replace richmem_compress with compress

Source: https://pyzstd.readthedocs.io/en/latest/deprecated.html

Use the `pyzstd.compress` function for memory-efficient compression instead of `richmem_compress`.

```python
# before
data_out = pyzstd.richmem_compress(data_in, level_or_option=5)
```

```python
# after
data_out = pyzstd.compress(data_in, level_or_option=5)
```

--------------------------------

### ZstdCompressor Object Usage

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Demonstrates the basic usage of the ZstdCompressor object for compression and flushing data.

```APIDOC
## ZstdCompressor Object Usage

### Description
This snippet shows how to instantiate a `ZstdCompressor` object with a specified compression option and then use it to compress raw data and flush any remaining compressed data.

### Code Example
```python
c = ZstdCompressor(level_or_option=option)
compressed_dat1 = c.compress(raw_dat)
compressed_dat2 = c.flush()
```
```

--------------------------------

### Compressing with a Zstd Dictionary Prefix

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Compress data using a Zstd dictionary as a prefix. Ensure windowLog is clamped to the valid range and enable long distance matching for optimal results. The dictionary must be created with `is_raw=True`.

```python
v1 = ZstdDict(VER_1, is_raw=True)

# let the window cover the longest version.
# don't forget to clamp windowLog to valid range.
# enable "long distance matching".
windowLog = max(len(VER_1), len(VER_2)).bit_length()
option = {CParameter.windowLog: windowLog,
          CParameter.enableLongDistanceMatching: 1}

# get a small PATCH
PATCH = compress(VER_2, level_or_option=option, zstd_dict=v1.as_prefix)
```

--------------------------------

### Replace decompress_stream with pyzstd.open

Source: https://pyzstd.readthedocs.io/en/latest/deprecated.html

Use `pyzstd.open` for stream decompression instead of `decompress_stream`. For more control, consider using `EndlessZstdDecompressor`.

```python
# before
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        decompress_stream(ifh, ofh)
```

```python
# after
with pyzstd.open(input_file_path) as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        shutil.copyfileobj(ifh, ofh)
```

```python
# after: more complex alternative
with io.open(input_file_path, 'rb') as ifh:
    with io.open(output_file_path, 'wb') as ofh:
        decompressor = EndlessZstdDecompressor()
        while True:
            if decompressor.needs_input:
                data = input_stream.read(read_size)
                if not data:
                    break
            else:
                data = b"")
            ofh.write(decompressor.decompress(data, write_size))
            callback_progress(ifh.tell(), ofh.tell())  # optional
        if not decompressor.at_frame_edge:
            raise ValueError("zstd data ends in an incomplete frame")
```

--------------------------------

### strategy

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Selects the compression strategy, with higher values indicating more complex and slower compression. Special value 0 uses the default.

```APIDOC
## strategy

### Description
See :py:attr:`Strategy` class definition.

The higher the value of selected strategy, the more complex it is, resulting in stronger and slower compression.

Special: value ``0`` means "use default strategy", then the value is dynamically set, see "strat" column in `this table <https://github.com/facebook/zstd/blob/release/lib/compress/clevels.h>`_.
```

--------------------------------

### ZstdDict Initialization

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Initializes a ZstdDict object with dictionary content. The `is_raw` parameter allows for advanced usage with raw dictionary content.

```APIDOC
## ZstdDict.__init__

### Description
Initialize a ZstdDict object.

### Parameters
* **dict_content** (_bytes-like object_) – Dictionary’s content.
* **is_raw** (_bool_) – This parameter is for advanced user. `True` means _dict_content_ argument is a “raw content” dictionary, free of any format restriction. `False` means _dict_content_ argument is an ordinary zstd dictionary, was created by zstd functions, follow a specified format.
```

--------------------------------

### Module-level variables

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Provides access to Zstandard library version information and compression level constants.

```APIDOC
## Module-level variables

### zstd_version

Underlying zstd library's version as a string.

#### Example
```python
import pyzstd

print(pyzstd.zstd_version)
```

### zstd_version_info

Underlying zstd library's version as a tuple.

#### Example
```python
import pyzstd

print(pyzstd.zstd_version_info)
```

### compressionLevel_values

A namedtuple containing the minimum, maximum, and default compression levels supported by the underlying zstd library.

#### Example
```python
import pyzstd

print(pyzstd.compressionLevel_values)
```

### zstd_support_multithread

A boolean indicating whether the underlying zstd library was compiled with multi-threaded compression support.

#### Example
```python
import pyzstd

print(pyzstd.zstd_support_multithread)
```
```

--------------------------------

### Replace richmem_compress with compress

Source: https://pyzstd.readthedocs.io/en/latest/_sources/deprecated.md.txt

Use `pyzstd.compress` instead of `pyzstd.richmem_compress` for in-memory compression.

```python
# before
data_out = pyzstd.richmem_compress(data_in, level_or_option=5)

# after
data_out = pyzstd.compress(data_in, level_or_option=5)
```

--------------------------------

### Compress with Digested vs. Undigested Dictionary

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Illustrates how to explicitly use a digested or undigested dictionary for compression. Undigested dictionaries are used by default and may offer better performance by avoiding a costly digestion operation.

```python
compress(dat, zstd_dict=zd.as_digested_dict)
```

```python
compress(dat, zstd_dict=zd)
```

--------------------------------

### chainLog

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Configures the size of the multi-probe search table, influencing compression quality and speed. Special value 0 uses the default.

```APIDOC
## chainLog

### Description
Size of the multi-probe search table, as a power of 2, resulting memory usage is ``1 << (chainLog+2)`` bytes.

Must be clamped between ``ZSTD_CHAINLOG_MIN`` and ``ZSTD_CHAINLOG_MAX``.

Larger tables result in better and slower compression.

This parameter is useless for :py:attr:`~Strategy.fast` strategy.

It's still useful when using :py:attr:`~Strategy.dfast` strategy, in which case it defines a secondary probe table.

Special: value ``0`` means "use default chainLog", then the value is dynamically set, see "C" column in `this table <https://github.com/facebook/zstd/blob/release/lib/compress/clevels.h>`_.
```

--------------------------------

### compress()

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Compresses data using zstd. It can accept an integer for compression level or a dictionary for advanced parameters. A pre-trained dictionary can also be provided.

```APIDOC
## compress(data, level_or_option=None, zstd_dict=None)

### Description
Compress _data_, return the compressed data. Compressing `b''` will get an empty content frame (9 bytes or more).

### Parameters

* **data** (_bytes-like object_) – Data to be compressed.
* **level_or_option** (_int_ _or_ _dict_) – When it’s an `int` object, it represents compression level. When it’s a `dict` object, it contains advanced compression parameters. The default value `None` means to use zstd’s default compression level/parameters.
* **zstd_dict** (_ZstdDict_) – Pre-trained dictionary for compression.

### Returns

Compressed data

### Return type

bytes

### Request Example
```python
# int compression level
compressed_dat = compress(raw_dat, 10)

# dict option, use 6 threads to compress, and append a 4-byte checksum.
option = {
    CParameter.compressionLevel : 10,
    CParameter.nbWorkers : 6,
    CParameter.checksumFlag : 1
}
compressed_dat = compress(raw_dat, option)
```
```

--------------------------------

### Compression Strategies (Strategy Enum)

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

Defines compression strategies from fastest to strongest, used for CParameter.strategy.

```APIDOC
## Strategy(IntEnum)

### Description
Used for :py:attr:`~CParameter.strategy`. Compression strategies, listed from fastest to strongest. Note: new strategies might be added in the future, only the order (from fast to strong) is guaranteed.

### Members
- `fast`
- `dfast`
- `greedy`
- `lazy`
- `lazy2`
- `btlazy2`
- `btopt`
- `btultra`
- `btultra2

### Example
```python
option = {CParameter.strategy : Strategy.lazy2,
          CParameter.checksumFlag : 1}
compressed_dat = compress(raw_dat, option)
```
```

--------------------------------

### ZstdCompressor Methods

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

This section covers the methods available for the ZstdCompressor class, including compress, flush, and attributes related to compression modes.

```APIDOC
## ZstdCompressor.compress

### Description
Provide data to the compressor object.

### Method
compress(self, data, mode=ZstdCompressor.CONTINUE)

### Parameters
#### Parameters
- **data** (bytes-like object) - Data to be compressed.
- **mode** (ZstdCompressor.CONTINUE, ZstdCompressor.FLUSH_BLOCK, ZstdCompressor.FLUSH_FRAME) - Compression mode.

### Returns
- A chunk of compressed data if possible, or ``b''`` otherwise.

### Example
```python
c = ZstdCompressor()
compressed_data = c.compress(b'some data')
```

## ZstdCompressor.flush

### Description
Flush any remaining data in internal buffer. Since zstd data consists of one or more independent frames, the compressor object can still be used after this method is called.

**Note**: Abuse of this method will reduce compression ratio, and some programs can only decompress single frame data. Use it only when necessary.

### Method
flush(self, mode=ZstdCompressor.FLUSH_FRAME)

### Parameters
#### Parameters
- **mode** (ZstdCompressor.FLUSH_FRAME, ZstdCompressor.FLUSH_BLOCK) - Flush mode.

### Returns
- Flushed data.

### Example
```python
c = ZstdCompressor()
data_to_flush = c.flush()
```

## ZstdCompressor.last_mode

### Description
The last mode used to this compressor, its value can be :py:attr:`~ZstdCompressor.CONTINUE`, :py:attr:`~ZstdCompressor.FLUSH_BLOCK`, :py:attr:`~ZstdCompressor.FLUSH_FRAME`. Initialized to :py:attr:`~ZstdCompressor.FLUSH_FRAME`. It can be used to get the current state of a compressor, such as, data flushed, a frame ended.

## ZstdCompressor.CONTINUE

### Description
Used for *mode* parameter in :py:meth:`~ZstdCompressor.compress` method. Collect more data, encoder decides when to output compressed result, for optimal compression ratio. Usually used for traditional streaming compression.

## ZstdCompressor.FLUSH_BLOCK

### Description
Used for *mode* parameter in :py:meth:`~ZstdCompressor.compress`, :py:meth:`~ZstdCompressor.flush` methods. Flush any remaining data, but don't close the current :ref:`frame<frame_block>`. Usually used for communication scenarios. If there is data, it creates at least one new :ref:`block<frame_block>`, that can be decoded immediately on reception. If no remaining data, no block is created, return ``b''``. **Note**: Abuse of this mode will reduce compression ratio. Use it only when necessary.

## ZstdCompressor.FLUSH_FRAME

### Description
Used for *mode* parameter in :py:meth:`~ZstdCompressor.compress`, :py:meth:`~ZstdCompressor.flush` methods. Flush any remaining data, and close the current :ref:`frame<frame_block>`. Usually used for traditional flush. Since zstd data consists of one or more independent frames, data can still be provided after a frame is closed. **Note**: Abuse of this mode will reduce compression ratio, and some programs can only decompress single frame data. Use it only when necessary.

### Example
```python
c = ZstdCompressor()
compressed_dat1 = c.compress(raw_dat1, c.FLUSH_BLOCK)
compressed_dat2 = c.compress(raw_dat2, c.FLUSH_FRAME)
```
```

--------------------------------

### Replace RichMemZstdCompressor with compress

Source: https://pyzstd.readthedocs.io/en/latest/_sources/deprecated.md.txt

Use `pyzstd.compress` for in-memory compression instead of `pyzstd.RichMemZstdCompressor`.

```python
# before
compressor = pyzstd.RichMemZstdCompressor(level_or_option=5)
data_out1 = compressor.compress(data_in1)
data_out2 = compressor.compress(data_in2)
data_out3 = compressor.compress(data_in3)

# after
data_out1 = pyzstd.compress(data_in1, level_or_option=5)
data_out2 = pyzstd.compress(data_in2, level_or_option=5)
data_out3 = pyzstd.compress(data_in3, level_or_option=5)
```

--------------------------------

### chainLog

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Configures the size of the multi-probe search table as a power of 2. Impacts compression quality and speed.

```APIDOC
## chainLog

### Description
Sets the size of the multi-probe search table, expressed as a power of 2. Memory usage is `1 << (chainLog + 2)` bytes. Larger tables generally lead to better compression but slower performance. This parameter is not effective for the `fast` strategy.

### Parameter Details
- **chainLog**: Size of the multi-probe search table, as a power of 2. Memory usage is `1 << (chainLog + 2)` bytes. Must be clamped between `ZSTD_CHAINLOG_MIN` and `ZSTD_CHAINLOG_MAX`. Larger tables result in better and slower compression. This parameter is useless for `fast` strategy. It’s still useful when using `dfast` strategy, in which case it defines a secondary probe table. Special value `0` means “use default chainLog”, then the value is dynamically set.

### Bounds
Refer to `CParameter.chainLog.bounds()` for specific bounds.
```

--------------------------------

### strategy

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Selects the compression strategy, influencing the trade-off between compression strength and speed.

```APIDOC
## strategy

### Description
Chooses the compression strategy to use. Higher strategy values involve more complex algorithms, resulting in stronger compression but slower performance. A value of `0` indicates the default strategy, which is dynamically determined.

### Parameter Details
- **strategy**: The higher the value of selected strategy, the more complex it is, resulting in stronger and slower compression. Special value `0` means “use default strategy”, then the value is dynamically set.

### Bounds
Refer to `CParameter.strategy.bounds()` for specific bounds.
```

--------------------------------

### compressionLevel

Source: https://pyzstd.readthedocs.io/en/latest/pyzstd.html

Sets compression parameters based on a pre-defined compression level. Manually set parameters are preserved.

```APIDOC
## compressionLevel

### Description
Configures compression settings according to a predefined `compressionLevel` table. This affects dynamically adjustable parameters, while manually set parameters remain unchanged.

### Parameter Details
- **compressionLevel**: Sets compression parameters according to a pre-defined table. See compression level for details. Setting a compression level does not set all other compression parameters to default. Setting this will dynamically impact the compression parameters which have not been manually set; the manually set ones will “stick”.

### Bounds
Refer to `CParameter.compressionLevel.bounds()` for specific bounds.
```

--------------------------------

### ZstdDecompressor Methods

Source: https://pyzstd.readthedocs.io/en/latest/_sources/pyzstd.rst.txt

This section details the ZstdDecompressor class for streaming decompression, including its initialization and decompression method.

```APIDOC
## ZstdDecompressor.__init__

### Description
Initialize a ZstdDecompressor object.

### Method
__init__(self, zstd_dict=None, option=None)

### Parameters
#### Parameters
- **zstd_dict** (ZstdDict) - Pre-trained dictionary for decompression.
- **option** (dict) - A ``dict`` object that contains :ref:`advanced decompression parameters<DParameter>`. The default value ``None`` means to use zstd's default decompression parameters.

## ZstdDecompressor.decompress

### Description
Decompress *data*, returning decompressed data as a ``bytes`` object.

### Method
decompress(self, data, max_length=-1)

### Parameters
#### Parameters
- **data** (bytes-like object) - Data to be decompressed.
- **max_length** (int) - Maximum length of data to decompress. Defaults to -1 (no limit).

### Returns
- Decompressed data as a ``bytes`` object.

### Example
```python
d = ZstdDecompressor()
decompressed_data = d.decompress(b'compressed data')
```
```
Tokens:
Raw