### Testing Package as Installed (Julia) Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/CONTRIBUTING.md Simulates the testing process for a package as if it were installed, by first copying it to a temporary directory, then updating its dependencies within that temporary environment, and finally running its tests. This ensures that local changes in other monorepo packages do not interfere with the test results. ```sh julia -e 'mkdir("temp"); cp("LibZlib", "temp/LibZlib")' julia --project=temp/LibZlib -e 'import Pkg; Pkg.update()' julia --project=temp/LibZlib/test temp/LibZlib/test/runtests.jl ``` -------------------------------- ### Creating GzipEncodeOptions in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This example shows how to instantiate `GzipEncodeOptions` with a specific compression level (e.g., `level=4`) and verifies that the created object is an instance of `ChunkCodecCore.EncodeOptions`, allowing for performance tuning. ```julia-repl e = GzipEncodeOptions(;level=4) GzipEncodeOptions(4) e isa ChunkCodecCore.EncodeOptions true ``` -------------------------------- ### Compressing and Decompressing Data with Brotli in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/LibBrotli/README.md This example demonstrates the basic usage of ChunkCodecLibBrotli to compress a byte array using Brotli and then decompress it. It shows how to specify encoding options like quality and how to provide size hints during decoding to ensure correct data reconstruction. ```Julia julia> using ChunkCodecLibBrotli julia> data = [0x00, 0x01, 0x02, 0x03]; julia> compressed_data = encode(BrotliEncodeOptions(;quality=6), data); julia> decompressed_data = decode(BrotliCodec(), compressed_data; max_size=length(data), size_hint=length(data)); julia> data == decompressed_data true ``` -------------------------------- ### Decoding Gzipped Data in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This comprehensive example shows the full process of decoding previously gzipped data back to its original form using `GzipCodec` and verifies that the decoded data matches the original input, ensuring data integrity. ```julia-repl using ChunkCodecLibZlib data = zeros(UInt8, 1000); gzipped_data = encode(GzipEncodeOptions(;level=4), data); un_gzipped_data = decode(GzipCodec(), gzipped_data); un_gzipped_data == data true ``` -------------------------------- ### Compressing and Decompressing Data with LZ4FrameCodec in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/LibLz4/README.md This example demonstrates how to use `LZ4FrameCodec` from `ChunkCodecLibLz4` to compress and decompress a byte array in Julia. It shows the `encode` function with `LZ4FrameEncodeOptions` and the `decode` function with `LZ4FrameCodec`, verifying that the original and decompressed data match. ```Julia julia> using ChunkCodecLibLz4 julia> data = [0x00, 0x01, 0x02, 0x03]; julia> compressed_data = encode(LZ4FrameEncodeOptions(;compressionLevel=3), data); julia> decompressed_data = decode(LZ4FrameCodec(), compressed_data; max_size=length(data), size_hint=length(data)); julia> data == decompressed_data true ``` -------------------------------- ### Compressing and Decompressing Data with BZ2Codec in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/LibBzip2/README.md This example demonstrates the basic usage of ChunkCodecLibBzip2 to compress and decompress a byte array. It uses `BZ2EncodeOptions` for encoding and `BZ2Codec` for decoding, verifying that the original and decompressed data are identical. The `max_size` and `size_hint` parameters are provided for decoding. ```Julia julia> using ChunkCodecLibBzip2 julia> data = [0x00, 0x01, 0x02, 0x03]; julia> compressed_data = encode(BZ2EncodeOptions(), data); julia> decompressed_data = decode(BZ2Codec(), compressed_data; max_size=length(data), size_hint=length(data)); julia> data == decompressed_data true ``` -------------------------------- ### Limiting Decoded Output Size with max_size in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This example demonstrates using the `max_size` keyword argument with `decode` to prevent 'zip bomb' attacks. If the decoded output size exceeds the specified limit, a `DecodedSizeError` is thrown. ```julia-repl decode(GzipCodec(), gzipped_data; max_size=100); ERROR: DecodedSizeError: decoded size is greater than max size: 100 ``` -------------------------------- ### Compressing and Decompressing Data with GzipCodec in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/LibZlib/README.md This example demonstrates how to use `GzipCodec` from `ChunkCodecLibZlib` to compress and decompress a byte array in Julia. It shows the `encode` function with `GzipEncodeOptions` and the `decode` function with `GzipCodec`, verifying that the original and decompressed data are identical. The `max_size` and `size_hint` parameters are used for decoding. ```Julia julia> using ChunkCodecLibZlib julia> data = [0x00, 0x01, 0x02, 0x03]; julia> compressed_data = encode(GzipEncodeOptions(;level=6), data); julia> decompressed_data = decode(GzipCodec(), compressed_data; max_size=length(data), size_hint=length(data)); julia> data == decompressed_data true ``` -------------------------------- ### Compressing and Decompressing Data with BloscCodec in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/LibBlosc/README.md This example demonstrates how to use `ChunkCodecLibBlosc` to compress and decompress a byte array. It shows the basic workflow of encoding data using `BloscEncodeOptions` and then decoding it back with `BloscCodec`, verifying that the original and decompressed data are identical. The `max_size` and `size_hint` parameters are crucial for successful decompression. ```julia-repl julia> using ChunkCodecLibBlosc julia> data = [0x00, 0x01, 0x02, 0x03]; julia> compressed_data = encode(BloscEncodeOptions(), data); julia> decompressed_data = decode(BloscCodec(), compressed_data; max_size=length(data), size_hint=length(data)); julia> data == decompressed_data true ``` -------------------------------- ### Handling Encode Function Errors in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This example demonstrates common `ArgumentError` exceptions that occur when the `encode` function receives invalid input, such as non-contiguous memory or data sizes outside the allowed range, emphasizing the importance of input validation. ```julia-repl encode(ChunkCodecLibBlosc.BloscEncodeOptions(), @view(zeros(UInt8, 8)[1:2:end])) ERROR: ArgumentError: vector is not contiguous in memory encode(ChunkCodecLibBlosc.BloscEncodeOptions(), zeros(UInt8, Int64(2)^32)) ERROR: ArgumentError: src_size ∈ 0:1:1073741824 must hold. Got src_size => 4294967296 encode(ChunkCodecLibBlosc.BloscEncodeOptions(;typesize=3), zeros(UInt8,7)) ERROR: ArgumentError: src_size ∈ 0:3:1073741823 must hold. Got src_size => 7 ``` -------------------------------- ### Compressing and Decompressing Data with Zstd in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/LibZstd/README.md This example demonstrates how to compress and decompress a byte array using `ZstdEncodeOptions` and `ZstdCodec` from `ChunkCodecLibZstd`. It shows the full round-trip process, including setting a compression level and verifying data integrity after decompression. The `max_size` and `size_hint` parameters are used during decoding to pre-allocate memory and improve performance. ```julia-repl julia> using ChunkCodecLibZstd julia> data = [0x00, 0x01, 0x02, 0x03]; julia> compressed_data = encode(ZstdEncodeOptions(;compressionLevel=3), data); julia> decompressed_data = decode(ZstdCodec(), compressed_data; max_size=length(data), size_hint=length(data)); julia> data == decompressed_data true ``` -------------------------------- ### Instantiating Workspace Dependencies (Julia) Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/CONTRIBUTING.md This command instantiates the Julia project workspace, ensuring all package dependencies defined in the root `Project.toml` are updated and available for development. It's a prerequisite for running any tests or development tasks within the monorepo. ```sh julia --project=. -e 'import Pkg; Pkg.update()' ``` -------------------------------- ### Creating Sample Data for Encoding in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This snippet demonstrates how to initialize a `UInt8` array of a specified size, which serves as the input data for subsequent encoding operations. ```julia-repl data = zeros(UInt8, 1000); ``` -------------------------------- ### Instantiating Compatibility Test Workspace (Julia) Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/CONTRIBUTING.md Updates the package dependencies specifically for the main `test` directory, which contains more complex and fragile compatibility tests. This step is necessary before running any of the compatibility test scripts. ```sh julia --project=test -e 'import Pkg; Pkg.update()' ``` -------------------------------- ### Running ImageCodecs Compatibility Tests (Julia) Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/CONTRIBUTING.md Executes the compatibility tests for `imagecodecs` from the main `test` directory. These tests verify interoperability with external libraries and formats, often having complex dependencies. ```sh julia --project=test test/imagecodecs-compat.jl ``` -------------------------------- ### Running Basic Package Tests (Julia) Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/CONTRIBUTING.md Executes the standard test suite for a specific package, `LibZlib`, by setting the project environment to its test directory and running the `runtests.jl` script. This is used to verify the core functionality of an individual package. ```sh julia --project=LibZlib/test LibZlib/test/runtests.jl ``` -------------------------------- ### Generating New Julia Package (Julia REPL) Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/CONTRIBUTING.md Generates a new Julia package named `LibFoo` within a new subdirectory, using the `Pkg.generate` function. This is the initial step for creating a new `ChunkCodec` package, especially when wrapping a C library (indicated by the `Lib` prefix). ```julia-repl using Pkg; Pkg.generate("LibFoo") ``` -------------------------------- ### Loading ChunkCodecLibZlib Package in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This code loads the `ChunkCodecLibZlib` package, which is necessary to access Gzip encoding and decoding functionalities within the ChunkCodecs framework. ```julia-repl using ChunkCodecLibZlib ``` -------------------------------- ### Compressing and Decompressing Data with SnappyCodec in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/LibSnappy/README.md This snippet demonstrates how to use ChunkCodecLibSnappy to compress and decompress a byte array. It utilizes `SnappyEncodeOptions` for compression and `SnappyCodec` for decompression, requiring `max_size` and `size_hint` parameters for the decode operation to ensure correct buffer allocation. ```Julia julia> using ChunkCodecLibSnappy julia> data = [0x00, 0x01, 0x02, 0x03]; julia> compressed_data = encode(SnappyEncodeOptions(), data); julia> decompressed_data = decode(SnappyCodec(), compressed_data; max_size=length(data), size_hint=length(data)); julia> data == decompressed_data true ``` -------------------------------- ### Encoding Data with GzipEncodeOptions in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This code illustrates the final step of the encoding process, where the `encode` function is called with the configured `EncodeOptions` and the input data to produce the compressed output. ```julia-repl gzipped_data = encode(e, data) 29-element Vector{UInt8}: ``` -------------------------------- ### Optimizing Decoding Performance with size_hint in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This snippet compares the performance of decoding with and without a `size_hint`. Providing an accurate `size_hint` significantly improves decoding speed and reduces memory allocations by pre-allocating the output buffer. ```julia-repl @time decode(GzipCodec(), gzipped_data; size_hint=1000, max_size=1000); 0.000016 seconds (4 allocations: 8.227 KiB) @time decode(GzipCodec(), gzipped_data; max_size=1000); 0.000018 seconds (9 allocations: 42.195 KiB) ``` -------------------------------- ### Accessing Codec Metadata from EncodeOptions in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This snippet demonstrates how to extract the `Codec` object from the `EncodeOptions` instance. The `Codec` object contains essential metadata required for successful decoding, and its type is verified. ```julia-repl gz_codec = e.codec GzipCodec() gz_codec isa ChunkCodecCore.Codec true ``` -------------------------------- ### Running Large Memory Package Tests (Julia) Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/CONTRIBUTING.md Runs memory-intensive tests for the `LibZlib` package, allocating a larger heap size (15GB) to accommodate demanding test cases. This command is suitable for machines with ample RAM (more than 24 GB) to ensure stability under high memory usage. ```sh julia --project=LibZlib/test --heap-size-hint=15G LibZlib/test/big-mem-tests.jl ``` -------------------------------- ### Handling Decoding Errors for Invalid Data in Julia Source: https://github.com/juliaio/chunkcodecs.jl/blob/main/README.md This snippet illustrates how the `decode` function throws a `LibzDecodingError` when it encounters corrupted or invalid gzipped data, and confirms that this specific error type is a subtype of `ChunkCodecCore.DecodingError`. ```julia-repl bad_gzipped_data = copy(gzipped_data); bad_gzipped_data[end-4] ⊻= 0x01; decode(GzipCodec(), bad_gzipped_data); ERROR: LibzDecodingError: incorrect data check LibzDecodingError <: ChunkCodecCore.DecodingError true ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.