LZ4 Java

https://github.com/yawkat/lz4-java
Admin
LZ4 Java provides fast and high-compression LZ4 compression and decompression algorithms for Java,...
Tokens:3,918
Snippets:32
Trust Score:9.7
License:Apache-2.0
Update:2 months ago
Show doc for...
Tokens:
Raw
### Compress and Decompress Data with LZ4

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/lz4/package.html

Demonstrates the standard workflow for compressing data and decompressing it using both fast and safe decompressor variants.

```java
    LZ4Factory factory = LZ4Factory.fastestInstance();

    byte[] data = "12345345234572".getBytes("UTF-8");
    final int decompressedLength = data.length;

    // compress data
    LZ4Compressor compressor = factory.fastCompressor();
    int maxCompressedLength = compressor.maxCompressedLength(decompressedLength);
    byte[] compressed = new byte[maxCompressedLength];
    int compressedLength = compressor.compress(data, 0, decompressedLength, compressed, 0, maxCompressedLength);

    // decompress data
    // - method 1: when the decompressed length is known
    LZ4FastDecompressor decompressor = factory.fastDecompressor();
    byte[] restored = new byte[decompressedLength];
    int compressedLength2 = decompressor.decompress(compressed, 0, restored, 0, decompressedLength);
    // compressedLength == compressedLength2

    // - method 2: when the compressed length is known (a little slower)
    // the destination buffer needs to be over-sized
    LZ4SafeDecompressor decompressor2 = factory.safeDecompressor();
    int decompressedLength2 = decompressor2.decompress(compressed, 0, compressedLength, restored, 0);
    // decompressedLength == decompressedLength2
```

--------------------------------

### LZ4Factory

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/lz4/package.html

The entry point for the LZ4 API. Use this class to obtain instances of compressors and decompressors.

```APIDOC
## LZ4Factory

### Description
Provides access to LZ4Compressor, LZ4FastDecompressor, and LZ4SafeDecompressor instances. Use `fastestInstance()` to get the most efficient implementation for the current environment.

### Usage
```java
LZ4Factory factory = LZ4Factory.fastestInstance();
LZ4Compressor compressor = factory.fastCompressor();
LZ4FastDecompressor fastDecompressor = factory.fastDecompressor();
LZ4SafeDecompressor safeDecompressor = factory.safeDecompressor();
```
```

--------------------------------

### Calculate Streaming Hash with XXHashFactory

Source: https://github.com/yawkat/lz4-java/blob/main/README.md

Demonstrates how to use XXHashFactory to compute a 32-bit hash for a stream of data using a specified seed.

```java
XXHashFactory factory = XXHashFactory.fastestInstance();

byte[] data = "12345345234572".getBytes("UTF-8");
ByteArrayInputStream in = new ByteArrayInputStream(data);

int seed = 0x9747b28c; // used to initialize the hash value, use whatever
                       // value you want, but always the same
StreamingXXHash32 hash32 = factory.newStreamingHash32(seed);
byte[] buf = new byte[8]; // for real-world usage, use a larger buffer, like 8192 bytes
for (;;) {
  int read = in.read(buf);
  if (read == -1) {
    break;
  }
  hash32.update(buf, 0, read);
}
int hash = hash32.getValue();
```

--------------------------------

### Use LZ4 Frame Streams

Source: https://github.com/yawkat/lz4-java/blob/main/README.md

Utilize LZ4FrameOutputStream and LZ4FrameInputStream for interoperable file-based compression.

```java
byte[] data = "12345345234572".getBytes("UTF-8");
final int decompressedLength = data.length;

LZ4FrameOutputStream outStream = new LZ4FrameOutputStream(new FileOutputStream(new File("test.lz4")));
outStream.write(data);
outStream.close();

byte[] restored = new byte[decompressedLength];
LZ4FrameInputStream inStream = new LZ4FrameInputStream(new FileInputStream(new File("test.lz4")));
inStream.read(restored);
inStream.close();
```

--------------------------------

### Add Maven Dependency

Source: https://github.com/yawkat/lz4-java/blob/main/README.md

Include this dependency in your project's pom.xml to use the library.

```xml
<dependency>
    <groupId>at.yawk.lz4</groupId>
    <artifactId>lz4-java</artifactId>
</dependency>
```

--------------------------------

### LZ4Factory Class Modifications

Source: https://github.com/yawkat/lz4-java/wiki/1.8.1-diffoscope-output

Shows the addition of insecure instance methods and the updated constructor logic for LZ4Factory.

```diff
@@ -8,53 +8,72 @@
 import java.lang.reflect.Field;
 import net.jpountz.util.Native;
 import net.jpountz.util.Utils;
 
 public final class LZ4Factory
 {
     private static LZ4Factory NATIVE_INSTANCE;
+    private static LZ4Factory NATIVE_INSECURE_INSTANCE;
     private static LZ4Factory JAVA_UNSAFE_INSTANCE;
+    private static LZ4Factory JAVA_UNSAFE_INSECURE_INSTANCE;
     private static LZ4Factory JAVA_SAFE_INSTANCE;
     private final String impl;
     private final LZ4Compressor fastCompressor;
     private final LZ4Compressor highCompressor;
     private final LZ4FastDecompressor fastDecompressor;
     private final LZ4SafeDecompressor safeDecompressor;
     private final LZ4Compressor[] highCompressors;
     
-    private static LZ4Factory instance(final String impl) {
+    private static LZ4Factory instance(final String impl, final boolean insecureFastDecompressor) {
         try {
-            return new LZ4Factory(impl);
+            return new LZ4Factory(impl, insecureFastDecompressor);
         }
         catch (final Exception e) {
             throw new AssertionError((Object)e);
         }
     }
     
     public static synchronized LZ4Factory nativeInstance() {
         if (LZ4Factory.NATIVE_INSTANCE == null) {
-            LZ4Factory.NATIVE_INSTANCE = instance("JNI");
+            LZ4Factory.NATIVE_INSTANCE = instance("JNI", false);
         }
         return LZ4Factory.NATIVE_INSTANCE;
     }
     
+    @Deprecated
+    public static synchronized LZ4Factory nativeInsecureInstance() {
+        if (LZ4Factory.NATIVE_INSECURE_INSTANCE == null) {
+            LZ4Factory.NATIVE_INSECURE_INSTANCE = instance("JNI", true);
+        }
+        return LZ4Factory.NATIVE_INSECURE_INSTANCE;
+    }
+    
     public static synchronized LZ4Factory safeInstance() {
         if (LZ4Factory.JAVA_SAFE_INSTANCE == null) {
-            LZ4Factory.JAVA_SAFE_INSTANCE = instance("JavaSafe");
+            LZ4Factory.JAVA_SAFE_INSTANCE = instance("JavaSafe", false);
         }
         return LZ4Factory.JAVA_SAFE_INSTANCE;
     }
     
+    @Deprecated
     public static synchronized LZ4Factory unsafeInstance() {
         if (LZ4Factory.JAVA_UNSAFE_INSTANCE == null) {
-            LZ4Factory.JAVA_UNSAFE_INSTANCE = instance("JavaUnsafe");
+            LZ4Factory.JAVA_UNSAFE_INSTANCE = safeInstance();
         }
         return LZ4Factory.JAVA_UNSAFE_INSTANCE;
     }
     
+    @Deprecated
+    public static synchronized LZ4Factory unsafeInsecureInstance() {
+        if (LZ4Factory.JAVA_UNSAFE_INSECURE_INSTANCE == null) {
+            LZ4Factory.JAVA_UNSAFE_INSECURE_INSTANCE = instance("JavaUnsafe", true);
+        }
+        return LZ4Factory.JAVA_UNSAFE_INSECURE_INSTANCE;
+    }
+    
     public static LZ4Factory fastestJavaInstance() {
         if (Utils.isUnalignedAccessAllowed()) {
             try {
                 return unsafeInstance();
             }
             catch (final Throwable t) {
                 return safeInstance();
@@ -82,20 +101,25 @@
         ClassLoader loader = LZ4Factory.class.getClassLoader();
         loader = ((loader == null) ? ClassLoader.getSystemClassLoader() : loader);
         final Class<?> c = loader.loadClass(cls);
         final Field f = c.getField("INSTANCE");
         return (T)f.get(null);
     }
     
-    private LZ4Factory(final String impl) throws ClassNotFoundException, NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException, NoSuchMethodException, InstantiationException, InvocationTargetException {
+    private LZ4Factory(final String impl, final boolean insecureFastDecompressor) throws ClassNotFoundException, NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException, NoSuchMethodException, InstantiationException, InvocationTargetException {
         this.highCompressors = new LZ4Compressor[18];
         this.impl = impl;
         this.fastCompressor = classInstance("net.jpountz.lz4.LZ4" + impl + "Compressor");
         this.highCompressor = classInstance("net.jpountz.lz4.LZ4HC" + impl + "Compressor");
-        this.fastDecompressor = classInstance("net.jpountz.lz4.LZ4" + impl + "FastDecompressor");
+        if (insecureFastDecompressor) {
+            this.fastDecompressor = classInstance("net.jpountz.lz4.LZ4" + impl + "FastDecompressor");
+        }
+        else {
+            this.fastDecompressor = LZ4JavaSafeFastDecompressor.INSTANCE;
+        }
         this.safeDecompressor = classInstance("net.jpountz.lz4.LZ4" + impl + "SafeDecompressor");
         final Constructor<? extends LZ4Compressor> highConstructor = this.highCompressor.getClass().getDeclaredConstructor(Integer.TYPE);
         this.highCompressors[9] = this.highCompressor;
         for (int level = 1; level <= 17; ++level) {
             if (level != 9) {
                 this.highCompressors[level] = (LZ4Compressor)highConstructor.newInstance(Integer.valueOf(level));
             }
```

--------------------------------

### Perform block hashing with xxHash

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/xxhash/package.html

Use this approach for hashing data already loaded into memory.

```java
    XXHashFactory factory = XXHashFactory.fastestInstance();

    byte[] data = "12345345234572".getBytes("UTF-8");

    XXHash32 hash32 = factory.hash32();
    int seed = 0x9747b28c; // used to initialize the hash value, use whatever
                           // value you want, but always the same
    int hash = hash32.hash(data, 0, data.length, seed);
```

--------------------------------

### Perform streaming hashing with xxHash

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/xxhash/package.html

Use this approach for hashing data streams to avoid loading the entire stream into memory.

```java
    XXHashFactory factory = XXHashFactory.fastestInstance();

    byte[] data = "12345345234572".getBytes("UTF-8");
    ByteArrayInputStream in = new ByteArrayInputStream(data);

    int seed = 0x9747b28c; // used to initialize the hash value, use whatever
                           // value you want, but always the same
    StreamingXXHash32 hash32 = factory.newStreamingHash32(seed);
    byte[] buf = new byte[8]; // for real-world usage, use a larger buffer, like 8192 bytes
    for (;;) {
      int read = in.read(buf);
      if (read == -1) {
        break;
      }
      hash32.update(buf, 0, read);
    }
    int hash = hash32.getValue();
```

--------------------------------

### LZ4Compressor

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/lz4/package.html

Interface for compressing data using the LZ4 algorithm.

```APIDOC
## LZ4Compressor

### Description
Handles the compression of byte arrays. Users must calculate the maximum possible compressed length before performing the compression.

### Methods
- `maxCompressedLength(int length)`: Returns the maximum size of the compressed data.
- `compress(byte[] src, int srcOff, int srcLen, byte[] dest, int destOff, int maxDestLen)`: Compresses the source array into the destination array.
```

--------------------------------

### StreamingXXHash32

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/xxhash/package.html

Interface for streaming hashing, allowing data to be processed in chunks.

```APIDOC
## StreamingXXHash32

### Description
Provides methods for incremental hashing of data streams.

### Methods
- **update(byte[] buf, int off, int len)**: Updates the hash with a chunk of data.
- **getValue()**: Returns the final computed hash value.
```

--------------------------------

### XXHash32

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/xxhash/package.html

Interface for block-based hashing.

```APIDOC
## XXHash32.hash(byte[] data, int off, int len, int seed)

### Description
Computes the 32-bit hash of a byte array.

### Parameters
- **data** (byte[]) - The input data to hash.
- **off** (int) - The offset in the array.
- **len** (int) - The length of the data to hash.
- **seed** (int) - The seed value for initialization.
```

--------------------------------

### XXHashFactory

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/xxhash/package.html

The factory class used to obtain instances of hashing interfaces.

```APIDOC
## XXHashFactory

### Description
Factory class to retrieve instances of xxHash hashing implementations.

### Usage
```java
XXHashFactory factory = XXHashFactory.fastestInstance();
```
```

--------------------------------

### LZ4Decompressors

Source: https://github.com/yawkat/lz4-java/blob/main/src/java/net/jpountz/lz4/package.html

Interfaces for decompressing LZ4-compressed data.

```APIDOC
## LZ4FastDecompressor and LZ4SafeDecompressor

### Description
Provides methods to restore compressed data. Use `LZ4FastDecompressor` when the original length is known, or `LZ4SafeDecompressor` when only the compressed length is known.

### Methods
- `LZ4FastDecompressor.decompress(byte[] src, int srcOff, byte[] dest, int destOff, int destLen)`
- `LZ4SafeDecompressor.decompress(byte[] src, int srcOff, int srcLen, byte[] dest, int destOff)`
```

--------------------------------

### Resolve Gradle Capability Conflict

Source: https://github.com/yawkat/lz4-java/wiki/Gradle-and-org.lz4:lz4‐java:1.8.1

Use these snippets in your build configuration to force Gradle to select the at.yawk.lz4 version when a conflict occurs with the old org.lz4 group ID.

```kotlin
configurations.configureEach {
    resolutionStrategy {
        capabilitiesResolution {
            withCapability("org.lz4:lz4-java") {
                select(candidates.first { (it.id as ModuleComponentIdentifier).group == "at.yawk.lz4" })
            }
        }
    }
}
```

```groovy
configurations.configureEach {
    resolutionStrategy {
        capabilitiesResolution {
            withCapability("org.lz4:lz4-java") {
                select(candidates.find { ((ModuleComponentIdentifier) it.id).group == "at.yawk.lz4" })
            }
        }
    }
}
```
Tokens:
Raw