### Install (JavaScript / TypeScript) Source: https://github.com/milesgranger/cramjam/blob/master/README.md Install cramjam using npm. ```commandline npm install cramjam ``` -------------------------------- ### Install (Python) Source: https://github.com/milesgranger/cramjam/blob/master/README.md Install cramjam using pip. ```commandline pip install --upgrade cramjam # Requires no Python or system dependencies! ``` -------------------------------- ### Install Benchmark Requirements Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Installs all Python libraries required to run the benchmarks. ```bash pip install -r benchmark-requirements.txt ``` -------------------------------- ### Install cramjam-cli Source: https://github.com/milesgranger/cramjam/blob/master/docs/index.md Install the cramjam-cli for command-line usage. ```bash pip install cramjam-cli ``` -------------------------------- ### Install cramjam for Python Source: https://github.com/milesgranger/cramjam/blob/master/docs/index.md Install the cramjam library for Python using pip or conda. ```bash pip install cramjam ``` ```bash conda install -c conda-forge cramjam ``` -------------------------------- ### TypeScript Example Source: https://github.com/milesgranger/cramjam/blob/master/README.md A basic example of using the cramjam-js library in TypeScript for Brotli compression and decompression. ```typescript import {Compress, Decompress} from 'cramjam'; const decoder = new TextDecoder(); const encoder = new TextEncoder(); const str = 'hello, world'; const encoded = encoder.encode(str); const compressed = Compress.brotli(encoded); const decompressed = Decompress.brotli(compressed); const decoded = decoder.decode(decompressed); ``` -------------------------------- ### Brotli Compression and Decompression Source: https://github.com/milesgranger/cramjam/blob/master/cramjam-js/README.md Example of using Brotli compression and decompression with cramjam-js. ```typescript import {Compress, Decompress} from 'cramjam'; const decoder = new TextDecoder(); const encoder = new TextEncoder(); const str = 'hello, world'; const encoded = encoder.encode(str); const compressed = Compress.brotli(encoded); const decompressed = Decompress.brotli(compressed); const decoded = decoder.decode(decompressed); ``` -------------------------------- ### Basic Compression and Decompression Source: https://github.com/milesgranger/cramjam/blob/master/README.md Demonstrates the basic usage of compress and decompress functions for the Snappy algorithm, including handling different input types and converting the output buffer to bytes or a NumPy array. ```python >>> import cramjam >>> import numpy as np >>> compressed = cramjam.snappy.compress(b"bytes here") >>> decompressed = cramjam.snappy.decompress(compressed) >>> decompressed cramjam.Buffer(len=10) # an object which implements the buffer protocol >>> bytes(decompressed) b"bytes here" >>> np.frombuffer(decompressed, dtype=np.uint8) array([ 98, 121, 116, 101, 115, 32, 104, 101, 114, 101], dtype=uint8) ``` -------------------------------- ### Run Brotli Benchmark Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to execute the Brotli benchmark test. ```bash make bench-brotli ``` -------------------------------- ### Benchmark Gzip Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the gzip benchmark. ```bash make bench-gzip ``` -------------------------------- ### Compress and Decompress Into Source: https://github.com/milesgranger/cramjam/blob/master/README.md Illustrates the use of `compress_into` and `decompress_into` for writing compressed data directly into a buffer and decompressing data into a pre-allocated output. ```python >>> import numpy as np >>> from cramjam import snappy, Buffer >>> >>> data = np.frombuffer(b'some bytes here', dtype=np.uint8) >>> data array([115, 111, 109, 101, 32, 98, 121, 116, 101, 115, 32, 104, 101, 114, 101], dtype=uint8) >>> >>> compressed = Buffer() >>> snappy.compress_into(data, compressed) 33 # 33 bytes written to compressed buffer >>> >>> compressed.tell() # Where is the buffer position? 33 # goodie! >>> >>> compressed.seek(0) # Go back to the start of the buffer so we can prepare to decompress >>> decompressed = b'0' * len(data) # let's write to `bytes` as output >>> decompressed b'000000000000000' >>> >>> snappy.decompress_into(compressed, decompressed) 15 # 15 bytes written to decompressed >>> decompressed b'some bytes here' ``` -------------------------------- ### Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the snappy raw benchmarks. ```bash make bench-snappy-raw ``` -------------------------------- ### Bzip2 Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the Bzip2 benchmark. ```bash make bench-bzip2 ``` -------------------------------- ### Snappy Framed Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the benchmark for Snappy's framed format. ```bash make bench-snappy-framed ``` -------------------------------- ### Snappy Raw Benchmark Results Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Benchmark results for Snappy raw compression. ```text --------------------------------------------------------------------------------------------------------------- benchmark: 28 tests ---------------------------------------------------------------------------------------------------------------- Name (time in us) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- test_snappy_raw[Mark.Twain-Tom.Sawyer.txt-cramjam] 28.0010 (4.79) 60.4390 (1.73) 31.9080 (4.53) 1.8590 (3.44) 31.7450 (4.87) 2.4550 (16.48) 3749;131 31,340.0822 (0.22) 12606 1 test_snappy_raw[Mark.Twain-Tom.Sawyer.txt-snappy] 26.9300 (4.61) 76.8340 (2.21) 28.9253 (4.11) 0.9898 (1.83) 28.7750 (4.41) 0.3340 (2.24) 1566;2560 34,571.8473 (0.24) 16034 1 ``` -------------------------------- ### Blosc2 Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the Blosc2 benchmark. ```bash make bench-blosc2 ``` -------------------------------- ### Bzip2 Benchmark Results Table Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Detailed benchmark results comparing Cramjam and Bzip2 across various file types and sizes. Includes metrics like Min, Max, Mean, StdDev, Median, IQR, Outliers, OPS, Rounds, and Iterations. ```text ------------------------------------------------------------------------------------------------------- benchmark: 28 tests ------------------------------------------------------------------------------------------------------- Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- test_bzip2[Mark.Twain-Tom.Sawyer.txt-bzip2] 1.2711 (1.00) 1.6461 (1.0) 1.3491 (1.01) 0.0385 (1.0) 1.3472 (1.02) 0.0512 (1.14) 204;5 741.2337 (0.99) 643 1 test_bzip2[Mark.Twain-Tom.Sawyer.txt-cramjam] 1.2670 (1.0) 1.8671 (1.13) 1.3389 (1.0) 0.0483 (1.26) 1.3258 (1.0) 0.0448 (1.0) 73;34 746.8702 (1.0) 630 1 test_bzip2[alice29.txt-bzip2] 10.5259 (8.31) 11.9644 (7.27) 10.9831 (8.20) 0.2470 (6.42) 10.9820 (8.28) 0.2934 (6.55) 19;3 91.0490 (0.12) 90 1 test_bzip2[alice29.txt-cramjam] 10.0803 (7.96) 11.6217 (7.06) 10.6278 (7.94) 0.3660 (9.51) 10.5032 (7.92) 0.5597 (12.49) 31;0 94.0926 (0.13) 94 1 test_bzip2[asyoulik.txt-bzip2] 8.5538 (6.75) 10.2334 (6.22) 9.2801 (6.93) 0.4153 (10.79) 9.1760 (6.92) 0.6618 (14.76) 30;0 107.7571 (0.14) 106 1 test_bzip2[asyoulik.txt-cramjam] 8.3083 (6.56) 9.0135 (5.48) 8.5585 (6.39) 0.1398 (3.63) 8.5525 (6.45) 0.1770 (3.95) 39;2 116.8429 (0.16) 116 1 test_bzip2[fifty-four-mb-random-bzip2] 5,798.4294 (>1000.0) 6,637.8179 (>1000.0) 6,010.0714 (>1000.0) 356.8759 (>1000.0) 5,825.5740 (>1000.0) 314.4360 (>1000.0) 1;1 0.1664 (0.00) 5 1 test_bzip2[fifty-four-mb-random-cramjam] 5,373.8872 (>1000.0) 5,395.6856 (>1000.0) 5,381.7519 (>1000.0) 9.1684 (238.21) 5,379.5500 (>1000.0) 14.1904 (316.55) 1;0 0.1858 (0.00) 5 1 test_bzip2[fifty-four-mb-repeating-bzip2] 9,565.3209 (>1000.0) 10,487.7696 (>1000.0) 9,907.5835 (>1000.0) 462.9329 (>1000.0) 9,575.6350 (>1000.0) 800.6694 (>1000.0) 1;0 0.1009 (0.00) 5 1 test_bzip2[fifty-four-mb-repeating-cramjam] 8,290.9926 (>1000.0) 8,564.3140 (>1000.0) 8,405.8197 (>1000.0) 100.4702 (>1000.0) 8,379.8931 (>1000.0) 105.7075 (>1000.0) 2;0 0.1190 (0.00) 5 1 test_bzip2[fireworks.jpeg-bzip2] 11.4166 (9.01) 13.8160 (8.39) 11.9222 (8.90) 0.4338 (11.27) 11.8242 (8.92) 0.3818 (8.52) 10;5 83.8773 (0.11) 85 1 test_bzip2[fireworks.jpeg-cramjam] 10.5497 (8.33) 12.2991 (7.47) 11.0318 (8.24) 0.2900 (7.53) 10.9575 (8.26) 0.2596 (5.79) 16;5 90.6472 (0.12) 92 1 test_bzip2[geo.protodata-bzip2] 8.0638 (6.36) 9.1159 (5.54) 8.2840 (6.19) 0.1215 (3.16) 8.2872 (6.25) 0.1452 (3.24) 16;2 127.4257 (0.17) 127 1 test_bzip2[geo.protodata-cramjam] 7.5771 (5.98) 8.8989 (5.41) 7.8477 (5.86) 0.1479 (3.84) 7.8153 (5.89) 0.1452 (3.24) 16;2 127.4257 (0.17) 127 1 test_bzip2[html-bzip2] 6.4125 (5.06) 7.2505 (4.40) 6.6252 (4.95) 0.1029 (2.67) 6.6151 (4.99) 0.0885 (1.97) 22;7 150.9397 (0.20) 150 1 test_bzip2[html-cramjam] 6.0190 (4.75) 6.6557 (4.04) 6.1884 (4.62) 0.0864 (2.25) 6.1636 (4.65) 0.1016 (2.27) 35;4 161.5915 (0.22) 158 1 test_bzip2[html_x_4-bzip2] 31.2213 (24.64) 33.4331 (20.31) 31.5566 (23.57) 0.4104 (10.66) 31.4843 (23.75) 0.2987 (6.66) 1;1 31.6891 (0.04) 30 1 test_bzip2[html_x_4-cramjam] 29.1961 (23.04) 31.7037 (19.26) 29.6869 (22.17) 0.4969 (12.91) 29.5098 (22.26) 0.4508 (10.06) 5;1 33.6849 (0.05) 32 1 ``` -------------------------------- ### Add libcramjam with C API for Rust Source: https://github.com/milesgranger/cramjam/blob/master/docs/index.md Add the libcramjam library with C API support for Rust projects. ```bash cargo add libcramjam --features capi ``` -------------------------------- ### LZMA Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the LZMA benchmark tests. ```bash make bench-lzma ``` -------------------------------- ### LZ4 Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the LZ4 benchmarks. ```bash make bench-lz4 ``` -------------------------------- ### Add libcramjam for Rust Source: https://github.com/milesgranger/cramjam/blob/master/docs/index.md Add the libcramjam library for Rust projects. ```bash cargo add libcramjam ``` -------------------------------- ### Snappy Benchmark Results Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Benchmark results comparing Cramjam with Snappy compression for various file types and sizes. ```text test_snappy_raw[paper-100k.pdf-cramjam] 12.2860 (2.10) 44.2950 (1.27) 13.1896 (1.87) 0.8487 (1.57) 13.0200 (2.00) 0.1490 (1.0) 1002;3367 75,817.5069 (0.53) 11359 1 test_snappy_raw[paper-100k.pdf-snappy] 10.2890 (1.76) 35.7210 (1.03) 11.0991 (1.58) 0.5402 (1.0) 11.0640 (1.70) 0.3180 (2.13) 1434;1328 90,097.6017 (0.63) 27306 1 test_snappy_raw[plrabn12.txt-cramjam] 1,512.8650 (258.88) 1,881.1250 (54.00) 1,609.2872 (228.41) 48.7215 (90.19) 1,591.1230 (243.96) 37.5485 (251.98) 66;46 621.3931 (0.00) 525 1 test_snappy_raw[plrabn12.txt-snappy] 1,716.5850 (293.74) 2,070.5130 (59.43) 1,804.1379 (256.07) 68.1815 (126.21) 1,799.4450 (275.90) 57.6188 (386.67) 96;34 554.2814 (0.00) 499 1 test_snappy_raw[urls.10K-cramjam] 1,357.1110 (232.22) 2,421.0440 (69.49) 1,634.3708 (231.97) 283.8191 (525.37) 1,483.5370 (227.47) 277.4218 (>1000.0) 120;68 611.8563 (0.00) 543 1 test_snappy_raw[urls.10K-snappy] 1,333.5560 (228.19) 2,160.5230 (62.02) 1,579.1927 (224.14) 226.8612 (419.94) 1,461.6870 (224.12) 342.2740 (>1000.0) 138;0 633.2350 (0.00) 569 1 ``` -------------------------------- ### LZ4 Block Format Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the LZ4 block format benchmark. ```bash make bench-lz4-block ``` -------------------------------- ### Snappy Benchmark Results Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Benchmark results comparing cramjam and Snappy for various file types and compression levels. ```text #### Snappy test_snappy_framed[html-snappy] 419.1740 (12.13) 518.9120 (8.10) 440.4698 (11.21) 19.0325 (11.97) 432.7900 (11.10) 23.5960 (62.41) 425;63 2,270.3031 (0.09) 1874 1 test_snappy_framed[html_x_4-cramjam] 498.1220 (14.42) 626.0840 (9.78) 538.0052 (13.69) 17.7006 (11.13) 533.8565 (13.70) 22.1410 (58.56) 294;39 1,858.7179 (0.07) 1406 1 test_snappy_framed[html_x_4-snappy] 1,774.0160 (51.35) 2,062.4400 (32.21) 1,799.3159 (45.77) 30.9255 (19.45) 1,790.9070 (45.95) 20.1712 (53.35) 37;36 555.7668 (0.02) 463 1 test_snappy_framed[kppkn.gtb-cramjam] 363.9480 (10.54) 445.5770 (6.96) 386.9350 (9.84) 12.8919 (8.11) 383.9880 (9.85) 13.1481 (34.78) 581;84 2,584.4136 (0.10) 1986 1 test_snappy_framed[kppkn.gtb-snappy] 948.8640 (27.47) 1,818.3100 (28.40) 991.6895 (25.23) 69.9524 (43.99) 964.2670 (24.74) 58.9637 (155.96) 98;38 1,008.3801 (0.04) 851 1 test_snappy_framed[lcet10.txt-cramjam] 1,211.4980 (35.07) 1,411.0670 (22.04) 1,266.7555 (32.23) 32.8859 (20.68) 1,269.7810 (32.58) 43.5995 (115.32) 201;22 789.4183 (0.03) 712 1 test_snappy_framed[lcet10.txt-snappy] 2,746.5900 (79.51) 3,359.2020 (52.46) 2,815.2247 (71.62) 72.5785 (45.64) 2,801.3160 (71.87) 55.8608 (147.76) 16;9 355.2114 (0.01) 323 1 test_snappy_framed[paper-100k.pdf-cramjam] 40.7860 (1.18) 71.1230 (1.11) 42.7810 (1.09) 2.1702 (1.36) 41.8275 (1.07) 1.8580 (4.91) 2532;810 23,374.8378 (0.92) 13638 1 test_snappy_framed[paper-100k.pdf-snappy] 355.6370 (10.29) 417.0000 (6.51) 363.4989 (9.25) 11.6659 (7.34) 357.4060 (9.17) 11.1997 (29.62) 358;220 2,751.0397 (0.11) 2321 1 test_snappy_framed[plrabn12.txt-cramjam] 1,619.8560 (46.89) 2,237.4490 (34.94) 1,774.7065 (45.15) 73.1317 (45.99) 1,758.2970 (45.11) 91.5220 (242.08) 145;11 563.4735 (0.02) 521 1 test_snappy_framed[plrabn12.txt-snappy] 3,366.8110 (97.46) 4,179.6740 (65.27) 3,670.3920 (93.37) 221.7947 (139.48) 3,587.1925 (92.03) 398.7299 (>1000.0) 98;0 272.4505 (0.01) 246 1 test_snappy_framed[urls.10K-cramjam] 1,572.6960 (45.52) 2,352.0380 (36.73) 1,782.1609 (45.34) 168.3364 (105.86) 1,712.2195 (43.93) 103.8710 (274.75) 109;108 561.1166 (0.02) 566 1 test_snappy_framed[urls.10K-snappy] 3,668.1250 (106.18) 4,344.0750 (67.84) 3,754.5744 (95.52) 134.4371 (84.54) 3,698.2595 (94.88) 68.2920 (180.64) 30;33 266.3418 (0.01) 240 1 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ``` -------------------------------- ### ZSTD Benchmark Command Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Command to run the ZSTD benchmark, comparing Cramjam's ZSTD implementation against the standard Zstd library. ```bash make bench-zstd ``` -------------------------------- ### Benchmark Results Source: https://github.com/milesgranger/cramjam/blob/master/benchmarks/README.md Benchmark results comparing Cramjam and Bzip2 compression performance on different datasets. ```text #### Bzip2 test_bzip2[kppkn.gtb-bzip2] 9.9215 (7.83) 11.7609 (7.14) 10.4842 (7.83) 0.3484 (9.05) 10.3495 (7.81) 0.3751 (8.37) 22;6 95.3820 (0.13) 94 1 test_bzip2[kppkn.gtb-cramjam] 9.5839 (7.56) 10.6962 (6.50) 9.9063 (7.40) 0.2034 (5.28) 9.8640 (7.44) 0.2405 (5.36) 18;3 100.9457 (0.14) 101 1 test_bzip2[lcet10.txt-bzip2] 28.7752 (22.71) 31.1098 (18.90) 29.2280 (21.83) 0.4010 (10.42) 29.1479 (21.99) 0.3664 (8.17) 2;1 34.2138 (0.05) 33 1 test_bzip2[lcet10.txt-cramjam] 27.5969 (21.78) 29.8887 (18.16) 28.3710 (21.19) 0.6627 (17.22) 28.1787 (21.25) 0.8508 (18.98) 8;0 35.2472 (0.05) 34 1 test_bzip2[paper-100k.pdf-bzip2] 8.5453 (6.74) 9.3542 (5.68) 8.8012 (6.57) 0.1380 (3.59) 8.7832 (6.62) 0.1122 (2.50) 25;11 113.6214 (0.15) 113 1 test_bzip2[paper-100k.pdf-cramjam] 8.1157 (6.41) 9.4451 (5.74) 8.3298 (6.22) 0.1797 (4.67) 8.2959 (6.26) 0.0947 (2.11) 10;7 120.0510 (0.16) 119 1 test_bzip2[plrabn12.txt-bzip2] 36.0196 (28.43) 38.5789 (23.44) 36.7931 (27.48) 0.7330 (19.05) 36.4790 (27.51) 0.9648 (21.52) 6;0 27.1790 (0.04) 26 1 test_bzip2[plrabn12.txt-cramjam] 34.6344 (27.34) 37.5419 (22.81) 35.3211 (26.38) 0.5382 (13.98) 35.3134 (26.64) 0.4735 (10.56) 4;1 28.3117 (0.04) 27 1 test_bzip2[urls.10K-bzip2] 51.0884 (40.32) 52.2594 (31.75) 51.6389 (38.57) 0.2879 (7.48) 51.6694 (38.97) 0.4628 (10.32) 5;0 19.3652 (0.03) 18 1 test_bzip2[urls.10K-cramjam] 47.3159 (37.35) 48.2918 (29.34) 47.8151 (35.71) 0.2691 (6.99) 47.8155 (36.07) 0.3339 (7.45) 6;0 20.9139 (0.03) 19 1 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ```