### Setup Development Environment Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/AGENTS.md Commands to initialize a virtual environment and install project dependencies. ```bash python -m venv .venv ``` ```bash source .venv/bin/activate ``` ```bash .venv\Scripts\activate ``` ```bash pip install -r requirements.txt ``` ```bash pip install -e . ``` -------------------------------- ### Install from Source Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Clone the repository and install in development mode. ```bash # Clone the repository git clone https://github.com/egorserdyuk/JSONBrotliMinifyer.git cd JSONBrotliMinifyer # Install in development mode pip install -e . ``` -------------------------------- ### Verify installation Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Check if the package is correctly installed by importing it. ```bash python -c "import jsonbrotliminifyer; print('Installation successful!')" ``` -------------------------------- ### Development Installation Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Install the package with additional dependencies required for development and testing. ```bash # Clone the repository git clone https://github.com/egorserdyuk/JSONBrotliMinifyer.git cd JSONBrotliMinifyer # Install with development dependencies pip install -e .[dev] # or pip install -r requirements_dev.txt ``` -------------------------------- ### Error Handling Examples Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Examples of how the tool reports common errors. ```bash # Invalid JSON input echo '{"invalid": json}' | jsonbrotlim compress # Error: Invalid JSON input: Expecting ',' delimiter: line 1 column 17 # Non-existent input file jsonbrotlim compress -i nonexistent.json -o output.br # Error: [Errno 2] No such file or directory: 'nonexistent.json' # Invalid compressed data echo "not compressed data" | jsonbrotlim decompress # Error: Invalid Brotli-compressed data ``` -------------------------------- ### Install and Verify jsonbrotlim Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Install the package via pip and verify the installation by checking the help menu. ```bash pip install jsonbrotliminifyer ``` ```bash jsonbrotlim --help ``` -------------------------------- ### Install dependencies Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Install the package in editable mode and any additional development requirements. ```bash pip install -e . pip install -r requirements.txt # if additional dev dependencies exist ``` -------------------------------- ### Installation Source: https://context7.com/egorserdyuk/jsonbrotliminifyer/llms.txt Install the JSONBrotliMinifyer library using pip. ```APIDOC ## Installation Install the library using pip. ```bash pip install jsonbrotliminifyer ``` ``` -------------------------------- ### Install Xcode Command Line Tools Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Install necessary build tools for macOS. ```bash xcode-select --install ``` -------------------------------- ### Install on macOS Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Use Homebrew to install Brotli before installing the package via pip. ```bash brew install brotli pip install jsonbrotliminifyer ``` -------------------------------- ### Setup Virtual Environment Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Create and activate a virtual environment for isolated package management. ```bash # Create virtual environment python -m venv jsonbrotli_env # Activate (Linux/macOS) source jsonbrotli_env/bin/activate # Activate (Windows) jsonbrotli_env\Scripts\activate # Install pip install jsonbrotliminifyer ``` -------------------------------- ### Verify Installation Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Confirm the package is correctly installed by checking the import and CLI tool availability. ```bash # Check Python import python -c "import jsonbrotliminifyer; print('Installation successful!')" # Check CLI tool jsonbrotlim --help ``` -------------------------------- ### Quality Level Examples Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Demonstrating the impact of the quality parameter on compression. ```bash # Fastest compression (largest output) jsonbrotlim compress -i data.json -o data.br -q 0 # Balanced (recommended for most cases) jsonbrotlim compress -i data.json -o data.br -q 6 # Best compression (slowest, smallest output) jsonbrotlim compress -i data.json -o data.br -q 11 ``` -------------------------------- ### Docker Integration Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Example of using the tool within a Dockerfile. ```dockerfile FROM python:3.9-slim RUN pip install jsonbrotliminifyer # Copy and compress configuration COPY config.json /app/ RUN jsonbrotlim compress -i /app/config.json -o /app/config.br ``` -------------------------------- ### Install jsonbrotliminifyer from source Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Install the library from its source code repository for development or custom builds. ```bash git clone https://github.com/egorserdyuk/JSONBrotliMinifyer.git cd JSONBrotliMinifyer pip install -e . ``` -------------------------------- ### Decompress Command Examples Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Examples of decompressing Brotli files or streams back to JSON. ```bash # Decompress a compressed file jsonbrotlim decompress -i data.json.br -o restored.json # Decompress from stdin cat compressed.br | jsonbrotlim decompress # Decompress from stdin to file jsonbrotlim decompress -i compressed.br -o output.json ``` -------------------------------- ### Install System Dependencies on Ubuntu/Debian Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Install build tools required if the Brotli library fails to compile. ```bash sudo apt-get update sudo apt-get install build-essential python3-dev ``` -------------------------------- ### Install Brotli on Linux Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Install the system-level Brotli package using the distribution's package manager. ```bash # Ubuntu/Debian sudo apt-get install python3-brotli # CentOS/RHEL sudo yum install python3-brotli ``` -------------------------------- ### Compress Command Examples Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Various ways to compress JSON data using files or standard input/output streams. ```bash # Compress a JSON file jsonbrotlim compress -i data.json -o data.json.br # Compress with custom quality jsonbrotlim compress -i data.json -o data.fast.br -q 0 # Compress from stdin echo '{"name": "test"}' | jsonbrotlim compress > output.br # Compress from stdin to file cat data.json | jsonbrotlim compress -o compressed.br ``` -------------------------------- ### Handle Permission Issues Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Use user-level installation or virtual environments to avoid permission errors. ```bash # Install to user directory pip install --user jsonbrotliminifyer # Or use a virtual environment python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install jsonbrotliminifyer ``` -------------------------------- ### Commit message examples Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Examples of valid commit messages following the Conventional Commits specification. ```text feat: add support for custom compression quality levels fix: handle empty JSON objects in decompress_json function docs: update README with new CLI examples refactor: simplify compression algorithm in compress_json_file test: add unit tests for error handling in CLI ``` -------------------------------- ### Install jsonbrotliminifyer using pip Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Install the library using pip for easy integration into your Python projects. ```bash pip install jsonbrotliminifyer ``` -------------------------------- ### Create a feature branch Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Command to start a new branch for development. ```bash git checkout -b feature/your-feature-name ``` -------------------------------- ### Display Command-Specific Help Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Get detailed help for specific jsonbrotlim commands, such as 'compress' or 'decompress'. ```bash # Command-specific help jsonbrotlim compress --help jsonbrotlim decompress --help ``` -------------------------------- ### Display jsonbrotlim Version Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Show the installed version of the jsonbrotlim command-line tool. ```bash jsonbrotlim --version # Displays version information ``` -------------------------------- ### Runtime Decompression Example Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Decompress Brotli-compressed JSON data from standard input and pipe it to another application. ```bash jsonbrotlim decompress -i /app/config.br | myapp ``` -------------------------------- ### Data Pipeline Integration with Compression Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md Processes multiple JSON files, compressing them using Brotli and then performing further actions. This example demonstrates batch compression for storage or transfer. ```python import jsonbrotliminifyer from pathlib import Path import logging class DataPipeline: """Example data processing pipeline with compression.""" def __init__(self, temp_dir: str = "temp"): self.temp_dir = Path(temp_dir) self.temp_dir.mkdir(exist_ok=True) self.logger = logging.getLogger(__name__) def process_dataset(self, input_files: list, output_dir: str): """Process multiple JSON files with compression.""" output_path = Path(output_dir) output_path.mkdir(exist_ok=True) # Stage 1: Compress all input files self.logger.info(f"Compressing {len(input_files)} files...") compressed_files = [] results = jsonbrotliminifyer.compress_json_files( input_files=input_files, output_dir=str(self.temp_dir), quality=9, # High compression for storage max_workers=8 ) # Collect successful compressions for i, result in enumerate(results): if result is None: compressed_files.append( self.temp_dir / f"{Path(input_files[i]).stem}.br" ) else: self.logger.error(f"Failed to compress {input_files[i]}: {result}") # Stage 2: Further processing (e.g., upload to cloud storage) self.logger.info(f"Processing {len(compressed_files)} compressed files...") for compressed_file in compressed_files: # Simulate processing/upload self.logger.info(f"Processing {compressed_file}") # Move to final destination final_path = output_path / compressed_file.name compressed_file.rename(final_path) return len(compressed_files) # Usage pipeline = DataPipeline() processed = pipeline.process_dataset( input_files=["data1.json", "data2.json", "data3.json"], output_dir="processed" ) print(f"Successfully processed {processed} files") ``` -------------------------------- ### Reinstall jsonbrotliminifyer Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Uninstall and then reinstall the jsonbrotliminifyer package to resolve potential installation issues. ```bash # Reinstall pip uninstall jsonbrotliminifyer pip install jsonbrotliminifyer ``` -------------------------------- ### Check if jsonbrotlim is Installed Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Verify if the jsonbrotliminifyer package is installed in the current Python environment using pip list. ```bash # Check if installed pip list | grep jsonbrotliminifyer ``` -------------------------------- ### Set up virtual environment Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Commands to create and activate a Python virtual environment for dependency isolation. ```bash python -m venv venv # On Windows: venv\Scripts\activate # On macOS/Linux: source venv/bin/activate ``` -------------------------------- ### Configure Swap File (Linux) Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Create and enable a 4GB swap file to provide additional virtual memory for handling large files when system RAM is insufficient. ```bash # Use swap file if needed sudo fallocate -l 4G /swapfile sudo mkswap /swapfile sudo swapon /swapfile ``` -------------------------------- ### Display Main Help Information Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Show the main help message for the jsonbrotlim CLI, listing available commands and global options. ```bash # Main help jsonbrotlim --help ``` -------------------------------- ### Clone the repository Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Initial step to obtain the project source code from GitHub. ```bash git clone https://github.com/YOUR_USERNAME/JSONBrotliMinifyer.git cd JSONBrotliMinifyer ``` -------------------------------- ### Basic CLI Usage Syntax Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md General syntax for executing commands with the tool. ```bash jsonbrotlim [options] ``` -------------------------------- ### Configure Pre-commit Hooks Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/AGENTS.md Initializes automatic linting and formatting hooks. ```bash pre-commit install ``` -------------------------------- ### Run Linting and Formatting Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/AGENTS.md Commands to check code quality and apply automatic formatting. ```bash ruff check --fix ``` ```bash ruff format ``` -------------------------------- ### Integration with Other Tools Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Piping output directly to cloud storage tools like AWS S3 CLI. ```bash # Compress and upload jsonbrotlim compress -i config.json | aws s3 cp - s3://my-bucket/config.br # Download and decompress aws s3 cp s3://my-bucket/data.br - | jsonbrotlim decompress > data.json ``` -------------------------------- ### Performance Monitoring Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Using system tools to monitor resource usage during compression. ```bash # Monitor memory usage /usr/bin/time -v jsonbrotlim compress -i large_file.json -o large_file.br ``` -------------------------------- ### Run tests Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Execute the test suite using either unittest or pytest. ```bash # Using unittest python -m unittest tests.test_jsonbrotliminifyer # Or using pytest (if installed) pytest tests/test_jsonbrotliminifyer.py ``` -------------------------------- ### Execute Tests Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/AGENTS.md Commands for running the test suite or specific tests. ```bash pytest ``` ```bash pytest -k "" ``` -------------------------------- ### Run tests with unittest Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Execute the library's tests using Python's built-in unittest module. ```bash python -m unittest tests.test_jsonbrotliminifyer ``` -------------------------------- ### Check Python Version Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Verify the current Python version to ensure compatibility. ```bash python --version # Should show Python 3.9.0 or higher ``` -------------------------------- ### Upgrade Package Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Update the package to the latest version. ```bash pip install --upgrade jsonbrotliminifyer ``` -------------------------------- ### File Compression Workflow Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md A standard workflow for compressing, verifying, and restoring JSON files. ```bash # 1. Compress a large JSON file jsonbrotlim compress -i large_dataset.json -o large_dataset.json.br -q 9 # 2. Verify file was created ls -lh large_dataset.json* # 3. Decompress to verify integrity jsonbrotlim decompress -i large_dataset.json.br -o verified.json # 4. Compare original and decompressed diff large_dataset.json verified.json || echo "Files differ!" ``` -------------------------------- ### Check File Permissions Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md List detailed information about the permissions of input and output files using ls -la. ```bash # Check file permissions for input/output ls -la input.json output.br ``` -------------------------------- ### Batch Processing with Shell Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Using loops to process multiple files in a directory. ```bash # Compress all JSON files in directory for file in *.json; do jsonbrotlim compress -i "$file" -o "${file%.json}.br" done # Decompress all .br files for file in *.br; do jsonbrotlim decompress -i "$file" -o "${file%.br}.json" done ``` -------------------------------- ### Pipeline Usage Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Integrating compression and decompression into shell pipelines. ```bash # Compress API response curl -s https://api.example.com/data | jsonbrotlim compress > response.br # Decompress and format jsonbrotlim decompress -i response.br | jq '.' ``` -------------------------------- ### Compress Command Syntax Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Syntax for the compress command with optional input, output, and quality parameters. ```bash jsonbrotlim compress [-i INPUT_FILE] [-o OUTPUT_FILE] [-q QUALITY] ``` -------------------------------- ### Check jsonbrotlim PATH Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Determine the location of the jsonbrotlim executable and display the current system PATH environment variable. ```bash # Check PATH which jsonbrotlim echo $PATH ``` -------------------------------- ### Uninstall Package Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/installation.md Remove the package from the environment. ```bash pip uninstall jsonbrotliminifyer ``` -------------------------------- ### Handle File Not Found Error Source: https://context7.com/egorserdyuk/jsonbrotliminifyer/llms.txt Demonstrates how to catch a ValueError when a Brotli-compressed file is not found during decompression. ```python try: jsonbrotliminifyer.decompress_json_file( input_path="nonexistent.br", output_path="output.json" ) except ValueError as e: print(f"Error: {e}") # Output: Error: Input file does not exist: nonexistent.br ``` -------------------------------- ### Compress JSON file using CLI Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Compress a JSON file to a Brotli-compressed file using the command-line interface. ```bash jsonbrotlim compress --input-file input.json --output-file output.br ``` ```bash jsonbrotlim compress --input-file input.json --output-file output.br --quality 0 ``` ```bash echo '{"name": "test"}' | jsonbrotlim compress > output.br ``` -------------------------------- ### I/O Optimization Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Adjusting process priority for I/O and CPU intensive tasks. ```bash # Use ionice for I/O priority (Linux) ionice -c 3 jsonbrotlim compress -i large.json -o large.br # Use nice for CPU priority nice -n 10 jsonbrotlim compress -i large.json -o large.br ``` -------------------------------- ### Cache User Data Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md Demonstrates storing and retrieving user data using a cache. Ensure the cache object is properly initialized before use. ```python user_data = {"id": 123, "profile": {"name": "Alice", "preferences": {...}}} cache.set("user_123", user_data) retrieved = cache.get("user_123") assert retrieved == user_data ``` -------------------------------- ### Run tests with pytest Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Execute the library's tests using the pytest framework. ```bash pytest tests/test_jsonbrotliminifyer.py ``` -------------------------------- ### Run Type Checking Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/AGENTS.md Executes static type analysis using mypy. ```bash mypy . ``` -------------------------------- ### CLI: Compress JSON file Source: https://context7.com/egorserdyuk/jsonbrotliminifyer/llms.txt Compresses a JSON file to Brotli format using the command-line tool. Supports custom quality levels and compression from stdin. ```bash # Compress a JSON file to Brotli format jsonbrotlim compress -i data.json -o data.json.br # Compress with custom quality (0=fastest, 11=best compression) jsonbrotlim compress -i data.json -o data.fast.br -q 0 jsonbrotlim compress -i data.json -o data.best.br -q 11 # Compress from stdin echo '{"name": "test", "value": 123}' | jsonbrotlim compress > output.br # Compress from stdin to file cat data.json | jsonbrotlim compress -o compressed.br # Compress API response directly curl -s https://api.example.com/data | jsonbrotlim compress > response.br # Batch compress with shell loop for file in *.json; do jsonbrotlim compress -i "$file" -o "${file%.json}.br" -q 6 done # Parallel batch compression (4 files at a time) ls *.json | xargs -n 1 -P 4 -I {} jsonbrotlim compress -i {} -o {}.br ``` -------------------------------- ### JSON Compression with Different Quality Levels Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md Demonstrates how different quality levels affect the compressed size of JSON data. Higher quality means better compression but potentially slower performance. ```python import jsonbrotliminifyer data = {"large": "x" * 10000} # 10KB of data # Different quality levels for quality in [0, 6, 11]: compressed = jsonbrotliminifyer.compress_json(data, quality=quality) ratio = len(compressed) / len(json.dumps(data)) print(f"Quality {quality}: {len(compressed)} bytes ({ratio:.2f}x)") ``` -------------------------------- ### Ensure Execute Permissions for jsonbrotlim Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Grant execute permissions to the jsonbrotlim executable. This is usually not required but can help resolve 'command not found' errors if permissions are incorrect. ```bash # Ensure execute permissions (usually not needed) chmod +x $(which jsonbrotlim) ``` -------------------------------- ### Decompress JSON file using CLI Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Decompress a Brotli-compressed file back to a JSON file using the command-line interface. ```bash jsonbrotlim decompress --input-file output.br --output-file restored.json ``` ```bash cat output.br | jsonbrotlim decompress ``` -------------------------------- ### Convert Encoding before Compression Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Convert a file from Latin-1 encoding to UTF-8 using iconv before piping it to jsonbrotlim for compression. This addresses potential encoding issues. ```bash # Convert encoding first iconv -f latin1 -t utf8 input.json | jsonbrotlim compress > output.br ``` -------------------------------- ### Compress Multiple JSON Files Concurrently Source: https://context7.com/egorserdyuk/jsonbrotliminifyer/llms.txt Compresses multiple JSON files to Brotli format concurrently. Creates output files in a specified directory and returns a list of exceptions for failed compressions. ```python import jsonbrotliminifyer from pathlib import Path # Create sample JSON files for i in range(5): with open(f"file{i}.json", "w") as f: import json json.dump({"id": i, "data": "x" * 1000}, f) # Compress all files concurrently input_files = [f"file{i}.json" for i in range(5)] results = jsonbrotliminifyer.compress_json_files( input_files=input_files, output_dir="compressed/", quality=6, # Balanced compression max_workers=4 # Use 4 threads ) # Check results for each file successful = 0 for i, result in enumerate(results): if result is None: print(f"✓ {input_files[i]} -> compressed/file{i}.br") successful += 1 else: print(f"✗ {input_files[i]} failed: {result}") print(f"\nCompressed {successful}/{len(input_files)} files successfully") # Output: Compressed 5/5 files successfully ``` -------------------------------- ### Safe File Compression with Error Handling Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md Compresses a single file with comprehensive error handling for various exceptions like validation errors, file not found, and permission issues. Returns a boolean indicating success. ```python import jsonbrotliminifyer def safe_compress_file(input_path: str, output_path: str) -> bool: """Compress a file with comprehensive error handling.""" try: jsonbrotliminifyer.compress_json_file(input_path, output_path) print(f"Successfully compressed {input_path}") return True except ValueError as e: print(f"Validation error: {e}") return False except FileNotFoundError: print(f"Input file not found: {input_path}") return False except PermissionError: print(f"Permission denied accessing files") return False except Exception as e: print(f"Unexpected error: {e}") return False ``` -------------------------------- ### Conventional commit format Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/CONTRIBUTING.md Template for writing structured commit messages. ```text [optional scope]: [optional body] [optional footer(s)] ``` -------------------------------- ### Compress JSON Backups with Shell Script Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md A bash script to iterate through JSON files in a backup directory, compress them using jsonbrotlim, and save them to a compressed directory. It includes error handling for compression failures. ```bash #!/bin/bash # compress_backup.sh - Compress JSON backups BACKUP_DIR="/backups" COMPRESSED_DIR="/compressed" mkdir -p "$COMPRESSED_DIR" for json_file in "$BACKUP_DIR"/*.json; do if [[ -f "$json_file" ]]; then base_name=$(basename "$json_file" .json) compressed_file="$COMPRESSED_DIR/${base_name}.br" echo "Compressing $json_file -> $compressed_file" if jsonbrotlim compress -i "$json_file" -o "$compressed_file"; then echo "✓ Compressed successfully" else echo "✗ Compression failed" exit 1 fi fi done echo "All backups compressed!" ``` -------------------------------- ### Benchmark Compression and Decompression Performance Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md This function benchmarks the time taken for compression and decompression operations over a specified number of iterations. It also reports the original and compressed sizes, compression ratio, and speeds in MB/s. ```python import jsonbrotliminifyer import time import json def benchmark_compression(data, quality: int = 11, iterations: int = 10): """Benchmark compression performance.""" json_str = json.dumps(data) json_size = len(json_str.encode('utf-8')) # Benchmark compression compress_times = [] compressed_sizes = [] for _ in range(iterations): start = time.time() compressed = jsonbrotliminifyer.compress_json(data, quality=quality) compress_times.append(time.time() - start) compressed_sizes.append(len(compressed)) # Benchmark decompression decompress_times = [] for _ in range(iterations): start = time.time() jsonbrotliminifyer.decompress_json(compressed) decompress_times.append(time.time() - start) avg_compress_time = sum(compress_times) / len(compress_times) avg_decompress_time = sum(decompress_times) / len(decompress_times) avg_compressed_size = sum(compressed_sizes) / len(compressed_sizes) compression_ratio = avg_compressed_size / json_size print(f"Original size: {json_size} bytes") print(f"Compressed size: {avg_compressed_size:.0f} bytes") print(f"Compression ratio: {compression_ratio:.3f}x") print(f"Compress speed: {json_size / avg_compress_time / 1024 / 1024:.1f} MB/s") print(f"Decompress speed: {json_size / avg_decompress_time / 1024 / 1024:.1f} MB/s") # Usage test_data = { "users": [{"id": i, "data": "x" * 100} for i in range(1000)], "metadata": {"version": "1.0", "large_field": "y" * 50000} } benchmark_compression(test_data, quality=6) ``` -------------------------------- ### Compress and Decompress JSON Files in Python Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Compress JSON files to Brotli format and decompress them back. Supports default and custom compression quality. ```python import jsonbrotliminifyer # Compress a JSON file (with default quality 11) jsonbrotliminifyer.compress_json_file('input.json', 'output.br') # Compress with custom quality jsonbrotliminifyer.compress_json_file('input.json', 'output_fast.br', quality=0) # Decompress a compressed file back to JSON jsonbrotliminifyer.decompress_json_file('output.br', 'restored.json') ``` -------------------------------- ### Library type definitions Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Imports required for type hinting throughout the library. ```python from typing import Any, Union, List, Optional, Sequence, Tuple from pathlib import Path ``` -------------------------------- ### Compress JSON Data using Python CLI Integration Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Python function to compress JSON data by invoking the jsonbrotlim CLI tool. It handles temporary file creation, subprocess execution, and cleanup. Requires `tempfile`, `subprocess`, and `os` modules. ```python import subprocess import jsonbrotliminifyer import tempfile import os def compress_with_cli(data: dict, quality: int = 11) -> bytes: """Compress using CLI tool instead of library.""" # Write to temporary file with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f: json.dump(data, f) temp_input = f.name # Compress with CLI temp_output = temp_input + '.br' result = subprocess.run([ 'jsonbrotlim', 'compress', '-i', temp_input, '-o', temp_output, '-q', str(quality) ], capture_output=True, text=True) if result.returncode != 0: raise RuntimeError(f"CLI compression failed: {result.stderr}") # Read compressed data with open(temp_output, 'rb') as f: compressed = f.read() # Cleanup os.unlink(temp_input) os.unlink(temp_output) return compressed # Usage data = {"test": "data"} compressed = compress_with_cli(data) ``` -------------------------------- ### Decompress Command Syntax Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Syntax for the decompress command. ```bash jsonbrotlim decompress [-i INPUT_FILE] [-o OUTPUT_FILE] ``` -------------------------------- ### Implement a Compressed JSON Caching System Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md A class that provides a simple caching mechanism for JSON data, storing compressed versions to disk and retrieving them. Uses MD5 hashing for cache keys. ```python import jsonbrotliminifyer import hashlib from pathlib import Path class CompressedJSONCache: """A simple compressed JSON caching system.""" def __init__(self, cache_dir: str = ".cache"): self.cache_dir = Path(cache_dir) self.cache_dir.mkdir(exist_ok=True) def _get_cache_key(self, key: str) -> str: """Generate cache file path from key.""" hash_key = hashlib.md5(key.encode()).hexdigest() return str(self.cache_dir / f"{hash_key}.br") def set(self, key: str, data: dict, quality: int = 6): """Store compressed data.""" compressed = jsonbrotliminifyer.compress_json(data, quality=quality) cache_path = self._get_cache_key(key) with open(cache_path, 'wb') as f: f.write(compressed) def get(self, key: str) -> dict: """Retrieve and decompress data.""" cache_path = self._get_cache_key(key) if not Path(cache_path).exists(): raise KeyError(f"Cache key '{key}' not found") with open(cache_path, 'rb') as f: compressed = f.read() return jsonbrotliminifyer.decompress_json(compressed) def has(self, key: str) -> bool: """Check if key exists in cache.""" return Path(self._get_cache_key(key)).exists() # Usage cache = CompressedJSONCache() ``` -------------------------------- ### Batch Compress and Decompress JSON Files Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md Compresses multiple JSON files concurrently to a specified output directory and then decompresses them. Allows specifying the number of worker threads. ```python import jsonbrotliminifyer from pathlib import Path # Compress multiple files concurrently input_files = [ "user_data.json", "config.json", "cache.json" ] # Compress all files to output directory results = jsonbrotliminifyer.compress_json_files( input_files=input_files, output_dir="compressed/", quality=6, # Balanced speed/size max_workers=4 # Use 4 threads ) # Check for errors for i, result in enumerate(results): if result is not None: print(f"Failed to compress {input_files[i]}: {result}") else: print(f"Successfully compressed {input_files[i]}") # Decompress all files compressed_files = [f"compressed/{Path(f).stem}.br" for f in input_files] results = jsonbrotliminifyer.decompress_json_files( input_files=compressed_files, output_dir="restored/", max_workers=4 ) ``` -------------------------------- ### compress_json_files Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Compresses multiple JSON files concurrently to an output directory. ```APIDOC ## compress_json_files(input_files, output_dir, quality=11, max_workers=None) ### Description Compresses multiple JSON files concurrently to an output directory. ### Parameters #### Path Parameters - **input_files** (Sequence[Union[str, Path]]) - Required - List of input JSON file paths - **output_dir** (Union[str, Path]) - Required - Directory to save compressed files - **quality** (int) - Optional - Compression quality level (0-11). Default: 11 - **max_workers** (Optional[int]) - Optional - Max worker threads. Default: None ### Response - **List[Optional[Exception]]** - List of exceptions (None for success, Exception for failure) ``` -------------------------------- ### Decompress JSON Files API Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Decompresses multiple Brotli-compressed files concurrently to an output directory. ```APIDOC ## POST /api/decompress/files ### Description Decompresses multiple Brotli-compressed files concurrently to an output directory. Creates an output directory if it doesn't exist. Returns per-file error status. ### Method POST ### Endpoint /api/decompress/files ### Parameters #### Request Body - **input_files** (array[string]) - Required - List of input compressed file paths - **output_dir** (string) - Required - Directory to save decompressed JSON files - **max_workers** (integer) - Optional - Maximum number of worker threads. Defaults to a reasonable value. ### Request Example ```json { "input_files": ["file1.json.br", "file2.json.br", "file3.json.br"], "output_dir": "restored/", "max_workers": 4 } ``` ### Response #### Success Response (200) - **results** (array[object | null]) - List of exceptions (None for success, Exception object for failure). #### Response Example ```json { "results": [ null, { "error_type": "ValueError", "message": "Failed to decompress file2.json.br" }, null ] } ``` #### Error Handling - **ValueError**: If `max_workers` <= 0, duplicate input paths, duplicate output paths, file access errors, or path validation issues. - **BrotliError**: If the Brotli data is invalid and cannot be decompressed. - **JSONError**: If the decompressed data is not valid JSON. #### Notes - Output files will have the `.json` extension. - Concurrent processing is used for better performance. - The output directory is created if it does not exist. ``` -------------------------------- ### Decompress JSON files Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Restores a compressed file to a JSON file, outputting with 2-space indentation. ```python import jsonbrotliminifyer # Basic file decompression jsonbrotliminifyer.decompress_json_file( "data.json.br", "restored.json" ) ``` -------------------------------- ### Parallel Processing Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/cli.md Using xargs to process multiple files in parallel. ```bash # Compress multiple files in parallel (4 at a time) ls *.json | xargs -n 1 -P 4 -I {} jsonbrotlim compress -i {} -o {}.br ``` -------------------------------- ### Compress and Decompress JSON Files Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md Compresses a single JSON file to a Brotli-compressed file and then decompresses it back to its original JSON format. ```python import jsonbrotliminifyer # Compress a JSON file jsonbrotliminifyer.compress_json_file( input_path="data.json", output_path="data.json.br", quality=11 ) # Decompress back to JSON jsonbrotliminifyer.decompress_json_file( input_path="data.json.br", output_path="restored.json" ) ``` -------------------------------- ### Decompress JSON Files Source: https://context7.com/egorserdyuk/jsonbrotliminifyer/llms.txt Decompresses a Brotli-compressed file back into a JSON file with pretty formatting. ```python import jsonbrotliminifyer import json # Decompress a compressed file jsonbrotliminifyer.decompress_json_file( input_path="data.json.br", output_path="restored.json" ) # Verify the restored file with open("restored.json", "r") as f: restored_data = json.load(f) with open("data.json", "r") as f: original_data = json.load(f) assert restored_data == original_data print("File decompression verified!") ``` -------------------------------- ### Compress JSON Files API Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Compresses multiple JSON files into Brotli format concurrently. ```APIDOC ## POST /api/compress/files ### Description Compresses multiple JSON files into Brotli format concurrently. Creates an output directory if it doesn't exist. Returns per-file error status. ### Method POST ### Endpoint /api/compress/files ### Parameters #### Request Body - **input_files** (array[string]) - Required - List of input JSON file paths - **output_dir** (string) - Required - Directory to save compressed files - **quality** (integer) - Optional - Compression quality level (0-11). Defaults to a reasonable value. - **max_workers** (integer) - Optional - Maximum number of worker threads for concurrent processing. Defaults to a reasonable value. ### Request Example ```json { "input_files": ["file1.json", "file2.json", "file3.json"], "output_dir": "compressed/", "quality": 6, "max_workers": 4 } ``` ### Response #### Success Response (200) - **results** (array[object | null]) - List of results, where each element is either None for success or an Exception object for failure. #### Response Example ```json { "results": [ null, null, { "error_type": "ValueError", "message": "Failed to compress file3.json" } ] } ``` #### Error Handling - **ValueError**: If `quality` is not in the range 0-11, or if there are file access errors, or path validation issues. - **BrotliError**: If the Brotli data is invalid during compression. - **JSONError**: If the input data is not valid JSON. #### Notes - Output files will have the `.br` extension. - Concurrent processing is used for better performance. - The output directory is created if it does not exist. ``` -------------------------------- ### Compress multiple JSON files Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Compresses a list of JSON files concurrently into a specified output directory. ```python import jsonbrotliminifyer ``` -------------------------------- ### compress_json_file Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Compresses a JSON file using Brotli compression with atomic write operations. ```APIDOC ## compress_json_file(input_path, output_path, quality=11) ### Description Compresses a JSON file using Brotli compression with atomic write operations. ### Parameters #### Path Parameters - **input_path** (Union[str, Path]) - Required - Path to input JSON file - **output_path** (Union[str, Path]) - Required - Path to output compressed file - **quality** (int) - Optional - Compression quality level (0-11). Default: 11 ``` -------------------------------- ### JSONBrotliMinifyer Python API Reference Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Core functions for compressing and decompressing JSON data and files. ```APIDOC ## compress_json(json_obj, quality=11) ### Description Compresses a JSON-serializable Python object into Brotli-compressed bytes. ### Parameters - **json_obj** (Any) - Required - Any JSON-serializable Python object - **quality** (int) - Optional - Compression quality level (0-11), default 11 ### Response - **Returns** (bytes) - The compressed data --- ## decompress_json(compressed_bytes) ### Description Decompresses Brotli-compressed data back to the original JSON object. ### Parameters - **compressed_bytes** (bytes) - Required - The compressed data as bytes ### Response - **Returns** (Any) - The original Python object --- ## compress_json_file(input_path, output_path, quality=11) ### Description Compresses a JSON file using Brotli compression. ### Parameters - **input_path** (str) - Required - Path to the input JSON file - **output_path** (str) - Required - Path to the output compressed file - **quality** (int) - Optional - Compression quality level (0-11), default 11 --- ## decompress_json_file(input_path, output_path) ### Description Decompresses a Brotli-compressed file back to a JSON file. ### Parameters - **input_path** (str) - Required - Path to the input compressed file - **output_path** (str) - Required - Path to the output JSON file ``` -------------------------------- ### Compress and Decompress JSON Data in Python Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/README.md Compress JSON-serializable Python objects to bytes and decompress them back. Supports default and custom compression quality. ```python import jsonbrotliminifyer # Original JSON data data = { "name": "John Doe", "age": 30, "items": ["apple", "banana", "cherry"], "active": True } # Compress the data (with default quality 11) compressed = jsonbrotliminifyer.compress_json(data) print(f"Compressed size: {len(compressed)} bytes") # Compress with custom quality (0-11, lower = faster but larger) compressed_fast = jsonbrotliminifyer.compress_json(data, quality=0) print(f"Fast compression size: {len(compressed_fast)} bytes") # Decompress the data decompressed = jsonbrotliminifyer.decompress_json(compressed) print(decompressed) # Should match the original data ``` -------------------------------- ### Compress multiple JSON files Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Compresses a list of JSON files concurrently into a specified directory. ```python input_files = ["file1.json", "file2.json", "file3.json"] results = jsonbrotliminifyer.compress_json_files( input_files=input_files, output_dir="compressed/", quality=6, max_workers=4 ) # Check results for i, result in enumerate(results): if result is None: print(f"✓ {input_files[i]} compressed successfully") else: print(f"✗ {input_files[i]} failed: {result}") ``` -------------------------------- ### Monitor Memory Usage During Compression/Decompression Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md This function monitors the Resident Set Size (RSS) memory usage of the current process before and after executing a given function. It's useful for understanding the memory overhead of compression and decompression operations. ```python import jsonbrotliminifyer import psutil import os def monitor_memory_usage(func, *args, **kwargs): """Monitor memory usage during compression/decompression.""" process = psutil.Process(os.getpid()) initial_memory = process.memory_info().rss / 1024 / 1024 # MB result = func(*args, **kwargs) final_memory = process.memory_info().rss / 1024 / 1024 # MB memory_delta = final_memory - initial_memory print(f"Memory usage: {initial_memory:.1f}MB → {final_memory:.1f}MB " f"(Δ{memory_delta:+.1f}MB)") return result # Usage large_data = {"data": "x" * 10_000_000} # ~10MB string compressed = monitor_memory_usage( jsonbrotliminifyer.compress_json, large_data, quality=6 ) decompressed = monitor_memory_usage( jsonbrotliminifyer.decompress_json, compressed ) ``` -------------------------------- ### Decompress multiple Brotli files Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Decompresses a list of Brotli-compressed files concurrently into a specified directory. ```python import jsonbrotliminifyer # Decompress multiple files compressed_files = ["file1.json.br", "file2.json.br", "file3.json.br"] results = jsonbrotliminifyer.decompress_json_files( input_files=compressed_files, output_dir="restored/", max_workers=4 ) # Check results for i, result in enumerate(results): if result is None: print(f"✓ {compressed_files[i]} decompressed successfully") else: print(f"✗ {compressed_files[i]} failed: {result}") ``` -------------------------------- ### Decompress Multiple Brotli JSON Files Concurrently Source: https://context7.com/egorserdyuk/jsonbrotliminifyer/llms.txt Decompresses multiple Brotli-compressed JSON files concurrently. Creates the output directory if it doesn't exist and verifies data integrity. ```python import jsonbrotliminifyer from pathlib import Path # Decompress all compressed files compressed_files = list(Path("compressed").glob("*.br")) results = jsonbrotliminifyer.decompress_json_files( input_files=[str(f) for f in compressed_files], output_dir="restored/", max_workers=4 ) # Verify results for i, result in enumerate(results): if result is None: print(f"✓ Decompressed {compressed_files[i].name}") else: print(f"✗ Failed {compressed_files[i].name}: {result}") # Verify data integrity import json for i in range(5): with open(f"file{i}.json") as orig, open(f"restored/file{i}.json") as rest: assert json.load(orig) == json.load(rest) print("\nAll files verified!") ``` -------------------------------- ### API Response Compression with Brotli Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/usage.md Compresses API responses using Brotli if the client accepts it. This Flask endpoint serves compressed JSON data, reducing bandwidth usage. ```python from flask import Flask, request, Response import jsonbrotliminifyer app = Flask(__name__) @app.route('/api/data') def get_compressed_data(): """API endpoint that serves compressed JSON data.""" # Generate response data data = { "status": "success", "data": { "items": [...], # Large dataset "metadata": {...} }, "timestamp": "2024-01-01T00:00:00Z" } # Check if client accepts Brotli compression accept_encoding = request.headers.get('Accept-Encoding', '') if 'br' in accept_encoding: # Compress response compressed = jsonbrotliminifyer.compress_json(data, quality=6) return Response( compressed, mimetype='application/json', headers={'Content-Encoding': 'br'} ) else: # Return uncompressed JSON return data if __name__ == '__main__': app.run() ``` -------------------------------- ### decompress_json_file Source: https://github.com/egorserdyuk/jsonbrotliminifyer/blob/main/docs/api.md Decompresses a Brotli-compressed file back to a JSON file. ```APIDOC ## decompress_json_file(input_path, output_path) ### Description Decompresses a Brotli-compressed file back to a JSON file. ### Parameters #### Path Parameters - **input_path** (Union[str, Path]) - Required - Path to input compressed file - **output_path** (Union[str, Path]) - Required - Path to output JSON file ```