### Install Locally with Cythonization Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Install the library locally from the source directory, forcing Cythonization of the extension modules. This requires Cython and a C compiler. ```sh pip install --upgrade . --config-setting="--build-option=--cythonize" --verbose ``` -------------------------------- ### Install Cutting-Edge Code Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Install the cutting-edge code directly from the GitHub repository. This version is likely unstable and not recommended for production. ```sh pip install --upgrade git+https://github.com/tomerfiliba-org/reedsolomon ``` -------------------------------- ### Install with sdist (Force Use) Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst If you encounter installation issues with pip, this command forces the use of sdist instead of wheels, which may help resolve the problem. ```sh pip install reedsolo --no-binary="reedsolo" --no-cache ``` -------------------------------- ### Install with Conda Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Install a compiled version of the reedsolo library for various platforms using Conda from the conda-forge channel. ```sh conda install -c conda-forge reedsolo ``` -------------------------------- ### Install Latest Stable Release Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Use this command to install the latest stable release of the reedsolo library, compatible with Python 3.7 and above. ```sh pip install --upgrade reedsolo ``` -------------------------------- ### Install Specific Version (Python <= 3.6) Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst For Python 2.7 and Python versions up to 3.6, install version 1.7.0 of the reedsolo library. ```sh pip install --upgrade reedsolo==1.7.0 ``` -------------------------------- ### Install Latest Development Release Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Install the latest development release of reedsolo using this command. This version may be unstable. ```sh pip install --upgrade reedsolo --pre ``` -------------------------------- ### Cython Byte Array Manipulation Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/creedsolo-arrays-speedtests.ipynb Demonstrates basic byte array manipulation and concatenation using Cython. Ensure Cython is installed and configured for your environment. ```cython %%cython --annotate from cython cimport bytearray ctypedef unsigned char uint8_t cdef bytearray_setval(uint8_t[::1] a, int val): for i in range(len(a)): a[i] = val cdef uint8_t[::1] a = bytearray(3) bytearray_setval(a, 2) cdef uint8_t[::1] b = bytearray(5) bytearray_setval(b, 3) print(bytearray(a) + bytearray(b)) ``` -------------------------------- ### Converting String to Bytearray Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst The Cython implementation of RSCodec exclusively accepts bytearray objects. This example shows how to convert a string to a bytearray using a specified encoding. ```python bytearray("Hello World", "UTF-8") ``` -------------------------------- ### RSCodec with More ECC Symbols Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Using a larger number of ECC symbols allows for correcting more errors or erasures. This example demonstrates encoding and decoding with 12 ECC symbols. ```python >>> rsc = RSCodec(12) # using 2 more ecc symbols (to correct max 6 errors or 12 erasures) >>> rsc.encode(b'hello world') b'hello world?Ay\xb2\xbc\xdc\x01q\xb9\xe3\xe2=' >>> rsc.decode(b'hello worXXXXy\xb2XX\x01q\xb9\xe3\xe2=')[0] # 6 errors - ok, but any more would fail b'hello world' ``` ```python >>> rsc.decode(b'helXXXXXXXXXXy\xb2XX\x01q\xb9\xe3\xe2=', erase_pos=[3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16])[0] # 12 erasures - OK b'hello world' ``` -------------------------------- ### Initialize C extension codec Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Initializes the speed-optimized creedsolo codec. ```python >> import creedsolo as crs >> codec = crs.RSCodec(10) ``` -------------------------------- ### Initialize Galois Field Tables with init_tables() Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Sets up lookup tables for Galois field arithmetic. This must be called before using low-level functions. ```python import reedsolo as rs # Initialize with default parameters (GF(2^8), prim=0x11d) gf_log, gf_exp, field_charac = rs.init_tables(0x11d) print(f"Field characteristic: {field_charac}") # 255 # Initialize for larger Galois field (GF(2^12)) prim = rs.find_prime_polys(c_exp=12, fast_primes=True, single=True)[0] gf_log, gf_exp, field_charac = rs.init_tables(c_exp=12, prim=prim) print(f"Field characteristic: {field_charac}") # 4095 # Custom generator gf_log, gf_exp, field_charac = rs.init_tables(prim=0x11d, generator=2, c_exp=8) ``` -------------------------------- ### Run local build and checks Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/BUILD.txt Executes local build processes and formatting checks. ```bash pymake build ``` -------------------------------- ### Benchmark Array Initialization Methods Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/creedsolo-arrays-speedtests.ipynb Measures the time taken to allocate arrays using different Cython and C-level techniques. ```cython # cython: wraparound=False import time import sys from cpython.array cimport array, clone from cython.view cimport array as cvarray from libc.stdlib cimport malloc, free import numpy as numpy cimport numpy as numpy cdef int loops def timefunc(name): def timedecorator(f): cdef int L, i print("Running", name) for L in [1, 10, 100, 1000, 10000, 100000]: start = time.process_time() f(L) end = time.process_time() print(format((end-start) / loops * 1e6, "2f"), end=" ") sys.stdout.flush() print("μs") return timedecorator print() print("INITIALISATIONS") loops = 100000 @timefunc("cpython.array buffer") def _(int L): cdef int i cdef array[double] arr, template = array('d') for i in range(loops): arr = clone(template, L, False) # Prevents dead code elimination str(arr[0]) @timefunc("cpython.array memoryview") def _(int L): cdef int i cdef double[::1] arr cdef array template = array('d') for i in range(loops): arr = clone(template, L, False) # Prevents dead code elimination str(arr[0]) @timefunc("cpython.array raw C type") def _(int L): cdef int i cdef array arr, template = array('d') for i in range(loops): arr = clone(template, L, False) # Prevents dead code elimination str(arr[0]) @timefunc("numpy.empty_like memoryview") def _(int L): cdef int i cdef double[::1] arr template = numpy.empty((L,), dtype='double') for i in range(loops): arr = numpy.empty_like(template) # Prevents dead code elimination str(arr[0]) @timefunc("malloc") def _(int L): cdef int i cdef double* arrptr for i in range(loops): arrptr = malloc(sizeof(double) * L) free(arrptr) # Prevents dead code elimination str(arrptr[0]) @timefunc("malloc memoryview") def _(int L): cdef int i cdef double* arrptr cdef double[::1] arr for i in range(loops): arrptr = malloc(sizeof(double) * L) arr = arrptr free(arrptr) # Prevents dead code elimination str(arr[0]) @timefunc("cvarray memoryview") def _(int L): cdef int i cdef double[::1] arr for i in range(loops): arr = cvarray((L,),sizeof(double),'d') # Prevents dead code elimination str(arr[0]) ``` -------------------------------- ### Compile from Source (Latest Stable) Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst To compile the latest stable release from source, including the optional Cythonized extension, use this command. It requires Cython and a C compiler. ```sh pip install --upgrade reedsolo --no-binary "reedsolo" --no-cache --config-setting="--build-option=--cythonize" --use-pep517 --isolated --verbose ``` -------------------------------- ### Build with build tool (Native Compile) Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Use the 'build' tool to compile the package locally, skipping Cythonization and only compiling from existing C extensions. This is faster if Cython is not available or needed. ```sh python -sBm build --config-setting="--build-option=--native-compile" ``` -------------------------------- ### Compile from Source (Latest Development) Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Compile the latest development release from source with Cythonization. This command is for the pre-release version and requires Cython and a C compiler. ```sh pip install --upgrade reedsolo --no-binary "reedsolo" --no-cache --config-setting="--build-option=--cythonize" --use-pep517 --isolated --pre --verbose ``` -------------------------------- ### Initialize Low-Level Tables Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Initialize precomputed tables for direct library access, optionally specifying a custom Galois Field exponent. ```python >> import reedsolo as rs >> rs.init_tables(0x11d) ``` ```python >> prim = rs.find_prime_polys(c_exp=12, fast_primes=True, single=True)[0] >> rs.init_tables(c_exp=12, prim=prim) ``` -------------------------------- ### Upload build artifacts Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/BUILD.txt Uploads the built distribution packages. ```bash pymake upload ``` -------------------------------- ### Cython import for C extension Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Demonstrates how to cimport the creedsolo module for Cython integration. ```python >> import cython >> cimport cython >> cimport creedsolo.creedsolo as crs ``` -------------------------------- ### Low-Level API: init_tables() - Initialize Galois Field Tables Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Initializes the logarithm and anti-logarithm lookup tables for Galois field arithmetic. Required before using low-level encoding/decoding functions. ```APIDOC ## rs.init_tables() ### Description Initializes the logarithm and anti-logarithm lookup tables for Galois field arithmetic. Required before using low-level encoding/decoding functions. ### Method `init_tables(prim=0x11d, generator=2, c_exp=8)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **prim** (int) - The primitive polynomial for the Galois Field. Defaults to 0x11d for GF(2^8). - **generator** (int) - The generator for the Galois Field. Defaults to 2. - **c_exp** (int) - The exponent for the Galois Field (e.g., 8 for GF(2^8)). Defaults to 8. ### Request Example ```python # Initialize with default parameters (GF(2^8), prim=0x11d) gf_log, gf_exp, field_charac = rs.init_tables(0x11d) print(f"Field characteristic: {field_charac}") # Initialize for larger Galois field (GF(2^12)) prim = rs.find_prime_polys(c_exp=12, fast_primes=True, single=True)[0] gf_log, gf_exp, field_charac = rs.init_tables(c_exp=12, prim=prim) print(f"Field characteristic: {field_charac}") # Custom generator gf_log, gf_exp, field_charac = rs.init_tables(prim=0x11d, generator=2, c_exp=8) ``` ### Response #### Success Response (200) - **gf_log** (list[int]) - The logarithm lookup table. - **gf_exp** (list[int]) - The anti-logarithm lookup table. - **field_charac** (int) - The characteristic of the Galois Field. #### Response Example ```json { "example": "([0, 1, 2, ...], [1, 2, 4, ...], 255)" } ``` ``` -------------------------------- ### Build with build tool (Cythonization) Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Use the 'build' tool to compile the package locally, including Cythonization. This process converts .pyx files to .c and then to .pyd (or equivalent). ```sh pip install build python -sBm build --config-setting="--build-option=--cythonize" ``` -------------------------------- ### Compile from Source (Cutting Edge) Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Compile the cutting-edge code from the GitHub repository with Cythonization. This is for the most recent, potentially unstable code and requires Cython and a C compiler. ```sh pip install --upgrade "reedsolo @ git+https://github.com/tomerfiliba-org/reedsolomon" --no-binary "reedsolo" --no-cache --config-setting="--build-option=--cythonize" --use-pep517 --isolated --pre --verbose ``` -------------------------------- ### Manage Multiple Codec Instances Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Demonstrates using multiple RSCodec instances simultaneously, where each instance maintains its own internal Galois field tables. ```python from reedsolo import RSCodec # Create codecs with different parameters codec_small = RSCodec(4) # Low redundancy codec_medium = RSCodec(10) # Medium redundancy codec_large = RSCodec(32) # High redundancy # Each codec can be used independently message = b"Important data" # Encode with different levels encoded_small = codec_small.encode(message) encoded_medium = codec_medium.encode(message) encoded_large = codec_large.encode(message) print(f"Small: {len(encoded_small)} bytes (corrects {codec_small.maxerrata()[0]} errors)") print(f"Medium: {len(encoded_medium)} bytes (corrects {codec_medium.maxerrata()[0]} errors)") print(f"Large: {len(encoded_large)} bytes (corrects {codec_large.maxerrata()[0]} errors)") # Each decodes only its own format decoded_s, _, _ = codec_small.decode(encoded_small) decoded_m, _, _ = codec_medium.decode(encoded_medium) decoded_l, _, _ = codec_large.decode(encoded_large) ``` -------------------------------- ### Initialize RSCodec Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Configure the codec with error correction symbols and Galois field parameters. Use custom parameters for specific compatibility requirements or larger message sizes. ```python from reedsolo import RSCodec, ReedSolomonError # Basic initialization with 10 error correction symbols # Can correct up to 5 errors or 10 erasures rsc = RSCodec(10) # Custom Galois field for longer messages (up to 4095 symbols per chunk) rsc_large = RSCodec(12, nsize=4095) # Alternative using exponent rsc_large = RSCodec(12, c_exp=12) # For ADSB UAT FEC compatibility rsc_adsb = RSCodec(nsym=10, fcr=120, prim=0x187) # For variable nsym encoding (use single_gen=False) rsc_variable = RSCodec(nsym=10, single_gen=False) ``` -------------------------------- ### Performance Comparison of Array Creation Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/creedsolo-arrays-speedtests.ipynb Compares the performance of creating arrays using Cython's cvarray, NumPy, raw C allocation, and CPython's array module. This benchmark measures the time taken to create arrays of varying lengths over a large number of iterations. ```cython %%cython --annotate from cython.view cimport array as cvarray import time import numpy as np cimport numpy as np from cpython cimport array from libc.stdlib cimport malloc, free cdef double* ptr cdef long N = 1000000 cdef array.array Lrange = array.array('i', [1,10,100,1000,10000]) cdef long i cdef int L for L in Lrange: t1 = time.process_time() for i in range(N): a = cvarray((L,),sizeof(double),'d') t2 = time.process_time() print('Elapsed time to make cython array with length of ' +str(L)+' is '+str((t2-t1)/N*1e6)+' us') for L in Lrange: t1 = time.process_time() for i in range(N): a = np.arange(L) t2 = time.process_time() print('Elapsed time to make numpy array with length of ' +str(L)+' is '+str((t2-t1)/N*1e6)+' us') for L in Lrange: t1 = time.process_time() for i in range(N): ptr = malloc(sizeof(double) * L) free(ptr) t2 = time.process_time() print('Elapsed time with raw c-allocation with length of ' +str(L)+' is '+str((t2-t1)/N*1e6)+' us') cdef array.array ar, template = array.array('d') for L in Lrange: t1 = time.process_time() for i in range(N): ar = array.clone(template, L, False) t2 = time.process_time() print('Elapsed time to make cpython array with length of ' +str(L)+' is '+str((t2-t1)/N*1e6)+' us') #TODO: C array https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html#quickstart #for L in [1,10,100,1000,10000]: # t1 = time.process_time() # for i in range(N): # cdef double[L] arc # t2 = time.process_time() ``` -------------------------------- ### Load Cython Extension Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/creedsolo-arrays-speedtests.ipynb This command loads the Cython extension for use in an IPython environment. ```python %load_ext Cython ``` -------------------------------- ### Run test coverage Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/BUILD.txt Verifies test coverage metrics locally. ```bash pymake testcoverage ``` -------------------------------- ### Benchmark Cython bytearray creation Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/creedsolo-arrays-speedtests.ipynb Measures the time to create Cython bytearrays of different lengths. Requires Cython and time modules. ```cython %%cython --annotate import time from cython cimport bytearray from cpython cimport array cdef long N = 1000000 cdef array.array Lrange = array.array('i', [1,10,100,1000,10000]) cdef long i cdef int L cdef bytearray barr for L in Lrange: t1 = time.process_time() for i in range(N): barr = bytearray(L) t2 = time.process_time() print('Elapsed time to make bytearray with length of ' +str(L)+' is '+str((t2-t1)/N*1e6)+' us') cdef unsigned char[::1] mbarr for L in Lrange: t1 = time.process_time() for i in range(N): mbarr = bytearray(L) t2 = time.process_time() print('Elapsed time to make bytearray memoryview with length of ' +str(L)+' is '+str((t2-t1)/N*1e6)+' us') ``` -------------------------------- ### Run security scan Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/BUILD.txt Tests for vulnerabilities using bandit. ```bash pymake bandit ``` -------------------------------- ### Importing RSCodec from Cython Extension Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst When using the optimized Cython implementation, import RSCodec from 'creedsolo' instead of 'reedsolo'. This version is significantly faster. ```python from creedsolo import RSCodec ``` -------------------------------- ### Initialize RSCodec Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Initialize the RSCodec with the desired number of error correction symbols. ```python # Initialization >>> from reedsolo import RSCodec, ReedSolomonError >>> rsc = RSCodec(10) # 10 ecc symbols ``` -------------------------------- ### MATLAB Compatibility Configuration Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Set specific prime and fcr parameters to ensure compatibility with MATLAB's rsenc codec. ```python >> rsc = RSCodec(12, prim=285, fcr=1) ``` -------------------------------- ### Cython Configuration Options Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/creedsolo-arrays-speedtests.ipynb Sets language level and disables bounds checking for Cython code. This can improve performance but requires careful attention to potential errors. ```cython %%cython --annotate # cython: language_level=3 # cython: boundscheck=False ``` -------------------------------- ### Benchmark Array Iteration Methods Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/creedsolo-arrays-speedtests.ipynb Measures the time taken to iterate over pre-allocated arrays using different Cython and C-level techniques. ```cython print() print("ITERATING") loops = 1000 @timefunc("cpython.array buffer") def _(int L): cdef int i cdef array[double] arr = clone(array('d'), L, False) cdef double d for i in range(loops): for i in range(L): d = arr[i] # Prevents dead-code elimination str(d) @timefunc("cpython.array memoryview") def _(int L): cdef int i cdef double[::1] arr = clone(array('d'), L, False) cdef double d for i in range(loops): for i in range(L): d = arr[i] # Prevents dead-code elimination str(d) @timefunc("cpython.array raw C type") def _(int L): cdef int i cdef array arr = clone(array('d'), L, False) cdef double d for i in range(loops): for i in range(L): d = arr[i] # Prevents dead-code elimination str(d) @timefunc("numpy.empty_like memoryview") def _(int L): cdef int i cdef double[::1] arr = numpy.empty((L,), dtype='double') cdef double d for i in range(loops): for i in range(L): d = arr[i] # Prevents dead-code elimination str(d) @timefunc("malloc") def _(int L): cdef int i cdef double* arrptr = malloc(sizeof(double) * L) cdef double d for i in range(loops): for i in range(L): d = arrptr[i] free(arrptr) # Prevents dead-code elimination str(d) @timefunc("malloc memoryview") def _(int L): cdef int i cdef double* arrptr = malloc(sizeof(double) * L) cdef double[::1] arr = arrptr cdef double d for i in range(loops): for i in range(L): d = arr[i] free(arrptr) # Prevents dead-code elimination str(d) @timefunc("cvarray memoryview") def _(int L): cdef int i cdef double[::1] arr = cvarray((L,),sizeof(double),'d') cdef double d for i in range(loops): for i in range(L): d = arr[i] # Prevents dead-code elimination str(d) ``` -------------------------------- ### High-Performance Encoding with Cython Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Utilizes the creedsolo Cython extension for faster encoding. Requires bytearray inputs for optimal performance. ```python # Import creedsolo (must be compiled first) try: from creedsolo import RSCodec print("Using Cython-optimized creedsolo") except ImportError: from reedsolo import RSCodec print("Falling back to pure-Python reedsolo") # Usage is identical to reedsolo rsc = RSCodec(10) # IMPORTANT: Always use bytearray with creedsolo for best performance message = bytearray(b"hello world") encoded = rsc.encode(message) print(encoded) # Decode decoded_msg, decoded_msgecc, errata_pos = rsc.decode(bytearray(encoded)) print(bytes(decoded_msg)) # For Cython cimport (advanced usage) # cimport creedsolo.creedsolo as crs ``` -------------------------------- ### RSCodec Class Initialization Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Initializes the Reed-Solomon codec with specific error correction parameters and Galois field configurations. ```APIDOC ## RSCodec Initialization ### Description Initializes the RSCodec instance with configurable error correction symbols and Galois field parameters. ### Parameters - **nsym** (int) - Required - Number of error correction symbols. - **nsize** (int) - Optional - Size of the Galois field (default 255). - **c_exp** (int) - Optional - Exponent for the Galois field. - **fcr** (int) - Optional - First consecutive root. - **prim** (int) - Optional - Primitive polynomial. - **single_gen** (bool) - Optional - Whether to use a single generator polynomial (default True). ``` -------------------------------- ### Low-Level Encoding and Decoding Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Perform manual encoding and decoding by managing generator polynomials and message buffers directly. ```python >> n = 255 # length of total message+ecc >> nsym = 12 # length of ecc >> mes = "a" * (n-nsym) # generate a sample message ``` ```python >> gen = rs.rs_generator_poly_all(n) ``` ```python >> mesecc = rs.rs_encode_msg(mes, nsym, gen=gen[nsym]) ``` ```python >> mesecc[1] = 0 ``` ```python >> rmes, recc, errata_pos = rs.rs_correct_msg(mesecc, nsym, erase_pos=erase_pos) ``` -------------------------------- ### Bitwise Operations for Bit Manipulation Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/TODO.txt Demonstrates common bitwise operations for manipulating individual bits within an integer. Useful for low-level data manipulation when working with bit arrays. ```python x = 1<<10 x |= 1<<19 x &= 1<<19 x &= ~(1<<19) x ^= 1<<19 x = ~x mask = ((1<<20)-1) x &= mask x ^= mask (x >> 19) & 1 (x >> 16) & 0xf ``` -------------------------------- ### Check and Repair Messages Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Demonstrates how to detect corruption in encoded messages and perform error correction using the RSCodec class. ```python tampered = bytearray(encoded) tampered[5] = ord('X') result = rsc.check(tampered) print(result) # [False] # Repair and verify decoded_msg, decoded_msgecc, errata_pos = rsc.decode(tampered) result = rsc.check(decoded_msgecc) print(result) # [True] # Check multi-chunk message long_encoded = rsc.encode(b'A' * 500) long_tampered = bytearray(long_encoded) long_tampered[10] = 0 # Corrupt first chunk result = rsc.check(long_tampered) print(result) # [False, True, True] - per-chunk results ``` -------------------------------- ### Encode Data with RSCodec Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Encode data using the RSCodec. Accepts lists of numbers, bytearrays, or byte strings. Bytearrays are recommended for consistent results. ```python # Encoding # just a list of numbers/symbols: >>> rsc.encode([1,2,3,4]) b'\x01\x02\x03\x04,\x9d\x1c+=\xf8h\xfa\x98M' ``` ```python # bytearrays are accepted and the output will be matched: >>> rsc.encode(bytearray([1,2,3,4])) bytearray(b'\x01\x02\x03\x04,\x9d\x1c+=\xf8h\xfa\x98M') ``` ```python # encoding a byte string is as easy: >>> rsc.encode(b'hello world') b'hello world\xed%T\xc4\xfd\xfd\x89\xf3\xa8\xaa' ``` -------------------------------- ### Configure Galois Field for Larger Data Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Increase the Galois Field size to support longer messages or values exceeding the default 256-bit limit. ```python # To use longer chunks or bigger values than 255 (may be very slow) >> rsc = RSCodec(12, nsize=4095) # always use a power of 2 minus 1 >> rsc = RSCodec(12, c_exp=12) # alternative way to set nsize=4095 >> mes = 'a' * (4095-12) >> mesecc = rsc.encode(mes) >> mesecc[2] = 1 >> mesecc[-1] = 1 >> rmes, rmesecc, errata_pos = rsc.decode(mesecc) >> rsc.check(mesecc) [False] >> rsc.check(rmesecc) [True] ``` -------------------------------- ### Restore global codec state Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Restores previously backed up global tables and performs encoding/decoding operations. ```python >> global gf_log, gf_exp, field_charac >> gf_log, gf_exp, field_charac = bak_gf_log, bak_gf_exp, bak_field_charac >> mesecc = rs.rs_encode_msg(mes, nsym) >> rmes, recc, errata_pos = rs.rs_correct_msg(mesecc, nsym) ``` -------------------------------- ### Backup global codec state Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Saves the current global Reed-Solomon tables to allow switching between different codec parameters. ```python >> rs.init_tables() >> global gf_log, gf_exp, field_charac >> bak_gf_log, bak_gf_exp, bak_field_charac = gf_log, gf_exp, field_charac ``` -------------------------------- ### find_prime_polys() Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Finds prime (irreducible) polynomials for a given Galois field, required when using custom field sizes. ```APIDOC ## find_prime_polys(c_exp, fast_primes, single, generator) ### Description Finds prime (irreducible) polynomials for a given Galois field. ### Parameters #### Query Parameters - **c_exp** (int) - Required - The exponent for the Galois field (e.g., 8 for GF(2^8)). - **fast_primes** (bool) - Optional - Whether to use a faster search method. - **single** (bool) - Optional - If True, returns only one prime polynomial. - **generator** (int) - Optional - Custom generator for the polynomial search. ``` -------------------------------- ### Calculate message syndromes Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Displays the syndromes for a given message and number of ECC symbols. ```python >> rs.rs_calc_syndromes(msg, nsym) ``` -------------------------------- ### Find Prime Polynomials for Galois Fields Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Identifies irreducible polynomials for specific Galois fields, which is necessary when configuring custom field sizes. ```python import reedsolo as rs # Find all prime polynomials for GF(2^8) primes = rs.find_prime_polys(c_exp=8, fast_primes=False) print(f"Found {len(primes)} prime polynomials for GF(2^8)") print(f"First few: {[hex(p) for p in primes[:5]]}") # Find a single prime polynomial quickly single_prime = rs.find_prime_polys(c_exp=8, fast_primes=True, single=True) print(f"Single prime: {hex(single_prime[0])}") # Find prime polynomials for larger field GF(2^12) primes_12 = rs.find_prime_polys(c_exp=12, fast_primes=True, single=True) print(f"Prime for GF(2^12): {hex(primes_12[0])}") # Custom generator primes_gen3 = rs.find_prime_polys(generator=3, c_exp=8, fast_primes=True) print(f"Primes with generator=3: {[hex(p) for p in primes_gen3[:3]]}") ``` -------------------------------- ### Calculate Syndrome Polynomials Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Computes syndrome polynomials to detect errors. Non-zero values indicate corruption in the message. ```python import reedsolo as rs # Initialize tables rs.init_tables(0x11d) nsym = 10 message = bytearray(b"hello world") encoded = rs.rs_encode_msg(message, nsym) # Calculate syndromes for valid message syndromes = rs.rs_calc_syndromes(encoded, nsym) print(f"Syndromes (valid): {syndromes}") # All zeros: [0, 0, 0, ...] # Introduce error and recalculate encoded[3] = 0 syndromes = rs.rs_calc_syndromes(encoded, nsym) print(f"Syndromes (corrupted): {syndromes}") # Non-zero values indicate errors print(f"Max syndrome value: {max(syndromes)}") # > 0 means errors present ``` -------------------------------- ### Calculate Correction Capacity with maxerrata() Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Determines the maximum number of errors and erasures a codec can handle. Useful for adaptive bitrate logic. ```python from reedsolo import RSCodec rsc = RSCodec(12) # Get independent max errors and erasures max_errors, max_erasures = rsc.maxerrata(verbose=True) # Output: This codec can correct up to 6 errors and 12 erasures independently print(f"Max errors: {max_errors}, Max erasures: {max_erasures}") # Calculate remaining error capacity given known erasures max_errors, erasures = rsc.maxerrata(erasures=6, verbose=True) # Output: This codec can correct up to 3 errors and 6 erasures simultaneously print(f"With 6 erasures, can still correct {max_errors} errors") # Calculate remaining erasure capacity given known errors errors, max_erasures = rsc.maxerrata(errors=5, verbose=True) # Output: This codec can correct up to 5 errors and 2 erasures simultaneously print(f"With 5 errors, can still correct {max_erasures} erasures") ``` -------------------------------- ### RSCodec Class Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt High-level interface for Reed-Solomon encoding and decoding. ```APIDOC ## RSCodec(nsym) ### Description Initializes a Reed-Solomon codec instance with a specific number of ECC symbols. ### Methods - **encode(message)**: Encodes the provided message. - **decode(encoded)**: Decodes the provided message and returns the original data. - **maxerrata()**: Returns the maximum number of errors the codec can correct. ``` -------------------------------- ### RSCodec Decode Output Usage Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Understand the three return values of RSCodec.decode(): the corrected message, the corrected message with ECC, and the errata positions. Errata positions can be converted to a list of integers. ```python >>> tampered_msg = b'heXlo worXd\xed%T\xc4\xfdX\x89\xf3\xa8\xaa' >>> decoded_msg, decoded_msgecc, errata_pos = rsc.decode(tampered_msg) >>> print(decoded_msg) # decoded/corrected message bytearray(b'hello world') >>> print(decoded_msgecc) # decoded/corrected message and ecc symbols bytearray(b'hello world\xed%T\xc4\xfd\xfd\x89\xf3\xa8\xaa') >>> print(errata_pos) # errata_pos is returned as a bytearray, hardly intelligible bytearray(b'\x10\t\x02') >>> print(list(errata_pos)) # convert to a list to get the errata positions as integer indices [16, 9, 2] ``` -------------------------------- ### Bit List to Integer Conversion Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/TODO.txt Converts a list of bits into an integer using bit shifting and OR operations. This function is useful for packing bit sequences into a single integer representation. ```python def shifting(bitlist): out = 0 for bit in bitlist: out = (out << 1) | bit return out ``` -------------------------------- ### Decode Data with RSCodec Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Decode (repair) data using the RSCodec. The decode method returns the corrected message, the corrected message with ECC symbols, and a list of error positions. It can handle a certain number of errors or erasures. ```python # Decoding (repairing) >>> rsc.decode(b'hello world\xed%T\xc4\xfd\xfd\x89\xf3\xa8\xaa')[0] # original b'hello world' ``` ```python >>> rsc.decode(b'heXlo worXd\xed%T\xc4\xfdX\x89\xf3\xa8\xaa')[0] # 3 errors b'hello world' ``` ```python >>> rsc.decode(b'hXXXo worXd\xed%T\xc4\xfdX\x89\xf3\xa8\xaa')[0] # 5 errors b'hello world' ``` ```python >>> rsc.decode(b'hXXXo worXd\xed%T\xc4\xfdXX\xf3\xa8\xaa')[0] # 6 errors - fail Traceback (most recent call last): ... reedsolo.ReedSolomonError: Too many (or few) errors found by Chien Search for the errata locator polynomial! ``` -------------------------------- ### RSCodec.check() Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Verifies the integrity of an encoded message. ```APIDOC ## RSCodec.check() ### Description Checks if a message with its ECC symbols is valid. ### Parameters - **data** (bytes/bytearray) - Required - The encoded data to verify. ### Response - **result** (list) - A list of boolean values, one per chunk, indicating if the chunk is valid. ``` -------------------------------- ### Low-Level API: rs_check() - Verify Single Message Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Checks if a single encoded message (message + ECC) is valid by computing the syndrome polynomial. ```APIDOC ## rs.rs_check() ### Description Checks if a single encoded message (message + ECC) is valid by computing the syndrome polynomial. ### Method `rs_check(encoded_message, nsym)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **encoded_message** (bytearray) - The encoded message to check. - **nsym** (int) - The number of ECC symbols used during encoding. ### Request Example ```python # Initialize tables rs.init_tables(0x11d) nsym = 12 message = bytearray(b"hello world") encoded = rs.rs_encode_msg(message, nsym) # Check uncorrupted message is_valid = rs.rs_check(encoded, nsym) print(f"Valid: {is_valid}") # Check corrupted message encoded[5] = 0 is_valid = rs.rs_check(encoded, nsym) print(f"Valid after corruption: {is_valid}") ``` ### Response #### Success Response (200) - **is_valid** (bool) - True if the message is valid and uncorrupted, False otherwise. #### Response Example ```json { "example": "True" } ``` ```json { "example": "False" } ``` ``` -------------------------------- ### Manually verify message integrity Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Checks if the repaired message is correct by validating the syndrome values. ```python >> rs.rs_check(rmes + recc, nsym) ``` -------------------------------- ### Verify Single Message with rs_check() Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Validates an encoded message by computing its syndrome polynomial. ```python import reedsolo as rs # Initialize tables rs.init_tables(0x11d) nsym = 12 message = bytearray(b"hello world") encoded = rs.rs_encode_msg(message, nsym) # Check uncorrupted message is_valid = rs.rs_check(encoded, nsym) print(f"Valid: {is_valid}") # True # Check corrupted message encoded[5] = 0 is_valid = rs.rs_check(encoded, nsym) print(f"Valid after corruption: {is_valid}") # False ``` -------------------------------- ### Verify Message Integrity Source: https://github.com/tomerfiliba-org/reedsolomon/blob/master/README.rst Use the check() method to verify if a message has been tampered with without performing a full decoding process. ```python # Checking >> rsc.check(b'hello worXXXXy\xb2XX\x01q\xb9\xe3\xe2=') # Tampered message will return False [False] >> rmes, rmesecc, errata_pos = rsc.decode(b'hello worXXXXy\xb2XX\x01q\xb9\xe3\xe2=') >> rsc.check(rmesecc) # Corrected or untampered message will return True [True] >> print('Number of detected errors and erasures: %i, their positions: %s' % (len(errata_pos), list(errata_pos))) Number of detected errors and erasures: 6, their positions: [16, 15, 12, 11, 10, 9] ``` -------------------------------- ### RSCodec.encode() Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Encodes data by appending Reed-Solomon error correction symbols, supporting automatic chunking. ```APIDOC ## RSCodec.encode() ### Description Encodes a message (bytes, bytearray, or list of integers) by appending Reed-Solomon error correction symbols. ### Parameters - **data** (bytes/bytearray/list) - Required - The input data to encode. - **nsym** (int) - Optional - Override the default number of ECC symbols (requires single_gen=False). ``` -------------------------------- ### RSCodec.decode() - Repair and Verify Message Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Decodes a potentially corrupted message and verifies its integrity. Returns the decoded message, the ECC symbols, and the positions of any corrected errors. ```APIDOC ## RSCodec.decode() ### Description Decodes a potentially corrupted message and verifies its integrity. Returns the decoded message, the ECC symbols, and the positions of any corrected errors. ### Method `decode(encoded_message)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **encoded_message** (bytearray) - The encoded message to decode and repair. ### Request Example ```python # Assuming 'tampered' is a bytearray representing a corrupted encoded message decoded_msg, decoded_msgecc, errata_pos = rsc.decode(tampered) print(decoded_msg) print(errata_pos) ``` ### Response #### Success Response (200) - **decoded_msg** (bytearray) - The corrected message. - **decoded_msgecc** (bytearray) - The corrected ECC symbols. - **errata_pos** (list[int]) - A list of indices where errors were found and corrected. #### Response Example ```json { "example": "(b'original message', b'corrected ecc', [5, 10])" } ``` ``` -------------------------------- ### Encode Single Message with rs_encode_msg() Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Encodes a message using low-level functions. Note that this does not support automatic chunking for messages exceeding field bounds. ```python import reedsolo as rs # Initialize tables rs.init_tables(0x11d) # Define parameters n = 255 # Total length (message + ECC) nsym = 12 # Number of ECC symbols message = b"a" * (n - nsym) # Max message length for this config # Precompute generator polynomial for efficiency gen = rs.rs_generator_poly_all(n) # Encode the message encoded = rs.rs_encode_msg(message, nsym, gen=gen[nsym]) print(f"Encoded length: {len(encoded)}") # 255 # Encode with custom parameters encoded_custom = rs.rs_encode_msg( message, nsym, fcr=0, # First consecutive root generator=2 # Generator polynomial base ) ``` -------------------------------- ### Decode and Repair Data Source: https://context7.com/tomerfiliba-org/reedsolomon/llms.txt Repair corrupted data within the codec's capacity or use known erasure positions to double correction power. Handle ReedSolomonError when corruption exceeds capacity. ```python from reedsolo import RSCodec, ReedSolomonError rsc = RSCodec(10) # Encode original message original = b'hello world' encoded = rsc.encode(original) # Decode uncorrupted message decoded_msg, decoded_msgecc, errata_pos = rsc.decode(encoded) print(decoded_msg) # b'hello world' # Decode message with 3 errors (within correction capacity) tampered = bytearray(encoded) tampered[0] = ord('X') # Introduce errors tampered[6] = ord('X') tampered[15] = ord('Z') decoded_msg, decoded_msgecc, errata_pos = rsc.decode(tampered) print(decoded_msg) # b'hello world' print(list(errata_pos)) # Positions of corrected errors # Decode with known erasure positions (doubles correction power) rsc12 = RSCodec(12) encoded12 = rsc12.encode(b'hello world') tampered12 = bytearray(encoded12) # Corrupt 12 positions (known erasures) erasure_positions = [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16] for pos in erasure_positions: tampered12[pos] = ord('X') decoded_msg, _, _ = rsc12.decode(tampered12, erase_pos=erasure_positions) print(decoded_msg) # b'hello world' # Handle decoding failures rsc_small = RSCodec(4) # Can only correct 2 errors try: encoded_small = rsc_small.encode(b'test') tampered_small = bytearray(encoded_small) tampered_small[0] = 0 tampered_small[1] = 0 tampered_small[2] = 0 # 3 errors, exceeds capacity rsc_small.decode(tampered_small) except ReedSolomonError as e: print(f"Decoding failed: {e}") ```