### Install Tox Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Install the tox testing tool if it is not already installed. ```bash python3 -m pip install tox ``` -------------------------------- ### Install Hachoir core library Source: https://github.com/vstinner/hachoir/blob/main/doc/install.md Standard installation command for the Hachoir package. ```bash python3 -m pip install -U hachoir ``` -------------------------------- ### Install Hachoir Source: https://context7.com/vstinner/hachoir/llms.txt Commands to install the Hachoir library via pip, including optional dependencies for terminal and GUI support. ```bash # Basic installation python3 -m pip install -U hachoir # With urwid terminal UI support python3 -m pip install -U hachoir[urwid] # With wxPython GUI support python3 -m pip install -U hachoir[wx] ``` -------------------------------- ### Find Files in a Hard Drive with hachoir-subfile Source: https://github.com/vstinner/hachoir/blob/main/doc/subfile.md This example demonstrates searching for files on a block device (e.g., /dev/sda) with a specified size limit and quiet output. It shows the detected file types and their offsets. ```bash $ hachoir-subfile /dev/sda --size=34200100 --quiet [+] Start search (32.6 MB) [+] Found file at 0: MS-DOS hard drive with Master Boot Record (MBR) [+] Found file at 32256: FAT16 filesystem [+] Found file at 346112 size=308280 (301.1 KB): Microsoft Bitmap version 3 [+] Found file at 32157696: MS-DOS executable [+] Found file at 32483328: MS-DOS executable [+] Found file at 32800768: MS-DOS executable [+] Found file at 32851968: MS-DOS executable [+] Found file at 32872448: MS-DOS executable [+] Found file at 33058816: MS-DOS executable [+] Found file at 33112064: MS-DOS executable [+] Found file at 33142784: MS-DOS executable [+] Found file at 33949936: Microsoft Windows Portable Executable: Intel 80386 or greater [+] Search done -- offset=34200100 (32.6 MB) Total time: 20.08 sec -- 1.6 MB/sec ``` -------------------------------- ### Install Hachoir optional dependencies Source: https://github.com/vstinner/hachoir/blob/main/doc/install.md Additional packages required for specific Hachoir sub-modules. ```bash python3 -m pip install -U urwid ``` ```bash python3 -m pip install -U wxPython ``` -------------------------------- ### Compute Min/Max Length of Regex (Example 2) Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Reiterates computing the minimum and maximum length of a regex pattern, showing the result for '(cat|horse)'. ```python >>> r=parse('(cat|horse)') >>> r.minLength(), r.maxLength() (3, 5) ``` -------------------------------- ### Run tests with tox Source: https://github.com/vstinner/hachoir/blob/main/doc/hacking.md Executes the test suite using the tox automation tool. Requires tox to be installed via pip. ```default tox ``` -------------------------------- ### Define a Parser with Nested Field Sets Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Use FieldSet to create nested structures within your parser. This example defines an Entry field set and a MyFormat parser that includes strings, unsigned integers, and dynamically generated Entry fields based on a count. ```python >>> from hachoir.field import FieldSet, UInt8, Character, String >>> class Entry(FieldSet): ... def createFields(self): ... yield Character(self, "letter") ... yield UInt8(self, "code") ... >>> class MyFormat(Parser): ... endian = LITTLE_ENDIAN ... def createFields(self): ... yield String(self, "signature", 3, charset="ASCII") ... yield UInt8(self, "count") ... for index in range(self["count"].value): ... yield Entry(self, "point[]") ... >>> data = b"MYF\3a\0b\2c\0" >>> stream = StringInputStream(data) >>> root = MyFormat(stream) ``` -------------------------------- ### Analyze PowerPoint Document with hachoir-subfile Source: https://github.com/vstinner/hachoir/blob/main/doc/subfile.md This example shows how hachoir-subfile can analyze a PowerPoint document (chiens.PPS) and list the embedded image files (JPEG and PNG) along with their sizes and types. ```bash $ hachoir-subfile chiens.PPS [+] Start search (828.5 KB) [+] Found file at 0: Microsoft Office document [+] Found file at 537 size=28449 (27.8 KB): JPEG picture: 433x300 pixels [+] Found file at 29011 size=34761 (33.9 KB): JPEG picture: 433x300 pixels [+] Found file at 63797 size=40326 (39.4 KB): JPEG picture: 433x300 pixels [+] Found file at 104148 size=30641 (29.9 KB): JPEG picture: 433x300 pixels [+] Found file at 134814 size=22782 (22.2 KB): JPEG picture: 384x325 pixels [+] Found file at 157621 size=24744 (24.2 KB): JPEG picture: 443x313 pixels [+] Found file at 182390 size=27241 (26.6 KB): JPEG picture: 443x290 pixels [+] Found file at 209656 size=27407 (26.8 KB): JPEG picture: 443x336 pixels [+] Found file at 237088 size=30088 (29.4 KB): JPEG picture: 388x336 pixels [+] Found file at 267201 size=30239 (29.5 KB): JPEG picture: 366x336 pixels [+] Found file at 297465 size=81634 (79.7 KB): JPEG picture: 630x472 pixels [+] Found file at 379124 size=36142 (35.3 KB): JPEG picture: 599x432 pixels [+] Found file at 415291 size=28801 (28.1 KB): JPEG picture: 443x303 pixels [+] Found file at 444117 size=28283 (27.6 KB): JPEG picture: 433x300 pixels [+] Found file at 472425 size=95913 (93.7 KB): PNG picture: 433x431x8 [+] Found file at 568363 size=219252 (214.1 KB): PNG picture: 532x390x8 [+] Found file at 811308 size=20644 (20.2 KB): Microsoft Windows Metafile (WMF) picture [+] Search done -- offset=848384 (828.5 KB) Total time: 1.30 sec -- 635.1 KB/sec ``` -------------------------------- ### Hachoir Stream Manipulation Example Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Demonstrates basic stream operations using StringInputStream, including reading bytes and bits with specified endianness. Note that stream sizes and addresses are in bits, not bytes. ```python data = b"point\0\3\0\2\0" from hachoir.stream import StringInputStream, LITTLE_ENDIAN stream = StringInputStream(data) print(stream.source) print(len(data), stream.size) print(data[1:6], stream.readBytes(8, 5)) print(data[6:8], stream.readBits(6*8, 16, LITTLE_ENDIAN)) print(data[8:10], stream.readBits(8*8, 16, LITTLE_ENDIAN)) ``` -------------------------------- ### Search for Patterns Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Searches a given string for all added patterns using the search() method and prints the start, end, and matched item. ```python >>> for start, end, item in p.search("a b c d"): ... print("%s..%s: %s" % (start, end, item)) ... 0..1: a 2..3: b 4..5: [cd] 6..7: [cd] ``` -------------------------------- ### Non-optimized regex patterns Source: https://github.com/vstinner/hachoir/blob/main/tests/regex_regression.rst Examples of patterns that the parser does not attempt to optimize or factorise. ```python >>> parse('(a*|b){3,}') >>> parse("(a{2,3}|b){3,}") >>> parse("(a{2,3}|b)*") >>> parse("(a{2,3}|b)+") >>> parse("(a+|b){4,5}") >>> parse("(a{2,}|b){4,5}") >>> parse("(a{2,3}|b){4,5}") ``` -------------------------------- ### Access Field from Root by Absolute Path Source: https://github.com/vstinner/hachoir/blob/main/doc/internals.md Access any field in the tree starting from the root by providing an absolute path. This is the most explicit way to locate a field. ```python field["/header/key"] ``` -------------------------------- ### Access Child Field by Path Source: https://github.com/vstinner/hachoir/blob/main/doc/internals.md Use this syntax to access a direct child of the current field node. No special setup is required beyond having a field object. ```python field["content"] ``` -------------------------------- ### List all strings in a binary file Source: https://github.com/vstinner/hachoir/blob/main/doc/grep.md Use the --all flag to output every string found within the specified binary file. ```default $ .hachoir-grep --all sheep_on_drugs.mp3 0:ID3 10:TAL 17:Bilmusik vol 1. Stainless Steel Providers (...) 143:COM 150:eng 154:Stainless Steel Provider is compilated to the car of Twinstar. ``` -------------------------------- ### PatternMatching Class Usage Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Demonstrates how to initialize PatternMatching, add string and regex patterns, and search for them within a text. It also shows how to access the matched pattern information. ```APIDOC ## PatternMatching Class Use `PatternMatching` to find multiple strings or regular expressions within a given string. ### Adding Patterns Use `addString()` to add literal strings and `addRegex()` to add regular expression patterns. ```python from hachoir.regex import PatternMatching p = PatternMatching() p.addString("a") p.addString("b") p.addRegex("[cd]") ``` ### Searching for Patterns Use the `search()` method to find all occurrences of the added patterns in a text. It returns an iterator yielding tuples of (start_index, end_index, pattern_object). ```python for start, end, item in p.search("a b c d"): print(f"{start}..{end}: {item}") ``` **Output:** ``` 0..1: a 2..3: b 4..5: [cd] 6..7: [cd] ``` ### Associating User Data You can associate a custom "user" value with each pattern. This value can be any Python object and is accessible via the `user` attribute of the returned pattern object. ```python p2 = PatternMatching() p2.addString("un", 1) p2.addString("deux", 2) p2.addRegex("(trois|three)", 3) for start, end, item in p2.search("un deux trois"): print(f"{item!r} at {start}: user={item.user!r}") ``` **Output:** ``` at 0: user=1 at 3: user=2 at 8: user=3 ``` **Note:** The `item` returned by `search()` is a `Pattern` object (either `StringPattern` or `RegexPattern`), not the matched string itself. The `user` attribute holds the associated data. ``` -------------------------------- ### Initialize Pattern Matching Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Initializes a PatternMatching object and adds strings and regex patterns to it. Use addString() and addRegex() for this. ```python >>> from hachoir.regex import PatternMatching >>> p = PatternMatching() >>> p.addString("a") >>> p.addString("b") >>> p.addRegex("[cd]") ``` -------------------------------- ### Create a Parser with createParser Source: https://context7.com/vstinner/hachoir/llms.txt Automatically detect file formats and iterate through the resulting field structure. ```python from hachoir.parser import createParser # Create a parser for any supported file parser = createParser("video.avi") if parser: print(f"Detected format: {parser.description}") print(f"MIME type: {parser.mime_type}") # Access fields in the parsed structure with parser: for field in parser: print(f"{field.address}) {field.name} = {field.display}") ``` -------------------------------- ### Define Static and Dynamic Parser Sizes Source: https://github.com/vstinner/hachoir/blob/main/doc/internals.md Illustrates using static_size for fixed-length parsers and setting _size in the constructor for dynamic-length parsers. ```pycon >>> from hachoir.field import Parser >>> class FourBytes(Parser): ... endian = BIG_ENDIAN ... static_size = 32 ... def createFields(self): ... yield Integer(self, "four", "uint32") ... >>> class DynamicSize(Parser): ... endian = BIG_ENDIAN ... def __init__(self, stream, nb_items): ... Parser.__init__(self, stream) ... assert 0 < nb_items ... self.nb_items = nb_items ... self._size = nb_items * 32 # 32 is the size of one item ... ... def createFields(self): ... for index in range(self.nb_items): ... yield Integer(self, "item[]", "uint32") ... >>> a = FourBytes(stream) >>> b = DynamicSize(stream, 1) >>> a.size, b.size (32, 32) >>> # Check that nothing is really read from stream ... a.current_length, b.current_length (0, 0) ``` -------------------------------- ### Compute Minimum and Maximum Length of Regex Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Demonstrates how to compute the minimum and maximum possible lengths of a matched pattern using minLength() and maxLength(). ```python >>> from hachoir.regex import parse >>> r=parse('(cat|horse)') >>> r.minLength(), r.maxLength() (3, 5) >>> r=parse('(a{2,}|b+)') >>> r.minLength(), r.maxLength() (1, None) ``` -------------------------------- ### Navigate to Hachoir Directory Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Change the current directory to the cloned Hachoir repository. ```bash cd hachoir ``` -------------------------------- ### Convert Regex to String and Representation Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Demonstrates converting regex objects to their string representation using str() and their detailed representation using repr(). ```python >>> from hachoir.regex import createRange, createString >>> str(createString('abc')) 'abc' >>> repr(createString('abc')) "" ``` -------------------------------- ### List file structure with hachoir-list Source: https://context7.com/vstinner/hachoir/llms.txt Display the decoded field structure of a binary file as text. ```bash $ hachoir-list audio.cda signature : "RIFF" filesize : 36 bytes type : "CDDA" cdda tag : "fmt " size : 24 bytes cda_version : 1 track_no : 4 disc_serial : 0008-5C48 # Options: $ hachoir-list file.bin --description # Show field descriptions $ hachoir-list file.bin --hide-value # Hide values $ hachoir-list file.bin --hide-size # Hide sizes $ hachoir-list file.bin --indent-width=4 # Custom indentation ``` -------------------------------- ### Define Custom Parsers Source: https://context7.com/vstinner/hachoir/llms.txt Create custom binary parsers by subclassing FieldSet or Parser and defining field structures. ```python from hachoir.field import Parser, FieldSet, CString, UInt8, UInt16, String from hachoir.stream import StringInputStream, LITTLE_ENDIAN class Entry(FieldSet): def createFields(self): yield Character(self, "letter") yield UInt8(self, "code") class Point(Parser): endian = LITTLE_ENDIAN def createFields(self): yield CString(self, "name", "Point name") yield UInt16(self, "x", "X coordinate") yield UInt16(self, "y", "Y coordinate") # Parse binary data data = b"point\0\x03\x00\x02\x00" stream = StringInputStream(data) point = Point(stream) # Access parsed fields for field in point: print(f"{field.address}) {field.name} = {field.display}") # Access specific field attributes x_field = point["x"] print(f"Path: {x_field.path}") # /x print(f"Value: {x_field.value}") # 3 print(f"Description: {x_field.description}") # X coordinate print(f"Address: {x_field.address}") # 48 (bits) print(f"Size: {x_field.size}") # 16 (bits) ``` -------------------------------- ### Explore binary files with hachoir-urwid Source: https://context7.com/vstinner/hachoir/llms.txt Interactive terminal-based binary file explorer using hachoir-urwid. ```bash # Open file in terminal UI $ hachoir-urwid binary_file.exe # Preload fields and focus on specific path $ hachoir-urwid archive.zip --preload=20 --path="/end_central_directory" # Force specific parser $ hachoir-urwid unknown_file --parser=jpeg # Key bindings: # up/down - Navigate fields # enter - Expand/collapse field set # h - Toggle human/raw display # v/d/s - Toggle value/description/size display # a - Toggle relative/absolute address # b - Toggle decimal/hex address # space - Parse embedded file # q - Quit ``` -------------------------------- ### Print All Decoded Fields Source: https://github.com/vstinner/hachoir/blob/main/doc/list.md Use this command to display all decoded fields of a binary file. The output format is designed for processing by text-oriented tools. ```bash $ hachoir-list cd_0008_5C48_1m53s.cda signature : "RIFF" filesize : 36 bytes type : "CDDA" cdda tag : "fmt " size : 24 bytes cda_version : 1 track_no : 4 disc_serial : 0008-5C48 hsg_offset : 19477 hsg_length : 8507 rb_offset frame : 52 second : 21 minute : 4 notused : "\0" rb_length frame : 32 second : 53 minute : 1 notused : "\0" ``` -------------------------------- ### Work with Input Streams Source: https://context7.com/vstinner/hachoir/llms.txt Manage binary data in-memory or from files using stream classes, supporting bit-level access and endianness. ```python from hachoir.stream import StringInputStream, FileInputStream, LITTLE_ENDIAN # Create stream from bytes data = b"point\0\x03\x00\x02\x00" stream = StringInputStream(data) # Stream properties print(f"Source: {stream.source}") print(f"Size in bits: {stream.size}") # 80 bits (10 bytes * 8) # Read bytes (address in bits) content = stream.readBytes(8, 5) # Start at bit 8, read 5 bytes print(content) # b'oint\x00' # Read integer with endianness value = stream.readBits(6*8, 16, LITTLE_ENDIAN) # Read 16-bit int at byte 6 print(value) # 3 # Create stream from file file_stream = FileInputStream("document.pdf") print(f"File size: {file_stream.size // 8} bytes") ``` -------------------------------- ### Group Prefix Optimization Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Demonstrates the 'group prefix' optimization where common prefixes of OR-ed strings are grouped. ```python >>> createString("blue") | createString("brown") >>> createString("moto") | parse("mot.") >>> parse("(ma|mb|mc)") >>> parse("(maa|mbb|mcc)") ``` -------------------------------- ### Extract Metadata via CLI Source: https://context7.com/vstinner/hachoir/llms.txt Use hachoir-metadata to extract file information from the command line. ```bash # Basic metadata extraction $ hachoir-metadata video.avi Common: - Duration: 4 min 25 sec - Comment: Has audio/video index (248.9 KB) - MIME type: video/x-msvideo - Endian: Little endian Video stream: - Image width: 600 - Image height: 480 - Compression: DivX v4 (fourcc:"divx") Audio stream: - Channel: stereo - Sample rate: 22.1 KHz # Get MIME type only $ hachoir-metadata --mime image.png audio.mp3 image.png: image/png audio.mp3: audio/mpeg # Brief file type description $ hachoir-metadata --type image.png audio.mp3 image.png: PNG picture: 331x90x8 (alpha layer) audio.mp3: MPEG v1 layer III, 128.0 Kbit/sec, 44.1 KHz, Joint stereo # Filter by priority level (1-9, lower = more important) $ hachoir-metadata --level=3 photo.jpg Image: - Image width: 2048 - Image height: 1536 - Bits/pixel: 24 ``` -------------------------------- ### Search Binary Files via CLI Source: https://context7.com/vstinner/hachoir/llms.txt Use hachoir-grep to search for text patterns within binary files. ```bash # List all strings in a file $ hachoir-grep --all music.mp3 0:ID3 10:TAL 17:Album Title 143:COM 154:Comment text here # Search for specific substring $ hachoir-grep "album" music.mp3 17:Album Title # Case-sensitive search with path display $ hachoir-grep --path --case "Title" music.mp3 78:/id3v2/field[2]/content/text:Album Title ``` -------------------------------- ### Logging Configuration Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Demonstrates how to change the verbosity of Hachoir logging. ```APIDOC ## Log ### Change Hachoir verbosity To make Hachoir quiet, set the `quiet` attribute to `True`. ```python from hachoir.core import config config.quiet = True ``` ``` -------------------------------- ### Factorise (a|b){x,} patterns Source: https://github.com/vstinner/hachoir/blob/main/tests/regex_regression.rst Shows how the parser handles factorisation for range-quantified alternation groups with no upper bound. ```python >>> parse("(a+|b){3,}") >>> parse("(a{2,}|b){3,}") ``` -------------------------------- ### Access Fields to Trigger Creation Source: https://github.com/vstinner/hachoir/blob/main/doc/internals.md Shows how accessing fields in a parser triggers the creation of the field list. ```pycon >>> from hachoir.stream import StringInputStream >>> stream = StringInputStream(b"\x2A\x00\x04\x05") >>> p = Point(stream) >>> p.current_length 1 ``` ```pycon >>> x = p["x"].value >>> p.current_length 3 ``` -------------------------------- ### Factorise (a|b)+ patterns Source: https://github.com/vstinner/hachoir/blob/main/tests/regex_regression.rst Shows how the parser handles factorisation for plus-quantified alternation groups. ```python >>> parse("(a*|b)+") >>> parse("(a+|b|)+") >>> parse("(a+|b)+") >>> parse("(a{5,}|b)+") ``` -------------------------------- ### Factorise (a|b){x,y} patterns Source: https://github.com/vstinner/hachoir/blob/main/tests/regex_regression.rst Shows how the parser handles factorisation for range-quantified alternation groups with defined bounds. ```python >>> parse("(a*|b|){4,5}") >>> parse("(a+|b|){4,5}") >>> parse("(a*|b){4,5}") ``` -------------------------------- ### Factorise (a|b)* patterns Source: https://github.com/vstinner/hachoir/blob/main/tests/regex_regression.rst Shows how the parser handles factorisation for star-quantified alternation groups. ```python >>> parse("(a*|b)*") >>> parse("(a+|b)*") >>> parse("(a{2,}|b)*") ``` -------------------------------- ### Search for a substring in a binary file Source: https://github.com/vstinner/hachoir/blob/main/doc/grep.md Provide a search string as the first argument to find matching occurrences in the file. ```default $ hachoir-grep "il" sheep_on_drugs.mp3 17:Bilmusik vol 1. Stainless Steel Providers 154:Stainless Steel Provider is compilated to the car of Twinstar. ``` -------------------------------- ### Attach User Data to Patterns Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Shows how to attach arbitrary user data to patterns using the 'user' argument when adding them to PatternMatching. This data is accessible via the 'item.user' attribute. ```python >>> p = PatternMatching() >>> p.addString("un", 1) >>> p.addString("deux", 2) >>> for start, end, item in p.search("un deux"): ... print("%r at %s: user=%r" % (item, start, item.user)) ... at 0: user=1 at 3: user=2 ``` -------------------------------- ### Define and Use a Basic Hachoir Parser Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Defines a custom Hachoir parser 'Point' with CString and UInt16 fields, then instantiates it with a stream to parse data. Fields are accessed by name, and their address, value, and display attributes can be inspected. ```python from hachoir.field import Parser, CString, UInt16 from hachoir.stream import LITTLE_ENDIAN class Point(Parser): endian = LITTLE_ENDIAN def createFields(self): yield CString(self, "name", "Point name") yield UInt16(self, "x", "X coordinate") yield UInt16(self, "y", "Y coordinate") # Assuming 'stream' is an initialized Hachoir stream object # For example: stream = StringInputStream(b"point\0\3\0\2\0") # point = Point(stream) # for field in point: # print("%s) %s=%s" % (field.address, field.name, field.display)) ``` -------------------------------- ### Regex OR and AND operations Source: https://github.com/vstinner/hachoir/blob/main/tests/regex_regression.rst Demonstrates parsing of complex alternation and concatenation patterns. ```python >>> from hachoir.regex import parse >>> parse("(M(SCF|Thd)|B(MP4|Zh))") >>> parse("(FWS1|CWS1|FWS2|CWS2)") >>> parse("(abcdeZ|abZ)") >>> parse("(00t003|10t003|00[12]0[1-9].abc\0|1CD001)") ``` -------------------------------- ### Create Empty String Regex Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Creates an empty regex using createString(), which results in a RegexEmpty object. ```python >>> from hachoir.regex import createString >>> createString('') ``` -------------------------------- ### Associate user data with patterns Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Assign custom user values to patterns during registration to retrieve them alongside match results. ```python >>> p2 = PatternMatching() >>> p2.addString("un", 1) >>> p2.addString("deux", 2) >>> p2.addRegex("(trois|three)", 3) >>> for start, end, item in p2.search("un deux trois"): ... print("%r at %s: user=%r" % (item, start, item.user)) ... at 0: user=1 at 3: user=2 at 8: user=3 ``` -------------------------------- ### Search Images, Videos, and SWF Files with hachoir-subfile Source: https://github.com/vstinner/hachoir/blob/main/doc/subfile.md This command searches for multiple categories of files (images, videos) and a specific parser (SWF) within an input file. ```bash hachoir-subfile input --category=image,video --parser=swf ``` -------------------------------- ### Create Simple String Regex Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Creates a regex object from a simple string literal using createString(). ```python >>> from hachoir.regex import createString >>> createString('abc') ``` -------------------------------- ### Search with case sensitivity and path display Source: https://github.com/vstinner/hachoir/blob/main/doc/grep.md Use --path to show the internal Hachoir field path and --case to enable case-sensitive matching. ```default $ hachoir-grep --path --case Car sheep_on_drugs.mp3 78:/id3v2/field[2]/content/text:Car music ``` -------------------------------- ### Display File Metadata with hachoir-metadata Source: https://github.com/vstinner/hachoir/blob/main/doc/strip.md Use the hachoir-metadata command to view the metadata of a file before stripping. This helps identify information that can be removed. ```bash $ hachoir-metadata KDE_Click.wav.new Common: - Creation date: 2001-02-21 <== here they are - Producer: Sound Forge 4.5 <== spy informations :-) - MIME type: audio/x-wav - Endian: Little endian Audio: - Duration: 39 ms ... ``` -------------------------------- ### Configure Parser Endianness Source: https://github.com/vstinner/hachoir/blob/main/doc/internals.md Demonstrates setting the endianness for a Parser class using the endian attribute. ```pycon >>> from hachoir.core.endian import LITTLE_ENDIAN >>> class UseLittleEndian(Parser): ... endian = LITTLE_ENDIAN ... ``` -------------------------------- ### Create Combined Strings Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Demonstrates combining two strings to form a new regex string. Use the '+' operator for concatenation. ```python >>> from hachoir.regex import createString >>> createString("bike") + createString("motor") ``` -------------------------------- ### Search Images with hachoir-subfile Source: https://github.com/vstinner/hachoir/blob/main/doc/subfile.md Use this command to search for files categorized as images. Specify the input file and the 'image' category. ```bash hachoir-subfile input --category=image ``` -------------------------------- ### Pattern Matching with Regex Source: https://context7.com/vstinner/hachoir/llms.txt Search for multiple patterns in binary data and manipulate regex objects. ```python from hachoir.regex import PatternMatching, parse, createString, createRange # Create pattern matcher p = PatternMatching() p.addString("RIFF") p.addString("PNG") p.addRegex("[A-Z]{4}") # Search in data data = "header RIFF content PNG footer" for start, end, item in p.search(data): print(f"{start}..{end}: {item}") # Attach user data to patterns p2 = PatternMatching() p2.addString("error", {"level": "high"}) p2.addString("warning", {"level": "medium"}) p2.addRegex("info|debug", {"level": "low"}) for start, end, item in p2.search("error: warning: info"): print(f"Found {item} at {start}, data: {item.user}") # Regex manipulation regex = createString("bike") | createString("motor") print(regex) # regex = parse('(foo|fooo|foot|football)') print(regex) # # Character ranges regex = createString("1") | createString("3") regex |= createRange("2", "4") print(regex) # # Min/max length computation r = parse('(cat|horse)') print(f"Min: {r.minLength()}, Max: {r.maxLength()}") # Min: 3, Max: 5 ``` -------------------------------- ### Define a Parser with Conditional Fields Source: https://github.com/vstinner/hachoir/blob/main/doc/internals.md Demonstrates creating a custom Parser class where fields are generated dynamically based on previous field values. ```pycon >>> from hachoir.field import Parser, Int8 >>> from hachoir.core.endian import BIG_ENDIAN >>> class Point(Parser): ... endian = BIG_ENDIAN ... def __init__(self, stream): ... Parser.__init__(self, stream) ... if self["color"].value == -1: ... self._description += " (no color)" ... ... def createFields(self): ... yield Int8(self, "color", "Point color (-1 for none)") ... yield Int8(self, "use_3d", "Does it use Z axis?") ... yield Int8(self, "x", "X axis value") ... yield Int8(self, "y", "Y axis value") ... if self["use_3d"] == 1: ... yield Int8(self, "z", "Z axis value") ... ``` -------------------------------- ### Build Nested Field Sets Source: https://context7.com/vstinner/hachoir/llms.txt Define hierarchical parsers by subclassing FieldSet and Parser to handle complex binary structures. ```python from hachoir.field import Parser, FieldSet, String, UInt8, Character from hachoir.stream import StringInputStream, LITTLE_ENDIAN class Entry(FieldSet): def createFields(self): yield Character(self, "letter") yield UInt8(self, "code") class MyFormat(Parser): endian = LITTLE_ENDIAN def createFields(self): yield String(self, "signature", 3, charset="ASCII") yield UInt8(self, "count") for index in range(self["count"].value): yield Entry(self, "point[]") # Auto-numbered as point[0], point[1], etc. data = b"MYF\x03a\x00b\x02c\x00" stream = StringInputStream(data) root = MyFormat(stream) # Display tree structure def display_tree(parent, indent=0): for field in parent: print(" " * indent + field.path) if field.is_field_set: display_tree(field, indent + 1) display_tree(root) ``` -------------------------------- ### Configure Hachoir Verbosity Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Set the global configuration to quiet mode to suppress Hachoir output. ```default from hachoir.core import config config.quiet = True ``` -------------------------------- ### Create and Modify Character Range Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Illustrates creating a character range and then expanding it using the bitwise OR assignment operator '|='. ```python >>> from hachoir.regex import createString, createRange >>> regex = createString("1") | createString("3") >>> regex >>> regex |= createRange("2", "4") >>> regex ``` -------------------------------- ### Extract Metadata with extractMetadata Source: https://context7.com/vstinner/hachoir/llms.txt Extract and export metadata from multimedia files, supporting both plaintext and dictionary formats. ```python from hachoir.parser import createParser from hachoir.metadata import extractMetadata filename = "photo.jpg" parser = createParser(filename) if not parser: print("Unable to parse file") exit(1) with parser: try: metadata = extractMetadata(parser) except Exception as err: print(f"Metadata extraction error: {err}") metadata = None if metadata: # Print all metadata as plaintext for line in metadata.exportPlaintext(): print(line) # Access specific metadata values width = metadata.get('width', default=0) height = metadata.get('height', default=0) print(f"Dimensions: {width}x{height}") # Export as dictionary meta_dict = metadata.exportDictionary() print(meta_dict) ``` -------------------------------- ### Search All Subfiles and Store Them with hachoir-subfile Source: https://github.com/vstinner/hachoir/blob/main/doc/subfile.md This command searches for all detectable subfiles in the input and stores them in the specified output directory. It does not require specific parser or category flags. ```bash hachoir-subfile input /tmp/subfiles/ ``` -------------------------------- ### Verify Stripped File Metadata Source: https://github.com/vstinner/hachoir/blob/main/doc/strip.md After using hachoir-strip, use hachoir-metadata again on the .new file to confirm that the targeted metadata has been removed. ```bash $ hachoir-metadata KDE_Click.wav.new Common: - MIME type: audio/x-wav - Endian: Little endian Audio: - Duration: 39 ms ... ``` -------------------------------- ### Configure Hachoir logging Source: https://context7.com/vstinner/hachoir/llms.txt Adjust library verbosity using the config module or standard logging. ```python from hachoir.core import config # Disable all output config.quiet = True # Alternative: configure logging level import logging logging.getLogger('hachoir').setLevel(logging.ERROR) ``` -------------------------------- ### Regex with Repetitions Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Shows how to parse regex patterns involving repetitions like {n,m}, *, etc. The library optimizes these into a single RegexRepeat object. ```python >>> from hachoir.regex import parse >>> parse("(a{2,}){3,4}") >>> parse("(a*|b)*") >>> parse("(a*|b|){4,5}") ``` -------------------------------- ### Filter metadata output with --level Source: https://github.com/vstinner/hachoir/blob/main/doc/metadata.md Use the --level option to control the verbosity of the metadata output. Lower levels provide more concise information. ```default $ hachoir-metadata logo-Kubuntu.png Image: - Image width: 331 - Image height: 90 - Bits/pixel: 8 - Image format: Color index - Creation date: 2006-05-26 09:41:46 - Compression: deflate - MIME type: image/png - Endian: Big endian ``` ```default $ hachoir-metadata --level=7 logo-Kubuntu.png Image: - Image width: 331 - Image height: 90 - Bits/pixel: 8 - Image format: Color index - Creation date: 2006-05-26 09:41:46 - Compression: deflate ``` ```default $ hachoir-metadata --level=3 logo-Kubuntu.png Image: - Image width: 331 - Image height: 90 - Bits/pixel: 8 - Image format: Color index ``` -------------------------------- ### Run tests manually Source: https://github.com/vstinner/hachoir/blob/main/doc/hacking.md Executes the test suite directly using the runtests.py script. ```python python3 runtests.py ``` -------------------------------- ### Extract Metadata from File Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Use Hachoir to parse a file and extract its metadata. Ensure the filename is provided as a command-line argument. ```python from hachoir.parser import createParser from hachoir.metadata import extractMetadata from sys import argv, stderr, exit if len(argv) != 2: print("usage: %s filename" % argv[0], file=stderr) exit(1) filename = argv[1] parser = createParser(filename) if not parser: print("Unable to parse file", file=stderr) exit(1) with parser: try: metadata = extractMetadata(parser) except Exception as err: print("Metadata extraction error: %s" % err) metadata = None if not metadata: print("Unable to extract metadata") exit(1) for line in metadata.exportPlaintext(): print(line) ``` -------------------------------- ### Clone Hachoir Git Repository Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Clone the Hachoir Git repository to your local machine. ```bash git clone https://github.com/vstinner/hachoir ``` -------------------------------- ### Strip Metadata from a File with hachoir-strip Source: https://github.com/vstinner/hachoir/blob/main/doc/strip.md Execute hachoir-strip on a file to remove metadata. The program indicates which fields are removed and saves the modified file with a .new extension. ```bash $ hachoir-strip KDE_Click.wav [+] Process file KDE_Click.wav Remove field /info Remove 56 bytes (3.1%) Save new file into KDE_Click.wav.new ``` -------------------------------- ### Search for patterns in a string Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Execute the search method to find all registered patterns within a target string and iterate over the results. ```python >>> for start, end, item in p.search("a b c d"): ... print("%s..%s: %s" % (start, end, item)) ... 0..1: a 2..3: b 4..5: [cd] 6..7: [cd] ``` -------------------------------- ### Parse and factorise regex repeats Source: https://github.com/vstinner/hachoir/blob/main/tests/regex_regression.rst Demonstrates how the parse function simplifies nested repeat patterns. ```python >>> from hachoir.regex import parse >>> parse("(a{2,3}){4,5}") >>> parse("(a{2,}){3,4}") >>> parse("(a{2,3})+") >>> parse("(a*){2,3}") >>> parse("(a+){2,3}") ``` -------------------------------- ### Create Character Range Regex Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Creates a character range regex using createRange() with multiple characters specified. ```python >>> from hachoir.regex import createRange >>> createRange('a', 'b', 'c') ``` -------------------------------- ### Search JPEG Images with hachoir-subfile Source: https://github.com/vstinner/hachoir/blob/main/doc/subfile.md Use this command to search for JPEG images within a binary input file. Specify the input file and the 'jpeg' parser. ```bash hachoir-subfile input --parser=jpeg ``` -------------------------------- ### Merge Ranges Optimization Source: https://github.com/vstinner/hachoir/blob/main/doc/regex.md Illustrates the 'merge ranges' optimization where adjacent or overlapping character ranges are combined into a single range. ```python >>> from hachoir.regex import createRange >>> regex = createString("1") | createString("3"); regex >>> regex = regex | createRange("2"); regex >>> regex = regex | createString("0"); regex >>> regex = regex | createRange("5", "6"); regex >>> regex = regex | createRange("4"); regex ``` -------------------------------- ### Remove metadata with hachoir-strip Source: https://context7.com/vstinner/hachoir/llms.txt Clean files by removing metadata, indexes, or padding using hachoir-strip. ```bash # Remove all metadata $ hachoir-strip recording.wav [+] Process file recording.wav Remove field /info Remove 56 bytes (3.1%) Save new file into recording.wav.new # Verify metadata was removed $ hachoir-metadata recording.wav.new Common: - MIME type: audio/x-wav - Endian: Little endian Audio: - Duration: 39 ms # Strip specific types $ hachoir-strip photo.jpg --strip=metadata # Remove EXIF/IPTC $ hachoir-strip video.avi --strip=index # Remove video index $ hachoir-strip file.bin --strip=useless # Remove padding only $ hachoir-strip file.bin --strip="useless,metadata" # Combine options ``` -------------------------------- ### FieldSet Class Overview Source: https://github.com/vstinner/hachoir/blob/main/doc/developer.md Details the attributes and methods of the Hachoir FieldSet class. ```APIDOC ## Field set class ### Read only attributes * **endian** (string) - BIG_ENDIAN or LITTLE_ENDIAN, the way the bits are written in input stream. * **stream** (InputStream) - The input stream. * **root** (FieldSet) - The root of all fields. * **eof** (bool) - End Of File: indicates if the end of the input stream has been reached. * **done** (bool) - Indicates if the parser is done. ### Read only and lazy attributes * **current_size** (long) - Current size in bits. * **current_length** (long) - Current number of children. ### Methods * **connectEvent(event, handler, local=True)**: Connects a handler to an event. * **raiseEvent(event, *args)**: Raises an event with given arguments. * **reset()**: Clears all caches but keeps its size if known. * **setUniqueFieldName()**: Replaces trailing '[]' in field names with unique identifiers (e.g., 'item[]' becomes 'item[0]'). * **seekBit(address, …)**: Creates a field to seek to a specified bit address, or returns None if already there. * **seekByte(address, …)**: Creates a field to seek to a specified byte address, or returns None if already there. * **replaceField(name, fields)**: Replaces a field with one or more new fields. * **writeFieldsIn(old, address, new)**: Helper function for replaceField(). * **getFieldByAddress(address, feed=True)**: Gets the field at the specified address. * **getFieldType()**: Gets the field type as a short string, potentially including charset information. ### Lazy methods * **array()**: Creates a FakeArray for easy field access by index. * **__len__()**: Returns the number of children in the field set. * **readFirstFields(number)**: Reads the first 'number' fields and returns the count of newly read fields. * **readMoreFields(number)**: Reads more fields and returns the count of newly read fields. * **__iter__()**: Iterates over the children fields. * **createFields()**: The main parser function; should not be called directly. ```