PyBufrKit

https://github.com/ywangd/pybufrkit
Admin
PyBufrKit is a pure Python toolkit for working with WMO BUFR (FM-94) messages, providing both...
Tokens:13,488
Snippets:133
Trust Score:9.2
License:MIT
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install PyBufrKit

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Install the PyBufrKit library using pip, conda, or from source.

```bash
pip install pybufrkit
```

```bash
conda install -c conda-forge pybufrkit
```

```bash
python setup.py install
```

--------------------------------

### Install PyBufrKit from source

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Install PyBufrKit directly from the source code. This method is useful for developers or when the latest changes are needed.

```bash
python setup.py install
```

--------------------------------

### Install PyBufrKit using pip

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Install the PyBufrKit package using pip. This is the recommended method for most users.

```bash
pip install pybufrkit
```

--------------------------------

### Query Examples

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/index.md

Examples of valid query expressions for template data. These demonstrate various ways to select descriptors, subsets, and associated fields.

```text
008042
```

```text
@[0] > 008042
```

```text
/008042
```

```text
/008042[0]
```

```text
303051/008042
```

```text
103000.031001
```

```text
021062.A21062
```

--------------------------------

### Install PyBufrKit using conda-forge

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Install the PyBufrKit package from the conda-forge channel. This is an alternative installation method.

```bash
conda install -c conda-forge pybufrkit
```

--------------------------------

### Basic PyBufrKit library usage

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/index.md

Example demonstrating fundamental usage of the PyBufrKit library in Python.

```Python
from pybufrkit.decoder import BUFRDecoder
from pybufrkit.encoder import BUFLEncoder

# Example usage would go here, but is not provided in the source text.
```

--------------------------------

### PyBufrKit Command-Line Interface Examples

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Utilize the `pybufrkit` command-line tool for decoding, encoding, and filtering BUFR files. Use subcommands like `decode` and `encode` with various flags for output format, error handling, and filtering.

```bash
# Decode to flat text
pybufrkit decode sample.bufr

# Decode to JSON
pybufrkit decode -j sample.bufr

# Decode to attributed/hierarchical text
pybufrkit decode -a sample.bufr

# Decode multi-message file, skip errors
pybufrkit decode -m --continue-on-error multi.bufr

# Decode only messages with data_category == 2
pybufrkit decode -m --filter '${%data_category} == 2' multi.bufr

# Encode from JSON back to BUFR
pybufrkit encode -j message.json output.bufr

# Pipe: decode then re-encode
pybufrkit decode sample.bufr | pybufrkit encode -
```

--------------------------------

### Filter Files Using Script

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use the 'script' command with a Python expression to filter files based on metadata. This example prints filenames with more than one subset.

```bash
pybufrkit script 'if ${%n_subsets} > 1: print(PBK_FILENAME)' /data/*.bufr
```

--------------------------------

### Filter BUFR files using script support

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Use the 'script' command to apply Python expressions for filtering BUFR files. This example finds files with more than one subset and prints their names.

```bash
pybufrkit script 'if ${%n_subsets} > 1: print(PBK_FILENAME)' DIRECTORY/*.bufr
```

--------------------------------

### Metadata Query Expression Syntax

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/internals.md

Defines the Extended Backus-Naur Form (EBNF) for constructing metadata query expressions. Queries start with '%' and can optionally specify a section index.

```ebnf
<query_expr> = '%'[<section_index>.]<parameter_name>
```

--------------------------------

### Query BUFR Message Metadata and Data

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use DataQuerent to extract specific metadata or data fields from a BUFR message. Metadata queries start with '%', while data queries use descriptor IDs or paths. QueryResult objects provide methods to access all values.

```python
n_subsets = querent.query(msg, '%n_subsets')       # returns scalar
edition   = querent.query(msg, '%edition')         # returns scalar

result = querent.query(msg, '005001')              # latitude
lats = result.all_values(flat=True)
# [[52.317], [48.133], ...]

result = querent.query(msg, '/301001/001002')      # station number as direct child
stations = result.all_values(flat=True)
```

--------------------------------

### Filter BUFR messages by data category

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Decode only those BUFR messages from a multi-message file that satisfy a specific filter expression. This example filters for messages where 'data_category' is equal to 2.

```bash
pybufrkit decode -m --filter '${%data_category} == 2' FILE
```

--------------------------------

### Show BUFR File Metadata

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use 'info' to quickly display metadata sections of a BUFR file without decoding the data. This is useful for a fast overview.

```bash
pybufrkit info sample.bufr
```

--------------------------------

### Query First Subset with Condition

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use '@[index]' to specify a subset and apply a query condition, such as checking if a descriptor value is greater than a certain number.

```bash
pybufrkit query '@[0] > 001002' sample.bufr
```

--------------------------------

### Unified BUFR query interface with BufrMessageQuerent.query()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

A convenience wrapper that dispatches queries to either MetadataQuerent or DataQuerent based on the query expression's prefix. This is the recommended single-entry-point API for querying BUFR messages.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.query import BufrMessageQuerent

decoder = Decoder()
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())

querent = BufrMessageQuerent()

```

--------------------------------

### BUFR Message Configuration File Format

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/internals.md

Defines the general structure of a BUFR message configuration file, including section metadata and parameter definitions. This format allows for flexible BUFR message parsing and generation.

```default
{
  "index": 0,  # zero-based section index

  "description": "Indicator section",

  "default": true,  # use this config if an edition-specific one is not available

  "optional": false, # whether this section is optional

  "end_of_message": false, # whether this is the last section

  "parameters": [  # a list of parameter configs
    {
      "name": "start_signature",  # parameter name

      "nbits": 32,  # number of bits

      "type": "bytes",  # parameter type determines how the value can be processed from the input bits

      "expected": "BUFR",  # expected value for this parameter (will be validated if not None)

      "as_property": false  # whether this parameter can be accessed from the parent message object
    },
    ...
  ]
}
```

--------------------------------

### Compile BUFR Template

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Compiles a BUFR template from a comma-separated list of descriptors.

```bash
pybufrkit lookup -l 020003
```

```bash
pybufrkit compile 309052,205060
```

--------------------------------

### Query BUFR metadata with MetadataQuerent.query()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Queries parameters from BUFR message header sections (0-3) using a %-prefixed expression. An optional section index prefix can disambiguate parameters with the same name across different sections.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.mdquery import MetadataExprParser, MetadataQuerent

decoder = Decoder()
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())

querent = MetadataQuerent(MetadataExprParser())

# Query common metadata fields
n_subsets = querent.query(msg, '%n_subsets')          # e.g. 128
edition   = querent.query(msg, '%edition')             # e.g. 4
category  = querent.query(msg, '%data_category')       # e.g. 0
centre    = querent.query(msg, '%originating_centre')  # e.g. 98 (ECMWF)

# Query unexpanded descriptor list
descriptors = querent.query(msg, '%unexpanded_descriptors')
# e.g. [309052, 205060]

# Disambiguate when same parameter name appears in multiple sections
# section_length exists in sections 1, 2, 3, 4; default returns section 1
sec1_len = querent.query(msg, '%section_length')       # section 1 length
sec3_len = querent.query(msg, '%3.section_length')     # section 3 length explicitly

print(f'Subsets: {n_subsets}, Edition: {edition}, Category: {category}')
```

--------------------------------

### DataQuerent.query()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Queries values from the decoded and wired BUFR template data using a path expression. Supports descendant search (`>`), direct child (`/`), attribute access (`.`), Python-style slice notation, and per-subset targeting (`@[N]`). Returns a `QueryResult` object.

```APIDOC
## `DataQuerent.query()` — Query BUFR data section values

Queries values from the decoded and wired BUFR template data using a path expression. Supports descendant search (`>`), direct child (`/`), attribute access (`.`), Python-style slice notation, and per-subset targeting (`@[N]`). Returns a `QueryResult` object.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.dataquery import NodePathParser, DataQuerent

decoder = Decoder()
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())

querent = DataQuerent(NodePathParser())

# All values of descriptor 001002 (WMO station number) across all subsets
result = querent.query(msg, '001002')
for i_subset in result.subset_indices():
    print(f'Subset {i_subset}: {result.get_values(i_subset, flat=True)}')
# Subset 0: [15]

# Root-level occurrences only (descriptor must be top-level in template)
result = querent.query(msg, '/001002')

# Direct child of sequence descriptor 301001
result = querent.query(msg, '/301001/001002')

# First subset only
result = querent.query(msg, '@[0] > 001002')

# Lat/lon from specific subsets using slice notation
lat_result = querent.query(msg, '@[::2] > 005001')   # every other subset, latitude
lon_result = querent.query(msg, '006001')              # all subsets, longitude

# Query associated field of descriptor 021062
result = querent.query(msg, '021062.A21062')

# Delayed replication factor of replication block 103000
result = querent.query(msg, '103000.031001')

# Flatten results for easy use
flat_lats = lat_result.all_values(flat=True)
# [[52.3, 48.1, ...], [51.9, ...]]  — list per subset
```
```

--------------------------------

### Query BUFR Metadata

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use the 'query' command to retrieve specific metadata fields from a BUFR file. Use format specifiers like %n_subsets or %edition.

```bash
pybufrkit query %n_subsets sample.bufr
```

```bash
pybufrkit query %edition sample.bufr
```

--------------------------------

### BufrMessageQuerent.query()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

A unified query interface that auto-dispatches to either MetadataQuerent or DataQuerent based on the query expression format. This is the preferred single-entry-point query API.

```APIDOC
## `BufrMessageQuerent.query()` — Unified metadata and data query interface

A convenience wrapper that auto-dispatches to either `MetadataQuerent` or `DataQuerent` based on whether the query expression starts with `%`. This is the preferred single-entry-point query API.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.query import BufrMessageQuerent

decoder = Decoder()
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())

querent = BufrMessageQuerent()

```
```

--------------------------------

### MetadataQuerent.query()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Queries parameters from BUFR message header sections (sections 0–3) using a `%`-prefixed expression. Returns a scalar value. An optional section index prefix resolves ambiguity when the same parameter name appears in multiple sections.

```APIDOC
## `MetadataQuerent.query()` — Query BUFR metadata sections

Queries parameters from BUFR message header sections (sections 0–3) using a `%`-prefixed expression. Returns a scalar value. An optional section index prefix (e.g., `%2.section_length`) resolves ambiguity when the same parameter name appears in multiple sections.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.mdquery import MetadataExprParser, MetadataQuerent

decoder = Decoder()
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())

querent = MetadataQuerent(MetadataExprParser())

# Query common metadata fields
n_subsets = querent.query(msg, '%n_subsets')          # e.g. 128
edition   = querent.query(msg, '%edition')             # e.g. 4
category  = querent.query(msg, '%data_category')       # e.g. 0
centre    = querent.query(msg, '%originating_centre')  # e.g. 98 (ECMWF)

# Query unexpanded descriptor list
descriptors = querent.query(msg, '%unexpanded_descriptors')
# e.g. [309052, 205060]

# Disambiguate when same parameter name appears in multiple sections
# section_length exists in sections 1, 2, 3, 4; default returns section 1
sec1_len = querent.query(msg, '%section_length')       # section 1 length
sec3_len = querent.query(msg, '%3.section_length')     # section 3 length explicitly

print(f'Subsets: {n_subsets}, Edition: {edition}, Category: {category}')
```
```

--------------------------------

### Query Direct Child of a Sequence

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Query for a descriptor that is a direct child within a sequence by specifying the path using '/'.

```bash
pybufrkit query /301001/001002 sample.bufr
```

--------------------------------

### Query BUFR Data Values

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use 'query' with a descriptor ID (e.g., 001002) to retrieve all instances of that data value across all subsets in the file.

```bash
pybufrkit query 001002 sample.bufr
```

--------------------------------

### Query root level descriptor

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Query for a descriptor at the root level of the BUFR Template. The path notation '/descriptor' is used.

```bash
pybufrkit query /001002 BUFR_FILE
```

--------------------------------

### Filter Files by BUFR Template ID

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/internals.md

Use this script to filter files based on the BUFR Template ID. Query expressions are embedded using `${...}`. `PBK_FILENAME` is an injected variable holding the current file's name. Ensure to use the function version of `print`.

```default
if 309052 in ${%unexpanded_descriptors}: print(PBK_FILENAME)
```

--------------------------------

### Subset a BUFR file

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Use the 'subset' command to extract specific subsets from a BUFR file and save them into a new file. Provide the indices of the desired subsets.

```bash
pybufrkit subset 0,3,6,9 BUFR_FILE
```

--------------------------------

### Query BUFR data with DataQuerent.query()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Queries values from decoded BUFR template data using a path expression. Supports descendant search (>), direct child (/), attribute access (.), slice notation, and per-subset targeting (@[N]). Returns a QueryResult object.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.dataquery import NodePathParser, DataQuerent

decoder = Decoder()
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())

querent = DataQuerent(NodePathParser())

# All values of descriptor 001002 (WMO station number) across all subsets
result = querent.query(msg, '001002')
for i_subset in result.subset_indices():
    print(f'Subset {i_subset}: {result.get_values(i_subset, flat=True)}')
# Subset 0: [15]

# Root-level occurrences only (descriptor must be top-level in template)
result = querent.query(msg, '/001002')

# Direct child of sequence descriptor 301001
result = querent.query(msg, '/301001/001002')

# First subset only
result = querent.query(msg, '@[0] > 001002')

# Lat/lon from specific subsets using slice notation
lat_result = querent.query(msg, '@[::2] > 005001')   # every other subset, latitude
lon_result = querent.query(msg, '006001')              # all subsets, longitude

# Query associated field of descriptor 021062
result = querent.query(msg, '021062.A21062')

# Delayed replication factor of replication block 103000
result = querent.query(msg, '103000.031001')

# Flatten results for easy use
flat_lats = lat_result.all_values(flat=True)
# [[52.3, 48.1, ...], [51.9, ...]]  — list per subset
```

--------------------------------

### Query all values for a specific descriptor

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Query all values associated with a specific descriptor, such as '001002', from the data section of a BUFR file.

```bash
pybufrkit query 001002 BUFR_FILE
```

--------------------------------

### Encode JSON to BUFR binary with Encoder.process()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Encodes a JSON object (previously rendered by FlatJsonRenderer) back into a binary BUFR message. Section lengths are calculated automatically. Can override master table version during encoding.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.encoder import Encoder
from pybufrkit.renderer import FlatJsonRenderer
import json

# Round-trip: decode → JSON → encode
decoder = Decoder()
encoder = Encoder()

with open('input.bufr', 'rb') as f:
    original = decoder.process(f.read())

# Get the JSON representation
json_data = FlatJsonRenderer().render(original)

# Encode back to BUFR
re_encoded = encoder.process(json_data)

with open('output.bufr', 'wb') as f:
    f.write(re_encoded.serialized_bytes)

# Encode from a JSON file (flat format)
with open('message.json', 'r') as f:
    json_string = f.read()

bufr_out = encoder.process(json_string)
with open('output.bufr', 'wb') as f:
    f.write(bufr_out.serialized_bytes)

# Override master table version during encoding
encoder_override = Encoder(master_table_version=25)
bufr_out = encoder_override.process(json_data)
with open('output_v25.bufr', 'wb') as f:
    f.write(bufr_out.serialized_bytes)
```

--------------------------------

### Encoder.process()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Encodes a JSON object (previously produced by FlatJsonRenderer) back into a binary BUFR message. Section lengths are automatically calculated.

```APIDOC
## `Encoder.process()` — Encode a JSON structure to BUFR binary

Takes a JSON object (or its string serialization) previously produced by `FlatJsonRenderer` and encodes it back to a binary BUFR message. Section lengths are automatically calculated. Returns a `BufrMessage` whose `serialized_bytes` attribute contains the binary output.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.encoder import Encoder
from pybufrkit.renderer import FlatJsonRenderer
import json

# Round-trip: decode → JSON → encode
decoder = Decoder()
encoder = Encoder()

with open('input.bufr', 'rb') as f:
    original = decoder.process(f.read())

# Get the JSON representation
json_data = FlatJsonRenderer().render(original)

# Encode back to BUFR
re_encoded = encoder.process(json_data)

with open('output.bufr', 'wb') as f:
    f.write(re_encoded.serialized_bytes)

# Encode from a JSON file (flat format)
with open('message.json', 'r') as f:
    json_string = f.read()

bufr_out = encoder.process(json_string)
with open('output.bufr', 'wb') as f:
    f.write(bufr_out.serialized_bytes)

# Override master table version during encoding
encoder_override = Encoder(master_table_version=25)
bufr_out = encoder_override.process(json_data)
with open('output_v25.bufr', 'wb') as f:
    f.write(bufr_out.serialized_bytes)
```
```

--------------------------------

### Run Python Scripts with Embedded BUFR Queries

Source: https://context7.com/ywangd/pybufrkit/llms.txt

ScriptRunner executes Python scripts with embedded BUFR query expressions in ${...} syntax. Results are injected as variables. Use PBK_FILENAME for the current message's path. Control nesting levels with '#$ data_values_nest_level = N'.

```python
from pybufrkit.decoder import Decoder, generate_bufr_message
from pybufrkit.script import ScriptRunner

decoder = Decoder()

# Filter script: print filename if message has multiple subsets
script = "if ${%n_subsets} > 1: print(PBK_FILENAME)"
runner = ScriptRunner(script)

with open('multi.bufr', 'rb') as f:
    data = f.read()

for msg in generate_bufr_message(decoder, data):
    runner.run(msg)
# /path/to/multi.bufr  (if condition is met)

# Print lat/lon values with default nesting level 1 (flat list)
runner2 = ScriptRunner("print(${005001}, ${006001})")
for msg in generate_bufr_message(decoder, data):
    runner2.run(msg)
# [52.317, 48.1] [13.2, 11.5]

# Control nesting level via magic pragma comment
# Level 0: scalar (first value only), 1: flat list (default),
# 2: list-per-subset, 4: fully hierarchical
script_nested = """
#$ data_values_nest_level = 2
print(${005001})
"""
runner3 = ScriptRunner(script_nested)
for msg in generate_bufr_message(decoder, data):
    runner3.run(msg)
# [[52.317], [48.1], ...]  — one sub-list per subset

# Use eval mode for boolean filter expressions
filter_runner = ScriptRunner('${%data_category} == 0', mode='eval')
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())
matched = filter_runner.run(msg)  # True or False
print('Surface data?', matched)

# Filter by template presence
tpl_filter = ScriptRunner('309052 in ${%unexpanded_descriptors}', mode='eval')
for msg in generate_bufr_message(decoder, data):
    if tpl_filter.run(msg):
        print('Uses template 309052:', msg.filename)
```

--------------------------------

### Query Associated Field

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Query for associated fields by specifying the descriptor ID followed by '.A' and the associated field ID.

```bash
pybufrkit query 021062.A21062 sample.bufr
```

--------------------------------

### Generate Multiple BUFR Messages from File

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Iterates through and decodes multiple BUFR messages from a single file. Requires a Decoder instance and the file's content.

```Python
from pybufrkit.decoder import generate_bufr_message
with open(SOME_FILE, 'rb') as ins:
    for bufr_message in generate_bufr_message(decoder, ins.read()):
        pass  # do something with the decoded message object
```

--------------------------------

### Decode only metadata sections of a BUFR file

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Use the 'info' command to extract and display only the metadata sections (0, 1, 2, 3) of a BUFR file, without decoding the actual data.

```bash
pybufrkit info BUFR_FILE
```

--------------------------------

### Query child descriptor within a parent

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Query for a descriptor that is a direct child of another descriptor. The path notation '/parent/child' is used.

```bash
pybufrkit query /301001/001002 BUFR_FILE
```

--------------------------------

### Query number of subsets in metadata

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Query the number of subsets present in the metadata sections (0, 1, 2, 3) of a BUFR file using the 'query' command. The '%n_subsets' special query string is used.

```bash
pybufrkit query %n_subsets BUFR_FILE
```

--------------------------------

### Decode a BUFR file

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/index.md

Opens a BUFR file in binary read mode and processes its content to decode a BUFR message.

```python
from pybufrkit.decoder import Decoder
decoder = Decoder()
with open(SOME_BUFR_FILE, 'rb') as ins:
    bufr_message = decoder.process(ins.read())
```

--------------------------------

### Render BufrMessage and QueryResult Objects

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use renderers to convert BufrMessage or QueryResult objects into text or JSON. Choose between flat (FlatTextRenderer, FlatJsonRenderer) or nested (NestedTextRenderer, NestedJsonRenderer) structures to preserve hierarchy.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.renderer import (
    FlatTextRenderer,
    FlatJsonRenderer,
    NestedTextRenderer,
    NestedJsonRenderer,
)
from pybufrkit.dataquery import NodePathParser, DataQuerent
import json

decoder = Decoder()
with open('sample.bufr', 'rb') as f:
    msg = decoder.process(f.read())

# Flat text — numbered lines with descriptor ID, name, and value
print(FlatTextRenderer().render(msg))
# <<<<<< section 0 >>>>>>
# edition = 4
# ...
# ###### subset 1 of 1 ######
#     1 001001 WMO block number                                       7
#     2 001002 WMO station number                                     15

# Flat JSON — round-trip ready; pass back to Encoder.process()
flat = FlatJsonRenderer().render(msg)
print(json.dumps(flat, indent=2))
# [[4, 0, ...], [...], [...], [...], [[7, 15, ...]], ['7777']]

# Nested text — preserves sequence/replication hierarchy
print(NestedTextRenderer().render(msg))
# 301001 WMO block and station numbers
#     001001 WMO block number  7
#     001002 WMO station number  15

# Nested JSON — full hierarchy with descriptor metadata
nested = NestedJsonRenderer().render(msg)
print(json.dumps(nested[4], indent=2))  # section 4 (data section)

# Render a QueryResult
querent = DataQuerent(NodePathParser())
result = querent.query(msg, '005001')
print(FlatTextRenderer().render(result))
# ###### subset 1 of 1 ######
# 52.317
```

--------------------------------

### Decode and Render BUFR File to JSON

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Decodes a BUFR file and converts the message into a flat JSON format. Ensure the BUFR file path is correctly specified.

```Python
from pybufrkit.decoder import Decoder
decoder = Decoder()
with open(SOME_BUFR_FILE, 'rb') as ins:
    bufr_message = decoder.process(ins.read())
```

```Python
from pybufrkit.renderer import FlatJsonRenderer
json_data = FlatJsonRenderer().render(bufr_message)
```

--------------------------------

### Split a BUFR file into individual messages

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Use the 'split' command to separate a BUFR file containing multiple messages into individual files, with each file containing a single message.

```bash
pybufrkit split BUFR_FILE
```

--------------------------------

### Lookup descriptor information

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/index.md

Look up information about a specific Element Descriptor, including its code table if applicable.

```Bash
pybufrkit lookup -l 020003
```

--------------------------------

### Update WMO BUFR Tables

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use 'update-wmo-tables' to update the bundled WMO BUFR tables to a specified version number.

```bash
pybufrkit update-wmo-tables 32
```

--------------------------------

### Subset BUFR File by Indices

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use 'subset' to extract specific data subsets from a BUFR file based on their indices. This allows for targeted data retrieval.

```bash
pybufrkit subset 0,3,6,9 sample.bufr
```

--------------------------------

### Decoder.process()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Decodes a raw bytes object containing a BUFR message into a BufrMessage object. The message is automatically wired into a hierarchical structure. Supports optional info-only mode, ignoring value expectations, and skipping the wiring step.

```APIDOC
## Decoder.process()

### Description
Decodes a raw bytes object containing a BUFR message into a `BufrMessage` object. The message is automatically wired into a hierarchical structure. Supports optional info-only mode (metadata sections only), ignoring value expectations, and skipping the wiring step.

### Method
```python
decoder.process(data: bytes, info_only: bool = False, ...) -> BufrMessage
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **data** (bytes) - Required - A bytes object containing the raw BUFR message.
- **info_only** (bool) - Optional - If True, decodes only metadata sections, which is faster.

### Request Example
```python
from pybufrkit.decoder import Decoder

decoder = Decoder()

with open('sample.bufr', 'rb') as f:
    bufr_message = decoder.process(f.read())

# Info-only decode
with open('sample.bufr', 'rb') as f:
    info_msg = decoder.process(f.read(), info_only=True)
```

### Response
#### Success Response (BufrMessage)
- **edition** (int) - The BUFR edition of the message.
- **n_subsets** (int) - The number of subsets in the message.
- **data_category** (int) - The data category of the message.
- **length** (int) - The total length of the BUFR message (when info_only=True).
- **unexpanded_descriptors** (list[int]) - A list of descriptors (when info_only=True).

#### Response Example
```json
{
  "edition": 4,
  "n_subsets": 1,
  "data_category": 0
}
```

### Additional Notes
- The `Decoder` can be initialized with `compiled_template_cache_max` for performance improvements when processing many messages.
- The decoded `BufrMessage` can be rendered into different formats using renderers like `FlatTextRenderer` and `FlatJsonRenderer`.
```

--------------------------------

### Extract Lat/Lon from Files using Script

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Use the 'script' command to extract specific data values (e.g., latitude and longitude) from multiple BUFR files using descriptor IDs within a Python expression.

```bash
pybufrkit script 'print(${005001}, ${006001})' /data/*.bufr
```

--------------------------------

### Iterate Multi-Message BUFR Files with generate_bufr_message()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

A generator function to scan a byte string for all BUFR messages in sequence. Supports error recovery, info-only mode, and filter expressions to skip non-matching messages.

```python
from pybufrkit.decoder import Decoder, generate_bufr_message

decoder = Decoder()

# Iterate all messages in a concatenated BUFR file
with open('multi_message.bufr', 'rb') as f:
    data = f.read()

for bufr_message in generate_bufr_message(decoder, data):
    print(f"File: {bufr_message.filename}, Subsets: {bufr_message.n_subsets.value}")

# Skip erroneous messages and continue
for bufr_message in generate_bufr_message(decoder, data, continue_on_error=True):
    print(bufr_message.n_subsets.value)

# Filter: only decode messages where data_category == 2 (vertical soundings)
# The filter is evaluated on info-only decode first — full decode only when matched
for bufr_message in generate_bufr_message(
    decoder, data,
    filter_expr='${%data_category} == 2'
):
    print('Matched sounding message with', bufr_message.n_subsets.value, 'subsets')

# Info-only pass for quick metadata scanning
for bufr_message in generate_bufr_message(decoder, data, info_only=True):
    print('Template:', bufr_message.unexpanded_descriptors.value)
```

--------------------------------

### Run Script with BUFR Data

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Executes a Python script that can access BUFR message properties like the number of subsets. Ensure to use the function version of print for Python 3 compatibility.

```Python
from pybufrkit.script import ScriptRunner
# NOTE: must use the function version of print (Python 3), NOT the statement version
code = """print('Multiple' if ${%n_subsets} > 1 else 'Single')"""
runner = ScriptRunner(code)
runner.run(bufr_message)
```

--------------------------------

### Decode BUFR file to default flat text format

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Use the 'decode' command to convert a BUFR file into a human-readable flat text format. This is the default output when no specific format is requested.

```bash
pybufrkit decode BUFR_FILE
```

--------------------------------

### Decode BUFR file with hierarchical structure and attribute association

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Decode a BUFR file and display its content in a hierarchical structure based on BUFR Descriptors. Attribute descriptors are automatically associated with their corresponding descriptors.

```bash
pybufrkit decode -a BUFR_FILE
```

--------------------------------

### Decode multi-message BUFR file with error handling

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Decode a BUFR file containing multiple messages, skipping erroneous messages and continuing the process. Use the '-m' flag for multi-message files and '--continue-on-error' for robustness.

```bash
pybufrkit decode -m --continue-on-error FILE
```

--------------------------------

### Decode BUFR file to JSON format

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Decode a BUFR file and output the data in a flat JSON format. Use the '-j' flag for JSON output.

```bash
pybufrkit decode -j BUFR_FILE
```

--------------------------------

### Set Data Values Nesting Level

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/internals.md

Control the nesting level of returned data values using a magic comment at the beginning of the script. The command-line option `-n` or `--data-values-nest-level` takes precedence.

```default
#$ data_values_nest_level = 0
```

--------------------------------

### Query associated field

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Query for an associated field of a given descriptor. The notation 'descriptor.AssociatedDescriptor' is used.

```bash
pybufrkit query 021062.A21062 BUFR_FILE
```

--------------------------------

### Encode flat JSON file to BUFR

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Encode data from a flat JSON file into a BUFR file. Use the '-j' flag to specify JSON input and provide the output BUFR file name.

```bash
pybufrkit encode -j JSON_FILE BUFR_FILE
```

--------------------------------

### Query descriptor within a specific subset

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Query for a descriptor within a specific subset of the BUFR message. The syntax '@[subset_index] > descriptor' is used to target the first subset.

```bash
pybufrkit query '@[0] > 001002' BUFR_FILE
```

--------------------------------

### Encode JSON back to BUFR File

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Encodes a JSON data structure back into a BUFR message and writes it to a file. The output file path should be specified.

```Python
from pybufrkit.encoder import Encoder
encoder = Encoder()
bufr_message_new = encoder.process(json_data)
with open(BUFR_OUTPUT_FILE, 'wb') as outs:
    outs.write(bufr_message_new.serialized_bytes)
```

--------------------------------

### Query BUFR Metadata

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Queries specific metadata from a BUFR message, such as the number of subsets. Requires a MetadataQuerent and MetadataExprParser.

```Python
from pybufrkit.mdquery import MetadataExprParser, MetadataQuerent
n_subsets = MetadataQuerent(MetadataExprParser()).query(bufr_message, '%n_subsets')
```

--------------------------------

### Decode BUFR Message with Decoder.process()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

Decode a raw bytes object containing a BUFR message into a BufrMessage object. Supports info-only mode for metadata, and template compilation caching for performance.

```python
from pybufrkit.decoder import Decoder
from pybufrkit.renderer import FlatTextRenderer, FlatJsonRenderer

decoder = Decoder()

# Basic decoding from a file
with open('sample.bufr', 'rb') as f:
    bufr_message = decoder.process(f.read())

# Access top-level message properties
print('Edition:', bufr_message.edition.value)       # e.g. 4
print('Subsets:', bufr_message.n_subsets.value)     # e.g. 1
print('Data category:', bufr_message.data_category.value)  # e.g. 0 (surface data)

# Render as flat text
text_output = FlatTextRenderer().render(bufr_message)
print(text_output)
# <<<<<< section 0 >>>>>>
# edition = 4
# ...
# ###### subset 1 of 1 ######
#     1 001001 WMO block number                                       7
#     2 001002 WMO station number                                     15
# ...

# Render as JSON (round-trip capable)
json_data = FlatJsonRenderer().render(bufr_message)
# json_data is a list of section arrays suitable for re-encoding

# Info-only decode (metadata sections only, fast)
with open('sample.bufr', 'rb') as f:
    info_msg = decoder.process(f.read(), info_only=True)
print('Message length:', info_msg.length.value)
print('Descriptors:', info_msg.unexpanded_descriptors.value)  # e.g. [309052, 205060]

# Decode with template compilation cache for repeated processing (10-30% speedup)
fast_decoder = Decoder(compiled_template_cache_max=100)
with open('sample.bufr', 'rb') as f:
    bufr_message = fast_decoder.process(f.read())
```

--------------------------------

### Pipe decoded BUFR to encoder

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Decode a BUFR file, pipe its output to the encoder to convert it back into a BUFR format. The '-' argument tells the encoder to read from standard input.

```bash
pybufrkit decode BUFR_FILE | pybufrkit encode -
```

--------------------------------

### generate_bufr_message()

Source: https://context7.com/ywangd/pybufrkit/llms.txt

A generator function that scans a byte string for all BUFR messages in sequence, yielding each decoded BufrMessage. Supports error recovery, info-only mode, and filter expressions to skip non-matching messages without full decoding.

```APIDOC
## generate_bufr_message()

### Description
A generator function that scans a byte string for all BUFR messages in sequence, yielding each decoded `BufrMessage`. Supports error recovery, info-only mode, and filter expressions to skip non-matching messages without full decoding.

### Method
```python
generate_bufr_message(decoder: Decoder, data: bytes, continue_on_error: bool = False, info_only: bool = False, filter_expr: str = None) -> Iterator[BufrMessage]
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **decoder** (Decoder) - Required - An instance of the `Decoder` class.
- **data** (bytes) - Required - A bytes object containing the raw BUFR data, potentially with multiple messages.
- **continue_on_error** (bool) - Optional - If True, errors encountered during decoding of a message will be skipped, and processing will continue with the next message.
- **info_only** (bool) - Optional - If True, decodes only metadata sections for each message, which is faster.
- **filter_expr** (str) - Optional - A filter expression (XPath-like syntax) to apply to each message. Only messages that match the filter will be fully decoded and yielded. The expression is evaluated on info-only decode first.

### Request Example
```python
from pybufrkit.decoder import Decoder, generate_bufr_message

decoder = Decoder()

with open('multi_message.bufr', 'rb') as f:
    data = f.read()

# Iterate all messages
for bufr_message in generate_bufr_message(decoder, data):
    print(f"Subsets: {bufr_message.n_subsets.value}")

# Filter messages
for bufr_message in generate_bufr_message(decoder, data, filter_expr='${%data_category} == 2'):
    print('Matched sounding message')
```

### Response
#### Success Response (Iterator[BufrMessage])
Yields `BufrMessage` objects that have been successfully decoded according to the specified parameters.

#### Response Example
```json
{
  "filename": "multi_message.bufr",
  "edition": 4,
  "n_subsets": 1
}
```

### Additional Notes
- The `filter_expr` uses a syntax like `${%field_name} operator value`, e.g., `${%data_category} == 2`.
```

--------------------------------

### Query BUFR Data

Source: https://github.com/ywangd/pybufrkit/blob/master/README.rst

Queries specific data points from a BUFR message using a descriptor. Requires a DataQuerent and NodePathParser.

```Python
from pybufrkit.dataquery import NodePathParser, DataQuerent
query_result = DataQuerent(NodePathParser()).query(bufr_message, '001002')
```

--------------------------------

### EBNF for Query Expressions

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/index.md

The Extended Backus-Naur Form (EBNF) defines the syntax for query expressions used to retrieve template data. This grammar specifies how to construct queries involving subsets and path specifications.

```default
<query_expr> = [<subset_spec>] <path_spec>+
<subset_spec> = '@'<slice>
<path_spec> = <separator> <descriptor_id> [<slice>]
<separator> = '/' | '.' | '>'
```

--------------------------------

### Extract Latitude and Longitude Data

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/internals.md

This script extracts latitude and longitude values from BUFR files. Data values can be nested; default nesting level is one, flattening all values into a single list. Control nesting via `-n` command-line option or a magic comment.

```default
print(${005001}, ${006001})
```

--------------------------------

### Convert BUFR message to JSON

Source: https://github.com/ywangd/pybufrkit/blob/master/docs/index.md

Renders a decoded BUFR message into a flat JSON format for easier data manipulation and inspection.

```python
from pybufrkit.renderer import FlatJsonRenderer
json_data = FlatJsonRenderer().render(bufr_message)
```
Tokens:
Raw