PyBufrKit

https://pybufrkit.readthedocs.io/
Admin
PyBufrKit is a pure Python package for decoding and encoding WMO BUFR (FM-94) messages with support...
Tokens:3,842
Snippets:55
Trust Score:9.6
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install PyBufrKit using pip

Source: http://pybufrkit.readthedocs.io

Install the PyBufrKit package from PyPi using pip. This is the recommended method for most users.

```bash
pip install pybufrkit
```

--------------------------------

### Install PyBufrKit from source

Source: http://pybufrkit.readthedocs.io

Install PyBufrKit by building from the source code. This method is useful for development or when a specific version is needed.

```bash
python setup.py install
```

--------------------------------

### Query Template Data Examples

Source: http://pybufrkit.readthedocs.io

Illustrates various valid query expressions for template data, demonstrating how to target specific descriptors, subsets, root elements, and children. It highlights the use of different separators like '/', '.', and '>' for various query purposes.

```text
008042
```

```text
@[0] > 008042
```

```text
/008042
```

```text
/008042[0]
```

```text
303051/008042
```

```text
103000.031001
```

```text
021062.A21062
```

--------------------------------

### Query metadata from BUFR sections

Source: http://pybufrkit.readthedocs.io

Query specific values from the metadata sections (0, 1, 2, 3) of a BUFR file. This example queries the number of subsets.

```bash
pybufrkit query %n_subsets BUFR_FILE
```

--------------------------------

### Filter BUFR files using script support

Source: http://pybufrkit.readthedocs.io

Use the script command to process multiple BUFR files and apply custom logic. This example finds files with more than one subset and prints their names.

```bash
pybufrkit script 'if ${%n_subsets} > 1: print(PBK_FILENAME)' DIRECTORY/*.bufr
```

--------------------------------

### Decode BUFR messages with a filter

Source: http://pybufrkit.readthedocs.io

Filter BUFR messages during decoding based on specified criteria. This example decodes only messages where 'data_category' is equal to 2.

```bash
pybufrkit decode -m --filter '${%data_category} == 2' FILE
```

--------------------------------

### Decode a BUFR file using the library

Source: http://pybufrkit.readthedocs.io

Basic library usage for decoding a BUFR file. This involves creating a Decoder instance and processing the file content.

```python
# Decode a BUFR file
from pybufrkit.decoder import Decoder
decoder = Decoder()
with open(SOME_BUFR_FILE, 'rb') as ins:
    bufr_message = decoder.process(ins.read())
```

--------------------------------

### Compile a BUFR Template from descriptors

Source: http://pybufrkit.readthedocs.io

Compile a BUFR Template from a comma-separated list of descriptors. This is useful for creating or validating BUFR structures.

```bash
pybufrkit compile 309052,205060
```

--------------------------------

### Subset a BUFR file

Source: http://pybufrkit.readthedocs.io

Create a new BUFR file by selecting specific subsets from an existing BUFR file. Specify the subset indices to extract.

```bash
pybufrkit subset 0,3,6,9 BUFR_FILE
```

--------------------------------

### Decode and re-encode BUFR using pipes

Source: http://pybufrkit.readthedocs.io

Demonstrates piping the output of the decode command directly to the encode command to convert a BUFR file back to BUFR format. The '-' argument indicates stdin/stdout.

```bash
pybufrkit decode BUFR_FILE | pybufrkit encode -
```

--------------------------------

### Decode a BUFR file (default format)

Source: http://pybufrkit.readthedocs.io

Decode a BUFR file and output its content in the default flat text format. Replace BUFR_FILE with the actual path to your file.

```bash
pybufrkit decode BUFR_FILE
```

--------------------------------

### Filter Files by BUFR Template using Script

Source: http://pybufrkit.readthedocs.io

Use this script to filter files based on a specific BUFR Template ID. Query expressions are embedded within `${...}`. PBK_FILENAME is an injected variable holding the current file's name. Ensure to use the function version of print.

```python
if 309052 in ${%unexpanded_descriptors}: print(PBK_FILENAME)

```

--------------------------------

### Query root level descriptor

Source: http://pybufrkit.readthedocs.io

Query for a specific descriptor at the root level of the BUFR Template. The '/' prefix indicates a root level query.

```bash
pybufrkit query /001002 BUFR_FILE
```

--------------------------------

### Extract Data Values using Script

Source: http://pybufrkit.readthedocs.io

This script demonstrates how to extract specific data values, such as latitude and longitude, using query expressions. The output format of data values can be controlled by nesting levels.

```python
print(${005001}, ${006001})

```

--------------------------------

### BUFR Configuration JSON Structure

Source: http://pybufrkit.readthedocs.io

Defines the structure for BUFR message configuration files, including section metadata and parameter definitions. Used for flexible BUFR message composition.

```json
{
  "index": 0,  # zero-based section index

  "description": "Indicator section",

  "default": true,  # use this config if an edition-specific one is not available

  "optional": false, # whether this section is optional

  "end_of_message": false, # whether this is the last section

  "parameters": [  # a list of parameter configs
    {
      "name": "start_signature",  # parameter name

      "nbits": 32,  # number of bits

      "type": "bytes",  # parameter type determines how the value can be processed from the input bits

      "expected": "BUFR",  # expected value for this parameter (will be validated if not None)

      "as_property": false  # whether this parameter can be accessed from the parent message object
    },
    ...
  ]
}
```

--------------------------------

### Decode BUFR file with hierarchical structure

Source: http://pybufrkit.readthedocs.io

Decode a BUFR file and display its content in a hierarchical structure that reflects the BUFR Descriptors. Attribute descriptors are associated with their corresponding descriptors.

```bash
pybufrkit decode -a BUFR_FILE
```

--------------------------------

### Query nested descriptor

Source: http://pybufrkit.readthedocs.io

Query for a descriptor that is a direct child of another descriptor within the BUFR Template. This uses a path-like syntax.

```bash
pybufrkit query /301001/001002 BUFR_FILE
```

--------------------------------

### Encode JSON file to BUFR format

Source: http://pybufrkit.readthedocs.io

Encode data from a flat JSON file into a BUFR file. Use the -j flag to specify JSON input.

```bash
pybufrkit encode -j JSON_FILE BUFR_FILE
```

--------------------------------

### Split a BUFR file into individual messages

Source: http://pybufrkit.readthedocs.io

Split a BUFR file that contains multiple messages into separate files, with each file containing a single BUFR message.

```bash
pybufrkit split BUFR_FILE
```

--------------------------------

### Query descriptor in a specific subset

Source: http://pybufrkit.readthedocs.io

Query for a descriptor within a specific subset of the BUFR message. The '@[index]' syntax targets a particular subset.

```bash
pybufrkit query '@[0] > 001002' BUFR_FILE
```

--------------------------------

### Encode JSON to BUFR File

Source: http://pybufrkit.readthedocs.io

Encodes JSON data into a BUFR message and writes it to a file. Ensure BUFR_OUTPUT_FILE is defined.

```python
from pybufrkit.encoder import Encoder
encoder = Encoder()
bufr_message_new = encoder.process(json_data)
with open(BUFR_OUTPUT_FILE, 'wb') as outs:
    outs.write(bufr_message_new.serialized_bytes)
```

--------------------------------

### Query data section descriptor

Source: http://pybufrkit.readthedocs.io

Query all values associated with a specific descriptor (e.g., 001002) in the data section of a BUFR file.

```bash
pybufrkit query 001002 BUFR_FILE
```

--------------------------------

### Decode BUFR file to JSON format

Source: http://pybufrkit.readthedocs.io

Decode a BUFR file and output the data in a flat JSON format. Use the -j flag for JSON output.

```bash
pybufrkit decode -j BUFR_FILE
```

--------------------------------

### Decode multi-message file with error handling

Source: http://pybufrkit.readthedocs.io

Decode a BUFR file containing multiple messages, skipping erroneous messages and continuing the process. Use the -m flag for multi-message files.

```bash
pybufrkit decode -m --continue-on-error FILE
```

--------------------------------

### Lookup information about a BUFR descriptor

Source: http://pybufrkit.readthedocs.io

Look up detailed information about a specific BUFR Element Descriptor, including its code table if applicable. Use the -l flag for lookup.

```bash
pybufrkit lookup -l 020003
```

--------------------------------

### Decode Multiple BUFR Messages from a File

Source: http://pybufrkit.readthedocs.io

Iterates through multiple BUFR messages within a single file. Requires a decoder object and the file content.

```python
from pybufrkit.decoder import generate_bufr_message
with open(SOME_FILE, 'rb') as ins:
    for bufr_message in generate_bufr_message(decoder, ins.read()):
        pass  # do something with the decoded message object
```

--------------------------------

### Control Data Values Nesting Level with Magic Comment

Source: http://pybufrkit.readthedocs.io

Specify the nesting level for data values directly within the script using a magic comment. This comment must appear before any other lines. Command-line options take precedence over this setting.

```python
#$ data_values_nest_level = 0

```

--------------------------------

### Decode only metadata sections of a BUFR file

Source: http://pybufrkit.readthedocs.io

Extract and display only the metadata sections (0, 1, 2, 3) of a given BUFR file. This is useful for inspecting the structure without decoding the full data.

```bash
pybufrkit info BUFR_FILE
```

--------------------------------

### EBNF Grammar for Query Expressions

Source: http://pybufrkit.readthedocs.io

Defines the Extended Backus-Naur Form (EBNF) for constructing query expressions to retrieve template data. This grammar specifies the structure of subset and path specifications, including allowed separators and descriptor ID formats.

```ebnf
<query_expr> = [<subset_spec>] <path_spec>+
<subset_spec> = '@'<slice>
<path_spec> = <separator> <descriptor_id> [<slice>]
<separator> = '/' | '.' | '>'

```

--------------------------------

### Convert BUFR message to JSON using the library

Source: http://pybufrkit.readthedocs.io

Convert a decoded BUFR message into a flat JSON format using the FlatJsonRenderer. This is a common step for data analysis or interoperability.

```python
# Convert the BUFR message to JSON
from pybufrkit.renderer import FlatJsonRenderer
json_data = FlatJsonRenderer().render(bufr_message)
```

--------------------------------

### Run Script within BUFR Message Context

Source: http://pybufrkit.readthedocs.io

Executes a Python script within the context of a BUFR message, allowing access to message properties like '%n_subsets'. Uses the function version of print.

```python
from pybufrkit.script import ScriptRunner
# NOTE: must use the function version of print (Python 3), NOT the statement version
code = """print('Multiple' if ${%n_subsets} > 1 else 'Single')"""
runner = ScriptRunner(code)
runner.run(bufr_message)
```

--------------------------------

### Query BUFR Message Metadata

Source: http://pybufrkit.readthedocs.io

Queries specific metadata from a BUFR message using a MetadataQuerent. Requires a parsed metadata expression.

```python
from pybufrkit.mdquery import MetadataExprParser, MetadataQuerent
n_subsets = MetadataQuerent(MetadataExprParser()).query(bufr_message, '%n_subsets')
```

--------------------------------

### Query associated field descriptor

Source: http://pybufrkit.readthedocs.io

Query for an associated field descriptor, often used for quality information or statistics related to a primary data field. The format is 'descriptor.AssociatedDescriptor'.

```bash
pybufrkit query 021062.A21062 BUFR_FILE
```

--------------------------------

### Query BUFR Message Data

Source: http://pybufrkit.readthedocs.io

Queries specific data points from a BUFR message using a DataQuerent and a NodePathParser. The query string specifies the data to retrieve.

```python
from pybufrkit.dataquery import NodePathParser, DataQuerent
query_result = DataQuerent(NodePathParser()).query(bufr_message, '001002')
```

--------------------------------

### BUFR Query Expression EBNF

Source: http://pybufrkit.readthedocs.io

Defines the Extended Backus-Naur Form for query expressions used in BUFR metadata sections. It specifies the format for querying parameters within different sections of a BUFR message.

```ebnf
<query_expr> = '%'[<section_index>.]<parameter_name>

```
Tokens:
Raw