Asterix Libs

https://github.com/zoranbosnjak/asterix-libs
Admin
Asterix Libs provides code generators and data processing libraries in Python and Haskell for...
Tokens:39,316
Snippets:155
Trust Score:7.9
License:BSD-3-Clause
Update:1 week ago
Show doc for...
Tokens:
Raw
### Install libasterix from GitHub (default and devel branches)

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Install the libasterix package directly from the GitHub repository. The first command installs from the default branch, the second from the 'devel' branch.

```bash
# (default branch)
pip install -e "git+https://github.com/zoranbosnjak/asterix-libs.git#egg=libasterix&subdirectory=libs/python"
```

```bash
# ('devel' branch)
pip install -e "git+https://github.com/zoranbosnjak/asterix-libs.git@devel#egg=libasterix&subdirectory=libs/python"
```

--------------------------------

### Run README examples

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/DEVEL.md

Run examples from README.md: extract code snippets with entangled tangle, then execute the run-samples.sh script inside a nix-shell.

```bash
nix-shell
entangled tangle                        # extract code snippets from README.md
./readme-samples/run-samples.sh         # run all samples

exit # from nix-shell
```

--------------------------------

### Run README examples with entangled

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/DEVEL.md

Manual process to extract and test examples from README.md using the entangled tool, which is not yet part of nix. Run inside a nix-shell; optionally monitor README.md with entr to re-run samples automatically.

```bash
nix-shell
entangled tangle                        # extract code snippets from README.md
./readme-samples/run-samples.sh         # run all samples

# or monitor README.md file and re-run samples automatically
echo README.md | entr sh -c \
    'clear && date && entangled tangle && ./readme-samples/run-samples.sh'

exit # from nix-shell
```

--------------------------------

### Install libasterix from PyPI

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Install the libasterix package from the Python Package Index using pip.

```bash
pip install libasterix
```

--------------------------------

### Verify libasterix installation

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Commands to verify that the asterix.base and asterix.generated modules are installed correctly and accessible.

```bash
python3 -c "import asterix.base as base; print(base.AstSpec)"
```

```bash
python3 -c "import asterix.generated as gen; print(gen.manifest['CATS'].keys())"
```

--------------------------------

### RFS handling example in Haskell

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

This example demonstrates creating a record with regular and RFS items using the Asterix library, extracting RFS items with getRfsItem, and verifying that non-RFS items are not present. It uses the Cat_008_1_3 specification and shows that RFS items can repeat.

```haskell
-- | file: readme-samples/rfs.hs
{-# LANGUAGE DataKinds #-}

import Control.Monad
import Data.Maybe
import qualified Data.Text.IO as T
import Asterix.Coding
import Asterix.Generated as Gen

-- cat008 contains RFS indicator, so we are able to add RFS items
type Spec = Gen.Cat_008_1_3

assert :: Bool -> IO ()
assert True = pure ()
assert False = error "Assertion error"

rec1 :: Record (RecordOf Spec)
rec1 = record
    -- add some regular items
    ( item @"000" 1
   *: item @"010" (group (item @"SAC" 1 *: item @"SIC" 2 *: nil))
    -- add items as RFS (may repeat)
   *: rfs
        ( item @"010" (group (item @"SAC" 1 *: item @"SIC" 2 *: nil))
       *: item @"010" (group (item @"SAC" 1 *: item @"SIC" 2 *: nil))
       *: nil)
   *: nil )

main :: IO ()
main = do
    -- extract regular item 010
    let i010Regular = fromJust $ getItem @"010" rec1
    T.putStrLn $ debugBits $ unparse @Bits i010Regular

    -- extract RFS items 010, expecting 2 such items
    let i010Rfs = getRfsItem @"010" rec1
    assert (length i010Rfs == 2)
    forM_ i010Rfs $ \i -> do
        T.putStrLn $ debugBits $ unparse @Bits i

    -- but item '000' is not present in RFS
    assert (null $ getRfsItem @"000" rec1)
```

--------------------------------

### Generic ASTERIX record and datablock encoding example

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

Shows how to build a URecord and UDatablock from a schema (Cat_062_1_20), encode it to bytes, and verify the result against an expected hex string. Uses helper functions hexlify/unhexlify and a custom assert. The example is a complete main program.

```haskell
-- | file: readme-samples/generic-zero.hs
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}

import Data.Text (Text)
import qualified Data.Text.IO as T
import qualified Data.ByteString as BS
import qualified Data.ByteString.Base16 as B16
import Data.Base16.Types (assertBase16, extractBase16)
import Asterix.Coding
import Asterix.Generated as Gen

-- | Convert bytestring to hex representation.
hexlify :: BS.ByteString -> Text
hexlify = extractBase16 . B16.encodeBase16

-- | Convert hex representation back to a bytestring.
unhexlify :: Text -> BS.ByteString
unhexlify = B16.decodeBase16' . assertBase16

assert :: Bool -> IO ()
assert True = pure ()
assert False = error "Assertion error"

-- we could even randomly select a category/edition from the 'manifest',
-- but for simplicity just use a particular spec's schema
schAsterix :: VAsterix
schAsterix = schema @Cat_062_1_20 Proxy

schCat :: Int
schCat = case schAsterix of
    GAsterixBasic cat _ed _uap -> cat
    GAsterixExpansion cat _ed _exp -> cat

schRecord :: VRecord
schRecord = case schAsterix of
    GAsterixBasic _cat _ed (GUap r) -> r
    _ -> error "unexpected"

schItems :: [VUapItem]
schItems =
    let GRecord lst = schRecord
    in lst

rec :: URecord
rec = URecord bld items
  where
    goVar :: VVariation -> Maybe UVariation
    goVar = \case
        GElement o n _cont -> Just $ UElement $ integerToBits o n 0
        GGroup _o lst -> UGroup <$> mapM goItem lst
        _ -> Nothing -- skip for this test

    goItem :: VItem -> Maybe UItem
    goItem = \case
        GSpare o n -> Just $ USpare $ integerToBits o n 0
        GItem nsp -> UItem <$> goNsp nsp

    goRv :: VRule VVariation -> Maybe URuleVar
    goRv = \case
        GContextFree var -> URuleVar <$> goVar var
        GDependent _lst1 var _lst2 -> URuleVar <$> goVar var

    goNsp :: VNonSpare -> Maybe UNonSpare
    goNsp (GNonSpare _name _title rv) = UNonSpare <$> goRv rv

    f :: VUapItem -> Maybe (RecordItem UNonSpare)
    f = \case
        GUapItem nsp -> RecordItem <$> goNsp nsp
        _ -> Nothing
    items = fmap f schItems
    bld = rebuildRecord items

db :: UDatablock
db = UDatablock bld records
  where
    records = [(Nothing, rec)]
    bld = datablockBuilder schCat (fmap snd records)

main :: IO ()
main = do
    let expected = unhexlify
            "3e0038bfe9bd5000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
        result = toByteString $ unparse @SBuilder db
    T.putStrLn $ hexlify result
    assert (result == expected)
```

--------------------------------

### Development environment commands

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/DEVEL.md

Start a nix-shell and run common development tasks: format Python files with autopep8, run tests via module or pytest, run mypy and pytest, generate samples, and watch for changes with entr.

```bash
nix-shell

# prettify python source code
autopep8 --in-place --aggressive --aggressive <filename> <filename>...
find . | grep "\.py$" \
    | grep -v "generated" \
    | xargs autopep8 --in-place --aggressive --aggressive

# run test from command line (as a module or via pytest), for example
python -m tests.test_asterix
python -c 'import tests.test_asterix; tests.test_asterix.test_parse1()'
pytest -v tests/test_asterix.py::test_raises

# run static code check and tests once
mypy
pytest

# generate record samples
python generate-samples.py

# monitor changes in .py files, check automatically on any change
find . | grep "\.py" | entr sh -c 'clear && date && mypy && pytest'
```

--------------------------------

### Python example demonstrating strict and partial parsing with Cat_063_1_6 and Cat_063_1_7

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

This example shows how to create a record using a newer specification (Cat_063_1_7), unparse it to bytes, and then attempt to parse it with both strict and partial modes using an older specification (Cat_063_1_6). Strict parsing fails with the old spec, but partial parsing succeeds and extracts items '010' and '015' correctly, though the resulting record is not identical to the original.

```python
#| file: readme-samples/example-partial-parsing.py
from asterix.base import *
from asterix.generated import *

SpecOld = Cat_063_1_6
SpecNew = Cat_063_1_7

# In the new spec, item 060 is extended to contain 3 groups,
# while in the old spec it only contain2 groups.
# Create record according to the new spec
rec0 = SpecNew.cv_record.create({
    '010': (('SAC', 1), ('SIC', 2)),
    '015': 3,
    '060': ((1, 2, 3)),
    })

# This bytestring represents the record of SpecNew
bs = rec0.unparse()

expected = unhexlify("c8010203030506")
assert (bs.to_bytes() == expected)

# We should be able to parse the record, using the new spec
# and get the same record back.
result1 = SpecNew.cv_record.parse(ParsingMode.StrictParsing, bs)
assert not isinstance(result1, ValueError)
rec1, _bs1  = result1
assert (rec1.unparse() == rec0.unparse())

# Strict parsing with the old spec fails.
result2 = SpecOld.cv_record.parse(ParsingMode.StrictParsing, bs)
assert isinstance(result2, ValueError)

# However, we can still try to parse using the PartialParsing mode.
result3 = SpecOld.cv_record.parse(ParsingMode.PartialParsing, bs)
assert not isinstance(result3, ValueError)
rec3, _ignore = result3

# We accept the fact that resulting record might not be complete, but
# items "010" and "015" are valid, even if parsing using the old edition.
i010 = rec3.get_item('010')
i015 = rec3.get_item('015')
assert i010 is not None
assert i015 is not None
assert (i010.as_uint() == 0x0102)
assert (i015.as_uint() == 3)

# Note, that the result in this case in not equal to the original record.
assert (rec3.unparse() != rec0.unparse())
```

--------------------------------

### Asterix encoding and decoding example with type annotations

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Complete example showing how to encode custom tokens into Asterix datablocks (Cat034 and Cat048) and decode received bytes back into the original message list. Includes type annotations for static checking with mypy; in untyped environments, annotations and assertions can be skipped. Demonstrates use of generated category classes, cv_record, cv_uap, RawDatablock, and Bits.

```python
#| file: readme-samples/example0.py
from typing import *
from binascii import hexlify, unhexlify
from dataclasses import dataclass

from asterix.base import *
import asterix.generated as gen

# Select particular asterix categories and editions
Cat034 = gen.Cat_034_1_29
Cat048 = gen.Cat_048_1_32

# Example messages for this application
class Token:
    pass

@dataclass
class NorthMarker(Token):
    pass

@dataclass
class SectorCrossing(Token):
    azimuth: float

@dataclass
class Plot(Token):
    rho: float
    theta: float
    ssr: str

# example message to be encoded
tx_message = [
    NorthMarker(),
    SectorCrossing(0.0),
    Plot(rho=10.0, theta=45.0, ssr='7777'),
    SectorCrossing(45.0),
]
print('sending message:', tx_message)

# encode token to datablock
def encode(token: Token) -> bytes:
    if isinstance(token, NorthMarker):
        rec034 = Cat034.cv_record.create({
            '000': 1, # North marker message
            '010': (('SAC', 1), ('SIC', 2)),
        })
        datablock034 = Cat034.create([rec034])
        return datablock034.unparse().to_bytes()
    if isinstance(token, SectorCrossing):
        rec034 = Cat034.cv_record.create({
            '000': 2, # Sector crossing message
            '010': (('SAC', 1), ('SIC', 2)),
            '020': ((token.azimuth, "°")),
        })
        datablock034 = Cat034.create([rec034])
        return datablock034.unparse().to_bytes()
    if isinstance(token, Plot):
        rec048 = Cat048.cv_record.create({
            '010': (('SAC', 1), ('SIC', 2)),
            '040': (('RHO', (token.rho, "NM")), ('THETA', (token.theta, "°"))),
            '070': (0, 0, 0, 0, ('MODE3A', token.ssr)),
        })
        datablock048= Cat048.create([rec048])
        return datablock048.unparse().to_bytes()
    raise Exception('unexpected token', token)

datablocks = [encode(token) for token in tx_message]
tx = b''.join(datablocks)
print('bytes on the wire:', hexlify(tx))

assert hexlify(tx) == \
    b'220007c0010201220008d00102020030000c9801020a0020000fff220008d001020220'

# decode bytes to message list
def decode(rx_bytes: bytes) -> List[Token]:
    message: List[Token] = []

    raw_datablocks = RawDatablock.parse(Bits.from_bytes(tx))
    assert not isinstance(raw_datablocks, ValueError)
    for db in raw_datablocks:
        cat = db.get_category()
        if cat == 34:
            result034 = Cat034.cv_uap.parse(db.get_raw_records())
            assert not isinstance(result034, ValueError)
            for rec034 in result034:
                i000 = rec034.get_item('000')
                assert i000 is not None
                val = i000.as_uint()
                if val == 1:
                    message.append(NorthMarker())
                elif val == 2:
                    i020 = rec034.get_item('020')
                    assert i020 is not None
                    azimuth = i020.variation.content.as_quantity("°")
                    message.append(SectorCrossing(azimuth = azimuth))
                else:
                    pass
        elif cat == 48:
            result048 = Cat048.cv_uap.parse(db.get_raw_records())
            assert not isinstance(result048, ValueError)
            for rec048 in result048:
                i040 = rec048.get_item('040')
                i070 = rec048.get_item('070')
                assert i040 is not None
                assert i070 is not None
                rho = i040.variation.get_item('RHO').variation.content.as_quantity("NM")
                theta = i040.variation.get_item('THETA').variation.content.as_quantity("°")
                ssr = i070.variation.get_item('MODE3A').variation.content.as_string()
                message.append(Plot(rho = rho, theta = theta, ssr = ssr))
        else:
            pass
    return message

rx = tx
rx_message = decode(rx)

# expect the same message
print('received message:', rx_message)
assert rx_message == tx_message
```

--------------------------------

### Show raw content of all toplevel items of each record

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

This Haskell example demonstrates generic asterix processing by parsing raw datablocks and printing the raw bits of non-spare items for each record. It uses a map of category numbers to record schemas, and handles unsupported categories by printing a message. The example includes sample input bytes and uses the Asterix library.

```haskell
-- | file: readme-samples/generic-names.hs
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}

import Control.Monad
import Data.Word
import Data.Either
import Data.Map as Map
import Data.Text (Text)
import Data.ByteString (ByteString)
import qualified Data.ByteString.Base16 as B16
import Data.Base16.Types (assertBase16)

import Asterix.Coding
import Asterix.Generated as Gen

-- | Convert hex representation back to a bytestring.
unhexlify :: Text -> ByteString
unhexlify = B16.decodeBase16' . assertBase16

specs :: Map Word8 VRecord
specs = Map.fromList
    [ (48, schema @(RecordOf Cat_048_1_31) Proxy)
    , (62, schema @(RecordOf Cat_062_1_19) Proxy)
    , (63, schema @(RecordOf Cat_063_1_6) Proxy)
    -- , ...
    ]

-- some test input bytes
s :: ByteString
s = mconcat $ fmap unhexlify
    [ "3e00a5254327d835a95a0d0a2baf256af940e8a8d0caa1a594e1e525f2e32bc0448b"
    , "0e34c0b6211b5847038319d1b88d714b990a6e061589a414209d2e1d00ba5602248e"
    , "64092c2a0410138b2c030621c2043080fe06182ee40d2fa51078192cce70e9af5435"
    , "aeb2e3c74efc7107052ce9a0a721290cb5b2b566137911b5315fa412250031b95579"
    , "03ed2ef47142ed8a79165c82fb803c0e38c7f7d641c1a4a77740960737"
    ]


handleNonspare :: Word8 -> (GUapItem ValueLevel, Maybe (RecordItem UNonSpare))
    -> IO ()
handleNonspare cat = \case
    (GUapItem (GNonSpare name _title _rv), Just (RecordItem nsp)) -> do
        print (cat, name, debugBits $ unparse @Bits nsp)
        -- depending on the application, we might want to display
        -- deep subitems, which is possible by examining 'nsp' object
    _ -> pure ()

main :: IO ()
main = do
    let rawDatablocks = fromRight (error "unexpected") $ parseRawDatablocks s
    forM_ rawDatablocks $ \db -> do
        let cat = rawDatablockCategory db
        case Map.lookup cat specs of
            Nothing -> print ("unsupported category" :: String, cat)
            Just (GRecord sch) -> do
                let act = parseRecords (GRecord sch)
                    records = fromRight (error "unexpected")
                        (parse @StrictParsing act (getRawRecords db))
                forM_ records $ \rec -> do
                    mapM_ (handleNonspare cat) (zip sch $ uRecItems rec)

```

--------------------------------

### Haskell example: handling content dependency with I062/380/IAS

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

This Haskell code demonstrates how to work with content dependency in the Asterix library, specifically for item I062/380/IAS. It shows how to create records using raw values, default cases, and quantities (NM/s and Mach), then parses the datablock and prints raw and converted values. The example uses the generated specification Cat_062_1_20 and expects a specific hex-encoded datablock.

```haskell
-- | file: readme-samples/dep-content.hs
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE OverloadedStrings #-}

import Control.Monad
import Data.Maybe
import Data.Either
import Data.ByteString (ByteString)
import Data.Text (Text)
import qualified Data.ByteString as BS
import qualified Data.ByteString.Base16 as B16
import Data.Base16.Types (assertBase16)

import Asterix.Coding
import Asterix.Generated as Gen

type Spec = Gen.Cat_062_1_20

-- | Convert hex representation back to a bytestring.
unhexlify :: Text -> BS.ByteString
unhexlify = B16.decodeBase16' . assertBase16

assert :: Bool -> IO ()
assert True = pure ()
assert False = error "Assertion error"

-- create records by different methods

-- set raw value
rec0 :: Record (RecordOf Spec)
rec0 = record
    ( item @"380" (compound
        ( item @"IAS" (group
            ( item @"IM" 0 -- set IM to 0
           *: item @"IAS" 1 -- set IAS to raw value 1 (no unit conversion)
           *: nil))
       *: nil))
   *: nil )

-- set raw value using default case
rec1 :: Record (RecordOf Spec)
rec1 = record
    ( item @"380" (compound
        ( item @"IAS" (group
            ( item @"IM" 0 -- set IM to 0
           *: item @"IAS" 1 -- same as above
           *: nil))
       *: nil))
   *: nil )

-- set IAS speed (NM/s)
rec2 :: Record (RecordOf Spec)
rec2 = record
    ( item @"380" (compound
        ( item @"IAS" (group
            ( item @"IM" 0 -- set IM to 0
           -- use case with index 0 (IAS), set IAS to 1.2 NM/s
           *: item @"IAS" (quantity @"NM/s" @('Just 0) 1.2)
           *: nil))
       *: nil))
   *: nil )

-- set Mach speed
rec3 :: Record (RecordOf Spec)
rec3 = record
    ( item @"380" (compound
        ( item @"IAS" (group
            ( item @"IM" 1 -- set IM to 1 (Mach)
           -- use case with index 1 (Mach), set IAS to 0.8 Mach
           *: item @"IAS" (quantity @"Mach" @('Just 1) 0.8)
           *: nil))
       *: nil))
   *: nil )

db0 :: Datablock (DatablockOf Spec)
db0 = datablock (rec0 *: rec1 *: rec2 *: rec3 *: nil)

expected :: ByteString
expected = unhexlify "3e0017011010000101101000010110104ccd0110108320"

main :: IO ()
main = do
    assert (toByteString (unparse @SBuilder db0) == expected)

    -- parse and interpret data from the example above
    let rx = expected
        rawDatablocks = fromRight (error "unexpected") (parseRawDatablocks rx)
    forM_ rawDatablocks $ \db -> do
        assert (rawDatablockCategory db == 62)
        let act = parseRecords (schema @(RecordOf Spec) Proxy)
            records :: [Record (RecordOf Spec)]
            records = either (const (error "unexpected")) (fmap Record)
                (parse @StrictParsing act (getRawRecords db))
        forM_ (zip [0::Int ..] records) $ \(cnt, rec) -> do
            let i380 = fromJust $ getItem @"380" rec
                iIAS1 = fromJust $ getItem @"IAS" $ getVariation i380
                iIM = getItem @"IM" iIAS1
                iIAS2 = getItem @"IAS" iIAS1
                value :: Double
                value = case asUint @Int iIM of
                    -- this is IAS, convert to 'NM/s', use case with index (0,)
                    0 -> unQuantity $ asQuantity @"NM/s" @('Just 0) iIAS2
                    -- this is Mach, convert to 'Mach', use case with index (1,)
                    1 -> unQuantity $ asQuantity @"Mach" @('Just 1) iIAS2
                    _ -> error "unexpected value"

            print ("--- record" :: String, cnt, "---" :: String)
            print ("I062/380/IAS/IM raw value:" :: String, asUint @Integer iIM)
            print ("I062/380/IAS/IAS raw value:" :: String, asUint @Integer iIAS2)
            print ("converted value" :: String, value)

```

--------------------------------

### Spare bits: get_spares and setting spare bits

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Demonstrates handling of spare bits in item I062/120. Shows that spare bits can be set to any value (including non-zero) and that get_spares returns the actual values of all spare bit groups. The example creates two records, one with spare bits set to 0 and one set to 0xf, and asserts the returned spare values.

```python
#| file: readme-samples/example-spare.py
from asterix.generated import *

# I062/120 contain single group of spare bits
Spec = Cat_062_1_20

# create regular record with spare bits set to '0'
rec1 = Spec.cv_record.create({
    '120': (0, ('MODE2', 0x1234)),
})
i120a = rec1.get_item('120')
assert i120a is not None
spares1 = i120a.variation.get_spares()
assert spares1 == [0]

# create record, abuse spare bits, set to '0xf'
rec2 = Spec.cv_record.create({
    '120': (0xf, ('MODE2', 0x1234)),
})
i120b = rec2.get_item('120')
assert i120b is not None
spares2 = i120b.variation.get_spares()
assert spares2 == [0xf]
```

--------------------------------

### Category filter example (drop datablocks if category == 1)

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

Shows a complete Haskell program that receives UDP data, parses raw datablocks, filters out those with category 1, and sends the remaining datablocks. Includes hexlify/unhexlify helpers and test send/receive functions. Requires DataKinds and OverloadedStrings.

```haskell
-- | file: readme-samples/catflt.hs
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE OverloadedStrings #-}

import Control.Monad
import Data.ByteString (ByteString)
import Data.Text (Text)
import qualified Data.ByteString.Base16 as B16
import Data.Base16.Types (assertBase16, extractBase16)
import qualified Data.Text.IO as T
import Asterix.Coding

-- | Convert bytestring to hex representation.
hexlify :: ByteString -> Text
hexlify = extractBase16 . B16.encodeBase16

-- | Convert hex representation back to a bytestring.
unhexlify :: Text -> ByteString
unhexlify = B16.decodeBase16' . assertBase16

-- UDP rx test function
receiveFromUdp :: IO ByteString
receiveFromUdp = pure $ unhexlify $ mconcat
    [ "01000401" -- cat1 datablock
    , "02000402" -- cat2 datablock
    ]

-- UDP tx test function
sendToUdp :: SBuilder -> IO ()
sendToUdp = T.putStrLn . hexlify . toByteString

main :: IO ()
main = do
    inputData <- receiveFromUdp
    let rawDatablocks = case parseRawDatablocks inputData of
            Left _ -> error "unable to parse"
            Right val -> val
        validDatablocks = do
            db <- rawDatablocks
            guard $ rawDatablockCategory db /= 1
            pure $ unparse @SBuilder db
        outputData = mconcat validDatablocks
    sendToUdp outputData
```

--------------------------------

### Typical Usage of get_item and variation

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Python example showing how to access subitems from an Asterix record using get_item and the variation property, including error handling for missing items and conversion to quantities and strings.

```python
rec048 = # some valid asterix record
i040 = rec048.get_item('040')
i070 = rec048.get_item('070')
if i040 is None: raise Exception('can not continue')
if i070 is None: raise Exception('can not continue')
rho = i040.variation.get_item('RHO').variation.content.as_quantity("NM")
theta = i040.variation.get_item('THETA').variation.content.as_quantity("°")
ssr = i070.variation.get_item('MODE3A').variation.content.as_string()
```

--------------------------------

### Development environment and code generation commands

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/code-generator/README.md

Sets up the development environment with nix-shell, runs linting and style checks, and shows commands to run the program with ghcid or runhaskell, including generating code for test runs or with all defined specs.

```bash
nix-shell

# fix permissions if necessary
chmod go-w .ghci
chmod go-w .

# check defined and used references
tagref

# lint
hlint {path}
find . | grep "\.hs$" | xargs hlint

# auto adjust style
stylish-haskell --inplace {path}
find . | grep "\.hs$" | xargs stylish-haskell --inplace

# run 'ghcid'
ghcid --no-title --lint "--command=ghci -Wall -iother -isrc src/Main.hs"

# run program, show usage
runhaskell $EXTENSIONS -iother -isrc ./src/Main.hs --help

# select target language
lang=...

# generate code for test run
runhaskell $EXTENSIONS -iother -isrc ./src/Main.hs \
    --language $lang --test \
    --ast-specs-ref "test" --ast-specs-date "test" \
    $TEST_ASTERIX_SPECS_FILES

# generate code with all defined specs
runhaskell $EXTENSIONS -iother -isrc ./src/Main.hs \
    --language $lang \
    --ast-specs-ref $ASTERIX_SPECS_REF --ast-specs-date $ASTERIX_SPECS_DATE \
    $ASTERIX_SPECS_FILES

exit
```

--------------------------------

### Build and run code-generator with Nix

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/code-generator/README.md

Builds the project with Nix and runs the resulting binary with the -h flag to show usage.

```bash
nix-build
./result/bin/code-generator -h
```

--------------------------------

### modifyExtendedSubitemIfPresent example

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

Shows how to use modifyExtendedSubitemIfPresent to apply a modifier function to a subitem only if it is present. The example constructs an extended item with only the first group present, modifies the present subitem "TYP", and verifies that modifying the absent subitem "TST" has no effect. Requires DataKinds and imports from Asterix.Coding and Asterix.Generated.

```haskell
-- | file: readme-samples/modify-extended-subitem.hs
{-# LANGUAGE DataKinds #-}
import Asterix.Coding
import Asterix.Generated as Gen

type Cat048 = Gen.Cat_048_1_32

assert :: Bool -> IO ()
assert True = pure ()
assert False = error "Assertion error"

main :: IO ()
main = do
    -- construct extended item with only the first group present
    let i020 :: NonSpare (Cat048 ~> "020")
        i020 = extendedGroups (0 *: nil)

        -- "TYP" is part of the first group (present), we are changing it
        result1 = modifyExtendedSubitemIfPresent @"TYP" (const 1) i020

        -- "TST" is part of the second group (not present),
        -- so the function call shall have no effect
        result2 = modifyExtendedSubitemIfPresent @"TST" (const 1) i020

    assert (unparse @Bits i020 /= unparse result1)
    assert (unparse @Bits i020 == unparse result2)
```

--------------------------------

### Convert Haskell package to Nix with cabal2nix

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/nix/README.md

Use cabal2nix to generate a .nix file directly from a repository revision, instead of using nix-prefetch-git.

```bash
hsh=...
cabal2nix --revision ${hsh} https://github.com/user/proj.git > nix/proj.nix
```

--------------------------------

### Update niv sources

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/nix/README.md

Run these commands in the project base directory (outside of nix/) to initialize niv and update nixpkgs to a specific branch.

```bash
# update niv
nix-shell -p niv --run "niv init"

# update packages
nix-shell -p niv --run "niv update nixpkgs -b master"
nix-shell -p niv --run "niv update nixpkgs -b release-..."
```

--------------------------------

### Demonstrate strict and partial parsing modes

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

Haskell example showing strict parsing fails with an old spec when the transmitter uses a new spec, while partial parsing succeeds and extracts valid items. The example uses a record with item 060 extended to 3 groups in the new spec, and asserts the expected byte string.

```haskell
-- | file: readme-samples/parsing-partial-mode.hs
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE OverloadedStrings #-}

import Data.Maybe
import Data.Either
import Data.ByteString (ByteString)
import Data.Text (Text)
import qualified Data.ByteString as BS
import qualified Data.ByteString.Base16 as B16
import Data.Base16.Types (assertBase16)

import Asterix.Coding
import Asterix.Generated as Gen

type SpecOld = Gen.Cat_063_1_6
type SpecNew = Gen.Cat_063_1_7

-- | Convert hex representation back to a bytestring.
unhexlify :: Text -> BS.ByteString
unhexlify = B16.decodeBase16' . assertBase16

assert :: Bool -> IO ()
assert True = pure ()
assert False = error "Assertion error"

-- In the new spec, item 060 is extended to contain 3 groups,
-- while in the old spec it only contain2 groups.
-- Create record according to the new spec
rec0 :: Record (RecordOf SpecNew)
rec0 = record
    ( item @"010" (group (item @"SAC" 1 *: item @"SIC" 2 *: nil))
   *: item @"015" 3
   *: item @"060" (extendedGroups (1 *: 2 *: 3 *: nil))
   *: nil )

-- This bytestring represents the record of SpecNew
bs :: ByteString
bs = toByteString $ unparse @SBuilder rec0

main :: IO ()
main = do
    let expected = unhexlify "c8010203030506"
    assert (bs == expected)

    -- We should be able to parse the record, using the new spec
    -- and get the same record back.
    let act1 = parseRecord (schema @(RecordOf SpecNew) Proxy)
        rec1 = fromRight (error "unexpected") (parse @StrictParsing act1 bs)
    assert (unparse @Bits rec1 == unparse rec0)

    -- Strict parsing with the old spec fails.
    let act2 = parseRecord (schema @(RecordOf SpecOld) Proxy)
        rec2 = parse @StrictParsing act2 bs
    assert $ isLeft rec2

    -- However, we can still try to parse using the PartialParsing mode.
    let act3 = parseRecord (schema @(RecordOf SpecOld) Proxy)
        rec3 :: Record (RecordOf SpecOld)
        rec3 = Record $ fromRight (error "unexpected")
            (parse @PartialParsing act3 bs)

    -- We accept the fact that resulting record might not be complete, but
    -- items "010" and "015" are valid, even if parsing using the old edition.
    let i010 = fromJust $ getItem @"010" rec3
        i015 = fromJust $ getItem @"015" rec3
    assert (asUint @Int i010 == 0x0102)
    assert (asUint @Int i015 == 3)

    -- Note, that the result in this case in not equal to the original record.
    assert (unparse @Bits rec3 /= unparse rec0)
```

--------------------------------

### Handle content dependency with I062/380/IAS

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Demonstrates creating records with content dependency (I062/380/IAS) using different methods: setting raw values, using default case, setting IAS speed in NM/s, and setting Mach speed. Then parses the unparsed data and converts values based on the IM item. Requires user to set dependent item values correctly.

```python
#| file: readme-samples/example-dep-content.py
from asterix.base import *
from asterix.generated import *

Spec = Cat_062_1_20 # Cat 062, edition 1.20

# create records by different methods

# set raw value
rec0 = Spec.cv_record.create({
    '380': {
        'IAS': (
            ('IM', 0),  # set IM to 0
            ('IAS', 1)  # set IAS to raw value 1 (no unit conversion)
        ) },
    })

# set raw value using default case
rec1 = Spec.cv_record.create({
    '380': {
        'IAS': (
            ('IM', 0),  # set IM to 0
            ('IAS',
                (None,  # use default raw case
                 1))    # set IAS to raw value 1 (same as above)
        ) },
    })

# set IAS speed (NM/s)
rec2 = Spec.cv_record.create({
    '380': {
        'IAS': (
            ('IM', 0), # airspeed = IAS
            ('IAS',
                ((0,), # use case with index 0 (IAS)
                 (1.2, 'NM/s'))), # set IAS to 1.2 NM/s
        ) },
    })

# set Mach speed
rec3 = Spec.cv_record.create({
    '380': {
        'IAS': (
            ('IM', 1), # airspeed = Mach
            ('IAS',
                ((1,), # use case with index 1 (Mach)
                 (0.8, 'Mach'))), # set speed to 0.8 Mach
        ) },
    })

db = Spec.create([rec0, rec1, rec2, rec3])
expected_output = b'3e0017011010000101101000010110104ccd0110108320'
assert hexlify(db.unparse().to_bytes()) == expected_output

# parse and interpret data from the example above
input_bytes = unhexlify(expected_output)
raw_datablocks = RawDatablock.parse(Bits.from_bytes(input_bytes))
assert not isinstance(raw_datablocks, ValueError)
for db in raw_datablocks:
    assert db.get_category() == 62
    result = Spec.cv_uap.parse(db.get_raw_records())
    assert not isinstance(result, ValueError)
    for (cnt, rec) in enumerate(result):
        i380 = rec.get_item('380')
        assert i380 is not None
        item_IAS1 = i380.variation.get_item('IAS')
        assert item_IAS1 is not None
        item_IM = item_IAS1.variation.get_item('IM')
        item_IAS2 = item_IAS1.variation.get_item('IAS')
        assert item_IM is not None
        assert item_IAS2 is not None
        match item_IM.as_uint(): # check value of I062/380/IAS/IM and convert
            case 0: # this is IAS, convert to 'NM/s', use case with index (0,)
                value = ('NM/s', item_IAS2.variation.rule.content((0,)).as_quantity('NM/s'))
            case 1: # this is Mach, convert to 'Mach', use case with index (1,)
                value = ('Mach', item_IAS2.variation.rule.content((1,)).as_quantity('Mach'))
            case _:
                raise Exception('unexpected value')
        print('--- record', cnt, '---')
        print('I062/380/IAS/IM raw value:', item_IM.as_uint())
        print('I062/380/IAS/IAS raw value:', item_IAS2.as_uint())
        print('converted value', value)
```

--------------------------------

### Manually publish to PyPI

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/DEVEL.md

Manually publish the project to PyPI. Normally not needed because a GitHub action handles it. Build the distribution, then upload to testpypi or production pypi with twine.

```bash
nix-shell
# from clean repository
git status
python3 -m build
ls -l dist/*

# upload to testpypi
twine upload --repository testpypi dist/*

# upload to production pypi
twine upload dist/*
```

--------------------------------

### Handle ParsingError when parsing raw datablocks

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

Shows how to handle a ParsingError returned by parseRawDatablocks. The example returns an empty list on error; adjust the error handling to suit your needs.

```haskell
parseDataBlocks :: ByteString -> [RawDatablock]
parseDataBlocks s = case parseRawDatablocks s of
    Left (ParsingError _err) -> [] -- decide what to do in case of error
    Right val -> val
```

--------------------------------

### Extract latest editions from manifest

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/python/README.md

Builds a dictionary of category-to-spec mappings by taking the last element of each category's list in the manifest, then prints the category and edition for each spec.

```python
#| file: readme-samples/example7.py
from asterix.generated import *

Specs = {cat: manifest['CATS'][cat][-1] for cat in manifest['CATS']}

for spec in Specs.values():
    print(spec.cv_category, spec.cv_edition)
```

--------------------------------

### Extract latest editions from manifest

Source: https://github.com/zoranbosnjak/asterix-libs/blob/main/libs/haskell/README.md

Uses 'latestEditionsBasic' to get the latest edition for each category from the manifest and prints the category and edition pairs. This is a convenient way to iterate over all available categories.

```haskell
-- | file: readme-samples/generic-latest.hs
import Control.Monad
import Data.Map

import Asterix.Base (latestEditionsBasic)
import Asterix.Generated (manifest)

main :: IO ()
main = forM_ (assocs $ latestEditionsBasic manifest) $ \(cat, (ed, _uap)) -> do
    print (cat, ed)
```
Tokens:
Raw