### Example Schema for Foo Instrument Metadata

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Provides a sample ASDF schema definition including $schema declaration, title, description, and properties for investigator and exposure_time.

```APIDOC
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"

title: |
  Metadata for the foo instrument.
description: |
  This stores some information about an exposure from the foo instrument.
examples:
  - 
    - A minimal description of an exposure.
    - |
        tag:example.org:foo/metadata-1.0.0
          exposure_time: 0.001

type: object
properties:
  investigator:
    type: string
    description: |
      The name of the principal investigator who requested the
      exposure.

  exposure_time:
    type: number
    description: |
      The time of the exposure, in nanoseconds.
```

--------------------------------

### Example Schema ID for Foo Instrument Metadata

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Demonstrates the schema ID URI for a custom ASDF schema, following the established naming conventions.

```APIDOC
id: "http://example.org/schemas/foo/metadata-1.0.0"
```

--------------------------------

### ASDF File Example with Version Information

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/versioning.rst

Demonstrates a typical ASDF file structure, highlighting the presence of various version numbers including file format, standard, tag, extension, and software versions.

```yaml
#ASDF 1.0.0
#ASDF_STANDARD 1.6.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/ 
---
!core/asdf-1.1.0
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf', 
  name: asdf, version: 4.1.0}
history:
  extensions:
  - !core/extension_metadata-1.0.0
    extension_class: asdf.extension._manifest.ManifestExtension
    extension_uri: asdf://asdf-format.org/core/extensions/core-1.6.0
    manifest_software: !core/software-1.0.0 {name: asdf_standard, version: 1.1.1}
    software: !core/software-1.0.0 {name: asdf, version: 4.1.0}
data: !core/ndarray-1.1.0
  data: [0, 1, 2, 3, 4, 5, 6, 7]
  datatype: int64
  shape: [8]
...
```

--------------------------------

### ASDF Schema Descriptive Fields

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Schemas can optionally include descriptive fields like title, description, and examples, which support core markdown syntax for documentation rendering.

```APIDOC
Descriptive information
^^^^^^^^^^^^^^^^^^^^^^^^

Each schema may optionally contain descriptive fields: ``title``,
``description`` and ``examples``.  These fields may contain core markdown
syntax (which will be used for the purposes of rendering schema documentation
by, for example, `sphinx-asdf
<https://github.com/spacetelescope/sphinx-asdf>`__).

- ``title``: A one-line summary of the data type described by the schema

- ``description``: A lengthier prose description of the schema

- ``examples``: A list of example content that conforms to the schema,
  illustrating how to use it.
```

--------------------------------

### Example ASDF Schema ID for ndarray

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Illustrates the application of the schema ID URI convention using the ASDF core ndarray schema.

```APIDOC
id: "http://stsci.edu/schemas/asdf/core/ndarray-1.1.0"
```

--------------------------------

### ASDF Schema Example with Custom Unit

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Defines a schema for metadata including a custom exposure time unit, referencing the ASDF core unit schema.

```APIDOC
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "http://example.org/schemas/foo/metadata-1.0.0"

title: |
  Metadata for the foo instrument.
description: |
  This stores some information about an exposure from the foo instrument.
examples:
  - 
    - A minimal description of an exposure.
    - |
        tag:example.org:foo/metadata-1.0.0
          exposure_time: 0.001

type: object
properties:
  investigator:
    type: string
    description: |
      The name of the principal investigator who requested the
      exposure.

  exposure_time:
    type: number
    description: |
      The time of the exposure, in nanoseconds.

  exposure_time_units:
    allOf:
      - $ref: "http://stsci.edu/schemas/asdf/unit/unit-1.0.0"
    description: |
      The unit of the exposure time.

required: [exposure_time]
additionalProperties: true
...
```

--------------------------------

### ASDF File Header Example

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/file_layout.rst

All ASDF files must begin with a specific header line that identifies the file as ASDF and specifies the file format version. This header is designed to be a YAML comment.

```ASDF
#ASDF 1.0.0
```

--------------------------------

### ASDF Tree YAML Document Example

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/file_layout.rst

Illustrates the structure of the YAML tree within an ASDF file, including the YAML version declaration, document start/end markers, and sample core ASDF tags for ndarray.

```YAML
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
---
!core/asdf-1.0.0
data: !core/ndarray-1.0.0
  source: 0
  datatype: float64
  shape: [1024, 1024]
...
```

--------------------------------

### ASDF Schema Installation in Non-Python Environments

Source: https://github.com/asdf-format/asdf-standard/wiki/2.0.0-roadmap

This section addresses how ASDF schemas, particularly transform schemas, can be made available to non-Python environments. It explores options like including schemas as submodules and auto-generating header files.

```APIDOC
Open Questions for Non-Python Environments (e.g., asdf-cpp):

- Include schema repositories as submodules.
- Auto-generate header files containing schema content as strings.
- Install schemas to a specific path known by the executable.
```

--------------------------------

### ASDF Block Index Structure

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/file_layout.rst

Defines the structure of the ASDF block index, including its header and the YAML document containing block offsets. It also provides an example of a valid block index.

```ASDF
#ASDF BLOCK INDEX
%YAML 1.1
---
[2043, 16340]
...
```

--------------------------------

### Create and Push Git Tag

Source: https://github.com/asdf-format/asdf-standard/wiki/Release-Process

Commands to create an annotated Git tag for a new release and push it to the upstream remote repository.

```git
git tag -a a.b.c -m "Tagging a.b.c release"
git push upstream a.b.c
```

--------------------------------

### ASDF Schema $schema Attribute Example

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

The $schema attribute declares the metaschema used to validate the schema itself. Most custom ASDF schemas conform to YAML Schema.

```YAML
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
```

--------------------------------

### ASDF Specification Default Value Example

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Illustrates how the 'default' attribute in JSON Schema can be used for descriptive purposes rather than literal value insertion in newer ASDF specifications.

```APIDOC
default: An array of zeros matching the dimensions of the data array.
```

--------------------------------

### ASDF Standard Developer API Reference

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/dev_api.rst

This section details the developer API for the ASDF library, highlighting components integrated into ASDF Standard to manage circular build dependencies. It specifically points to the 'asdf_standard.resource' module for API details.

```APIDOC
.. automodapi:: asdf_standard.resource
```

--------------------------------

### ASDF Core Schemas Manifests

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas/manifest.rst

This section details the ASDF core schema manifests for various ASDF specification versions. It uses the `asdf-autoschemas` directive to generate documentation from schema definitions.

```python
.. asdf-autoschemas::
   :schema_root: ../../resources/manifests
   :standard_prefix: asdf-format.org/core

   core-1.0.0
   core-1.1.0
   core-1.2.0
   core-1.3.0
   core-1.4.0
   core-1.5.0
   core-1.6.0
```

--------------------------------

### ASDF Tagging for Core Schemas

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/tree.rst

Demonstrates the use of ASDF custom tags, specifically for the core ASDF and ndarray schemas, showing both the full tag URI and a user-friendly version with a %TAG declaration.

```YAML
%YAML 1.1
---
!tag:stsci.edu:asdf/core/asdf-1.1.0
data: !tag:stsci.edu:asdf/core/ndarray-1.1.0
  source: 0
  datatype: float64
  shape: [1024, 1024]
  byteorder: little
...
```

```YAML
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
---
!core/asdf-1.1.0
data: !core/ndarray-1.1.0
  source: 0
  datatype: float64
  shape: [1024, 1024]
  byteorder: little
```

--------------------------------

### ASDF Compression Algorithms

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/file_layout.rst

Lists the supported compression algorithms for ASDF blocks as defined in the specification.

```APIDOC
Compression Algorithms:

- zlib: The zlib lossless compression algorithm.
- bzp2: The bzip2 lossless compression algorithm.
```

--------------------------------

### ASDF Tag Convention

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Shows the recommended format for ASDF tags, which includes organization, standard, and name-version components.

```APIDOC
tag:example.org:foo/metadata-1.0.0
```

--------------------------------

### ASDF Schema Management Options

Source: https://github.com/asdf-format/asdf-standard/wiki/2.0.0-roadmap

This section details various strategies for managing ASDF schemas within projects, focusing on Python package integration, submodule inclusion, and external schema repositories. It highlights the trade-offs for different software environments.

```APIDOC
Project Structure Options:

1. Create repository with Python package for transform schemas:
   - Astropy lists as optional dependency.
   - Requires 'all' install or manual installation of both packages.
   - Potential for Astropy to add 'asdf' extras category.

2. Create repository without Python package:
   - Astropy incorporates schemas as submodule.
   - Files installed with Astropy.
   - Non-Python software may 'vendorize' schemas, leading to duplicates.

3. Schemas downloaded over HTTP from a centralized repository:
   - Undesirable to require network connection for opening files.
   - Software could vendorize schemas and use HTTP as a backup.
   - Caching can mitigate network dependency.

4. Create schema package and move Astropy tag classes to 'asdf-astropy':
   - Astropy users install a single dependency.
   - Other packages (transform schemas, ASDF) become hard dependencies of 'asdf-astropy'.
   - Allows independent work on schemas and tag code.
   - Requires Astropy tests to ensure tag code maintenance.

5. Keep schemas in the ASDF Standard:
   - Simplifies dependency tree.
   - Requires coordinated releases of 'asdf-standard' and 'asdf' for new schema material.

6. Move all ASDF related packages to a dedicated GitHub organization.

7. Consider using pipfile and pipenv.
```

--------------------------------

### ASDF Core Software Schema

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

The base schema for describing a software package, including its name, author, homepage, and version.

```APIDOC
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "http://stsci.sci.edu/schemas/asdf/core/software-1.0.0"
title: |
  Describes a software package.
description: |
  General-purpose description of a software package.

type: object
properties:
  name:
    description: |
      The name of the application or library.
    type: string

  author:
    description: |
      The author (or institution) that produced the software package.
    type: string

  homepage:
    description: |
      A URI to the homepage of the software.
    type: string
    format: uri

  version:
    description: |
      The version of the software used.  It is recommended, but not
      required, that this follows the (Semantic Versioning
      Specification)[http://semver.org/spec/v2.0.0.html].
    type: string

required: [name, version]
additionalProperties: true
...
```

--------------------------------

### Adding a Schema to ASDF Standard

Source: https://github.com/asdf-format/asdf-standard/wiki/FAQ

This section outlines the process for incorporating a new schema into the ASDF standard. It involves placing the schema file, updating version maps, and adjusting ASDF and Astropy repositories.

```markdown
1. Add the schema to the appropriate subfolder in asdf-standard.

2. Add the schema tag to the latest supported *version_map* file in *asdf-standard/schemas/stsci.edu/asdf/*.

3. Update the hash of *asdf-standard* in *asdf* to point to the one with the new schemas. 

4. Add the tag implementation to *asdf* or *astropy*. 

**Note** *Transform* schemas are stored in *asdf-standard* but *transform* tags live in *astropy*. In order for the tests to pass, both the tag and the schema should be available. This is a chicken and egg problem. It's easiest to test this locally. Online, on Travis if all PRs are merged, the *asdf* tests with *astropy-dev* should be passing, while the tests with stable astropy will be failing.
```

--------------------------------

### ASDF References using JSON Pointer and JSON Reference

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/tree.rst

Explains how ASDF uses IETF standards JSON Pointer (RFC 6901) and JSON Reference (Draft 3) for internal and external file referencing. It details the '$ref' keyword, URI fragments, and the encoding of special characters ('~' as '~0', '/' as '~1') in reference tokens.

```APIDOC
ASDF References:
  - Based on JSON Pointer (RFC 6901) and JSON Reference (Draft 3).
  - Represented as a mapping with a single key/value pair: '$ref': URI.
  - URI can contain a fragment (e.g., '#/path/to/element') for specific element referencing.
  - If no fragment, the reference points to the top of the tree.
  - Special characters in reference tokens:
    - '~' is encoded as '~0'
    - '/' is encoded as '~1'
  - Reference resolution occurs after the entire tree is read, allowing forward references.
  - YAML anchors/aliases are for internal references only, not external.

Example (External Reference):
  $ref: "http://www.nowhere.com/reference.asdf#/wavelengths/0"

Example (Internal Reference):
  my_mask: !core/ndarray
    source: 0
    shape: [256, 256]
    datatype: uint8
    byteorder: little
  data:
    $ref: "#/my_mask"
```

--------------------------------

### ASDF Astronomy Schemas Manifests

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas/manifest.rst

This section describes the ASDF non-core (astronomy) tags, which are documented separately. It utilizes the `asdf-autoschemas` directive with a specific standard prefix for astronomy schemas.

```python
.. asdf-autoschemas::
   :schema_root: ../../resources/manifests
   :standard_prefix: asdf-format.org/astronomy

   astronomy-1.0.0
```

--------------------------------

### ASDF Schema Reorganization

Source: https://github.com/asdf-format/asdf-standard/wiki/2.0.0-roadmap

Details the process of moving ASDF schemas out of the main ASDF Standard repository into separate, dedicated repositories. This reorganization aims to improve modularity and maintainability.

```APIDOC
Move other schemas out of the standard (Complete as of ASDF Standard 1.6.0):

- Core schemas: ndarray, top-level node, etc.
- FITS schema: Supporting nesting FITS files within ASDF.
- Unit schema: Supporting units and quantities.
- Time schema: Supporting time objects, with emphasis on astronomical time scales.

Candidate schemas for moving to separate repositories:
- fits
- gwcs
- astropy.coordinates
```

--------------------------------

### Proposed Schema Version Mismatch Handling

Source: https://github.com/asdf-format/asdf-standard/wiki/The-revenge-of-2.0.0-Roadmap

This snippet outlines the proposed wording for handling files that mix schema versions from different versions of the same extension, emphasizing support for such files.

```APIDOC
It is recommended that extension maintainers and implementation developers avoid writing files that mix tags from different versions of the same extension in a single file. However all implementations must support loading files with tags from different versions of the same extension and deserialize the tagged contents when deserialization is requested.
```

--------------------------------

### Update Stable Branch

Source: https://github.com/asdf-format/asdf-standard/wiki/Release-Process

Commands to update the 'stable' branch to point to the newly created release tag and force push the changes to the upstream remote.

```git
git checkout stable
git reset --hard a.b.c
git push upstream stable --force
```

--------------------------------

### ASDF Comment Convention

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/tree.rst

Demonstrates the ASDF convention for associating comments with YAML mappings using the reserved '//' key. ASDF parsers must ignore these comments programmatically.

```APIDOC
filter:
  //: |
    We set this to filter B here, even though C was used, because B
    is handled with more accuracy by our software.
  type: B
```

--------------------------------

### ASDF Extended Software Schema with Email

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Extends the ASDF core software schema to include an author's email address, demonstrating schema extension using 'allOf'.

```APIDOC
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "http://somewhere.org/schemas/software_extended-1.0.0"
title: |
  Describes a software package.
description: |
  Extension of ASDF core software schema to include the
  software author's contact email.

allOf:
  - $ref: http://stsci.edu/schemas/asdf/core/software-1.0.0
  - properties:
      author_email:
        description: |
          The contact email of the software author.
        type: string
    required: [author_email]
...
```

--------------------------------

### ASDF Schema YAML Header

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

All ASDF schemas begin with a YAML header indicating the version. This header is required for schemas encoded in YAML that conform to a superset of JSON Schema called yaml-schema.

```YAML
%YAML 1.1
---
```

--------------------------------

### ASDF Schema Version Mismatch Handling

Source: https://github.com/asdf-format/asdf-standard/wiki/The-revenge-of-2.0.0-Roadmap

This snippet presents the current ASDF specification's recommendation regarding mixing schema versions within a file, which the proposal seeks to modify.

```APIDOC
Writing out a file that mixes versions of schema from different versions of the ASDF standard is not recommended, though such a file should be accepted by readers given the rules above.
```

--------------------------------

### ASDF Legacy Schemas

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas/legacy.rst

Lists legacy ASDF schemas that are not part of the most recent ASDF core schemas. These include various versions of core, FITS, time, and unit schemas.

```python
core/asdf-1.0.0
core/column-1.0.0
core/integer-1.0.0
core/ndarray-1.0.0
core/subclass_metadata-1.0.0
core/table-1.0.0
fits/fits-1.0.0
time/time-1.0.0
time/time-1.1.0
unit/quantity-1.1.0
```

--------------------------------

### ASDF Block Header Structure

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/file_layout.rst

Defines the fields and their specifications for the ASDF block header, used for identifying and managing binary data blocks within an ASDF file.

```APIDOC
Block Header Fields:

- block_magic_token (4 bytes): Indicates the start of the block. Hexadecimal: d3, 42, 4c, 4b. ASCII: "\323BLK".
- header_size (16-bit unsigned integer, big-endian): Size of the remainder of the header in bytes.
- flags (32-bit unsigned integer, big-endian): Bit field containing flags.
- compression (4-byte byte string): Name of the compression algorithm (e.g., '\0\0\0\0' for no compression).
- allocated_size (64-bit unsigned integer, big-endian): Allocated space for the block (excluding header) in bytes.
- used_size (64-bit unsigned integer, big-endian): Used space for the block on disk (excluding header) in bytes.
- data_size (64-bit unsigned integer, big-endian): Size of the block when decoded in bytes. Must equal used_size if no compression.
- checksum (16-byte string): Optional MD5 checksum of the used data. All zeros indicate no checksum verification.
```

--------------------------------

### ASDF Schema Naming Conventions

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

Schema 'id' attributes must be valid URIs, and 'tag' URIs must follow specific conventions for global uniqueness.

```APIDOC
Naming Conventions
^^^^^^^^^^^^^^^^^^

Schema ``id`` attributes must be valid URIs. Furthermore ``tag`` uris must be
```

--------------------------------

### ASDF Header EBNF Definition

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/file_layout.rst

Defines the Extended Backus-Naur Form (EBNF) for the ASDF file header, specifying the required ASDF token, a space, and the format version.

```EBNF
asdf_token = "#ASDF"
header     = asdf_token " " format_version ["\r"] "\n"
```

--------------------------------

### YAML Mapping with Restricted Keys

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/tree.rst

Illustrates a valid YAML document with mapping keys that are restricted to scalar types (bool, int, str) as required by the ASDF standard for portability.

```YAML
%YAML 1.1
---
{foo: bar}:
  3.14159: baz
  [1, 2, 3]: qux
...
```

--------------------------------

### ASDF YAML Schema Definition

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas/yaml_schema.rst

Defines the ASDF YAML Schema, which extends JSON Schema Draft 4 with YAML-specific features. It references the base JSON Schema standard and outlines potential future requirements for 'id' and 'tag' keywords.

```APIDOC
ASDF YAML Schema:
  Extends: JSON Schema Draft 4
  Features:
    - YAML-specific extensions
  Keywords:
    - id: (Optional, inherited from JSON Schema, may become mandatory)
    - tag: (Optional, may become mandatory)
  Notes:
    - Metaschema extension process is complex and not well-supported by all libraries.
    - Creation of custom metaschemas is discouraged.
  Schema Root Reference: ../../resources/schemas/stsci.edu
  Standard Prefix: 
  Schema File: yaml-schema/draft-01
```

--------------------------------

### ASDF Schema References

Source: https://github.com/asdf-format/asdf-standard/blob/main/docs/source/schemas.rst

JSON Schema allows referencing other schemas using a '$ref' key with the URI of the referenced schema.

```APIDOC
References
^^^^^^^^^^

`JSON Schema`_ allows rerencing other schemas by including
a mapping containing a single key ``$ref`` and value containing the ``uri``
of the referenced schema.
```

--------------------------------

### ASDF Schema Updates and Terminology

Source: https://github.com/asdf-format/asdf-standard/blob/main/CHANGES.rst

Updates to ASDF Standard terminology and schema versions, including new time formats and ndarray schema enhancements.

```APIDOC
ASDF Standard Version Updates:

- **1.4.0 (unreleased)**: Updated terminology to distinguish between ASDF specification and ASDF core schemas.
- **1.3.0 (2025-06-11)**: Added `utime` and `tai_seconds` formats to `time-1.3.0` schema.
- **1.2.0 (2025-05-21)**: Updated reference files, removed discouragement of YAML anchors/aliases, improved documentation theme and navigation, added reference files for v1.6.0, anchor, endian, structured, and scalar types. Clarified schema references, block compression, tag version mismatches, and padding.
- **1.1.0 (2024-03-05)**: Added `quantity-1.2.0` schema with `datatype` option. Fixed `base_format` in `time-1.2.0`. Added `ndarray-1.1.0` schema. Removed erroneous `unit-1.1.0`. Fixed URI fragment format in metaschema and quantity schemas. Dropped Python 3.8 support. Added `float16` to `ndarray-1.1.0`. Removed `tag` keyword from `fits` schema. Moved non-core tags to `astronomy` manifest.
- **1.0.3 (2022-08-08)**: Updated documentation for consistency. Added `time-1.2.0` schema for bugfix. Moved packaging to `pyproject.toml`. Removed `tag` from `time-1.1.0` and other schemas.
- **1.0.2 (2022-04-15)**: Pinned `astropy` minimum version to 5.0.4.
- **1.0.1 (2022-02-23)**: Removed `asdf` as an install dependency for `asdf-standard`.
- **1.0.0 (2022-02-14)**: Added installable Python package to replace submodule usage.
```

--------------------------------

### ASDF Block Index Offset Handling

Source: https://github.com/asdf-format/asdf-standard/wiki/The-revenge-of-2.0.0-Roadmap

This documentation outlines a proposed strategy for handling invalid absolute byte offsets in ASDF file block indices. It suggests prioritizing relative offsets and regenerating the index if the initial offset is incorrect, aiming to provide access to block data even when the index is compromised by manual edits.

```APIDOC
ASDF File Block Index Offset Handling:

Problem:
  Manual edits to the ASDF tree can invalidate absolute byte offsets in the block index, leading to potential data access issues.

Proposed Solution:
  Implement a graceful handling mechanism that prioritizes relative offsets and allows for index regeneration.

Process:
1. Search for a block index upon first block request.
2. If no index is found, search for the first BLOCK_MAGIC, read the header, and seek to the requested block.
3. If an index is found, seek to the absolute offset for the requested block.
4. If BLOCK_MAGIC is not found at the absolute offset:
   a. Search for the first BLOCK_MAGIC.
   b. Regenerate the index assuming relative offsets based on the found position of the first BLOCK_MAGIC.
   c. Seek to the newly generated offset.
   d. If BLOCK_MAGIC is still not found, regenerate the block index as if none existed.

Recommendation:
  When using an offset in the block index, always ensure that the block magic token exists at that offset before reading data.
```