### Install Development Dependencies

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/HANDOFF-2026-05-30-bench-codspeed-profiling.md

Checkout the specific branch and install the development dependencies using pixi.

```bash
git checkout feat/bench-codspeed-profiling
pixi install -e dev
```

--------------------------------

### Install and Verify Dependency

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-29-codspeed-perf-tracking.md

Install new dependencies using 'pixi install' and verify the installation by checking the version of pytest-codspeed.

```bash
pixi install -e dev
```

```bash
pixi run -e dev python -c "import pytest_codspeed; print(pytest_codspeed.__version__)"
```

--------------------------------

### Install Pre-commit Hooks with Pixi

Source: https://github.com/mcvickerlab/genvarloader/blob/main/README.md

Install pre-commit hooks using Pixi for development. This ensures code quality and consistency.

```bash
pixi run -e dev pre-commit
```

--------------------------------

### Verify filelock Installation

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-06-03-robust-ondisk-artifacts.md

After adding the dependency and running `pixi install`, verify that `filelock` is installed in the development environment by checking its version.

```python
import filelock; print(filelock.__version__)
```

--------------------------------

### RTK for PNPM Install Output

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Use 'rtk pnpm install' for token-optimized PNPM installation output.

```bash
rtk pnpm install        # Compact install output (90%)
```

--------------------------------

### Verify SemanticVersion Installation

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-21-svar-link-replacement.md

Install dependencies and verify that `pydantic-extra-types` is correctly installed by attempting to parse a semantic version string. This confirms the `SemanticVersion` class is available and functional.

```python
from pydantic_extra_types.semantic_version import SemanticVersion; print(SemanticVersion.parse('0.18.0'))
```

--------------------------------

### Install GenVarLoader

Source: https://github.com/mcvickerlab/genvarloader/blob/main/README.md

Install the GenVarLoader package using pip. PyTorch is not included by default.

```bash
pip install genvarloader
```

--------------------------------

### RTK for Dependency Overview

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Use 'rtk deps' to get a compact overview of project dependencies.

```bash
rtk deps                # Dependency overview
```

--------------------------------

### Audit Documentation Format Example

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phases-1-3.md

Example markdown structure for documenting polymorphism gaps, including missing combinations and files with low line coverage.

```markdown
```markdown
### Output mode matrix

Missing combinations (verified against coverage baseline):

- `haplotypes × padded × jitter=5 × rc_neg=True` — not exercised by any current test.
- `annotated × ragged × jitter=0 × rc_neg=False` — covered only via `test_ds_haps.py`, no assertion on return type.
- ...

Files with low line coverage in this area: `_dataset/_impl.py` lines NNN-NNN (with_seqs branch for `variants` mode).
```
```

--------------------------------

### Example _Flat Container Initialization

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-31-flat-buffer-getitem-pipeline.md

Illustrates the construction of a _Flat container using data, offsets, and outer dimensions. This is typically used within the context of data loading and manipulation pipelines.

```python
return _Flat(self.data, self.offsets, (*outer, None))
```

--------------------------------

### Sanity-check Dataloader Benchmark Setup

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-29-dataloader-bench.md

This command-line snippet performs a quick check of the dataloader benchmark prerequisites. It verifies the available threads, the number of cells for a single thread, and the total count of cells.

```bash
pixi run -e dev python -c "
import sys; sys.path.insert(0, 'experiments/dataloader')
import _common as C
print('threads union:', C.ALL_THREADS)
print('cells @ threads=1:', len(C.cells_for_threads(1)))
print('total cells:', len(C.enumerate_cells()))
"
```

--------------------------------

### Lint Workflow with actionlint

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phase7-ci-coverage.md

Optionally, lint the workflow file using `actionlint` if installed. This helps catch CI-specific issues before committing.

```bash
which actionlint && actionlint .github/workflows/test.yaml || echo "actionlint not installed; skipping (CI's own validation will catch issues)"

```

--------------------------------

### Verify Pyrefly Command Help

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-23-refactor-campaign-pr0-pyrefly.md

Run this command to verify that pyrefly is installed and accessible via pixi. It should display help text without configuration errors.

```bash
pixi run -e dev pyrefly check --help
```

--------------------------------

### Get Latest Pixi Version

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-21-release-pipeline.md

Use the GitHub API to fetch the latest release tag for the setup-pixi action. This tag will be used as the new pixi-version in the workflow file.

```bash
gh api repos/prefix-dev/setup-pixi/releases/latest --jq .tag_name
```

--------------------------------

### Atomic Write Context Manager Setup

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-06-03-robust-ondisk-artifacts.md

Replaces the original path with a temporary directory for atomic operations. This is used when the existing body of the write function needs to be preserved with minimal changes.

```python
    dest = Path(path)
    _atomic_ctx = atomic_dir(dest, overwrite=overwrite)
    path = _atomic_ctx.__enter__()
```

--------------------------------

### RTK Meta Command: Init

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Use 'rtk init' to add RTK instructions to the current CLAUDE.md file. Use '--global' for ~/.claude/CLAUDE.md.

```bash
rtk init                # Add RTK instructions to CLAUDE.md
rtk init --global       # Add RTK to ~/.claude/CLAUDE.md
```

--------------------------------

### gvl Dataset with_seqs Variants Example

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-31-gvl-phase2.md

Yields RaggedVariants when using with_seqs("variants"), providing variant data like start, alt, and optional AF.

```python
.with_seqs("variants") yields RaggedVariants (awkward array, fields at least `start`,`alt`, and one of `ref`/`ilen`; optional `dosage` + INFO fields like `AF`).
```

--------------------------------

### Add Per-Cell Measurement Protocol

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-29-dataloader-bench.md

This snippet shows the context for adding a per-cell measurement protocol within the Dataloader benchmarking setup. It includes variables like git SHA, host, and start time.

```python
{
        "git_sha": git_sha,
        "host": host,
        "started_at": started_at,
    }
```
```

--------------------------------

### Create a Dummy Dataset

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/source/dataset.md

Use `gvl.get_dummy_dataset()` to create a sample dataset for testing and demonstration purposes.

```python
ds = gvl.get_dummy_dataset()
```

--------------------------------

### Prepare BigWig Sample Table

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/source/geuvadis.ipynb

Read a CSV file for BigWig paths and join it with actual download paths. The table must contain 'sample' and 'path' columns.

```python
bigwig_table = (
    pl.read_csv(bw_table_path)
    .join(
        pl.Series(bw_paths).to_frame("realpath"),
        left_on="path",
        right_on=pl.col("realpath").str.split("/").list.get(-1),
    )
    .drop("path")
    .rename({"realpath": "path"})
)
bigwig_table.head()
```

--------------------------------

### Build Documentation

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Generates project documentation using the specified build command.

```bash
# Build docs
pixi run -e docs doc
```

--------------------------------

### Minimal Implementation for Dataset Preparation

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-29-dataloader-bench.md

Implements `prepare_datasets` to create `.gvl` datasets for different region lengths. It uses `generate_bed` to create the BED files and `genvarloader.write` to create the datasets. Datasets are stored in a temporary directory.

```python
from pathlib import Path


def prepare_datasets(
    region_lengths: list[int],
    svar_path: str | Path,
    regions_bed: str | Path,
    tmp_dir: str | Path,
) -> dict[int, Path]:
    """Write one fresh ``.gvl`` dataset per region length, keyed by length.

    Amortized once at bench startup. Returns ``{length: dataset_path}``.
    """
    import genvarloader as gvl

    tmp_dir = Path(tmp_dir)
    tmp_dir.mkdir(parents=True, exist_ok=True)
    out: dict[int, Path] = {}
    for length in region_lengths:
        bed = generate_bed(regions_bed, length)
        ds_path = tmp_dir / f"dataset_rL{length}.gvl"
        gvl.write(
            path=ds_path,
            bed=bed,
            variants=Path(svar_path),
            overwrite=True,
        )
        out[length] = ds_path
    return out
```

--------------------------------

### Test Table Initialization from Path with Per-Sample Dictionary

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-08-tracks-generalize.md

Confirms that `Table.from_path` can correctly load data when provided with a dictionary mapping sample IDs to individual file paths.

```python
def test_table_from_path_per_sample_dict(long_df, tmp_path):
    s0 = long_df.filter(pl.col("sample_id") == "s0").drop("sample_id")
    s1 = long_df.filter(pl.col("sample_id") == "s1").drop("sample_id")
    p0 = tmp_path / "s0.parquet"
    p1 = tmp_path / "s1.parquet"
    s0.write_parquet(p0)
    s1.write_parquet(p1)
    t = Table.from_path("signal", {"s0": p0, "s1": p1})
    assert t.samples == ["s0", "s1"]
```

--------------------------------

### RTK for Kubectl Get

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Use 'rtk kubectl get' for a compact list of Kubernetes resources.

```bash
rtk kubectl get         # Compact resource list
```

--------------------------------

### Create Test Directories and .gitkeep Files

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phases-1-3.md

Creates the necessary directories for unit and integration tests and adds .gitkeep files to ensure they are tracked by Git.

```bash
mkdir -p tests/unit tests/integration tests/_builders
touch tests/unit/.gitkeep tests/integration/.gitkeep
```

--------------------------------

### Profiling Script Configuration Example

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/specs/2026-05-29-codspeed-perf-tracking-design.md

Example of how the profiling script is invoked with different modes to analyze specific components under single-threaded execution.

```bash
NUMBA_NUM_THREADS=1 python profiling/profile.py --mode haplotypes
NUMBA_NUM_THREADS=1 python profiling/profile.py --mode tracks
NUMBA_NUM_THREADS=1 python profiling/profile.py --mode variants

```

--------------------------------

### End-to-end GenVarLoader Dataset Workflow

Source: https://github.com/mcvickerlab/genvarloader/blob/main/skills/genvarloader/SKILL.md

Demonstrates the complete process of writing a GenVarLoader dataset, opening it with various configurations, and performing eager indexing.

```python
import genvarloader as gvl

# 1. Preprocess variants outside Python (see "Variant preprocessing")
# 2. Write the dataset
gvl.write(
    path="ds.gvl",
    bed="rois.bed",
    variants="normed.bcf",  # or .pgen, or .svar directory
    tracks=[gvl.BigWigs.from_table("signal", "bw_table.tsv")],
    max_jitter=128,
)

# 3. Open and configure (chainable fluent API)
ds = (
    gvl.Dataset
    .open("ds.gvl", reference="ref.fa")
    .with_seqs("haplotypes")
    .with_tracks(["signal"])
    .with_insertion_fill(gvl.Repeat5pNormalized())
    .with_len(2048)  # or "ragged" / "variable"
    .with_settings(jitter=32, deterministic=False)
)

# 4. Eager indexing: dataset[region_idx, sample_idx]
batch = ds[0:8, :]  # shape depends on with_* state — see "Output shapes"

```

--------------------------------

### Test Zero-Based Start Conversion

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-08-get-splice-bed.md

Ensures that the 1-based start positions from the GTF file are correctly converted to 0-based chromStart in the BED output.

```python
def test_zero_based_start(gtf_path: Path):
    """GTF starts (1-based) become BED chromStart (0-based) by subtracting 1."""
    bed = gvl.get_splice_bed(gtf_path)
    starts = bed.sort("chromStart")["chromStart"].to_list()
    # T1 had GTF starts 100 and 300 -> 99 and 299
    assert starts == [99, 299]
```

--------------------------------

### Run Pre-push Hooks and Tests

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-23-refactor-campaign-pr0-pyrefly.md

Execute the pre-push verification and testing commands. These commands ensure code quality and correctness before pushing changes. The `prek run --all-files` command covers static analysis tools like Ruff and Pyrefly, while `pixi run -e dev test` runs the project's test suite.

```bash
pixi run -e dev prek run --all-files --show-diff-on-failure
```

```bash
pixi run -e dev test
```

--------------------------------

### Verify Pyrefly Executability

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-23-refactor-campaign-pr0-pyrefly.md

Tests if the 'pyrefly' command can be executed successfully after installation in the development environment. This confirms that the installation was successful and the command is in the system's PATH.

```bash
pixi run -e dev pyrefly --version
```

--------------------------------

### Test intervals_to_tracks with offset query start

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-25-test-coverage-implementation.md

Tests the `intervals_to_tracks` function when the query starts at a non-zero offset. Verifies that intervals are correctly interpreted in absolute coordinates.

```python
def test_intervals_to_tracks_offset_query_start():
    """Query starts at non-zero — intervals are in absolute coords."""
    offset_idxs = np.array([0], dtype=np.intp)
    starts = np.array([10], dtype=np.int32)
    itv_starts = np.array([11], dtype=np.int32)
    itv_ends = np.array([13], dtype=np.int32)
    itv_values = np.array([7.0], dtype=np.float32)
    itv_offsets = np.array([0, 1], dtype=np.int64)
    out = np.empty(4, dtype=np.float32)
    out_offsets = np.array([0, 4], dtype=np.int64)

    intervals_to_tracks(
        offset_idxs,
        starts,
        itv_starts,
        itv_ends,
        itv_values,
        itv_offsets,
        out,
        out_offsets,
    )
    np.testing.assert_equal(out, np.array([0.0, 7.0, 7.0, 0.0], dtype=np.float32))
```

--------------------------------

### Initialize `insertion_fill` in `Tracks.from_path`

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-11-track-insertion-options.md

Modify the `Tracks.from_path` method to initialize the `insertion_fill` field with `Repeat5p()` for each active track.

```python
        insertion_fill = {name: Repeat5p() for name in all_tracks}
        return cls(
            intervals,
            all_tracks,
            all_tracks,
            kind,
            n_regions,
            n_samples,
            insertion_fill,
        )
```

--------------------------------

### Test get_diffs_sparse Slow Path: Deletion Clipped at Start

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-25-test-coverage-implementation.md

Tests the slow path of `get_diffs_sparse` where a deletion starts before the query region, clipping the counted length.

```python
def test_get_diffs_spanning_del_clipped_at_start():
    """Deletion starts before region — only the in-region portion counts."""
    geno_offset_idx = np.array([[0]], dtype=np.intp)
    geno_v_idxs = np.array([0], dtype=np.int32)
    geno_offsets = np.array([0, 1], dtype=np.int64)
    v_starts = np.array([0], dtype=np.int32)
    ilens = np.array([-3], dtype=np.int32)  # deletes 3bp starting at pos 0
    q_starts = np.array([2], dtype=np.int32)  # region starts inside the deletion
    q_ends = np.array([10], dtype=np.int32)

    diffs = get_diffs_sparse(
        geno_offset_idx=geno_offset_idx,
        geno_v_idxs=geno_v_idxs,
        geno_offsets=geno_offsets,
        ilens=ilens,
        q_starts=q_starts,
        q_ends=q_ends,
        v_starts=v_starts,
    )

    # Atomized DEL: v_end = 0 - min(0, -3) + 1 = 4
    # v_ilen += max(0, q_starts - v_start - 1) = max(0, 2 - 0 - 1) = 1 -> -2
    # v_ilen += max(0, v_end - q_ends) = max(0, 4 - 10) = 0
    # final -> -2
    np.testing.assert_equal(diffs, np.array([[-2]], dtype=np.int32))
```

--------------------------------

### Run initial test suite baseline

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phase5-ref-fasta.md

Execute the test file before any changes to establish a baseline. This verifies the current state of the tests.

```bash
pixi run -e dev pytest tests/integration/test_ref_ds.py -v
```

--------------------------------

### FASTA Cache Building with Atomic Publish

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/specs/2026-06-02-robust-ondisk-artifacts-design.md

Illustrates the process of ensuring a FASTA cache, using `atomic_dir` for safe publishing. Includes lock acquisition and a double-check mechanism.

```python
from contextlib import contextmanager
from typing import Iterator
from pathlib import Path

# Assume necessary imports and definitions like DEFAULT_LOCK_TIMEOUT, etc.

# Placeholder for the actual atomic_dir context manager
@contextmanager
def atomic_dir(dest: Path, *, overwrite: bool = False, lock: bool = True, timeout: float = 60.0) -> Iterator[Path]:
    # ... implementation ...
    pass

def ensure_cache(gvlfa_dir: Path, fasta_path: Path):
    # ... other logic for dispatching build/migrate/rebuild/reuse ...

    # If a build is needed:
    T = 60.0 # Example timeout
    with atomic_dir(gvlfa_dir, lock=True, timeout=T) as tmp:
        # lock acquired (or timed out -> proceed)
        # double-check: re-classify gvlfa_dir; if another job just published a fresh cache, skip building and reuse it
        # else write sequence.bin + metadata.json into tmp
        pass # Placeholder for writing cache files
    # context exit -> os.replace(tmp, gvlfa_dir)
    # ... return meta, data_path ...
    pass # Placeholder for actual implementation
```

--------------------------------

### Install GenVarLoader with Local Genoray

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-19-max-mem-index-aware.md

Installs a local, editable version of the genoray library into the development environment using pixi and uv/pip. This is a prerequisite for developing against the latest genoray changes without waiting for a release.

```bash
cd /Users/david/projects/GenVarLoader
rtk git checkout -b feat/max-mem-index-aware
```

```bash
pixi run -e dev uv pip install -e /Users/david/projects/genoray
```

```bash
pixi run -e dev pip install -e /Users/david/projects/genoray
```

--------------------------------

### Install Dependencies with Pixi

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-08-tracks-generalize.md

This command refreshes the project's environment using pixi, ensuring that all declared dependencies, including the newly added `polars-bio`, are installed and the lockfile is updated. This is crucial for maintaining a consistent development environment.

```bash
pixi install -e dev
```

--------------------------------

### Add Spliced Reference-Only Dataset Example

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-21-refdataset-splicing.md

Append a new section to the splicing.ipynb notebook demonstrating how to create and use a RefDataset for spliced reference-only sequences. This example shows initializing RefDataset with a reference and splice BED, then retrieving sequences per transcript.

```python
import genvarloader as gvl

ref = gvl.Reference.from_path("hg38.fa.bgz")
bed = gvl.get_splice_bed("annotations.gtf")
ref_ds = gvl.RefDataset(ref, bed, splice_info="transcript_id")
seqs = ref_ds[:]  # Ragged[S1], one row per transcript

```

--------------------------------

### RTK for Cargo Build and Check

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Use 'rtk cargo build' and 'rtk cargo check' for token-optimized build and check outputs.

```bash
rtk cargo build         # Cargo build output
rtk cargo check         # Cargo check output
```

--------------------------------

### Get latest commit hash

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phase5-dataset-polymorphism.md

Retrieve the short hash of the latest commit in the git log.

```bash
git log -1 --format=%h
```

--------------------------------

### genvarloader.get_dummy_dataset

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/source/api.md

Function to get a dummy dataset. Specific details on its usage and parameters are not provided in the source.

```APIDOC
## get_dummy_dataset

### Description
Function to get a dummy dataset.

### Method
Not specified

### Endpoint
Not specified

### Parameters
Not specified

### Request Example
Not specified

### Response
Not specified
```

--------------------------------

### Dataset Writing with Atomic Publish

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/specs/2026-06-02-robust-ondisk-artifacts-design.md

Demonstrates writing a dataset using `atomic_dir` to ensure all components are published atomically. Handles overwrites and potential interruptions.

```python
from contextlib import contextmanager
from typing import Iterator
from pathlib import Path

# Assume necessary imports and definitions like DEFAULT_LOCK_TIMEOUT, etc.

# Placeholder for the actual atomic_dir context manager
@contextmanager
def atomic_dir(dest: Path, *, overwrite: bool = False, lock: bool = True, timeout: float = 60.0) -> Iterator[Path]:
    # ... implementation ...
    pass

def write_dataset(path: Path, overwrite: bool = False):
    T = 60.0 # Example timeout
    with atomic_dir(path, overwrite=overwrite, lock=True, timeout=T) as tmp:
        # existing-dest handled up front (FileExistsError unless overwrite)
        # every write (input_regions.arrow, regions.npy, genotypes/*, intervals/**, metadata.json with format_version) targets `tmp`
        pass # Placeholder for writing dataset files to tmp
    # exit -> atomic publish to path. An interrupted write leaves only an orphan <path>.tmp.*, never a half-written path.
    pass # Placeholder for actual implementation
```

--------------------------------

### RTK for Cargo Clippy

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Use 'rtk cargo clippy' to get token-optimized Clippy warnings, grouped by file.

```bash
rtk cargo clippy        # Clippy warnings grouped by file (80%)
```

--------------------------------

### Run tests and commit changes for Dataset.open

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-25-test-coverage-implementation.md

Shell commands to execute the tests for Dataset.open error paths and commit the changes.

```shell
pixi run -e dev pytest tests/dataset/test_open.py -v
rtk git add tests/dataset/test_open.py
rtk git commit -m "test(open): cover error paths on Dataset.open"
```

--------------------------------

### Uninstall Genoray and Refresh Lockfile

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-19-max-mem-index-aware.md

Removes the editable install of genoray and then updates the pixi lockfile to resolve the new version.

```bash
pixi run -e dev pip uninstall -y genoray
pixi install -e dev
```

--------------------------------

### Test Table Initialization from Path with Unknown Extension

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-08-tracks-generalize.md

Ensures that `Table.from_path` raises a ValueError when attempting to load a file with an unsupported or unknown file extension.

```python
def test_table_from_path_unknown_extension(tmp_path):
    p = tmp_path / "data.bogus"
    p.write_text("nope")
    with pytest.raises(ValueError, match="extension"):
        Table.from_path("signal", p)
```

--------------------------------

### Enable GenVarLoader Logging

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/source/geuvadis.ipynb

Configures loguru to display GenVarLoader messages at the INFO level. Ensure loguru is installed and imported.

```python
logger.remove()
logger.add(sys.stderr, level="INFO")
logger.enable("genvarloader")
```

--------------------------------

### Running Local Benchmarks

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/HANDOFF-2026-05-30-bench-codspeed-profiling.md

Execute the local benchmark suite using pixi. This command runs the defined local benchmarks.

```bash
pixi run -e dev bench-local
```

--------------------------------

### Test BED Generation and Dataset Preparation

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-29-dataloader-bench.md

This test verifies that `prepare_datasets` writes one `.gvl` dataset per specified region length and that each dataset contains a `metadata.json` file. It also checks that `generate_bed` correctly resizes regions to the target length.

```python
import pytest


@pytest.mark.slow
def test_prepare_datasets_writes_one_gvl_per_region_length(tmp_path):
    repo = Path(__file__).resolve().parents[3]
    svar = repo / "tests" / "data" / "1kg" / "filtered.svar"
    regions = repo / "tests" / "data" / "1kg" / "regions.bed"
    if not svar.is_dir():
        pytest.skip("missing tests/data/1kg/filtered.svar; run pixi run -e dev gen")

    paths = C.prepare_datasets([1_000, 2_500], svar, regions, tmp_path)

    assert set(paths) == {1_000, 2_500}
    for length, p in paths.items():
        assert p.is_dir(), p
        assert (p / "metadata.json").exists()


@pytest.mark.slow
def test_generate_bed_resizes_to_target_length():
    import seqpro as sp

    repo = Path(__file__).resolve().parents[3]
    regions = repo / "tests" / "data" / "1kg" / "regions.bed"
    if not regions.exists():
        pytest.skip("missing regions.bed")

    bed = C.generate_bed(regions, 2_500)
    lengths = (bed["chromEnd"] - bed["chromStart"])
    assert lengths == [2_500]
    assert bed.height == 100  # regions.bed has 100 regions
```

--------------------------------

### Get Reference Data (Un-spliced)

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-22-splice-zero-copy.md

Retrieves reference data, casting it to a Ragged array of bytes. This is used for the non-spliced path.

```python
ref = get_reference(
            regions=regions,
            out_offsets=out_offsets,
            reference=self.reference.reference,
            ref_offsets=self.reference.offsets,
            pad_char=self.reference.pad_char,
        ).view("S1")
        ref = cast(
            Ragged[np.bytes_],
            Ragged.from_offsets(ref, (batch_size, None), out_offsets),
        )
        return ref
```

--------------------------------

### List Files in tests/_builders/

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phase5-reconstruct.md

Lists the files in the tests/_builders directory to verify the presence of necessary builder files, such as 'reconstruct.py'.

```bash
/bin/ls tests/_builders/
```

--------------------------------

### Converting Ragged to Nested Tensor

Source: https://github.com/mcvickerlab/genvarloader/blob/main/skills/genvarloader/SKILL.md

Convert a ragged data structure into a PyTorch nested tensor. This requires the PyTorch library to be installed.

```python
gvl.to_nested_tensor(ragged)
```

--------------------------------

### Dataset Opening Process

Source: https://github.com/mcvickerlab/genvarloader/blob/main/CLAUDE.md

Describes the steps involved in opening a dataset for reading, including metadata loading and lazy reader initialization.

```python
# Reading: Dataset.open(path, reference?) → RaggedDataset
# 1. Loads metadata and region index map
# 2. Initializes lazy readers (Haps from genotypes, or Ref from reference)
# 3. Eager indexing dataset[region_idx, sample_idx] triggers data loading
```

--------------------------------

### Run and Commit Choose Exonic Variants Tests

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-25-test-coverage-implementation.md

Commands to execute the unit tests for `choose_exonic_variants` and then add and commit the modified test file.

```bash
pixi run -e dev pytest tests/unit/dataset/genotypes/test_choose_exonic_variants.py -v
rtk git add tests/unit/dataset/genotypes/test_choose_exonic_variants.py
rtk git commit -m "test(kernels): fill choose_exonic_variants scenario gaps"
```

--------------------------------

### Run Pytest and Get Latest Git Hash

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phase5-ref-fasta.md

Execute unit tests and capture the latest commit hash for documentation updates.

```bash
pixi run -e dev pytest tests/unit -q 2>&1 | tail -3
pixi run -e dev pytest -q 2>&1 | tail -3
git log -1 --format=%h
```

--------------------------------

### Verify DataLoader Prerequisite

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-29-dataloader-bench.md

This command verifies that the Dataset.to_dataloader method accepts 'mode' and 'buffer_bytes' parameters, and that the _output_bytes_per_instance method exists. It's crucial for ensuring the DataLoader API is ready before running the benchmark.

```bash
pixi run -e dev python -c "import genvarloader as gvl, inspect; sig=inspect.signature(gvl.Dataset.to_dataloader); assert 'mode' in sig.parameters and 'buffer_bytes' in sig.parameters, sig; assert hasattr(gvl.Dataset, '_output_bytes_per_instance'); print('prereq OK')"
```

--------------------------------

### Check Phased Genotypes with plink2

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/source/faq.md

Provides a command-line example using `plink2` to check genotype phasing information from a PLINK prefix.

```bash
# for PLINK
plink2 --pgen-info $prefix
```

--------------------------------

### Test Fasta Zero-Length Range Read

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-25-test-coverage-implementation.md

Verify that reading a zero-length range (start == end) from Fasta returns an empty sequence.

```python
def test_fasta_zero_length_range(ref_fasta):
    """start == end -> empty result."""
    fasta = Fasta("ref", ref_fasta, pad="N")
    seq = fasta.read("chr1", 100, 100)
    assert len(seq) == 0
```

--------------------------------

### Verify Docs Build

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-19-max-mem-index-aware.md

Run the documentation build command to ensure that the docstring changes do not cause any build errors.

```bash
pixi run -e docs doc
```

--------------------------------

### Smoke Test for SpliceMap.from_bed

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-21-refdataset-splicing.md

This script tests the `SpliceMap.from_bed` method with different grouping keys and verifies the output of `parse_rows`. It requires `polars` and `genvarloader` to be installed.

```python
import polars as pl
from genvarloader._dataset._splice import SpliceMap

bed = pl.DataFrame({
    "chrom": ["chr1"] * 4,
    "chromStart": [0, 100, 200, 300],
    "chromEnd": [10, 110, 210, 310],
    "transcript_id": ["T1", "T1", "T2", "T2"],
    "exon_number": [1, 2, 1, 2],
})
sm, sp_bed = SpliceMap.from_bed("transcript_id", bed)
assert sm.n_rows == 2
print("ok", sm.n_rows, sp_bed.height)

sm2, sp_bed2 = SpliceMap.from_bed(("transcript_id", "exon_number"), bed)
assert sm2.n_rows == 2
print("ok2", sm2.n_rows)

flat, offsets, reshape, squeeze = sm.parse_rows([0, 1])
print("parse", flat, offsets, reshape, squeeze)
```

--------------------------------

### Download BED File

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/source/geuvadis.ipynb

Downloads the BED file containing eQTL gene annotations for chromosome 22 using pooch. Ensure pooch is installed.

```python
# BED
bed_path = pooch.retrieve(
    url="doi:10.5281/zenodo.13656224/chr22_egenes.bed",
    known_hash="md5:ccb55548e4ddd416d50dbe6638459421",
    progressbar=True,
)
```

--------------------------------

### RaggedVariants.to_packed() Crash Reproduction

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/specs/2026-06-07-ragged-variants-pack-lazy-views-design.md

Demonstrates scenarios where RaggedVariants.to_packed() fails with an AttributeError on sliced or reordered views. These examples highlight the problem with indexed or ListArray layouts.

```python
rv = <freshly built RaggedVariants>          # canonical: to_packed() OK
rv[::-1].to_packed()                          # AttributeError: 'IndexedArray' has no 'size'
rv[perm].to_packed()                          # same
<explicit ListArray variant level>.to_packed()# AttributeError: 'ListArray' has no 'offsets'
```

--------------------------------

### View Git Log

Source: https://github.com/mcvickerlab/genvarloader/blob/main/docs/superpowers/plans/2026-05-24-test-suite-overhaul-phases-1-3.md

Display the last 20 commits in one-line format. This is used to verify the number and formatting of commits since the plan started.

```bash
git log --oneline -20
```