### Accessor Usage Examples Source: https://github.com/scverse/anndata/blob/main/docs/accessors.rst Examples demonstrating how to use accessors to create references to AnnData arrays. ```APIDOC ## Accessor Usage Examples ### Description Examples demonstrating how to use accessors to create references to AnnData arrays. ### Code Examples Create references to arrays within an AnnData object using the `A` accessor. ```python from anndata.acc import A # Reference to `adata[:, "gene-3"].X` as a 1D vector ref1 = A.X[:, "gene-3"] # Reference to `adata.varm["PCs"]` ref2 = A.varm["PCs"] # Reference to `adata.obs["louvain"]` ref3 = A.obs["louvain"] # Reference to `adata.obsm["pca"][:, 0]` ref4 = A.obsm["pca"][:, 0] # Reference to `adata.obsp["connectivities"]` with multiple indices ref5 = A.obsp["connectivities"][:, ["cell0", "cell1"]] ``` Inspect properties of references. ```python # Inspecting properties of a reference print(ref4.idx) # Output: 0 print(ref4.acc) # Output: A.obsm['pca'] print(A.var["symbol"].dims) # Output: {{'var'}} print(ref4.acc.k) # Output: 'pca' ``` Check for existence and extract data using references. ```python # Check if a reference exists in an AnnData object # Assuming 'adata' is an AnnData object # print(ref2[:, 30] in adata) # Extract data using a reference # print(adata[ref3].categories[:2]) ``` ``` -------------------------------- ### Show benchmarks for a pattern Source: https://github.com/scverse/anndata/blob/main/benchmarks/README.md Use 'asv show' to view benchmarks. Filter results by a pattern using the -b flag. This example shows benchmarks related to 'views'. ```bash asv show -b "views" ``` -------------------------------- ### Install Zarrs Python Package Source: https://github.com/scverse/anndata/blob/main/docs/tutorials/zarr-v3.md Install the `zarrs` package to utilize the Rust implementation of Zarr for potential performance improvements in writing and sharding. ```bash uv pip install zarrs ``` -------------------------------- ### Custom Reader and Writer Examples Source: https://context7.com/scverse/anndata/llms.txt Demonstrates how to implement custom readers and writers for AnnData objects, such as converting float32 to float64 or logging element names during writing. ```APIDOC ## Custom Reader: Convert float32 to float64 ### Description This function `float64_reader` can be used as a callback during data reading to ensure all float32 arrays are converted to float64. ### Usage ```python import h5py import numpy as np # Assuming read_dispatched and adata are defined elsewhere # with h5py.File("test_dispatch.h5ad", "r") as f: # adata_loaded = read_dispatched(f, callback=float64_reader) # print(adata_loaded.X.dtype) # Expected output: float64 ``` ## Custom Writer: Log Element Names ### Description This function `logging_writer` logs the names of elements being written to storage, useful for tracking data flow. ### Usage ```python import h5py # Assuming write_dispatched, adata, and written are defined elsewhere # with h5py.File("logged.h5ad", "w") as f: # write_dispatched(f, "/", adata, callback=logging_writer) # print([w for w in written if w]) # Expected output: ["/X", "/obs", "/var", ...] ``` ``` -------------------------------- ### Concrete Transcript Model Example Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Provides a concrete example of a transcript model, showing specific values for tx_id, seq_name, exon_seq_start, exon_seq_end, and ensembl_id. This serves as a sample data point conforming to the defined type. ```python transcript_models[0] { x_id: 'ENST00000450305', seq_name: '1', exon_seq_start: [12010, 12179, 12613, 12975, 13221, 13453], exon_seq_end: [12057, 12227, 12697, 13052, 13374, 13670], ensembl_id: 'ENSG00000223972'} ``` -------------------------------- ### MetaAcc Reference Creation Example Source: https://github.com/scverse/anndata/blob/main/docs/accessors.rst Shows how to create references to observation (`obs`) or variable (`var`) metadata using `MetaAcc`. This is achieved through `A.obs[key]` or `A.var[key]`. ```python A.obs["a"], A.var["b"] ``` -------------------------------- ### Show benchmarks for a specific commit Source: https://github.com/scverse/anndata/blob/main/benchmarks/README.md View benchmark results for a particular commit by specifying the commit hash after the pattern. This example shows 'views' benchmarks for commit '0ebe187e'. ```bash asv show -b "views" 0ebe187e ``` -------------------------------- ### Example Data Structure Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Illustrates a list of dictionaries, each representing a transcript with associated exon sequence start and end positions. This format is used for representing genomic transcript data. ```python {tx_id: 'ENST00000477740', seq_name: '1', exon_seq_start: [...], ...}, {tx_id: 'ENST00000495576', seq_name: '1', exon_seq_start: [...], ...}] ``` -------------------------------- ### Compare benchmarks between two commits Source: https://github.com/scverse/anndata/blob/main/benchmarks/README.md Use 'asv compare' to compare benchmark results between two commits. This example compares commit 'e9ccfc' and '0ebe187e'. ```bash asv compare e9ccfc 0ebe187e ``` -------------------------------- ### LayerAcc Reference Creation Example Source: https://github.com/scverse/anndata/blob/main/docs/accessors.rst Demonstrates how to create a reference to a layer within an AnnData object using `LayerAcc`. This is done via `A.layers[key]`. ```python A.X[:, :], A.layers["c"][:, "g0"] ``` -------------------------------- ### GraphAcc Reference Creation Example Source: https://github.com/scverse/anndata/blob/main/docs/accessors.rst Demonstrates creating references to graph-related arrays like `obsp` or `varp` using `GraphAcc`. This typically involves specifying a key and then indices, e.g., `A.obsp[key][index1, index2]`. ```python A.obsp["e"][:, "c1"], A.vbsp["e"]["g0", :] ``` -------------------------------- ### Write AnnData to Zarr and H5AD Source: https://github.com/scverse/anndata/blob/main/tests/data/archives/v0.11.4/readme.md Use this command to create an AnnData object and write it to both Zarr (within a zip archive) and H5AD formats. Ensure you have anndata and zarr installed. ```bash uvx '--with=anndata==0.11.4' '--with=zarr<3' python -c ' import zarr from anndata import AnnData adata = AnnData(shape=(10, 20)) adata.write_zarr(zarr.ZipStore("tests/data/archives/v0.11.4/adata.zarr.zip")) adata.write_h5ad("tests/data/archives/v0.11.4/adata.h5ad")' ``` -------------------------------- ### MultiAcc Reference Creation Example Source: https://github.com/scverse/anndata/blob/main/docs/accessors.rst Illustrates creating references to multi-dimensional arrays like `obsm` or `varm` using `MultiAcc`. This involves accessing a specific key and then an index, e.g., `A.obsm[key][:, index]`. ```python A.obsm["d"][:, 2] ``` -------------------------------- ### Publish and preview benchmarks Source: https://github.com/scverse/anndata/blob/main/benchmarks/README.md Use 'asv publish' to generate benchmark results and 'asv preview' to view them in a browser. Add local branches to 'asv.conf.json' to include them. ```bash asv publish asv preview ``` -------------------------------- ### Build and Inspect AnnData Object Source: https://context7.com/scverse/anndata/llms.txt Demonstrates how to construct an AnnData object from scratch using various data structures and inspect its basic properties. Use this for initializing AnnData with experimental data. ```python import anndata as ad import numpy as np import pandas as pd from scipy.sparse import csr_matrix # ── Build an AnnData from scratch ────────────────────────────────────────── n_obs, n_vars = 200, 500 X = csr_matrix(np.random.poisson(1, (n_obs, n_vars)).astype(np.float32)) obs = pd.DataFrame( {"cell_type": np.random.choice(["T cell", "B cell", "NK"], n_obs), "batch": np.random.choice(["batch1", "batch2"], n_obs)}, index=[f"cell_{i}" for i in range(n_obs)], ) var = pd.DataFrame( {"highly_variable": np.random.choice([True, False], n_vars)}, index=[f"gene_{i}" for i in range(n_vars)], ) adata = ad.AnnData( X=X, obs=obs, var=var, uns={"experiment": "scRNA-seq demo"}, obsm={"X_pca": np.random.randn(n_obs, 50)}, ) print(adata) # AnnData object with n_obs × n_vars = 200 × 500 # obs: 'cell_type', 'batch' # var: 'highly_variable' # uns: 'experiment' # obsm: 'X_pca' # ── Inspection ────────────────────────────────────────────────────────────── print(adata.shape) # (200, 500) print(adata.obs_names[:3]) # Index(['cell_0', 'cell_1', 'cell_2'], dtype='object') print(adata.var_names[:3]) # Index(['gene_0', 'gene_1', 'gene_2'], dtype='object') print(adata.obs["cell_type"].value_counts()) # ── Subsetting / views ────────────────────────────────────────────────────── t_cells = adata[adata.obs["cell_type"] == "T cell", :] print(t_cells) # View of AnnData … t_cells_copy = t_cells.copy() # materialise view into own object # ── Layers ────────────────────────────────────────────────────────────────── adata.layers["counts"] = adata.X.copy() adata.layers["log1p"] = np.log1p(adata.X.toarray()) print(list(adata.layers)) # ['counts', 'log1p'] # ── Transpose ─────────────────────────────────────────────────────────────── adata_T = adata.T # variables become observations print(adata_T.shape) # (500, 200) ``` -------------------------------- ### Run benchmarks for a specific commit Source: https://github.com/scverse/anndata/blob/main/benchmarks/README.md Execute benchmarks for a given commit using 'asv run'. The --steps 1 and -b flags can be used for specific configurations. ```bash asv run {commit} --steps 1 -b ``` -------------------------------- ### Run benchmarks for a range of commits Source: https://github.com/scverse/anndata/blob/main/benchmarks/README.md Execute benchmarks for a range of commits using 'asv run' with the '..' operator. ```bash asv run {commit1}..{commit2} ``` -------------------------------- ### Create a Custom Matplotlib Reference Accessor Source: https://github.com/scverse/anndata/blob/main/docs/accessors.rst Subclass `AdRef` and `str` to create a custom reference accessor that Matplotlib can use. This example demonstrates how to integrate custom references with AnnData's accessor system for plotting. ```python from matplotlib import pyplot as plt from anndata.acc import AdAcc, AdRef class MplRef(AdRef, str): """Matplotlib will only treat strings as references, so we subclass `str`. """ def __new__(cls, acc, idx) -> None: obj = str.__new__(cls, str(AdRef(acc, idx))) AdRef.__init__(obj, acc, idx) return obj A = AdAcc(ref_class=MplRef) adata = sc.datasets.pbmc3k_processed() plt.scatter(*A.obsm["X_umap"][:, [0, 1]], c=A.obs["n_counts"], data=adata) ``` -------------------------------- ### AnnData - Core Annotated Data Container Source: https://context7.com/scverse/anndata/llms.txt Demonstrates the creation and basic usage of the AnnData object, including initialization, inspection, subsetting, and handling of layers. ```APIDOC ## AnnData ### Description The central class storing a data matrix together with obs/var annotations, multi-dimensional embeddings, pairwise arrays, layers, and unstructured metadata. Observations correspond to rows and variables to columns. ### Usage ```python import anndata as ad import numpy as np import pandas as pd from scipy.sparse import csr_matrix # Build an AnnData from scratch n_obs, n_vars = 200, 500 X = csr_matrix(np.random.poisson(1, (n_obs, n_vars)).astype(np.float32)) obs = pd.DataFrame( {"cell_type": np.random.choice(["T cell", "B cell", "NK"], n_obs), "batch": np.random.choice(["batch1", "batch2"], n_obs)}, index=[f"cell_{i}" for i in range(n_obs)], ) var = pd.DataFrame( {"highly_variable": np.random.choice([True, False], n_vars)}, index=[f"gene_{i}" for i in range(n_vars)], ) adata = ad.AnnData( X=X, obs=obs, var=var, uns={"experiment": "scRNA-seq demo"}, obsm={"X_pca": np.random.randn(n_obs, 50)}, ) print(adata) # Inspection print(adata.shape) print(adata.obs_names[:3]) print(adata.var_names[:3]) print(adata.obs["cell_type"].value_counts()) # Subsetting / views t_cells = adata[adata.obs["cell_type"] == "T cell", :] print(t_cells) # Layers adata.layers["counts"] = adata.X.copy() adata.layers["log1p"] = np.log1p(adata.X.toarray()) print(list(adata.layers)) # Transpose adata_T = adata.T print(adata_T.shape) ``` ``` -------------------------------- ### On-Disk Sparse Matrix Interface with `sparse_dataset` Source: https://context7.com/scverse/anndata/llms.txt Wrap a sparse matrix stored in HDF5 or Zarr into a `BaseCompressedSparseDataset` for slice-based access without loading the entire matrix. This enables random access into backed sparse data. ```python import anndata as ad import h5py import numpy as np from scipy.sparse import csr_matrix # Write a sparse matrix matrix = csr_matrix(np.eye(1000, 2000, dtype=np.float32)) with h5py.File("sparse.h5", "w") as f: ad.io.write_elem(f, "X", matrix) # Access slices without full load with h5py.File("sparse.h5", "r") as f: ds = ad.io.sparse_dataset(f["X"]) print(ds.shape) # (1000, 2000) print(ds.format) # 'csr' row_slice = ds[0:10] # returns a scipy sparse matrix for rows 0-9 print(row_slice.shape) # (10, 2000) print(type(row_slice)) # ``` -------------------------------- ### Create a New AdAcc Subclass with Custom Accessors Source: https://github.com/scverse/anndata/blob/main/docs/accessors.rst Subclass `AdAcc` to define new accessors. This example shows how to add a 'tem' accessor using `MetaAcc` within a frozen dataclass, demonstrating initialization and attribute setting for custom accessors. ```python from dataclasses import dataclass, field from anndata.acc import AdAcc, MetaAcc @dataclass(frozen=True) class EHRAcc(AdAcc): tem: MetaAcc = field(init=False) def __post_init__(self) -> None: super().__post_init__() tem = MetaAcc("tem", ref_class=self.ref_class) object.__setattr__(self, "tem", tem) # necessary because it’s frozen A = EHRAcc() A.tem["visit_id"] ``` -------------------------------- ### Type Definition for Transcript Models Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Defines the type for transcript models, specifying optional string fields for tx_id, seq_name, and ensembl_id, and optional lists of integers for exon sequence start and end positions. This schema is used for validating transcript data. ```python type: 5 * { tx_id: ?string, seq_name: ?string, exon_seq_start: option[var * ?int64], exon_seq_end: option[var * ?int64], ensembl_id: ?string } ``` -------------------------------- ### Class Overview Source: https://github.com/scverse/anndata/blob/main/docs/_templates/autosummary/class-minimal.rst This section details the class structure, including its inheritance, attributes, and methods. ```APIDOC .. currentmodule:: {{ module }} .. autoclass:: {{ objname }} :show-inheritance: {% block attributes %} {%- for item in attributes %} {%- if loop.first %} .. rubric:: Attributes {% endif %} .. autoattribute:: {{ item }} {%- endfor %} {% endblock %} {% block methods %} {%- for item in methods if item != "__init__" and item not in inherited_members %} {%- if loop.first %} .. rubric:: Methods {% endif %} .. automethod:: {{ item }} {%- endfor %} {% endblock %} ``` -------------------------------- ### anndata.settings - Global Configuration Source: https://context7.com/scverse/anndata/llms.txt Manage global configuration settings for the anndata package, with options for inspection, global modification, and temporary overrides using a context manager. ```APIDOC ## `anndata.settings` — Global Configuration ### Description Control package-wide behaviour through a typed settings object. Settings can be temporarily overridden with a context manager. ### Inspect Defaults ```python import anndata as ad print(ad.settings) # check_uniqueness: bool # ... (default: True) # ... ``` ### Modify Globally ```python import anndata as ad ad.settings.check_uniqueness = False ad.settings.remove_unused_categories = True ``` ### Temporary Override via Context Manager ```python import anndata as ad import numpy as np from scipy.sparse import csr_matrix adata = ad.AnnData(X=csr_matrix(np.eye(3))) with ad.settings.override(check_uniqueness=False): # duplicate obs_names are allowed here adata2 = ad.AnnData( X=csr_matrix(np.eye(3)), obs={"batch": ["a", "a", "b"]} # duplicate not checked ) print(adata2.obs) ``` ### Environment Variable Control Set environment variables to control settings, e.g.: ```bash ANNDATA_CHECK_UNIQUENESS=0 python script.py ``` ``` -------------------------------- ### Display String Array in HDF5 Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Shows how a string array is represented in HDF5. This is useful for understanding storage. ```python >>> store["var"][store["var"].attrs["_index"]] ``` -------------------------------- ### Display String Array in Zarr Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Shows how a string array is represented in Zarr. This is useful for understanding storage. ```python >>> store["var"][store["var"].attrs["_index"]] ``` -------------------------------- ### Load Dataset Source: https://github.com/scverse/anndata/blob/main/docs/benchmark-read-write.ipynb Loads the pbmc3k dataset using Scanpy. ```python adata = sc.datasets.pbmc3k() ``` -------------------------------- ### List Keys in HDF5 and Zarr Stores Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Opens an AnnData file in read mode and lists the top-level keys. Use this to inspect the structure of an on-disk AnnData object. ```python >>> import h5py >>> store = h5py.File("for-ondisk-docs/cart-164k-processed.h5ad", mode="r") >>> list(store.keys()) ['X', 'layers', 'obs', 'obsm', 'obsp', 'uns', 'var', 'varm', 'varp'] ``` ```python >>> import zarr >>> store = zarr.open("for-ondisk-docs/cart-164k-processed.zarr", mode="r") >>> list(store.keys()) ['X', 'layers', 'obs', 'obsm', 'obsp', 'uns', 'var', 'varm', 'varp'] ``` -------------------------------- ### Configure Zarr to Use Zarrs Codec Pipeline Source: https://github.com/scverse/anndata/blob/main/docs/tutorials/zarr-v3.md Configure `zarr-python` to use the `zarrs` codec pipeline for potentially faster I/O operations. This is recommended for writing full datasets and reading contiguous regions. ```python import zarr import zarrs zarr.config.set({"codec_pipeline.path": "zarrs.ZarrsCodecPipeline"}) ``` -------------------------------- ### Virtual Collection of AnnData Objects with `AnnCollection` Source: https://context7.com/scverse/anndata/llms.txt Create a lightweight virtual collection over multiple AnnData objects or `.h5ad` files using `experimental.AnnCollection`. This acts as a single large dataset without concatenating in memory, supporting lazy access and serving as the multi-file backend for `AnnLoader`. ```python import anndata as ad import numpy as np from scipy.sparse import csr_matrix import pandas as pd def make_adata(n, offset, genes): X = csr_matrix(np.random.rand(n, len(genes)).astype(np.float32)) obs = pd.DataFrame({"cell_type": ["T"] * n}, index=[f"c{i+offset}" for i in range(n)]) var = pd.DataFrame(index=genes) return ad.AnnData(X=X, obs=obs, var=var) a1 = make_adata(300, 0, ["g0", "g1", "g2"]) a2 = make_adata(200, 300, ["g0", "g1", "g2"]) coll = ad.experimental.AnnCollection([a1, a2]) print(len(coll)) # 500 print(coll.obs_names[:3]) # ['c0', 'c1', 'c2'] # Random-access a batch batch = coll[10:20] print(batch.X.shape) # (10, 3) # Use with AnnLoader for training loader = ad.experimental.AnnLoader(coll, batch_size=32, shuffle=True) for batch in loader: print(batch["X"].shape) break ``` -------------------------------- ### Mutate Existing Zarr Store with Consolidated Metadata Source: https://github.com/scverse/anndata/blob/main/docs/tutorials/zarr-v3.md Use this snippet to mutate an existing Zarr store on disk. Open the store unconsolidated, perform the mutation using `write_elem`, and then reconsolidate the metadata. Alternatively, delete the `.zmetadata` file before rewriting. ```python g = zarr.open_group(orig_path, mode="a", use_consolidated=False) ad.io.write_elem( g, "obs", obs, dataset_kwargs=dict(chunks=(250,)), ) zarr.consolidate_metadata(g.store) ``` -------------------------------- ### Set AnnData.X with JAX Array Source: https://github.com/scverse/anndata/blob/main/docs/release-notes/2071.feat.md Demonstrates setting the `.X` attribute of an AnnData object with a JAX NumPy array. This leverages array-API compatibility for seamless integration with JAX. ```python adata.X = jax.numpy.array([[0, 1], [2, 3]]) ``` -------------------------------- ### List Sparse Array Components (Zarr) Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Iterate through and print the names of the components of a sparse array stored in Zarr format. ```python store["X"].visititems(print) data indices indptr ``` -------------------------------- ### Settings API Source: https://github.com/scverse/anndata/blob/main/docs/api.md API for configuring AnnData settings. ```APIDOC ## Settings API ### Description API for configuring AnnData settings. ### Functions - settings - settings.override ``` -------------------------------- ### Virtual Collection of AnnData Objects Source: https://context7.com/scverse/anndata/llms.txt Create a lightweight virtual collection over multiple AnnData objects or `.h5ad` files that acts as a single large dataset without concatenating them in memory. Supports lazy access and is the multi-file backend for `AnnLoader`. ```APIDOC ## `experimental.AnnCollection` — Virtual Collection of AnnData Objects Create a lightweight virtual collection over multiple AnnData objects or `.h5ad` files that acts as a single large dataset without concatenating them in memory. Supports lazy access and is the multi-file backend for `AnnLoader`. ```python import anndata as ad import numpy as np from scipy.sparse import csr_matrix import pandas as pd def make_adata(n, offset, genes): X = csr_matrix(np.random.rand(n, len(genes)).astype(np.float32)) obs = pd.DataFrame({"cell_type": ["T"] * n}, index=[f"c{i+offset}" for i in range(n)]) var = pd.DataFrame(index=genes) return ad.AnnData(X=X, obs=obs, var=var) a1 = make_adata(300, 0, ["g0", "g1", "g2"]) a2 = make_adata(200, 300, ["g0", "g1", "g2"]) coll = ad.experimental.AnnCollection([a1, a2]) print(len(coll)) # 500 print(coll.obs_names[:3]) # ['c0', 'c1', 'c2'] # Random-access a batch batch = coll[10:20] print(batch.X.shape) # (10, 3) # Use with AnnLoader for training loader = ad.experimental.AnnLoader(coll, batch_size=32, shuffle=True) for batch in loader: print(batch["X"].shape) break ``` ``` -------------------------------- ### List Sparse Array Components (HDF5) Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Iterate through and print the names of the components of a sparse array stored in HDF5 format. ```python store["X"].visititems(print) data indices indptr ``` -------------------------------- ### Reading Matrix Market Files Source: https://context7.com/scverse/anndata/llms.txt Import an expression matrix from a Matrix Market (.mtx) file into an AnnData object. Note that .mtx files store variables x observations, and the function transposes the result. ```APIDOC ## read_mtx ### Description Reads an AnnData object from a Matrix Market (.mtx) file. ### Method GET ### Endpoint `ad.io.read_mtx(filename: str, backed: Optional[Literal['r', 'r+']] = None, ...)` ### Parameters #### Path Parameters - **filename** (str) - Required - Path to the .mtx file. ### Request Example ```python import anndata as ad adata_mtx = ad.io.read_mtx("matrix.mtx") print(adata_mtx.shape) ``` ### Response #### Success Response (200) - **AnnData object** - An AnnData object containing the data from the .mtx file. ``` -------------------------------- ### Import Libraries Source: https://github.com/scverse/anndata/blob/main/docs/benchmark-read-write.ipynb Imports necessary libraries for AnnData and Scanpy operations. ```python from __future__ import annotations import scanpy as sc import anndata as ad ``` -------------------------------- ### Reading Text Files Source: https://context7.com/scverse/anndata/llms.txt Import an expression matrix from a whitespace-delimited text file into an AnnData object. ```APIDOC ## read_text ### Description Reads an AnnData object from a whitespace-delimited text file. ### Method GET ### Endpoint `ad.io.read_text(filename: str, backed: Optional[Literal['r', 'r+']] = None, ...)` ### Parameters #### Path Parameters - **filename** (str) - Required - Path to the text file. ### Response #### Success Response (200) - **AnnData object** - An AnnData object containing the data from the text file. ``` -------------------------------- ### Write Nullable Integer Array to HDF5 Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Demonstrates writing a Pandas nullable integer array to an HDF5 file using `anndata.write_elem`. Shows the resulting group structure with 'mask' and 'values' datasets. ```python >>> from anndata import write_elem >>> null_store = h5py.File("tmp.h5", mode="w") >>> int_array = pd.array([1, None, 3, 4]) >>> int_array [1, , 3, 4] Length: 4, dtype: Int64 >>> write_elem(null_store, "nullable_integer", int_array) >>> null_store.visititems(print) nullable_integer nullable_integer/mask nullable_integer/values ``` -------------------------------- ### Experimental API Source: https://github.com/scverse/anndata/blob/main/docs/api.md Experimental classes and functions for advanced use cases, such as batched access and out-of-core operations. ```APIDOC ## Experimental API ### Description Experimental classes and functions for advanced use cases, such as batched access and out-of-core operations. ### Classes - experimental.AnnCollection - experimental.AnnLoader ### Functions - experimental.concat_on_disk - experimental.read_elem_lazy - experimental.read_lazy - experimental.read_dispatched - experimental.write_dispatched ``` -------------------------------- ### Write Nullable Integer Array to Zarr Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Demonstrates writing a Pandas nullable integer array to a Zarr store using `anndata.write_elem`. Shows the resulting group structure with 'mask' and 'values' arrays. ```python >>> from anndata import write_elem >>> null_store = zarr.open() >>> int_array = pd.array([1, None, 3, 4]) >>> int_array [1, , 3, 4] Length: 4, dtype: Int64 >>> write_elem(null_store, "nullable_integer", int_array) >>> null_store.visititems(print) nullable_integer nullable_integer/mask nullable_integer/values ``` -------------------------------- ### Inspect Ragged Array Structure in HDF5 Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Use `visititems` to inspect the structure of a ragged array stored in HDF5 format. This shows the individual components of the array. ```python >>> store["varm/transcript"].visititems(print) node1-mask node10-data node11-mask node12-offsets node13-mask node14-data node16-offsets node17-data node2-offsets node3-data node4-mask node5-offsets node6-data node7-mask node8-offsets node9-mask ``` -------------------------------- ### AnnData.write_h5ad / io.read_h5ad - HDF5 Persistence Source: https://context7.com/scverse/anndata/llms.txt Provides methods for saving AnnData objects to the `.h5ad` HDF5 format and loading them back, including options for compression and backed (memory-mapped) mode. ```APIDOC ## AnnData.write_h5ad / io.read_h5ad ### Description Save and load AnnData objects to/from the native `.h5ad` HDF5 format. Supports optional gzip/lzf compression, backed (memory-mapped) mode, and writing sparse matrices as dense. ### Usage ```python import anndata as ad import numpy as np from scipy.sparse import csr_matrix adata = ad.AnnData(X=csr_matrix(np.eye(100, 200, dtype=np.float32))) # Write adata.write_h5ad("data.h5ad") adata.write_h5ad("data_compressed.h5ad", compression="gzip", compression_opts=4) # Read (full, into memory) loaded = ad.read_h5ad("data.h5ad") print(loaded) # Backed / memory-mapped mode backed = ad.read_h5ad("data.h5ad", backed="r") print(backed.isbacked) print(backed.filename) # Slice a backed object and load that slice into memory subset_mem = backed[10:30, :50].to_memory() print(subset_mem.shape) backed.file.close() # Write dense for a sparse matrix adata.write_h5ad("dense.h5ad", as_dense=["X"]) ``` ``` -------------------------------- ### List items in Zarr categorical array Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Use `visititems` to list the 'categories' and 'codes' arrays within a categorical array group in Zarr. This helps understand how categorical data is stored. ```python >>> categorical.visititems(print) categories codes ``` -------------------------------- ### On-Disk Sparse Matrix Interface Source: https://context7.com/scverse/anndata/llms.txt Wrap a CSR/CSC sparse matrix stored as an HDF5 or Zarr group into a `BaseCompressedSparseDataset`. This supports slice-based access without loading the full matrix, enabling random access into backed sparse data. ```APIDOC ## `io.sparse_dataset` — On-Disk Sparse Matrix Interface Wrap a CSR/CSC sparse matrix stored as an HDF5 or Zarr group into a `BaseCompressedSparseDataset`, which supports slice-based access without loading the full matrix. Enables random access into backed sparse data. ```python import anndata as ad import h5py import numpy as np from scipy.sparse import csr_matrix # Write a sparse matrix matrix = csr_matrix(np.eye(1000, 2000, dtype=np.float32)) with h5py.File("sparse.h5", "w") as f: ad.io.write_elem(f, "X", matrix) # Access slices without full load with h5py.File("sparse.h5", "r") as f: ds = ad.io.sparse_dataset(f["X"]) print(ds.shape) # (1000, 2000) print(ds.format) # 'csr' row_slice = ds[0:10] # returns a scipy sparse matrix for rows 0-9 print(row_slice.shape) # (10, 2000) print(type(row_slice)) # ``` ``` -------------------------------- ### Write Sharded AnnData with Zarr V3 Source: https://github.com/scverse/anndata/blob/main/docs/tutorials/zarr-v3.md Custom callback for `anndata.experimental.write_dispatched` to enable automatic sharding for Zarr V3 arrays and sparse matrices. Tune shard and chunk sizes based on your data and use case. ```python import zarr import anndata as ad from collections.abc import Mapping from typing import Any g = zarr.open_group(orig_path, mode="a", use_consolidated=False, zarr_version=3) # zarr_version 3 is default but note that sharding only works with v3! def write_sharded(group: zarr.Group, adata: ad.AnnData): def callback( func: ad.experimental.Write, g: zarr.Group, k: str, elem: ad.typing.RWAble, dataset_kwargs: Mapping[str, Any], iospec: ad.experimental.IOSpec, ): if iospec.encoding_type in {"array"}: dataset_kwargs = { "shards": tuple(int(2 ** (16 / len(elem.shape))) for _ in elem.shape), **dataset_kwargs, } dataset_kwargs["chunks"] = tuple(i // 2 for i in dataset_kwargs["shards"]) elif iospec.encoding_type in {"csr_matrix", "csc_matrix"}: dataset_kwargs = {"shards": (2**16,), "chunks": (2**8,), **dataset_kwargs} func(g, k, elem, dataset_kwargs=dataset_kwargs) return ad.experimental.write_dispatched(group, "/", adata, callback=callback) ``` -------------------------------- ### Anndata Settings: Inspect and Modify Defaults Source: https://context7.com/scverse/anndata/llms.txt Access and modify global configuration settings for anndata. Settings can be inspected directly or changed globally. ```python import anndata as ad # Inspect defaults print(ad.settings) # check_uniqueness: bool # ... (default: True) # ... # Modify globally ad.settings.check_uniqueness = False ad.settings.remove_unused_categories = True ``` -------------------------------- ### Inspect scalar parameters in Zarr uns group Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Use `visititems` to list scalar parameters stored within the `uns/neighbors/params` group in Zarr. This shows the keys and types of saved parameters. ```python >>> store["uns/neighbors/params"].visititems(print) method metric n_neighbors random_state ``` -------------------------------- ### Write AnnData to .h5ad Source: https://github.com/scverse/anndata/blob/main/docs/benchmark-read-write.ipynb Writes the AnnData object to the native .h5ad file format. This operation is timed to measure performance. ```python %%time adata.write("test.h5ad") ``` -------------------------------- ### List items in HDF5 categorical array Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Use `visititems` to list the 'categories' and 'codes' datasets within a categorical array group in HDF5. This helps understand how categorical data is stored. ```python >>> categorical.visititems(print) categories codes ``` -------------------------------- ### Use BloscCodec with AnnData.write_zarr Source: https://github.com/scverse/anndata/blob/main/docs/tutorials/zarr-v3.md Explicitly pass `zarr.codecs.BloscCodec` to the `compressor` argument of `anndata.AnnData.write_zarr` to revert to the previous default compression behavior of `blosc`. ```python adata.write_zarr(..., compressor=zarr.codecs.BloscCodec()) ``` -------------------------------- ### Inspect scalar parameters in HDF5 uns group Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Use `visititems` to list scalar parameters stored within the `uns/neighbors/params` group in HDF5. This shows the keys and types of saved parameters. ```python >>> store["uns/neighbors/params"].visititems(print) method metric n_neighbors random_state ``` -------------------------------- ### Custom I/O Hooks with `read_dispatched` / `write_dispatched` Source: https://context7.com/scverse/anndata/llms.txt Intercept element-level read or write processes using `experimental.read_dispatched` and `experimental.write_dispatched` to customize data encoding/decoding. This is useful for adding support for new array backends or transforming data on-the-fly. ```python import anndata as ad from anndata.experimental import read_dispatched, write_dispatched import h5py import numpy as np from scipy.sparse import csr_matrix adata = ad.AnnData(X=csr_matrix(np.eye(5, dtype=np.float32))) adata.write_h5ad("test_dispatch.h5ad") ``` -------------------------------- ### Inspect Ragged Array Structure in Zarr Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Use `visititems` to inspect the structure of a ragged array stored in Zarr format. This shows the individual components of the array. ```python >>> store["varm/transcript"].visititems(print) node1-mask node10-data node11-mask node12-offsets node13-mask node14-data node16-offsets node17-data node2-offsets node3-data node4-mask node5-offsets node6-data node7-mask node8-offsets node9-mask ``` -------------------------------- ### Read and Write Individual Elements with h5py Source: https://context7.com/scverse/anndata/llms.txt Use `io.write_elem` and `io.read_elem` to save and load individual AnnData elements like DataFrames or sparse matrices to/from HDF5 files. This is useful for custom serialization pipelines. ```python import anndata as ad import h5py import numpy as np import pandas as pd from scipy.sparse import csr_matrix obs = pd.DataFrame({"cell_type": ["T", "B", "NK"], "n_counts": [1200, 800, 950]}, index=["c0", "c1", "c2"]) matrix = csr_matrix(np.eye(3, dtype=np.float32)) # ── Write individual elements ───────────────────────────────────────────────── with h5py.File("elements.h5", "w") as f: ad.io.write_elem(f, "obs", obs) ad.io.write_elem(f, "X", matrix) # ── Read them back ──────────────────────────────────────────────────────────── with h5py.File("elements.h5", "r") as f: obs_loaded = ad.io.read_elem(f["obs"]) X_loaded = ad.io.read_elem(f["X"]) print(type(obs_loaded)) # print(obs_loaded) print(X_loaded.toarray()) ``` -------------------------------- ### Low-Level Element I/O Source: https://context7.com/scverse/anndata/llms.txt Read or write individual elements like DataFrames or sparse matrices to/from HDF5 or Zarr groups without constructing a full AnnData object. This is useful for custom serialization pipelines. ```APIDOC ## `io.read_elem` / `io.write_elem` — Low-Level Element I/O Read or write individual elements (e.g., a single obs DataFrame or sparse matrix) to/from an HDF5 or Zarr group without constructing a full AnnData object. Useful for custom serialisation pipelines. ```python import anndata as ad import h5py import numpy as np import pandas as pd from scipy.sparse import csr_matrix obs = pd.DataFrame({"cell_type": ["T", "B", "NK"], "n_counts": [1200, 800, 950]}, index=["c0", "c1", "c2"]) matrix = csr_matrix(np.eye(3, dtype=np.float32)) # ── Write individual elements ───────────────────────────────────────────────── with h5py.File("elements.h5", "w") as f: ad.io.write_elem(f, "obs", obs) ad.io.write_elem(f, "X", matrix) # ── Read them back ──────────────────────────────────────────────────────────── with h5py.File("elements.h5", "r") as f: obs_loaded = ad.io.read_elem(f["obs"]) X_loaded = ad.io.read_elem(f["X"]) print(type(obs_loaded)) # print(obs_loaded) print(X_loaded.toarray()) ``` ``` -------------------------------- ### List items in Zarr uns group Source: https://github.com/scverse/anndata/blob/main/docs/fileformat-prose.md Use `visititems` to list all items within a group in the AnnData object's `uns` attribute when stored in Zarr. This is useful for inspecting the structure of mappings. ```python >>> store["uns"].visititems(print) [...] pca pca/variance pca/variance_ratio [...] ``` -------------------------------- ### Errors and Warnings Source: https://github.com/scverse/anndata/blob/main/docs/api.md Custom exceptions and warnings used within the AnnData library. ```APIDOC ## Errors and Warnings ### Description Custom exceptions and warnings used within the AnnData library. ### Warning ImplicitModificationWarning ```