Zarr

https://github.com/zarr-developers/zarr-python
Admin
Zarr is a Python package providing an implementation of compressed, chunked, N-dimensional arrays,...
Tokens:160,797
Snippets:1,171
Trust Score:8.6
License:MIT
Update:1 week ago
Show doc for...
Tokens:
Raw
### Run the example with uv

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-indexing/examples/lazy_indexing_numpy/README.md

Use `uv` to run the example script, which installs dependencies declared inline via PEP 723 automatically.

```bash
cd packages/zarr-indexing
uv run --with-editable . examples/lazy_indexing_numpy/lazy_indexing_numpy.py
```

--------------------------------

### Run the example with plain Python

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-indexing/examples/lazy_indexing_numpy/README.md

Run the example with plain Python after manually installing `zarr-indexing`, `numpy`, and `pytest`.

```bash
cd packages/zarr-indexing
python examples/lazy_indexing_numpy/lazy_indexing_numpy.py
```

--------------------------------

### Quickstart: wrap an array and compose a lazy view

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-indexing/docs/index.md

Wrap an array, compose a lazy view with view[...], and call result() to get values. The example is included from an external file and is not shown on this page.

```python
--8<-- "snippets/canonical_slice.py:landing-quickstart"
```

--------------------------------

### Run the example with uv (PEP 723)

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-indexing/examples/lazy_indexing_dask/README.md

Use uv to run the script, which installs dependencies automatically from inline metadata.

```bash
cd packages/zarr-indexing
uv run --with-editable . examples/lazy_indexing_dask/lazy_indexing_dask.py
```

--------------------------------

### Run custom data type example with uv

Source: https://github.com/zarr-developers/zarr-python/blob/main/examples/custom_dtype/README.md

Use uv to run the example script, which automatically installs dependencies declared inline via PEP 723.

```bash
uv run examples/custom_dtype/custom_dtype.py
```

--------------------------------

### Run the example with plain Python

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-indexing/examples/lazy_indexing_dask/README.md

Run the script with plain Python after manually installing zarr, zarr-indexing, dask[array], numpy, and pytest.

```bash
cd packages/zarr-indexing
python examples/lazy_indexing_dask/lazy_indexing_dask.py
```

--------------------------------

### Run the sharding coalescing example

Source: https://github.com/zarr-developers/zarr-python/blob/main/examples/sharding_coalescing/README.md

Executes the example script with uv. Requires the project environment with zarr and numpy installed.

```bash
uv run sharding_coalescing.py
```

--------------------------------

### Run custom data type example with plain Python

Source: https://github.com/zarr-developers/zarr-python/blob/main/examples/custom_dtype/README.md

Run the example with plain Python after manually installing zarr, ml_dtypes, and pytest.

```bash
python examples/custom_dtype/custom_dtype.py
```

--------------------------------

### Import dependencies and set up environment for Zarr quick start

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/quick-start.md

Sets up the Python environment for the quick start examples. Imports numpy, pprint, io, and warnings, and suppresses a specific UserWarning about Numcodecs codecs not being in the Zarr version 3 specification. Seeds the random number generator for reproducibility.

```python
import numpy as np
from pprint import pprint
import io
import warnings

warnings.filterwarnings(
  "ignore",
  message="Numcodecs codecs are not in the Zarr version 3 specification*",
  category=UserWarning
)
np.random.seed(0)
```

--------------------------------

### Create and print a 2D array for oindex examples

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/arrays.md

Creates a 2D array of shape (3,5) and prints it. This is the setup for the oindex property examples.

```python
data = np.arange(15).reshape(3, 5)
z = zarr.create_array(store='data/example-15.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
z[:] = data
print(z.oindex[[0, 2], :])  # select first and third rows
```

--------------------------------

### Example directory structure

Source: https://github.com/zarr-developers/zarr-python/blob/main/examples/README.md

Shows the directory layout for each example, with a subdirectory containing a README.md and a Python source file.

```text
examples/
├── example_name/
│   ├── README.md          # Documentation for the example
│   └── example_name.py    # Python source code
└── ...
```

--------------------------------

### Install development tools with uv

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/contributing.md

Install Hatch and just using uv, then list available commands and set up the default test environment.

```bash
uv tool install hatch==1.16.5
uv tool install rust-just==1.58.0   # or any option from https://just.systems/man/en/packages.html
just                 # list available commands
just envs            # list Python environments
just setup           # create the default test environment
just test
```

--------------------------------

### Run example script with uv

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-http-server/README.md

Runs the example script from the examples directory using uv. The script is executed by the test suite to prevent drift.

```bash
uv run examples/serve.py
```

--------------------------------

### Quick start: serve a Zarr array with node_app and serve_background

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-http-server/docs/index.md

Creates an in-memory Zarr array and serves it over HTTP using node_app and serve_background. The server URL is printed. Only GET and HEAD are enabled by default; other methods return 405.

```python
import zarr
from zarr_http_server import node_app, serve_background

store = zarr.storage.MemoryStore()
array = zarr.create_array(store, shape=(100,), chunks=(10,), dtype="float64")

with serve_background(node_app(array)) as server:
    print(server.url)   # e.g. http://127.0.0.1:8000
```

--------------------------------

### Start the background server

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-http-server/examples/serve_notebook.ipynb

Starts the server with `serve_background` and prints the URL. The server runs in a daemon thread and is ready immediately.

```python
server = serve_background(node_app(array), host="127.0.0.1")

print(f"serving at {server.url}")
assert server.url is not None
```

--------------------------------

### Run the system memory chunk cache example

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-indexing/examples/system_memory_chunk_cache/README.md

Run the example from the packages/zarr-indexing directory using uv with the editable package. The script is located at examples/system_memory_chunk_cache/system_memory_chunk_cache.py.

```bash
cd packages/zarr-indexing
uv run --with-editable . examples/system_memory_chunk_cache/system_memory_chunk_cache.py
```

--------------------------------

### Install zarr-http-server

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/subprojects.md

Install the zarr-http-server package, an HTTP server for Zarr stores, arrays, and groups.

```bash
pip install zarr-http-server
```

--------------------------------

### Iterate over group members

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/groups.md

Use root.members() to get (name, member) pairs for immediate members. The example prints each pair.

```python
for name, member in root.members():
    print(name, member)
```

--------------------------------

### Create and print a 2D array for orthogonal indexing

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/arrays.md

Creates a 2D array of shape (3,5) and prints it. This is the setup for the orthogonal selection examples.

```python
data = np.arange(15).reshape(3, 5)
z = zarr.create_array(store='data/example-14.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
z[:] = data
print(z[:])
```

--------------------------------

### just commands for development

Source: https://github.com/zarr-developers/zarr-python/blob/main/packages/zarr-metadata/README.md

Run the test suite, lint, typecheck, docs-check, check, and docs-serve from the package directory or from anywhere in the repository using `just packages/zarr-metadata/<recipe>`.

```bash
just test        # run the test suite (extra args go to pytest)
just lint        # ruff, same invocation as CI
just typecheck   # pyright, pinned to the version CI uses
just docs-check  # strict build of the docs site
just check       # all of the above
just docs-serve  # serve the docs site locally
```

--------------------------------

### Create and print a 1D array for mask indexing

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/arrays.md

Creates a 1D array of squared integers and prints it. This is the setup for the subsequent mask selection examples.

```python
data = np.arange(10) ** 2
z = zarr.create_array(store='data/example-12.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
z[:] = data
print(z[:])
```

--------------------------------

### Create CacheStore with LocalStore and cache store

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/experimental.md

Creates a local source store and a separate cache store, then wraps them with CacheStore. Requires a source store and a cache store; the cache store can be any Store implementation. The example imports numpy and uses mkdtemp for a temporary directory.

```python
import zarr
from zarr.storage import LocalStore
import numpy as np
from tempfile import mkdtemp
from zarr.experimental.cache_store import CacheStore

# Create a local store and a separate cache store
local_store_path = mkdtemp(suffix='.zarr')

```

--------------------------------

### Create a 2D array for block indexing

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/arrays.md

Creates a 10x10 array with chunk shape (3,3) and fills it with data. This is the setup for the subsequent block selection examples.

```python
data = np.arange(100).reshape(10, 10)
z = zarr.create_array(store='data/example-17.zarr', shape=data.shape, dtype=data.dtype, chunks=(3, 3), overwrite=True)
z[:] = data
```

--------------------------------

### Create and print a 2D array for multidimensional mask indexing

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/arrays.md

Creates a 2D array of shape (3,5) and prints it. This is the setup for the multidimensional mask selection examples.

```python
data = np.arange(15).reshape(3, 5)
z = zarr.create_array(store='data/example-13.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
z[:] = data
print(z[:])
```

--------------------------------

### Create FsspecStore from URL for remote storage

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/storage.md

Creates an FsspecStore from a URL for remote storage, using storage_options to configure the backend. Requires s3fs to be installed. The example uses an anonymous read-only connection to an S3 bucket.

```python
# Note: requires s3fs to be installed
store = zarr.storage.FsspecStore.from_url(
   's3://noaa-nwm-retro-v2-zarr-pds',
   read_only=True,
   storage_options={'anon': True}
)
group = zarr.open_group(store=store, mode='r')
print(group)
```

--------------------------------

### Run the test suite

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/contributing.md

Invoke the default test suite using just.

```bash
just test
```

--------------------------------

### Get array view with different config using with_config

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/arrays.md

Shows how to obtain an array view with a different configuration using the `with_config` method. The example changes the `order` to `"F"` and prints the new config.

```python
arr_f = arr.with_config({"order": "F"})
print(arr_f.config)
```

--------------------------------

### Create array with custom config using config keyword

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/user-guide/arrays.md

Shows how to pass a configuration dictionary when creating an array. The example sets `write_empty_chunks` to `True` and prints the resulting config.

```python
arr = zarr.create_array({}, shape=(10,), dtype='int8', config={"write_empty_chunks": True})
print(arr.config)
```

--------------------------------

### Open array with unknown codec and print error

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/blog/posts/3.4.0-release.md

This example creates a Zarr array metadata with an unsupported codec ('imagecodecs_jpeg') and attempts to open it, catching the UnknownCodecError to display the useful error message. It requires the 'imagecodecs' package to be installed to actually read the array.

```python
import json

import zarr
from zarr.buffer.cpu import buffer_prototype

metadata = {
    "zarr_format": 3,
    "node_type": "array",
    "shape": [4, 4],
    "data_type": "uint8",
    "chunk_grid": {"name": "regular", "configuration": {"chunk_shape": [4, 4]}},
    "chunk_key_encoding": {"name": "default"},
    "fill_value": 0,
    "codecs": [{"name": "bytes"}, {"name": "imagecodecs_jpeg"}],
    "attributes": {},
}
store = {"zarr.json": buffer_prototype.buffer.from_bytes(json.dumps(metadata).encode())}

try:
    zarr.open_array(store, mode="r")
except zarr.errors.UnknownCodecError as e:
    print(e)
```

--------------------------------

### Serve documentation during development

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/contributing.md

Serves a continuously updating version of the documentation at http://127.0.0.1:8000/.

```bash
just docs-serve
```

--------------------------------

### Install Zarr with pip

Source: https://github.com/zarr-developers/zarr-python/blob/main/README.md

Installs the latest version of Zarr from PyPI.

```bash
pip install zarr
```

--------------------------------

### Build documentation locally

Source: https://github.com/zarr-developers/zarr-python/blob/main/docs/contributing.md

Builds the documentation locally using mkdocs. The built documentation is available in the 'site' folder.

```bash
just docs-build
```
Tokens:
Raw