===============
LIBRARY RULES
===============
From library maintainers:
- Use Store.read(path) for streaming reads, Store.write(path, content) for streaming writes (returns WriteResult), and Store.list_files(path) for directory listings.
- Backends declare capabilities (LAZY_READ, ATOMIC_WRITE, ATOMIC_MOVE, METADATA, GLOB, ...). Query with store.supports(Capability.X); see FEATURES.md for the full matrix.
- Install optional backends and extensions via pip extras. Backends: remote-store[s3], [s3-pyarrow], [sftp], [azure], [sql], [sql-query]. Extensions: [arrow] (PyArrow + Parquet), [otel] (tracing), [pydantic], [yaml], [dagster].
- For async workloads, import AsyncStore from remote_store.aio. To cross the sync/async boundary, use SyncBackendAdapter (sync Backend in async code, via asyncio.to_thread) or AsyncBackendSyncAdapter (async Backend in sync code, daemon-thread event loop).
- Built-in backends: LocalBackend, MemoryBackend, ReadOnlyHttpBackend, S3Backend, S3PyArrowBackend, SFTPBackend, AzureBackend, SQLBlobBackend, SQLQueryBackend.
- Streaming reads: with store.read(path) as f: data = f.read(). Atomic writes: with store.open_atomic(path) as f: f.write(...). The destination appears only after the block exits cleanly.
- Errors typed in remote_store.errors (re-exported from remote_store). Base RemoteStoreError; subclasses: NotFound, AlreadyExists, PermissionDenied, InvalidPath, CapabilityNotSupported, DirectoryNotEmpty, BackendUnavailable. SDK exceptions never leak.
- Configuration loads from TOML, YAML, dict, or Pydantic models via Registry; ${ENV_VAR} placeholders are resolved and credentials are wrapped in Secret automatically.
- Extensions in remote_store.ext (always available): batch, cache, glob, integrity, observe, partition, streams, transfer, write. Optional via extras: arrow (PyArrow + Parquet), otel (OpenTelemetry), pydantic, yaml, dagster (IO manager). See FEATURES.md.
- Write methods return WriteResult(path, size, source, digest, etag, version_id, last_modified, metadata). source: NativeSource (write response), BasicSource (post-write stat), or SidecarSource (ext.write hash). Rich fields need WRITE_RESULT_NATIVE.
- Write*() methods accept optional metadata= dict[str, str]; backend must declare USER_METADATA, otherwise non-empty raises CapabilityNotSupported. store.head(path) returns WriteResult metadata (size, etag, last_modified) without reading content.
- Capabilities split into method gates (READ, WRITE, DELETE, LIST, GLOB, MOVE, COPY, ATOMIC_WRITE, METADATA, required to call) and quality flags (SEEKABLE_READ, LAZY_READ, ATOMIC_MOVE, WRITE_RESULT_NATIVE, method works either way).

### Quick Start Example

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/index.md

A basic example demonstrating the installation of remote-store with S3 support and performing a simple write and read operation. This is a common starting point for users.

```bash
pip install remote-store[s3]
```

```python
from remote_store import Store
from remote_store.backends import S3Backend

store = Store(S3Backend(bucket="my-bucket"))
store.write_text("file.txt", "hello")
print(store.read_text("file.txt"))  # 'hello'
```

--------------------------------

### Install Dependencies for Medallion Dagster Example

Source: https://github.com/haalfi/remote-store/blob/master/examples/medallion_dagster/README.md

Install remote-store with required extras and showcase dependencies. This command is used to set up the project environment.

```bash
cd examples/medallion_dagster

# Install remote-store with required extras + showcase dependencies
pip install -e "../../[dagster,arrow,otel,requests]" polars dagster-webserver opentelemetry-sdk
```

--------------------------------

### Install remote-store

Source: https://github.com/haalfi/remote-store/blob/master/README.md

Standard installation command for the remote-store package from PyPI.

```bash
pip install remote-store
```

--------------------------------

### Direct Instantiation

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/custom-backend-guide.md

Example of directly instantiating a custom RedisStore backend.

```python
from remote_store.stores import Store
from remote_store.paths import RemotePath
from remote_store.backends.redis import RedisStore


# Assuming Redis is running on localhost:6379
redis_store = RedisStore(url="redis://localhost:6379/0", prefix="app:")

# You can then use redis_store as a Store instance
# For example:
# redis_store.write("my_file.txt", b"Hello, world!")

```

--------------------------------

### Async Store Quick Start

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/async.md

Demonstrates the basic usage of the async store API for writing data. This snippet is intended for a quick start and assumes the necessary imports and store object are available.

```python
import asyncio

from remote_store import AsyncStore


async def async_quick_start(store: AsyncStore):
    # Write bytes
    await store.write("my-file.txt", b"Hello world!")

    # Write text
    await store.write_text("my-text-file.txt", "Hello world!")

    # Write atomically
    await store.write_atomic("my-atomic-file.txt", b"Hello world!")

    # Read bytes
    data = await store.read_bytes("my-file.txt")
    assert data == b"Hello world!"

    # Read text
    text = await store.read_text("my-text-file.txt")
    assert text == "Hello world!"

    # Read as an async iterator
    async for chunk in store.read("my-file.txt"):
        process(chunk)

    # Delete a file
    await store.delete("my-file.txt")


def process(chunk: bytes):
    # Placeholder for processing chunks of data
    pass

```

--------------------------------

### Install integration extras

Source: https://github.com/haalfi/remote-store/blob/master/README.md

Commands to install optional dependencies for HTTP clients, data formats, and observability.

```bash
pip install "remote-store[requests]"       # HTTP backend with requests (connection pooling)
pip install "remote-store[httpx]"          # HTTP backend with httpx (HTTP/2)
pip install "remote-store[arrow]"          # PyArrow filesystem adapter
pip install "remote-store[otel]"           # OpenTelemetry instrumentation
pip install "remote-store[yaml]"           # YAML config support
pip install "remote-store[pydantic]"       # Pydantic BaseSettings config
pip install "remote-store[toml]"           # TOML config on Python < 3.11
```

--------------------------------

### Install backend dependencies

Source: https://github.com/haalfi/remote-store/blob/master/README.md

Commands to install specific backend support for cloud storage and database systems.

```bash
pip install "remote-store[s3]"           # Amazon S3 / MinIO
pip install "remote-store[s3-pyarrow]"   # S3 via PyArrow (analytical workloads)
pip install "remote-store[sftp]"         # SFTP / SSH
pip install "remote-store[azure]"        # Azure Blob / ADLS Gen2
pip install "remote-store[graph]"        # Microsoft Graph (OneDrive / SharePoint / Teams), async-only
pip install "remote-store[sql]"          # SQL Blob (SQLite, PostgreSQL, ...)
pip install "remote-store[sql-query]"    # SQL Query (read-only, SQLAlchemy + PyArrow)
```

--------------------------------

### Install ParquetDatasetStore

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/parquet-datasets.md

Install the `remote-store` package with the `arrow` extra to include `ParquetDatasetStore`.

```bash
pip install "remote-store[arrow]"
```

--------------------------------

### App-only Authentication Setup

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/graph.md

Example of setting up GraphAuth for app-only authentication using client credentials against a work or school tenant. This is suitable for CI, daemons, and services.

```python
# App-only (client-credentials) against a work/school tenant:
auth = GraphAuth(
    tenant_id="<tenant-guid>",
    client_id="<entra-app-id>",
    client_secret="<secret>",  # accepts a Secret; masked in repr
)
```

--------------------------------

### Install S3-PyArrow Backend

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/s3-pyarrow.md

Install the S3-PyArrow backend along with its dependencies, s3fs and pyarrow.

```bash
pip install "remote-store[s3-pyarrow]"
```

--------------------------------

### Install SQL Query Backend

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/sql-query.md

Install the SQL query backend with its dependencies. Requires sqlalchemy and pyarrow.

```bash
pip install remote-store[sql-query]
```

--------------------------------

### Install S3 Backend

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/s3.md

Install the remote-store package with S3 support using pip.

```bash
pip install "remote-store[s3]"
```

--------------------------------

### Install SFTP Backend

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/sftp.md

Install the remote-store package with SFTP support. This includes paramiko and tenacity for SSH operations and retries.

```bash
pip install "remote-store[sftp]"
```

--------------------------------

### Quick Start: Batch Operations

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/batch-operations.md

Demonstrates the basic usage of `batch_exists`, `batch_copy`, and `batch_delete` for common file operations.

```python
from remote_store import Store, batch_delete, batch_copy, batch_exists
from remote_store.backends import MemoryBackend

store = Store(backend=MemoryBackend())
store.write("a.txt", b"hello")
store.write("b.txt", b"world")

# Check which files exist
exists_map = batch_exists(store, ["a.txt", "b.txt", "c.txt"])
# {"a.txt": True, "b.txt": True, "c.txt": False}

# Copy multiple files
result = batch_copy(store, [("a.txt", "a_copy.txt"), ("b.txt", "b_copy.txt")])
assert result.all_succeeded

# Delete multiple files
result = batch_delete(store, ["a.txt", "b.txt"], missing_ok=True)
assert result.all_succeeded
```

--------------------------------

### Install SQL Blob Backend

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/sql-blob.md

Install the remote-store package with SQL support. Requires sqlalchemy version 2.0 or higher.

```bash
pip install remote-store[sql]
```

--------------------------------

### Quick Start with PyArrow Filesystem

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/pyarrow-adapter.md

Create a PyArrow filesystem from a remote-store instance and use it with PyArrow for reading and writing Parquet files.

```python
import pyarrow as pa
import pyarrow.parquet as pq

from remote_store import Store
from remote_store.backends import MemoryBackend
from remote_store.ext.arrow import pyarrow_fs

store = Store(backend=MemoryBackend())
fs = pyarrow_fs(store)

# Now use `fs` anywhere PyArrow accepts a filesystem:
table = pa.table({"col": [1, 2, 3]})
pq.write_table(table, "data.parquet", filesystem=fs)
result = pq.read_table("data.parquet", filesystem=fs)
```

--------------------------------

### Initialize AsyncAzureBackend and AsyncStore

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/async.md

Demonstrates initializing AsyncAzureBackend for ADLS Gen2 or Blob Storage and using it with AsyncStore. Requires installation of 'remote-store[azure]'.

```python
from remote_store.aio import AsyncStore, AsyncAzureBackend

backend = AsyncAzureBackend(
    container="my-container",
    hns=True,
    account_name="myaccount",
    account_key="...",
)
async with AsyncStore(backend, root_path="data") as store:
    await store.write("report.csv", b"col1,col2\n1,2", overwrite=True)
```

--------------------------------

### Store.to_key Composition Example

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/010-native-path-resolution.md

Illustrates how Store.to_key composes backend conversion with root path stripping. Use this to understand the two-step path resolution process.

```python
# SFTP backend with base_path="/srv/sftp", store with root_path="data"
store = Store(backend=sftp, root_path="data")

# Full chain: "/srv/sftp/data/reports/q1.csv"
#   → backend.to_key → "data/reports/q1.csv"
#   → strip root_path → "reports/q1.csv"
store.to_key("/srv/sftp/data/reports/q1.csv")  # → "reports/q1.csv"
```

--------------------------------

### Initialize GraphBackend with Device-Code Authentication

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/async.md

Shows how to set up GraphBackend for OneDrive/SharePoint using device-code authentication for personal accounts. Requires installation of 'remote-store[graph]'.

```python
from remote_store.aio import AsyncStore, GraphAuth, GraphBackend, GraphUtils

# Device-code auth against a personal Microsoft account (consumer OneDrive).
auth = GraphAuth(tenant_id="consumers", client_id="<entra-app-id>")
# Inside async code, use the async resolver — the sync GraphUtils.resolve_drive_id
# runs its own event loop internally and raises RuntimeError from a running one.
drive_id = await GraphUtils.aresolve_drive_id("me", token_provider=auth)

backend = GraphBackend(drive_id, token_provider=auth)
async with AsyncStore(backend, root_path="Documents") as store:
    await store.write("report.csv", b"col1,col2\n1,2", overwrite=True)
```

--------------------------------

### LocalBackend to_key Example

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/010-native-path-resolution.md

Demonstrates how `LocalBackend.to_key` strips the filesystem root directory from a native path. Paths without the root prefix are returned unchanged.

```python
backend = LocalBackend("/tmp/store")
backend.to_key("/tmp/store/data/file.txt")  # → "data/file.txt"
backend.to_key("data/file.txt")             # → "data/file.txt" (no prefix, unchanged)
```

--------------------------------

### Initialize Store with LocalBackend

Source: https://github.com/haalfi/remote-store/blob/master/README.md

Demonstrates the simplest way to initialize a store using a local filesystem backend.

```python
from remote_store import Store
from remote_store.backends import LocalBackend

store = Store(LocalBackend(root="/tmp/data"))
store.write_text("hello.txt", "Hello, world!")
print(store.read_text("hello.txt"))  # 'Hello, world!'
```

--------------------------------

### Optional Dependency Import Guard

Source: https://github.com/haalfi/remote-store/blob/master/sdd/adrs/0008-extension-architecture.md

This pattern is used in extension modules to handle optional dependencies. It raises a helpful error message if the required dependency is not installed, guiding the user on how to install it.

```python
try:
    import pyarrow as pa
except ModuleNotFoundError as _exc:
    raise ModuleNotFoundError(
        "PyArrow is required for the arrow extension. "
        "Install it with: pip install 'remote-store[arrow]'"
    ) from _exc
```

--------------------------------

### Quick Start: Upload, Download, and Transfer

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/transfer-operations.md

Demonstrates the basic usage of upload, download, and transfer functions with a MemoryBackend store. Ensure necessary imports are included.

```python
from remote_store import Store, upload, download, transfer
from remote_store.backends import MemoryBackend

store = Store(backend=MemoryBackend())

# Upload a local file to the store
upload(store, "local/report.csv", "reports/report.csv")

# Download a remote file to a local path
download(store, "reports/report.csv", "local/copy.csv")

# Transfer between two stores
other = Store(backend=MemoryBackend())
transfer(store, "reports/report.csv", other, "archive/report.csv")
```

--------------------------------

### SFTPBackend to_key Example

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/010-native-path-resolution.md

Shows how `SFTPBackend.to_key` removes the configured `base_path` from a native path. Paths not starting with the `base_path` are returned as is.

```python
backend = SFTPBackend(host="srv", base_path="/srv/sftp")
backend.to_key("/srv/sftp/data/file.txt")   # → "data/file.txt"
backend.to_key("data/file.txt")             # → "data/file.txt" (no prefix, unchanged)
```

--------------------------------

### Quick Start: Initialize and Ping Store

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/health-check.md

Initialize the Store with a backend and use `ping()` to verify connectivity. The `ping()` method raises an exception on failure and is silent on success.

```python
from remote_store import Store
from remote_store.backends import LocalBackend

store = Store(LocalBackend(root="/data/inbox"))
store.ping()  # raises on failure, silent on success
```

--------------------------------

### Initialize SQLQueryBackend and Store

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/sql-query.md

Initialize the SQLQueryBackend with a database URL and a dictionary of queries mapped to file paths. The file extension determines the output format. Then, create a Store instance with this backend.

```python
from remote_store import Store
from remote_store.backends import SQLQueryBackend

backend = SQLQueryBackend(
    url="sqlite:///analytics.db",
    queries={
        "reports/daily_sales.parquet": "SELECT date, SUM(amount) FROM orders GROUP BY date",
        "reports/user_summary.csv": "SELECT * FROM user_summary_mv",
    },
)
store = Store(backend=backend)

# Read executes the query and serializes to the format implied by the extension
data = store.read_bytes("reports/daily_sales.parquet")  # Parquet bytes
```

--------------------------------

### Store.to_key InvalidPath Example

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/010-native-path-resolution.md

Demonstrates when Store.to_key raises InvalidPath. This occurs if the path, after backend stripping, does not start with the store's root_path.

```python
store = Store(backend=local, root_path="data")
store.to_key("/tmp/store/other/file.txt")  # → InvalidPath (not under "data/")
```

--------------------------------

### Quick Start: Basic Glob Operations

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/glob-pattern-matching.md

Demonstrates initializing a store, writing files, and performing basic file listing with both simple name filtering and recursive glob patterns.

```python
from remote_store import Store, glob_files
from remote_store.backends import MemoryBackend

store = Store(backend=MemoryBackend())
store.write("data/report.csv", b"r1")
store.write("data/summary.csv", b"r2")
store.write("data/readme.txt", b"r3")
store.write("logs/app.log", b"l1")

# Tier 1: simple name filtering (works with every backend)
csvs = list(store.list_files("data", pattern="*.csv"))
# [FileInfo("data/report.csv", ...), FileInfo("data/summary.csv", ...)]

# Tier 3: full recursive glob (works with every backend)
all_csvs = list(glob_files(store, "**/*.csv"))
# [FileInfo("data/report.csv", ...), FileInfo("data/summary.csv", ...)]
```

--------------------------------

### Initialize SQLite Backend and Store

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/sql-blob.md

Demonstrates the simplest usage with SQLite, creating a local store file. Writes and reads binary data.

```python
from remote_store import Store
from remote_store.backends import SQLBlobBackend

backend = SQLBlobBackend(url="sqlite:///store.db")
store = Store(backend=backend)

store.write("models/v3.pkl", model_bytes)
data = store.read_bytes("models/v3.pkl")
```

--------------------------------

### Quick Start with observe

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/observe.md

Wrap a store with the `observe` extension and provide a callback function to execute after each operation. The returned `ObservedStore` is a full `Store` subclass.

```python
from remote_store import observe

def on_write(event):
    print(f"Wrote {event.path} in {event.duration_ms:.1f}ms")

observed = observe(store, on_write=on_write)
observed.write("data/report.csv", csv_bytes)
# prints: Wrote data/report.csv in 12.3ms
```

--------------------------------

### Initialize store from configuration

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/choosing-a-backend.md

Load a configuration file into the Registry to instantiate a store, maintaining a consistent API across different backends.

```python
from remote_store import RegistryConfig, Registry

config = RegistryConfig.from_toml("dev.toml")  # or "prod.toml"
registry = Registry(config)
store = registry.get_store("default")
# Same API regardless of backend
```

--------------------------------

### Initialize Remote Store

Source: https://github.com/haalfi/remote-store/blob/master/examples/notebooks/02_file_operations.ipynb

Sets up the registry and store with a local backend. Ensure the temporary directory is cleaned up afterwards.

```python
import tempfile

from remote_store import BackendConfig, Registry, RegistryConfig, StoreProfile

tmpdir = tempfile.mkdtemp()
config = RegistryConfig(
    backends={"local": BackendConfig(type="local", options={"root": tmpdir})},
    stores={"files": StoreProfile(backend="local")},
)
registry = Registry(config)
store = registry.get_store("files")
print("Store ready.")
```

--------------------------------

### Switching Backends

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/index.md

Illustrates how to instantiate a Store with a different backend, such as S3Backend, without altering the application's core logic. This example requires AWS credentials to be configured.

```python
from remote_store import Store
from remote_store.backends import S3Backend

store = Store(S3Backend(bucket="my-bucket"))
store.write_text("file.txt", "hello")
print(store.read_text("file.txt"))  # same API, different backend
```

--------------------------------

### Install OpenTelemetry Optional Dependency

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/observe.md

Install the optional 'otel' dependency for OpenTelemetry integration. This only requires the 'opentelemetry-api' package.

```bash
pip install "remote-store[otel]"
```

--------------------------------

### Install remote-store with optional dependencies

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/troubleshooting.md

Install the necessary extras for your specific backend when encountering ImportError for optional dependencies.

```bash
pip install "remote-store[s3]"

```

```bash
pip install "remote-store[s3-pyarrow]"

```

```bash
pip install "remote-store[sftp]"

```

```bash
pip install "remote-store[azure]"

```

```bash
pip install "remote-store[all]"

```

--------------------------------

### Initialize HTTP Backend and Store

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/http.md

Instantiate the ReadOnlyHttpBackend with a base URL and optional headers, then create a Store instance.

```python
from remote_store import Store
from remote_store.backends import ReadOnlyHttpBackend

backend = ReadOnlyHttpBackend(
    base_url="https://data.example.com/datasets/",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
)
store = Store(backend=backend)

content = store.read_bytes("population/2024.csv")
info = store.get_file_info("population/2024.csv")
print(f"Size: {info.size}, Modified: {info.modified_at}")
```

--------------------------------

### Async Usage Example

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/graph.md

Demonstrates constructing and using the GraphBackend with AsyncStore for file operations. It shows interactive device-code authentication and drive ID resolution.

```python
import asyncio

from remote_store.aio import AsyncStore, GraphAuth, GraphBackend, GraphUtils


async def main() -> None:
    # 1. A token provider. Device-code (interactive) auth against a personal
    #    Microsoft account needs only tenant + client id — no secret.
    auth = GraphAuth(tenant_id="consumers", client_id="<entra-app-id>")

    # 2. Resolve the target drive ("me" = the signed-in user's OneDrive).
    #    Inside async code, use the async resolver — the sync
    #    GraphUtils.resolve_drive_id runs its own event loop internally and
    #    raises RuntimeError when called from a running one.
    drive_id = await GraphUtils.aresolve_drive_id("me", token_provider=auth)

    # 3. Construct the backend and use it through AsyncStore. On the event loop,
    #    prefer the async auth.aget_token (off-loop acquisition + single-flight).
    backend = GraphBackend(drive_id, token_provider=auth.aget_token)
    async with AsyncStore(backend, root_path="Documents") as store:
        await store.write("report.csv", b"col1,col2\n1,2\n", overwrite=True)
        data = await store.read_bytes("report.csv")


asyncio.run(main())
```

--------------------------------

### Install Optional Extensions

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/extensions.md

Install specific extensions by appending their names in brackets to the remote-store package. This ensures the necessary dependencies are installed for features like PyArrow filesystem adapters, OpenTelemetry tracing, Pydantic settings, YAML configuration, or Dagster integration.

```bash
pip install "remote-store[arrow]"     # PyArrow filesystem adapter + parquet datasets
pip install "remote-store[otel]"      # OpenTelemetry tracing and metrics
pip install "remote-store[pydantic]"  # Pydantic BaseSettings adapter
pip install "remote-store[yaml]"      # YAML config file loader
pip install "remote-store[dagster]"   # Dagster IO Manager adapter
```

--------------------------------

### Configure Lake Backend in Python

Source: https://github.com/haalfi/remote-store/blob/master/examples/medallion_dagster/README.md

Shows how to initialize the remote-store lake with different backends: local filesystem (default), S3, and Azure ADLS Gen2. Ensure necessary cloud backend packages are installed.

```python
# Local (default)
lake = otel_observe(Store(LocalBackend(root=_LAKE_ROOT)))

# S3 — S3Backend has no prefix= param; scope to a sub-prefix with .child()
lake = otel_observe(Store(S3Backend(bucket="my-bucket")).child("showcase"))

# Azure ADLS Gen2 (Hierarchical Namespace)
lake = otel_observe(
    Store(
        AzureBackend(
            container="my-filesystem",
            hns=True,  # required for ADLS Gen2 — the backend does not auto-detect it
            connection_string=os.environ["AZURE_STORAGE_CONNECTION_STRING"],
        )
    )
)
```

--------------------------------

### Install remote-store with Dagster support

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/dagster.md

Install the necessary packages for remote-store Dagster integration. Include 'arrow' for Parquet serializer support.

```bash
pip install "remote-store[dagster]"

# For Parquet serializer support:
pip install "remote-store[dagster,arrow]"
```

--------------------------------

### Initialize Store via Registry

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/sql-blob.md

Configures and uses the SQLBlobBackend through the remote-store Registry, defining backend and store profiles.

```python
from remote_store import BackendConfig, RegistryConfig, Registry, StoreProfile

config = RegistryConfig(
    backends={"db": BackendConfig(type="sql-blob", options={"url": "sqlite:///store.db"})},
    stores={"files": StoreProfile(backend="db", root_path="data")},
)

with Registry(config) as registry:
    store = registry.get_store("files")
    store.write("readme.txt", b"Hello!")
```

--------------------------------

### Initialize PostgreSQL Backend

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/sql-blob.md

Configure the SQLBlobBackend to connect to a PostgreSQL database using a connection URL.

```python
backend = SQLBlobBackend(url="postgresql://user:pass@localhost/mydb")
store = Store(backend=backend)
```

--------------------------------

### Install Graph Extras for Remote Store

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/async.md

Command to install the necessary extras for using Graph backends with remote-store. This includes Microsoft Graph API integration.

```bash
pip install "remote-store[graph]"
```

--------------------------------

### Install Azure Extras for Remote Store

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/async.md

Command to install the necessary extras for using Azure backends with remote-store. This includes async Azure SDK clients.

```bash
pip install "remote-store[azure]"
```

--------------------------------

### Initialize Memory Backend and Store

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/memory.md

Instantiate the MemoryBackend and a Store with it. Use this for basic in-memory file operations.

```python
from remote_store import Store
from remote_store.backends import MemoryBackend

backend = MemoryBackend()
store = Store(backend=backend, root_path="data")

store.write_text("hello.txt", "Hello, world!")
print(store.read_text("hello.txt"))  # 'Hello, world!'
```

--------------------------------

### Tier 3: glob_files() - Portable Full Glob Examples

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/glob-pattern-matching.md

Provides examples of using the `glob_files` function for recursive and subdirectory pattern matching that works across all backends.

```python
from remote_store.ext.glob import glob_files

# Recursive: find all logs at any depth
for info in glob_files(store, "**/*.log"):
    print(info.path, info.size)

# Subdirectory wildcard
for info in glob_files(store, "data/2024/*.csv"):
    print(info.path)

# Match everything
for info in glob_files(store, "**/*"):
    print(info.path)
```

--------------------------------

### AsyncBackend.name

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/reference/api/aio/backend.md

Gets the name of the backend.

```APIDOC
## AsyncBackend.name

### Description
Gets the name of the backend.

### Method
GET

### Endpoint
/name

### Response
#### Success Response (200)
- **name** (string) - The name of the backend.
```

--------------------------------

### AsyncBackend.capabilities

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/reference/api/aio/backend.md

Gets the capabilities supported by the backend.

```APIDOC
## AsyncBackend.capabilities

### Description
Gets the capabilities supported by the backend.

### Method
GET

### Endpoint
/capabilities

### Response
#### Success Response (200)
- **capabilities** (list of Capability) - A list of capabilities supported by the backend.
```

--------------------------------

### Setting up Data Lake Layers with Store.child()

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/data-lake-patterns.md

Demonstrates how to set up bronze, silver, and gold data lake layers using `Store.child()` for independent namespaces. Supports swapping backends like MemoryBackend or LocalBackend for development and testing without code changes.

```python
from remote_store import BackendConfig, Registry, RegistryConfig, StoreProfile

config = RegistryConfig(
    backends={
        "lake": BackendConfig(
            type="s3",
            options={"bucket": "my-data-lake"},
        ),
    },
    stores={"lake": StoreProfile(backend="lake", root_path="v1")},
)

with Registry(config) as registry:
    lake = registry.get_store("lake")

    bronze = lake.child("bronze")   # raw ingestion
    silver = lake.child("silver")   # cleaned and typed
    gold = lake.child("gold")       # aggregated / business-ready
```

--------------------------------

### GET /dataset_exists

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/042-ext-parquet.md

Checks if a dataset is fully committed and available.

```APIDOC
## GET /dataset_exists

### Description
Returns true if the _SUCCESS marker exists for the given dataset key.

### Parameters
#### Query Parameters
- **key** (str) - Required - The dataset storage key.

### Response
#### Success Response (200)
- **exists** (bool) - True if the dataset is complete.
```

--------------------------------

### Cleanup temporary directories

Source: https://github.com/haalfi/remote-store/blob/master/examples/notebooks/03_configuration.ipynb

Remove temporary directories created during the configuration examples.

```python
import shutil

shutil.rmtree(tmpdir)
shutil.rmtree(tmpdir2)
print("Cleaned up.")
```

--------------------------------

### Quick Start: Write and Read Parquet Datasets

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/parquet-datasets.md

Initialize `ParquetDatasetStore` and use it to write an Arrow table to a dataset and then read it back. Column projection is also demonstrated.

```python
import pyarrow as pa
from remote_store import Store
from remote_store.backends import LocalBackend
from remote_store.ext.parquet import ParquetDatasetStore

store = Store(LocalBackend("/data/warehouse"))
pds = ParquetDatasetStore(store)

# Write
table = pa.table({"id": [1, 2, 3], "name": ["Alice", "Bob", "Charlie"]})
manifest = pds.write_dataset(table, "silver/customers")

# Read
customers = pds.read_dataset("silver/customers")

# Read with column projection
ids_only = pds.read_dataset("silver/customers", columns=["id"])
```

--------------------------------

### GET /read_dataset

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/042-ext-parquet.md

Reads a Parquet dataset from the store and returns it as a concatenated PyArrow Table.

```APIDOC
## GET /read_dataset

### Description
Reads a dataset by key. Verifies the existence of the _SUCCESS marker and all parts listed in the manifest before reading.

### Parameters
#### Query Parameters
- **key** (str) - Required - The dataset storage key.
- **columns** (list[str]) - Optional - List of columns to project during read.

### Response
#### Success Response (200)
- **table** (pa.Table) - The concatenated PyArrow table.
```

--------------------------------

### BackendConfig with Retry Policy

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/025-retry-policy.md

Example of a backend configuration dictionary including a nested retry configuration.

```json
{"type": "sftp", "options": {...}, "retry": {"max_attempts": 5}}
```

--------------------------------

### Configure and Use Local Backend

Source: https://github.com/haalfi/remote-store/blob/master/docs-src/guides/backends/local.md

Configure the local backend with a root directory and use it to write text to a file. Ensure the root directory is specified in BackendConfig.

```python
from remote_store import BackendConfig, RegistryConfig, Registry, StoreProfile

config = RegistryConfig(
    backends={"local": BackendConfig(type="local", options={"root": "/data"})},
    stores={"files": StoreProfile(backend="local", root_path="files")},
)

with Registry(config) as registry:
    store = registry.get_store("files")
    store.write_text("readme.txt", "Hello!")
```

--------------------------------

### Limit folder traversal depth

Source: https://github.com/haalfi/remote-store/blob/master/sdd/specs/038-nonrecursive-folder-info.md

Examples of using max_depth to control how many levels of subdirectories are aggregated.

```python
store.get_folder_info("data", max_depth=0)
# Aggregates: data/file_a.csv, data/file_b.csv
# Excludes:   data/raw/file_c.csv

store.get_folder_info("data", max_depth=1)
# Aggregates: data/file_a.csv, data/raw/file_c.csv
# Excludes:   data/raw/2026/file_d.csv
```