### Quick Start with S3 Backend

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-docs-landing-page

A minimal example demonstrating the installation and basic usage of remote-store with an S3 backend for reading and writing text files.

```bash
pip install remote-store[s3]
```

```python
from remote_store import Store
from remote_store.backends.s3 import S3Backend

store = Store(S3Backend(bucket="my-bucket"))
store.write_text("file.txt", "hello")
print(store.read_text("file.txt"))  # 'hello'
```

--------------------------------

### Quick Start with otel_observe

Source: https://docs.remotestore.dev/stable/guides/observe

Quick start example using otel_observe to wrap a store instance. This automatically emits OpenTelemetry spans and metrics for store operations like write.

```python
from remote_store.ext.otel import otel_observe

observed = otel_observe(store)
observed.write("data/report.csv", csv_bytes)
# -> OTel span "store.write" + metrics recorded automatically

```

--------------------------------

### Quick Start Async Store

Source: https://docs.remotestore.dev/stable/guides/async

Initialize and use an AsyncStore with a MemoryBackend. This example demonstrates writing and reading bytes from the store.

```python
import asyncio

from remote_store.aio import AsyncStore
from remote_store.backends import MemoryBackend

async def main() -> None:
    async with AsyncStore(MemoryBackend(), root_path="reports") as store:
        result = await store.write("summary.txt", b"Q1 results", overwrite=True)
        print(f"wrote {result.size} bytes")
        data = await store.read_bytes("summary.txt")
        print(data.decode())

asyncio.run(main())

```

--------------------------------

### PyArrow FileSystem Adapter Demo

Source: https://docs.remotestore.dev/stable/tutorial/examples/pyarrow-adapter

This example requires PyArrow to be installed (`pip install "remote-store[arrow]"`). It shows how to create a PyArrow filesystem from a Store, write and read Parquet files, and discover dataset partitions.

```python
from __future__ import annotations

from typing import Any

try:
    import pyarrow as pa
    import pyarrow.parquet as pq
except ImportError as _exc:
    print("This example requires PyArrow: pip install 'remote-store[arrow]'")
    raise SystemExit(1) from _exc

from remote_store import Store
from remote_store.backends import MemoryBackend
from remote_store.ext.arrow import pyarrow_fs


def demo(store: Store) -> dict[str, Any]:
    """PyArrow filesystem: Parquet round-trip and dataset discovery. Returns results dict."""
    import pyarrow.dataset as ds

    results: dict[str, Any] = {}

    fs = pyarrow_fs(store)
    results["type_name"] = fs.type_name
    print(f"Filesystem type: {fs.type_name}")

    # Write a Parquet file
    table = pa.table({"id": [1, 2, 3], "name": ["Alice", "Bob", "Charlie"]})
    pq.write_table(table, "people.parquet", filesystem=fs)
    print("Wrote people.parquet")

    # Read it back
    result = pq.read_table("people.parquet", filesystem=fs)
    results["people_rows"] = result.num_rows
    results["people_data"] = result.to_pydict()
    print(f"Read back {result.num_rows} rows:")
    print(result.to_pydict())

    # File info
    info = fs.get_file_info("people.parquet")
    results["file_size"] = info.size
    print(f"\nFile info: type={info.type}, size={info.size} bytes")

    # Write multiple files for dataset discovery
    for i in range(3):
        part = pa.table({"value": [i * 10 + j for j in range(5)]})
        pq.write_table(part, f"dataset/part{i}.parquet", filesystem=fs)
    print("\nWrote 3 partitions to dataset/")

    # Discover and read all partitions
    dataset = ds.dataset("dataset", filesystem=fs, format="parquet")
    all_data = dataset.to_table()
    results["dataset_rows"] = all_data.num_rows
    results["dataset_files"] = len(dataset.files)
    print(f"Dataset: {all_data.num_rows} total rows from {len(dataset.files)} files")

    return results


def main() -> None:
    backend = MemoryBackend()
    store = Store(backend=backend)
    demo(store)
    store.close()
    print("\nDone!")


if __name__ == "__main__":
    main()

```

--------------------------------

### Navigation Structure - Getting Started Section

Source: https://docs.remotestore.dev/stable/explanation/design/audits/audit-012-docs-structure

This YAML snippet shows a part of the navigation structure where 'Tutorial' is nested under 'Getting Started'. The issue was that 'Tutorial' should be a top-level section according to project rules.

```yaml
- Getting Started:
    - Tutorial: getting-started.md
    - Examples: examples/

```

--------------------------------

### Setup OpenTelemetry SDK and Instrument Store

Source: https://docs.remotestore.dev/stable/tutorial/examples/otel-tracing

Initializes the OpenTelemetry SDK with in-memory exporters for traces and metrics, then wraps a MemoryBackend Store with otel_observe for instrumentation. Requires 'remote-store[otel]' and 'opentelemetry-sdk' to be installed.

```python
from __future__ import annotations

try:
    from opentelemetry.sdk.metrics import MeterProvider
    from opentelemetry.sdk.metrics.export import InMemoryMetricReader
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import SimpleSpanProcessor
    from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
except ImportError as _exc:
    print("This example requires: pip install 'remote-store[otel]' opentelemetry-sdk")
    raise SystemExit(1) from _exc

from opentelemetry import metrics, trace

from remote_store import Store
from remote_store.backends import MemoryBackend
from remote_store.ext.otel import otel_observe


def demo(observed: Store) -> None:
    """Store operations under OTel instrumentation."""
    observed.write("data/report.csv", b"id,value\n1,100\n2,200")
    print("Wrote data/report.csv")

    content = observed.read_bytes("data/report.csv")
    print(f"Read {len(content)} bytes from data/report.csv")

    observed.copy("data/report.csv", "data/report_backup.csv")
    print("Copied to data/report_backup.csv")

    observed.exists("data/report.csv")
    print("Checked existence")

    observed.delete("data/report_backup.csv")
    print("Deleted backup\n")


def main() -> None:
    # -- Set up OTel SDK with in-memory exporters (for demo) --------
    span_exporter = InMemorySpanExporter()
    tracer_provider = TracerProvider()
    tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter))
    trace.set_tracer_provider(tracer_provider)

    metric_reader = InMemoryMetricReader()
    meter_provider = MeterProvider(metric_readers=[metric_reader])
    metrics.set_meter_provider(meter_provider)

    # -- Create and instrument a Store ------------------------------
    store = Store(backend=MemoryBackend())
    observed = otel_observe(store)
    print("Store instrumented with OpenTelemetry\n")

    # -- Perform operations -----------------------------------------
    demo(observed)

    # -- Inspect collected spans ------------------------------------
    spans = span_exporter.get_finished_spans()
    print(f"--- {len(spans)} spans collected ---")
    for span in spans:
        attrs = dict(span.attributes or {})
        status = "OK" if span.status.is_ok else "ERROR"
        print(f"  {span.name:24s}  status={status:5s}  backend={attrs.get('remote_store.backend', '?')}")

    # -- Inspect collected metrics ----------------------------------
    data = metric_reader.get_metrics_data()
    assert data is not None
    print("\n--- Metrics ---")
    for rm in data.resource_metrics:
        for sm in rm.scope_metrics:
            for m in sm.metrics:
                points = list(m.data.data_points)
                total = sum(getattr(p, "value", 0) or getattr(p, "count", 0) for p in points)
                print(f"  {m.name:40s}  unit={m.unit:3s}  data_points={len(points)}  total={total}")

    # -- Cleanup ----------------------------------------------------
    store.close()
    tracer_provider.shutdown()
    meter_provider.shutdown()
    print("\nDone!")


if __name__ == "__main__":
    main()

```

--------------------------------

### Python Graph Setup Backend Documentation Snippet

Source: https://docs.remotestore.dev/stable/explanation/design/audits/audit-016-graph-backend-review

This example from graph-setup.md describes a forthcoming Graph backend feature using future tense, indicating outdated documentation.

```python
the forthcoming Graph backend will take one opaque `drive_id`
GraphUtils.resolve_drive_id(...) (shipping with the backend) will accept three target shapes
the libraries the built-in `GraphAuth` helper will wrap
the built-in `GraphAuth` helper, when it ships, will persist its MSAL cache
```

--------------------------------

### Download Example

Source: https://docs.remotestore.dev/stable/guides/transfer-operations

Example of downloading a file from a remote store to a local path.

```python
download(store, "reports/output.csv", "/tmp/output.csv")

```

--------------------------------

### Install remote-store with httpx HTTP Adapter

Source: https://docs.remotestore.dev/stable/reference/FEATURES

Install the httpx HTTP adapter for the ReadOnlyHttpBackend.

```bash
pip install remote-store[httpx]       # httpx HTTP adapter for ReadOnlyHttpBackend
```

--------------------------------

### Install Remote Store with Arrow Backend

Source: https://docs.remotestore.dev/stable/tutorial/getting-started

Install remote-store with the PyArrow filesystem adapter.

```bash
pip install "remote-store[arrow]"

```

--------------------------------

### Install ParquetDatasetStore

Source: https://docs.remotestore.dev/stable/guides/parquet-datasets

Install the ParquetDatasetStore by including the 'arrow' extra for PyArrow support.

```bash
pip install "remote-store[arrow]"
```

--------------------------------

### Unwrap Sync-Safe Handle Example

Source: https://docs.remotestore.dev/stable/reference/api/aio/adapters

Example demonstrating how to unwrap a sync-safe handle from an adapter.

```python
class _SafeBackend(AsyncBackend, _SyncSafeHandleProvider):
    def sync_safe_unwrap(self, type_hint):
        return self._sync_client

adapter = AsyncBackendSyncAdapter(_SafeBackend())
client = adapter.unwrap(SyncClient)


```

--------------------------------

### Install remote-store with SFTP

Source: https://docs.remotestore.dev/stable/reference/FEATURES

Install SFTP support for remote-store using the paramiko library.

```bash
pip install remote-store[sftp]        # SFTP via paramiko
```

--------------------------------

### Upload Example

Source: https://docs.remotestore.dev/stable/guides/transfer-operations

Example of uploading a local file to a remote store, with overwrite enabled.

```python
upload(store, "/data/input.csv", "input.csv", overwrite=True)

```

--------------------------------

### Quickstart: Direct Store Construction and Usage

Source: https://docs.remotestore.dev/stable/tutorial/examples/quickstart

Demonstrates the simplest way to use RemoteStore by directly constructing a Store object with a LocalBackend. This method is suitable for basic, single-backend use cases.

```Python
from __future__ import annotations

import tempfile

from remote_store import Registry, RegistryConfig, Store
from remote_store.backends import LocalBackend


def demo_direct(root: str) -> None:
    """Simplest usage: construct a Store directly."""
    store = Store(LocalBackend(root=root))
    store.write_text("hello.txt", "Hello, world!")
    print(store.read_text("hello.txt"))  # 'Hello, world!'


def demo_registry(root: str) -> None:
    """Registry usage: declarative config, multiple stores."""
    config = RegistryConfig.from_dict(
        {
            "backends": {"main": {"type": "local", "options": {"root": root}}},
            "stores": {"data": {"backend": "main", "root_path": ""}},
        }
    )

    with Registry(config) as registry:
        store = registry.get_store("data")
        store.write_text("hello.txt", "Hello, world!")
        print(store.read_text("hello.txt"))  # 'Hello, world!'


if __name__ == "__main__":
    with tempfile.TemporaryDirectory() as tmp:
        print("-- Direct construction --")
        demo_direct(f"{tmp}/direct")

        print("-- Registry config --")
        demo_registry(f"{tmp}/registry")

```

--------------------------------

### Add MkDocs Material documentation site

Source: https://docs.remotestore.dev/stable/explanation/development-story

This commit introduces a MkDocs Material documentation site, providing API references, getting started guides, and backend documentation.

```git
7fdde35  Add MkDocs Material documentation site
```

--------------------------------

### Install pytest-recording

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-bk-181-cassette-replay-poc

Installs pytest-recording and its dependency vcrpy. This is a one-time setup step.

```bash
uv pip install pytest-recording
```

--------------------------------

### Quick Start with Registry and Declarative Config

Source: https://docs.remotestore.dev/stable/tutorial/getting-started

Using a Registry with a declarative configuration to manage multiple backends and stores. Demonstrates writing and reading a text file.

```python
from remote_store import Registry, RegistryConfig

config = RegistryConfig.from_dict({
    "backends": {"main": {"type": "local", "options": {"root": "/tmp/data"}}},
    "stores": {"data": {"backend": "main", "root_path": ""}},
})

with Registry(config) as registry:
    store = registry.get_store("data")
    store.write_text("hello.txt", "Hello, world!")
    print(store.read_text("hello.txt"))  # 'Hello, world!'

```

--------------------------------

### Quick Start: Upload, Download, and Transfer

Source: https://docs.remotestore.dev/stable/guides/transfer-operations

Demonstrates the basic usage of upload, download, and transfer functions with a MemoryBackend store.

```python
from remote_store import Store, upload, download, transfer
from remote_store.backends import MemoryBackend

store = Store(backend=MemoryBackend())

# Upload a local file to the store
upload(store, "local/report.csv", "reports/report.csv")

# Download a remote file to a local path
download(store, "reports/report.csv", "local/copy.csv")

# Transfer between two stores
other = Store(backend=MemoryBackend())
transfer(store, "reports/report.csv", other, "archive/report.csv")

```

--------------------------------

### Missing `ext.write` imports in 'always-available' example

Source: https://docs.remotestore.dev/stable/explanation/design/audits/audit-011-docs-v023-gaps

The 'always-available' import example block in the extensions guide is missing imports for `ext.write`. Ensure these imports are included for users following the example.

```python
from remote_store import ext

# ...

ext.write(...) 
```

--------------------------------

### Module Docstring Example

Source: https://docs.remotestore.dev/stable/explanation/design/design

Every module should start with a 1-2 sentence docstring explaining its purpose.

```python
"""Normalized error hierarchy for remote_store."""

```

```python
"""Backend implementations for remote_store."""

```

--------------------------------

### Initialize and Use Local Backend

Source: https://docs.remotestore.dev/stable/guides/backends/local

Demonstrates how to configure and use the local backend to store a text file. Ensure the 'root' option points to a valid directory on your filesystem.

```python
from remote_store import BackendConfig, RegistryConfig, Registry, StoreProfile

config = RegistryConfig(
    backends={"local": BackendConfig(type="local", options={"root": "/data"})},
    stores={"files": StoreProfile(backend="local", root_path="files")},
)

with Registry(config) as registry:
    store = registry.get_store("files")
    store.write_text("readme.txt", "Hello!")

```

--------------------------------

### Initialize and Use Local Store

Source: https://docs.remotestore.dev/stable

Demonstrates initializing a Store with a LocalBackend and performing basic read/write operations.

```python
from remote_store import Store
from remote_store.backends import LocalBackend

store = Store(LocalBackend(root="/tmp/data"))
store.write_text("hello.txt", "Hello, world!")
print(store.read_text("hello.txt"))  # 'Hello, world!'

```

--------------------------------

### Pydantic Configuration Loading

Source: https://docs.remotestore.dev/stable/tutorial/examples/config-loaders

Loads configuration using pydantic for validation. This example is skipped if pydantic is not installed.

```python
notes.write("todo.txt", b"Ship config loaders!")
pydantic_content = notes.read_bytes("todo.txt")
results["pydantic_content"] = pydantic_content
print(f"  wrote: {pydantic_content.decode()}")
```

--------------------------------

### SFTPBackend.to_key Example

Source: https://docs.remotestore.dev/stable/explanation/design/specs/010-native-path-resolution

Shows SFTPBackend.to_key stripping the configured base_path from a native path. Paths not starting with the base_path are returned as is.

```python
backend = SFTPBackend(host="srv", base_path="/srv/sftp")
backend.to_key("/srv/sftp/data/file.txt")   # → "data/file.txt"
backend.to_key("data/file.txt")             # → "data/file.txt" (no prefix, unchanged)
```

--------------------------------

### Example Test for Graph Backend

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-bk-283-example-replay-design

This test runs the graph backend example script in both live and replay modes. It uses pytest parameterization to switch between modes and includes environment variable setup and assertions on the script's output. Use this pattern to integrate examples into the conformance suite for robust testing.

```python
pytest.importorskip("httpx", reason="httpx not installed (graph extra)")
pytest.importorskip("msal", reason="msal not installed (graph extra)")

EXAMPLE = Path(__file__).parents[3] / "examples" / "backends" / "graph_backend.py"

@pytest.mark.parametrize(
    "mode",
    [
        pytest.param("live", id="graph_live", marks=pytest.mark.live),
        pytest.param("replay", id="graph_replay", marks=pytest.mark.vcr(record_mode="none")),
    ],
)
def test_graph_backend_example(mode, monkeypatch, capsys):
    if mode == "live":
        if os.environ.get("RS_TEST_LIVE_GRAPH") != "1":
            pytest.skip("graph_live opt-in via RS_TEST_LIVE_GRAPH=1")
        require_graph_live_credentials()  # real env vars stay in place
    else:
        monkeypatch.setenv("GRAPH_TENANT_ID", "consumers")
        monkeypatch.setenv("GRAPH_CLIENT_ID", "00000000-0000-0000-0000-000000000000")
        monkeypatch.setenv("GRAPH_DRIVE_ID", FAKE_DRIVE_ID)
        monkeypatch.setattr("remote_store.aio.GraphAuth", _StubGraphAuth)

    runpy.run_path(str(EXAMPLE), run_name="__main__")

    out = capsys.readouterr().out
    assert "Wrote 2 files." in out
    assert "revenue,profit" in out  # demonstrated content, not just banners
    assert "Cleaned up all example files." in out
    assert "Done!" in out

```

--------------------------------

### Initialize and Use MemoryBackend

Source: https://docs.remotestore.dev/stable/guides/backends/memory

Demonstrates basic initialization of MemoryBackend and its integration with the Store for writing and reading text files.

```python
from remote_store import Store
from remote_store.backends import MemoryBackend

backend = MemoryBackend()
store = Store(backend=backend, root_path="data")

store.write_text("hello.txt", "Hello, world!")
print(store.read_text("hello.txt"))  # 'Hello, world!'

```

--------------------------------

### Backend Configuration Example

Source: https://docs.remotestore.dev/stable/explanation/contributing

An example demonstrating how to configure a new backend for RemoteStore. This snippet is part of the process for adding a new backend.

```python
examples/configuration/configuration.py
```

--------------------------------

### Running the Medallion Dagster Showcase

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-medallion-dagster-showcase

Commands to navigate to the showcase directory, install dependencies, and start the Dagster development server.

```bash
cd examples/medallion_dagster
pip install -e "../../[dagster,arrow,otel]" polars duckdb
dagster dev -f definitions.py

```

--------------------------------

### Initialize Store with Local Backend

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-docs-landing-page

Demonstrates how to initialize the Store with a LocalBackend, specifying a root directory. Use this for local file system operations.

```python
from remote_store import Store
from remote_store.backends.local import LocalBackend

store = Store(LocalBackend(root="/tmp/data"))
store.write_text("hello.txt", "Hello, world!")
print(store.read_text("hello.txt"))  # 'Hello, world!'
```

--------------------------------

### Importing OpenTelemetry API with Error Handling

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-logging-monitoring-tracing

Demonstrates how to import OpenTelemetry tracing and metrics APIs, including error handling for cases where the optional dependency is not installed. This ensures a clear error message guides the user to install the necessary package.

```python
# ext/notify/otel.py — top of file
try:
    from opentelemetry import trace, metrics
except ImportError as exc:
    raise ImportError(
        "Install remote-store[otel] for OpenTelemetry support"
    ) from exc
```

--------------------------------

### Direct RedisBackend Instantiation and Usage

Source: https://docs.remotestore.dev/stable/guides/custom-backend-guide

Demonstrates how to directly instantiate RedisBackend, wrap it in a Store, and perform basic operations like writing, reading, and listing files.

```python
from remote_store import Store

backend = RedisBackend(url="redis://localhost:6379/0", prefix="myapp:")
store = Store(backend=backend)

store.write("reports/q1.csv", b"revenue,100\n")
data = store.read_bytes("reports/q1.csv")
print(data)  # b'revenue,100\n'

for info in store.list_files("reports"):
    print(f"{info.name}: {info.size} bytes")
```

--------------------------------

### Initialize AzureFileSystem with Account Key

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-azure-pyarrow-optimization

Demonstrates how to create an AzureFileSystem instance using an account key for authentication. This is a direct PyArrow integration.

```python
from pyarrow.fs import AzureFileSystem

# Account-key auth
fs = AzureFileSystem(account_name="mystorageacct", account_key="...")
```

--------------------------------

### Example Classifications

Source: https://docs.remotestore.dev/stable/explanation/design/authoring

Illustrates the application of classification markers for different file types, including new guides, contributor docs, operational files, and specifications.

```markdown
<!-- A new guide. docs-src/ is docs-only by default; no marker needed. -->
# Streaming reads
...
```

```markdown
<!-- A repo-root contributor doc. Explicit dual marker; bridge emits docs/contributing/. -->
<!-- doc: dual dest=contributing.md -->
# Contributing
...
```

```markdown
<!-- A Claude-only operational file. Explicit repo-only marker; never reaches the docs site. -->
<!-- doc: repo-only -->
# Claude Code Reference
...
```

```markdown
<!-- A new spec. sdd/specs/ has a directory default; no marker needed. -->
# Spec 048 -- Some Topic
...
```

--------------------------------

### LocalBackend.to_key Example

Source: https://docs.remotestore.dev/stable/explanation/design/specs/010-native-path-resolution

Demonstrates how LocalBackend.to_key strips the filesystem root directory from a native path. If the path does not start with the backend's root, it is returned unchanged.

```python
backend = LocalBackend("/tmp/store")
backend.to_key("/tmp/store/data/file.txt")  # → "data/file.txt"
backend.to_key("data/file.txt")             # → "data/file.txt" (no prefix, unchanged)
```

--------------------------------

### Main Execution Block

Source: https://docs.remotestore.dev/stable/tutorial/examples/s3-listing-strategies

Sets up the storage registry, retrieves a store instance, executes demonstration functions, and ensures cleanup. This block orchestrates the entire demo process.

```python
if __name__ == "__main__":
    registry = setup_registry()

    with registry:
        store = registry.get_store("demo")

        try:
            setup_sample_data(store)
            demo_shallow_listing(store)
            demo_recursive_listing(store)
            demo_filtering_on_stream(store)
            demo_performance_comparison(store)
            demo_antipattern(store)

        finally:
            cleanup(store)

    print("Done! See docs-src/guides/backends/s3.md for detailed performance analysis.")
```

--------------------------------

### See Also Section

Source: https://docs.remotestore.dev/stable/explanation/design/documentation

A footer section that must include at least one entry, typically a primary guide or a matching example. It provides links for further reading.

```markdown
## See also

- [Guide name](https://github.com/haalfi/remote-store/blob/master/guide-path.md) — one-line description
- [Example name](https://github.com/haalfi/remote-store/blob/master/examples/example-path.md) — one-line description
```

--------------------------------

### Initialize SQL Query Backend and Store

Source: https://docs.remotestore.dev/stable/guides/backends/sql-query

Initialize the SQLQueryBackend with a database URL and a dictionary of queries mapped to file paths. Then, create a Store instance using this backend.

```python
from remote_store import Store
from remote_store.backends import SQLQueryBackend

backend = SQLQueryBackend(
    url="sqlite:///analytics.db",
    queries={
        "reports/daily_sales.parquet": "SELECT date, SUM(amount) FROM orders GROUP BY date",
        "reports/user_summary.csv": "SELECT * FROM user_summary_mv",
    },
)
store = Store(backend=backend)

# Read executes the query and serializes to the format implied by the extension
data = store.read_bytes("reports/daily_sales.parquet")  # Parquet bytes

```

--------------------------------

### Quick Start: Basic Globbing

Source: https://docs.remotestore.dev/stable/guides/glob-pattern-matching

Demonstrates basic usage of `list_files` for name filtering and `glob_files` for recursive globbing. Ensure the `remote_store` library and `MemoryBackend` are imported.

```python
from remote_store import Store, glob_files
from remote_store.backends import MemoryBackend

store = Store(backend=MemoryBackend())
store.write("data/report.csv", b"r1")
store.write("data/summary.csv", b"r2")
store.write("data/readme.txt", b"r3")
store.write("logs/app.log", b"l1")

# Tier 1: simple name filtering (works with every backend)
csvs = list(store.list_files("data", pattern="*.csv"))
# [FileInfo("data/report.csv", ...), FileInfo("data/summary.csv", ...)]

# Tier 3: full recursive glob (works with every backend)
all_csvs = list(glob_files(store, "**/*.csv"))
# [FileInfo("data/report.csv", ...), FileInfo("data/summary.csv", ...)]

```

--------------------------------

### Quick start Dagster IO Manager with remote-store

Source: https://docs.remotestore.dev/stable/guides/dagster

Configure a Dagster IOManager using the dagster_io_manager utility with a remote-store Store. This example uses the pickle serializer.

```python
from dagster import Definitions, asset, IOManager, io_manager
from remote_store import Store
from remote_store.backends import LocalBackend
from remote_store.ext.dagster import dagster_io_manager


@io_manager
def my_io_manager() -> IOManager:
    store = Store(LocalBackend(root="/data/dagster"))
    return dagster_io_manager(store, serializer="pickle")


@asset
def raw_data() -> dict:
    return {"rows": [1, 2, 3]}

defs = Definitions(
    assets=[raw_data],
    resources={"io_manager": my_io_manager},
)

```

--------------------------------

### Store.to_key With Unrelated Path Example

Source: https://docs.remotestore.dev/stable/explanation/design/specs/010-native-path-resolution

Demonstrates the behavior when an unrelated path is provided to `Store.to_key`. If the path, after backend stripping, does not start with the `root_path`, an `InvalidPath` exception is raised.

```python
store = Store(backend=local, root_path="data")
store.to_key("/tmp/store/other/file.txt")  # → InvalidPath (not under "data/")

```

--------------------------------

### Quick Start: Batch Operations

Source: https://docs.remotestore.dev/stable/guides/batch-operations

Demonstrates how to use batch_exists, batch_copy, and batch_delete for efficient operations on multiple paths. Supports concurrent execution for cloud backends.

```python
from remote_store import Store, batch_delete, batch_copy, batch_exists
from remote_store.backends import MemoryBackend

store = Store(backend=MemoryBackend())
store.write("a.txt", b"hello")
store.write("b.txt", b"world")

# Check which files exist
exists_map = batch_exists(store, ["a.txt", "b.txt", "c.txt"])
# {"a.txt": True, "b.txt": True, "c.txt": False}

# Copy multiple files
result = batch_copy(store, [("a.txt", "a_copy.txt"), ("b.txt", "b_copy.txt")])
assert result.all_succeeded

# Delete multiple files
result = batch_delete(store, ["a.txt", "b.txt"], missing_ok=True)
assert result.all_succeeded

```

--------------------------------

### Optional Dependency Check in Extension Module

Source: https://docs.remotestore.dev/stable/explanation/contributing

In the extension module, check for the presence of an optional dependency and raise a helpful error if it's missing, guiding the user on how to install it.

```python
# In ext/<name>.py:
try:
    import some_lib
except ModuleNotFoundError as _exc:
    raise ModuleNotFoundError(
        "some_lib is required for the <name> extension. "
        "Install it with: pip install 'remote-store[<name>]'"
    ) from _exc

```

--------------------------------

### Configure and Use HTTP Backend via Registry

Source: https://docs.remotestore.dev/stable/guides/backends/http

This example shows how to configure an HTTP backend within a registry and then retrieve a store instance. It's useful for managing multiple backends and stores centrally.

```python
from remote_store import BackendConfig, RegistryConfig, Registry, StoreProfile

config = RegistryConfig(
    backends={
        "opendata": BackendConfig(
            type="http",
            options={
                "base_url": "https://data.example.com/datasets/",
                "timeout": 60,
                "headers": {"X-API-Key": "YOUR_KEY"},
            },
        ),
    },
    stores={"data": StoreProfile(backend="opendata")},
)

with Registry(config) as registry:
    store = registry.get_store("data")
    content = store.read_bytes("population/2024.csv")

```

--------------------------------

### Configure and Use S3 Backend

Source: https://docs.remotestore.dev/stable/tutorial/examples/s3-backend

Sets up an S3 backend and demonstrates various file operations like write, read, list, copy, move, and delete. It also shows how to handle S3's virtual folder semantics and perform streaming reads. Ensure the RS_S3_BUCKET environment variable is set.

```python
from __future__ import annotations

import os
import sys

from remote_store import BackendConfig, Registry, RegistryConfig, StoreProfile

BUCKET = os.environ.get("RS_S3_BUCKET", "")

if not BUCKET:
    print(
        "Set RS_S3_BUCKET to run this example.\n"
        "Optional: RS_S3_KEY, RS_S3_SECRET, RS_S3_ENDPOINT, RS_S3_REGION\n\n"
        "Example with MinIO:\n"
        "  RS_S3_BUCKET=my-bucket RS_S3_KEY=minioadmin RS_S3_SECRET=minioadmin "
        "RS_S3_ENDPOINT=http://localhost:9000 python examples/s3_backend.py"
    )
    sys.exit(1)

if __name__ == "__main__":
    # --- Build options from environment ---
    options: dict[str, object] = {"bucket": BUCKET}
    if val := os.environ.get("RS_S3_KEY"):
        options["key"] = val
    if val := os.environ.get("RS_S3_SECRET"):
        options["secret"] = val
    if val := os.environ.get("RS_S3_ENDPOINT"):
        options["endpoint_url"] = val
    if val := os.environ.get("RS_S3_REGION"):
        options["region_name"] = val

    # --- Two stores on one bucket, different root_path ---
    config = RegistryConfig(
        backends={"s3": BackendConfig(type="s3", options=options)},
        stores={
            "data": StoreProfile(backend="s3", root_path="example/data"),
            "logs": StoreProfile(backend="s3", root_path="example/logs"),
        },
    )

    with Registry(config) as registry:
        data = registry.get_store("data")
        logs = registry.get_store("logs")

        # --- Write ---
        data.write("report.csv", b"revenue,profit\n100,20\n200,40\n")
        data.write("notes/todo.txt", b"Ship v1.0")
        logs.write("app.log", b"[INFO] started\n[WARN] disk 80%\n")
        print("Wrote 3 files across 2 stores.")

        # --- Read ---
        content = data.read_bytes("report.csv")
        print(f"\nreport.csv:\n{content.decode()}")

        # --- Metadata ---
        info = data.get_file_info("report.csv")
        print(f"report.csv  size={info.size}  modified={info.modified_at}")

        # --- List files (shallow) ---
        print("\ndata/ (shallow):")
        for f in data.list_files(""):
            print(f"  {f.name}  ({f.size} bytes)")

        # --- List files (recursive) ---
        print("\ndata/ (recursive):")
        for f in data.list_files("", recursive=True):
            print(f"  {f.path}  ({f.size} bytes)")

        # --- List folders ---
        print("\nFolders in data/:")
        for folder in data.list_folders(""):
            print(f"  {folder.name}/")

        # --- Folder info ---
        folder_info = data.get_folder_info("notes")
        print(f"\nnotes/ totals: {folder_info.file_count} files, {folder_info.total_size} bytes")

        # --- Copy ---
        data.copy("report.csv", "report_backup.csv")
        print(f"\nCopied report.csv -> report_backup.csv (exists: {data.exists('report_backup.csv')})")

        # --- Move ---
        data.move("report_backup.csv", "archive/report_old.csv")
        print(f"Moved to archive/report_old.csv (original exists: {data.exists('report_backup.csv')})")

        # --- Streaming read ---
        reader = data.read("report.csv")
        print("\nStreaming read (line by line):")
        newline = b"\n"
        for line in reader:
            text = line.rstrip(newline).decode()
            if text:
                print(f"  {text}")

        # --- Cleanup ---
        # S3 folders are virtual (prefix-based). Deleting all files under a
        # prefix makes the "folder" vanish automatically.
        for f in data.list_files("", recursive=True):
            data.delete(str(f.path))
        for f in logs.list_files("", recursive=True):
            logs.delete(str(f.path))
        print("\nCleaned up all example files.")

    print("\nDone!")

```

--------------------------------

### Quick Start: Observe Store Operations

Source: https://docs.remotestore.dev/stable/guides/observe

Use the `observe` function to wrap a store and define callbacks for operations like writes. This example shows how to log write events.

```python
from remote_store import observe

def on_write(event):
    print(f"Wrote {event.path} in {event.duration_ms:.1f}ms")

observed = observe(store, on_write=on_write)
observed.write("data/report.csv", csv_bytes)
# prints: Wrote data/report.csv in 12.3ms
```

--------------------------------

### OpenTelemetry Tracing API Usage

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-logging-monitoring-tracing

Implement distributed tracing for library operations using the OpenTelemetry Tracing API. This example shows how to start a span, add attributes, and record results.

```python
from opentelemetry import trace

tracer = trace.get_tracer("remote_store")

with tracer.start_as_current_span("store.read", attributes={
    "remote_store.backend": "s3",
    "remote_store.path": "data/file.csv",
}) as span:
    result = backend.read(path)
    span.set_attribute("remote_store.bytes", len(result))

```

--------------------------------

### Usage Examples for `from_toml()`

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-store-config

Demonstrates how to use the `from_toml` method to load configuration, both from a standalone TOML file and from a nested table within `pyproject.toml`.

```python
# Standalone file
config = RegistryConfig.from_toml("remote-store.toml")

# From pyproject.toml
config = RegistryConfig.from_toml("pyproject.toml", table=("tool", "remote-store"))

```

--------------------------------

### PipelinedStore Middleware Example

Source: https://docs.remotestore.dev/stable/explanation/design/research/research-store-middleware-architecture

Demonstrates setting up a Store with a middleware pipeline, where operations are processed through an ordered list of middleware components. This approach centralizes proxy layers but can be complex for diverse method signatures.

```python
store = pipeline(
    store,
    ChecksumMiddleware(verify=True),
    CacheMiddleware(ttl=300),
    ProgressMiddleware(callback=update_bar),
    ObserveMiddleware(on_read=log_it),
)
```

--------------------------------

### Get RemotePath Parent

Source: https://docs.remotestore.dev/stable/reference/api/models

Access the parent path. Returns None if the path has only one component. For example, `RemotePath("a/b").parent` returns `RemotePath("a")`, but `RemotePath("a").parent` returns `None`.

```python
parent: RemotePath | None
```