### Run Obstore Progress Bar Example

Source: https://github.com/developmentseed/obstore/blob/main/examples/progress-bar/README.md

Execute the main Python script to start the progress bar example.

```shell
uv run python main.py
```

--------------------------------

### Run Zarr Example with Obstore

Source: https://github.com/developmentseed/obstore/blob/main/examples/zarr/README.md

Execute the main Python script to run the Zarr example using Obstore.

```bash
uv run main.py
```

--------------------------------

### Install obstore using pip

Source: https://github.com/developmentseed/obstore/blob/main/README.md

Use this command to install the obstore library via pip.

```sh
pip install obstore
```

--------------------------------

### Run Obstore Progress Bar with Custom URL

Source: https://github.com/developmentseed/obstore/blob/main/examples/progress-bar/README.md

Test the progress bar example by providing an arbitrary URL for streaming data.

```shell
uv run python main.py https://ookla-open-data.s3.us-west-2.amazonaws.com/parquet/performance/type=fixed/year=2019/quarter=1/2019-01-01_performance_fixed_tiles.parquet
```

--------------------------------

### Install obstore using conda

Source: https://github.com/developmentseed/obstore/blob/main/README.md

Use this command to install the obstore library via conda from the conda-forge channel.

```sh
conda install -c conda-forge obstore
```

--------------------------------

### Read and Plot Zarr Data from Azure Blob Storage

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/zarr.md

This example shows how to open a Zarr dataset stored in Azure Blob Storage using Obstore and Xarray. It utilizes pystac-client to find the Zarr store's metadata on Microsoft's Planetary Computer and then plots a subset of the data using Matplotlib. Ensure you have the necessary libraries installed.

```python
import matplotlib.pyplot as plt
import pystac_client
import xarray as xr
from zarr.storage import ObjectStore

from obstore.auth.planetary_computer import PlanetaryComputerCredentialProvider
from obstore.store import AzureStore

# These first lines are specific to Zarr stored in the Microsoft Planetary
# Computer. We use pystac-client to find the metadata for this specific Zarr
# store.
catalog = pystac_client.Client.open(
    "https://planetarycomputer.microsoft.com/api/stac/v1/",
)
collection = catalog.get_collection("daymet-daily-hi")
asset = collection.assets["zarr-abfs"]

# We construct an AzureStore because this Zarr dataset is stored in Azure
# storage
azure_store = AzureStore(
    credential_provider=PlanetaryComputerCredentialProvider.from_asset(asset),
)

# Next we use the Zarr ObjectStorage adapter and pass it to xarray.
zarr_store = ObjectStore(azure_store, read_only=True)
ds = xr.open_dataset(zarr_store, consolidated=True, engine="zarr")

# And plot with matplotlib
fig, ax = plt.subplots(figsize=(12, 12))
ds.sel(time="2009")["tmax"].mean(dim="time").plot.imshow(ax=ax, cmap="inferno")
fig.savefig("zarr-example.png")

```

--------------------------------

### Using Obstore S3Store with Obspec Download Function

Source: https://github.com/developmentseed/obstore/blob/main/docs/obspec.md

This example shows how to pass an Obstore S3Store instance to the generic download_file function, demonstrating compatibility between Obstore and Obspec.

```python
from obstore.store import S3Store

store = S3Store("bucket", ...)
download_file(store)
```

--------------------------------

### Example: List Sentinel COGS Objects with Arrow and Pandas

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Demonstrates listing objects from the sentinel-cogs bucket using `S3Store`, returning Arrow `RecordBatch` objects, and converting them to a Pandas DataFrame for display. This example requires `pandas` and `pyarrow`.

```python
import pandas as pd
import pyarrow as pa
from obstore.store import S3Store

store = S3Store("sentinel-cogs", region="us-west-2", skip_signature=True)
stream = store.list(chunk_size=20, return_arrow=True)

for record_batch in stream:
    # Convert to pyarrow (zero-copy), then to pandas for easy export to a
    # Markdown table
    df = pa.record_batch(record_batch).to_pandas()
    print(df.iloc[:5].to_markdown(index=False))
    break
```

--------------------------------

### Download File Generically with Obspec Get Protocol

Source: https://github.com/developmentseed/obstore/blob/main/docs/obspec.md

This snippet demonstrates how to download a file using the Obspec Get protocol. It defines a function that accepts any client implementing the Get protocol and processes the file in chunks.

```python
from obspec import Get

def download_file(client: Get):
    response = client.get("my-file.txt")
    for buffer in response:
        # Process each buffer chunk as needed
        print(f"Received buffer of size: {len(memoryview(buffer))} bytes")
```

--------------------------------

### Synchronous Download with tqdm Progress Bar

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/tqdm.md

Wrap synchronous obstore downloads with a tqdm progress bar to visualize download progress. Ensure tqdm is installed.

```python
from obstore.store import HTTPStore
from tqdm import tqdm

store = HTTPStore.from_url("https://ookla-open-data.s3.us-west-2.amazonaws.com")
path = "parquet/performance/type=fixed/year=2019/quarter=1/2019-01-01_performance_fixed_tiles.parquet"
response = obs.get(store, path)
file_size = response.meta["size"]
with tqdm(total=file_size) as pbar:
    for bytes_chunk in response:
        # Do something with buffer
        pbar.update(len(bytes_chunk))
```

--------------------------------

### Create obstore store from URL

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.4.md

Use `obstore.store.from_url` to create a store instance from a URL. Pass store-specific parameters as keyword arguments. This example demonstrates listing objects and downloading a file from a public S3 bucket.

```python
import obstore as obs
from obstore.store import from_url

# The base path within the bucket to "mount" to
url = "s3://sentinel-cogs/sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2A_12SUF_20220601_0_L2A"

# Pass in store-specific parameters as keyword arguments
# Here, we pass `skip_signature=True` because it's a public bucket
store = from_url(url, region="us-west-2", skip_signature=True)

# Print filenames in this directory
print([meta["path"] for meta in obs.list_with_delimiter(store)["objects"]])
# ['AOT.tif', 'B01.tif', 'B02.tif', 'B03.tif', 'B04.tif', 'B05.tif', 'B06.tif', 'B07.tif', 'B08.tif', 'B09.tif', 'B11.tif', 'B12.tif', 'B8A.tif', 'L2A_PVI.tif', 'S2A_12SUF_20220601_0_L2A.json', 'SCL.tif', 'TCI.tif', 'WVP.tif', 'granule_metadata.xml', 'thumbnail.jpg', 'tileinfo_metadata.json']

# Download thumbnail
with open("thumbnail.jpg", "wb") as f:
    f.write(obs.get(store, "thumbnail.jpg").bytes())

```

--------------------------------

### Run MinIO Locally with Docker

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/minio.md

Use this command to start a MinIO server instance locally. Ensure the ports are not in use by other applications.

```shell
docker run -p 9000:9000 -p 9001:9001 \
    quay.io/minio/minio server /data --console-address ":9001"
```

--------------------------------

### Open and Read File with FsspecStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/integrations/fsspec.md

Use the open method of FsspecStore to get a file-like object for reading.

```python
from obstore.fsspec import FsspecStore

fs = FsspecStore("s3", region="us-west-2", skip_signature=True)

with fs.open(
    "s3://sentinel-cogs/sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2B_12SUF_20220609_0_L2A/thumbnail.jpg",
) as file:
    content = file.read()
```

--------------------------------

### Interact with MemoryStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/getting-started.md

Shows basic interactions with a MemoryStore, including putting and getting data. The `get` operation returns an object with metadata.

```python
from obstore.store import MemoryStore

store = MemoryStore()

store.put("file.txt", b"hello world!")
response = store.get("file.txt")
response.meta
# {"path": "file.txt",
#  "last_modified": datetime.datetime(2024, 10, 21, 16, 19, 45, 102620, tzinfo=datetime.timezone.utc),
#  "size": 12,
#  "e_tag": "0",
```

--------------------------------

### Method-based API Example

Source: https://github.com/developmentseed/obstore/blob/main/docs/method-vs-functional-api.md

Use the method-based API for direct interaction with store objects. This approach is generally less verbose and integrates with Obspec protocols.

```python
from obstore.store import S3Store

store = S3Store("bucket", ...)
buffer = store.get_range("path", start=0, end=16384)
```

--------------------------------

### Install Maturin Import Hook

Source: https://github.com/developmentseed/obstore/blob/main/DEVELOP.md

Ensure obstore is automatically recompiled when changes are made by installing the Maturin import hook. This streamlines the development cycle by providing live updates upon Python import.

```bash
uv run python -m maturin_import_hook site install
```

--------------------------------

### Define a Synchronous Custom Credential Provider

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.5.md

Provides an example of a simple synchronous Python function that can be used as a custom credential provider for Obstore, returning necessary authentication details.

```python
from datetime import datetime, timedelta, UTC

def get_credentials() -> S3Credential:
    return {
        "access_key_id": "...",
        "secret_access_key": "...",
        # Not always required
        "token": "...",
        "expires_at": datetime.now(UTC) + timedelta(minutes=30),
    }
```

--------------------------------

### Use Google Auth for GCS Credentials

Source: https://github.com/developmentseed/obstore/blob/main/docs/authentication.md

Use GoogleCredentialProvider to integrate with google.auth for GCS credentials. Ensure google.auth is installed and configured.

```python
from obstore.auth.google import GoogleCredentialProvider
from obstore.store import GCSStore

credential_provider = GoogleCredentialProvider(credentials=...)
store = GCSStore("bucket_name", credential_provider=credential_provider)
```

--------------------------------

### Functional API Example

Source: https://github.com/developmentseed/obstore/blob/main/docs/method-vs-functional-api.md

Utilize the functional API for operations that require passing the store object as an argument. This API supports features like signed URLs and file-like objects.

```python
import obstore as obs
from obstore.store import S3Store

store = S3Store("bucket", ...)
buffer = obs.get_range(store, "path", start=0, end=16384)
```

--------------------------------

### Construct and List Files with FsspecStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/integrations/fsspec.md

Instantiate FsspecStore to create an fsspec-compatible filesystem. Use the ls method to list items in a given prefix.

```python
from obstore.fsspec import FsspecStore

fs = FsspecStore("s3", region="us-west-2", skip_signature=True)
prefix = (
    "s3://sentinel-cogs/sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2B_12SUF_20220609_0_L2A/"
)
items = fs.ls(prefix)
# [{'name': 'sentinel-cogs/sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2B_12SUF_20220609_0_L2A/AOT.tif',
#   'size': 80689,
#   'type': 'file',
#   'e_tag': '"c93b0f6b0e2cf8e375968f41161f9df7"'},
#   ...
```

--------------------------------

### Serve Documentation Website Locally

Source: https://github.com/developmentseed/obstore/blob/main/DEVELOP.md

Serve the mkdocs documentation website locally to preview changes. This command uses uv to manage the environment and run the mkdocs serve command.

```bash
uv run --group docs mkdocs serve
```

--------------------------------

### Interact with MinIO using S3Store

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/minio.md

This Python code demonstrates how to initialize an S3Store for MinIO, upload, list, download, and delete files. Ensure a bucket named 'test-bucket' is created in MinIO before running.

```python
from obstore.store import S3Store

store = S3Store(
    "test-bucket",
    endpoint="http://localhost:9000",
    access_key_id="minioadmin",
    secret_access_key="minioadmin",
    virtual_hosted_style_request=False,
    client_options={\"allow_http\": True},
)

# Add files
store.put("a.txt", b"foo")
store.put("b.txt", b"bar")
store.put("c/d.txt", b"baz")

# List files
files = store.list().collect()
print(files)

# Download a file
resp = store.get("a.txt")
print(resp.bytes())

# Delete a file
store.delete("a.txt")
```

--------------------------------

### Using NASA Earthdata Credential Provider with S3Store

Source: https://github.com/developmentseed/obstore/blob/main/docs/authentication.md

Demonstrates how to instantiate an S3Store using a custom NASA Earthdata credential provider.

```python
credential_provider = NasaEarthdataCredentialProvider(username="...", password="...")
store = S3Store("bucket_name", credential_provider=credential_provider)
```

--------------------------------

### Initialize FastAPI App with Obstore Imports

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/fastapi.md

Import necessary modules from FastAPI and obstore to set up your application.

```python
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

from obstore.store import HTTPStore, S3Store

app = FastAPI()
```

--------------------------------

### Configure fsspec.filesystem with Obstore

Source: https://github.com/developmentseed/obstore/blob/main/docs/integrations/fsspec.md

Register Obstore for the 's3' protocol and then create an fsspec filesystem, passing configuration parameters to fsspec.filesystem.

```python
import fsspec

from obstore.fsspec import register

register("s3")

fs = fsspec.filesystem("s3", region="us-west-2", skip_signature=True)
```

--------------------------------

### Initialize AzureStore with PlanetaryComputerCredentialProvider

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.6.md

Construct an AzureStore using PlanetaryComputerCredentialProvider to automatically handle token access and refresh for Microsoft Planetary Computer data. The account, container, and prefix are passed down to AzureStore automatically.

```python
from obstore.store import AzureStore
from obstore.auth.planetary_computer import PlanetaryComputerCredentialProvider

url = "https://naipeuwest.blob.core.windows.net/naip/v002/mt/2023/mt_060cm_2023/"

# Construct an AzureStore with this credential provider.
#
# The account, container, and container prefix are passed down to AzureStore
# automatically.
store = AzureStore(credential_provider=PlanetaryComputerCredentialProvider(url))
```

--------------------------------

### Initialize S3Store with inferred bucket region

Source: https://github.com/developmentseed/obstore/blob/main/docs/troubleshooting/aws.md

Demonstrates how to initialize an S3Store by first inferring the bucket's region using the `find_bucket_region` function and then passing it to the S3Store constructor.

```python
bucket_name = "sentinel-cogs"
store = S3Store(
    bucket_name, skip_signature=True, region=find_bucket_region(bucket_name)
)
```

--------------------------------

### Custom Authentication Callback for Obstore

Source: https://github.com/developmentseed/obstore/blob/main/docs/authentication.md

Illustrates how to provide custom synchronous or asynchronous authentication callbacks for Obstore's storage clients (S3Store, GCSStore, AzureStore). Asynchronous providers require an active event loop.

```python
# A custom AWS credential provider, passed in to [`S3Store`][obstore.store.S3Store] must return an [`S3Credential`][obstore.store.S3Credential].
# A custom GCS credential provider, passed in to [`GCSStore`][obstore.store.GCSStore] must return a [`GCSCredential`][obstore.store.GCSCredential].
# A custom Azure credential provider, passed in to [`AzureStore`][obstore.store.AzureStore] must return an [`AzureCredential`][obstore.store.AzureCredential].
```

--------------------------------

### Initialize AzureStore from STAC Asset with PlanetaryComputerCredentialProvider

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.6.md

Create a PlanetaryComputerCredentialProvider from a STAC asset and use it to initialize an AzureStore. This is useful for accessing data referenced by the Planetary Computer STAC API.

```python
import pystac_client

from obstore.auth.planetary_computer import PlanetaryComputerCredentialProvider
from obstore.store import AzureStore

stac_url = "https://planetarycomputer.microsoft.com/api/stac/v1/"
# Open the STAC Catalog
catalog = pystac_client.Client.open(stac_url)

# Access a specific Collection and Asset
collection = catalog.get_collection("daymet-daily-hi")
asset = collection.assets["zarr-abfs"]

# Then we can pass this directly to `from_asset`
credential_provider = PlanetaryComputerCredentialProvider.from_asset(asset)

# Print objects at the root of this directory
store = AzureStore(credential_provider=credential_provider)
print(store.list_with_delimiter()["objects"])
```

--------------------------------

### Access Store Configuration Parameters

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.4.md

Demonstrates how to access configuration parameters like config, client_options, and retry_config from an S3Store instance. This behavior is consistent across S3Store, GCSStore, and AzureStore.

```python
from obstore.store import S3Store

store = S3Store.from_url(
    "s3://ookla-open-data/parquet/performance/type=fixed/year=2024/quarter=1",
    region="us-west-2",
    skip_signature=True,
)
new_store = S3Store(
    config=store.config,
    prefix=store.prefix,
    client_options=store.client_options,
    retry_config=store.retry_config,
)
assert store.config == new_store.config
assert store.prefix == new_store.prefix
assert store.client_options == new_store.client_options
assert store.retry_config == new_store.retry_config
```

--------------------------------

### Asynchronous Download with tqdm Progress Bar

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/tqdm.md

Integrate tqdm progress bars with asynchronous obstore downloads. This is useful for non-blocking download operations.

```python
from obstore.store import HTTPStore
from tqdm import tqdm

store = HTTPStore.from_url("https://ookla-open-data.s3.us-west-2.amazonaws.com")
path = "parquet/performance/type=fixed/year=2019/quarter=1/2019-01-01_performance_fixed_tiles.parquet"
response = await obs.get_async(store, path)
file_size = response.meta["size"]
with tqdm(total=file_size) as pbar:
    async for bytes_chunk in response:
        # Do something with buffer
        pbar.update(len(bytes_chunk))
```

--------------------------------

### obstore.open_writer_async

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/file.md

Opens a writable file to an object store asynchronously.

```APIDOC
## obstore.open_writer_async

### Description
Opens a writable file to an object store asynchronously.

### Method
Python async function call

### Endpoint
N/A (Python async function)

### Parameters
(Details for parameters are not provided in the source text)

### Request Example
```python
# Example usage (details not provided in source)
async def main():
    writer = await obstore.open_writer_async(path)
    await writer.write(data)
```

### Response
#### Success Response
- Returns an async file-like object (obstore.AsyncWritableFile).

#### Response Example
```json
# Example response structure (details not provided in source)
{
  "status": "success"
}
```
```

--------------------------------

### Fetch and save image thumbnail from AzureStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.6.md

Fetch the content of a specific file (e.g., an image thumbnail) from an AzureStore and write it to a local file.

```python
path = "44106/m_4410602_nw_13_060_20230712_20240103.200.jpg"
image_content = store.get(path).bytes()

# Write out the image content to a file in the current directory
with open("thumbnail.jpg", "wb") as f:
    f.write(image_content)
```

--------------------------------

### Run FastAPI Development Server

Source: https://github.com/developmentseed/obstore/blob/main/examples/fastapi/README.md

Use `uvicorn` to run the FastAPI development server. Ensure your main application file is named `main.py`.

```bash
uv run fastapi dev main.py
```

--------------------------------

### Asynchronous File Operations with MemoryStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/getting-started.md

Demonstrates basic file operations using the asynchronous API of MemoryStore. Use this for I/O-bound operations in async applications.

```python
from obstore.store import MemoryStore

store = MemoryStore()

await store.put_async("file.txt", b"hello world!")
response = await store.get_async("file.txt")
response.meta
# {'path': 'file.txt',
#  'last_modified': datetime.datetime(2024, 10, 21, 16, 20, 36, 477418, tzinfo=datetime.timezone.utc),
#  'size': 12,
#  'e_tag': '0',
#  'version': None}
assert await response.bytes_async() == b"hello world!"

byte_range = await store.get_range_async("file.txt", start=0, end=5)
assert byte_range == b"hello"

await store.copy_async("file.txt", "other.txt")
resp = await store.get_async("other.txt")
assert await resp.bytes_async() == b"hello world!"
```

--------------------------------

### Download Object to Disk

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Demonstrates downloading an object from storage to a local file. Iterating over the response stream prevents buffering the entire file into memory, making it suitable for large files.

```python
resp = store.get(path)

with open("output/file", "wb") as f:
    for chunk in resp:
        f.write(chunk)
```

--------------------------------

### Open Remote Objects as File-like Readers/Writers

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.4.md

Shows how to use `obstore.open_writer` to write data to a remote object and `obstore.open_reader` to read data from it. This feature allows treating remote objects like local files.

```python
import os

import obstore as obs
from obstore.store import MemoryStore

# Create an in-memory store
store = MemoryStore()

# Iteratively write to the file
with obs.open_writer(store, "new_file.csv") as writer:
    writer.write(b"col1,col2,col3\n")
    writer.write(b"a,1,True\n")
    writer.write(b"b,2,False\n")
    writer.write(b"c,3,True\n")


# Open a reader from the file
reader = obs.open_reader(store, "new_file.csv")
file_length = reader.seek(0, os.SEEK_END)
print(file_length) # 43
reader.seek(0)
buf = reader.read()
print(buf)
# Bytes(b"col1,col2,col3\na,1,True\nb,2,False\nc,3,True\n")
```

--------------------------------

### Compile obstore with Maturin

Source: https://github.com/developmentseed/obstore/blob/main/DEVELOP.md

Use this command to compile the obstore library and add it to your uv-managed Python environment. This is the standard command for local development builds.

```bash
uv run maturin dev -m obstore/Cargo.toml
```

--------------------------------

### Synchronous File Operations with MemoryStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/getting-started.md

Demonstrates basic file operations using the synchronous API of MemoryStore. Use this for simple, non-blocking operations.

```python
from obstore.store import MemoryStore

store = MemoryStore()

store.put("file.txt", b"hello world!")
response = store.get("file.txt")
assert response.meta['path'] == 'file.txt'
assert response.bytes() == b"hello world!"

byte_range = store.get_range("file.txt", start=0, end=5)
assert byte_range == b"hello"

store.copy("file.txt", "other.txt")
assert store.get("other.txt").bytes() == b"hello world!"
```

--------------------------------

### obstore.open_writer

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/file.md

Opens a writable file to an object store synchronously.

```APIDOC
## obstore.open_writer

### Description
Opens a writable file to an object store synchronously.

### Method
Python function call

### Endpoint
N/A (Python function)

### Parameters
(Details for parameters are not provided in the source text)

### Request Example
```python
# Example usage (details not provided in source)
writer = obstore.open_writer(path)
writer.write(data)
```

### Response
#### Success Response
- Returns a file-like object (obstore.WritableFile).

#### Response Example
```json
# Example response structure (details not provided in source)
{
  "status": "success"
}
```
```

--------------------------------

### Read Parquet Schema using PyArrow and FsspecStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/pyarrow.md

Demonstrates how to initialize an FsspecStore for S3 and use PyArrow's ParquetFile to read the schema of a remote Parquet file. Ensure you have the necessary AWS credentials and permissions configured.

```python
import pyarrow.parquet as pq

from obstore.fsspec import FsspecStore

fs = FsspecStore("s3", skip_signature=True, region="us-west-2")

url = "s3://overturemaps-us-west-2/release/2025-02-19.0/theme=addresses/type=address/part-00010-e084a2d7-fea9-41e5-a56f-e638a3307547-c000.zstd.parquet"
parquet_file = pq.ParquetFile(url, filesystem=fs)
print(parquet_file.schema_arrow)
```

--------------------------------

### obstore.open_reader_async

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/file.md

Opens a readable file from an object store asynchronously.

```APIDOC
## obstore.open_reader_async

### Description
Opens a readable file from an object store asynchronously.

### Method
Python async function call

### Endpoint
N/A (Python async function)

### Parameters
(Details for parameters are not provided in the source text)

### Request Example
```python
# Example usage (details not provided in source)
async def main():
    reader = await obstore.open_reader_async(path)
```

### Response
#### Success Response
- Returns an async file-like object (obstore.AsyncReadableFile).

#### Response Example
```json
# Example response structure (details not provided in source)
{
  "content": "..."
}
```
```

--------------------------------

### Use Top-Level obstore.put Function

Source: https://github.com/developmentseed/obstore/blob/main/docs/dev/functional-api.md

Users must use the top-level `obstore.put` function, passing the store as the first argument. This ensures a consistent API across different store types.

```python
import obstore as obs
from obstore.store import AzureStore

store = AzureStore()
obs.put(store, ....)
```

--------------------------------

### Register Obstore as Global fsspec Handler

Source: https://github.com/developmentseed/obstore/blob/main/docs/integrations/fsspec.md

Register Obstore to handle specified protocols globally. Then use fsspec.filesystem or fsspec.open to interact with storage.

```python
import fsspec
from obstore.fsspec import register

# Register obstore as the default handler for all protocols supported by
# obstore.
# You may wish to register only specific protocols, instead.
register()

# Create a new fsspec filesystem for the given protocol
fs = fsspec.filesystem("https")
content = fs.cat_file("https://example.com/")

# Or, open the file directly
url = "https://github.com/opengeospatial/geoparquet/raw/refs/heads/main/examples/example.parquet"
with fsspec.open(url) as file:
    content = file.read()
```

--------------------------------

### Create Zip Archive and Upload to Store

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/stream-zip.md

This function copies files from an input store into a zip archive and uploads the archive to an output store. It uses `stream_zip.async_stream_zip` to create the archive on the fly, passing a generator of member files directly to the `put_async` method of the output store. This approach minimizes memory usage.

```python
from __future__ import annotations

import asyncio
from pathlib import Path
from stat import S_IFREG
from typing import TYPE_CHECKING

import stream_zip
from stream_zip import ZIP_32, AsyncMemberFile

from obstore.store import LocalStore, MemoryStore

if TYPE_CHECKING:
    from collections.abc import AsyncIterable, Iterable

    from obstore.store import ObjectStore


async def member_file(store: ObjectStore, path: str) -> AsyncMemberFile:
    """Create a member file for the zip archive."""
    resp = await store.get_async(path)
    last_modified = resp.meta["last_modified"]
    mode = S_IFREG | 0o644
    # Unclear why but we need to wrap the response in an async generator
    return (path, last_modified, mode, ZIP_32, (byte async for byte in resp.stream()))


async def member_files(
    store: ObjectStore,
    paths: Iterable[str],
) -> AsyncIterable[AsyncMemberFile]:
    """Create an async iterable of files for the zip archive."""
    for path in paths:
        yield await member_file(store, path)


async def zip_copy() -> None:
    """Copy files from one store into a zip archive that we upload to another store."""
    # Input store with source data
    input_store = MemoryStore()
    input_store.put("foo", b"hello")
    input_store.put("bar", b"world")

    # Output store where the zip file will be saved
    output_store = LocalStore(Path())

    # We can pass the streaming zip directly to `put`
    await output_store.put_async(
        "my.zip",
        stream_zip.async_stream_zip(
            member_files(input_store, ["foo", "bar"]),
            chunk_size=10 * 1024 * 1024,
        ),
    )

```

--------------------------------

### Compile obstore with Release Optimizations

Source: https://github.com/developmentseed/obstore/blob/main/DEVELOP.md

Compile obstore with release optimizations enabled for benchmarking purposes. This command ensures that performance tests reflect production-like build configurations.

```bash
uv run maturin dev -m obstore/Cargo.toml --release
```

--------------------------------

### Construct S3Store Manually for Cloudflare R2

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/r2.md

Alternatively, construct an S3Store instance manually by providing the bucket name, endpoint URL, Access Key ID, and Secret Access Key.

```python
from obstore.store import S3Store

access_key_id = "..."
secret_access_key = "..."
bucket = "kylebarron-public"
endpoint = "https://f0b62eebfbdde1133378bfe3958325f6.r2.cloudflarestorage. com"
store = S3Store(
    bucket,
    access_key_id=access_key_id,
    secret_access_key=secret_access_key,
    endpoint=endpoint,
)
store.list_with_delimiter()
```

--------------------------------

### Use Class Method Wrapper for Obstore Operations

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.5.md

Demonstrates the new class method wrapper for performing operations like 'put' directly on a store object, replacing the previous functional API.

```python
import obstore as obs
from obstore.store import AzureStore

store = AzureStore()
obs.put(store, ...)
```

```python
from obstore.store import AzureStore

store = AzureStore()
store.put(...) # (1)!
```

--------------------------------

### Type Hinting Store Configuration with fsspec

Source: https://github.com/developmentseed/obstore/blob/main/docs/integrations/fsspec.md

Use a typed configuration dictionary with fsspec.filesystem for better type checking and IDE suggestions. Import S3Config within an if TYPE_CHECKING block.

```python
from __future__ import annotations

from typing import TYPE_CHECKING

import fsspec

from obstore.fsspec import register

if TYPE_CHECKING:
    from obstore.store import S3Config

register("s3")

config: S3Config = {"region": "us-west-2", "skip_signature": True}
fs = fsspec.filesystem("s3", config=config)
```

--------------------------------

### Configure FsspecStore Directly

Source: https://github.com/developmentseed/obstore/blob/main/docs/integrations/fsspec.md

Pass configuration parameters like region and skip_signature directly to the FsspecStore constructor for S3.

```python
from obstore.fsspec import FsspecStore

fs = FsspecStore("s3", region="us-west-2", skip_signature=True)
```

--------------------------------

### Copy object to local file

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Copy an object from one store to another by downloading it to a temporary local file and then uploading.

```python
from pathlib import Path

store1 = ... # store of your choice
store2 = ... # store of your choice

path1 = "data/file1"
path2 = "data/file2"

resp = store1.get(path1)

with open("temporary_file", "wb") as f:
    for chunk in resp:
        f.write(chunk)

# Upload the path
store2.put(path2, Path("temporary_file"))
```

--------------------------------

### Register Obstore for Fsspec Protocol Handling

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.5.md

Demonstrates how to register obstore as the default handler for protocols like 's3', 'gs', and 'az' within the fsspec ecosystem.

```python
import obstore.fsspec
obstore.fsspec.register()
```

--------------------------------

### obstore.open_reader

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/file.md

Opens a readable file from an object store synchronously.

```APIDOC
## obstore.open_reader

### Description
Opens a readable file from an object store synchronously.

### Method
Python function call

### Endpoint
N/A (Python function)

### Parameters
(Details for parameters are not provided in the source text)

### Request Example
```python
# Example usage (details not provided in source)
reader = obstore.open_reader(path)
```

### Response
#### Success Response
- Returns a file-like object (obstore.ReadableFile).

#### Response Example
```json
# Example response structure (details not provided in source)
{
  "content": "..."
}
```
```

--------------------------------

### Integrate Boto3 Session for S3 Credentials

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.5.md

Shows how to use the Boto3CredentialProvider to leverage a boto3 Session for handling S3 object storage credentials.

```python
from boto3 import Session
from obstore.auth.boto3 import Boto3CredentialProvider
from obstore.store import S3Store

session = Session(...)
credential_provider = Boto3CredentialProvider(session)
store = S3Store("bucket_name", credential_provider=credential_provider)
```

--------------------------------

### Construct Anonymous S3Store Client

Source: https://github.com/developmentseed/obstore/blob/main/docs/getting-started.md

Demonstrates multiple ways to create an anonymous S3Store client for public buckets. Ensure `skip_signature=True` is used for anonymous access.

```python
from obstore.store import S3Store, from_url

from_url("s3://bucket-name", region="us-east-1", skip_signature=True)
from_url("https://bucket-name.s3.us-east-1.amazonaws.com", skip_signature=True)
store = S3Store("bucket-name", region="us-east-1", skip_signature=True)
```

--------------------------------

### Create S3Store from URL for Cloudflare R2

Source: https://github.com/developmentseed/obstore/blob/main/docs/examples/r2.md

Use S3Store.from_url to connect to a Cloudflare R2 bucket. Ensure you have your API token's Access Key ID and Secret Access Key, and the bucket's S3 API URL.

```python
from obstore.store import S3Store

access_key_id = "..."
secret_access_key = "..."
store = S3Store.from_url(
    "https://f0b62eebfbdde1133378bfe3958325f6.r2.cloudflarestorage.com/ kylebarron-public",
    access_key_id=access_key_id,
    secret_access_key=secret_access_key,
)
store.list_with_delimiter()
```

--------------------------------

### Fix AzureStore Creation from HTTPS URL

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.7.md

Correctly creates an `AzureStore` from an HTTPS URL by properly identifying the container name. Previously, it might misinterpret the container name as a prefix.

```python
url = "https://overturemapswestus2.blob.core.windows.net/release"
store = AzureStore.from_url(url, skip_signature=True)

assert store.config.get("container_name") == "release"
assert store.config.get("account_name") == "overturemapswestus2"
assert store.prefix is None
```

--------------------------------

### Put local file

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Upload a local file to the store using `obstore.put`.

```python
from pathlib import Path

store = ... # store of your choice
path = "data/file1"
content = Path("path/to/local/file")
store.put(path, content)
```

--------------------------------

### List Objects as Arrow RecordBatches

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Pass `return_arrow=True` to `obstore.list` to receive results as Arrow `RecordBatch` objects, which can significantly reduce overhead when listing large buckets. This requires the `arro3-core` dependency.

```python
store = ... # store of your choice

# Get a stream of Arrow RecordBatches of metadata
list_stream = store.list(prefix="data", return_arrow=True)
for record_batch in list_stream:
    # Perform zero-copy conversion to your arrow-backed library of choice
    #
    # To pyarrow:
    # pyarrow.record_batch(record_batch)
    #
    # To polars:
    # polars.DataFrame(record_batch)
    #
    # To pandas (with Arrow-backed data-types):
    # pyarrow.record_batch(record_batch).to_pandas(types_mapper=pd.ArrowDtype)
    #
    # To arro3:
    # arro3.core.RecordBatch(record_batch)
    print(record_batch.num_rows)
```

--------------------------------

### list

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/list.md

Functional API for listing objects in Obstore.

```APIDOC
## list

### Description
Provides a functional interface for listing objects within Obstore. This is part of the Functional API design.

### Method
(Function signature or relevant details would be here if available)

### Parameters
(Parameters would be listed here if explicitly documented)

### Request Example
(Request example would be here if explicitly documented)

### Response
#### Success Response
(Success response details would be here if explicitly documented)

#### Response Example
(Response example would be here if explicitly documented)
```

--------------------------------

### List Objects Recursively

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Use the `obstore.list` method to recursively list all files under a specified prefix. This is useful for browsing directory structures in object storage.

```python
store = ... # store of your choice

# Recursively list all files below the 'data' path.
# 1. On AWS S3 this would be the 'data/' prefix
# 2. On a local filesystem, this would be the 'data' directory
prefix = "data"

# Get a stream of metadata objects:
list_stream = store.list(prefix)

# Print info
for batch in list_stream:
    for meta in batch:
        print(f'Name: {meta["path"]}, size: {meta["size"]}')
```

--------------------------------

### List items in AzureStore container

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.6.md

List items within the container of an AzureStore. The prefix is automatically set on the AzureStore if provided during initialization.

```python
items = next(store.list())
print(items[:2])
```

--------------------------------

### Put file-like object

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Upload content from a file-like object using `obstore.put`.

```python
store = ... # store of your choice
path = "data/file1"
with open("path/to/local/file", "rb") as content:
    store.put(path, content)
```

--------------------------------

### Azure Authentication with AzureCredentialProvider

Source: https://github.com/developmentseed/obstore/blob/main/docs/authentication.md

Use AzureCredentialProvider for synchronous Azure credential handling with AzureStore. Ensure you have the necessary credentials configured.

```python
from obstore.auth.azure import AzureCredentialProvider
from obstore.store import AzureStore

credential_provider = AzureAsyncCredentialProvider(credential=...)
store = AzureStore("container", credential_provider=credential_provider)
print(store.list().collect())
```

--------------------------------

### obstore.get_async

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/get.md

Asynchronously retrieves an object from the store using the Functional API.

```APIDOC
## obstore.get_async

### Description
Asynchronously retrieves a single object from the store using the functional API.

### Method
(Functional API)

### Endpoint
(Functional API)

### Parameters
(Not specified in source)

### Request Example
(Not specified in source)

### Response
(Not specified in source)
```

--------------------------------

### obstore.get

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/get.md

Retrieves an object from the store using the Functional API.

```APIDOC
## obstore.get

### Description
Retrieves a single object from the store using the functional API.

### Method
(Functional API)

### Endpoint
(Functional API)

### Parameters
(Not specified in source)

### Request Example
(Not specified in source)

### Response
(Not specified in source)
```

--------------------------------

### Pass Custom Credential Provider to S3Store

Source: https://github.com/developmentseed/obstore/blob/main/docs/blog/posts/obstore-0.5.md

Illustrates how to pass a custom credential provider function, such as 'get_credentials', directly to the S3Store constructor.

```python
S3Store(..., credential_provider=get_credentials)
```

--------------------------------

### Fetch Object Content

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Use the `store.get` method to retrieve an object's content along with its metadata. The content can be accessed as bytes or streamed.

```python
store = ... # store of your choice

# Retrieve a specific file
path = "data/file01.parquet"

# Fetch the object including metadata
result = store.get(path)
assert result.meta == meta

# Buffer the entire object in memory
buffer = result.bytes()
assert len(buffer) == meta.size

# Alternatively stream the bytes from object storage
stream = store.get(path).stream()

# We can now iterate over the stream
total_buffer_len = 0
for chunk in stream:
    total_buffer_len += len(chunk)

assert total_buffer_len == meta.size
```

--------------------------------

### ObjectStoreMethods.get_async

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/get.md

Asynchronously retrieves an object from the store using the Method API.

```APIDOC
## ObjectStoreMethods.get_async

### Description
Asynchronously retrieves a single object from the store.

### Method
(Method API)

### Endpoint
(Method API)

### Parameters
(Not specified in source)

### Request Example
(Not specified in source)

### Response
(Not specified in source)
```

--------------------------------

### Use PyObjectStore as a Function Parameter

Source: https://github.com/developmentseed/obstore/blob/main/pyo3-object_store/README.md

Accepts a PyObjectStore as a parameter in a function exported to Python. The store can then be converted into an Arc<dyn ObjectStore> for use in Rust.

```rust
#[pyfunction]
pub fn use_object_store(store: PyObjectStore) {
    let store: Arc<dyn ObjectStore> = store.into_dyn();
}
```

--------------------------------

### AzureStore

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/store/azure.md

Represents a store for interacting with Azure Blob Storage. It allows for uploading, downloading, and managing files in Azure.

```APIDOC
## AzureStore

### Description
Represents a store for interacting with Azure Blob Storage. It allows for uploading, downloading, and managing files in Azure.

### Class
obstore.store.AzureStore

### Options
- `inherited_members: true` - Includes inherited members in the documentation.
```

--------------------------------

### PlanetaryComputerCredentialProvider

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/auth/planetary-computer.md

Provides synchronous credential retrieval for the Planetary Computer.

```APIDOC
## PlanetaryComputerCredentialProvider

### Description
This class provides synchronous credential retrieval for the Planetary Computer.

### Usage
```python
from obstore.auth.planetary_computer import PlanetaryComputerCredentialProvider

provider = PlanetaryComputerCredentialProvider()
credentials = provider.get_credentials()
```

### Methods
- `get_credentials()`: Retrieves the Planetary Computer credentials.
```

--------------------------------

### ObjectStoreMethods.get

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/get.md

Retrieves an object from the store using the Method API.

```APIDOC
## ObjectStoreMethods.get

### Description
Retrieves a single object from the store.

### Method
(Method API)

### Endpoint
(Method API)

### Parameters
(Not specified in source)

### Request Example
(Not specified in source)

### Response
(Not specified in source)
```

--------------------------------

### PlanetaryComputerAsyncCredentialProvider

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/auth/planetary-computer.md

Provides asynchronous credential retrieval for the Planetary Computer.

```APIDOC
## PlanetaryComputerAsyncCredentialProvider

### Description
This class provides asynchronous credential retrieval for the Planetary Computer.

### Usage
```python
from obstore.auth.planetary_computer import PlanetaryComputerAsyncCredentialProvider

async def get_planetary_credentials():
    provider = PlanetaryComputerAsyncCredentialProvider()
    credentials = await provider.get_credentials()
    return credentials
```

### Methods
- `get_credentials()`: Asynchronously retrieves the Planetary Computer credentials.
```

--------------------------------

### List S3 bucket objects with correct region

Source: https://github.com/developmentseed/obstore/blob/main/docs/troubleshooting/aws.md

Successfully listing objects in an S3 bucket by providing the correct region parameter.

```python
from obstore.store import S3Store

store = S3Store("sentinel-cogs", skip_signature=True, region="us-west-2")
next(store.list())
```

--------------------------------

### put

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/put.md

Puts an object into the store using the Functional API.

```APIDOC
## obstore.put

### Description
Puts an object into the store using the Functional API.

### Method
(Method signature details not provided in source)

### Endpoint
(Endpoint details not provided in source)

### Parameters
(Parameter details not provided in source)

### Request Example
(Request example not provided in source)

### Response
#### Success Response
(Success response details not provided in source)

#### Response Example
(Response example not provided in source)
```

--------------------------------

### S3Config

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/store/aws.md

Configuration options for connecting to AWS S3.

```APIDOC
## Class S3Config

### Description
Defines the configuration parameters for an AWS S3 connection.

### Fields
(Specific fields are not detailed in the source, but typically include bucket name, region, etc.)
```

--------------------------------

### Fetch Object Metadata

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Use the `store.head` method to retrieve only the metadata for a specific object without downloading its content. This is efficient for checking file size or modification times.

```python
store = ... # store of your choice

# Retrieve a specific file
path = "data/file01.parquet"

# Fetch just the file metadata
meta = store.head(path)
print(meta)
```

--------------------------------

### obstore.head_async

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/head.md

Asynchronously retrieves the head of an object store using the Functional API.

```APIDOC
## obstore.head_async

### Description
Asynchronously retrieves the head of an object store using the Functional API.

### Method
[Method details not specified in source]

### Endpoint
[Endpoint details not specified in source]

### Parameters
[Parameters not specified in source]

### Request Example
[Request example not specified in source]

### Response
#### Success Response
[Success response details not specified in source]

#### Response Example
[Response example not specified in source]
```

--------------------------------

### ObjectStoreMethods.list_with_delimiter_async

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/list.md

Asynchronously lists objects with a specified delimiter. Part of the Method API.

```APIDOC
## ObjectStoreMethods.list_with_delimiter_async

### Description
Asynchronously lists objects within the ObjectStore, supporting a delimiter for hierarchical listing. This method is part of the Method API design.

### Method
(Method signature or relevant details would be here if available)

### Parameters
(Parameters would be listed here if explicitly documented)

### Request Example
(Request example would be here if explicitly documented)

### Response
#### Success Response
(Success response details would be here if explicitly documented)

#### Response Example
(Response example would be here if explicitly documented)
```

--------------------------------

### Async Azure Authentication with AzureAsyncCredentialProvider

Source: https://github.com/developmentseed/obstore/blob/main/docs/authentication.md

Utilize AzureAsyncCredentialProvider for asynchronous Azure credential management. This is suitable for applications using Obstore's async APIs and requires an active event loop.

```python
import asyncio
from obstore.auth.azure import AzureCredentialProvider
from obstore.store import AzureStore

credential_provider = AzureAsyncCredentialProvider(credential=...)
store = AzureStore("container", credential_provider=credential_provider)

async def fetch_blobs():
    blobs = await store.list().collect_async()
    print(blobs)

asyncio.run(fetch_blobs())
```

--------------------------------

### Put object with bytes

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Use `obstore.put` to atomically write data. It automatically uses multipart uploads for large input data.

```python
store = ... # store of your choice
path = "data/file1"
content = b"hello"
store.put(path, content)
```

--------------------------------

### Copy object in memory

Source: https://github.com/developmentseed/obstore/blob/main/docs/cookbook.md

Copy an object from one store to another by downloading its bytes into memory and then uploading.

```python
store1 = ... # store of your choice
store2 = ... # store of your choice

path1 = "data/file1"
path2 = "data/file2"

buffer = store1.get(path1).bytes()
store2.put(path2, buffer)
```

--------------------------------

### ObjectStoreMethods.list

Source: https://github.com/developmentseed/obstore/blob/main/docs/api/list.md

Lists objects within the ObjectStore. This is part of the Method API design.

```APIDOC
## ObjectStoreMethods.list

### Description
Lists objects within the ObjectStore. This method is part of the Method API design, providing an object-oriented approach to interacting with the store.

### Method
(Method signature or relevant details would be here if available)

### Parameters
(Parameters would be listed here if explicitly documented)

### Request Example
(Request example would be here if explicitly documented)

### Response
#### Success Response
(Success response details would be here if explicitly documented)

#### Response Example
(Response example would be here if explicitly documented)
```