### Set up Python virtual environment and install dependencies

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Creates a Python virtual environment and installs essential dependencies like pip, setuptools, and numpy. This is a prerequisite for installing TensorStore from source.

```shell
python3 -m venv ts-venv
source ts-venv/bin/activate
python3 -m pip install --upgrade pip setuptools numpy
```

--------------------------------

### Open Stack Driver with Layers

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/stack/index.rst

Example of opening a stack driver with two array layers. The second layer is transformed to start after the first. This demonstrates how to configure layers within the stack driver.

```python
ts.open({
    'driver':
        'stack',
    'layers': [
        {
            'driver': 'array',
            'dtype': 'int32',
            'array': [1, 2, 3, 4],
        },
        {
            'driver': 'array',
            'dtype': 'int32',
            'array': [1, 2, 3, 4],
            'transform': {
                'input_inclusive_min': [4],
                'input_exclusive_max': [8],
                'output': [{
                    'input_dimension': 0,
                    'offset': -4
                }]
            },
        },
    ]
}).result().schema.dtype
```

--------------------------------

### Install TensorStore Python API from Local Source

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Installs TensorStore from a local git repository checkout. This command clones the repository, navigates into the directory, and then installs the package using pip, which invokes Bazel for building the C++ extension module.

```shell
git clone https://github.com/google/tensorstore
cd tensorstore
python3 -m pip install .
```

--------------------------------

### Open and Read Array in C++

Source: https://context7.com/google/tensorstore/llms.txt

Demonstrates opening and reading a Zarr array using the TensorStore C++ API. This example requires the TensorStore C++ library to be installed and linked.

```cpp
#include "tensorstore/tensorstore.h"
#include "tensorstore/open.h"
#include "tensorstore/context.h"
#include <iostream>

int main() {
  // Create a context for shared resources
  auto context = tensorstore::Context::Default();

  // Open/create a Zarr array
  TENSORSTORE_ASSIGN_OR_RETURN(
      auto store,
      tensorstore::Open({
          {"driver", "zarr"},
          {"kvstore", {{"driver", "file"}, {"path", "/tmp/cpp_dataset/"}}},
          {"metadata", {
              {"dtype", "<f4"},
              {"shape", {100, 200, 300}},
              {"chunks", {50, 50, 50}}
          }}
      },
      context,
      tensorstore::OpenMode::create | tensorstore::OpenMode::delete_existing,
      tensorstore::ReadWriteMode::read_write).result());

  std::cout << "Rank: " << store.rank() << std::endl;
  std::cout << "Domain: " << store.domain() << std::endl;

  // Read a slice
  TENSORSTORE_ASSIGN_OR_RETURN(
      auto slice,
      store | tensorstore::Dims(0).HalfOpenInterval(0, 10));

  TENSORSTORE_ASSIGN_OR_RETURN(
      auto data,
      tensorstore::Read(slice).result());

  std::cout << "Read shape: " << data.shape() << std::endl;

  return 0;
}
```

--------------------------------

### Install TensorStore Python API from PyPI on Windows

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Alternative command for installing the TensorStore Python package on Windows systems. Use the -vv option to show progress.

```shell
py -3 -m pip install tensorstore -vv
```

--------------------------------

### Install TensorStore via pip

Source: https://github.com/google/tensorstore/blob/master/README.md

Use this command to install the TensorStore package from PyPI.

```bash
pip install tensorstore
```

--------------------------------

### Install TensorStore Python API from PyPI

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Installs the latest published version of the TensorStore Python package using pip. Use the -vv option to show progress. This is the recommended method for most users.

```shell
python3 -m pip install tensorstore -vv
```

--------------------------------

### Initialize TensorStore Array

Source: https://github.com/google/tensorstore/blob/master/docs/python/indexing.rst

Example of creating a basic TensorStore array representation.

```python
TensorStore({
  'array': [0, 2, 3],
  'context': {'data_copy_concurrency': {}},
  'driver': 'array',
  'dtype': 'int32',
  'transform': {'input_exclusive_max': [3], 'input_inclusive_min': [0]},
})
```

--------------------------------

### File Key-Value Store Driver Configuration

Source: https://github.com/google/tensorstore/blob/master/tensorstore/kvstore/file/index.rst

Configuration example for the 'file' driver, demonstrating how to disable write durability for potentially faster performance.

```APIDOC
## File Key-Value Store Driver Configuration

### Description
This configuration example shows how to use the `file` driver and disable write durability by setting `file_io_sync` to `false`. This can improve write performance when data durability is not critical.

### Method
N/A (Configuration Example)

### Endpoint
N/A (Configuration Example)

### Parameters
#### Request Body
- **driver** (string) - Required - Specifies the driver type, which is "file" for this driver.
- **path** (string) - Required - The root directory on the filesystem where keys will be stored as files.
- **file_io_sync** (boolean) - Optional - If set to `false`, write durability is disabled, potentially increasing write performance. Defaults to `true`.

### Request Example
```json
{
  "driver": "file",
  "path": "/local/path/",
  "file_io_sync": false
}
```

### Response
N/A (Configuration Example)
```

--------------------------------

### Install build dependencies on Debian

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Install the necessary packages to satisfy build requirements on Debian-based systems.

```shell
sudo apt-get install build-essential git nasm python3 python3-dev python3-pip python3-venv
```

--------------------------------

### Zarr v3 Metadata to TensorStore Schema Example

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr3/index.rst

This example demonstrates how Zarr v3 metadata maps to a TensorStore schema, specifically for an array without sharding.

```APIDOC
## Example: Zarr v3 Metadata to TensorStore Schema Mapping

### Description
This example illustrates the conversion of Zarr v3 metadata to a TensorStore schema, focusing on an array configuration without sharding.

### Request Example (Zarr v3 Metadata)
```json
{
  "zarr_format": 3,
  "node_type": "array",
  "shape": [1000, 2000, 3000],
  "chunk_grid": {"name": "regular", "configuration": {"chunk_shape": [100, 200, 300]}},
  "chunk_key_encoding": {"name": "default"},
  "data_type": "uint16",
  "codecs": [{"name": "bytes", "configuration": {"endian": "little"}}],
  "fill_value": 42
}
```

### Response Example (TensorStore Schema)
```json
{
  "chunk_layout": {
    "grid_origin": [0, 0, 0],
    "inner_order": [0, 1, 2],
    "read_chunk": {"shape": [100, 200, 300]},
    "write_chunk": {"shape": [100, 200, 300]}
  },
  "codec": {
    "codecs": [{"configuration": {"endian": "little"}, "name": "bytes"}],
    "driver": "zarr3"
  },
  "domain": {"exclusive_max": [[1000], [2000], [3000]], "inclusive_min": [0, 0, 0]},
  "dtype": "uint16",
  "fill_value": 42,
  "rank": 3
}
```
```

--------------------------------

### Create Arrays with Different Formats

Source: https://context7.com/google/tensorstore/llms.txt

Examples of creating N5, Zarr v3, and Zarr v2 datasets with specific metadata and compression configurations.

```python
import tensorstore as ts
import numpy as np

# Create an N5 dataset with gzip compression
n5_store = ts.open({
    'driver': 'n5',
    'kvstore': {
        'driver': 'file',
        'path': 'tmp/n5_dataset/',
    },
    'metadata': {
        'compression': {'type': 'gzip'},
        'dataType': 'uint32',
        'dimensions': [1000, 20000],
        'blockSize': [100, 100],
    },
    'create': True,
    'delete_existing': True,
}).result()

# Create a Zarr v3 dataset with blosc compression
zarr3_store = ts.open({
    'driver': 'zarr3',
    'kvstore': {'driver': 'memory'},
},
    create=True,
    dtype=ts.uint16,
    shape=[1000, 2000, 3000],
    chunk_layout=ts.ChunkLayout(
        read_chunk_shape=[64, 64, 64],
        write_chunk_shape=[512, 512, 512],
    ),
).result()

# Create Zarr v2 with explicit metadata
zarr2_store = ts.open({
    'driver': 'zarr',
    'kvstore': {'driver': 'memory'},
    'metadata': {
        'compressor': {
            'id': 'blosc',
            'cname': 'lz4',
            'clevel': 5,
            'shuffle': 1
        },
        'dtype': '<f4',
        'order': 'C',
    }
},
    create=True,
    shape=[500, 500, 500],
).result()
```

--------------------------------

### Open GCS Storage

Source: https://context7.com/google/tensorstore/llms.txt

Examples for reading from a public GCS bucket and creating a new array in GCS.

```python
public_store = ts.open({
    'driver': 'neuroglancer_precomputed',
    'kvstore': 'gs://neuroglancer-janelia-flyem-hemibrain/v1.1/segmentation/',
    'context': {
        'cache_pool': {'total_bytes_limit': 100_000_000}  # 100MB cache
    },
    'recheck_cached_data': 'open',
}).result()

print(f"Domain: {public_store.domain}")

# Create array in GCS (requires authentication)
# Set GOOGLE_APPLICATION_CREDENTIALS or use gcloud auth
gcs_store = ts.open({
    'driver': 'zarr3',
    'kvstore': {
        'driver': 'gcs',
        'bucket': 'my-bucket-name',
        'path': 'path/to/dataset/'
    }
},
    create=True,
    dtype=ts.float32,
    shape=[1000, 1000, 1000]
).result()

# Read a slice from remote storage
slice_data = gcs_store[100:200, 100:200, 0].read().result()
```

--------------------------------

### Get TensorStore Schema from Zarr v3 Metadata

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr3/index.rst

This example demonstrates how to obtain the TensorStore schema corresponding to a given Zarr v3 metadata object. It involves opening a JSON representation of the metadata and then opening a Zarr v3 array with that metadata to retrieve its schema.

```python
>>> def get_schema(metadata):
...     context = ts.Context()
...     kvstore = {'driver': 'memory'}
...     ts.open(
...         {
...             'driver': 'json',
...             'kvstore': kvstore,
...             'path': 'zarr.json'
...         },
...         context=context).result().write(metadata).result()
...     return ts.open({
...         'driver': 'zarr3',
...         'kvstore': kvstore,
...     },
...                    context=context).result().schema
>>> metadata = json.loads(OUTPUT)  # doctest:+JSON_OUTPUT
... metadata
{
  "zarr_format": 3,
  "node_type": "array",
  "shape": [1000, 2000, 3000],
  "chunk_grid": {"name": "regular", "configuration": {"chunk_shape": [100, 200, 300]}},
  "chunk_key_encoding": {"name": "default"},
  "data_type": "uint16",
  "codecs": [{"name": "bytes", "configuration": {"endian": "little"}}],
  "fill_value": 42
}
```

```python
>>> get_schema(metadata).to_json()  # doctest:+JSON_OUTPUT
{
  "chunk_layout": {
    "grid_origin": [0, 0, 0],
    "inner_order": [0, 1, 2],
    "read_chunk": {"shape": [100, 200, 300]},
    "write_chunk": {"shape": [100, 200, 300]}
  },
  "codec": {
    "codecs": [{"configuration": {"endian": "little"}, "name": "bytes"}],
    "driver": "zarr3"
  },
  "domain": {"exclusive_max": [[1000], [2000], [3000]], "inclusive_min": [0, 0, 0]},
  "dtype": "uint16",
  "fill_value": 42,
  "rank": 3
}
```

--------------------------------

### Zarr3 Metadata and IndexDomain Mapping

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr3/index.rst

Examples showing the conversion of Zarr3 metadata into a TensorStore IndexDomain.

```json
{
  "zarr_format": 3,
  "node_type": "array",
  "shape": [1000, 2000, 3000],
  "dimension_names": ["x", "y", "z"],
  "chunk_grid": {"name": "regular", "configuration": {"chunk_shape": [100, 200, 300]}},
  "chunk_key_encoding": {"name": "default"},
  "data_type": "uint16",
  "codecs": [{"name": "bytes", "configuration": {"endian": "little"}}],
  "fill_value": 0
}
```

```json
{
  "exclusive_max": [[1000], [2000], [3000]],
  "inclusive_min": [0, 0, 0],
  "labels": ["x", "y", "z"]
}
```

--------------------------------

### Open dataset with auto-detection

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/auto/index.rst

Example of attempting to open a dataset where auto-detection fails due to an unrecognized format.

```python
>>> # Open with auto-detection
>>> await ts.open("file:///tmp/dataset2.ocdbt")
Traceback (most recent call last):
    ...
ValueError: FAILED_PRECONDITION: Error opening "auto" driver: Failed to detect format for "" in OCDBT database at local file "/tmp/dataset2.ocdbt/"...
```

--------------------------------

### Chunk Layout with Separate Read and Write Constraints

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr3/index.rst

Example of configuring distinct read and write chunk constraints using ChunkLayout.

```python
>>> ts.open(
...     {
...         'driver': 'zarr3',
...         'kvstore': {
...             'driver': 'memory'
...         }
...     },
...     create=True,
...     dtype=ts.uint16,
...     chunk_layout=ts.ChunkLayout(
...         chunk_aspect_ratio=[2, 1, 1],
...         read_chunk_elements=2000000,
...         write_chunk_elements=1000000000,
...     ),
...     shape=[1000, 2000, 3000],
... ).result().chunk_layout
ChunkLayout({
  'grid_origin': [0, 0, 0],
  'inner_order': [0, 1, 2],
  'read_chunk': {'shape': [200, 100, 100]},
  'write_chunk': {'shape': [1000, 1000, 1000]},
})
```

--------------------------------

### Zip Key-Value Store JSON Specification

Source: https://github.com/google/tensorstore/blob/master/tensorstore/kvstore/zip/index.rst

Example JSON specification for configuring the zip driver. This requires specifying the driver as 'zip' and providing the path to the ZIP file in the 'kvstore' field.

```json
{
  "driver": "zip",
  "kvstore": "gs://my-bucket/path/to/file.zip"
}
```

--------------------------------

### Use System Dependencies with CMake

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Configures TensorStore's CMake build to use system-provided dependencies instead of vendored ones. This example shows how to enable system CURL and ZLIB, emphasizing the need for consistency to avoid symbol collisions.

```cmake
-DTENSORSTORE_USE_SYSTEM_CURL=ON -DTENSORSTORE_USE_SYSTEM_ZLIB=ON
```

--------------------------------

### Interval Slicing Basics

Source: https://github.com/google/tensorstore/blob/master/docs/python/indexing.rst

Shows basic interval slicing on a TensorStore array where start is inclusive and stop is exclusive.

```python
>>> a = ts.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=ts.int32)
>>> a[1:5]
TensorStore({
  'array': [1, 2, 3, 4],
  'context': {'data_copy_concurrency': {}},
  'driver': 'array',
  'dtype': 'int32',
  'transform': {
    'input_exclusive_max': [5],
    'input_inclusive_min': [1],
    'output': [{'input_dimension': 0, 'offset': -1}],
  },
})
```

--------------------------------

### Chunk Layout with Explicit Shapes

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr3/index.rst

Example of initializing a Zarr3 array with explicit read chunk shape constraints.

```python
>>> ts.open(
...     {
...         'driver': 'zarr3',
...         'kvstore': {
...             'driver': 'memory'
...         }
...     },
...     create=True,
...     dtype=ts.uint16,
...     chunk_layout=ts.ChunkLayout(
...         read_chunk_shape=[64, 64, 64],

```

--------------------------------

### Unconstrained Chunk Layout Creation

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr3/index.rst

Example of opening a new Zarr3 array in memory without explicit chunk layout constraints.

```python
>>> ts.open(
...     {
...         'driver': 'zarr3',
...         'kvstore': {
...             'driver': 'memory'
...         }
...     },
...     create=True,
...     dtype=ts.uint16,
...     shape=[1000, 2000, 3000],
... ).result().chunk_layout
ChunkLayout({
  'grid_origin': [0, 0, 0],
  'inner_order': [0, 1, 2],
  'read_chunk': {'shape': [101, 101, 101]},
  'write_chunk': {'shape': [101, 101, 101]},
})
```

--------------------------------

### Neuroglancer Precomputed Chunk Layout Example

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/neuroglancer_precomputed/index.rst

Illustrates the configuration of a ChunkLayout for the Neuroglancer Precomputed driver, specifying grid origin, inner order, and read/write chunk shapes.

```python
ChunkLayout({
    'grid_origin': [20, 30, 40, 0],
    'inner_order': [3, 2, 1, 0],
    'read_chunk': {'shape': [64, 64, 64, 2]},
    'write_chunk': {'shape': [512, 512, 512, 2]},
  })
```

--------------------------------

### Build TensorStore Extension Module

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Builds the TensorStore extension module and specifies a directory for pre-built extensions. This is useful for installing TensorStore without compiling from source.

```shell
python3 setup.py build_ext -b /tmp/prebuilt
TENSORSTORE_PREBUILT_DIR=/tmp/prebuilt pip wheel .
```

--------------------------------

### Unconstrained Chunk Layout Example

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/n5/index.rst

Demonstrates creating an N5 array with an unconstrained chunk layout. TensorStore automatically determines a suitable chunk shape based on the default of 1 million elements per chunk.

```python
ts.open({
        'driver': 'n5',
        'kvstore': {
            'driver': 'memory'
        }
    },
        create=True,
        dtype=ts.uint16,
        shape=[1000, 2000, 3000]).result().chunk_layout
```

--------------------------------

### Configure GCS HTTP URL for Testing

Source: https://github.com/google/tensorstore/blob/master/tensorstore/kvstore/gcs/index.rst

Set this environment variable to point to a fake GCS server for testing purposes. Example uses http://localhost:4443.

```shell
TENSORSTORE_GCS_HTTP_URL=http://localhost:4443
```

--------------------------------

### Chunk Aspect Ratio and Elements Constraint Example

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/n5/index.rst

Demonstrates creating an N5 array with both chunk aspect ratio and element count constraints. TensorStore determines a chunk shape that satisfies both conditions.

```python
ts.open({
        'driver': 'n5',
        'kvstore': {
            'driver': 'memory'
        }
    },
        create=True,
        dtype=ts.uint16,
        shape=[1000, 2000, 3000],
        chunk_layout=ts.ChunkLayout(
            chunk_aspect_ratio=[1, 2, 2],
            chunk_elements=2000000)).result().chunk_layout
```

--------------------------------

### Configure Bazel for TensorStore on Windows

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Sets a Bazel startup option to mitigate MAX_PATH limitations on Windows when building TensorStore. This helps avoid 'Cannot open include file' errors.

```shell
TENSORSTORE_BAZEL_STARTUP_OPTIONS="--output_base=C:\\Out"
```

--------------------------------

### Build documentation with Bazelisk

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Use this command to generate the TensorStore documentation.

```shell
python3 bazelisk.py run //tools/docs:build_docs -- --output /tmp/tensorstore-docs
```

--------------------------------

### Get Zarr v2 Schema for Structured Data Type (Field 'y')

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr/index.rst

This example demonstrates retrieving the TensorStore schema for the 'y' field of a Zarr v2 array with a structured data type. It uses the provided Zarr metadata and specifies the field name.

```json
{
  "zarr_format": 2,
  "shape": [1000, 2000, 3000],
  "chunks": [100, 200, 300],
  "dtype": [["x", "<u2", [2, 3]], ["y", "<f4", [5]]],
  "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1},
  "fill_value": "AQACAAMABAAFAAYAAAAgQQAAMEEAAEBBAABQQQAAYEE=",
  "order": "F",
  "filters": null
}
```

```json
{
  "chunk_layout": {
    "grid_origin": [0, 0, 0, 0],
    "inner_order": [2, 1, 0, 3],
    "read_chunk": {"shape": [100, 200, 300, 5]},
    "write_chunk": {"shape": [100, 200, 300, 5]}
  },
  "codec": {
    "compressor": {"blocksize": 0, "clevel": 5, "cname": "lz4", "id": "blosc", "shuffle": 1},
    "driver": "zarr",
    "filters": null
  },
  "domain": {
    "exclusive_max": [[1000], [2000], [3000], 5],
    "inclusive_min": [0, 0, 0, 0]
  },
  "dtype": "float32",
  "fill_value": 0.0,
  "rank": 4
}
```

--------------------------------

### Get Zarr v2 Schema for Structured Data Type (Field 'x')

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr/index.rst

This example shows how to retrieve the TensorStore schema for a specific field ('x') within a Zarr v2 array that uses a structured data type. It requires the Zarr metadata and the field name as input.

```json
{
  "zarr_format": 2,
  "shape": [1000, 2000, 3000],
  "chunks": [100, 200, 300],
  "dtype": [["x", "<u2", [2, 3]], ["y", "<f4", [5]]],
  "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1},
  "fill_value": "AQACAAMABAAFAAYAAAAgQQAAMEEAAEBBAABQQQAAYEE=",
  "order": "F",
  "filters": null
}
```

```json
{
  "chunk_layout": {
    "grid_origin": [0, 0, 0, 0, 0],
    "inner_order": [2, 1, 0, 3, 4],
    "read_chunk": {"shape": [100, 200, 300, 2, 3]},
    "write_chunk": {"shape": [100, 200, 300, 2, 3]}
  },
  "codec": {
    "compressor": {"blocksize": 0, "clevel": 5, "cname": "lz4", "id": "blosc", "shuffle": 1},
    "driver": "zarr",
    "filters": null
  },
  "domain": {
    "exclusive_max": [[1000], [2000], [3000], 2, 3],
    "inclusive_min": [0, 0, 0, 0, 0]
  },
  "dtype": "uint16",
  "fill_value": [[1, 2, 3], [4, 5, 6]],
  "rank": 5
}
```

--------------------------------

### Get Zarr v2 Schema for Scalar Data Type

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/zarr/index.rst

This example demonstrates how to obtain the TensorStore schema for a Zarr v2 array with a scalar data type. It involves opening a JSON schema, writing Zarr metadata, and then opening the Zarr v2 driver to retrieve the schema.

```python
def get_schema(metadata, field=''):
    context = ts.Context()
    kvstore = {'driver': 'memory'}
    ts.open(
        {
            'driver': 'json',
            'kvstore': kvstore,
            'path': '.zarray'
        },
        context=context).result().write(metadata).result()
    return ts.open(
        {
            'driver': 'zarr2',
            'kvstore': kvstore,
            'field': field,
        },
        context=context).result().schema
```

```json
{
  "zarr_format": 2,
  "shape": [1000, 2000, 3000],
  "chunks": [100, 200, 300],
  "dtype": "<u2",
  "compressor": null,
  "fill_value": 42,
  "order": "C",
  "filters": null
}
```

```json
{
  "chunk_layout": {
    "grid_origin": [0, 0, 0],
    "inner_order": [0, 1, 2],
    "read_chunk": {"shape": [100, 200, 300]},
    "write_chunk": {"shape": [100, 200, 300]}
  },
  "codec": {"compressor": null, "driver": "zarr", "filters": null},
  "domain": {"exclusive_max": [[1000], [2000], [3000]], "inclusive_min": [0, 0, 0]},
  "dtype": "uint16",
  "fill_value": 42,
  "rank": 3
}
```

--------------------------------

### Open or Create TensorStore Arrays

Source: https://context7.com/google/tensorstore/llms.txt

Demonstrates initializing a new Zarr array in memory and opening an existing array from the local filesystem.

```python
import tensorstore as ts
import numpy as np

# Create a new Zarr array in memory
store = ts.open({
    'driver': 'zarr',
    'kvstore': {
        'driver': 'memory'
    }
},
    create=True,
    dtype=ts.float32,
    shape=[1000, 2000, 3000],
    chunk_layout=ts.ChunkLayout(chunk_shape=[100, 100, 100])
).result()

# Access array properties
print(f"Shape: {store.shape}")      # (1000, 2000, 3000)
print(f"Dtype: {store.dtype}")      # float32
print(f"Domain: {store.domain}")    # { [0, 1000), [0, 2000), [0, 3000) }

# Open an existing array from local filesystem
existing = ts.open({
    'driver': 'zarr',
    'kvstore': {
        'driver': 'file',
        'path': '/path/to/dataset/'
    }
}).result()
```

--------------------------------

### Initialize Google Cloud Storage Backend

Source: https://context7.com/google/tensorstore/llms.txt

Initializes a connection to Google Cloud Storage using the gcs driver.

```python
import tensorstore as ts
import numpy as np
```

--------------------------------

### Create and Write to N5 Dataset

Source: https://github.com/google/tensorstore/blob/master/docs/python/tutorial.rst

Initializes a new N5 dataset on the local filesystem and performs asynchronous and synchronous write operations.

```python
>>> import tensorstore as ts
>>> import numpy as np
>>> dataset = ts.open({
...     'driver': 'n5',
...     'kvstore': {
...         'driver': 'file',
...         'path': 'tmp/dataset/',
...     },
...     'metadata': {
...         'compression': {
...             'type': 'gzip'
...         },
...         'dataType': 'uint32',
...         'dimensions': [1000, 20000],
...         'blockSize': [100, 100],
...     },
...     'create': True,
...     'delete_existing': True,
... }).result()
```

```python
>>> write_future = dataset[80:82, 99:102].write([[1, 2, 3], [4, 5, 6]])
```

```python
>>> write_future.result()
```

```python
>>> await write_future
```

```python
>>> dataset[80:82, 99:102] = [[1, 2, 3], [4, 5, 6]]
```

```python
>>> dataset[80:83, 99:102].read().result()
array([[1, 2, 3],
       [4, 5, 6],
       [0, 0, 0]], dtype=uint32)
```

--------------------------------

### Run tests with system-provided libraries

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Configure the environment to use system-provided libraries before running tests.

```shell
export TENSORSTORE_SYSTEM_LIBS=curl,jpeg,boringssl
python3 bazelisk.py test //...
```

--------------------------------

### Chaining adapters with auto-detection

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/auto/index.rst

Illustrates how to combine the auto driver with other adapters like cast.

```python
>>> ts.Spec("file:///tmp/dataset.zarr|cast:int64")
Spec({
  'base': {
    'driver': 'auto',
    'kvstore': {
      'driver': 'file',
      'path': '/tmp/dataset.zarr',
    },
  },
  'driver': 'cast',
  'dtype': 'int64',
})
>>> ts.Spec("file:///tmp/dataset.zarr|auto|cast:int64")
Spec({
  'base': {
    'driver': 'auto',
    'kvstore': {
      'driver': 'file',
      'path': '/tmp/dataset.zarr',
    },
  },
  'driver': 'cast',
  'dtype': 'int64',
})
>>> await ts.open("file:///tmp/dataset.zarr|cast:int64")
TensorStore({
  'base': {
    'driver': 'zarr3',
```

--------------------------------

### Retrieve Schema as JSON

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/neuroglancer_precomputed/index.rst

Example of converting a metadata object to its corresponding TensorStore schema representation in JSON format.

```python
>>> get_schema(metadata).to_json()  # doctest:+JSON_OUTPUT
      {
        "chunk_layout": {
          "grid_origin": [20, 30, 40, 0],
          "inner_order": [3, 2, 1, 0],
          "read_chunk": {"shape": [64, 64, 64, 2]},
          "write_chunk": {"shape": [2048, 2048, 2048, 2]}
        },
        "codec": {
          "driver": "neuroglancer_precomputed",
          "encoding": "raw",
          "shard_data_encoding": "gzip"
        },
        "dimension_units": [[8.0, "nm"], [8.0, "nm"], [8.0, "nm"], null],
        "domain": {
          "exclusive_max": [34452, 39582, 51548, 2],
          "inclusive_min": [20, 30, 40, 0],
          "labels": ["x", "y", "z", "channel"]
        },
        "dtype": "uint8",
        "rank": 4
      }
```

--------------------------------

### Run tests with Bazelisk

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Execute the test suite for the project.

```shell
python3 bazelisk.py test //...
```

--------------------------------

### Benchmark kvstore IO operations

Source: https://github.com/google/tensorstore/blob/master/tensorstore/internal/benchmark/README.md

Run benchmarks for basic IO operations using a target size or a specific duration.

```bash
bazel run -c opt \
  //tensorstore/internal/benchmark:kvstore_benchmark -- \
  --kvstore_spec='"memory://abc/"' \
  --chunk_size=4194304 \
  --total_bytes=4294967296 \
  --read_blowup=500 \
  --repeat_writes=10 \
  --repeat_reads=100
```

```bash
bazel run -c opt \
  //tensorstore/internal/benchmark:kvstore_duration -- \
  --kvstore_spec='"file:///tmp/kvstore"' --duration=1m
```

--------------------------------

### Specify Bazel Cache Directory

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Sets the TENSORSTORE_BAZEL_STARTUP_OPTIONS environment variable to specify a non-standard cache directory for Bazel. This is useful for managing Bazel's output and cache locations.

```shell
TENSORSTORE_BAZEL_STARTUP_OPTIONS="--output_user_root /path/to/bazel_cache"
```

--------------------------------

### Configure Cache Pools in Python

Source: https://context7.com/google/tensorstore/llms.txt

Demonstrates configuring cache pools and concurrency limits for TensorStore contexts. Multiple stores can share a cache pool.

```python
import tensorstore as ts
import numpy as np

# Create context with shared cache
context = ts.Context({
    'cache_pool': {
        'total_bytes_limit': 1_000_000_000  # 1GB cache
    },
    'data_copy_concurrency': {'limit': 8},
    'file_io_concurrency': {'limit': 4}
})

# Multiple stores sharing the cache
store1 = ts.open({
    'driver': 'zarr',
    'kvstore': {'driver': 'file', 'path': '/data/dataset1/'}
},
    context=context,
    recheck_cached_data': False  # Trust cache
).result()

store2 = ts.open({
    'driver': 'n5',
    'kvstore': {'driver': 'file', 'path': '/data/dataset2/'}
},
    context=context,
    recheck_cached_data': 'open'  # Revalidate on open only
).result()

# Remote storage with caching
gcs_cached = ts.open({
    'driver': 'zarr3',
    'kvstore': 'gs://bucket/dataset/',
    'context': {
        'cache_pool': {'total_bytes_limit': 500_000_000},
        'gcs_request_concurrency': {'limit': 32}
    },
    'recheck_cached_data': False
}).result()
```

--------------------------------

### Create and Open Zarr3 in OCDBT

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/auto/index.rst

Examples of creating a new Zarr3 array within an OCDBT database and opening it using auto-detection.

```python
>>> # Create new array within new OCDBT database
>>> await ts.open("file:///tmp/dataset.ocdbt|ocdbt|zarr3",
...               dtype="int32",
...               shape=[5],
...               create=True)
```

```python
>>> # Open with auto-detection
>>> await ts.open("file:///tmp/dataset.ocdbt")
```

--------------------------------

### Define Metadata and Domain

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/neuroglancer_precomputed/index.rst

Demonstrates creating metadata for a multiscale volume and extracting the resulting IndexDomain.

```python
>>> metadata = get_metadata({
      ...     'multiscale_metadata': {
      ...         'num_channels': 2,
      ...         'data_type': 'uint8'
      ...     },
      ...     'scale_metadata': {
      ...         'resolution': [8, 8, 8],
      ...         'chunk_size': [64, 64, 64],
      ...         'size': [1000, 2000, 3000],
      ...         'voxel_offset': [20, 30, 40]
      ...     }
      ... })
      >>> metadata  # doctest:+JSON_OUTPUT
      {
        "@type": "neuroglancer_multiscale_volume",
        "data_type": "uint8",
        "num_channels": 2,
        "scales": [ {
            "chunk_sizes": [[64, 64, 64]],
            "encoding": "raw",
            "key": "8_8_8",
            "resolution": [8.0, 8.0, 8.0],
            "size": [1000, 2000, 3000],
            "voxel_offset": [20, 30, 40]
          }],
        "type": "image"
      }
```

```python
>>> get_schema(metadata).domain.to_json()  # doctest:+JSON_OUTPUT
      {
        "exclusive_max": [1020, 2030, 3040, 2],
        "inclusive_min": [20, 30, 40, 0],
        "labels": ["x", "y", "z", "channel"]
      }
```

--------------------------------

### Select dimensions using ts.d

Source: https://github.com/google/tensorstore/blob/master/docs/python/indexing.rst

Examples of creating DimSelection objects using various selection criteria like indices, labels, and slices.

```python
>>> ts.d[0, 1, 2]
d[0,1,2]
>>> ts.d[0:1, 2, "x"]
d[0:1,2,'x']
>>> ts.d[[0, 1], [2]]
d[0,1,2]
>>> ts.d[[0, 1], ts.d[2, 3]]
d[0,1,2,3]
```

--------------------------------

### Benchmark multi-tensorstore operations

Source: https://github.com/google/tensorstore/blob/master/tensorstore/internal/benchmark/README.md

Generate configuration specs and run concurrent read or write benchmarks for multiple tensorstores.

```bash
bazel run -c opt \
  //tensorstore/internal/benchmark/multi_genspec  \
  --base_spec='{
    "driver": "zarr3",
    "kvstore": { "driver": "ocdbt", "base": "file:///tmp/checkpoint" }
  }'  \
  --config='[
    { "name": "array1", "dtype":"float32", "shape": [8192], "chunks": [2048] },
    { "name": "array2", "dtype":"float32", "shape": [8192,64,28672], "chunks": [512,32,7168] },
    { "name": "array3", "dtype":"float32", "shape": [32000,8192], "chunks":[8000,512] },
    { "name": "array4", "dtype":"float32", "shape": [28672,64,8192], "chunks":[7168,32,512] }
  ]'  > /tmp/config.json
```

```bash
bazel run -c opt \
  //tensorstore/internal/benchmark/multi_write_benchmark  \
  --context_spec='{"file_io_concurrency": { "limit": 128 }}'  \
  --base_spec='{
    "driver": "zarr3",
    "kvstore": { "driver": "ocdbt", "base": "file:///tmp/checkpoint" }
  }'  \
  --repeat_writes=25 \
  --write_config=/tmp/config.json
```

```bash
bazel run -c opt \
  //tensorstore/internal/benchmark/multi_read_benchmark  \
  --context_spec='{"file_io_concurrency": { "limit": 128 }}'  \
  --base_spec='{
    "driver": "zarr3",
    "kvstore": { "driver": "ocdbt", "base": "file:///tmp/checkpoint" }
  }'  \
  --repeat_reads=25 \
  --read_config=/tmp/config.json
```

--------------------------------

### Create Zarr3 at Path within OCDBT

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/auto/index.rst

Example of creating a Zarr3 array at a specific path within an OCDBT database, where auto-detection is not applicable.

```python
>>> # Create new array within new OCDBT database
>>> await ts.open(
...     "file:///tmp/dataset2.ocdbt|ocdbt:path/within/database|zarr3",
...     dtype="int32",
...     shape=[5],
...     create=True)
```

--------------------------------

### Auto-detecting a Zarr3 array

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/auto/index.rst

Demonstrates creating a Zarr3 array and subsequently opening it using automatic format detection.

```python
>>> # Create new array
>>> await ts.open("file:///tmp/dataset.zarr|zarr3",
...               dtype="int32",
...               shape=[5],
...               create=True)
TensorStore({
  'context': {
    'cache_pool': {},
    'data_copy_concurrency': {},
    'file_io_concurrency': {},
    'file_io_locking': {},
    'file_io_mode': {},
    'file_io_sync': True,
  },
  'driver': 'zarr3',
  'dtype': 'int32',
  'kvstore': {
    'driver': 'file',
    'path': '/tmp/dataset.zarr/',
  },
  'metadata': {
    'chunk_grid': {'configuration': {'chunk_shape': [5]}, 'name': 'regular'},
    'chunk_key_encoding': {'name': 'default'},
    'codecs': [{'configuration': {'endian': 'little'}, 'name': 'bytes'}],
    'data_type': 'int32',
    'fill_value': 0,
    'node_type': 'array',
    'shape': [5],
    'zarr_format': 3,
  },
  'transform': {'input_exclusive_max': [[5]], 'input_inclusive_min': [0]},
})
>>> # Open with auto-detection
>>> await ts.open("file:///tmp/dataset.zarr")
TensorStore({
  'context': {
    'cache_pool': {},
    'data_copy_concurrency': {},
    'file_io_concurrency': {},
    'file_io_locking': {},
    'file_io_mode': {},
    'file_io_sync': True,
  },
  'driver': 'zarr3',
  'dtype': 'int32',
  'kvstore': {
    'driver': 'file',
    'path': '/tmp/dataset.zarr/',
  },
  'metadata': {
    'chunk_grid': {'configuration': {'chunk_shape': [5]}, 'name': 'regular'},
    'chunk_key_encoding': {'name': 'default'},
    'codecs': [{'configuration': {'endian': 'little'}, 'name': 'bytes'}],
    'data_type': 'int32',
    'fill_value': 0,
    'node_type': 'array',
    'shape': [5],
    'zarr_format': 3,
  },
  'transform': {'input_exclusive_max': [[5]], 'input_inclusive_min': [0]},
})
```

--------------------------------

### Integer array indexing in TensorStore

Source: https://github.com/google/tensorstore/blob/master/docs/python/indexing.rst

Selects coordinates of a dimension using an array-like index of integer values. This example selects elements at specified indices from a 1D array.

```python
a = ts.array([5, 4, 3, 2], dtype=ts.int32)
a[[0, 3, 3]]
```

--------------------------------

### Slice a TensorStore array with mixed slice and scalar

Source: https://github.com/google/tensorstore/blob/master/docs/python/indexing.rst

A sequence may be combined with a scalar value for indexing. This example shows a slice combined with a tuple representing bounds.

```python
a = ts.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]],
             dtype=ts.int32)
a[1:(3, 4)]
```

--------------------------------

### Run TensorStore IPython Shell

Source: https://github.com/google/tensorstore/blob/master/docs/installation.rst

Launches the TensorStore IPython shell using Bazel. This command is used to interact with TensorStore in an interactive Python environment.

```shell
python3 bazelisk.py run -c opt //python/tensorstore:shell
```

--------------------------------

### Authenticate with Google Cloud SDK

Source: https://github.com/google/tensorstore/blob/master/tensorstore/kvstore/gcs/index.rst

Use this command to log in to Google Cloud SDK and set up application default credentials for accessing non-public GCS buckets.

```shell
gcloud auth application-default login
```

--------------------------------

### Open neuroglancer_uint64_sharded with murmurhash3_x86_128 hash and 1GB cache

Source: https://github.com/google/tensorstore/blob/master/tensorstore/kvstore/neuroglancer_uint64_sharded/index.rst

Example JSON specification for opening a neuroglancer_uint64_sharded key-value store. Uses the murmurhash3_x86_128 hash and configures a 1GB cache.

```json
{
  "driver": "neuroglancer_uint64_sharded",
  "kvstore": "gs://my-bucket/path/to/sharded/data/",
  "metadata": {
    "@type": "neuroglancer_uint64_sharded_v1",
    "hash": "murmurhash3_x86_128",
    "preshift_bits": 0,
    "minishard_bits": 3,
    "shard_bits": 3,
    "data_encoding": "raw",
    "minishard_index_encoding": "gzip"
  },
  "context": {
    "cache_pool": {"total_bytes_limit": 1000000000}
  }
}
```

--------------------------------

### webp Driver Overview

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/image/webp/index.rst

Details regarding the webp driver configuration and indexing structure.

```APIDOC
## webp Driver

### Description
The webp driver specifies a TensorStore backed by a webp image file. It supports either 3 (RGB) or 4 (RGBA) channel WebP images. Lossy WebP images are encoded as YUV 422.

### Indexing
The read volume is indexed by the following dimensions:
- **height** (y)
- **width** (x)
- **channel**

### Auto Detection
This driver supports auto-detection based on the signature at the start of the file.
```

--------------------------------

### Open Existing Multiscale Volume Scale (Default)

Source: https://github.com/google/tensorstore/blob/master/tensorstore/driver/neuroglancer_precomputed/index.rst

Opens the first or only scale of an existing multiscale volume. Specify the driver and key-value store path.

```json
{
  "driver": "neuroglancer_precomputed",
  "kvstore": "gs://my-bucket/path/to/volume/"
}
```

--------------------------------

### Open neuroglancer_uint64_sharded with identity hash and 1GB cache

Source: https://github.com/google/tensorstore/blob/master/tensorstore/kvstore/neuroglancer_uint64_sharded/index.rst

Example JSON specification for opening a neuroglancer_uint64_sharded key-value store. Uses an identity hash and configures a 1GB cache.

```json
{
  "driver": "neuroglancer_uint64_sharded",
  "kvstore": "gs://my-bucket/path/to/sharded/data/",
  "metadata": {
    "@type": "neuroglancer_uint64_sharded_v1",
    "hash": "identity",
    "preshift_bits": 1,
    "minishard_bits": 3,
    "shard_bits": 3,
    "data_encoding": "raw",
    "minishard_index_encoding": "gzip"
  },
  "context": {
    "cache_pool": {"total_bytes_limit": 1000000000}
  }
}
```