TensorStore

https://google.github.io/tensorstore
Admin
TensorStore is a C++ library for efficiently reading and writing large multi-dimensional arrays with...
Tokens:705,809
Snippets:5,578
Trust Score:9.7
Update:6 months ago
Show doc for...
Tokens:
Raw
### Array Construction Examples

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.Array.Array-element_pointer-shape-order.html

Examples demonstrating array creation with C-style and Fortran-style layout orders.

```cpp
int data[6] = {1, 2, 3, 4, 5, 6}; auto c_array = Array(&data[0], {2, 3}); EXPECT_EQ(MakeArray<int>({{1, 2, 3}, {4, 5, 6}}), array);
auto f_array = Array(&data[0], {3, 2}, fortran_order); EXPECT_EQ(MakeArray<int>({{1, 4}, {2, 5}, {3, 6}}), array);
```

--------------------------------

### Relative Path Example

Source: https://google.github.io/tensorstore/driver/image/jpeg/index.html

Example demonstrating the use of the 'path' member for specifying an additional path relative to the kvstore.

```string
"path/to/data"
```

--------------------------------

### Zip Driver Example JSON

Source: https://google.github.io/tensorstore/kvstore/zip/index.html

An example of how to specify the zip driver in a JSON configuration.

```APIDOC
## Example JSON specifications

```json
{
  "driver": "zip",
  "kvstore": "gs://my-bucket/path/to/file.zip"
}
```
```

--------------------------------

### Define base_url examples

Source: https://google.github.io/tensorstore/kvstore/http/index.html

Examples of valid base_url strings for the HTTP driver.

```json
"https://example.com"
```

```json
"https://example.com:8000/path/to/data"
```

```json
"https://example.com:8000/path/to/data?query=value"
```

--------------------------------

### Install TensorStore Python API from local source

Source: https://google.github.io/tensorstore/installation.html

Installs TensorStore from a local git repository checkout. This command invokes Bazel to build the C++ extension module. Ensure build prerequisites are installed.

```bash
git clone https://github.com/google/tensorstore
cd tensorstore
python3 -m pip install .

```

--------------------------------

### Install TensorStore Python API from PyPI on Windows

Source: https://google.github.io/tensorstore/installation.html

Alternative command for installing the TensorStore Python API from PyPI on Windows systems.

```bash
py -3 -m pip install tensorstore -vv

```

--------------------------------

### Example of HalfOpenInterval Usage in C++

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.HalfOpenInterval.html

Demonstrates the effect of HalfOpenInterval on dimension selection and input domain. The example shows how specifying start, stop, and strides transforms the domain and equivalent input indices.

```cpp
Dims(0, 2).HalfOpenInterval({1, 8}, {4, 3}, {1, -2})
```

--------------------------------

### Constructing IndexDomains

Source: https://google.github.io/tensorstore/python/api/tensorstore.IndexDomain.__init__-components.html

Examples showing how to initialize an IndexDomain by specifying rank or shape.

```python
>>> ts.IndexDomain(rank=5)
{ (-inf*, +inf*), (-inf*, +inf*), (-inf*, +inf*), (-inf*, +inf*), (-inf*, +inf*) }
>>> ts.IndexDomain(shape=[2, 3])
{ [0, 2), [0, 3) }
```

--------------------------------

### N5 URL representation examples

Source: https://google.github.io/tensorstore/driver/n5/index.html

Examples showing the mapping between URL strings and JSON configuration objects for N5 datasets.

```json
{"driver": "n5",
 "kvstore": {"driver": "file",
             "path": "/tmp/dataset.n5/"}
}
```

```json
{"driver": "n5",
 "kvstore": {"driver": "file",
             "path": "/tmp/dataset.n5/path/within/hierarchy/"}
}
```

--------------------------------

### Install Build Dependencies

Source: https://google.github.io/tensorstore/installation.html

Install required system packages on Debian-based systems.

```bash
sudo apt-get install build-essential git nasm python3 python3-dev python3-pip python3-venv
```

--------------------------------

### Configure Zlib Compressor

Source: https://google.github.io/tensorstore/driver/zarr/index.html

Example configuration for Zlib or Gzip compression.

```json
{"id": "gzip", "level": 9}
```

--------------------------------

### S3 Bucket Endpoint Example

Source: https://google.github.io/tensorstore/kvstore/s3/index.html

Example of an S3 bucket URL, potentially including a virtual-hosted-style endpoint.

```json
"https://mybucket.cwobject.com"
```

--------------------------------

### Example: MurmurHash3 Configuration

Source: https://google.github.io/tensorstore/kvstore/neuroglancer_uint64_sharded/index.html

Example JSON specification for opening the neuroglancer_uint64_sharded driver with murmurhash3_x86_128 hash and a 1GB cache pool.

```APIDOC
### Example: Opening with murmurhash3_x86_128 hash and 1GB cache

```json
{
  "driver": "neuroglancer_uint64_sharded",
  "kvstore": "gs://my-bucket/path/to/sharded/data/",
  "metadata": {
    "@type": "neuroglancer_uint64_sharded_v1",
    "hash": "murmurhash3_x86_128",
    "preshift_bits": 0,
    "minishard_bits": 3,
    "shard_bits": 3,
    "data_encoding": "raw",
    "minishard_index_encoding": "gzip"
  },
  "context": {
    "cache_pool": {"total_bytes_limit": 1000000000}
  }
}
```
```

--------------------------------

### Example: Identity Hash Configuration

Source: https://google.github.io/tensorstore/kvstore/neuroglancer_uint64_sharded/index.html

Example JSON specification for opening the neuroglancer_uint64_sharded driver with an identity hash and a 1GB cache pool.

```APIDOC
## Example JSON specifications

### Example: Opening with identity hash and 1GB cache

```json
{
  "driver": "neuroglancer_uint64_sharded",
  "kvstore": "gs://my-bucket/path/to/sharded/data/",
  "metadata": {
    "@type": "neuroglancer_uint64_sharded_v1",
    "hash": "identity",
    "preshift_bits": 1,
    "minishard_bits": 3,
    "shard_bits": 3,
    "data_encoding": "raw",
    "minishard_index_encoding": "gzip"
  },
  "context": {
    "cache_pool": {"total_bytes_limit": 1000000000}
  }
}
```
```

--------------------------------

### Install TensorStore Python API from PyPI

Source: https://google.github.io/tensorstore/installation.html

Installs the latest published version of the TensorStore Python API using pip. Use the -vv option to show progress. This is the recommended method for most users.

```bash
python3 -m pip install tensorstore -vv

```

--------------------------------

### DimExpression::Stride Usage Examples

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.Stride.html

Examples of how to invoke the Stride method using dimension selection and stride vectors.

```cpp
Dims(0, 2).Stride({-2, 3})
```

```cpp
Stride({1, 2, 3})
```

```cpp
Stride(5)
```

--------------------------------

### ChainResult Example with Result-wrapped and Plain Values

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.ChainResult.html

Demonstrates using ChainResult with an initial value that is already a Result, and another example with a plain initial value. Assumes the existence of functions func1, func2, and func3 with appropriate signatures.

```cpp
float func1(int x);
Result<std::string> func2(float x);
bool func3(std::string_view x);

Result<bool> y1 = ChainResult(Result<int>(3), func1, func2, func3);
Result<bool> y2 = ChainResult(3, func1, func2, func3);


```

--------------------------------

### Downsample Driver Schema Example

Source: https://google.github.io/tensorstore/driver/downsample/index.html

An example of a TensorStore schema configuration using the `downsample` driver. This includes specifying downsample factors, the downsample method, and the base TensorStore configuration.

```json
{
  "driver": "downsample",
  "downsample_factors": [1, 2, 2],
  "downsample_method": "mean",
  "base": {"driver": "zarr", "kvstore": {"driver": "gcs", "bucket": "my-bucket"}}
}
```

--------------------------------

### Set up Python virtual environment and dependencies

Source: https://google.github.io/tensorstore/installation.html

Creates a Python virtual environment and installs necessary dependencies like pip, setuptools, and numpy for building TensorStore from source.

```bash
python3 -m venv ts-venv
source ts-venv/bin/activate
python3 -m pip install --upgrade pip setuptools numpy

```

--------------------------------

### Get BoxView Origin in C++

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.BoxView.origin.html

Retrieves the origin array of length `rank()` from a `BoxView`. This is useful for understanding the starting coordinates of the view.

```cpp
#include "tensorstore/box.h"

span<IndexType, Rank>  
tensorstore::BoxView<Rank, Mutable>::origin() const;
```

--------------------------------

### Build TensorStore Documentation

Source: https://google.github.io/tensorstore/installation.html

Run the documentation build tool using bazelisk.

```bash
python3 bazelisk.py run //tools/docs:build_docs -- --output /tmp/tensorstore-docs
```

--------------------------------

### Get TensorStore Spec Transform

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.Spec.transform.html

Retrieves the transform applied on top of the driver for a TensorStore specification. No setup or imports are required beyond including the necessary header.

```cpp
#include "tensorstore/spec.h"



const IndexTransform<>& tensorstore::Spec::transform() const;
```

--------------------------------

### Get Index Expression with Inclusive and Exclusive Bounds

Source: https://google.github.io/tensorstore/python/api/tensorstore.IndexDomain.index_exp.html

Generates an index expression from inclusive minimum and exclusive maximum bounds, resulting in slices with specified start and end points.

```python
>>> ts.IndexDomain(inclusive_min=[1, 2], exclusive_max=[5, 10]).index_exp
(slice(1, 5, None), slice(2, 10, None))
```

--------------------------------

### Get Output Rank of Transform

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.OutputIndexMapRange.size.html

Use this method to retrieve the output rank of the transform represented by `OutputIndexMapRange`. No specific setup or imports are required beyond including the necessary header.

```cpp
#include "tensorstore/index_space/output_index_map.h"

StaticOrDynamicRank<OutputRank>
tensorstore::OutputIndexMapRange<InputRank, OutputRank, CKind>::size() const;
```

--------------------------------

### Open and Use Memory KvStore

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.html

Demonstrates opening a memory-based KvStore, writing a key-value pair, reading it back, and listing available keys. Use this for in-memory storage or testing.

```python
store = await ts.KvStore.open({'driver': 'memory'})
await store.write(b'a', b'value')
KvStore.TimestampedStorageGeneration(...)
await store.read(b'a')
KvStore.ReadResult(state='value', value=b'value', stamp=KvStore.TimestampedStorageGeneration(...))
await store.read(b'b')
KvStore.ReadResult(state='missing', value=b'', stamp=KvStore.TimestampedStorageGeneration(...))
await store.list()
[b'a']
```

--------------------------------

### Open KvStore with Path

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.path.html

Demonstrates opening a KvStore with a specified path prefix using the 'gcs' driver. The path is stored and can be retrieved later.

```python
>>> store = await ts.KvStore.open({
...     'driver': 'gcs',
...     'bucket': 'my-bucket',
...     'path': 'path/to/object'
... })
>>> store.spec()
KvStore.Spec({'bucket': 'my-bucket', 'driver': 'gcs', 'path': 'path/to/object'})
>>> store.path
'path/to/object'
```

--------------------------------

### TranslateBoxSlice Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.TranslateBoxSlice.html

Example demonstrating the effect of TranslateBoxSlice on dimension selection.

```cpp
Dims(0, 2).TranslateBoxSlice(BoxView({1, 4}, {3, 4}))
```

--------------------------------

### Read and List Keys in KvStore

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.read.html

Demonstrates reading values from a memory-backed store, handling missing keys, and listing keys.

```python
>>> store = await ts.KvStore.open({'driver': 'memory'})
>>> await store.write(b'a', b'value')
KvStore.TimestampedStorageGeneration(...)
>>> await store.read(b'a')
KvStore.ReadResult(state='value', value=b'value', stamp=KvStore.TimestampedStorageGeneration(...))
>>> store[b'a']
b'value'
>>> await store.read(b'b')
KvStore.ReadResult(state='missing', value=b'', stamp=KvStore.TimestampedStorageGeneration(...))
>>> store[b'b']
Traceback (most recent call last):
    ...
KeyError...
```

```python
>>> store[b'a'] = b'value'
>>> store[b'b'] = b'value'
>>> store.list().result()
```

--------------------------------

### SizedInterval Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.SizedInterval.html

Example demonstrating the effect of SizedInterval on dimension selection.

```cpp
Dims(0, 2).SizedInterval({1, 8}, {3, 2}, {1, -2})
```

--------------------------------

### KvStore Constructor with Driver

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.kvstore.KvStore.KvStore-driver.html

Constructs a KvStore using only a driver. This is a basic initialization.

```cpp
tensorstore::kvstore::KvStore::KvStore(DriverPtr driver);
```

--------------------------------

### Corresponding IndexDomain Example

Source: https://google.github.io/tensorstore/driver/zarr3/index.html

The IndexDomain representation corresponding to the example Zarr v3 metadata.

```json
{
  "exclusive_max": [[1000], [2000], [3000]],
  "inclusive_min": [0, 0, 0],
  "labels": ["x", "y", "z"]
}
```

--------------------------------

### Configure and Access Context Resources in C++

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.Context.html

Demonstrates creating a context from a JSON specification, retrieving a resource, and executing tasks using the resource's executor.

```cpp
auto spec_result = Context::Spec::FromJson({
  {"data_copy_concurrency", {{"limit", 4}}},
  {"data_copy_concurrency#a", {{"limit", 8}}},
});
if (!spec_result) {
  // Handle error
}
auto resource_spec_result =
    Context::Resource<internal::DataCopyConcurrencyResource>
    ::FromJson("data_copy_concurrency");
if (!resource_spec_result) {
  // Handle error
}
auto context = Context(*spec_result);
auto resource_result = context.GetResource(
    resource_spec_result->value());
if (!resource_result) {
  // Handle error
}
// Can access executor member of context resource.
(*resource_result)->executor([] {
  // task body
});
// Access other executor.
auto resource2 = context.GetResource(
    Context::Resource<internal::DataCopyConcurrencyResource>
    ::FromJson("data_copy_concurrency#a").value()).value();
resource2->executor([] { /* task body */ });
```

--------------------------------

### StridedLayout Example Usage

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.StridedLayout.html

Demonstrates how to create and use the StridedLayout class with different configurations.

```APIDOC
## Example Usage

```cpp
tensorstore::StridedLayout<2> x(tensorstore::c_order, sizeof(int),
                                {3, 4});
EXPECT_EQ(3, x.rank());

// Indexing with `operator()` requires all dimensions be specified.
EXPECT_EQ((4 + 2) * sizeof(int), x(1, 2));
EXPECT_EQ((4 + 2) * sizeof(int), x({1, 2}));

// Indexing with `operator[]` supports partial indexing.
EXPECT_EQ(4 * sizeof(int), x[1]);
EXPECT_EQ((4 + 2) * sizeof(int), x[{1, 2}]);

// Create a StridedLayoutView that refers to x.
tensorstore::StridedLayoutView<2> x_ref = x;
```
```

--------------------------------

### All Unlabeled Dimensions Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.IndexDomain.operator-call-transform.html

Example demonstrating slicing when both transform and domain have only unlabeled dimensions.

```text
* transform: `[0, 5), [1, 7)`
* domain: `[2, 4), [3, 6)`
* result: `[2, 4), [3, 6)`
```

--------------------------------

### Define File KvStore via JSON

Source: https://google.github.io/tensorstore/kvstore/file/index.html

Examples of mapping file URLs to their corresponding JSON configuration objects.

```json
{"driver": "file",
 "path": "/tmp/dataset/"}
```

```json
{"driver": "file",
 "path": "C:/Users/abc/dataset/"}
```

--------------------------------

### Downsample Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.Downsample-Spec.html

Example of calling the Downsample function with a braced list for factors.

```cpp
Downsample(base_spec, {2, 3}, DownsampleMethod::kMean)
```

--------------------------------

### MarkBoundsExplicit Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.MarkBoundsExplicit.html

An example demonstrating the effect of calling MarkBoundsExplicit on selected dimensions.

```cpp
Dims(0, 2).MarkBoundsExplicit()
```

--------------------------------

### Construct a Batch View

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.Batch.View.View-no_batch.html

Includes the necessary header and demonstrates the default and no-batch constructors for Batch::View.

```cpp
#include "tensorstore/batch.h"
```

```cpp
constexpr tensorstore::Batch::View::View();
```

```cpp
constexpr tensorstore::Batch::View::View(no_batch_t);
```

--------------------------------

### Initialize and index a StridedLayout

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.StridedLayout.html

Demonstrates creating a 2D StridedLayout with C-order, calculating byte offsets, and creating a view.

```cpp
tensorstore::StridedLayout<2> x(tensorstore::c_order, sizeof(int),
                                {3, 4});
EXPECT_EQ(3, x.rank());

// Indexing with `operator()` requires all dimensions be specified.
EXPECT_EQ((4 + 2) * sizeof(int), x(1, 2));
EXPECT_EQ((4 + 2) * sizeof(int), x({1, 2}));

// Indexing with `operator[]` supports partial indexing.
EXPECT_EQ(4 * sizeof(int), x[1]);
EXPECT_EQ((4 + 2) * sizeof(int), x[{1, 2}]);

// Create a StridedLayoutView that refers to x.
tensorstore::StridedLayoutView<2> x_ref = x;
```

--------------------------------

### ClosedInterval Usage Examples

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.ClosedInterval.html

Examples demonstrating the effect of ClosedInterval on dimension selection and domain transformation.

```cpp
Dims(0, 2).ClosedInterval({1, 8}, {4, 3}, {1, -2})
```

```cpp
Dims(0, 2).ClosedInterval({1, 9}, {4, 3}, {1, -2})
```

--------------------------------

### Open KvStore with Spec and Options

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.kvstore.Open.html

Opens a KvStore using a pre-parsed `kvstore::Spec` object and an `OpenOptions` object. This is useful when the specification is already available in a structured format.

```cpp
#include "tensorstore/kvstore/kvstore.h"

Future<KvStore>  
tensorstore::kvstore::Open(Spec spec, OpenOptions&& options);
```

--------------------------------

### DimSelection Initialization and Examples

Source: https://google.github.io/tensorstore/python/api/tensorstore.DimSelection.html

Demonstrates how to construct DimSelection objects using various indexing methods with ts.d.

```APIDOC
## DimSelection Construction Examples

### Description
Examples of creating `DimSelection` objects using `tensorstore.d` with different indexing specifications.

### Examples
```python
>>> import tensorstore as ts

# Select dimensions by index
>>> ts.d[0, 1, 2]
d[0,1,2]

# Select dimensions using slices and labels
>>> ts.d[0:1, 2, "x"]
d[0:1,2,'x']

# Select dimensions using lists of indices
>>> ts.d[[0, 1], [2]]
d[0,1,2]

# Combine list indexing with dimension selection
>>> ts.d[[0, 1], ts.d[2, 3]]
d[0,1,2,3]
```
```

--------------------------------

### Writing and Deleting Keys in KvStore

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.write.html

Demonstrates opening a memory-backed store, writing a value, reading it back, and deleting the key by writing None.

```python
>>> store = await ts.KvStore.open({'driver': 'memory'})
>>> await store.write(b'a', b'value')
KvStore.TimestampedStorageGeneration(...)
>>> await store.read(b'a')
KvStore.ReadResult(state='value', value=b'value', stamp=KvStore.TimestampedStorageGeneration(...))
>>> await store.write(b'a', None)
KvStore.TimestampedStorageGeneration(...)
>>> await store.read(b'a')
KvStore.ReadResult(state='missing', value=b'', stamp=KvStore.TimestampedStorageGeneration(...))
```

--------------------------------

### MapFutureValue Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.MapFutureValue.html

Example demonstrating how to combine two integer futures using an inline executor.

```cpp
Future<int> future_a = ...;
Future<int> future_b = ...;
Future<int> mapped_future = MapFutureValue(
    InlineExecutor{}, [](int a, int b) { return a + b; },
    future_a, future_b);
```

--------------------------------

### Create and Index TensorStore Dataset with vindex

Source: https://google.github.io/tensorstore/python/api/tensorstore.TensorStore.vindex.html

Demonstrates creating a Zarr dataset in memory and using the vindex method with array indexing for a virtual view. Ensure the TensorStore library is imported as 'ts'.

```python
>>> dataset = await ts.open(
...     {
...         'driver': 'zarr',
...         'kvstore': {
...             'driver': 'memory'
...         }
...     },
...     dtype=ts.uint32,
...     shape=[60, 70, 80],
...     create=True)
>>> view = dataset.vindex[:, [5, 10, 20], [7, 8, 10]]
>>> view
TensorStore({
  'context': {
    'cache_pool': {},
    'data_copy_concurrency': {},
    'memory_key_value_store': {},
  },
  'driver': 'zarr',
  'dtype': 'uint32',
  'kvstore': {'driver': 'memory'},
  'metadata': {
    'chunks': [60, 70, 80],
    'compressor': {
      'blocksize': 0,
      'clevel': 5,
      'cname': 'lz4',
      'id': 'blosc',
      'shuffle': -1,
    },
    'dimension_separator': '.',
    'dtype': '<u4',
    'fill_value': None,
    'filters': None,
    'order': 'C',
    'shape': [60, 70, 80],
    'zarr_format': 2,
  },
  'transform': {
    'input_exclusive_max': [3, [60]],
    'input_inclusive_min': [0, 0],
    'output': [
      {'input_dimension': 1},
      {'index_array': [[5], [10], [20]]},
      {'index_array': [[7], [8], [10]]},
    ],
  },
})
```

--------------------------------

### Fully Labeled Dimensions Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.IndexDomain.operator-call-transform.html

Example demonstrating slicing when both transform and domain have fully labeled dimensions.

```text
* transform: `"x": [0, 5), "y": [1, 7), "z": [2, 8)`
* domain: `"y": [2, 6), "x": [3, 4)`
* result: `"x": [3, 4), "y": [2, 6), "z": [2, 8)`
```

--------------------------------

### Configure AVIF driver with JSON

Source: https://google.github.io/tensorstore/driver/image/avif/index.html

Example of a basic AVIF driver configuration pointing to a Google Cloud Storage bucket.

```json
{"driver": "avif", "kvstore": "gs://my-bucket/path-to-image.avif"}
```

--------------------------------

### N5 Metadata Example

Source: https://google.github.io/tensorstore/driver/n5/index.html

Example N5 metadata showing dimensions, block size, data type, and compression.

```json
{
  "dimensions": [1000, 2000, 3000],
  "blockSize": [100, 200, 300],
  "dataType": "uint16",
  "compression": {"type": "raw"}
}

```

--------------------------------

### Create a local N5 dataset

Source: https://google.github.io/tensorstore/python/tutorial.html

Initializes a new N5 dataset on the local filesystem using the file Key-Value Store driver.

```python
>>> import tensorstore as ts
>>> import numpy as np
>>> dataset = ts.open({
...     'driver': 'n5',
...     'kvstore': {
...         'driver': 'file',
...         'path': 'tmp/dataset/',
...     },
...     'metadata': {
...         'compression': {
...             'type': 'gzip'
...         },
...         'dataType': 'uint32',
...         'dimensions': [1000, 20000],
...         'blockSize': [100, 100],
...     },
...     'create': True,
...     'delete_existing': True,
... }).result()
```

--------------------------------

### StaticRankCast Usage Examples

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.StaticRankCast.html

Examples demonstrating checked and unchecked static rank casting for different TensorStore types.

```cpp
StaticRank<3> r_static = StaticRankCast<3,
unchecked>(DimensionIndex(3)); Box<> box = ...; Result<Box<3>>
checked_result = StaticRankCast<3>(box); Box<3> unchecked_result =
StaticRankCast<3, unchecked>(box);
```

--------------------------------

### TranslateBackwardBy Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.TranslateBackwardBy.html

Example demonstrating the effect of shifting dimensions 0 and 2 by offsets 10 and 20.

```cpp
Dims(0, 2).TranslateBackwardBy({10, 20})
```

--------------------------------

### Construct KVStore Spec with Driver

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.kvstore.Spec.Spec-driver.html

Constructs a KVStore Spec from a driver with an empty path. Include the necessary header for Spec and DriverSpecPtr.

```cpp
#include "tensorstore/kvstore/spec.h"


tensorstore::kvstore::Spec::Spec(DriverSpecPtr driver);
```

--------------------------------

### Transpose Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.Transpose-target_dimensions.html

Example showing the effect of transposing dimensions 2 and 0 to indices 1 and 2.

```cpp
Dims(2, 0).Transpose({1, 2})
```

--------------------------------

### OuterIndexArraySlice Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.OuterIndexArraySlice.html

Example demonstrating the effect of slicing dimensions 2 and 0 with specific index arrays.

```cpp
Dims(2, 0).OuterIndexArraySlice(MakeArray<Index>({{2, 3}, {4, 5}}),
                                MakeArray<Index>({6, 7}))
```

--------------------------------

### Open KvStore with Spec and Variadic Options

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.kvstore.Open.html

Opens a KvStore using a `kvstore::Spec` and a variadic sequence of options. This overload allows specifying options directly as arguments, offering flexibility in configuration.

```cpp
template <typename... Option>  
requires IsCompatibleOptionSequence<OpenOptions, Option...>  
static Future<KvStore>  
tensorstore::kvstore::Open(Spec spec, Option&&... option);
```

--------------------------------

### List keys in a memory-backed KvStore

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.list.html

Opens an in-memory key-value store, adds two key-value pairs, and then lists all keys. This demonstrates basic usage of KvStore.list().

```python
>>> store = ts.KvStore.open({'driver': 'memory'}).result()
>>> store[b'a'] = b'value'
>>> store[b'b'] = b'value'
>>> store.list().result()
[b'a', b'b']
```

--------------------------------

### Build pre-built extension module and create wheel

Source: https://google.github.io/tensorstore/installation.html

Skips building the extension module directly and instead uses a pre-built module. First, build the extension, then use pip to create a wheel package.

```bash
python3 setup.py build_ext -b /tmp/prebuilt
TENSORSTORE_PREBUILT_DIR=/tmp/prebuilt pip wheel .

```

--------------------------------

### Zarr v3 Metadata Example

Source: https://google.github.io/tensorstore/driver/zarr3/index.html

Example Zarr v3 metadata structure used to define an array's properties.

```json
{
  "zarr_format": 3,
  "node_type": "array",
  "shape": [1000, 2000, 3000],
  "dimension_names": ["x", "y", "z"],
  "chunk_grid": {"name": "regular", "configuration": {"chunk_shape": [100, 200, 300]}},
  "chunk_key_encoding": {"name": "default"},
  "data_type": "uint16",
  "codecs": [{"name": "bytes", "configuration": {"endian": "little"}}],
  "fill_value": 0
}
```

--------------------------------

### KvStore Open Options

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.kvstore.OpenOptions.html

The `OpenOptions` struct contains driver-agnostic options that can be specified when opening a `KvStore`. Refer to the `kvstore::Open` documentation for detailed usage.

```APIDOC
## KvStore Open Options

### Description
Driver-agnostic options that may be specified when opening a `KvStore`.

### Method
N/A (Struct definition)

### Endpoint
N/A (Struct definition)

### Parameters
N/A (Struct definition)

### Request Example
N/A (Struct definition)

### Response
N/A (Struct definition)
```

--------------------------------

### Open KvStore from URL

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.open.html

Use this when opening a key-value store directly from a URL string. Requires asynchronous execution.

```python
kvstore = await ts.KvStore.open('memory://abc/')
kvstore.spec()
```

--------------------------------

### Mixed Labeled and Unlabeled Dimensions Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.IndexDomain.operator-call-transform.html

Example demonstrating slicing with a mix of labeled and unlabeled dimensions in both transform and domain.

```text
* transform: `"x": [0, 10), "": [0, 10), "": [0, 10), "y": [0, 10)`
* domain: `"y": [1, 6), "": [2, 7), "x": [3, 8), "": [4, 9)`
* result: `"x": [3, 8), "": [2, 7), "": [4, 9), "y": [1, 6)`
```

--------------------------------

### TranslateBy Usage Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.TranslateBy.html

A practical example showing the effect of shifting dimensions 0 and 2 by offsets of 10 and 20.

```cpp
Dims(0, 2).TranslateBy({10, 20})
```

--------------------------------

### Retrieve KvStore string representation

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.__repr__.html

Demonstrates opening a file-based KvStore and viewing its string representation.

```python
>>> kvstore = await ts.KvStore.open({
...     'driver': 'file',
...     'path': 'tmp/data/'
... })
>>> kvstore
KvStore({
  'context': {
    'file_io_concurrency': {},
    'file_io_locking': {},
    'file_io_mode': {},
    'file_io_sync': True,
  },
  'driver': 'file',
  'path': 'tmp/data/',
})
```

--------------------------------

### Define Inclusive Start Index

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimRangeSpec.inclusive_start.html

Specifies the inclusive start index for a dimension range, defaulting based on the step direction.

```cpp
std::optional<DimensionIndex>  
tensorstore::DimRangeSpec::inclusive_start;
```

--------------------------------

### Verifying Unit parsing behavior

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.Unit.Unit-string.html

Examples demonstrating how various string inputs are parsed into multiplier and base unit components.

```cpp
EXPECT_EQ(Unit(4, "nm"), Unit("4nm"));
EXPECT_EQ(Unit(4e-3, "nm"), Unit("4e-3nm"));
EXPECT_EQ(Unit(4e-3, "nm"), Unit("+4e-3nm"));
EXPECT_EQ(Unit(-4e-3, "nm"), Unit("-4e-3nm"));
EXPECT_EQ(Unit(4.5, "nm"), Unit("4.5nm"));
EXPECT_EQ(Unit(1, "nm"), Unit("nm"));
EXPECT_EQ(Unit(4, ""), Unit("4"));
EXPECT_EQ(Unit(1, ""), Unit(""));
```

--------------------------------

### S3 Server Endpoint Example

Source: https://google.github.io/tensorstore/kvstore/s3/index.html

Example of an S3 server endpoint URL. This is used when not connecting to the public Amazon S3 endpoints.

```json
"http://localhost:1234"
```

--------------------------------

### Create and Index a TensorStore Dataset

Source: https://google.github.io/tensorstore/python/api/tensorstore.TensorStore.__getitem__-indices.html

Opens a TensorStore dataset using Zarr driver and memory kvstore, then creates a virtual view using NumPy-style indexing. Data is not read until `read` is called on the view.

```python
>>> dataset = await ts.open(
...     {
...         'driver': 'zarr',
...         'kvstore': {
...             'driver': 'memory'
...         }
...     },
...     dtype=ts.uint32,
...     shape=[70, 80],
...     create=True)
>>> view = dataset[[5, 10, 20], 6:10]
>>> view
```

--------------------------------

### Construct KvStore.Spec from JSON or URL

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.Spec.__new__.html

Create a specification object by providing either a dictionary representing the driver configuration or a URL string.

```python
>>> spec = ts.KvStore.Spec({'driver': 'file', 'path': 'tmp/data/'})
>>> spec
KvStore.Spec({'driver': 'file', 'path': 'tmp/data/'})
```

```python
>>> spec = ts.KvStore.Spec('file:///path/to/data/')
>>> spec
KvStore.Spec({'driver': 'file', 'path': '/path/to/data/'})
```

--------------------------------

### Get IndexDomain Shape

Source: https://google.github.io/tensorstore/python/api/tensorstore.IndexDomain.shape.html

Access the shape attribute of an IndexDomain object to get its dimensions. This is useful for understanding the size and structure of the domain.

```python
>>> domain = ts.IndexDomain(inclusive_min=[1, 2, 3], shape=[3, 4, 5])
>>> domain.shape
(3, 4, 5)
```

--------------------------------

### Accessing TensorStore KvStore

Source: https://google.github.io/tensorstore/python/api/tensorstore.TensorStore.kvstore.html

Demonstrates opening a TensorStore with a memory-backed kvstore and accessing the kvstore property.

```python
>>> store = await ts.open(
...     {
...         'driver': 'n5',
...         'kvstore': {
...             'driver': 'memory',
...             'path': 'abc/',
...         }
...     },
...     create=True,
...     shape=[100, 200],
...     dtype=ts.uint32,
... )
>>> store.kvstore
KvStore({'context': {'memory_key_value_store': {}}, 'driver': 'memory', 'path': 'abc/'})
```

--------------------------------

### Select Interval with Slice

Source: https://google.github.io/tensorstore/python/indexing.html

Use a slice object `start:stop` to select an interval. The `start` value is inclusive, and the `stop` value is exclusive.

```python
>>> a = ts.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=ts.int32)
>>> a[1:5]
```

--------------------------------

### Start DimExpression with a dimension range

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.html

Initiates a DimExpression with a specified range of dimensions. Supports inclusive start, optional exclusive stop, and a step value.

```cpp
auto tensorstore::DimRange(
DimensionIndex inclusive_start,
std::optional<DimensionIndex> exclusive_stop = std::nullopt,
const DimensionIndex step = 1);
```

--------------------------------

### Create and Use a Transaction

Source: https://google.github.io/tensorstore/python/api/tensorstore.Transaction.html

Demonstrates creating a transaction, opening a dataset, performing writes within the transaction, and observing changes with transactional and non-transactional reads.

```python
>>> txn = ts.Transaction()
>>> store = ts.open({
...     'driver': 'n5',
...     'kvstore': {
...         'driver': 'file',
...         'path': 'tmp/dataset/'
...     },
...     'metadata': {
...         'dataType': 'uint16',
...         'blockSize': [2, 3],
...         'dimensions': [5, 6],
...         'compression': {
...             'type': 'raw'
...         }
...     },
...     'create': True,
...     'delete_existing': True
... }).result()
>>> store.with_transaction(txn)[1:4, 2:5] = 42
>>> store.with_transaction(txn)[0:2, 4] = 43
```

```python
>>> store.with_transaction(txn).read().result()
array([[ 0,  0,  0,  0, 43,  0],
       [ 0,  0, 42, 42, 43,  0],
       [ 0,  0, 42, 42, 42,  0],
       [ 0,  0, 42, 42, 42,  0],
       [ 0,  0,  0,  0,  0,  0]], dtype=uint16)
```

```python
>>> store.read().result()
array([[0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0]], dtype=uint16)
```

--------------------------------

### Get TensorStore IndexTransform Input Shape

Source: https://google.github.io/tensorstore/python/api/tensorstore.IndexTransform.input_shape.html

Use the `input_shape` property to get the shape of the input domain for an IndexTransform. This is an alias for the `shape` property of the `domain`.

```python
transform = ts.IndexTransform(input_shape=[3, 4, 5])
transform.input_shape
```

--------------------------------

### Open or Create a TensorStore

Source: https://google.github.io/tensorstore/python/api/tensorstore.open.html

Initializes a new TensorStore using the Zarr driver in memory with specified shape, dtype, and chunk layout.

```python
>>> store = await ts.open(
...     {
...         'driver': 'zarr',
...         'kvstore': {
...             'driver': 'memory'
...         }
...     },
...     create=True,
...     dtype=ts.int32,
...     shape=[1000, 2000, 3000],
...     chunk_layout=ts.ChunkLayout(inner_order=[2, 1, 0]),
... )
>>> store
TensorStore({
  'context': {
    'cache_pool': {},
    'data_copy_concurrency': {},
    'memory_key_value_store': {},
  },
  'driver': 'zarr',
  'dtype': 'int32',
  'kvstore': {'driver': 'memory'},
  'metadata': {
    'chunks': [101, 101, 101],
    'compressor': {
      'blocksize': 0,
      'clevel': 5,
      'cname': 'lz4',
      'id': 'blosc',
      'shuffle': -1,
    },
    'dimension_separator': '.',
    'dtype': '<i4',
    'fill_value': None,
    'filters': None,
    'order': 'F',
    'shape': [1000, 2000, 3000],
    'zarr_format': 2,
  },
  'transform': {
    'input_exclusive_max': [[1000], [2000], [3000]],
    'input_inclusive_min': [0, 0, 0],
  },
})
```

--------------------------------

### Get Input Rank of IndexTransform

Source: https://google.github.io/tensorstore/python/api/tensorstore.IndexTransform.input_rank.html

Access the `input_rank` property of an `IndexTransform` object to get the rank of the input space. This is determined by the `input_shape` when the transform is created.

```python
>>> transform = ts.IndexTransform(input_shape=[3, 4, 5],
...                               input_labels=["x", "y", "z"])
>>> transform.input_rank
3
```

--------------------------------

### Accessing and updating Schema codec

Source: https://google.github.io/tensorstore/python/api/tensorstore.Schema.codec.html

Demonstrates initializing a schema, checking the default codec, and updating it with specific Zarr driver configurations.

```python
>>> schema = ts.Schema()
>>> print(schema.codec)
None
>>> schema.update(codec=ts.CodecSpec({
...     'driver': 'zarr',
...     'compressor': None
... }))
>>> schema.update(codec=ts.CodecSpec({'driver': 'zarr', 'filters': None}))
>>> schema.codec
CodecSpec({'compressor': None, 'driver': 'zarr', 'filters': None})
```

--------------------------------

### Initialize a tiff TensorStore via JSON

Source: https://google.github.io/tensorstore/driver/image/tiff/index.html

Example configuration for opening a TIFF file stored in a Google Cloud Storage bucket.

```json
{"driver": "tiff", "kvstore": "gs://my-bucket/path-to-image.tiff"}
```

--------------------------------

### S3 Host Header Override Example

Source: https://google.github.io/tensorstore/kvstore/s3/index.html

Example of overriding the HTTP host header for S3 requests. This is useful for testing with local services like LocalStack.

```json
"mybucket.s3.af-south-1.localstack.localhost.com"
```

--------------------------------

### Unsafe DimExpression Construction Example

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.html

This example demonstrates an unsafe way to construct a DimExpression by storing it in a variable that outlives the temporary values it references. Avoid this pattern.

```c++
auto do_not_use =
  Dims(std::string("x"), std::string("y")).IndexSlice({1, 2});
```

--------------------------------

### Open and Access KvStore Base

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.base.html

Demonstrates opening an 'ocdbt' driver key-value store with a 'memory://' base and accessing its base property. This is useful for understanding the structure of adapter key-value stores.

```python
>>> store = await ts.KvStore.open({
...     'driver': 'ocdbt',
...     'base': 'memory://'
... })
>>> store.base
KvStore({'context': {'memory_key_value_store': {}}, 'driver': 'memory'})
```

--------------------------------

### Get KvStore URL

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.url.html

Access the `url` property of an opened KvStore object to get its string representation. This is useful for identifying the store's location and configuration.

```python
store = await ts.KvStore.open({
    'driver': 'gcs',
    'bucket': 'my-bucket',
    'path': 'path/to/object'
})
store.url
```

--------------------------------

### Synchronously read a key from KvStore

Source: https://google.github.io/tensorstore/python/api/tensorstore.KvStore.__getitem__.html

Demonstrates opening a memory-backed KvStore, setting a value, and reading it back. Note that accessing a non-existent key raises a KeyError.

```python
>>> store = ts.KvStore.open({'driver': 'memory'}).result()
>>> store[b'a'] = b'value'
>>> store[b'a']
b'value'
>>> store[b'b']
Traceback (most recent call last):
    ...
KeyError...
```

--------------------------------

### KvStore Constructor with Driver, Path, and Transaction

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.kvstore.KvStore.KvStore-driver.html

Constructs a KvStore with a driver, an optional path, and an optional transaction. The path specifies a location within the driver, and the transaction can be omitted.

```cpp
explicit tensorstore::kvstore::KvStore::KvStore(
DriverPtr driver,
std::string path,
Transaction transaction = no_transaction
);
```

--------------------------------

### TranslateTo Example Usage

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.TranslateTo.html

Demonstrates how to use the TranslateTo method with a specific origin vector. This example shifts dimensions 0 and 2 to origins 10 and 20 respectively.

```cpp
Dims(0, 2).TranslateTo({10, 20})
```

--------------------------------

### Create a new N5 TensorStore with schema constraints

Source: https://google.github.io/tensorstore/python/api/tensorstore.open.html

Demonstrates applying schema constraints to an N5 driver, showing how storage characteristics are specified independently of the format.

```python
>>> store = await ts.open(
...     {
...         'driver': 'n5',
...         'kvstore': {
...             'driver': 'memory'
...         },
...     },
...     create=True,
...     dtype=ts.float32,
...     shape=[1000, 2000, 3000],
...     chunk_layout=ts.ChunkLayout(
...         chunk_shape=[10, None, None],
...         chunk_aspect_ratio=[None, 2, 1],
...         chunk_elements=10000000,
...     ),
... )
>>> store
```

--------------------------------

### TensorStore IndexArraySlice Example Usage

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.DimExpression.IndexArraySlice.html

Example demonstrating the use of IndexArraySlice with multiple index arrays for slicing dimensions. Ensure index arrays have compatible shapes and ranks.

```cpp
Dims(0, 2).IndexArraySlice(MakeArray<Index>({{1, 2, 3}, {4, 5, 6}}),
                           MakeArray<Index>({{7, 8, 9}, {0, 1, 2}}))
```

--------------------------------

### Reorder all selected dimensions to start at a positive index

Source: https://google.github.io/tensorstore/python/api/tensorstore.DimExpression.transpose.html

A single non-negative target index reorders all selected dimensions to start at that index. For `k > 1` dimensions, `i` is equivalent to `i:i+k`.

```python
>>> transform = ts.IndexTransform(input_shape=[2, 3, 4, 5],
...                               input_labels=["a", "b", "c", "d"])
>>> transform[ts.d['a', 'd'].transpose[1]]
Rank 4 -> 4 index space transform:
  Input domain:
    0: [0, 3) "b"
    1: [0, 2) "a"
    2: [0, 5) "d"
    3: [0, 4) "c"
  Output index maps:
    out[0] = 0 + 1 * in[1]
    out[1] = 0 + 1 * in[0]
    out[2] = 0 + 1 * in[3]
    out[3] = 0 + 1 * in[2]


```

--------------------------------

### Retrieve context resources using __getitem__

Source: https://google.github.io/tensorstore/python/api/tensorstore.Context.__getitem__.html

Demonstrates accessing context resources by key, including specific resource IDs and general resource types.

```python
>>> context = ts.Context(
...     {'cache_pool#a': {
...         'total_bytes_limit': 10000000
...     }})
>>> context['cache_pool#a']
Context.Resource({'total_bytes_limit': 10000000})
>>> context['cache_pool']
Context.Resource({})
```

--------------------------------

### Get exclusive_max of a Dimension

Source: https://google.github.io/tensorstore/python/api/tensorstore.Dim.exclusive_max.html

Access the `exclusive_max` property of a `ts.Dim` object to get the exclusive upper bound of the interval. This is equal to `inclusive_max + 1` or a special value for unbounded intervals.

```python
>>> ts.Dim(inclusive_min=5, inclusive_max=10).exclusive_max
11
```

```python
>>> ts.Dim(exclusive_max=5).exclusive_max
5
```

```python
>>> ts.Dim().exclusive_max
4611686018427387904
```

--------------------------------

### Run TensorStore Tests

Source: https://google.github.io/tensorstore/installation.html

Execute the test suite using bazelisk.

```bash
python3 bazelisk.py test //...
```

--------------------------------

### Get KvStore Spec with Options

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.kvstore.KvStore.spec.html

Use this function to get a Spec for re-opening a KvStore. Options like ContextBindingMode can be specified to control how context resources are handled in the returned Spec.

```cpp
#include "tensorstore/kvstore/kvstore.h"

template <typename... Option>
requires IsCompatibleOptionSequence<SpecRequestOptions, Option...>
Result<Spec>
tensorstore::kvstore::KvStore::spec(Option&&... option) const;
```

```cpp
Result<Spec> tensorstore::kvstore::KvStore::spec(
SpecRequestOptions&& options) const;
```

--------------------------------

### Create and Use a Promise/Future Pair

Source: https://google.github.io/tensorstore/python/api/tensorstore.Promise.html

Demonstrates the basic usage of creating a Promise and Future pair, setting a result, and retrieving it. Ensure the Promise is used to set the result or exception.

```python
>>> promise, future = ts.Promise.new()
>>> future.done()
False
>>> promise.set_result(5)
>>> future.done()
True
>>> future.result()
5
```

--------------------------------

### Get Input Shape of IndexTransform

Source: https://google.github.io/tensorstore/cpp/api/tensorstore.IndexTransform.input_shape.html

Use the `input_shape()` method on a valid `IndexTransform` object to get a span of extents for its input dimensions. The transform must be valid before calling this method.

```cpp
#include "tensorstore/index_space/index_transform.h"

span<const Index, InputRank> tensorstore::  
IndexTransform<InputRank, OutputRank, CKind>::input_shape() const;
```
Tokens:
Raw