### Install Dependencies with setup.sh

Source: https://github.com/nvidia/aistore/blob/main/scripts/gitlab/runners/k8s/README.md

Execute the setup script to install necessary dependencies like Docker, Minikube, kubectl, and GitLab Runner. Ensure you log out and back in after installation for rootless Docker access.

```bash
sudo ./setup.sh [--data-root <absolute_path>]
```

--------------------------------

### Example Workflow: Populate and Benchmark

Source: https://github.com/nvidia/aistore/blob/main/python/tests/perf/get_batch/README.md

A complete example workflow demonstrating the steps to populate an AIStore bucket and then run the batch get performance benchmark. This includes editing configuration files for both scripts.

```bash
# 1. Create 10,000 objects of 100 KiB each
# Edit populate_bucket.py configuration:
# BUCKET_NAME = "benchmark-100KiB"
# NUM_OBJECTS = 10000
# OBJECT_SIZE = 100 * 1024
python populate_bucket.py

# 2. Run comprehensive benchmark
# Edit get_batch_bench.py configuration:
# BUCKET_NAME = "benchmark-100KiB"
# TOTAL_OBJECTS = 10000
python get_batch_bench.py

# 3. Review results in console output and JSON file
```

--------------------------------

### Build and Run aisloader

Source: https://github.com/nvidia/aistore/blob/main/docs/howto_benchmark.md

Build the aisloader binary and run it to get started. Ensure you are in the aistore root directory.

```bash
$ make aisloader
$ $GOPATH/bin/aisloader
```

--------------------------------

### Start Maintenance Example

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/cluster.md

Demonstrates putting a target node into maintenance mode and observing the status change in the cluster view.

```bash
$ ais cluster add-remove-nodes start-maintenance 147665t8084
```

--------------------------------

### Setup: Create AIStore Client and Bucket

Source: https://github.com/nvidia/aistore/blob/main/python/examples/sdk/multipart-upload.ipynb

Initializes the AIStore client and creates a bucket for multipart upload examples. Ensure the AIStore cluster is running at the specified URL.

```python
from aistore import Client

# Connect to AIStore cluster
ais_url = "http://localhost:8080"
client = Client(ais_url)

# Create or get a bucket for our multipart upload examples
bucket = client.bucket("multipart-demo-bck").create(exist_ok=True)
print(f"Using bucket: {bucket.name}")

```

--------------------------------

### SIE#50 Error Example

Source: https://github.com/nvidia/aistore/blob/main/docs/troubleshooting.md

Example of a target failing to start due to SIE#50, indicating a lost or mismatched mountpath.

```text
storage integrity error sie#50:
lost or missing mountpath "<path>"
```

--------------------------------

### AuthN Server Configuration Example

Source: https://github.com/nvidia/aistore/blob/main/docs/authn.md

Example JSON configuration for the AuthN server. Ensure this file exists in the configuration directory when starting the server.

```json
{
    "log": {
        "dir": "/tmp/ais/authn/log",
        "level": "3"
    },
    "net": {
        "http": {
            "port": 52001,
            "use_https": false,
            "server_crt": "",
            "server_key": ""
        }
    },
    "auth": {
        "expiration_time": "24h"
    },
    "timeout": {
        "default_timeout": "30s"
    }
}
```

--------------------------------

### Install xmeta using go install

Source: https://github.com/nvidia/aistore/blob/main/cmd/README.md

Installs the xmeta utility by navigating to its directory and running 'go install'.

```bash
cd cmd/xmeta
go install
```

--------------------------------

### Example AIStore operations

Source: https://github.com/nvidia/aistore/blob/main/docs/distributed-tracing.md

Demonstrates basic AIStore operations like creating a bucket, putting a file, and getting a file. These operations can be traced once tracing is enabled.

```sh
ais bucket create ais://nnn
ais put README.md ais://nnn
ais get ais://nnn/README.md /dev/null
```

--------------------------------

### Install Dependencies & Setup Registry

Source: https://github.com/nvidia/aistore/blob/main/scripts/gitlab/runners/README.md

Run this script to install Docker, GitLab Runner, and set up a local pull-through registry cache. It accepts optional arguments for data root, registry username, and password.

```bash
sudo ./setup.sh [--data-root <absolute_path>] [--registry-username <username>] [--registry-password <password]
```

--------------------------------

### Install aisloader using go install

Source: https://github.com/nvidia/aistore/blob/main/cmd/README.md

Installs the aisloader binary using the 'go install' command.

```bash
go install github.com/NVIDIA/aistore/cmd/aisloader@latest
```

--------------------------------

### Install Go and Set Up Environment Variables

Source: https://github.com/nvidia/aistore/blob/main/python/examples/google_colab/aistore_deployment.ipynb

Installs Go version 1.23.1 and configures the GOPATH and PATH environment variables for AIStore deployment. Verifies the Go installation.

```python
# Install Go (v1.23.1)
!wget https://go.dev/dl/go1.23.1.linux-amd64.tar.gz
!sudo tar -C /usr/local -xzf go1.23.1.linux-amd64.tar.gz
!mkdir go

# Add Go to environment variables
import os
os.environ['GOPATH'] = '/content/go'
os.environ['PATH'] += ':/usr/local/go/bin:/content/go/bin'

# Check Go version to ensure installation
!go version
```

--------------------------------

### List, Get, and Evict Object from Remote Bucket

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/bucket.md

This example demonstrates a complete workflow: listing objects in a remote bucket, retrieving a specific object, viewing bucket properties, evicting the bucket, and verifying the eviction by listing objects again.

```bash
$ ais ls gs://wrQkliptRt
NAME             SIZE            
TDXBNBEZNl.tar   8.50KiB         
qFpwOOifUe.tar   8.50KiB         
thmdpZXetG.tar   8.50KiB         
```

```bash
$ ais get gcp://wrQkliptRt/qFpwOOifUe.tar /tmp/qFpwOOifUe.tar
GET "qFpwOOifUe.tar" from bucket "gcp://wrQkliptRt" as "/tmp/qFpwOOifUe.tar" [8.50KiB]
```

```bash
$ ais ls gs://wrQkliptRt --props all
NAME             SIZE            CHECKSUM                                ATIME                   VERSION                 CACHED  STATUS  COPIES
TDXBNBEZNl.tar   8.50KiB         33345a69bade096a30abd42058da4537                                1622133976984266        no      ok      0
qFpwOOifUe.tar   8.50KiB         47dd59e41f6b7723                        28 May 21 12:02 PDT     1622133846120151        yes     ok      1
thmdpZXetG.tar   8.50KiB         cfe0c386e91daa1571d6a659f49b1408                                1622137609269706        no      ok      0
```

```bash
$ ais bucket evict gcp://wrQkliptRt
"gcp://wrQkliptRt" bucket evicted
```

```bash
$ ais ls gs://wrQkliptRt --props all
NAME             SIZE            CHECKSUM                                ATIME   VERSION                 CACHED  STATUS  COPIES
TDXBNBEZNl.tar   8.50KiB         33345a69bade096a30abd42058da4537                1622133976984266        no      ok      0
qFpwOOifUe.tar   8.50KiB         8b5919c0850a07d931c3c46ed9101eab                1622133846120151        no      ok      0
thmdpZXetG.tar   8.50KiB         cfe0c386e91daa1571d6a659f49b1408                1622137609269706        no      ok      0
```

--------------------------------

### Example Template for CLI Help

Source: https://github.com/nvidia/aistore/blob/main/cmd/cli/cli/help_style_guide.md

Defines a template for constructing command usage strings with examples. It specifies the format for example headers, commands, descriptions, and multi-line support.

```go
const cmdUsage = "Brief description.\n" +
    indent1 + "e.g.:\n" +
    indent1 + "\t- 'ais <command> <arg1> <arg2>'\t- short description;\n" +
    indent1 + "\t- 'ais <command> <arg1> --flag'\t- another description;"
```

--------------------------------

### Implicit Bucket Creation Examples

Source: https://github.com/nvidia/aistore/blob/main/docs/bucket.md

Demonstrates implicit bucket creation (lazy discovery) by accessing buckets using 'ais ls' and 'ais get' commands.

```bash
ais ls s3://images --all
ais get s3://logs/foo.txt
```

--------------------------------

### Install ishard using go install

Source: https://github.com/nvidia/aistore/blob/main/cmd/README.md

Installs the ishard utility using the 'go install' command.

```bash
go install github.com/NVIDIA/aistore/cmd/ishard@latest
```

--------------------------------

### Example: Show Mountpaths for a Target

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/storage.md

An example of the output when showing mountpaths for a specific target ID.

```console
$ ais show storage mountpath t[TqPtghbiRw]

TqPtghbiRw
        Used Capacity (all disks): avg 15% max 18%
                                                /ais/mp1/2 /dev/nvme0n1(xfs)
                                                /ais/mp2/2 /dev/nvme1n1(xfs)
                                                /ais/mp3/2 /dev/nvme2n1(xfs)
                                                /ais/mp4/2 /dev/nvme3n1(xfs)
```

--------------------------------

### Install Benchmark Dependencies

Source: https://github.com/nvidia/aistore/blob/main/python/tests/perf/nbi/README.md

Install the required Python packages, aistore and boto3, for running the benchmark.

```bash
pip install aistore boto3
```

--------------------------------

### Install AIStore SDK

Source: https://github.com/nvidia/aistore/blob/main/python/examples/sdk/accessing-objects.ipynb

Install the AIStore SDK using pip. This is a prerequisite for using the SDK.

```python
! pip install aistore
```

--------------------------------

### Install and Show AIStore SDK

Source: https://github.com/nvidia/aistore/blob/main/python/examples/sdk/ais-batch-requests.ipynb

Installs the AIStore SDK and displays its name and version. This is a prerequisite for using the SDK.

```python
! pip show aistore | grep -E "Name|Version"
```

--------------------------------

### PyAISLoader GET Benchmark Help

Source: https://github.com/nvidia/aistore/blob/main/python/pyaisloader/README.md

To get detailed information on options specific to the GET benchmark, run this command.

```shell
pyaisloader GET --help
```

--------------------------------

### Install xmeta using Makefile

Source: https://github.com/nvidia/aistore/blob/main/cmd/README.md

Builds and installs the xmeta utility using the project's Makefile.

```bash
make xmeta
```

--------------------------------

### Install Kaggle API

Source: https://github.com/nvidia/aistore/blob/main/python/examples/sdk/writing-webdataset.ipynb

Installs the Kaggle API client, which is required for downloading datasets from Kaggle.

```bash
pip install kaggle
```

--------------------------------

### Install aisloader using Makefile

Source: https://github.com/nvidia/aistore/blob/main/cmd/README.md

Builds and installs the aisloader binary using the project's Makefile.

```bash
make aisloader
```

--------------------------------

### Install Grafana on Debian/Ubuntu

Source: https://github.com/nvidia/aistore/blob/main/docs/monitoring-grafana.md

Install Grafana OSS on Debian or Ubuntu systems using the official APT repository. Ensure to start the Grafana server service after installation.

```bash
# Debian/Ubuntu
sudo apt-get install -y apt-transport-https software-properties-common
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo apt-get update
sudo apt-get install grafana

# Start Grafana
sudo systemctl start grafana-server
```

--------------------------------

### Start Grafana Container

Source: https://github.com/nvidia/aistore/blob/main/bench/tools/aisloader-composer/README.md

Starts the Grafana container for visualizing metrics. Ensure docker is installed and accessible on the target host.

```bash
./start_grafana.sh
```

--------------------------------

### Display Help for Install Script

Source: https://github.com/nvidia/aistore/blob/main/cmd/cli/README.md

Shows all available options and usage instructions for the `install_from_binaries.sh` script.

```console
./scripts/install_from_binaries.sh --help
```

--------------------------------

### Install AIStore SDK

Source: https://github.com/nvidia/aistore/blob/main/python/examples/sdk/bucket-management.ipynb

Install the AIStore Python SDK using pip. This is a prerequisite for using the SDK.

```python
from aistore import Client

ais_url = "http://localhost:8080"
client = Client(ais_url)

# Default provider is AIS when instantiating a bucket object
bucket = client.bucket("first-bck")
# You can also set the backend provider and namespace
# client.bucket("my-aws-bck", provider=ProviderAmazon, namespace="my-namespace")
```

--------------------------------

### Create Bucket, Put, List, and Get Objects

Source: https://github.com/nvidia/aistore/blob/main/docs/getting_started.md

Demonstrates how to create a bucket, upload an object, list objects in the bucket, and download an object using the AIStore CLI.

```console
# Create a new bucket
$ ais create ais://mybucket

# Put an object using CLI
$ echo "Hello AIStore" > hello.txt
$ ais put hello.txt ais://mybucket

# List objects in the bucket
$ ais ls ais://mybucket

# Get the object
$ ais get ais://mybucket/hello.txt downloaded.txt
$ cat downloaded.txt
```

--------------------------------

### Start Netdata Container

Source: https://github.com/nvidia/aistore/blob/main/bench/tools/aisloader-composer/README.md

Starts the Netdata container for collecting and displaying system statistics. Ensure docker is installed and accessible on the target host.

```bash
./start_netdata.sh
```

--------------------------------

### Install AIS CLI from GitHub Releases

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/help.md

Installs the AIS CLI and aisloader to a specified directory. The default destination is /usr/local/bin, but this example installs to /tmp/www. Autocompletions for bash and zsh are also downloaded and configured.

```bash
$ scripts/install_from_binaries.sh --dstdir /tmp/www
Installing aisloader => /tmp/www/aisloader
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 9847k  100 9847k    0     0  3553k      0  0:00:02  0:00:02 --:--:-- 4301k
aisloader
Installing CLI => /tmp/www/ais
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  9.9M  100  9.9M    0     0  4436k      0  0:00:02  0:00:02 --:--:-- 5901k
ais
Downloading CLI autocompletions (bash & zsh)...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2350  100  2350    0     0   7000      0 --:--:-- --:--:-- --:--:--  6994
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   578  100   578    0     0   1673      0 --:--:-- --:--:-- --:--:--  1675
NOTE:
   *** CLI autocompletions are now copied to /etc/bash_completion.d/ais ***
   *** To enable, simply run: source /etc/bash_completion.d/ais         ***

Done.

$ ls /tmp/www
ais  aisloader
```

--------------------------------

### Conventional API GET Example with curl

Source: https://github.com/nvidia/aistore/blob/main/docs/http_api.md

Shows the equivalent GET operation using the conventional AIS RESTful API, contrasting with the 'Easy URL' format.

```console
$ curl -s -L -X GET 'http://aistore/v1/objects/my-google-bucket/abc-train-0001.tar?provider=gs' -o abc-train-0001.tar
```

--------------------------------

### Build xmeta using go install (default)

Source: https://github.com/nvidia/aistore/blob/main/cmd/xmeta/README.md

Builds and installs the xmeta binary directly using the 'go install' command from the repository root or the cmd/xmeta directory. This method typically preserves full debug information, resulting in a larger binary.

```console
# from the repository root
go install ./cmd/xmeta

# or, from inside cmd/xmeta
go install .
```

--------------------------------

### Install Docker CE

Source: https://github.com/nvidia/aistore/blob/main/docs/docker_main.md

Installs the Docker Community Edition. Note that for Ubuntu 16.04 and up, docker-ce might not be in the default repository and requires following a specific guide.

```bash
$ sudo apt-get install docker-ce
```

--------------------------------

### Install AIStore with ETL support

Source: https://github.com/nvidia/aistore/blob/main/python/aistore/sdk/etl/webserver/README.md

Install the AIStore package with the necessary extras for ETL functionality using pip.

```bash
pip install aistore[etl]
```

--------------------------------

### Create Bucket and Add Object

Source: https://github.com/nvidia/aistore/blob/main/docs/authn.md

Demonstrates the basic workflow of creating a bucket and uploading an object using the AIStore CLI.

```sh
ais bucket create ais://nnn
"ais://nnn" created
ais put README.md ais://nnn
PUT "README.md" => ais://nnn/README.md
```

--------------------------------

### Batch Get With Object Names

Source: https://github.com/nvidia/aistore/blob/main/python/aistore/sdk/batch/README.md

Retrieve multiple objects by providing a list of their names (strings). This is a minimal example demonstrating the basic usage of the batch get functionality.

```APIDOC
## Batch Get With Object Names (`str`)

### Description
If you just have a list of object names (strings), you can batch get them easily.

### Method Signature
`client.batch(object_names: list[str], bucket: Bucket)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from aistore.sdk import Client

client = Client("http://localhost:8080")
bucket = client.bucket("example")

# Provide a list of object names as strings
object_names = ["obj1", "obj2", "dir/file3.jpg"]
batch = client.batch(object_names, bucket=bucket)

for obj_info, data in batch.get():
    print(f"Object: {obj_info.obj_name}, Size: {len(data)}")
```

### Response
#### Success Response (200)
Returns an iterator yielding tuples of `ObjectInfo` and the object's data.
```

--------------------------------

### Easy URL GET Example with curl

Source: https://github.com/nvidia/aistore/blob/main/docs/http_api.md

Demonstrates how to perform a GET request using the 'Easy URL' format with curl. This is an alternative to the conventional AIS RESTful API.

```console
$ curl -s -L -X GET 'http://aistore/gs/my-google-bucket/abc-train-0001.tar' -o abc-train-0001.tar
```

--------------------------------

### Quick Start with AWS CLI

Source: https://github.com/nvidia/aistore/blob/main/docs/s3compat.md

Demonstrates basic S3 operations like creating a bucket, copying a file, and listing bucket contents using the AWS CLI with a specified endpoint.

```bash
AWS_EP=http://localhost:8080/s3
aws --endpoint-url "$AWS_EP" s3 mb s3://demo
aws --endpoint-url "$AWS_EP" s3 cp README.md s3://demo/
aws --endpoint-url "$AWS_EP" s3 ls s3://demo/
```

--------------------------------

### Display Help for 'ais archive get'

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/archive.md

Shows the help message for the 'ais archive get' command, detailing its usage, options, and examples for retrieving and extracting archived content.

```console
$ ais archive get --help
```

--------------------------------

### Start dSort Job with Inline YAML Specification

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/dsort.md

Starts a dSort job using an inline YAML specification. This example shows shuffling shards from 0 to 9 with a defined algorithm kind.

```console
$ ais start dsort -f - <<EOM
  extension: .tar
  input_bck:
      name: dsort-testing
  input_format:
      template: shard-{0..9}
  output_format: new-shard-{0000..1000}
  output_shard_size: 10KB
  description: shuffle shards from 0 to 9
  algorithm:
      kind: shuffle
  EOM
```

--------------------------------

### Show Make Options

Source: https://github.com/nvidia/aistore/blob/main/docs/development.md

Displays available options and usage examples for the `make` command.

```console
$ make help
```

--------------------------------

### Build and Run aisloader for Benchmarking

Source: https://github.com/nvidia/aistore/blob/main/docs/getting_started.md

Builds the aisloader tool and then runs it to benchmark the AIStore cluster for 2 minutes with specific configurations.

```bash
make aisloader
```

```bash
aisloader -bucket=ais://abc -duration 2m -numworkers=8 -minsize=1K -maxsize=1K -pctput=100 --cleanup=false
```

--------------------------------

### Stress Test AIS Downloads

Source: https://github.com/nvidia/aistore/blob/main/docs/docker_main.md

Performs GET requests on files previously downloaded into AIS. Requires the 'jq' command to be installed.

```bash
./playground/stress_get.sh imagenet # do gets on saved files (requires jq command)
```

--------------------------------

### AIS Operation Logs Example

Source: https://github.com/nvidia/aistore/blob/main/docs/monitoring-logs.md

Operation logs detail the completion of specific AIS actions like list, put, or get operations.

```log
I 21:00:43.063816 base:211 x-list[ApJcaebM5]-ais://yodas-21:00:03.062994-00:00:00.000000 finished
I 21:00:44.482430 base:211 x-list[J1qpgaWbxG]-ais://yodas-21:00:04.481894-00:00:00.000000 finished
```

--------------------------------

### Quick Start with s3cmd

Source: https://github.com/nvidia/aistore/blob/main/docs/s3compat.md

Shows how to upload a file to an S3-compatible bucket using s3cmd with explicit host and port configuration, disabling SSL.

```bash
s3cmd put README.md s3://demo \
  --no-ssl \
  --host=localhost:8080/s3 \
  --host-bucket="localhost:8080/s3/%(bucket)"
```

--------------------------------

### Object Operations

Source: https://github.com/nvidia/aistore/blob/main/docs/http_api.md

Examples for performing basic operations on objects, such as listing, getting, putting, reading ranges, deleting, and checking object properties.

```APIDOC
## GET /v1/objects/{bucket}/{object_name}

### Description
Lists a given object, potentially from a specific backend provider.

### Method
GET

### Endpoint
`/v1/objects/{bucket}/{object_name}`

### Query Parameters
- **provider** (string) - Optional - Specifies the backend provider (e.g., aws, gcp).
- **archpath** (string) - Optional - Path within an archive file.
- **check_cached** (boolean) - Optional - If true, checks if the object is cached.

### Request Example
```console
$ curl -s -L -X GET 'http://G/v1/objects/myS3bucket/myobject?provider=aws'
```

## GET /v1/objects/{bucket}/{object_name}

### Description
Retrieves an archived file from a remote tar archive.

### Method
GET

### Endpoint
`/v1/objects/{bucket}/{object_name}`

### Query Parameters
- **provider** (string) - Required - Specifies the backend provider (e.g., gcp).
- **archpath** (string) - Required - The path to the file within the archive.

### Request Example
```console
$ curl -s -L -X GET 'http://localhost:8080/v1/objects/myGCPbucket/train-1234.tar?provider=gcp&archpath=567.jpg' --output /tmp/567.jpg
```

## POST /v1/buckets/{bucket_name}

### Description
Creates a new bucket.

### Method
POST

### Endpoint
`/v1/buckets/{bucket_name}`

### Request Body
- **action** (string) - Required - Must be "create-bck".

### Request Example
```console
$ curl -i -X POST -H 'Content-Type: application/json' -d '{"action": "create-bck"}' 'http://G/v1/buckets/abc'
```

## DELETE /v1/buckets/{bucket_name}

### Description
Destroys an existing bucket.

### Method
DELETE

### Endpoint
`/v1/buckets/{bucket_name}`

### Request Body
- **action** (string) - Required - Must be "destroy-bck".

### Request Example
```console
$ curl -i -X DELETE -H 'Content-Type: application/json' -d '{"action": "destroy-bck"}' 'http://G/v1/buckets/abc'
```

## PUT /v1/objects/{bucket_name}/{object_name}

### Description
Uploads an object to a bucket.

### Method
PUT

### Endpoint
`/v1/objects/{bucket_name}/{object_name}`

### Request Example
```console
$ curl -s -L -X PUT 'http://G/v1/objects/myS3bucket/myobject' -T filenameToUpload
```

## GET /v1/objects/{bucket_name}/{object_name}

### Description
Retrieves an object from a bucket.

### Method
GET

### Endpoint
`/v1/objects/{bucket_name}/{object_name}`

### Query Parameters
- **provider** (string) - Optional - Specifies the backend provider (e.g., s3).

### Request Example
```console
$ curl -s -L -X GET 'http://G/v1/objects/myS3bucket/myobject?provider=s3' -o myobject
```

## GET /v1/objects/{bucket_name}/{object_name}

### Description
Reads a specific range of bytes from an object.

### Method
GET

### Endpoint
`/v1/objects/{bucket_name}/{object_name}`

### Headers
- **Range** (string) - Required - Specifies the byte range (e.g., `bytes=1024-1535`).

### Query Parameters
- **provider** (string) - Optional - Specifies the backend provider (e.g., s3).

### Request Example
```console
$ curl -s -L -X GET -H 'Range: bytes=1024-1535' 'http://G/v1/objects/myS3bucket/myobject?provider=s3' -o myobject
```

## DELETE /v1/objects/{bucket_name}/{object_name}

### Description
Deletes an object from a bucket.

### Method
DELETE

### Endpoint
`/v1/objects/{bucket_name}/{object_name}`

### Request Example
```console
$ curl -i -X DELETE -L 'http://G/v1/objects/mybucket/myobject'
```

## HEAD /v1/objects/{bucket_name}/{object_name}

### Description
Retrieves the properties (metadata) of an object.

### Method
HEAD

### Endpoint
`/v1/objects/{bucket_name}/{object_name}`

### Request Example
```console
$ curl -s -L --head 'http://G/v1/objects/mybucket/myobject'
```

## HEAD /v1/objects/{bucket_name}/{object_name}

### Description
Checks if a remote object is cached locally.

### Method
HEAD

### Endpoint
`/v1/objects/{bucket_name}/{object_name}`

### Query Parameters
- **check_cached** (boolean) - Required - Set to `true` to check cache status.

### Request Example
```console
$ curl -s -L --head 'http://G/v1/objects/mybucket/myobject?check_cached=true'
```

## HEAD /v1/buckets/{bucket_name}

### Description
Retrieves properties (metadata) of a bucket.

### Method
HEAD

### Endpoint
`/v1/buckets/{bucket_name}`

### Request Example
```console
$ curl -s -L --head 'http://G/v1/buckets/mybucket'
```
```

--------------------------------

### Create Go ETL Webserver

Source: https://github.com/nvidia/aistore/blob/main/ext/etl/webserver/README.md

Defines an EchoServer that implements the Transform method to return input data as is. This is a basic example for starting with the framework.

```go
import 	"github.com/NVIDIA/aistore/ext/etl/webserver"

type EchoServer struct {
	webserver.ETLServer
}

func (*EchoServer) Transform(input io.ReadCloser, _, _ string) (io.ReadCloser, error) {
	data, err := io.ReadAll(input)
	if err != nil {
		return nil, err
	}
	input.Close()
	return io.NopCloser(bytes.NewReader(data)), nil
}

func main() {
	svr := &EchoServer{}
	if err := webserver.Run(svr, "0.0.0.0", 8080); err != nil {
		log.Fatalf("Failed to start server: %v", err)
	}
}
```

--------------------------------

### Get Archived File from Remote Tar

Source: https://github.com/nvidia/aistore/blob/main/docs/http_api.md

Retrieves a specific file from a remote tar archive. This example targets a GCP bucket and specifies the archive path.

```console
# Get archived file from a remote tar
$ curl -s -L -X GET 'http://localhost:8080/v1/objects/myGCPbucket/train-1234.tar?provider=gcp&archpath=567.jpg' --output /tmp/567.jpg
```

--------------------------------

### Initialize AIStore Client and Create Bucket

Source: https://github.com/nvidia/aistore/blob/main/python/examples/sdk/sdk-etl-tutorial.ipynb

Sets up necessary constants, initializes the AIStore client, and creates a bucket. Note that AIS-ETLs require Kubernetes.

```python
from aistore import Client
from aistore.sdk.etl_templates import MD5
from aistore.sdk.etl.webserver.fastapi_server import FastAPIServer
import hashlib
from itertools import cycle

BUCKET_NAME = "bucket-demo"
SPEC_ETL_NAME = "etl-spec-demo"
CODE_ETL_NAME = "etl-code-demo"

# Note: AIS-ETLs require Kubernetes.
client = Client("http://192.168.49.2:8080")
client.bucket(bck_name=BUCKET_NAME).create(exist_ok=True)
```

--------------------------------

### Run Dsort Example

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/dsort.md

This example demonstrates how to run the dsort command with a specification file. Ensure the source and destination buckets exist and the source bucket has at least 10 shards matching the input format. The `--verbose` flag provides detailed output.

```console
$ ais start dsort ais://src ais://dst -f ais/test/scripts/dsort-ex1-spec.json --verbose
PROPERTY                         VALUE
algorithm.content_key_type       -
algorithm.decreasing             false
algorithm.extension              -
algorithm.kind                   alphanumeric
algorithm.seed                   -
create_concurrency_max_limit     0
description                      sort shards alphanumerically
dry_run                          false
dsorter_type                     -
extension                        .tar
extract_concurrency_max_limit    0
input_bck                        ais://src
input_format.objnames            -
input_format.template            shard-{0..9}
max_mem_usage                    -
ekm_file                       -
ekm_file_sep                   	
output_bck                       ais://dst
output_format                    new-shard-{0000..1000}
output_shard_size                10KB

Config override:                 none

srt-M8ld-VU_i
```

--------------------------------

### Basic Endpoint Annotation

Source: https://github.com/nvidia/aistore/blob/main/docs/_posts/2025-08-29-automated-api-documentation-generation-with-gendocs.md

Document a standard GET endpoint with parameters and provide an example using curl. This annotation defines the HTTP method, path, and parameters.

```go
// +gen:endpoint GET /v1/buckets/{bucket-name}/objects/{object-name} [provider=string]
// Retrieves an object from the specified bucket.
// Supports streaming for large objects and conditional requests.
func GetObject(w http.ResponseWriter, r *http.Request) {
    // implementation...
}
```

--------------------------------

### Build AuthN Server

Source: https://github.com/nvidia/aistore/blob/main/docs/authn.md

Build the AuthN server binary. This is the first step in setting up the AuthN server.

```shell
make authn
```

--------------------------------

### Prepare Dataset and Upload to AIStore

Source: https://github.com/nvidia/aistore/blob/main/docs/tutorials/etl/etl_imagenet_pytorch.md

This snippet shows how to inspect a tarball, create an AIStore bucket, and upload the dataset.

```console
$ tar -tvf raw-train.tar | head -n 5
-rw-r--r--  0 aditya86 users   27024 Jul  4  2012 n02085620_10074.JPEG
-rw-r--r--  0 aditya86 users   34446 Jul  4  2012 n02085620_10131.JPEG
-rw-r--r--  0 aditya86 users   12891 Jul  4  2012 n02085620_10621.JPEG
-rw-r--r--  0 aditya86 users   34837 Jul  4  2012 n02085620_1073.JPEG
-rw-r--r--  0 aditya86 users   18126 Jul  4  2012 n02085620_10976.JPEG
$ ais create ais://imagenet
"ais://imagenet" bucket created
$ ais put raw-train.tar imagenet
PUT "raw-train.tar" into bucket "imagenet"
```

--------------------------------

### Download Log Examples

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/log.md

Illustrates various ways to download logs using the 'ais log get' command, including specifying nodes, output paths, and filtering by severity.

```console
- 'ais log get NODE_ID /tmp'                    - download the current log from NODE_ID and save it in the specified directory;

```

```console
- 'ais log get NODE_ID /tmp/out' --refresh 10   - download the current log from NODE_ID as /tmp/out
                                                       and continuously update it every 10 seconds;

```

```console
- 'ais log get cluster /tmp'                    - download TAR.GZ archives from all cluster nodes
                                                       and save them in the specified directory ('get cluster' implies '--all');

```

```console
- 'ais log get NODE_ID --all'                   - download the complete log archive (TAR.GZ) from NODE_ID;

```

```console
- 'ais log get NODE_ID --all --severity error'  - download only errors and warnings from NODE_ID;

```

```console
- 'ais log get NODE_ID --all --severity e'      - same as above.

```

--------------------------------

### Initiate Cluster-Wide Rebalancing via REST API

Source: https://github.com/nvidia/aistore/blob/main/docs/configuration.md

This example shows how to manually start a cluster-wide rebalancing process using the AIS REST API. This is an alternative to enabling automatic rebalancing.

```bash
PUT {"action": "start", "value": {"kind": "rebalance"}} v1/cluster
```

--------------------------------

### Start AuthN Server

Source: https://github.com/nvidia/aistore/blob/main/docs/authn.md

Start the AuthN server using its configuration file. Ensure the configuration directory is correctly specified.

```shell
authn -config=<authn_config_dir>
```

--------------------------------

### Show download jobs matching a regex

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/download.md

Filters and displays download jobs whose descriptions match the provided regular expression. This example shows jobs with descriptions starting with 'downloads '.

```console
$ ais show job download --regex "^downloads (.*)"
JOB ID	 STATUS	 ERRORS	 DESCRIPTION
cudIYMAqg	 Finished	 0	 downloads whole imagenet bucket
fjwiIEMfa	 Finished	 0	 downloads range lpr-bucket from gcp://lpr-bucket
```

--------------------------------

### Show Help

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/bucket.md

Display help information for the CLI command.

```bash
--help
```

```bash
-h
```

--------------------------------

### Start dSort Job with Inline JSON Specification

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/dsort.md

Initiates a dSort job using an inline JSON specification. This example demonstrates packing records into categorized shards with specific input and output formats.

```console
$ ais start dsort '{ "extension": ".tar", "input_bck": {"name": "dsort-testing"}, "input_format": {"template": "shard-{0..9}"}, "output_shard_size": "200KB", "description": "pack records into categorized shards", "ekm_file": "http://website.web/static/ekm_file.txt", "ekm_file_sep": " " }'
```

--------------------------------

### URL Encode Unicode Object Names with Curl

Source: https://github.com/nvidia/aistore/blob/main/docs/unicode.md

When accessing objects with Unicode names programmatically using curl, the object name must be URL-encoded. This example demonstrates setting an environment variable with a Japanese string and then using it in a curl GET request.

```bash
$ export helloworld="こんにちは世界"
$ curl -L -X GET "http://ais-endpoint/v1/objects/onebucket/$helloworld"
```

--------------------------------

### Setup AIStore client and create a bucket

Source: https://github.com/nvidia/aistore/blob/main/python/examples/pytorch/dataset_example.ipynb

Initialize the AIStore client and create a bucket for storing data. Uses AIS_ENDPOINT environment variable or defaults to localhost:8080.

```python
ais_url = os.getenv("AIS_ENDPOINT", "http://localhost:8080")
client = Client(ais_url)
bucket = client.bucket("my-bck").create(exist_ok=True)
```

--------------------------------

### Accessing External Cloud Storage Buckets with Python SDK

Source: https://github.com/nvidia/aistore/blob/main/docs/_posts/2022-07-20-python-sdk.md

Demonstrates how to use the AIStore Python SDK to interact with buckets from external cloud providers (GCP in this example). It shows how to get a reader for a specific object and how to list and read objects with a given prefix.

```APIDOC
## Accessing External Cloud Storage Buckets

### Description
This section demonstrates how to use the AIStore Python SDK to read objects from external cloud storage buckets. The `Client.bucket()` method accepts a `provider` argument to specify the external cloud provider.

### Usage

#### Reading a specific object
```python
# Getting compressed TinyImageNet dataset from [gcp://tinyimagenet_compressed]
BUCKET_NAME = "tinyimagenet_compressed"
OBJECT_NAME = "tinyimagenet-compressed.zip"
reader = client.bucket(BUCKET_NAME, provider="gcp").object(OBJECT_NAME).get_reader()
```

#### Listing and reading objects from a bucket
```python
# Getting uncompressed TinyImageNet dataset from [gcp://tinyimagenet_uncompressed]
BUCKET_NAME = "tinyimagenet_uncompressed"
# List all objects and read them
objects = client.bucket(BUCKET_NAME, provider="gcp").list_objects().get_entries()
for obj_info in objects:
    client.bucket(BUCKET_NAME, provider="gcp").object(obj_info.name).get_reader()

# Getting only objects with prefix "validation/" from bucket [gcp://tinyimagenet_uncompressed]
BUCKET_NAME = "tinyimagenet_uncompressed"
validation_objects = client.bucket(BUCKET_NAME).list_objects(prefix="validation/").get_entries()
for obj_info in validation_objects:
    client.bucket(BUCKET_NAME).object(obj_info.name).get_reader()
```

### Parameters

#### `Client.bucket()`
- **bucket_name** (string) - Required - The name of the bucket.
- **provider** (string) - Optional - Specifies the external cloud provider (e.g., "gcp", "aws"). Defaults to "ais".

#### `Bucket.list_objects()`
- **prefix** (string) - Optional - Filters objects by a given prefix.
```

--------------------------------

### Install and Use cpupower for CPU Governor Settings

Source: https://github.com/nvidia/aistore/blob/main/docs/performance.md

Installs the 'cpupower' package and demonstrates its usage to set the CPU scaling governor to 'performance'. This is an alternative to directly writing to sysfs.

```bash
$ apt-get install -y linux-tools-$(uname -r) # install `cpupower`
$ cpupower frequency-info # check current settings
$ cpupower frequency-set -r -g performance # set `performance` setting to all CPU's
$ cpupower frequency-info # check settings after the change
```

--------------------------------

### Example: Entire Manifest as Single TAR

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/ml.md

This example shows how to process an entire Lhotse manifest (`manifest.jsonl.gz`) and package all its cuts into a single TAR archive named `output.tar`.

```console
ais ml lhotse-get-batch --cuts manifest.jsonl.gz output.tar
```

--------------------------------

### Install AIS CLI using go install

Source: https://github.com/nvidia/aistore/blob/main/cmd/README.md

Installs the AIS CLI using 'go install' and renames the binary to 'ais'. Ensure your $GOPATH/bin is in your PATH.

```bash
go install github.com/NVIDIA/aistore/cmd/cli@latest && mv $GOPATH/bin/cli $GOPATH/bin/ais
```

--------------------------------

### Build AIStore Node

Source: https://github.com/nvidia/aistore/blob/main/docs/docker_main.md

Install the 'aisnode' binary to build the AIStore application. Ensure Go is installed and GOPATH is set.

```bash
go install github.com/NVIDIA/aistore/cmd/aisnode@latest
```

--------------------------------

### PromQL for GET Latency

Source: https://github.com/nvidia/aistore/blob/main/docs/monitoring-grafana.md

Calculate the average GET latency in milliseconds. This query divides the total GET nanoseconds by the GET count, then converts to milliseconds over a 5-minute rate.

```promql
# GET latency (milliseconds) using PromQL
sum(rate(ais_target_get_ns_total[5m])) / sum(rate(ais_target_get_count[5m])) / 1000000
```

--------------------------------

### Verify Docker-Compose Installation

Source: https://github.com/nvidia/aistore/blob/main/docs/docker_main.md

Checks the installed version of Docker Compose to confirm successful installation.

```bash
$ docker-compose --version
```

--------------------------------

### Deploy Single Target with Loopback Devices

Source: https://github.com/nvidia/aistore/blob/main/docs/getting_started.md

Use the 'make' command to deploy a single AIStore target with two loopback devices, each 1GB in size. This command automates the setup process.

```console
$ make kill clean cli deploy <<< $'1\n1\n4\ny\ny\nn\n1G\n'
```

```console
$ TAGS=aws TEST_LOOPBACK_SIZE=1G make kill clean cli deploy <<< $'1\n1\n'
```

--------------------------------

### Average GET Latency (ms)

Source: https://github.com/nvidia/aistore/blob/main/docs/monitoring-prometheus.md

Calculates the average latency of GET operations in milliseconds by dividing total GET nanoseconds by the count of GET operations. Converts nanoseconds to milliseconds.

```promql
sum(rate(ais_target_get_ns_total[5m]))
/
sum(rate(ais_target_get_count[5m]))
/
1e6   # convert ns → ms
```

--------------------------------

### Install Docker

Source: https://github.com/nvidia/aistore/blob/main/bench/tools/aisloader-composer/README.md

Installs Docker on AIS target nodes. This script automates the Docker installation process.

```bash
./install_docker.sh
```

--------------------------------

### Explore AIStore CLI Help and Aliases

Source: https://github.com/nvidia/aistore/blob/main/docs/getting_started.md

Shows how to access the AIStore CLI help and view configured aliases.

```console
$ ais --help
$ ais alias
$ ais <TAB-TAB>
```

--------------------------------

### Build AIStore Binaries

Source: https://github.com/nvidia/aistore/blob/main/deploy/dev/slurm/README.md

Build the AIStore binaries with specified backend providers. Ensure the AIStore repository path is correctly set.

```bash
cd $AISTORE_REPO
AIS_BACKEND_PROVIDERS=aws make node cli
```

--------------------------------

### Install Docker-Compose with Pip

Source: https://github.com/nvidia/aistore/blob/main/docs/docker_main.md

Installs Docker Compose using the Python pip package manager. Requires pip to be installed first.

```bash
$ sudo apt-get install -y python-pip
$ sudo pip install docker-compose
```

--------------------------------

### Example: Batching with Output Template

Source: https://github.com/nvidia/aistore/blob/main/docs/cli/ml.md

This example illustrates how to generate multiple output archives from a single Lhotse manifest (`m.jsonl.lz4`). It uses `--batch-size 1000` to specify the number of cuts per archive and `--output-template "a-{001..999}.tar"` to name the archives sequentially.

```console
ais ml lhotse-get-batch --cuts m.jsonl.lz4 --batch-size 1000 --output-template "a-{001..999}.tar"
```

--------------------------------

### Install AIS CLI using Makefile

Source: https://github.com/nvidia/aistore/blob/main/cmd/README.md

Builds and installs the AIS CLI binary to your $GOPATH/bin. The installed binary is named 'ais'.

```bash
make cli
```

--------------------------------

### Install AIS CLI from Release Binaries

Source: https://github.com/nvidia/aistore/blob/main/cmd/cli/README.md

Installs the CLI binary to a specified destination directory and optionally installs command autocompletions.

```console
./scripts/install_from_binaries.sh --dstdir /tmp/www --completions
```

--------------------------------

### PyAISLoader LIST Benchmark Help

Source: https://github.com/nvidia/aistore/blob/main/python/pyaisloader/README.md

To get detailed information on options specific to the LIST benchmark, run this command.

```shell
pyaisloader LIST --help
```