### Build and Start S3Proxy with Java Builder API

Source: https://context7.com/gaul/s3proxy/llms.txt

Constructs an S3Proxy instance using the fluent builder API, configuring a local filesystem BlobStore and starting the server. Ensure the BlobStore is properly initialized before building S3Proxy.

```java
import java.net.URI;
import java.util.Properties;
import org.eclipse.jetty.util.component.AbstractLifeCycle;
import org.gaul.s3proxy.AuthenticationType;
import org.gaul.s3proxy.S3Proxy;
import org.jclouds.ContextBuilder;
import org.jclouds.blobstore.BlobStore;
import org.jclouds.blobstore.BlobStoreContext;

// Build a BlobStore backed by the local filesystem
Properties props = new Properties();
props.setProperty("jclouds.filesystem.basedir", "/tmp/blobstore");

BlobStoreContext context = ContextBuilder.newBuilder("filesystem-nio2")
        .credentials("identity", "credential")
        .overrides(props)
        .build(BlobStoreContext.class);
BlobStore blobStore = context.getBlobStore();

// Build and start S3Proxy
S3Proxy s3Proxy = S3Proxy.builder()
        .blobStore(blobStore)
        .endpoint(URI.create("http://127.0.0.1:8080"))
        .awsAuthentication(AuthenticationType.AWS_V2_OR_V4, "mykey", "mysecret")
        .virtualHost("localhost")             // enables bucket.localhost:8080 style requests
        .maxSinglePartObjectSize(5L << 30)    // 5 GB default
        .ignoreUnknownHeaders(false)
        .jettyMaxThreads(200)
        .build();

s3Proxy.start();
while (!s3Proxy.getState().equals(AbstractLifeCycle.STARTED)) {
    Thread.sleep(1);
}
System.out.println("S3Proxy listening on port: " + s3Proxy.getPort());
// Output: S3Proxy listening on port: 8080

// Stop cleanly
s3Proxy.stop();
context.close();
```

--------------------------------

### Example Regex Blobstore Rule

Source: https://github.com/gaul/s3proxy/wiki/Middleware-regex

An example demonstrating how to configure a regex rule to partition and reformat object keys. The match expression captures parts of the original key, and the replace expression reconstructs it with new formatting.

```properties
s3proxy.regex-blobstore.match.type_partition=^prefix/(\w+)/(\d{4})/(\d{2})/(\d{2})/(.*)$
s3proxy.regex-blobstore.replace.type_partition=prefix/date=$2-$3-$4/type=$1/$5
```

--------------------------------

### Start S3Proxy Service

Source: https://github.com/gaul/s3proxy/wiki/Middleware-storage-class-override

Launch the S3Proxy service using the specified properties file. Ensure the service is running before attempting to send requests.

```bash
./s3proxy --properties s3proxy.properties
```

--------------------------------

### Instantiate S3Proxy with Filesystem Backend

Source: https://github.com/gaul/s3proxy/wiki/Using-S3Proxy-in-Java-projects

Create an S3Proxy instance using the filesystem-nio2 backend and configure its properties and endpoint. Ensure the S3Proxy is started and its state is verified.

```java
Properties properties = new Properties();
properties.setProperty("jclouds.filesystem.basedir", "/tmp/blobstore");

BlobStoreContext context = ContextBuilder
        .newBuilder("filesystem-nio2")
        .credentials("identity", "credential")
        .overrides(properties)
        .build(BlobStoreContext.class);

S3Proxy s3Proxy = S3Proxy.builder()
        .blobStore(context.getBlobStore())
        .endpoint(URI.create("http://127.0.0.1:8080"))
        .build();

s3Proxy.start();
while (!s3Proxy.getState().equals(AbstractLifeCycle.STARTED)) {
    Thread.sleep(1);
}
```

--------------------------------

### Example: Scoped Data Prefix

Source: https://github.com/gaul/s3proxy/wiki/Middleware-prefix-blobstore

Example configuration to expose 'scoped-data/' objects from backend storage as if they were at the top of 'customer-bucket'. All operations on 'customer-bucket' will then operate under the 'scoped-data/' prefix on the backend.

```properties
s3proxy.prefix-blobstore.customer-bucket=scoped-data/
```

--------------------------------

### Create S3Proxy Filesystem Basedir

Source: https://github.com/gaul/s3proxy/blob/master/README.md

Command to create the directory for the local filesystem storage backend. This must be executed before starting S3Proxy.

```bash
mkdir /tmp/s3proxy
```

--------------------------------

### Starting S3Proxy with Multiple Backend Configuration Files

Source: https://context7.com/gaul/s3proxy/llms.txt

Launch S3Proxy with multiple properties files to route individual buckets to separate storage backends. Each file defines buckets using `s3proxy.bucket-locator.*`.

```bash
# Start S3Proxy with two backends
./s3proxy --properties backend-a.conf --properties backend-b.conf
```

--------------------------------

### Starting S3Proxy with Multiple Properties Files

Source: https://github.com/gaul/s3proxy/wiki/Middleware-bucket-locator

Launch S3Proxy by specifying multiple configuration files using the `--properties` flag. This is useful when defining complex backend configurations with aliased buckets.

```bash
s3proxy --properties file1.conf --properties file2.conf
```

--------------------------------

### Rackspace Cloud Files Configuration

Source: https://github.com/gaul/s3proxy/wiki/Storage-backend-examples

Example configuration for Rackspace Cloud Files, including a user-specified region.

```properties
s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.authorization=aws-v2-or-v4
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.provider=rackspace-cloudfiles-us
jclouds.identity=xxxxxxxxxxxxxx
jclouds.credential=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
jclouds.region=HKG
```

--------------------------------

### SoftLayer Configuration

Source: https://github.com/gaul/s3proxy/wiki/Storage-backend-examples

Example configuration for SoftLayer, which uses Keystone authentication. This example targets the Dallas geo.

```properties
s3proxy.endpoint=http://localhost:8080
s3proxy.authorization=aws-v2-or-v4
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.provider=openstack-swift
jclouds.endpoint=https://dal05.objectstorage.softlayer.net/auth/v1.0
jclouds.identity=test:tester
jclouds.credential=testing
jclouds.keystone.credential-type=tempAuthCredentials
```

--------------------------------

### Configure S3Proxy with Local Filesystem Backend

Source: https://github.com/gaul/s3proxy/blob/master/README.md

Example properties file for S3Proxy using the local filesystem as the storage backend with anonymous access. Ensure the basedir exists before running.

```properties
s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=filesystem
jclouds.filesystem.basedir=/tmp/s3proxy
```

--------------------------------

### Build and Run S3Proxy Standalone

Source: https://context7.com/gaul/s3proxy/llms.txt

Build the S3Proxy JAR, configure it with a properties file for the filesystem-nio2 backend, and run it. Includes example cURL commands for testing bucket creation, listing, object upload, and download.

```bash
# Build the binary
mvn package
chmod +x target/s3proxy

# Create storage directory for filesystem-nio2 backend
mkdir /tmp/s3proxy

# Create s3proxy.conf
cat > s3proxy.conf << 'EOF'
s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=filesystem-nio2
jclouds.filesystem.basedir=/tmp/s3proxy
EOF

# Run
./target/s3proxy --properties s3proxy.conf

# Test: create a bucket
curl --request PUT http://localhost:8080/testbucket

# Test: list all buckets
curl http://localhost:8080/
# Output:
# <?xml version="1.0" ?><ListAllMyBucketsResult ...><Buckets><Bucket>
# <Name>testbucket</Name><CreationDate>2025-01-01T00:00:00.000Z</CreationDate>
# </Bucket></Buckets></ListAllMyBucketsResult>

# Test: upload an object
curl -X PUT -T myfile.txt http://localhost:8080/testbucket/myfile.txt

# Test: download an object
curl http://localhost:8080/testbucket/myfile.txt
```

--------------------------------

### Example Key Transformation

Source: https://github.com/gaul/s3proxy/wiki/Middleware-regex

Illustrates the transformation of an object key using the provided regex rule. The original key is transformed into a new structure based on the match and replace expressions.

```text
Original key:
prefix/test/2023/01/01/test.txt

Transformed key:
prefix/date=2023-01-01/type=test/test.txt
```

--------------------------------

### Configure S3Proxy Bucket Assignment to Backends

Source: https://github.com/gaul/s3proxy/blob/master/README.md

Example properties for S3Proxy to assign specific buckets or use glob syntax for assigning multiple buckets to different backends.

```properties
s3proxy.bucket-locator.1=bucket
s3proxy.bucket-locator.2=another-bucket
```

--------------------------------

### Libcloud Configuration for S3Proxy

Source: https://github.com/gaul/s3proxy/wiki/Client-compatibility-list

Configure Libcloud to connect to S3Proxy. Get the S3 driver and instantiate it with credentials, host, port, and SSL disabled.

```python
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver
cls = get_driver(Provider.S3)
driver = cls('local-identity', 'local-credential', host='localhost', port=8080, secure=False)
```

--------------------------------

### Kubernetes Deployment with SSL Configuration

Source: https://github.com/gaul/s3proxy/wiki/SSL-support

Example Kubernetes Deployment for S3Proxy, mounting the keystore secret and referencing the configuration secret. This enables SSL for the S3Proxy service.

```yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: s3proxy
  namespace: default
  labels:
    app: s3proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: s3proxy
  template:
    metadata:
      labels:
        app: s3proxy
    spec:
      containers:
        - name: s3proxy
          image: gaul/s3proxy
          ports:
            - name: https
              containerPort: 443
          envFrom:
            - secretRef:
                name: s3proxy
          resources:
            requests:
              cpu: 1
              memory: "1Gi"
            limits:
              memory: "1Gi"
          volumeMounts:
          - name: keystore
            mountPath: /opt/s3proxy/tls
      volumes:
        - name: keystore
          secret:
            secretName: s3proxy-keystore
            items:
              - key: keystore.jks
                path: keystore.jks

```

--------------------------------

### Mock Large Object Upload with AWS CLI

Source: https://context7.com/gaul/s3proxy/llms.txt

Uses the AWS CLI and 'dd' command to simulate uploading a large file instantly. The 'null-blobstore' feature ensures the upload is immediate and GET requests return the correct byte count filled with NUL bytes.

```bash
# Upload a 10 GB file instantly (no actual data written)
dd if=/dev/urandom bs=1M count=10240 | \
  aws s3 --endpoint-url http://localhost:8081 cp - s3://test/largefile.bin
# Immediate 200 OK; GET returns 10 GB of NUL bytes
```

--------------------------------

### Dynamic Multi-Backend Bucket Routing with BlobStoreLocator

Source: https://context7.com/gaul/s3proxy/llms.txt

Installs a custom BlobStoreLocator to route different buckets to distinct backend BlobStore instances. This enables dynamic multi-backend bucket routing based on container names.

```java
import org.gaul.s3proxy.BlobStoreLocator;
import org.gaul.s3proxy.S3Proxy;
import org.jclouds.blobstore.BlobStore;
import java.util.Map;

// Create two separate blobstore backends
BlobStore backendA = /* filesystem-nio2 blobstore for /tmp/a */ null;
BlobStore backendB = /* transient-nio2 blobstore */ null;

S3Proxy proxy = S3Proxy.builder()
        .blobStore(backendA)  // default backend
        .endpoint(URI.create("http://0.0.0.0:8080"))
        .build();

// Route "archive-*" buckets to backendB, everything else to backendA
proxy.setBlobStoreLocator((identity, container, blob) -> {
    if (container != null && container.startsWith("archive-")) {
        return Map.entry("archive-cred", backendB);
    }
    return Map.entry("default-cred", backendA);
});

proxy.start();
// PUT /archive-2024/data.csv  → routed to backendB
// PUT /production/data.csv    → routed to backendA
```

--------------------------------

### JUnit 5 Extension for S3Proxy

Source: https://context7.com/gaul/s3proxy/llms.txt

Integrates S3Proxy into the JUnit 5 lifecycle. The proxy starts before each test method and stops with cleanup afterwards. The filesystem backend is used by default, writing to a temporary directory.

```java
import org.gaul.s3proxy.AuthenticationType;
import org.gaul.s3proxy.junit.S3ProxyExtension;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
import java.net.URI;
import java.util.List;

import static org.assertj.core.api.Assertions.assertThat;

class MyS3Test {

    @RegisterExtension
    static S3ProxyExtension s3Proxy = S3ProxyExtension.builder()
            .withCredentials(AuthenticationType.AWS_V2_OR_V4, "access", "secret")
            .withBlobStoreProvider("transient-nio2")
            .build();

    @Test
    void uploadAndListObjects() {
        S3Client client = S3Client.builder()
                .endpointOverride(URI.create(s3Proxy.getUri().toString()))
                .credentialsProvider(StaticCredentialsProvider.create(
                        AwsBasicCredentials.create(
                                s3Proxy.getAccessKey(), s3Proxy.getSecretKey())))
                .region(Region.US_EAST_1)
                .forcePathStyle(true)
                .build();

        client.createBucket(CreateBucketRequest.builder().bucket("test").build());
        client.putObject(PutObjectRequest.builder().bucket("test").key("hello.txt").build(),
                software.amazon.awssdk.core.sync.RequestBody.fromString("Hello World"));

        ListObjectsV2Response resp = client.listObjectsV2(
                ListObjectsV2Request.builder().bucket("test").build());
        List<S3Object> objects = resp.contents();

        assertThat(objects).hasSize(1);
        assertThat(objects.get(0).key()).isEqualTo("hello.txt");
        assertThat(objects.get(0).size()).isEqualTo(11L);
        client.close();
    }
}
```

--------------------------------

### S3Proxy Mocking Configuration Properties

Source: https://github.com/gaul/s3proxy/wiki/Middleware-large-object-mocking

Configure S3Proxy to use a null blobstore for mocking large objects. This setup is useful for testing S3 interactions without actual storage.

```properties
s3proxy.endpoint=http://localhost:8081
s3proxy.authorization=aws-v2
s3proxy.identity=local-identity
s3proxy.credential=local-credential
s3proxy.null-blobstore=true

jclouds.provider=transient
jclouds.identity=remote-identity
jclouds.credential=remote-credential
```

--------------------------------

### Demonstrate No-Cache Behavior with Curl

Source: https://context7.com/gaul/s3proxy/llms.txt

Compares the behavior of conditional GET requests with and without the 'no-cache-blobstore' middleware enabled. The first curl command shows a potential 304 Not Modified response, while the second, with the middleware active, returns a 200 OK.

```bash
# Without no-cache: server may return 304 if ETag matches
curl -H 'If-None-Match: "abc123"' http://127.0.0.1:8080/bucket/file.txt
# HTTP/1.1 304 Not Modified

# With no-cache: 304 is suppressed, full object is returned
curl -H 'If-None-Match: "abc123"' http://127.0.0.1:8080/bucket/file.txt
# HTTP/1.1 200 OK
# <full file content>
```

--------------------------------

### Override Speed for Specific Operation

Source: https://github.com/gaul/s3proxy/wiki/Middleware-latency

Set a specific speed limit for a particular S3 operation, overriding the global setting. This example sets a lower speed for the 'get' operation.

```properties
s3proxy.latency-blobstore.get.speed=10 # use 10 kb/s for get blob instead
```

--------------------------------

### Run S3Proxy with Docker

Source: https://context7.com/gaul/s3proxy/llms.txt

Launches an S3Proxy instance using Docker with specified logging and S3Proxy configurations. Authorization is set to 'none' for easy testing.

```bash
docker run -e LOG_LEVEL=debug -e LOG_APPENDER=CONTAINER \
  -e S3PROXY_AUTHORIZATION=none \
  -e S3PROXY_ENDPOINT=http://0.0.0.0:80 \
  -e JCLOUDS_PROVIDER=transient-nio2 \
  -p 8080:80 andrewgaul/s3proxy
```

--------------------------------

### Disable Conditional GETs with Properties

Source: https://context7.com/gaul/s3proxy/llms.txt

Enables the 'no-cache-blobstore' feature in S3Proxy to drop conditional request headers from GET requests. This ensures the full object is always returned, bypassing potential issues with unreliable backend conditional request support.

```properties
s3proxy.no-cache-blobstore=true
s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=transient-nio2
jclouds.identity=x
jclouds.credential=x
```

--------------------------------

### Create and Copy Object to Virtual Bucket

Source: https://context7.com/gaul/s3proxy/llms.txt

Demonstrates creating a virtual bucket and copying a file into it using the AWS CLI with a specified endpoint.

```bash
aws s3 --endpoint-url http://127.0.0.1:8080 mb s3://big-data
aws s3 --endpoint-url http://127.0.0.1:8080 cp data.csv s3://big-data/data.csv
```

--------------------------------

### boto3 (Python) S3 Client Configuration

Source: https://context7.com/gaul/s3proxy/llms.txt

Shows how to configure and use the boto3 library in Python to interact with S3Proxy. It sets up a session with path-style addressing enabled and demonstrates creating a bucket and uploading an object.

```python
# boto3 (Python)
import boto3
session = boto3.session.Session(
    aws_access_key_id='local-identity',
    aws_secret_access_key='local-credential')
config = boto3.session.Config(s3={'addressing_style': 'path'})
s3 = session.client('s3', endpoint_url='http://localhost:8080', config=config)
s3.create_bucket(Bucket='mybucket')
s3.put_object(Bucket='mybucket', Key='hello.txt', Body=b'Hello')
resp = s3.get_object(Bucket='mybucket', Key='hello.txt')
print(resp['Body'].read())  # b'Hello'
```

--------------------------------

### S3Proxy.setBlobStoreLocator

Source: https://context7.com/gaul/s3proxy/llms.txt

Installs a custom routing function to dynamically route different buckets to different backend BlobStore instances and credential sets. This is useful for multi-backend configurations.

```APIDOC
## `S3Proxy.setBlobStoreLocator` — dynamic multi-backend bucket routing

`setBlobStoreLocator(BlobStoreLocator)` installs a custom routing function so that different buckets are handled by different backend `BlobStore` instances and credential sets. This is used when multiple `.conf` files are passed on the command line.

```java
import org.gaul.s3proxy.BlobStoreLocator;
import org.gaul.s3proxy.S3Proxy;
import org.jclouds.blobstore.BlobStore;
import java.util.Map;

// Create two separate blobstore backends
BlobStore backendA = /* filesystem-nio2 blobstore for /tmp/a */ null;
BlobStore backendB = /* transient-nio2 blobstore */ null;

S3Proxy proxy = S3Proxy.builder()
        .blobStore(backendA)  // default backend
        .endpoint(URI.create("http://0.0.0.0:8080"))
        .build();

// Route "archive-*" buckets to backendB, everything else to backendA
proxy.setBlobStoreLocator((identity, container, blob) -> {
    if (container != null && container.startsWith("archive-")) {
        return Map.entry("archive-cred", backendB);
    }
    return Map.entry("default-cred", backendA);
});

proxy.start();
// PUT /archive-2024/data.csv  → routed to backendB
// PUT /production/data.csv    → routed to backendA
```
```

--------------------------------

### S3Proxy Authentication Configuration Properties

Source: https://context7.com/gaul/s3proxy/llms.txt

Example properties file for configuring AWS Signature v4 authentication for S3Proxy. This sets the authorization mode, identity, credential, and endpoint.

```properties
# s3proxy.conf — AWS Signature v4 authentication
s3proxy.authorization=aws-v4
s3proxy.identity=myaccesskeyid
s3proxy.credential=mysecretaccesskey
s3proxy.endpoint=http://0.0.0.0:8080
jclouds.provider=transient-nio2
jclouds.identity=x
jclouds.credential=x
```

--------------------------------

### s3cmd Configuration

Source: https://context7.com/gaul/s3proxy/llms.txt

Shows how to configure the s3cmd command-line tool to work with S3Proxy by creating a .s3cfg file with the S3Proxy endpoint and credentials.

```bash
# s3cmd configuration
cat > ~/.s3cfg << 'EOF'
access_key = local-identity
secret_key = local-credential
host_base = localhost:8080
host_bucket = localhost:8080
EOF
s3cmd ls
```

--------------------------------

### Enable Wire Logging for S3Proxy

Source: https://github.com/gaul/s3proxy/wiki/Tips

To emit storage backend requests and responses, run S3Proxy with both -DLOG_LEVEL=trace and -Djclouds.wire=debug system properties.

```bash
java -DLOG_LEVEL=trace -Djclouds.wire=debug -jar s3proxy.jar
```

--------------------------------

### Override Latency for Specific Operation

Source: https://github.com/gaul/s3proxy/wiki/Middleware-latency

Set a specific latency for a particular S3 operation, overriding the global setting. This example sets a higher latency for the 'put' operation.

```properties
s3proxy.latency-blobstore.put.latency=2000 # use 2000ms latency for put blob instead
```

--------------------------------

### S3Proxy Java Builder API

Source: https://context7.com/gaul/s3proxy/llms.txt

Constructs an S3Proxy instance using a fluent builder, starting with a jclouds BlobStore. This is the primary method for embedding S3Proxy in Java applications.

```APIDOC
## S3Proxy Java Builder API

`S3Proxy` is constructed using a fluent builder. The `BlobStore` is created via the jclouds `ContextBuilder` and passed to `S3Proxy.builder()`. This is the primary API for embedding S3Proxy in Java applications.

```java
import java.net.URI;
import java.util.Properties;
import org.eclipse.jetty.util.component.AbstractLifeCycle;
import org.gaul.s3proxy.AuthenticationType;
import org.gaul.s3proxy.S3Proxy;
import org.jclouds.ContextBuilder;
import org.jclouds.blobstore.BlobStore;
import org.jclouds.blobstore.BlobStoreContext;

// Build a BlobStore backed by the local filesystem
Properties props = new Properties();
props.setProperty("jclouds.filesystem.basedir", "/tmp/blobstore");

BlobStoreContext context = ContextBuilder.newBuilder("filesystem-nio2")
        .credentials("identity", "credential")
        .overrides(props)
        .build(BlobStoreContext.class);
BlobStore blobStore = context.getBlobStore();

// Build and start S3Proxy
S3Proxy s3Proxy = S3Proxy.builder()
        .blobStore(blobStore)
        .endpoint(URI.create("http://127.0.0.1:8080"))
        .awsAuthentication(AuthenticationType.AWS_V2_OR_V4, "mykey", "mysecret")
        .virtualHost("localhost")             // enables bucket.localhost:8080 style requests
        .maxSinglePartObjectSize(5L << 30)    // 5 GB default
        .ignoreUnknownHeaders(false)
        .jettyMaxThreads(200)
        .build();

s3Proxy.start();
while (!s3Proxy.getState().equals(AbstractLifeCycle.STARTED)) {
    Thread.sleep(1);
}
System.out.println("S3Proxy listening on port: " + s3Proxy.getPort());
// Output: S3Proxy listening on port: 8080

// Stop cleanly
s3Proxy.stop();
context.close();
```
```

--------------------------------

### Builder.fromProperties

Source: https://context7.com/gaul/s3proxy/llms.txt

Loads S3Proxy configuration settings from a Properties object, typically loaded from a configuration file, and returns a configured builder. This is the standard deployment pattern.

```APIDOC
## `S3Proxy.Builder.fromProperties` — load configuration from a Properties file

`Builder.fromProperties(Properties)` reads all `s3proxy.*` settings from a properties object (typically loaded from a `.conf` file) and returns a configured builder. This is the mechanism used by the `Main` class and is the standard deployment pattern.

```java
import java.io.FileInputStream;
import java.util.Properties;
import org.gaul.s3proxy.S3Proxy;
import org.jclouds.ContextBuilder;
import org.jclouds.blobstore.BlobStore;
import org.jclouds.blobstore.BlobStoreContext;

Properties properties = new Properties();
try (var is = new FileInputStream("s3proxy.conf")) {
    properties.load(is);
}

// Properties file content:
// s3proxy.authorization=aws-v2-or-v4
// s3proxy.endpoint=http://127.0.0.1:8080
// s3proxy.identity=local-identity
// s3proxy.credential=local-credential
// jclouds.provider=transient-nio2
// jclouds.identity=x
// jclouds.credential=x

BlobStoreContext ctx = ContextBuilder.newBuilder(
        properties.getProperty("jclouds.provider"))
        .credentials(properties.getProperty("jclouds.identity"),
                     properties.getProperty("jclouds.credential"))
        .overrides(properties)
        .build(BlobStoreContext.class);

S3Proxy proxy = S3Proxy.Builder
        .fromProperties(properties)
        .blobStore(ctx.getBlobStore())
        .build();

proxy.start();
// S3Proxy is now serving requests configured from file
```
```

--------------------------------

### Run S3Proxy with Docker (HTTPS)

Source: https://context7.com/gaul/s3proxy/llms.txt

Configure S3Proxy to run with HTTPS using Docker. Requires a keystore file mounted into the container and specified via environment variables for the secure endpoint and keystore details. Uses the transient-nio2 backend.

```bash
# With HTTPS (requires a keystore)
docker run \
  -e S3PROXY_SECURE_ENDPOINT=https://0.0.0.0:443 \
  -e S3PROXY_KEYSTORE_PATH=/keystore.jks \
  -e S3PROXY_KEYSTORE_PASSWORD=changeit \
  -e JCLOUDS_PROVIDER=transient-nio2 \
  -p 8443:443 \
  -v /path/to/keystore.jks:/keystore.jks \
  andrewgaul/s3proxy
```

--------------------------------

### Debug S3Proxy with Curl

Source: https://github.com/gaul/s3proxy/wiki/Tips

Use curl commands to interact with S3Proxy when authentication is disabled. Examples include listing buckets, listing blobs, adding buckets, and adding blobs.

```bash
curl http://<proxy>/ 
```

```bash
curl http://<proxy>/<bucketname>
```

```bash
curl -X PUT http://<proxy>/<newbucketname>
```

```bash
curl -X PUT -T <filename> http://<proxy>/<bucketname>/<newblobname>
```

--------------------------------

### Generate Keystore with JDK keytool

Source: https://github.com/gaul/s3proxy/wiki/SSL-support

Use the JDK keytool to generate a keystore file for SSL certificates. This is a prerequisite for enabling HTTPS.

```bash
$ keytool -keystore keystore.jks -alias aws -genkey -keyalg RSA
```

--------------------------------

### JUnit 4 Rule for S3Proxy

Source: https://context7.com/gaul/s3proxy/llms.txt

Integrates S3Proxy into the JUnit 4 @Rule lifecycle, starting the proxy before each test and shutting it down afterwards. Requires AWS SDK for Java v1.

```java
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import org.gaul.s3proxy.junit.S3ProxyRule;
import org.junit.Rule;
import org.junit.Test;

import java.util.List;

import static org.assertj.core.api.Assertions.assertThat;

public class MyS3JUnit4Test {

    @Rule
    public S3ProxyRule s3Proxy = S3ProxyRule.builder()
            .withCredentials("access", "secret")
            .withBlobStoreProvider("transient-nio2")
            .build();

    @Test
    public void uploadFile() {
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                .withCredentials(new AWSStaticCredentialsProvider(
                        new BasicAWSCredentials(
                                s3Proxy.getAccessKey(), s3Proxy.getSecretKey())))
                .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(
                        s3Proxy.getUri().toString(), Regions.US_EAST_1.getName()))
                .withPathStyleAccessEnabled(true)
                .build();

        s3Client.createBucket("test-bucket");
        s3Client.putObject("test-bucket", "readme.txt", "Hello S3Proxy");

        List<S3ObjectSummary> summaries = 
                s3Client.listObjects("test-bucket").getObjectSummaries();
        assertThat(summaries).hasSize(1);
        assertThat(summaries.get(0).getKey()).isEqualTo("readme.txt");
    }
}
```

--------------------------------

### Configure boto3 Client for S3Proxy

Source: https://context7.com/gaul/s3proxy/llms.txt

Initialize a boto3 S3 client in Python, specifying the endpoint URL and credentials. The 'addressing_style' should be set to 'path' for S3Proxy.

```python
import boto3
s3 = boto3.client('s3',
    endpoint_url='http://127.0.0.1:8080',
    aws_access_key_id='myaccesskeyid',
    aws_secret_access_key='mysecretaccesskey',
    config=boto3.session.Config(s3={'addressing_style': 'path'}))
s3.create_bucket(Bucket='mybucket')
s3.put_object(Bucket='mybucket', Key='hello.txt', Body=b'Hello World')
```

--------------------------------

### Configure AWS CLI Credentials

Source: https://context7.com/gaul/s3proxy/llms.txt

Set up AWS CLI with access key and secret key to interact with S3Proxy. Ensure the endpoint URL is correctly configured.

```bash
aws configure set aws_access_key_id myaccesskeyid
aws configure set aws_secret_access_key mysecretaccesskey

aws s3 ls --endpoint-url http://127.0.0.1:8080
aws s3 cp myfile.txt s3://mybucket/myfile.txt --endpoint-url http://127.0.0.1:8080
```

--------------------------------

### Scrape Prometheus Metrics with Curl

Source: https://context7.com/gaul/s3proxy/llms.txt

Demonstrates how to scrape the Prometheus metrics endpoint exposed by S3Proxy using 'curl'. The output shows example metrics like total requests and their status codes.

```bash
# Scrape metrics
curl http://127.0.0.1:9090/metrics
# Output (example):
# # HELP s3proxy_requests_total Total S3 requests handled
# # TYPE s3proxy_requests_total counter
# s3proxy_requests_total{method="GET",status="200"} 42.0
# s3proxy_requests_total{method="PUT",status="200"} 17.0
```

--------------------------------

### Load S3Proxy Configuration from Properties

Source: https://context7.com/gaul/s3proxy/llms.txt

Loads S3Proxy configuration from a Properties object, typically from a .conf file, and builds the S3Proxy instance. This is the standard deployment pattern used by the Main class.

```java
import java.io.FileInputStream;
import java.util.Properties;
import org.gaul.s3proxy.S3Proxy;
import org.jclouds.ContextBuilder;
import org.jclouds.blobstore.BlobStore;
import org.jclouds.blobstore.BlobStoreContext;

Properties properties = new Properties();
try (var is = new FileInputStream("s3proxy.conf")) {
    properties.load(is);
}

// Properties file content:
// s3proxy.authorization=aws-v2-or-v4
// s3proxy.endpoint=http://127.0.0.1:8080
// s3proxy.identity=local-identity
// s3proxy.credential=local-credential
// jclouds.provider=transient-nio2
// jclouds.identity=x
// jclouds.credential=x

BlobStoreContext ctx = ContextBuilder.newBuilder(
        properties.getProperty("jclouds.provider"))
        .credentials(properties.getProperty("jclouds.identity"),
                     properties.getProperty("jclouds.credential"))
        .overrides(properties)
        .build(BlobStoreContext.class);

S3Proxy proxy = S3Proxy.Builder
        .fromProperties(properties)
        .blobStore(ctx.getBlobStore())
        .build();

proxy.start();
// S3Proxy is now serving requests configured from file
```

--------------------------------

### Run S3Proxy (Linux/Mac OS X)

Source: https://github.com/gaul/s3proxy/blob/master/README.md

Executable command to run S3Proxy on Linux and Mac OS X systems after making the jar executable. It uses the specified properties file for configuration.

```bash
chmod +x s3proxy
s3proxy --properties s3proxy.conf
```

--------------------------------

### Enable Trace Logging for S3Proxy

Source: https://github.com/gaul/s3proxy/wiki/Tips

Run the S3Proxy JAR with the -DLOG_LEVEL=trace system property to enable trace logging, which shows client requests and server responses.

```bash
java -DLOG_LEVEL=trace -jar s3proxy.jar
```

--------------------------------

### Run S3Proxy (Windows)

Source: https://github.com/gaul/s3proxy/blob/master/README.md

Command to run S3Proxy on Windows systems, explicitly invoking Java to execute the jar file with the specified properties file.

```bash
java -jar s3proxy --properties s3proxy.conf
```

--------------------------------

### Using Prefix BlobStore with AWS CLI

Source: https://context7.com/gaul/s3proxy/llms.txt

Demonstrates how to use the AWS CLI with an S3Proxy Prefix BlobStore configuration. Client writes to 'customer-bucket' are stored under the 'scoped-data/' prefix on the backend.

```bash
# Client writes to "customer-bucket"; backend stores at "scoped-data/myfile.txt"
aws s3 --endpoint-url http://127.0.0.1:8080 cp myfile.txt s3://customer-bucket/myfile.txt
```

--------------------------------

### Configure Latency and Throughput Limits in S3Proxy

Source: https://context7.com/gaul/s3proxy/llms.txt

Inject artificial latency and/or throughput limits per operation type. Use '*' for global settings and specific operation names (e.g., 'put', 'get') for overrides.

```properties
s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=transient-nio2
jclouds.identity=x
jclouds.credential=x

# Apply 200ms latency to all operations
s3proxy.latency-blobstore.*.latency=200

# Override: slower puts at 2000ms, faster gets at 50ms
s3proxy.latency-blobstore.put.latency=2000
s3proxy.latency-blobstore.get.latency=50

# Throttle upload speed to 100 kb/s globally, override get to 1 Mb/s
s3proxy.latency-blobstore.*.speed=100
s3proxy.latency-blobstore.get.speed=1024
```

--------------------------------

### Enable Prefix BlobStore Middleware

Source: https://github.com/gaul/s3proxy/wiki/Middleware-prefix-blobstore

Enable the middleware by adding a property for each bucket that should be scoped to a prefix. The format is 's3proxy.prefix-blobstore.<bucket-name>=<prefix>'.

```properties
s3proxy.prefix-blobstore.<bucket-name>=<prefix>
```

--------------------------------

### Run S3Proxy with Docker (Anonymous Access)

Source: https://context7.com/gaul/s3proxy/llms.txt

Run S3Proxy using a Docker container with anonymous access and the transient-nio2 backend for in-memory storage. Configuration is done via environment variables.

```bash
# Run with anonymous access and filesystem backend (in-memory for testing)
docker run \
  -e S3PROXY_AUTHORIZATION=none \
  -e S3PROXY_ENDPOINT=http://0.0.0.0:80 \
  -e JCLOUDS_PROVIDER=transient-nio2 \
  -p 8080:80 \
  andrewgaul/s3proxy
```

--------------------------------

### Run S3Proxy with Docker (AWS v4 Authentication)

Source: https://context7.com/gaul/s3proxy/llms.txt

Run S3Proxy via Docker with AWS v4 authentication. Specify access key and secret key using environment variables. Uses the transient-nio2 backend.

```bash
# With AWS v4 authentication
docker run \
  -e S3PROXY_AUTHORIZATION=aws-v4 \
  -e S3PROXY_IDENTITY=myaccesskey \
  -e S3PROXY_CREDENTIAL=mysecretkey \
  -e S3PROXY_ENDPOINT=http://0.0.0.0:80 \
  -e JCLOUDS_PROVIDER=transient-nio2 \
  -p 8080:80 \
  andrewgaul/s3proxy
```

--------------------------------

### Mock Large Object Uploads with Properties

Source: https://context7.com/gaul/s3proxy/llms.txt

Configures S3Proxy to use the 'null-blobstore' feature for mocking large object uploads. This is useful for performance testing as it creates stubs storing only object length and returns NUL bytes on GET, without storing actual data.

```properties
s3proxy.endpoint=http://localhost:8081
s3proxy.authorization=aws-v2
s3proxy.identity=local-identity
s3proxy.credential=local-credential
s3proxy.null-blobstore=true

jclouds.provider=transient-nio2
jclouds.identity=x
jclouds.credential=x
```

--------------------------------

### Basic Bucket Locator Configuration

Source: https://github.com/gaul/s3proxy/wiki/Middleware-bucket-locator

Configure S3Proxy to map specific buckets to backends using indexed properties. The index is only for differentiation.

```properties
s3proxy.bucket-locator.1=bucket1
s3proxy.bucket-locator.2=bucket2
```

--------------------------------

### Simulate Speed for All Supported Operations

Source: https://github.com/gaul/s3proxy/wiki/Middleware-latency

Configure a global speed limit for all operations that support speed throttling. Specify the speed in kb/s.

```properties
s3proxy.latency-blobstore.*.speed=100 # use 100 kb/s for all operations
```

--------------------------------

### S3Proxy Configuration for Backblaze B2

Source: https://context7.com/gaul/s3proxy/llms.txt

Configure S3Proxy to use Backblaze B2 as the storage backend. Requires B2 Account ID and Application Key.

```properties
s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.authorization=aws-v2-or-v4
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.provider=b2
jclouds.identity=<B2_ACCOUNT_ID>
jclouds.credential=<B2_APPLICATION_KEY>
```

--------------------------------

### Using Alias BlobStore with AWS CLI

Source: https://context7.com/gaul/s3proxy/llms.txt

Demonstrates how to use the AWS CLI with an S3Proxy Alias BlobStore configuration. The client interacts with the virtual bucket name ('content'), while the backend uses the aliased name ('backend-data').

```bash
# The client sees "content"; the backend stores in "backend-data"
aws s3 --endpoint-url http://127.0.0.1:8080 mb s3://content
aws s3 --endpoint-url http://127.0.0.1:8080 cp file.txt s3://content/file.txt
aws s3 --endpoint-url http://127.0.0.1:8080 ls
# Output: content   (backend-data is replaced by content in listings)
```

--------------------------------

### Swift (Keystone v2.0) Configuration

Source: https://github.com/gaul/s3proxy/wiki/Storage-backend-examples

Configuration for Swift storage using Keystone v2.0 authentication. Retrieve OS variables from your tenant's openrc.sh file.

```properties
s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.authorization=aws-v2-or-v4
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.provider=openstack-swift
jclouds.endpoint=$OS_AUTH_URL (e.g. https://keystone.example.com/v2.0/)
jclouds.region=$OS_REGION_NAME (e.g. paris1)
jclouds.identity=$OS_TENANT_NAME:$OS_USERNAME (e.g. production:user1)
jclouds.credential=$OS_PASSWORD (e.g. password)
```

--------------------------------

### Swift (Keystone v3.0) Configuration

Source: https://github.com/gaul/s3proxy/wiki/Storage-backend-examples

Configuration for Swift storage using Keystone v3.0 authentication. Retrieve OS variables from your project's openrc.sh file.

```properties
s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.authorization=aws-v2-or-v4
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.provider=openstack-swift
jclouds.endpoint=$OS_AUTH_URL (e.g. https://keystone.example.com/v3/)
jclouds.regions=$OS_REGION_NAME (e.g. paris1)
jclouds.region=$OS_REGION_NAME (e.g. paris1)
jclouds.identity=$OS_USER_DOMAIN_NAME:$OS_USERNAME (e.g. domain1:user1)
jclouds.keystone.version=3
jclouds.keystone.scope=project:$OS_PROJECT_NAME (e.g. project:project1)
jclouds.keystone.project_domain_name=$OS_PROJECT_DOMAIN_NAME (e.g. domain2 - only needed if user and project domains differ)
jclouds.credential=$OS_PASSWORD (e.g. password)
```

--------------------------------

### Boto3 Configuration for S3Proxy

Source: https://github.com/gaul/s3proxy/wiki/Client-compatibility-list

Configure boto3 to connect to S3Proxy. Set up a session with credentials and a configuration for path-style addressing. Instantiate both low-level client and S3 resource objects.

```python
session = boto3.session.Session(aws_access_key_id='identity',
                                aws_secret_access_key='credential')
config = boto3.session.Config(s3={'addressing_style': 'path'})
# low level S3 client
client = session.client('s3', endpoint_url='http://localhost:8080',
                        config=config)
# S3 resource object
resource = session.resource('s3', endpoint_url='http://localhost:8080',
                            config=config)
```

--------------------------------

### S3Proxy Backend Configuration for Local Filesystem

Source: https://context7.com/gaul/s3proxy/llms.txt

Configuration file for S3Proxy's backend A, which owns buckets matching the 'logs-*' pattern and stores them on the local filesystem.

```properties
# backend-a.conf — owns "logs-*" buckets on local filesystem
s3proxy.authorization=aws-v2-or-v4
s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.identity=local-identity
s3proxy.credential=local-credential
s3proxy.bucket-locator.1=logs-*
jclouds.provider=filesystem-nio2
jclouds.filesystem.basedir=/data/logs
jclouds.identity=x
jclouds.credential=x
```

--------------------------------

### Enable Prometheus Metrics Endpoint with Properties

Source: https://context7.com/gaul/s3proxy/llms.txt

Configures S3Proxy to expose a `/metrics` endpoint in Prometheus exposition format. The 's3proxy.metrics.enabled' property must be set to 'true'. Default port is 9090 and default host is 127.0.0.1.

```properties
s3proxy.metrics.enabled=true
s3proxy.metrics.port=9090      # default: 9090
s3proxy.metrics.host=0.0.0.0  # default: 127.0.0.1

s3proxy.authorization=none
s3proxy.endpoint=http://0.0.0.0:8080
jclouds.provider=transient-nio2
jclouds.identity=x
jclouds.credential=x
```

--------------------------------

### Accessing S3Proxy Buckets via Virtual Host Style

Source: https://context7.com/gaul/s3proxy/llms.txt

Configure DNS or /etc/hosts for virtual host access and use cURL or s3cmd to interact with buckets using the virtual host style.

```bash
# Add wildcard DNS entry (dnsmasq example) or /etc/hosts entry
echo "127.0.0.1  mybucket.localhost" | sudo tee -a /etc/hosts

# Access bucket via virtual host style
curl http://mybucket.localhost:8080/
curl http://mybucket.localhost:8080/myobject.txt

# s3cmd using virtual host
cat > ~/.s3cfg << 'EOF'
access_key = local-identity
secret_key = local-credential
host_base = localhost:8080
host_bucket = %(bucket)s.localhost:8080
EOF
s3cmd ls
```

--------------------------------

### AWS SDK for Ruby S3 Client Configuration

Source: https://context7.com/gaul/s3proxy/llms.txt

Demonstrates how to configure the AWS SDK for Ruby to interact with S3Proxy. It sets the credentials, endpoint, and enables force path style.

```ruby
# AWS SDK for Ruby
require 'aws-sdk'
Aws.config.update({
  credentials: Aws::Credentials.new('identity', 'credential'),
  endpoint: 'http://localhost:8080',
  force_path_style: true
})
s3 = Aws::S3::Client.new
s3.create_bucket(bucket: 'mybucket')
```

--------------------------------

### Fog Configuration for S3Proxy

Source: https://github.com/gaul/s3proxy/wiki/Client-compatibility-list

Initialize Fog storage with AWS provider, S3Proxy endpoint, and path-style access enabled.

```ruby
fog = Fog::Storage.new(provider: 'aws', endpoint: 'http://127.0.0.1:8080',
    path_style: true)
```

--------------------------------

### Bucket Locator with Glob Syntax

Source: https://github.com/gaul/s3proxy/wiki/Middleware-bucket-locator

The `bucket-locator` middleware supports glob syntax for matching bucket names, allowing for more flexible routing.

```properties
s3proxy.bucket-locator.1=bucket-*
```

--------------------------------

### AWS SDK for Java v1 S3Client Configuration

Source: https://context7.com/gaul/s3proxy/llms.txt

Demonstrates how to configure and use the AWS SDK for Java v1 to connect to S3Proxy. Path-style access and explicit endpoint configuration are required.

```java
// AWS SDK for Java v1
AmazonS3Client client = new AmazonS3Client(
    new BasicAWSCredentials("local-identity", "local-credential"),
    new ClientConfiguration().withSignerOverride("S3SignerType")));
client.setEndpoint("http://127.0.0.1:8080");
client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));
client.listBuckets().forEach(b -> System.out.println(b.getName()));
```

--------------------------------

### s3cmd Configuration (before 1.5.0) for S3Proxy

Source: https://github.com/gaul/s3proxy/wiki/Client-compatibility-list

Configure s3cmd (older than 1.5.0) for S3Proxy. Set access key, secret key, and host details. Also, configure S3Proxy's virtual host and ensure `bucket.localhost` resolves to `127.0.0.1`.

```ini
[default]
access_key = local-identity
secret_key = local-credential
host_base = localhost:8080
host_bucket = %(bucket)s.localhost:8080
```

```properties
s3proxy.virtual-host=localhost
```

```text
127.0.0.1       bucket.localhost
```

```bash
echo address=/localhost/127.0.0.1 | sudo tee /etc/NetworkManager/dnsmasq.d/localhost.conf
```

--------------------------------

### Enable Debug Logging for S3Proxy

Source: https://github.com/gaul/s3proxy/wiki/Tips

Run the S3Proxy JAR with the -DLOG_LEVEL=debug system property to enable debug logging for more detailed request information.

```bash
java -DLOG_LEVEL=debug -jar s3proxy.jar
```

--------------------------------

### Swift (non-Keystone) Configuration

Source: https://github.com/gaul/s3proxy/wiki/Storage-backend-examples

Configuration for Swift storage without Keystone authentication. This configuration might be outdated.

```properties
s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.authorization=aws-v2-or-v4
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.provider=swift
jclouds.endpoint=http://IP-OF-SWIFT:PORT/auth/v1.0
jclouds.api=swift
jclouds.identity=test:tester
jclouds.credential=testing
```