### Install S3Mock Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Clone the S3Mock repository and build the project, including the Docker image. ```bash git clone https://github.com/adobe/S3Mock.git cd S3Mock make install # Full build including Docker image ``` -------------------------------- ### Install S3Mock (Skip Docker) Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Install S3Mock without building the Docker image for a faster build process. ```bash make skip-docker ``` -------------------------------- ### Verify Prerequisites Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Verify that the minimum required versions of JDK, Maven, and Docker are installed. ```bash java -version # Should show 25.x ./mvnw --version # Should show 3.9+ docker info # Should respond without error ``` -------------------------------- ### Start S3Mock with Command-Line Arguments Source: https://context7.com/adobe/s3mock/llms.txt Initiate the S3Mock server using command-line style arguments for configuration. This is an alternative to using a properties map. ```kotlin import com.adobe.testing.s3mock.S3MockApplication // Start with command-line-style args val s3MockArgs = S3MockApplication.start("--server.port=9292", "--http.port=9191") ``` -------------------------------- ### Docker Compose Configuration for S3Mock Source: https://context7.com/adobe/s3mock/llms.txt Example Docker Compose setup for S3Mock, including image, environment variables, ports, and a health check. ```yaml # services: # s3mock: # image: adobe/s3mock:latest # environment: # - COM_ADOBE_TESTING_S3MOCK_STORE_INITIAL_BUCKETS=my-bucket # - COM_ADOBE_TESTING_S3MOCK_STORE_REGION=eu-west-1 # ports: # - 9090:9090 # - 9191:9191 # healthcheck: # test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/favicon.ico"] # interval: 5s # retries: 3 ``` -------------------------------- ### Build Commands for S3Mock Source: https://github.com/adobe/s3mock/blob/main/AGENTS.md Common make commands for building, testing, and running S3Mock from source. Use 'make install' for a full build. ```bash make install # Full build make skip-docker # Skip Docker make test # Unit tests only make integration-tests # Run integration tests make format # Format Kotlin code (ktlint) make run # Run S3Mock from source (Spring Boot) make sort # Sort POM files (sortpom) ``` -------------------------------- ### Start S3Mock with Docker Source: https://github.com/adobe/s3mock/blob/main/README.md Run S3Mock as a Docker container. Ensure the container is accessible on port 9090. ```shell docker run -p 9090:9090 adobe/s3mock ``` -------------------------------- ### Start S3Mock with Default Ports Source: https://context7.com/adobe/s3mock/llms.txt Launch the embedded S3Mock server using default HTTP and HTTPS ports. The returned object provides access to the server's ports. ```kotlin import com.adobe.testing.s3mock.S3MockApplication // Start with defaults (HTTP :9090, HTTPS :9191) val s3Mock = S3MockApplication.start() println("HTTP port: ${s3Mock.httpPort}") // 9090 println("HTTPS port: ${s3Mock.port}") // 9191 ``` -------------------------------- ### Start S3Mock with Custom Properties Source: https://context7.com/adobe/s3mock/llms.txt Configure S3Mock with custom settings by providing a map of properties. This allows specifying ports, initial buckets, region, and suppressing the banner. ```kotlin import com.adobe.testing.s3mock.S3MockApplication // Start with custom properties map val s3MockCustom = S3MockApplication.start( mutableMapOf( "server.port" to "0", // random HTTPS port "http.port" to "0", // random HTTP port "com.adobe.testing.s3mock.store.initialBuckets" to "a,b", "com.adobe.testing.s3mock.store.region" to "ap-southeast-1", "silent" to "true", // suppress banner ) ) // Register a KMS key reference at runtime s3MockCustom.registerKMSKeyRef("arn:aws:kms:us-east-1:1234567890:key/my-key") // Stop gracefully s3MockCustom.stop() ``` -------------------------------- ### Integration Test Example with S3TestBase Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Shows an integration test using S3TestBase for a pre-configured s3Client. Utilizes `givenBucket` for unique bucket names and follows the Arrange-Act-Assert pattern for testing S3 operations. ```kotlin internal class MyFeatureIT : S3TestBase() { @Test fun `should perform operation`(testInfo: TestInfo) { // Arrange val bucketName = givenBucket(testInfo) // Act s3Client.putObject( PutObjectRequest.builder().bucket(bucketName).key("key").build(), RequestBody.fromString("content") ) // Assert val response = s3Client.getObject( GetObjectRequest.builder().bucket(bucketName).key("key").build() ) assertThat(response.readAllBytes().decodeToString()).isEqualTo("content") } } ``` -------------------------------- ### Unit Test Example with Mockito Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Demonstrates a unit test for ObjectService using Spring Boot, Mockito for mocking dependencies, and AssertJ for assertions. Ensure to mock external dependencies and use the class under test (iut) for testing. ```kotlin @SpringBootTest(classes = [ServiceConfiguration::class], webEnvironment = SpringBootTest.WebEnvironment.NONE) @MockitoBean(types = [BucketService::class, MultipartService::class, MultipartStore::class]) internal class ObjectServiceTest : ServiceTestBase() { @Autowired private lateinit var iut: ObjectService @Test fun `should get object`() { whenever(bucketStore.getBucketMetadata("bucket")).thenReturn(bucket) whenever(objectStore.getObject(bucket, "key")).thenReturn(s3Object) assertThat(iut.getObject("bucket", "key")).isEqualTo(s3Object) } } ``` -------------------------------- ### Run Integration Tests Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Execute integration tests for S3Mock. Docker is required as these tests start a Docker container automatically via Testcontainers. ```bash make integration-tests # Integration tests (Docker required) ``` -------------------------------- ### Run S3Mock Service Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Start the S3Mock service using the provided make target, which runs on HTTP port 9090 and HTTPS port 9191. ```bash make run # Start S3Mock on HTTP :9090 / HTTPS :9191 ``` -------------------------------- ### Enable and Get Bucket Versioning Source: https://context7.com/adobe/s3mock/llms.txt Use PutBucketVersioning to enable versioning on a bucket and GetBucketVersioning to retrieve its status. This is useful for managing object lifecycles and recovering from accidental overwrites or deletes. ```kotlin import software.amazon.awssdk.services.s3.model.BucketVersioningStatus import software.amazon.awssdk.services.s3.model.VersioningConfiguration // Enable versioning s3Client.putBucketVersioning { it.bucket("my-bucket") it.versioningConfiguration( VersioningConfiguration.builder() .status(BucketVersioningStatus.ENABLED) .build() ) } // Get current versioning status val versioning = s3Client.getBucketVersioning { it.bucket("my-bucket") } println(versioning.status()) // ENABLED ``` -------------------------------- ### Configure TestNG with S3MockListener Source: https://context7.com/adobe/s3mock/llms.txt Integrate S3MockListener into your testng.xml to automatically start and stop S3Mock for your TestNG suite. Ensure the s3mock-testng dependency is included in your project. ```xml ``` -------------------------------- ### AssertJ Exception Handling Example Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Demonstrates how to assert exceptions using AssertJ's `assertThatThrownBy`. This is preferred over JUnit's `assertThrows` for clearer and more expressive error handling in tests. ```kotlin assertThatThrownBy { s3Client.deleteBucket { it.bucket(bucketName) } } .isInstanceOf(AwsServiceException::class.java) .hasMessageContaining("Status Code: 409") ``` -------------------------------- ### Running Server Tests Skipping Docker Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Command to run tests for the server module while skipping Docker-related setup, useful for faster feedback cycles. ```bash ./mvnw test -pl server -DskipDocker ``` -------------------------------- ### TestNG Test with S3Mock Client Source: https://context7.com/adobe/s3mock/llms.txt Example of a TestNG test class that uses an S3Client created by S3Mock. The S3MockListener automatically manages the S3Mock lifecycle. ```kotlin // Maven dependency: // // com.adobe.testing // s3mock-testng // 5.0.1 // test // import com.adobe.testing.s3mock.testng.S3Mock import com.adobe.testing.s3mock.testsupport.common.S3MockStarter import org.testng.annotations.Test import software.amazon.awssdk.services.s3.S3Client class MyS3Test { // S3Mock is started by S3MockListener before this test runs private val s3Client: S3Client = S3Mock.createS3ClientV2() @Test fun shouldCreateBucketAndPutObject() { s3Client.createBucket { it.bucket("testng-bucket") } s3Client.putObject( { it.bucket("testng-bucket").key("hello.txt") }, software.amazon.awssdk.core.sync.RequestBody.fromString("Hello TestNG!") ) val obj = s3Client.getObjectAsBytes { it.bucket("testng-bucket").key("hello.txt") } assert(obj.asUtf8String() == "Hello TestNG!") } } ``` -------------------------------- ### Declarative JUnit 5 Extension Registration Source: https://context7.com/adobe/s3mock/llms.txt Use the `@ExtendWith(S3MockExtension::class)` annotation for automatic S3Mock setup before tests. The S3Client is injected automatically. ```kotlin import com.adobe.testing.s3mock.junit5.S3MockExtension import org.junit.jupiter.api.Test import org.junit.jupiter.api.extension.ExtendWith import software.amazon.awssdk.core.sync.RequestBody import software.amazon.awssdk.services.s3.S3Client // ── Declarative registration ────────────────────────────────────────────────── @ExtendWith(S3MockExtension::class) class DeclarativeTest { @Test fun uploadAndVerifyChecksum(s3Client: S3Client) { s3Client.createBucket { it.bucket("test-bucket") } s3Client.putObject( { it.bucket("test-bucket").key("file.txt") }, RequestBody.fromString("content") ) val response = s3Client.getObjectAsBytes { it.bucket("test-bucket").key("file.txt") } assert(response.asUtf8String() == "content") } } ``` -------------------------------- ### Run S3Mock as Spring Boot Application Source: https://github.com/adobe/s3mock/blob/main/README.md Start S3Mock directly from the source code as a Spring Boot application. This is useful for local development and debugging. ```shell make run ``` -------------------------------- ### Configure AWS SDK v2 Endpoint Override Source: https://github.com/adobe/s3mock/blob/main/README.md Example of configuring the AWS SDK v2 client to use S3Mock's endpoint. Remember to supply dummy credentials and a region. ```java S3Client s3Client = S3Client.builder() .endpointOverride(URI.create("http://localhost:9090")) .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("foo", "bar"))) .region(Region.US_EAST_1) .build(); ``` -------------------------------- ### S3Mock Testcontainers Usage Example Source: https://github.com/adobe/s3mock/blob/main/README.md Demonstrates using S3MockContainer in a JUnit 5 test class. Initializes the container with an initial bucket and creates an S3 client pointing to the mock endpoint. ```kotlin @Testcontainers class MyTest { @Container val s3Mock = S3MockContainer("latest") .withInitialBuckets("test-bucket") @Test fun test() { val s3Client = S3Client.builder() .endpointOverride(URI.create(s3Mock.httpEndpoint)) .region(Region.US_EAST_1) .credentialsProvider(StaticCredentialsProvider.create( AwsBasicCredentials.create("foo", "bar") )) .build() s3Client.createBucket { it.bucket("my-bucket") } } } ``` -------------------------------- ### Activate Spring Debug Profile Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Run S3Mock with the 'debug' Spring profile activated via an environment variable, enabling debug logging and actuators. This example also maps ports for the Docker container. ```bash SPRING_PROFILES_ACTIVE=debug docker run -p 9090:9090 -p 9191:9191 adobe/s3mock ``` -------------------------------- ### Configure S3Mock Initial Buckets Source: https://context7.com/adobe/s3mock/llms.txt Create buckets on S3Mock startup by providing a comma-separated list of names to the COM_ADOBE_TESTING_S3MOCK_STORE_INITIAL_BUCKETS environment variable. ```shell # Comma-separated bucket names created on startup COM_ADOBE_TESTING_S3MOCK_STORE_INITIAL_BUCKETS=bucket-a,bucket-b,bucket-c ``` -------------------------------- ### PutObject from String and File with Metadata Source: https://context7.com/adobe/s3mock/llms.txt Demonstrates putting objects into a bucket from a string and a file. Includes options for setting content type, storage class, metadata, and server-side encryption. ```kotlin import software.amazon.awssdk.core.sync.RequestBody import software.amazon.awssdk.services.s3.model.NoSuchKeyException import software.amazon.awssdk.services.s3.model.ServerSideEncryption import software.amazon.awssdk.services.s3.model.StorageClass import java.io.File // Put from string s3Client.putObject( { it.bucket("my-bucket").key("notes/hello.txt") }, RequestBody.fromString("Hello, S3Mock!") ) ``` ```kotlin // Put from file with metadata s3Client.putObject( { it.bucket("my-bucket") it.key("images/photo.png") it.contentType("image/png") it.storageClass(StorageClass.STANDARD) it.metadata(mapOf("author" to "alice", "version" to "1.0")) it.serverSideEncryption(ServerSideEncryption.AWS_KMS) it.ssekmsKeyId("arn:aws:kms:us-east-1:1234567890:key/my-key") }, RequestBody.fromFile(File("photo.png")) ) ``` -------------------------------- ### Object Operations Source: https://context7.com/adobe/s3mock/llms.txt Perform fundamental single-object operations like putting, getting, heading, and deleting objects. ```APIDOC ## Object Operations — PutObject, GetObject, HeadObject, DeleteObject Fundamental single-object operations. ### PutObject Uploads an object to an S3 bucket. Can be from a string or a file, with optional metadata and encryption settings. #### Request Example (from string) ```kotlin s3Client.putObject( { it.bucket("my-bucket").key("notes/hello.txt") }, RequestBody.fromString("Hello, S3Mock!") ) ``` #### Request Example (from file with metadata) ```kotlin s3Client.putObject( { it.bucket("my-bucket") it.key("images/photo.png") it.contentType("image/png") it.storageClass(StorageClass.STANDARD) it.metadata(mapOf("author" to "alice", "version" to "1.0")) it.serverSideEncryption(ServerSideEncryption.AWS_KMS) it.ssekmsKeyId("arn:aws:kms:us-east-1:1234567890:key/my-key") }, RequestBody.fromFile(File("photo.png")) ) ``` ### GetObject Retrieves an object from an S3 bucket. The object can be read as bytes or streamed, with support for byte range requests. #### Request Example (read bytes) ```kotlin val bytes = s3Client.getObjectAsBytes { it.bucket("my-bucket").key("notes/hello.txt") } println(bytes.asUtf8String()) // "Hello, S3Mock!" ``` #### Request Example (with range) ```kotlin s3Client.getObject { it.bucket("my-bucket") it.key("notes/hello.txt") it.range("bytes=0-4") }.use { stream -> println(stream.readAllBytes().decodeToString()) // "Hello" } ``` ### HeadObject Retrieves metadata about an object without returning the object's body. Useful for checking object properties like content length and type. #### Request Example ```kotlin val head = s3Client.headObject { it.bucket("my-bucket").key("notes/hello.txt") } println(head.contentLength()) // 14 println(head.contentType()) // "text/plain; charset=utf-8" ``` ### DeleteObject Deletes a single object from an S3 bucket. Can also delete multiple objects at once. #### Request Example (single object) ```kotlin s3Client.deleteObject { it.bucket("my-bucket").key("notes/hello.txt") } try { s3Client.getObject { it.bucket("my-bucket").key("notes/hello.txt") } } catch (e: NoSuchKeyException) { println("Object deleted as expected") } ``` #### Request Example (multiple objects) ```kotlin import software.amazon.awssdk.services.s3.model.Delete import software.amazon.awssdk.services.s3.model.ObjectIdentifier s3Client.deleteObjects { it.bucket("my-bucket") it.delete( Delete.builder() .objects( ObjectIdentifier.builder().key("file1.txt").build(), ObjectIdentifier.builder().key("file2.txt").build(), ) .build() ) } ``` ``` -------------------------------- ### Paginate ListObjectsV2 with Prefix and Delimiter Source: https://context7.com/adobe/s3mock/llms.txt Lists objects in a bucket with pagination, filtering by prefix and using a delimiter to simulate folder structures. Continues fetching pages until all objects are retrieved. ```kotlin // Paginate with prefix and delimiter (simulate folders) var continuationToken: String? = null do { val page = s3Client.listObjectsV2 { it.bucket("my-bucket") it.prefix("images/") it.delimiter("/") it.maxKeys(100) continuationToken?.let { token -> it.continuationToken(token) } } page.contents().forEach { println(it.key()) } page.commonPrefixes().forEach { println(" prefix: ${it.prefix()}") } continuationToken = if (page.isTruncated) page.nextContinuationToken() else null } while (continuationToken != null) ``` -------------------------------- ### List All Make Targets Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Display a list of all available targets for the S3Mock project's Makefile. ```bash make help # List all available targets ``` -------------------------------- ### S3MockContainer Manual Lifecycle Management Source: https://context7.com/adobe/s3mock/llms.txt Manually start and stop the S3MockContainer. Access HTTP/HTTPS endpoints and ports directly. Useful when not using Testcontainers annotations. ```kotlin // Manual lifecycle (without @Testcontainers) val s3Mock = S3MockContainer("latest") .withInitialBuckets("bucket-a,bucket-b") s3Mock.start() val httpEndpoint = s3Mock.httpEndpoint // "http://localhost:" val httpsEndpoint = s3Mock.httpsEndpoint // "https://localhost:" val httpPort = s3Mock.httpServerPort val httpsPort = s3Mock.httpsServerPort s3Mock.stop() ``` -------------------------------- ### Run Type Checking and Sorting Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Compile main and test sources for type checking and sort POM files. ```bash make typecheck # Compile main + test sources make sort # Sort POM files ``` -------------------------------- ### Check S3Mock Readiness with Favicon Source: https://github.com/adobe/s3mock/blob/main/README.md Use this curl command to check the basic readiness of S3Mock. It returns HTTP 200 OK and is always available. ```shell curl -sf http://localhost:9090/favicon.ico ``` -------------------------------- ### Run Linting and Formatting Checks Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Perform linting using ktlint and Checkstyle, and auto-format Kotlin code. ```bash make lint # ktlint + Checkstyle (check only) make fmt # Auto-format Kotlin (ktlint) ``` -------------------------------- ### Docker Compose Health Check Source: https://github.com/adobe/s3mock/blob/main/README.md Example Docker Compose configuration for S3Mock, including a health check that uses wget to probe the /favicon.ico endpoint. ```yaml services: s3mock: image: adobe/s3mock:latest ports: - 9090:9090 - 9191:9191 healthcheck: test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/favicon.ico"] interval: 5s retries: 3 ``` -------------------------------- ### Package S3Mock Server and Run as Docker Container Source: https://github.com/adobe/s3mock/blob/main/README.md Build the S3Mock server JAR and then run it as a Docker container. This is a common way to deploy and run S3Mock. ```shell ./mvnw clean package -pl server -am -DskipTests docker run -p 9090:9090 -p 9191:9191 adobe/s3mock:latest ``` -------------------------------- ### Generate Presigned URLs with S3Presigner Source: https://context7.com/adobe/s3mock/llms.txt Use S3Presigner to generate presigned URLs for GET and PUT operations. S3Mock does not validate expiry, signatures, or method restrictions for these URLs. ```kotlin import software.amazon.awssdk.services.s3.presigner.S3Presigner import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest import software.amazon.awssdk.services.s3.presigner.model.PutObjectPresignRequest import software.amazon.awssdk.services.s3.model.GetObjectRequest import software.amazon.awssdk.services.s3.model.PutObjectRequest import java.time.Duration val presigner = S3Presigner.builder() .region(software.amazon.awssdk.regions.Region.US_EAST_1) .credentialsProvider( software.amazon.awssdk.auth.credentials.StaticCredentialsProvider.create( software.amazon.awssdk.auth.credentials.AwsBasicCredentials.create("foo", "bar") ) ) .serviceConfiguration( software.amazon.awssdk.services.s3.S3Configuration.builder() .pathStyleAccessEnabled(true) .build() ) .endpointOverride(java.net.URI.create("https://localhost:9191")) .build() // Generate a presigned GET URL val presignedGet = presigner.presignGetObject( GetObjectPresignRequest.builder() .signatureDuration(Duration.ofMinutes(60)) .getObjectRequest( GetObjectRequest.builder() .bucket("my-bucket") .key("report.pdf") .build() ) .build() ) println(presignedGet.url()) // https://localhost:9191/my-bucket/report.pdf?X-Amz-... // Download via the presigned URL (using any HTTP client) // curl --insecure "" -O // Generate a presigned PUT URL val presignedPut = presigner.presignPutObject( PutObjectPresignRequest.builder() .signatureDuration(Duration.ofMinutes(15)) .putObjectRequest(PutObjectRequest.builder().bucket("my-bucket").key("upload.bin").build()) .build() ) // curl --insecure -X PUT --upload-file ./upload.bin "" presigner.close() ``` -------------------------------- ### Running All Integration Tests Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Command to execute all integration tests for the S3Mock project. Requires Docker to be running. ```bash make integration-tests ``` -------------------------------- ### TestNG Access to S3Mock Endpoint Source: https://github.com/adobe/s3mock/blob/main/testsupport/AGENTS.md Retrieve the S3Mock HTTP endpoint using system properties within a TestNG listener setup. This approach will be removed in S3Mock 6.x. ```kotlin val endpoint = System.getProperty("s3mock.httpEndpoint") ``` -------------------------------- ### Build S3Mock with Docker Source: https://github.com/adobe/s3mock/blob/main/README.md Execute a full build of S3Mock, including the Docker image. This is the recommended approach for CI environments. ```shell make install ``` -------------------------------- ### Configure AWS SDK v2 S3Client for S3Mock (HTTPS) Source: https://context7.com/adobe/s3mock/llms.txt Build a synchronous S3Client pointing to S3Mock using path-style access, dummy credentials, and trusting all certificates for HTTPS connections. Ensure the Apache HTTP client is available. ```kotlin import software.amazon.awssdk.auth.credentials.AwsBasicCredentials import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider import software.amazon.awssdk.http.SdkHttpConfigurationOption import software.amazon.awssdk.http.apache.ApacheHttpClient import software.amazon.awssdk.regions.Region import software.amazon.awssdk.services.s3.S3Client import software.amazon.awssdk.services.s3.S3Configuration import software.amazon.awssdk.utils.AttributeMap import java.net.URI // Synchronous client (HTTPS, trust all certs) val s3Client: S3Client = S3Client.builder() .region(Region.US_EAST_1) .credentialsProvider( StaticCredentialsProvider.create(AwsBasicCredentials.create("foo", "bar")) ) .serviceConfiguration(S3Configuration.builder().pathStyleAccessEnabled(true).build()) .endpointOverride(URI.create("https://localhost:9191")) .httpClient( ApacheHttpClient.builder().buildWithDefaults( AttributeMap.builder() .put(SdkHttpConfigurationOption.TRUST_ALL_CERTIFICATES, true) .build() ) ) .build() ``` -------------------------------- ### Troubleshoot Compilation Errors Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Run this command to resolve compilation errors, skipping Docker and tests. This is useful when facing build issues. ```bash ./mvnw clean install -DskipDocker -DskipTests ``` -------------------------------- ### Initialize and Use S3MockContainer Source: https://github.com/adobe/s3mock/blob/main/testsupport/testcontainers/AGENTS.md Instantiate S3MockContainer with initial buckets and KMS keys. Connect to the container using either HTTP or HTTPS endpoints, with HTTPS requiring trust-all-certificates configuration. ```kotlin @Container val s3Mock = S3MockContainer("latest") .withInitialBuckets("test-bucket") .withValidKmsKeys("arn:aws:kms:us-east-1:1234567890:key/my-key") // Connect via HTTP (no TLS config needed): val s3Client = S3Client.builder() .endpointOverride(URI.create(s3Mock.httpEndpoint)) .build() // Or HTTPS (must trust the self-signed certificate): val s3Client = S3Client.builder() .endpointOverride(URI.create(s3Mock.httpsEndpoint)) .httpClient(UrlConnectionHttpClient.builder().buildWithDefaults( AttributeMap.builder().put(TRUST_ALL_CERTIFICATES, true).build() )) .build() ``` -------------------------------- ### Multipart Upload Operations Source: https://context7.com/adobe/s3mock/llms.txt Demonstrates the process of initiating a multipart upload, uploading individual parts, completing the upload, and handling potential errors by aborting the upload. It also shows how to list in-progress multipart uploads and uploaded parts. ```APIDOC ## Multipart Upload — CreateMultipartUpload, UploadPart, CompleteMultipartUpload Upload large objects in parts (each part must be ≥ 5 MB except the last). ```kotlin import software.amazon.awssdk.services.s3.model.CompletedPart import software.amazon.awssdk.services.s3.model.ChecksumAlgorithm val bucket = "my-bucket" val key = "large-file.bin" val FIVE_MB = 5 * 1024 * 1024 // 1. Initiate upload val createResponse = s3Client.createMultipartUpload { it.bucket(bucket) it.key(key) it.checksumAlgorithm(ChecksumAlgorithm.CRC32) } val uploadId = createResponse.uploadId() val completedParts = mutableListOf() try { // 2. Upload parts (each ≥ 5 MB except last) val part1Data = ByteArray(FIVE_MB + 1024) { it.toByte() } val part1 = s3Client.uploadPart( { it.bucket(bucket) it.key(key) it.uploadId(uploadId) it.partNumber(1) it.checksumAlgorithm(ChecksumAlgorithm.CRC32) }, software.amazon.awssdk.core.sync.RequestBody.fromBytes(part1Data) ) completedParts += CompletedPart.builder() .partNumber(1) .eTag(part1.eTag()) .checksumCRC32(part1.checksumCRC32()) .build() val part2Data = "final chunk".toByteArray() val part2 = s3Client.uploadPart( { it.bucket(bucket) it.key(key) it.uploadId(uploadId) it.partNumber(2) }, software.amazon.awssdk.core.sync.RequestBody.fromBytes(part2Data) ) completedParts += CompletedPart.builder().partNumber(2).eTag(part2.eTag()).build() // 3. Complete val completeResponse = s3Client.completeMultipartUpload { it.bucket(bucket) it.key(key) it.uploadId(uploadId) it.multipartUpload { mu -> mu.parts(completedParts) } } println("ETag: ${completeResponse.eTag()}") } catch (e: Exception) { // Abort on failure to free resources s3Client.abortMultipartUpload { it.bucket(bucket) it.key(key) it.uploadId(uploadId) } throw e } // List in-progress multipart uploads val uploads = s3Client.listMultipartUploads { it.bucket(bucket) } uploads.uploads().forEach { u -> println("uploadId=${u.uploadId()} key=${u.key()}") } // List uploaded parts s3Client.listParts { it.bucket(bucket) it.key(key) it.uploadId(uploadId) }.parts().forEach { p -> println("part ${p.partNumber()}: ${p.size()} bytes") } ``` ``` -------------------------------- ### Interact with S3Mock using cURL Source: https://context7.com/adobe/s3mock/llms.txt Utilize cURL commands to perform S3 operations against S3Mock, including PUT operations for creating buckets and uploading files, and GET operations for downloading files. ```shell # cURL curl -X PUT http://localhost:9090/my-bucket/ curl -X PUT --upload-file ./myfile http://localhost:9090/my-bucket/myfile curl http://localhost:9090/my-bucket/myfile -O curl --insecure https://localhost:9191/my-bucket/myfile -O ``` -------------------------------- ### Create Bucket with AWS CLI Source: https://github.com/adobe/s3mock/blob/main/README.md Use the AWS CLI to create a bucket in the S3Mock instance. Specify the endpoint URL to target S3Mock. ```shell aws s3api create-bucket --bucket my-bucket --endpoint-url http://localhost:9090 ``` -------------------------------- ### Run Full Check Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Perform a comprehensive check including linting, type checking, and unit tests. ```bash make check # lint + typecheck + unit tests ``` -------------------------------- ### Manage Object Versions Source: https://context7.com/adobe/s3mock/llms.txt After enabling versioning, you can put multiple versions of the same object key. Use listObjectVersions to retrieve all versions and getObjectAsBytes with a specific versionId to fetch a particular version. ```kotlin // Put two versions of the same key s3Client.putObject( { it.bucket("my-bucket").key("config.json") }, software.amazon.awssdk.core.sync.RequestBody.fromString("{\"version\":1}") ) s3Client.putObject( { it.bucket("my-bucket").key("config.json") }, software.amazon.awssdk.core.sync.RequestBody.fromString("{\"version\":2}") ) // Retrieve a specific version val versions = s3Client.listObjectVersions { it.bucket("my-bucket").prefix("config.json") } val oldVersionId = versions.versions().last().versionId() val oldContent = s3Client.getObjectAsBytes { it.bucket("my-bucket") it.key("config.json") it.versionId(oldVersionId) } println(oldContent.asUtf8String()) // {"version":1} ``` -------------------------------- ### ListObjectVersions Source: https://context7.com/adobe/s3mock/llms.txt Lists all versions of objects in a bucket, including regular versions and delete markers. Requires versioning to be enabled on the bucket. ```kotlin // List object versions (requires versioning enabled on bucket) val versionsResponse = s3Client.listObjectVersions { it.bucket("my-bucket") } versionsResponse.versions().forEach { v -> println("key=${v.key()} versionId=${v.versionId()} isLatest=${v.isLatest}") } versionsResponse.deleteMarkers().forEach { m -> println("delete-marker key=${m.key()} versionId=${m.versionId()}") } ``` -------------------------------- ### Running Unit Tests Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Command to execute only the unit tests for the S3Mock project. ```bash make test ``` -------------------------------- ### Configure AWS SDK v2 S3AsyncClient for S3Mock (HTTPS) Source: https://context7.com/adobe/s3mock/llms.txt Build an asynchronous S3Client using Netty for connections to S3Mock. This configuration includes path-style access, dummy credentials, and trusts all certificates for HTTPS. ```kotlin import software.amazon.awssdk.services.s3.S3AsyncClient import software.amazon.awssdk.http.nio.netty.NettyNioAsyncHttpClient import java.time.Duration import software.amazon.awssdk.auth.credentials.AwsBasicCredentials import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider import software.amazon.awssdk.http.SdkHttpConfigurationOption import software.amazon.awssdk.regions.Region import software.amazon.awssdk.services.s3.S3Configuration import software.amazon.awssdk.utils.AttributeMap import java.net.URI val s3AsyncClient: S3AsyncClient = S3AsyncClient.builder() .region(Region.US_EAST_1) .credentialsProvider( StaticCredentialsProvider.create(AwsBasicCredentials.create("foo", "bar")) ) .forcePathStyle(true) .endpointOverride(URI.create("https://localhost:9191")) .httpClient( NettyNioAsyncHttpClient.builder() .connectionTimeout(Duration.ofMinutes(5)) .maxConcurrency(100) .buildWithDefaults( AttributeMap.builder() .put(SdkHttpConfigurationOption.TRUST_ALL_CERTIFICATES, true) .build() ) ) .multipartEnabled(true) .build() ``` -------------------------------- ### Low-level Multipart Upload with S3 Client Source: https://context7.com/adobe/s3mock/llms.txt Initiate, upload parts, and complete a multipart upload. Abort the upload if an error occurs. Lists in-progress multipart uploads and uploaded parts. ```kotlin import software.amazon.awssdk.services.s3.model.CompletedPart import software.amazon.awssdk.services.s3.model.ChecksumAlgorithm val bucket = "my-bucket" val key = "large-file.bin" val FIVE_MB = 5 * 1024 * 1024 // 1. Initiate upload val createResponse = s3Client.createMultipartUpload { it.bucket(bucket) it.key(key) it.checksumAlgorithm(ChecksumAlgorithm.CRC32) } val uploadId = createResponse.uploadId() val completedParts = mutableListOf() try { // 2. Upload parts (each ≥ 5 MB except last) val part1Data = ByteArray(FIVE_MB + 1024) { it.toByte() } val part1 = s3Client.uploadPart( { it.bucket(bucket) it.key(key) it.uploadId(uploadId) it.partNumber(1) it.checksumAlgorithm(ChecksumAlgorithm.CRC32) }, software.amazon.awssdk.core.sync.RequestBody.fromBytes(part1Data) ) completedParts += CompletedPart.builder() .partNumber(1) .eTag(part1.eTag()) .checksumCRC32(part1.checksumCRC32()) .build() val part2Data = "final chunk".toByteArray() val part2 = s3Client.uploadPart( { it.bucket(bucket) it.key(key) it.uploadId(uploadId) it.partNumber(2) }, software.amazon.awssdk.core.sync.RequestBody.fromBytes(part2Data) ) completedParts += CompletedPart.builder().partNumber(2).eTag(part2.eTag()).build() // 3. Complete val completeResponse = s3Client.completeMultipartUpload { it.bucket(bucket) it.key(key) it.uploadId(uploadId) it.multipartUpload { mu -> mu.parts(completedParts) } } println("ETag: ${completeResponse.eTag()}") } catch (e: Exception) { // Abort on failure to free resources s3Client.abortMultipartUpload { it.bucket(bucket) it.key(key) it.uploadId(uploadId) } throw e } // List in-progress multipart uploads val uploads = s3Client.listMultipartUploads { it.bucket(bucket) } uploads.uploads().forEach { u -> println("uploadId=${u.uploadId()} key=${u.key()}") } // List uploaded parts s3Client.listParts { it.bucket(bucket) it.key(key) it.uploadId(uploadId) }.parts().forEach { p -> println("part ${p.partNumber()}: ${p.size()} bytes") } ``` -------------------------------- ### Run Unit Tests Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Execute only the unit tests for S3Mock. This does not require Docker. ```bash make test # Unit tests only (no Docker required) ``` -------------------------------- ### S3 Bucket Operations with AWS SDK v2 Source: https://context7.com/adobe/s3mock/llms.txt Perform core bucket lifecycle operations including creating standard and directory buckets, waiting for their existence, checking metadata with headBucket, listing all buckets, and deleting empty buckets. Ensure the S3Client is configured. ```kotlin import software.amazon.awssdk.services.s3.model.BucketType import software.amazon.awssdk.services.s3.model.NoSuchBucketException // Create a standard bucket s3Client.createBucket { it.bucket("my-bucket") } // Create a directory bucket s3Client.createBucket { it.bucket("my-dir-bucket") it.createBucketConfiguration { cfg.bucket { b -> b.type(BucketType.DIRECTORY) } } } // Wait for bucket existence s3Client.waiter().waitUntilBucketExists { it.bucket("my-bucket") } // Head bucket (check existence / metadata) val headResponse = s3Client.headBucket { it.bucket("my-bucket") } println(headResponse.sdkHttpResponse().statusCode()) // 200 // List all buckets val buckets = s3Client.listBuckets().buckets() buckets.forEach { println(it.name()) } // Delete bucket (must be empty first) s3Client.deleteBucket { it.bucket("my-bucket") } // Wait until gone s3Client.waiter().waitUntilBucketNotExists { it.bucket("my-bucket") } ``` -------------------------------- ### Configure S3Mock Valid KMS Keys Source: https://context7.com/adobe/s3mock/llms.txt Provide a comma-separated list of KMS key ARNs for SSE-KMS validation (no actual encryption) using COM_ADOBE_TESTING_S3MOCK_STORE_VALID_KMS_KEYS. ```shell # Comma-separated KMS key ARNs accepted for SSE-KMS (validation only, no encryption) COM_ADOBE_TESTING_S3MOCK_STORE_VALID_KMS_KEYS=arn:aws:kms:us-east-1:1234567890:key/my-key ``` -------------------------------- ### Running Specific Integration Test Class Source: https://github.com/adobe/s3mock/blob/main/docs/TESTING.md Command to run a specific integration test class, identified by its name. Skips Docker if specified. ```bash ./mvnw verify -pl integration-tests -Dit.test=BucketIT ``` -------------------------------- ### Run S3Mock with Docker and Persistent Storage Source: https://github.com/adobe/s3mock/blob/main/docs/SETUP.md Run S3Mock using Docker, configuring persistent storage by mounting a local directory to the container's data volume and setting environment variables for storage root and file retention. ```bash docker run \ -p 9090:9090 -p 9191:9191 \ -e COM_ADOBE_TESTING_S3MOCK_STORE_ROOT=/data \ -e COM_ADOBE_TESTING_S3MOCK_STORE_RETAIN_FILES_ON_EXIT=true \ -v /local/path:/data \ adobe/s3mock ``` -------------------------------- ### PUT Object Data Flow Source: https://github.com/adobe/s3mock/blob/main/docs/ARCHITECTURE.md Illustrates the step-by-step process for a PUT Object request, from the AWS SDK client to the S3Mock server's storage and response. ```markdown AWS SDK v2 Client │ PUT /{bucket}/{key} (path-style) ▼ Tomcat — encoded slashes decoded, backslashes allowed (S3MockConfiguration) ▼ KmsValidationFilter — validates x-amz-server-side-encryption-aws-kms-key-id if present; rejects unknown keys with 400 InvalidArgument ▼ ObjectController.putObject() — HTTP mapping only, no logic ▼ ObjectService.putObject() — validates bucket exists (BucketStore) — decodes AWS chunked transfer encoding (AwsChunkedDecodingChecksumInputStream) — computes ETag via DigestUtil.hexDigest() — verifies optional client-supplied checksum ▼ ObjectStore.storeS3ObjectMetadata() — acquires per-object lock: synchronized(lockStore[uuid]) — writes binary data → ///binaryData — writes metadata → ///objectMetadata.json — if versioning enabled: writes version file alongside existing data ▼ BucketStore.addKeyToBucket() — acquires per-bucket lock: synchronized(lockStore[bucketName]) — updates key→UUID mapping in bucketMetadata.json ▼ ObjectController — returns ETag header, 200 OK ``` -------------------------------- ### Enable Actuator with Minimal Logging Source: https://github.com/adobe/s3mock/blob/main/README.md Activate the 'actuator' profile to enable Spring Boot Actuator endpoints without additional verbose logging. ```shell docker run -p 9090:9090 -p 9191:9191 -e SPRING_PROFILES_ACTIVE=actuator adobe/s3mock ``` -------------------------------- ### Run S3Mock with Docker (Configuration) Source: https://github.com/adobe/s3mock/blob/main/README.md Launches the S3Mock Docker container with custom environment variables for initial buckets and debug profile. Ports 9090 (HTTP) and 9191 (HTTPS) are mapped. ```shell docker run -p 9090:9090 -p 9191:9191 \ -e COM_ADOBE_TESTING_S3MOCK_STORE_INITIAL_BUCKETS=test-bucket \ -e SPRING_PROFILES_ACTIVE=debug \ adobe/s3mock ``` -------------------------------- ### Manage Bucket Lifecycle Configuration Source: https://context7.com/adobe/s3mock/llms.txt Configure, read, and delete lifecycle rules for buckets using the S3 client. Rules can specify expiration based on days or prefixes. ```kotlin import software.amazon.awssdk.services.s3.model.BucketLifecycleConfiguration import software.amazon.awssdk.services.s3.model.LifecycleRule import software.amazon.awssdk.services.s3.model.LifecycleExpiration import software.amazon.awssdk.services.s3.model.LifecycleRuleFilter import software.amazon.awssdk.services.s3.model.ExpirationStatus // Set lifecycle rules s3Client.putBucketLifecycleConfiguration { it.bucket("my-bucket") it.lifecycleConfiguration( BucketLifecycleConfiguration.builder() .rules( LifecycleRule.builder() .id("expire-logs-after-30-days") .filter(LifecycleRuleFilter.builder().prefix("logs/").build()) .expiration(LifecycleExpiration.builder().days(30).build()) .status(ExpirationStatus.ENABLED) .build() ) .build() ) } // Read lifecycle rules val lifecycle = s3Client.getBucketLifecycleConfiguration { it.bucket("my-bucket") } lifecycle.rules().forEach { rule -> println("id=${rule.id()} prefix=${rule.filter().prefix()} days=${rule.expiration().days()}") } // Delete lifecycle configuration s3Client.deleteBucketLifecycle { it.bucket("my-bucket") } ``` -------------------------------- ### Configure Dockerized S3Mock with Environment Variables Source: https://context7.com/adobe/s3mock/llms.txt Customize S3Mock behavior in Docker using environment variables for initial buckets, debug logging, and storage configuration. ```shell docker run -p 9090:9090 -p 9191:9191 \ -e COM_ADOBE_TESTING_S3MOCK_STORE_INITIAL_BUCKETS=bucket1,bucket2 \ -e SPRING_PROFILES_ACTIVE=debug \ adobe/s3mock ``` -------------------------------- ### Enable Actuator via Environment Variables Source: https://github.com/adobe/s3mock/blob/main/README.md Configure actuator endpoints to be unrestricted and exposed by including all endpoints using environment variables, bypassing Spring profiles. ```shell docker run -p 9090:9090 -p 9191:9191 \ -e MANAGEMENT_ENDPOINTS_ACCESS_DEFAULT=unrestricted \ -e MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE='*' \ adobe/s3mock ``` -------------------------------- ### HeadBucket Source: https://github.com/adobe/s3mock/blob/main/README.md Confirms that the specified bucket exists and you have permission to access it. ```APIDOC ## HEAD Bucket ### Description Confirms that the specified bucket exists and you have permission to access it. ### Method HEAD ### Endpoint /{bucketName} ``` -------------------------------- ### ListObjectVersions Source: https://github.com/adobe/s3mock/blob/main/README.md Returns metadata about all versions of the objects in a bucket. ```APIDOC ## GET ListObjectVersions ### Description Returns metadata about all versions of the objects in a bucket. ### Method GET ### Endpoint /{bucketName}?versions ``` -------------------------------- ### ListObjectsV2 Source: https://context7.com/adobe/s3mock/llms.txt Lists all objects in a bucket. The response includes object key, size, and last modified timestamp. ```kotlin // List all objects val response = s3Client.listObjectsV2 { it.bucket("my-bucket") } response.contents().forEach { obj -> println("${obj.key()} — ${obj.size()} bytes — ${obj.lastModified()}") } ``` -------------------------------- ### Enable Actuator with Debug Logging Source: https://github.com/adobe/s3mock/blob/main/README.md Use the 'debug' profile to enable verbose logging for Spring Web, Apache, and request details, along with actuator endpoints. ```shell docker run -p 9090:9090 -p 9191:9191 -e SPRING_PROFILES_ACTIVE=debug adobe/s3mock ```