### Configure Gateway with Sidecar Metadata Source: https://github.com/versity/versitygw/wiki/POSIX-metadata Example of starting the gateway with the sidecar metadata option enabled. Ensure the sidecar directory is accessible and distinct from the object data. ```bash versitygw -a test -s test posix --sidecar /tmp/sidecar /tmp/gw ``` -------------------------------- ### Install Versity Gateway with Helm Source: https://github.com/versity/versitygw/wiki/Docker Installs Versity Gateway with a POSIX backend, persistent volume claim, and WebUI enabled. Use this for basic setup. ```bash helm install versitygw oci://ghcr.io/versity/versitygw/charts/versitygw \ --set auth.accessKey=myaccesskey \ --set auth.secretKey=mysecretkey \ --set gateway.backend.type=posix \ --set persistence.enabled=true \ --set admin.enabled=true \ --set webui.enabled=true ``` -------------------------------- ### Install Moq Source: https://github.com/versity/versitygw/wiki/Running-Test-Suite Install the moq tool, which is used for generating mock test definitions. ```bash go install github.com/matryer/moq@latest ``` -------------------------------- ### Load Example Bucket Policy Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Populates the policy JSON editor with a predefined example policy structure. This is useful for users who need a starting point for their bucket policy. The example policy grants full S3 access to specified users. ```javascript function loadExamplePolicy() { if (!currentPolicyBucket) return; const examplePolicy = { "Version": "2012-10-17", "Statement": [ { "Sid": "FullS3Access", "Effect": "Allow", "Principal": [ "user001", "user002" ], "Action": [ "s3:AbortMultipartUpload", "s3:DeleteObject", "s3:DeleteObjectTagging", "s3:GetBucketAcl", "s3:GetBucketPolicy", "s3:GetBucketTagging", "s3:GetBucketLocation", "s3:GetBucketCORS", "s3:GetObject", "s3:GetObjectAcl", "s3:GetObjectAttributes", "s3:GetObjectRetention", "s3:GetObjectTagging", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:ListMultipartUploadParts", "s3:PutBucketTagging", "s3:PutObject", "s3:PutObjectRetention", "s3:PutObjectTagging", "s3:PutBucketWebsite", "s3:GetBucketWebsite", "s3:RestoreObject" ], "Resource": [ `arn:aws:s3:::${currentPolicyBucket}`, `arn:aws:s3:::${currentPolicyBucket}/*` ] } ] }; document.getElementById('policy-json-editor').value = JSON.stringify(examplePolicy, null, 2); showPolicyStatusMessage('Example policy loaded. Modify as needed and click Save.', 'info'); } ``` -------------------------------- ### Enable and Start VersityGW Service Source: https://github.com/versity/versitygw/wiki/ZFS-Use-Case Enables the VersityGW service to start on boot and starts it immediately. This command assumes the configuration file is correctly placed at `/etc/versitygw.d/zfs.conf`. ```bash systemctl enable --now versitygw@zfs.service ``` -------------------------------- ### Start Versity Gateway with CLI Options Source: https://github.com/versity/versitygw/wiki/Quickstart Start the Versity Gateway using command-line arguments for access key, secret key, and configuration. Enables web UI and POSIX backend. ```bash versitygw --access myaccess --secret mysecret --iam-dir /tmp/vgw --webui :8080 posix posix --versioning-dir /tmp/vers /tmp/gw ``` -------------------------------- ### Enabling VersityGW WebGUI with a Specific Port Source: https://github.com/versity/versitygw/wiki/WebGUI Example command to start VersityGW with the WebGUI enabled on port 7071. ```bash versitygw --access test --secret test --iam-dir /tmp/gw --webui :7071 posix --versioning-dir /tmp/vers /tmp/gw ``` -------------------------------- ### Setup XFS Filesystem Source: https://github.com/versity/versitygw/wiki/MultiPart-Optimizations Commands to format and mount an XFS filesystem for testing. ```bash mkfs.xfs -f /dev/md0 mkdir /mnt/xfs mount /dev/md0 /mnt/xfs ``` -------------------------------- ### Setup ext4 Filesystem Source: https://github.com/versity/versitygw/wiki/MultiPart-Optimizations Commands to format and mount an ext4 filesystem for testing. ```bash mkfs.ext4 /dev/md0 mkdir /mnt/ext4 mount /dev/md0 /mnt/ext4 ``` -------------------------------- ### Start Versity Gateway Service Source: https://github.com/versity/versitygw/wiki/Linux-Install Start a specific Versity Gateway service instance named 'mygateway' using systemd. ```bash systemctl start versitygw@mygateway ``` -------------------------------- ### Start Versity Gateway with Versioning Enabled Source: https://github.com/versity/versitygw/wiki/POSIX-versioning Start the gateway with versioning support enabled by creating the necessary directories and running the versitygw command with the `--versioning-dir` option. This sets up a primary namespace and a separate namespace for non-current versions. ```bash mkdir -p /tmp/gw /tmp/versioning versitygw -a sccesskey -s secretkey posix --versioning-dir /tmp/versioning /tmp/gw ``` -------------------------------- ### Start Versitygw with Vault IAM (Command Flags) Source: https://github.com/versity/versitygw/wiki/IAM-Vault Starts the versitygw service using command-line flags to configure Vault IAM. This example specifies the Vault endpoint, root token, and secret storage path. ```bash # versitygw --iam-vault-endpoint-url http://127.0.0.1:8200 --iam-vault-root-token hvs.dqw7YKwMlvXmsxICYA0N7Krs --iam-vault-secret-storage-path veristygw --access test --secret test posix /tmp/gw ``` -------------------------------- ### Run Tests with Docker Compose (Default) Source: https://github.com/versity/versitygw/wiki/Running-Test-Suite Execute the test suite using docker-compose. This command assumes a standard setup and will start the 'direct' service. ```bash docker-compose -f tests/docker-compose-bats.yml --project-directory . up direct ``` -------------------------------- ### Start Vault Development Server Source: https://github.com/versity/versitygw/wiki/IAM-Vault Command to start a local Vault development server. This is useful for testing and examples, but not recommended for production environments. The output will provide a root token. ```bash # vault server -dev ``` -------------------------------- ### Install Fish Completion File Source: https://github.com/versity/versitygw/wiki/Linux-Install Copy the Versity Gateway Fish completion file to the appropriate directory for Fish shell completion. ```fish cp extra/versitygw.fish ~/.config/fish/completions/versitygw.fish ``` -------------------------------- ### Enable Versity Gateway Service on Boot Source: https://github.com/versity/versitygw/wiki/Linux-Install Configure a specific Versity Gateway service instance named 'mygateway' to start automatically on system boot. ```bash systemctl enable versitygw@mygateway ``` -------------------------------- ### List Buckets REST Script Example Source: https://github.com/versity/versitygw/blob/main/tests/README.md Example of how to execute the list_buckets.sh script. Ensure AWS credentials, endpoint URL, and output file are set. ```bash AWS_ACCESS_KEY_ID={id} AWS_SECRET_ACCESS_KEY={key} AWS_ENDPOINT_URL=https://s3.amazonaws.com OUTPUT_FILE=./output_file.xml ./tests/rest_scripts/list_buckets.sh ``` -------------------------------- ### Start Docker Compose and Create Directories Source: https://github.com/versity/versitygw/wiki/Docker Creates necessary data directories and starts the Docker Compose services in detached mode. ```bash mkdir -p ./data/s3 ./data/versioning ./data/iam docker compose up -d ``` -------------------------------- ### Start Versity Gateway with Environment Variables Source: https://github.com/versity/versitygw/wiki/Quickstart Start the Versity Gateway using environment variables for root credentials. Enables the web UI on port 8080 and POSIX backend. ```bash ROOT_ACCESS_KEY=myaccess ROOT_SECRET_KEY=mysecret ./versitygw --iam-dir /tmp/vgw --webui :8080 posix --versioning-dir /tmp/vers /tmp/vgw ``` -------------------------------- ### Install Versity Gateway with POSIX Backend Source: https://github.com/versity/versitygw/blob/main/chart/README.md Basic installation of the versitygw Helm chart for single-user mode with a POSIX backend. Ensure you have Kubernetes 1.19+ and Helm 3.8+ installed. ```bash helm install my-versitygw oci://ghcr.io/versity/versitygw/charts/versitygw \ --set auth.accessKey=myaccesskey \ --set auth.secretKey=mysecretkey \ --set gateway.backend.type=posix \ --set persistence.enabled=true ``` -------------------------------- ### Example Usage of Noop Plugin Source: https://github.com/versity/versitygw/blob/main/plugins/noop/README.md An example command to run the Versity Gateway with the noop plugin, specifying access and secret keys. The gateway will then respond to S3 API requests without persisting data. ```sh versitygw --access myaccesskey --secret mysecretkey plugin ./noop.so ``` -------------------------------- ### Install Bash Completion Script (System-wide) Source: https://github.com/versity/versitygw/wiki/Linux-Install Copy the Versity Gateway Bash completion script to the system-wide completion directory for all users. ```bash sudo cp extra/versitygw.sh /etc/bash_completion.d/versitygw ``` -------------------------------- ### Install Versity Gateway with Custom Values Source: https://github.com/versity/versitygw/wiki/Docker Installs Versity Gateway using a custom values.yaml file for advanced configurations like multi-user deployments. ```bash helm install versitygw oci://ghcr.io/versity/versitygw/charts/versitygw \ -f values.yaml ``` -------------------------------- ### Install ZFS on AlmaLinux Source: https://github.com/versity/versitygw/wiki/ZFS-Use-Case Installs the ZFS package from the official repository. Ensure you are using a compatible Linux distribution. ```bash dnf install https://zfsonlinux.org/epel/zfs-release-2-2$(rpm --eval "%{\dist}").noarch.rpm dnf install zfs ``` -------------------------------- ### Start Azurite and Versity Gateway with Docker Compose (Direct Command) Source: https://github.com/versity/versitygw/wiki/AzureBlob-Backend Alternative Docker Compose command to start Azurite and the versitygw with the Azure backend. Ensure to specify the project directory and environment file. ```bash docker-compose -f tests/docker-compose.yml --env-file .env.dev --project-directory . up azurite azuritegw ``` -------------------------------- ### Install RPM Package Source: https://github.com/versity/versitygw/wiki/Linux-Install Use this command to install the Versity Gateway RPM package on Red Hat-based systems. ```bash sudo dnf install ./versitygw_*.rpm ``` -------------------------------- ### Minimal POSIX Backend Configuration Source: https://github.com/versity/versitygw/wiki/Linux-Install A minimal configuration example for a Versity Gateway service using a POSIX backend. Ensure to replace placeholder values. ```ini VGW_BACKEND=posix VGW_BACKEND_ARG= ROOT_ACCESS_KEY_ID= ROOT_SECRET_ACCESS_KEY= ``` -------------------------------- ### Example Output of Multipart Upload Test Source: https://github.com/versity/versitygw/wiki/MultiPart-Optimizations This is an example of the output from the bash script, showing the progress of uploading parts and the final completion of the multipart upload, including the time taken. ```text make_bucket: testbucket create multipart upload uploading part 1 { "ETag": "cd573cfaace07e7949bc0c46028904ff" } uploading part 2 { "ETag": "cd573cfaace07e7949bc0c46028904ff" } uploading part 3 { "ETag": "cd573cfaace07e7949bc0c46028904ff" } uploading part 4 { "ETag": "cd573cfaace07e7949bc0c46028904ff" } uploading part 5 { "ETag": "cd573cfaace07e7949bc0c46028904ff" } complete multipart upload { "Bucket": "testbucket", "Key": "testkey", "ETag": "306cca04302861ed2620a328f286346f-5" } real 0m0.994s user 0m0.305s sys 0m0.066s ``` -------------------------------- ### Test Setup: mdadm RAID5 Configuration Source: https://github.com/versity/versitygw/wiki/MultiPart-Optimizations Demonstrates the command used to create a software RAID5 device with mdadm, which serves as the block device for filesystem testing. ```bash mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/vdb /dev/vdc /dev/vdd /dev/vde ``` -------------------------------- ### versitygw Admin CLI Create Bucket Source: https://github.com/versity/versitygw/wiki/Admin-APIs Example of how to create a bucket using the versitygw admin CLI. Ensure you replace placeholders with your actual access keys, endpoint, and owner ID. ```bash versitygw admin -a -s \ -er http://127.0.0.1:7070 create-bucket \ --owner \ --bucket test-bucket ``` -------------------------------- ### Run Tests with Docker Compose (Arm64) Source: https://github.com/versity/versitygw/wiki/Running-Test-Suite Build and run the test suite for arm64 platforms using docker-compose. This involves building the image first, then starting the 'direct' service. ```bash docker-compose -f tests/docker-compose-bats.yml build direct docker-compose -f tests/docker-compose-bats.yml --project-directory . up direct ``` -------------------------------- ### Start Azurite and Versity Gateway with Docker Compose Source: https://github.com/versity/versitygw/wiki/AzureBlob-Backend Command to launch Azurite (for local Azure testing) and the versitygw configured with the Azure backend using Docker Compose. ```bash make up-azurite ``` -------------------------------- ### Configure Gateway with Webhook URL Source: https://github.com/versity/versitygw/wiki/Webhook-log-entries Command to start the gateway and configure it to send logs to a specified webhook URL. Replace http://localhost:8080/s3logs with your actual webhook endpoint. ```bash ./versitygw -a user -s password --log-webhook-url http://localhost:8080/s3logs posix /tmp/gw ``` -------------------------------- ### Run Specific REST Bucket Test Source: https://github.com/versity/versitygw/blob/main/tests/README.md An example of running a specific test case, 'REST - HeadBucket', from the REST bucket test suite using a specified environment file. ```bash VERSITYGW_TEST_ENV=tests/.env tests/test_rest_bucket.sh -f "REST - HeadBucket" ``` -------------------------------- ### Programmatic Create Bucket with Go Source: https://github.com/versity/versitygw/wiki/Admin-APIs Demonstrates how to programmatically create a bucket using Go, including constructing the XML configuration, signing the request with AWS v4, and handling responses. Requires AWS SDK for Go. ```go package main import ( "bytes" "crypto/sha256" "encoding/hex" "encoding/xml" "fmt" "io" "log" "net/http" "time" "github.com/aws/aws-sdk-go-v2/aws" v4 "github.com/aws/aws-sdk-go-v2/aws/signer/v4" "github.com/aws/smithy-go" ) const ( adminEndpoint = "http://127.0.0.1:7070" adminAccess = "admin_access_key_id" adminSecret = "admin_secret_access_key" serverRegion = "us-east-1" ) type Tag struct { Key string `xml:"Key"` Value string `xml:"Value"` } type CreateBucketConfiguration struct { XMLName xml.Name `xml:"CreateBucketConfiguration"` Xmlns string `xml:"xmlns,attr"` LocationConstraint string `xml:"LocationConstraint,omitempty"` Tags []Tag `xml:"Tags>Tag,omitempty"` } func main() { bucket := "test-bucket" owner := "owner_access_key_id" cfg := CreateBucketConfiguration{ Xmlns: "http://s3.amazonaws.com/doc/2006-03-01/", LocationConstraint: "us-east-1", Tags: []Tag{ {Key: "env", Value: "test"}, }, } cfgxml, err := xml.Marshal(cfg) if err != nil { log.Fatalf("failed to serialize config: %v\n", err) } req, err := http.NewRequest( http.MethodPatch, fmt.Sprintf("%v/%v/create", adminEndpoint, bucket), bytes.NewBuffer(cfgxml), ) if err != nil { log.Fatalf("failed to create request: %v\n", err) } req.Header.Set("x-vgw-owner", owner) hashedPayload := sha256.Sum256(cfgxml) payloadHash := hex.EncodeToString(hashedPayload[:]) req.Header.Set("X-Amz-Content-Sha256", payloadHash) signer := v4.NewSigner() if err := signer.SignHTTP( req.Context(), aws.Credentials{AccessKeyID: adminAccess, SecretAccessKey: adminSecret}, req, payloadHash, "s3", serverRegion, time.Now(), ); err != nil { log.Fatalf("failed to sign the request: %v\n", err) } client := &http.Client{} resp, err := client.Do(req) if err != nil { log.Fatalf("failed to send the request: %v\n", err) } defer resp.Body.Close() if resp.StatusCode >= 400 { body, _ := io.ReadAll(resp.Body) log.Fatalln(parseApiError(body)) } } func parseApiError(body []byte) error { var apiErr smithy.GenericAPIError if err := xml.Unmarshal(body, &apiErr); err != nil { apiErr.Code = "InternalServerError" apiErr.Message = err.Error() } return &apiErr } ``` -------------------------------- ### Versity Gateway TLS Configuration with Cert-Manager Source: https://github.com/versity/versitygw/wiki/Docker Configuration values for enabling TLS with cert-manager for automatic certificate provisioning. This setup requires cert-manager to be installed and a ClusterIssuer configured. ```yaml # values.yaml auth: existingSecret: versitygw-creds gateway: backend: type: posix iam: enabled: true admin: enabled: true port: 7071 webui: enabled: true port: 8080 apiGateways: - https://s3.example.com adminGateways: - https://s3-admin.example.com persistence: enabled: true tls: enabled: true certificate: create: true dnsNames: - s3.example.com - s3-admin.example.com - versitygw.example.com issuerRef: group: cert-manager.io kind: ClusterIssuer name: letsencrypt-production ingress: enabled: true className: nginx hosts: - host: s3.example.com paths: - path: / pathType: Prefix servicePort: s3-api - host: s3-admin.example.com paths: - path: / pathType: Prefix servicePort: admin - host: versitygw.example.com paths: - path: / pathType: Prefix servicePort: webui tls: - secretName: versitygw-tls hosts: - s3.example.com - s3-admin.example.com - versitygw.example.com ``` -------------------------------- ### Create Bucket and Assign Owner with AWS CLI and versitygw CLI Source: https://github.com/versity/versitygw/wiki/Multi-Tenant This example demonstrates creating a bucket using the AWS CLI with admin credentials, then changing its owner to a specific user using the versitygw CLI. Ensure environment variables are set for AWS CLI. ```bash export AWS_ACCESS_KEY_ID=myadmin export AWS_SECRET_ACCESS_KEY=mysecret export AWS_ENDPOINT_URL=http://127.0.0.1:7070 aws s3 mb s3://bucket1 versitygw admin --access myaccess --secret mysecret --endpoint-url http://127.0.0.1:7070 change-bucket-owner --bucket bucket1 --owner myuser ``` -------------------------------- ### CPU Profile with go tool pprof Source: https://github.com/versity/versitygw/wiki/Debugging Use the 'go tool pprof' command to grab and display a CPU profile for a specified duration. This example captures a 30-second profile and serves it via a local webserver on port 9999. ```bash go tool pprof -http :9999 'http://localhost:6060/debug/pprof/profile?seconds=30' ``` -------------------------------- ### Install DEB Package Source: https://github.com/versity/versitygw/wiki/Linux-Install Use this command to install the Versity Gateway DEB package on Debian-based systems. ```bash sudo dpkg -i ./versitygw_*.deb ``` -------------------------------- ### Build Versity Gateway from Source Source: https://github.com/versity/versitygw/wiki/Quickstart Build the Versity Gateway project after cloning the repository. Requires a Go compiler and Make. ```bash cd versitygw make ``` -------------------------------- ### Build and Run Webhook Server Source: https://github.com/versity/versitygw/wiki/Webhook-log-entries Commands to compile and run the Go webhook server. Ensure the server is running before configuring the gateway. ```bash go build -o webhooktest ./webhooktest ``` -------------------------------- ### Initialize UI and Load Buckets Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Initializes the UI by checking the URL hash for bucket and prefix information, then loads the list of buckets. Handles both admin and regular user roles. ```javascript let currentBucket = null; let currentPrefix = ''; let objects = []; let folders = []; let bucketsList = []; let selectedObjects = new Set(); let currentObjectKey = null; // For info modal let searchPrefix = ''; // API prefix filter // Pagination state let currentPage = 1; let loadObjectsAbortController = null; let pageSize = 10; let continuationTokens = [null]; // continuationTokens[i] = S3 token needed to fetch page i+1 let isTruncated = false; // Versioning state let showVersions = false; let currentBucketVersioningStatus = null; let objectVersions = []; let objectVersionCounts = {}; // Maps object key to version count let deletedObjectsWithVersions = []; // Objects with delete marker but older versions exist let bucketsVersioningCache = {}; let bucketsObjectLockCache = {}; // Auth guard if (!requireAuth()) { // Will redirect to login } else { // Clear history state on startup (can happen when reloading with popup open) if (history.state?.modal) history.back(); initSidebarWithRole(); updateUserInfo(); init(); setupDragDrop(); } // ============================================ // Initialization // ============================================ async function init() { // Check URL hash for bucket/prefix // decodeURIComponent is needed because browsers percent-encode characters // like spaces when storing the hash const hash = decodeURIComponent(window.location.hash.slice(1)); if (hash) { const parts = hash.split('/'); const bucketName = parts[0]; if (bucketName) { currentBucket = bucketName; currentPrefix = parts.slice(1).join('/'); if (currentPrefix && !currentPrefix.endsWith('/')) { currentPrefix += '/'; } } } else { currentBucket = ""; currentPrefix = ""; } await loadBuckets(); } // Rerun init() on hash change to allow for back/forward when browsing a bucket. window.addEventListener('hashchange', async () => { await init(); }); // ============================================ // Logout // ============================================ function logout() { confirm('Are you sure you want to sign out?', () => { api.logout(); window.location.href = 'index.html'; }); } // ============================================ // Bucket Loading // ============================================ async function loadBuckets() { const tbody = document.getElementById('buckets-table'); tbody.innerHTML = `

Loading buckets...

`; try { if (api.isAdmin()) { // Admin: use admin API to get all buckets with owner info bucketsList = await api.listBuckets(); } else { // Regular user: use S3 API to get accessible buckets bucketsList = await api.listBucketsS3(); } // Load versioning status for all buckets await loadAllBucketsVersioning(); // Load object lock configuration for all buckets await loadAllBucketsObjectLock(); // If we have a bucket from URL, show it if (currentBucket) { showObjectsView(); await loadObjects(); } else { showBucketsView(); renderBuckets(); } } catch (error) { console.error('Error loading buckets:', error); showToast('Error loading buckets: ' + error.message, 'error'); tbody.innerHTML = `

Error loading buckets

${escapeHtml(error.message)}

`; } } async function loadAllBucketsVersioning() { const promises = bucketsList.map(async (bucket) => { const name = bucket.name || bucket.Name; try { const result = await api.getBucketVersioning(name); bucketsVersioningCache[name] = result.status || ''; } catch (error) { console.error(`Error loading versioning for ${name}:`, error); bucketsVersioningCache[name] = 'Unknown'; } }); await Promise.all(promises); } async function loadAllBucketsObjectLock() { const promises = bucketsList.map(async (bucket) => { const name = bucket.name || bucket.Name; try { const result = await api.getBucketObjectLockConfiguration(name); bucketsObjectLockCache[name] = result; } catch (error) { console.error(`Error loading object lock for ${name}:`, error); bucketsObjectLockCache[name] = { enabled: false, mode: '', days: null, years: null }; } }); await Promise.all(promises); } function renderBuckets() { const tbody = ``` -------------------------------- ### Install VersityGW on Kubernetes with Helm Source: https://github.com/versity/versitygw/blob/main/README.md Deploys VersityGW to a Kubernetes cluster using the provided Helm chart. This command initiates the installation process for the chart. ```bash helm install versitygw oci://ghcr.io/versity/versitygw/charts/versitygw ``` -------------------------------- ### Test Versity Gateway with AWS CLI Source: https://github.com/versity/versitygw/wiki/Docker Demonstrates creating a bucket, uploading a file, and listing objects using the AWS CLI against the running versitygw instance. ```bash export AWS_ACCESS_KEY_ID=myaccesskey export AWS_SECRET_ACCESS_KEY=mysecretkey # Create a bucket aws --endpoint-url http://127.0.0.1:7070 s3 mb s3://mybucket # Upload a file aws --endpoint-url http://127.0.0.1:7070 s3 cp myfile.txt s3://mybucket/myfile.txt # List objects aws --endpoint-url http://127.0.0.1:7070 s3 ls s3://mybucket ``` -------------------------------- ### Install Versity Gateway using Existing Secret Source: https://github.com/versity/versitygw/wiki/Docker Installs Versity Gateway using a pre-created Kubernetes Secret for authentication credentials. This is the recommended approach for production. ```bash helm install versitygw oci://ghcr.io/versity/versitygw/charts/versitygw \ --set auth.existingSecret=versitygw-creds \ --set gateway.backend.type=posix \ --set persistence.enabled=true \ --set admin.enabled=true \ --set webui.enabled=true ``` -------------------------------- ### Start Versitygw with Vault IAM (Environment Variables) Source: https://github.com/versity/versitygw/wiki/IAM-Vault Starts the versitygw service using environment variables for Vault IAM configuration. This method is an alternative to using command-line flags. ```bash # export VGW_IAM_VAULT_ENDPOINT_URL=http://127.0.0.1:8200 # export VGW_IAM_VAULT_ROOT_TOKEN=hvs.dqw7YKwMlvXmsxICYA0N7Krs # export VGW_IAM_VAULT_SECRET_STORAGE_PATH=versitygw # versitygw --access test --secret test posix /tmp/gw ``` -------------------------------- ### Example Webhook Log Entry Source: https://github.com/versity/versitygw/wiki/Webhook-log-entries An example of a JSON-encoded log entry sent by the gateway to the webhook URL. These logs contain fields similar to AWS S3 server logs. ```json {"BucketOwner":"acct1","Bucket":"bucket1","Time":"2023-08-01T21:25:10.004812-07:00","RemoteIP":"127.0.0.1","Requester":"user","RequestID":"ACA601CDB88E80CA","Operation":"ListObjectsV2","Key":"","RequestURI":"http://127.0.0.1:7070/bucket1?list-type=2\u0026prefix=\u0026delimiter=%2F\u0026encoding-type=url","HttpStatus":200,"ErrorCode":"","BytesSent":359,"ObjectSize":0,"TotalTime":1,"TurnAroundTime":1,"Referer":"","UserAgent":"aws-cli/2.2.25 Python/3.8.8 Darwin/22.5.0 exe/x86_64 prompt/off command/s3.ls","VersionID":"","HostID":"","SignatureVersion":"SigV4","CipherSuite":"","AuthenticationType":"AuthHeader","HostHeader":"s3.us-east-1.amazonaws.com","TLSVersion":"","AccessPointARN":"arn:aws:s3:::/bucket1","AclRequired":"Yes"} ``` -------------------------------- ### Create Gateway Backend Directories Source: https://github.com/versity/versitygw/wiki/Quickstart Create necessary directories for the gateway's backend storage and versioning. Versioning is optional. ```bash mkdir /tmp/vgw /tmp/vers ``` -------------------------------- ### Create CLI Command for New Backend Source: https://github.com/versity/versitygw/wiki/Adding-A-New-Backend Define a new command-line subcommand for the new backend, including its name, usage, description, and action. ```go func mystorageCommand() *cli.Command { return &cli.Command{ Name: "mystorage", Usage: "usage for mystorage", Description: "description for mystorage", Action: runMyStorage, } } ``` -------------------------------- ### Create User Programmatically with Go Source: https://github.com/versity/versitygw/wiki/Admin-APIs This Go program demonstrates how to create a new IAM user account programmatically. It includes setting up the request, signing it with AWS SigV4, and handling potential API errors. Ensure the AWS SDK for Go is installed and configured. ```go package main import ( "bytes" "crypto/sha256" "encoding/hex" "encoding/xml" "fmt" "io" "log" "net/http" "time" "github.com/aws/aws-sdk-go-v2/aws" v4 "github.com/aws/aws-sdk-go-v2/aws/signer/v4" "github.com/aws/smithy-go" ) const ( adminEndpoint = "http://127.0.0.1:7070" adminAccess = "admin_access_key_id" adminSecret = "admin_secret_access_key" serverRegion = "us-east-1" ) type Account struct { Access string `xml:"Access"` Secret string `xml:"Secret"` Role string `xml:"Role"` UserID int `xml:"UserID"` GroupID int `xml:"GroupID"` ProjectID int `xml:"ProjectID"` } func main() { acc := Account{ Access: "test_user_1", Secret: "test_user_1_secret", Role: "user", UserID: 1, GroupID: 3, ProjectID: 2, } accxml, err := xml.Marshal(acc) if err != nil { log.Fatalf("failed to serialize user data: %v\n", err) } req, err := http.NewRequest(http.MethodPatch, fmt.Sprintf("%v/create-user", adminEndpoint), bytes.NewBuffer(accxml)) if err != nil { log.Fatalf("failed to send the request: %v\n", err) } signer := v4.NewSigner() hashedPayload := sha256.Sum256(accxml) hexPayload := hex.EncodeToString(hashedPayload[:]) req.Header.Set("X-Amz-Content-Sha256", hexPayload) err = signer.SignHTTP(req.Context(), aws.Credentials{AccessKeyID: adminAccess, SecretAccessKey: adminSecret}, req, hexPayload, "s3", serverRegion, time.Now()) if err != nil { log.Fatalf("failed to sign the request: %v\n", err) } client := &http.Client{} resp, err := client.Do(req) if err != nil { log.Fatalf("failed to send the request: %v\n", err) } if resp.StatusCode >= 400 { defer resp.Body.Close() body, err := io.ReadAll(resp.Body) if err != nil { log.Fatalf("failed to read the request body: %v\n", err) } log.Fatalln(parseApiError(body)) } } func parseApiError(body []byte) error { var apiErr smithy.GenericAPIError err := xml.Unmarshal(body, &apiErr) if err != nil { apiErr.Code = "InternalServerError" apiErr.Message = err.Error() } return &apiErr } ``` -------------------------------- ### S3 Server Access Log Format Example Source: https://github.com/versity/versitygw/wiki/S3-server-access-log This is an example of the log file format generated by the gateway, which adheres to the AWS S3 log file format. The log can be rotated by sending SIGHUP to the gateway. ```text - - [01/August/2023:21:11:35 -0700] 127.0.0.1 admin 0C0C4A5F8759E883 ListBucket - http://127.0.0.1:7070/ 200 - 481 0 1 1 - aws-cli/2.2.25 Python/3.8.8 Darwin/22.5.0 exe/x86_64 prompt/off command/s3.ls - - SigV4 - AuthHeader s3.us-east-1.amazonaws.com - arn:aws:s3:::/ Yes acct1 bucket1 [01/August/2023:21:11:52 -0700] 127.0.0.1 acct1 0B9570A292BEE1E5 ListObjectsV2 - http://127.0.0.1:7070/bucket1?list-type=2&prefix=&delimiter=%2F&encoding-type=url 200 - 359 0 0 0 - aws-cli/2.2.25 Python/3.8.8 Darwin/22.5.0 exe/x86_64 prompt/off command/s3.ls - - SigV4 - AuthHeader s3.us-east-1.amazonaws.com - arn:aws:s3:::/bucket1 Yes ``` -------------------------------- ### Event Listeners for Folder and Bucket Creation Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Sets up event listeners for input fields to trigger folder and bucket creation when the 'Enter' key is pressed. ```javascript document.getElementById('new-folder-name'); if (folderInput) { folderInput.addEventListener('keydown', (e) => { if (e.key === 'Enter') { createFolder(); } }); } const bucketInput = document.getElementById('new-bucket-name'); if (bucketInput) { bucketInput.addEventListener('keydown', (e) => { if (e.key === 'Enter') { createBucket(); } }); } ``` -------------------------------- ### Get Selected Region Source: https://github.com/versity/versitygw/blob/main/webui/web/index.html Retrieves the currently selected region value from the region input element. ```javascript function getSelectedRegion() { return document.getElementById('region').value; } ``` -------------------------------- ### Extract Tarball Source: https://github.com/versity/versitygw/wiki/Linux-Install Extract the Versity Gateway files from a tar.gz archive for manual installation on other Linux distributions. ```bash tar -xzf versitygw_*.tar.gz ``` -------------------------------- ### Run Versity Gateway with Noop Plugin Source: https://github.com/versity/versitygw/blob/main/plugins/noop/README.md Pass the path to the compiled noop plugin to the `plugin` subcommand of `versitygw`. This backend requires no configuration. ```sh versitygw plugin /path/to/noop.so ``` -------------------------------- ### Utility Function: Get File Icon Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Determines the appropriate icon class for a file based on its extension. ```javascript function getFileIcon(name) { const ext = name.split('.').pop().toLowerCase(); const icons = { // Images 'jpg': 'text-pink-500', 'jpeg': 'text-pink-500', 'png': 'text-pink-500', 'gif': 'text-pink-500', 'svg': 'text-pink-500', 'webp': 'text-pink-500', // Documents 'pdf': 'text-red-500', 'doc': 'text-blue-500', 'docx': 'text-blue-500', 'xls': 'text-green-500', 'xlsx': 'text-green-500', // Code 'js': 'text-yellow-500', 'ts': 'text-blue-500', 'py': 'text-blue-500', 'go': 'text-cyan-500', 'html': 'text-orange-500', 'css': 'text-purple-500', 'json': 'text-yellow-600', // Archive 'zip': 'text-yellow-600', 'tar': 'text-yellow-600', 'gz': 'text-yellow-600', 'rar': 'text-yellow-600', // Video 'mp4': 'text-purple-500', 'webm': 'text-purple-500', 'mov': 'text-purple-500', // Audio 'mp3': 'text-green-500', 'wav': 'text-green-500', }; const color = icons[ext] || 'text-gray-400'; return ` `; } ``` -------------------------------- ### Verify Initial Directory Structure Source: https://github.com/versity/versitygw/wiki/POSIX-versioning Check the file system to confirm that the bucket directory has been created in the primary object namespace, but no versioning-specific directories exist yet. ```bash # find /tmp/gw /tmp/gw /tmp/gw/mybucket # find /tmp/versioning /tmp/versioning ``` -------------------------------- ### Get File Name from Key Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Extracts the file name from an object key. Removes any trailing slash if present. ```javascript function getFileName(key) { return key.endsWith('/') ? key.slice(0, -1) : key; } ``` -------------------------------- ### Get Folder Name from Key Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Extracts the folder name from a folder key. Assumes the key ends with a '/'. ```javascript function getFolderName(key) { return key.endsWith('/') ? key.slice(0, -1) : key; } ``` -------------------------------- ### Test Setup: AlmaLinux Version Source: https://github.com/versity/versitygw/wiki/MultiPart-Optimizations Specifies the operating system and kernel version used for testing the multipart upload optimizations. ```text AlmaLinux release 9.3 (Shamrock Pampas Cat) kernel version 5.14.0-362.13.1.el9_3.x86_64 ``` -------------------------------- ### Open Create Folder Dialog Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Opens a modal dialog for creating a new folder. It checks if a bucket is selected and sets the initial folder location before displaying the modal. Requires `currentBucket` and `currentPrefix` to be defined. ```javascript function openCreateFolderDialog() { if (!currentBucket) { showToast('Please select a bucket first', 'warning'); return; } document.getElementById('new-folder-name').value = ''; document.getElementById('folder-location').textContent = currentBucket + '/' + currentPrefix; openModal('create-folder-modal'); } ``` -------------------------------- ### Create User Account with versitygw CLI Source: https://github.com/versity/versitygw/wiki/Multi-Tenant Use this command to create a new regular user account. Provide the necessary admin credentials and the desired user access and secret keys. ```bash versitygw admin --access myaccess --secret mysecret --endpoint-url http://127.0.0.1:7070 create-user --access myuser --secret mysecret --role user ``` -------------------------------- ### Utility Function: Get File Name Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Extracts the last part of a key string, assuming it represents a file name. ```javascript function getFileName(key) { const parts = key.split('/'); return parts[parts.length - 1] || key; } ``` -------------------------------- ### Utility Function: Get Folder Name Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Extracts the last part of a prefix string, assuming it represents a folder name. ```javascript function getFolderName(prefix) { const parts = prefix.split('/').filter(p => p); return parts[parts.length - 1] || prefix; } ``` -------------------------------- ### Get File Extension Source: https://github.com/versity/versitygw/blob/main/webui/web/explorer.html Extracts the file extension from a given file name. Returns an empty string if no extension is found. ```javascript function getFileExtension(fileName) { const lastDot = fileName.lastIndexOf('.'); return lastDot > 0 ? fileName.substring(lastDot) : ''; } ``` -------------------------------- ### Display Versity Gateway Version Source: https://github.com/versity/versitygw/wiki/Global-Options Use the --version or -v option to print the current binary version and exit. ```bash --version, -v list versitygw version (default: false) ``` ```bash $ ./versitygw --version Version : v0.1 Build : a881893 BuildTime: 2023-05-29_05:16:34AM ```