### Configure S3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Launches an interactive setup wizard to configure S3 credentials, region, encryption, and proxy settings. Configuration is saved to ~/.s3cfg by default, but a custom config file can be specified. ```bash s3cmd --configure ``` ```bash s3cmd --config=/path/to/custom.cfg ls ``` -------------------------------- ### Download Objects (get) Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for downloading objects from S3 buckets to a local directory or standard output. ```APIDOC ## GET /s3cmd/get ### Description Downloads objects from an S3 bucket. Supports recursive downloads, resuming interrupted transfers, skipping existing files, overwriting, and downloading to standard output. ### Method GET (simulated via CLI command) ### Endpoint Not applicable (CLI command) ### Parameters - `--recursive` (flag) - Downloads objects recursively from a specified directory. - `--continue` (flag) - Resumes an interrupted download. - `--skip-existing` (flag) - Skips downloading files that already exist locally. - `--force` (flag) - Overwrites existing local files. - `-` (special argument) - Downloads the object content to standard output. ### Request Example ```bash s3cmd get --recursive s3://my-bucket/remote-dir/ ./local-dir/ s3cmd get --continue s3://my-bucket/large-file.zip s3cmd get --skip-existing s3://my-bucket/files/ ./local/ s3cmd get --force s3://my-bucket/file.txt ./existing-file.txt s3cmd get s3://my-bucket/config.json - ``` ### Response #### Success Response (200) Objects are downloaded to the specified local path. Output indicates the progress and completion of the download. #### Response Example ``` s3://my-bucket/file.txt -> ./file.txt [1 of 1] 123456 of 123456 100% in 2s 51.75 kB/s done ``` ``` -------------------------------- ### Display s3cmd Version in Error Report Source: https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ) This example demonstrates how an s3cmd version might appear within an error traceback, helping users identify if their version is outdated. ```text Problem: KeyErr: 'elapsed' **S3cmd: 1.1.0-beta3** Traceback (most recent call last): File "/usr/bin/s3cmd", line 1800, in ? main() ``` -------------------------------- ### Get Bucket and Object Information with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for retrieving detailed information about S3 buckets and objects. The 'info' command provides details such as location, ACL, storage class, metadata, file size, modification date, and MD5 sum. ```bash s3cmd info s3://my-bucket ``` ```bash s3cmd info s3://my-bucket/file.txt ``` -------------------------------- ### Get CloudFront Distribution Info using s3cmd (cfinfo) Source: https://context7.com/s3tools/s3cmd/llms.txt Retrieves detailed information about CloudFront distributions. Can be used to get information for all distributions or for a specific distribution identified by its ID. Useful for inspecting distribution settings. ```bash # Get info for all distributions s3cmd cfinfo # Get info for specific distribution s3cmd cfinfo cf://E1234ABCD5678 ``` -------------------------------- ### Get Disk Usage with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt The 'du' command calculates and displays disk usage for S3 buckets or specific prefixes within a bucket. ```bash s3cmd du ``` ```bash s3cmd du s3://my-bucket ``` ```bash s3cmd du s3://my-bucket/photos/ ``` -------------------------------- ### Manage S3 Buckets Source: https://github.com/s3tools/s3cmd/blob/master/README.md Commands for creating and listing S3 buckets. ```bash s3cmd ls s3cmd mb s3://public.s3tools.org ``` -------------------------------- ### Configure s3cmd Source: https://github.com/s3tools/s3cmd/blob/master/README.md Initializes the s3cmd configuration by prompting for AWS Access and Secret keys. ```bash s3cmd --configure ``` -------------------------------- ### List All Objects Across Buckets with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Lists all objects across all buckets, providing a comprehensive overview of S3 storage contents. Includes details like modification time, size, and object URI. ```bash s3cmd la ``` -------------------------------- ### Create S3 Bucket with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Creates a new S3 bucket. Supports specifying the region and access control settings like public read access. The output confirms bucket creation. ```bash s3cmd mb s3://my-new-bucket ``` ```bash s3cmd mb s3://my-eu-bucket --region=eu-west-1 ``` ```bash s3cmd mb s3://my-public-bucket --acl-public ``` -------------------------------- ### List Buckets and Objects with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Lists all buckets when no arguments are provided. When a bucket URI is given, it lists objects within that bucket. Supports recursive listing, long format, limiting results, and human-readable sizes. ```bash s3cmd ls ``` ```bash s3cmd ls s3://my-bucket ``` ```bash s3cmd ls s3://my-bucket/photos/ ``` ```bash s3cmd ls --recursive s3://my-bucket ``` ```bash s3cmd ls -l s3://my-bucket ``` ```bash s3cmd ls --limit=100 s3://my-bucket ``` ```bash s3cmd ls -H s3://my-bucket ``` -------------------------------- ### Create CloudFront Distribution using s3cmd (cfcreate) Source: https://context7.com/s3tools/s3cmd/llms.txt Creates a new CloudFront distribution for an S3 bucket. Supports specifying custom CNAMEs and adding comments to the distribution. This command simplifies the process of setting up a CDN for S3 content. ```bash # Create CloudFront distribution s3cmd cfcreate s3://my-bucket # Create with custom CNAME s3cmd cfcreate --cf-add-cname=cdn.example.com s3://my-bucket # Create with comment s3cmd cfcreate --cf-comment="Production CDN" s3://my-bucket ``` -------------------------------- ### Bucket and Object Information (info) Source: https://context7.com/s3tools/s3cmd/llms.txt Commands to retrieve detailed information about S3 buckets and objects. ```APIDOC ## GET /s3cmd/info ### Description Retrieves and displays detailed information about S3 buckets or objects, including location, ACLs, storage class, metadata, and more. ### Method GET (simulated via CLI command) ### Endpoint Not applicable (CLI command) ### Parameters - `s3://bucket` (string) - The S3 URI of the bucket to get information about. - `s3://bucket/object` (string) - The S3 URI of the object to get information about. ### Request Example ```bash s3cmd info s3://my-bucket s3cmd info s3://my-bucket/file.txt ``` ### Response #### Success Response (200) Detailed information about the specified bucket or object is returned. #### Response Example ``` s3://my-bucket (bucket): Location: us-east-1 Payer: BucketOwner Ownership: BucketOwnerEnforced Versioning: none Expiration rule: none Block Public Access: BlockPublicAcls,IgnorePublicAcls,BlockPublicPolicy,RestrictPublicBuckets Policy: none CORS: none ACL: owner-id: FULL_CONTROL s3://my-bucket/file.txt (object): File size: 12345 Last mod: Mon, 15 Jan 2024 10:30:00 GMT MIME type: text/plain Storage: STANDARD MD5 sum: d41d8cd98f00b204e9800998ecf8427e SSE: none ACL: owner-id: FULL_CONTROL ``` ``` -------------------------------- ### List and Retrieve Files Source: https://github.com/s3tools/s3cmd/blob/master/README.md Lists bucket contents and retrieves files or directory trees from S3, including recursive operations. ```bash s3cmd ls s3://public.s3tools.org s3cmd ls --recursive s3://public.s3tools.org s3cmd get s3://public.s3tools.org/somefile.xml some-file-2.xml s3cmd get --recursive s3://public.s3tools.org/somewhere ``` -------------------------------- ### Synchronize Directories (sync) Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for synchronizing directories between local filesystem and S3, or between two S3 locations. ```APIDOC ## POST /s3cmd/sync ### Description Synchronizes directories between a local filesystem and S3, or between two S3 locations. It compares files by size and MD5 checksum to transfer only changed files. Supports deletion of removed files, exclusion/inclusion patterns, and preserving file attributes. ### Method POST (simulated via CLI command) ### Endpoint Not applicable (CLI command) ### Parameters - `SOURCE` (string) - The source directory or S3 URI. - `DESTINATION` (string) - The destination directory or S3 URI. - `--delete-removed` (flag) - Deletes files in the destination that are not present in the source. - `--no-check-md5` (flag) - Skips MD5 checksum verification, relying only on file size for comparison (faster). - `--exclude="PATTERN"` (string) - Excludes files matching the specified pattern. - `--include="PATTERN"` (string) - Includes only files matching the specified pattern. - `--preserve` (flag) - Preserves file attributes such as permissions and timestamps. - `--dry-run` (flag) - Shows what changes would be made without actually performing them. - `--delete-after-fetch` (flag) - Deletes source files after they have been successfully transferred. - `--cf-invalidate` (flag) - Invalidates CloudFront cache for synchronized files. ### Request Example ```bash s3cmd sync ./local-dir/ s3://my-bucket/remote-dir/ s3cmd sync s3://my-bucket/remote-dir/ ./local-dir/ s3cmd sync s3://source-bucket/dir/ s3://dest-bucket/dir/ s3cmd sync --delete-removed ./local-dir/ s3://my-bucket/ s3cmd sync --no-check-md5 ./local-dir/ s3://my-bucket/ s3cmd sync --exclude="*.tmp" --exclude="*.log" ./local-dir/ s3://my-bucket/ s3cmd sync --include="*.jpg" --include="*.png" ./photos/ s3://my-bucket/photos/ s3cmd sync --preserve ./local-dir/ s3://my-bucket/ s3cmd sync --dry-run ./local-dir/ s3://my-bucket/ s3cmd sync --delete-after-fetch s3://my-bucket/ ./local-backup/ s3cmd sync --cf-invalidate ./website/ s3://my-website-bucket/ ``` ### Response #### Success Response (200) Directories are synchronized. Output indicates which files were uploaded, deleted, or skipped. #### Response Example ``` upload: './local-dir/newfile.txt' -> 's3://my-bucket/remote-dir/newfile.txt' Done. Uploaded 12345 bytes in 1.5 seconds, 8.23 kB/s. ``` ``` -------------------------------- ### Upload Files to S3 with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Uploads local files or directories to S3. Supports single files, multiple files, recursive directory uploads, public access, specific storage classes, server-side and GPG encryption, custom headers, MIME type overrides, and uploads from stdin. Includes a dry-run option. ```bash s3cmd put file.txt s3://my-bucket/ ``` ```bash s3cmd put localfile.txt s3://my-bucket/remote-name.txt ``` ```bash s3cmd put file1.txt file2.txt s3://my-bucket/ ``` ```bash s3cmd put --recursive ./local-dir/ s3://my-bucket/remote-dir/ ``` ```bash s3cmd put --acl-public image.jpg s3://my-bucket/ ``` ```bash s3cmd put --storage-class=STANDARD_IA archive.zip s3://my-bucket/ ``` ```bash s3cmd put --server-side-encryption secret.txt s3://my-bucket/ ``` ```bash s3cmd put --encrypt sensitive.txt s3://my-bucket/ ``` ```bash s3cmd put --add-header="Cache-Control:max-age=31536000" static.js s3://my-bucket/ ``` ```bash s3cmd put --mime-type="application/json" data.txt s3://my-bucket/data.json ``` ```bash cat data.txt | s3cmd put - s3://my-bucket/piped-data.txt ``` ```bash s3cmd put --dry-run ./local-dir/ s3://my-bucket/ ``` -------------------------------- ### Synchronize Directories with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for synchronizing directories between local filesystem and S3, or between two S3 locations. Compares files by size and MD5 checksum to transfer only changed files. Supports various options like deletion, skipping checksums, exclusions, inclusions, and dry runs. ```bash s3cmd sync ./local-dir/ s3://my-bucket/remote-dir/ ``` ```bash s3cmd sync s3://my-bucket/remote-dir/ ./local-dir/ ``` ```bash s3cmd sync s3://source-bucket/dir/ s3://dest-bucket/dir/ ``` ```bash s3cmd sync --delete-removed ./local-dir/ s3://my-bucket/ ``` ```bash s3cmd sync --no-check-md5 ./local-dir/ s3://my-bucket/ ``` ```bash s3cmd sync --exclude="*.tmp" --exclude="*.log" ./local-dir/ s3://my-bucket/ ``` ```bash s3cmd sync --include="*.jpg" --include="*.png" ./photos/ s3://my-bucket/photos/ ``` ```bash s3cmd sync --preserve ./local-dir/ s3://my-bucket/ ``` ```bash s3cmd sync --dry-run ./local-dir/ s3://my-bucket/ ``` ```bash s3cmd sync --delete-after-fetch s3://my-bucket/ ./local-backup/ ``` ```bash s3cmd sync --cf-invalidate ./website/ s3://my-website-bucket/ ``` -------------------------------- ### Upload and Sync Files Source: https://github.com/s3tools/s3cmd/blob/master/README.md Uploads individual files or entire directory trees to an S3 bucket using put or sync commands. ```bash s3cmd put some-file.xml s3://public.s3tools.org/somefile.xml s3cmd put --recursive dir1 dir2 s3://public.s3tools.org/somewhere/ s3cmd sync dir1 dir2 s3://public.s3tools.org/somewhere/ ``` -------------------------------- ### Verify File Integrity Source: https://github.com/s3tools/s3cmd/blob/master/README.md Uses md5sum to verify that a retrieved file matches the original source file. ```bash md5sum some-file.xml some-file-2.xml ``` -------------------------------- ### Download Files from S3 with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Downloads objects from S3 to local storage. Supports downloading single files to the current directory or a specified local path. Supports resumable downloads and recursive operations. ```bash s3cmd get s3://my-bucket/file.txt ``` ```bash s3cmd get s3://my-bucket/file.txt ./local-file.txt ``` -------------------------------- ### Manage S3 Bucket Policies with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Manages bucket access policies using JSON policy documents. Commands allow setting a policy from a file, retrieving the current policy, and deleting the policy. Policies control access permissions for the bucket. ```bash # Set bucket policy from file s3cmd setpolicy policy.json s3://my-bucket # Get current bucket policy s3cmd getpolicy s3://my-bucket # Delete bucket policy s3cmd delpolicy s3://my-bucket ``` -------------------------------- ### Download Files with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for downloading files and directories from S3. Supports recursive downloads, resuming interrupted transfers, skipping existing files, overwriting, and downloading to standard output. ```bash s3cmd get --recursive s3://my-bucket/remote-dir/ ./local-dir/ ``` ```bash s3cmd get --continue s3://my-bucket/large-file.zip ``` ```bash s3cmd get --skip-existing s3://my-bucket/files/ ./local/ ``` ```bash s3cmd get --force s3://my-bucket/file.txt ./existing-file.txt ``` ```bash s3cmd get s3://my-bucket/config.json - ``` -------------------------------- ### List CloudFront Distributions using s3cmd (cflist) Source: https://context7.com/s3tools/s3cmd/llms.txt Displays all CloudFront distribution points associated with the AWS account. The output includes the Distribution ID, S3 Origin, Enabled status, and any associated comment. This command helps in monitoring existing distributions. ```bash # List all CloudFront distributions s3cmd cflist ``` -------------------------------- ### Configure S3 Bucket Access Logging with s3cmd (accesslog) Source: https://context7.com/s3tools/s3cmd/llms.txt Enables or disables server access logging for an S3 bucket. Allows specifying a target prefix in another bucket for storing log files. Can also be used to check the current logging status and configuration. ```bash # Enable access logging s3cmd accesslog --access-logging-target-prefix=s3://log-bucket/access-logs/ s3://my-bucket # Disable access logging s3cmd accesslog --no-access-logging s3://my-bucket # Check current logging configuration s3cmd accesslog s3://my-bucket ``` -------------------------------- ### Manage S3 Bucket Lifecycle Policies with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Manages advanced object lifecycle control for S3 buckets. Allows uploading policies from JSON/XML files, retrieving the current policy, and deleting the policy. The policy file defines rules for transitions and expirations. ```bash # Upload lifecycle policy from JSON/XML file s3cmd setlifecycle lifecycle-policy.json s3://my-bucket # Get current lifecycle policy s3cmd getlifecycle s3://my-bucket # Delete lifecycle policy s3cmd dellifecycle s3://my-bucket ``` -------------------------------- ### Manage S3 Object and Bucket Tagging with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Manages tags on S3 buckets and objects for organization and cost allocation. Supports setting tags using key-value pairs, retrieving existing tags, and deleting tags. Tags can be applied to the bucket itself or individual objects. ```bash # Set tags on bucket or object s3cmd settagging s3://my-bucket "Environment=Production&Team=Engineering" # Set tags on specific object s3cmd settagging s3://my-bucket/file.txt "Project=Alpha&Owner=John" # Get tags s3cmd gettagging s3://my-bucket s3cmd gettagging s3://my-bucket/file.txt # Delete tags s3cmd deltagging s3://my-bucket s3cmd deltagging s3://my-bucket/file.txt ``` -------------------------------- ### Copy Objects with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for copying objects within S3, including server-side copies. Supports copying within the same bucket, to different buckets, recursive copies, and changing ACLs. ```bash s3cmd cp s3://my-bucket/file.txt s3://my-bucket/backup/file.txt ``` ```bash s3cmd cp s3://source-bucket/file.txt s3://dest-bucket/file.txt ``` ```bash s3cmd cp --recursive s3://source-bucket/data/ s3://dest-bucket/data/ ``` ```bash s3cmd cp --acl-public s3://my-bucket/private.txt s3://my-bucket/public.txt ``` -------------------------------- ### Disk Usage (du) Source: https://context7.com/s3tools/s3cmd/llms.txt Commands to calculate and display disk usage for S3 buckets or prefixes. ```APIDOC ## GET /s3cmd/du ### Description Calculates and displays the disk usage for S3 buckets or specific prefixes within a bucket. ### Method GET (simulated via CLI command) ### Endpoint Not applicable (CLI command) ### Parameters - `s3://bucket` (string, optional) - The S3 URI of the bucket to calculate disk usage for. If omitted, usage for all buckets is shown. - `s3://bucket/prefix/` (string, optional) - The S3 URI prefix to calculate disk usage for. ### Request Example ```bash s3cmd du s3cmd du s3://my-bucket s3cmd du s3://my-bucket/photos/ ``` ### Response #### Success Response (200) The disk usage in bytes for the specified bucket or prefix is returned. #### Response Example ``` 123456789 s3://my-bucket/photos/ ``` ``` -------------------------------- ### List of standard MIME type configuration files Source: https://github.com/s3tools/s3cmd/wiki/Understand-Content-Type,-Mime-type-guessing-and-Magic This list represents the standard locations s3cmd checks on Linux and macOS to load additional file extension rules for the guess_mime_type functionality. ```python knownfiles = [ "/etc/mime.types", "/etc/httpd/mime.types", "/etc/httpd/conf/mime.types", "/etc/apache/mime.types", "/etc/apache2/mime.types", "/usr/local/etc/httpd/conf/mime.types", "/usr/local/lib/netscape/mime.types", "/usr/local/etc/httpd/conf/mime.types", "/usr/local/etc/mime.types", ] ``` -------------------------------- ### Copy Objects (cp) Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for copying objects within S3, including server-side copy operations. ```APIDOC ## POST /s3cmd/cp ### Description Copies objects within S3 buckets. Supports copying within the same bucket, to a different bucket, recursive copying, and changing ACLs during the copy process. Utilizes server-side copy for efficiency. ### Method POST (simulated via CLI command) ### Endpoint Not applicable (CLI command) ### Parameters - `s3://source-bucket/source-object` (string) - The S3 URI of the source object. - `s3://dest-bucket/dest-object` (string) - The S3 URI of the destination. - `--recursive` (flag) - Copies objects recursively from a source directory to a destination. - `--acl-public` (flag) - Sets the ACL of the copied object to public. ### Request Example ```bash s3cmd cp s3://my-bucket/file.txt s3://my-bucket/backup/file.txt s3cmd cp s3://source-bucket/file.txt s3://dest-bucket/file.txt s3cmd cp --recursive s3://source-bucket/data/ s3://dest-bucket/data/ s3cmd cp --acl-public s3://my-bucket/private.txt s3://my-bucket/public.txt ``` ### Response #### Success Response (200) Objects are copied to the specified destination. Output confirms the copy operation. #### Response Example ``` remote copy: 's3://source-bucket/file.txt' -> 's3://dest-bucket/file.txt' ``` ``` -------------------------------- ### Set S3 Bucket Versioning with s3cmd (setversioning) Source: https://context7.com/s3tools/s3cmd/llms.txt Enables or disables bucket versioning for an S3 bucket. Versioning provides an additional layer of data protection against accidental deletions or overwrites by maintaining multiple versions of an object. Requires the bucket name and an 'enable' or 'disable' argument. ```bash # Enable versioning s3cmd setversioning s3://my-bucket enable ``` -------------------------------- ### Manage S3 CORS Configuration with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Manages Cross-Origin Resource Sharing (CORS) rules for an S3 bucket. Supports setting the CORS configuration from an XML file and deleting the existing configuration. CORS rules define which web applications can access resources in the bucket. ```bash # Set CORS configuration s3cmd setcors cors.xml s3://my-bucket # Delete CORS configuration s3cmd delcors s3://my-bucket ``` -------------------------------- ### Configure s3cmd for China (cn-north-1) Region Source: https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ) This configuration snippet shows the specific parameters needed in the .s3cfg file to enable s3cmd to work correctly with buckets in the 'cn-north-1' region. ```ini [s3cmd] bucket_location = cn-north-1 host_base = s3.cn-north-1.amazonaws.com.cn host_bucket = %(bucket)s.s3.cn-north-1.amazonaws.com.cn website_endpoint = http://%(bucket)s.s3-website-%(location)s.amazonaws.com.cn/ ``` -------------------------------- ### Move Objects with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for moving objects within S3, which involves a copy operation followed by a delete of the source. Supports moving/renaming, moving between buckets, and recursive moves. ```bash s3cmd mv s3://my-bucket/old-name.txt s3://my-bucket/new-name.txt ``` ```bash s3cmd mv s3://source-bucket/file.txt s3://dest-bucket/file.txt ``` ```bash s3cmd mv --recursive s3://my-bucket/old-folder/ s3://my-bucket/new-folder/ ``` -------------------------------- ### Delete Objects with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for deleting objects from S3. Supports single and multiple object deletion, recursive deletion, exclusion patterns, dry runs, and limiting the number of deletions. ```bash s3cmd del s3://my-bucket/file.txt ``` ```bash s3cmd del s3://my-bucket/file1.txt s3://my-bucket/file2.txt ``` ```bash s3cmd del --recursive --force s3://my-bucket/old-data/ ``` ```bash s3cmd del --recursive --exclude="*.log" s3://my-bucket/data/ ``` ```bash s3cmd del --dry-run --recursive s3://my-bucket/temp/ ``` ```bash s3cmd del --recursive --max-delete=100 s3://my-bucket/cleanup/ ``` -------------------------------- ### Set Object Expiration using s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Configures automatic expiration rules for objects in an S3 bucket. Supports setting expiration by days, a specific date, or a prefix. Can also be used to remove existing expiration rules. Requires the bucket name as input. ```bash # Set expiration after 30 days s3cmd expire --expiry-days=30 s3://my-bucket # Set expiration on specific date s3cmd expire --expiry-date=2024-12-31 s3://my-bucket # Set expiration for objects with specific prefix s3cmd expire --expiry-days=7 --expiry-prefix=temp/ s3://my-bucket # Remove expiration rule s3cmd expire s3://my-bucket ``` -------------------------------- ### Define IAM Policy for s3cmd Operations Source: https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ) This JSON policy provides the minimum required permissions for s3cmd to perform list, upload, download, and delete operations on a specific S3 bucket. Replace 'my_bucket' with your actual bucket name to apply the policy. ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt123456", "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets" ], "Resource": [ "arn:aws:s3:::*" ] }, { "Sid": "Stmt123457", "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload", "s3:GetObjectAcl", "s3:GetObjectVersion", "s3:DeleteObject", "s3:DeleteObjectVersion", "s3:GetObject", "s3:PutObjectAcl", "s3:PutObject", "s3:GetObjectVersionAcl" ], "Resource": [ "arn:aws:s3:::my_bucket", "arn:aws:s3:::my_bucket/*" ] } ] } ``` -------------------------------- ### Move Objects (mv) Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for moving objects within S3, which involves a copy operation followed by a delete operation. ```APIDOC ## POST /s3cmd/mv ### Description Moves objects within S3 buckets. This operation is performed as a copy followed by a delete of the source object. Supports renaming, moving between buckets, and recursive moves. ### Method POST (simulated via CLI command) ### Endpoint Not applicable (CLI command) ### Parameters - `s3://source-bucket/source-object` (string) - The S3 URI of the source object. - `s3://dest-bucket/dest-object` (string) - The S3 URI of the destination. - `--recursive` (flag) - Moves objects recursively from a source directory to a destination. ### Request Example ```bash s3cmd mv s3://my-bucket/old-name.txt s3://my-bucket/new-name.txt s3cmd mv s3://source-bucket/file.txt s3://dest-bucket/file.txt s3cmd mv --recursive s3://my-bucket/old-folder/ s3://my-bucket/new-folder/ ``` ### Response #### Success Response (200) Objects are moved to the specified destination. Output confirms the move operation. #### Response Example ``` move: 's3://my-bucket/old-name.txt' -> 's3://my-bucket/new-name.txt' ``` ``` -------------------------------- ### Delete Objects (del/rm) Source: https://context7.com/s3tools/s3cmd/llms.txt Commands for deleting objects from S3 buckets, with options for recursive deletion and exclusion patterns. ```APIDOC ## DELETE /s3cmd/del ### Description Removes objects from S3 buckets. Supports deleting single or multiple objects, recursive deletion, and filtering by exclusion patterns. Use `--force` with recursive deletion for safety. ### Method DELETE (simulated via CLI command) ### Endpoint Not applicable (CLI command) ### Parameters - `s3://bucket/object` (string) - The S3 URI of the object(s) to delete. - `--recursive` (flag) - Deletes objects recursively within a specified directory. - `--force` (flag) - Required for recursive deletion to confirm the action. - `--exclude="PATTERN"` (string) - Excludes files matching the specified pattern. - `--dry-run` (flag) - Shows what would be deleted without actually deleting. - `--max-delete=NUMBER` (integer) - Limits the number of objects to delete. ### Request Example ```bash s3cmd del s3://my-bucket/file.txt s3cmd del s3://my-bucket/file1.txt s3://my-bucket/file2.txt s3cmd del --recursive --force s3://my-bucket/old-data/ s3cmd del --recursive --exclude="*.log" s3://my-bucket/data/ s3cmd del --dry-run --recursive s3://my-bucket/temp/ s3cmd del --recursive --max-delete=100 s3://my-bucket/cleanup/ ``` ### Response #### Success Response (200) Objects are deleted from the S3 bucket. Output confirms the deletion. #### Response Example ``` delete: 's3://my-bucket/file.txt' ``` ``` -------------------------------- ### Define MIME type extension mapping Source: https://github.com/s3tools/s3cmd/wiki/Understand-Content-Type,-Mime-type-guessing-and-Magic This configuration defines how the Python mimetypes library maps file extensions to MIME types. The format requires the MIME type followed by a tab and the associated file extensions. ```text application/x-yaml yml yaml ``` -------------------------------- ### ParseError in s3cmd Source: https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ) Illustrates a ParseError in s3cmd, often caused by non-XML responses from the server due to network intermediaries. Updating s3cmd is recommended. ```text Problem: ParseError: no element found: line 1, column 0 S3cmd: 1.1.0-beta3 Traceback (most recent call last): File "/usr/bin/s3cmd", line 1800, in main() File "/usr/bin/s3cmd", line 1741, in main cmd_func(args) File "/usr/bin/s3cmd", line 462, in cmd_object_del subcmd_object_del_uri(uri_str) File "/usr/bin/s3cmd", line 488, in subcmd_object_del_uri response = s3.object_delete(S3Uri(item['object_uri_str'])) File "/usr/share/s3cmd/S3/S3.py", line 403, in object_delete response = self.send_request(request) File "/usr/share/s3cmd/S3/S3.py", line 613, in send_request e = S3Error(response) File "/usr/share/s3cmd/S3/Exceptions.py", line 48, in __init__ tree = getTreeFromXml(response["data"]) File "/usr/share/s3cmd/S3/Utils.py", line 69, in getTreeFromXml tree = ET.fromstring(xml) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML return parser.close() File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close self._raiseerror(v) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror raise err ParseError: no element found: line 1, column 0 ``` -------------------------------- ### Recursive S3 Object Retrieval Source: https://github.com/s3tools/s3cmd/blob/master/README.md Downloads files recursively from an S3 path. The inclusion of a trailing slash in the source path determines whether the parent directory name is preserved in the local file structure. ```bash s3cmd get --recursive s3://public.s3tools.org/somewhere/ ~/ ``` -------------------------------- ### Remove S3 Bucket with s3cmd Source: https://context7.com/s3tools/s3cmd/llms.txt Removes an S3 bucket. By default, the bucket must be empty. The `--force` flag can be used to remove the bucket and all its contents. ```bash s3cmd rb s3://my-old-bucket ``` ```bash s3cmd rb s3://my-old-bucket --force ``` -------------------------------- ### KeyError 'elapsed' in s3cmd Source: https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ) This snippet shows a typical traceback for a KeyError related to 'elapsed' in s3cmd, often resolved by updating to a newer version. ```text Traceback (most recent call last): File "/usr/bin/s3cmd", line 1800, in main() File "/usr/bin/s3cmd", line 1741, in main cmd_func(args) File "/usr/bin/s3cmd", line 309, in cmd_object_put (unicodise(full_name_orig), uri_final, response["size"], response["elapsed"], KeyError: 'elapsed' ``` -------------------------------- ### Recursive S3 Object Deletion Source: https://github.com/s3tools/s3cmd/blob/master/README.md Deletes all objects under a specific S3 prefix recursively using the del command. ```bash s3cmd del --recursive s3://public.s3tools.org/somewhere/ ``` -------------------------------- ### Forced S3 Bucket Removal Source: https://github.com/s3tools/s3cmd/blob/master/README.md Attempts to remove an S3 bucket. If the bucket is not empty, the --force flag can be used to automatically delete all contained objects before removing the bucket itself. ```bash s3cmd rb --force s3://public.s3tools.org/ ``` -------------------------------- ### Invalidate CloudFront Cache using s3cmd (cfinval) Source: https://context7.com/s3tools/s3cmd/llms.txt Creates invalidation requests to clear the CloudFront cache for specified objects or objects under a given prefix. This ensures that users receive the latest versions of updated files. Supports invalidating multiple files or all files under a path. ```bash # Invalidate specific objects s3cmd cfinval s3://my-bucket/updated-file.js s3://my-bucket/style.css # Invalidate all objects under prefix s3cmd cfinval s3://my-bucket/assets/* ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.