### Install S3Path from PyPI Source: https://github.com/liormizr/s3path/blob/master/README.rst Install the S3Path library using pip. ```bash $ pip install s3path ``` -------------------------------- ### Install S3Path from Conda Source: https://github.com/liormizr/s3path/blob/master/README.rst Install the S3Path library using conda. ```bash $ conda install -c conda-forge s3path ``` -------------------------------- ### Listing boto3 Path Contents Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Lists the contents of the 'boto3' path within the bucket. This example shows how to list specific files and index files within a subdirectory. ```python [path for path in bucket_path.boto3_path()] [S3Path('/pypi-proxy/boto3/boto3-1.4.1.tar.gz'), S3Path('/pypi-proxy/boto3/index.html')] ``` -------------------------------- ### S3Path Constructor Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Instantiate an S3Path object representing a concrete path in AWS S3. Requires boto3 to be installed. ```APIDOC ## S3Path Constructor ### Description Instantiates a concrete path object for AWS S3. The path segments define the bucket and key. ### Parameters * `*pathsegments` - Variable length argument list representing the bucket and key. ### Request Example ```python from s3path import S3Path S3Path('//') ``` ``` -------------------------------- ### S3Path instantiation with boto3 not installed Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Shows the ModuleNotFoundError when boto3 is not installed, preventing S3Path instantiation. ```python >>> import boto3 Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'boto3' >>> from s3path import S3Path >>> S3Path('//') Traceback (most recent call last): File "", line 1, in File "pathlib.py", line 798, in __new__ % (cls.__name__, )) NotImplementedError: cannot instantiate 'S3Path' on your system ``` -------------------------------- ### S3Path.get_presigned_url Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Generates a pre-signed URL for a given S3 key, allowing temporary GET access to the file. The expiration time can be specified. ```APIDOC ## S3Path.get_presigned_url(expire_in: timedelta | int = 3600) -> str ### Description Generates a pre-signed URL for a given S3 key, allowing temporary GET access to the file. The expiration time can be specified. ### Method Not applicable (Python method) ### Parameters #### Query Parameters - **expire_in** (timedelta or int) - Optional - The expiration time for the URL in seconds or as a timedelta object. Defaults to 3600 seconds (1 hour). ### Response #### Success Response (200) - **presigned_url** (string) - The generated pre-signed URL. ### Request Example ```python from s3path import S3Path file = S3Path("/my-bucket/toto.txt") presigned_url = file.get_presigned_url() print(presigned_url) ``` ### Response Example ``` https://my-bucket.s3.amazonaws.com/toto.txt?AWSAccessKeyId=...&Signature=...&Expires=... ``` ``` -------------------------------- ### PureS3Path.key Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Gets the key name from a PureS3Path object. ```APIDOC ## PureS3Path.key ### Description A string representing the AWS S3 Key name, if any. ### Returns (string) The key name. ### Example ```python >>> PureS3Path('/pypi-proxy/boto3/').key 'boto3' >>> PureS3Path('/pypi-proxy/boto3/index.html').key 'boto3/index.html' >>> PureS3Path.from_uri('s3://pypi-proxy/').key '' ``` ``` -------------------------------- ### PureS3Path.bucket Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Gets the bucket name from a PureS3Path object. ```APIDOC ## PureS3Path.bucket ### Description A string representing the AWS S3 Bucket name, if any. ### Returns (string) The bucket name. ### Example ```python >>> PureS3Path.from_uri('s3://pypi-proxy/boto3/').bucket 'pypi-proxy' >>> PureS3Path('/').bucket '' ``` ``` -------------------------------- ### Get Bucket Name from PureS3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Retrieves the AWS S3 bucket name from a PureS3Path object. ```python >>> PureS3Path.from_uri('s3://pypi-proxy/boto3/').bucket 'pypi-proxy' >>> PureS3Path('/').bucket '' ``` -------------------------------- ### Get Key Name from PureS3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Retrieves the AWS S3 key name from a PureS3Path object. ```python >>> PureS3Path('/pypi-proxy/boto3/').key 'boto3' >>> PureS3Path('/pypi-proxy/boto3/index.html').key 'boto3/index.html' >>> PureS3Path.from_uri('s3://pypi-proxy/').key '' ``` -------------------------------- ### Get S3Path stat information Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Retrieves and displays stat information for an S3 path, including size, last modified time, and version ID. ```python >>> path_stat = S3Path('/pypi-proxy/boto3/index.html').stat() >>> path_stat StatResult(size=188, last_modified=datetime.datetime(2018, 4, 4, 12, 26, 3, tzinfo=tzutc()), version_id=None) >>> path_stat.st_size 188 >>> path_stat.st_mtime 1522833963.0 >>> print(path_stat.st_version_id) None >>> path_stat.st_atime Traceback (most recent call last): ... io.UnsupportedOperation: StatResult do not support st_atime attribute ``` -------------------------------- ### Create an S3 Bucket Source: https://github.com/liormizr/s3path/blob/master/docs/comparison.rst Demonstrates creating a new S3 bucket using S3Path and Boto3. ```python >>> from s3path import S3Path >>> S3Path('/my-bucket/').mkdir() ``` ```python >>> import boto3 >>> s3 = boto3.resource('s3') >>> s3.create_bucket(Bucket='my-bucket') ``` -------------------------------- ### Instantiate S3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Demonstrates the basic instantiation of an S3Path object with a bucket and key. ```python >>> S3Path('//') S3Path('//') ``` -------------------------------- ### Initialize PureVersionedS3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Instantiate a PureVersionedS3Path object with bucket, key, and version ID. ```python >>> from s3path import PureVersionedS3Path >>> PureVersionedS3Path('//', version_id='') PureVersionedS3Path('//', version_id='') ``` -------------------------------- ### Initialize PureS3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Instantiate a PureS3Path object with bucket and key segments. ```python >>> PureS3Path('//') PureS3Path('//') ``` -------------------------------- ### Instantiate VersionedS3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Demonstrates how to instantiate a VersionedS3Path object, including the version_id. ```python >>> from s3path import VersionedS3Path >>> VersionedS3Path('//', version_id='') VersionedS3Path('//', version_id='') ``` -------------------------------- ### Upload a File to S3 Source: https://github.com/liormizr/s3path/blob/master/docs/comparison.rst Shows how to upload a local file to an S3 bucket using S3Path and Boto3. ```python >>> from pathlib import Path >>> from s3path import S3Path >>> local_path = Path('/tmp/hello.txt') >>> S3Path('/my-bucket/hello.txt').write_text(local_path.read_text()) ``` ```python >>> import shutil >>> from pathlib import Path >>> from s3path import S3Path >>> local_path = Path('/tmp/hello.txt') >>> remote_path = S3Path('/my-bucket/hello.txt') >>> with local_path.open('rb') as src, remote_path.open('wb') as dst: >>> shutil.copyfileobj(src, dst) ``` ```python >>> import boto3 >>> s3 = boto3.resource('s3') >>> bucket = s3.Bucket('my-bucket') >>> bucket.upload_file(Fileobj='/tmp/hello.txt', Key='hello.txt') ``` -------------------------------- ### Download a File from S3 Source: https://github.com/liormizr/s3path/blob/master/docs/comparison.rst Illustrates downloading a file from an S3 bucket to a local path using S3Path and Boto3. ```python >>> from pathlib import Path >>> from s3path import S3Path >>> local_path = Path('./my_local_image.jpg') >>> local_path.write_text(S3Path('/my-bucket/my_image_in_s3.jpg').read_text()) ``` ```python >>> import boto3 >>> import botocore >>> s3 = boto3.resource('s3') >>> >>> try: >>> bucket = s3.Bucket('my-bucket') >>> bucket.download_file(Key='my_image_in_s3.jpg', Filename='my_local_image.jpg') >>> except botocore.exceptions.ClientError as e: >>> if e.response['Error']['Code'] == "404": >>> print("The object does not exist.") >>> else: >>> raise ``` -------------------------------- ### S3Path.mkdir Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Creates a bucket at the specified path. Note that S3 does not support folders, so this method only creates buckets. If the bucket already exists, an error is raised unless exist_ok is True. The 'parents' and 'mode' arguments are handled according to S3's limitations. ```APIDOC ## S3Path.mkdir(mode=0o777, parents=False, exist_ok=False) ### Description Creates a bucket at the specified path. Note that S3 does not support folders, so this method only creates buckets. If the bucket already exists, an error is raised unless exist_ok is True. The 'parents' and 'mode' arguments are handled according to S3's limitations. ### Method Not applicable (Python method) ### Parameters #### Query Parameters - **mode** (integer) - Ignored. - **parents** (boolean) - Optional - If True, attempts to create parent buckets if they do not exist (behavior may be limited by S3). - **exist_ok** (boolean) - Optional - If True, ignores FileExistsError if the bucket already exists. ### Response #### Success Response Creates the specified bucket in AWS S3. ``` -------------------------------- ### Navigate and Construct S3 Paths Source: https://github.com/liormizr/s3path/blob/master/README.rst Construct a specific S3 path by joining directory and file names. ```python >>> bucket_path = S3Path('/pypi-proxy/') >>> boto3_package_path = bucket_path / 'boto3' / 'boto3-1.4.1.tar.gz' >>> boto3_package_path S3Path('/pypi-proxy/boto3/boto3-1.4.1.tar.gz') ``` -------------------------------- ### List S3 Buckets Source: https://github.com/liormizr/s3path/blob/master/docs/comparison.rst Compares listing all available S3 buckets using S3Path and Boto3. ```python >>> from s3path import S3Path >>> for bucket in S3Path('/').iterdir(): ... print(bucket) ``` ```python >>> import boto3 >>> # Create an S3 client >>> s3 = boto3.client('s3') >>> # Call S3 to list current buckets >>> response = s3.list_buckets() >>> # Get a list of all bucket names from the response >>> buckets = [bucket['Name'] for bucket in response['Buckets']] >>> # Print out the bucket list >>> for bucket in buckets: ... print(bucket) ``` -------------------------------- ### Create VersionedS3Path Instances Source: https://github.com/liormizr/s3path/blob/master/README.rst Instantiate VersionedS3Path using different methods, providing a version ID. ```python >>> from s3path import VersionedS3Path >>> bucket, key, version_id = 'my-bucket', 'my-key', 'my-version-id' >>> VersionedS3Path(f'/{bucket}/{key}', version_id=version_id) VersionedS3Path('/my-bucket/my-key', version_id='my-version-id') >>> VersionedS3Path.from_uri(f's3://{bucket}/{key}', version_id=version_id) VersionedS3Path('/my-bucket/my-key', version_id='my-version-id') >>> VersionedS3Path.from_bucket_key(bucket=bucket, key=key, version_id=version_id) VersionedS3Path('/my-bucket/my-key', version_id='my-version-id') ``` -------------------------------- ### Configure boto3 Session Source: https://github.com/liormizr/s3path/blob/master/docs/advance.rst Set up a default boto3 session with specific AWS credentials and region. This is necessary before interacting with S3 paths. ```python >>> import boto3 >>> from s3path import S3Path >>> boto3.setup_default_session( ... region_name='us-east-1', ... aws_access_key_id='', ... aws_secret_access_key='') >>> >>> bucket_path = S3Path('/pypi-proxy/') >>> [path for path in bucket_path.iterdir() if path.is_dir()] ... [S3Path('/pypi-proxy/requests/'), ... S3Path('/pypi-proxy/boto3/'), ... S3Path('/pypi-proxy/botocore/')] ``` -------------------------------- ### Create PureVersionedS3Path from Versioned Bucket and Key Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Constructs a PureVersionedS3Path object from bucket, key, and version ID. ```python >>> from s3path import PureVersionedS3Path >>> PureVersionedS3Path.from_bucket_key('pypi-proxy', 'boto3/index.html', version_id='') PureVersionedS3Path('/pypi-proxy/boto3/index.html', version_id='') ``` -------------------------------- ### S3Path.open() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Opens the Bucket key pointed to by the path, delegating to the smart_open library. Returns a file-like object for reading or writing. ```APIDOC ## S3Path.open(mode='r', buffering=-1, encoding=None, errors=None, newline=None) ### Description Opens the Bucket key pointed to by the path. This delegates to the smart_open library that handles the file streaming. Returns a file like object that you can read or write with. ### Method N/A (Method call) ### Endpoint N/A ### Parameters - **mode** (str): The mode in which to open the file (e.g., 'r', 'w', 'rb', 'wb'). Defaults to 'r'. - **buffering** (int): Optional buffering configuration. - **encoding** (str): The encoding to use for text mode. - **errors** (str): How encoding/decoding errors are handled. - **newline** (str): Controls universal newlines mode. ### Request Example ```python with S3Path('/pypi-proxy/botocore/index.html').open() as f: print(f.read()) ``` ### Response #### Success Response - **File-like object**: An object that supports read/write operations. #### Response Example ```html Package Index botocore-1.4.93.tar.gz
``` ``` -------------------------------- ### PureS3Path Constructor Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Initializes a PureS3Path object. Path segments are specified similarly to PurePath. ```APIDOC ## PureS3Path(*pathsegments) ### Description A subclass of PurePath, this path flavour represents AWS S3 Service semantics. ### Parameters * pathsegments: Variable length argument list for path components. ### Example ```python >>> from s3path import PureS3Path >>> PureS3Path('//') PureS3Path('//') ``` ``` -------------------------------- ### Writing and Reading Binary Content Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Demonstrates writing binary data to an S3 object and then reading it back. This is useful for handling non-textual data. ```python >>> S3Path('/test_bucket/test.txt').write_bytes(b'Binary file contents') >>> S3Path('/test_bucket/test.txt').read_bytes() b'Binary file contents' ``` -------------------------------- ### PureVersionedS3Path Constructor Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Initializes a PureVersionedS3Path object. Supports version_id for versioned S3 buckets. ```APIDOC ## PureVersionedS3Path(*pathsegments, version_id) ### Description A subclass of PureS3Path, this path flavour represents AWS S3 Service semantics for buckets in which S3 versioning is enabled. ### Parameters * pathsegments: Variable length argument list for path components. * version_id (string): A string that can be any valid AWS S3 version identifier. ### Example ```python >>> from s3path import PureVersionedS3Path >>> PureVersionedS3Path('//', version_id='') PureVersionedS3Path('//', version_id='') ``` ``` -------------------------------- ### List S3 Directory Contents Source: https://github.com/liormizr/s3path/blob/master/README.rst Import S3Path and iterate through directory contents to find subdirectories. ```python >>> from s3path import S3Path >>> bucket_path = S3Path('/pypi-proxy/') >>> [path for path in bucket_path.iterdir() if path.is_dir()] [S3Path('/pypi-proxy/requests/'), S3Path('/pypi-proxy/boto3/'), S3Path('/pypi-proxy/botocore/')] ``` -------------------------------- ### Handle Double Dots in PureS3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Demonstrates how PureS3Path handles double dots, differing from PurePath as S3 does not support symbolic links. ```python >>> PureS3Path('foo/../bar') PureS3Path('bar') ``` -------------------------------- ### VersionedS3Path Constructor Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Instantiate a VersionedS3Path object for S3 buckets with versioning enabled. Requires boto3. ```APIDOC ## VersionedS3Path Constructor ### Description Instantiates a concrete path object for versioned AWS S3 objects. Requires a version ID. ### Parameters * `*pathsegments` - Variable length argument list representing the bucket and key. * `version_id` (string) - The version ID of the S3 object. ### Request Example ```python from s3path import VersionedS3Path VersionedS3Path('//', version_id='') ``` ``` -------------------------------- ### Glob pattern matching in S3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Illustrates using the glob method to find paths matching a pattern within an S3 bucket or prefix. ```python >>> bucket_path = S3Path('/pypi-proxy/') >>> [path for path in bucket_path.glob('boto*')] [S3Path('/pypi-proxy/boto3/'), S3Path('/pypi-proxy/botocore/')] >>> [path for path in bucket_path.glob('*/*.html')] [S3Path('/pypi-proxy/requests/index.html'), S3Path('/pypi-proxy/boto3/index.html'), S3Path('/pypi-proxy/botocore/index.html')]] ``` -------------------------------- ### Globbing for HTML files Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Demonstrates how to find all HTML files within a bucket path recursively. Requires the S3Path library. ```python >>> bucket_path = S3Path('/pypi-proxy/') >>> list(bucket_path.glob('**/*.html')) [S3Path('/pypi-proxy/requests/index.html'), S3Path('/pypi-proxy/index.html'), S3Path('/pypi-proxy/boto3/index.html'), S3Path('/pypi-proxy/botocore/index.html')] ``` -------------------------------- ### S3Path.touch Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Creates a key at the specified path. If the key already exists, its modification time is updated if exist_ok is True. Otherwise, FileExistsError is raised. ```APIDOC ## S3Path.touch(exist_ok=True, **kwargs) ### Description Creates a key at the specified path. If the key already exists, its modification time is updated if exist_ok is True. Otherwise, FileExistsError is raised. ### Method Not applicable (Python method) ### Parameters #### Query Parameters - **exist_ok** (boolean) - Optional - If True, allows the operation to succeed if the key already exists, updating its modification time. ### Response #### Success Response Creates the key at the specified path or updates its modification time. ``` -------------------------------- ### Configure S3-Compatible Storage Source: https://github.com/liormizr/s3path/blob/master/docs/advance.rst Configure s3path to use S3-compatible storage services like LocalStack or MinIO by registering custom boto3 resources. This allows specifying endpoint URLs and credentials for these services. ```python >>> import boto3 >>> from botocore.client import Config >>> from s3path import PureS3Path, register_configuration_parameter >>> # Define path's for configuration >>> default_aws_s3_path = PureS3Path('/') >>> local_stack_bucket_path = PureS3Path('/LocalStackBucket/') >>> minio_bucket_path = PureS3Path('/MinIOBucket/') >>> # Define boto3 s3 resources >>> local_stack_resource = boto3.resource('s3', endpoint_url='http://localhost:4566') >>> minio_resource = boto3.resource( ... 's3', ... endpoint_url='http://localhost:9000', ... aws_access_key_id='minio', ... aws_secret_access_key='minio123', ... config=Config(signature_version='s3v4'), ... region_name='us-east-1') >>> # Configure and map root path's per boto3 parameters or resources >>> register_configuration_parameter(default_aws_s3_path, parameters={'ServerSideEncryption': 'AES256'}) >>> register_configuration_parameter(local_stack_bucket_path, resource=local_stack_resource) >>> register_configuration_parameter(minio_bucket_path, resource=minio_resource) ``` -------------------------------- ### Check if S3Path exists Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Demonstrates checking for the existence of an S3 path, which can be a bucket, key, or key prefix. ```python >>> S3Path('/pypi-proxy/boto3/index.html').exists() True >>> S3Path('/pypi-proxy/boto3/').exists() True >>> S3Path('/fake-bucket/').exists() False ``` -------------------------------- ### PureVersionedS3Path.from_bucket_key Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Creates a PureVersionedS3Path object from a bucket, key, and version ID. ```APIDOC ## PureVersionedS3Path.from_bucket_key(bucket, key, *, version_id) ### Description Represents a versioned AWS S3 Bucket and Key pairs as a PureVersionedS3Path. ### Parameters * bucket (string): The name of the S3 bucket. * key (string): The key within the S3 bucket. * version_id (string): The version ID associated with the S3 object. ### Returns (PureVersionedS3Path) A PureVersionedS3Path object representing the bucket, key, and version ID. ### Example ```python >>> from s3path import PureVersionedS3Path >>> PureVersionedS3Path.from_bucket_key('pypi-proxy', 'boto3/index.html', version_id='') PureVersionedS3Path('/pypi-proxy/boto3/index.html', version_id='') ``` ``` -------------------------------- ### Listing Directory Contents Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Iterates through the contents of a bucket path and filters for directories. This method yields path objects for each item within the specified directory. ```python >>> bucket_path = S3Path('/pypi-proxy/') >>> [path for path in bucket_path.iterdir() if path.is_dir()] [S3Path('/pypi-proxy/requests/'), S3Path('/pypi-proxy/boto3/'), S3Path('/pypy-proxy/botocore/')] ``` -------------------------------- ### PureS3Path.from_bucket_key Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Creates a PureS3Path object from a bucket and key. ```APIDOC ## PureS3Path.from_bucket_key(bucket, key) ### Description Represents a AWS S3 Bucket and Key pairs as a PureS3Path. ### Parameters * bucket (string): The name of the S3 bucket. * key (string): The key within the S3 bucket. ### Returns (PureS3Path) A PureS3Path object representing the bucket and key. ### Example ```python >>> PureS3Path.from_bucket_key('pypi-proxy', 'boto3/index.html') PureS3Path('/pypi-proxy/boto3/index.html') ``` ``` -------------------------------- ### Open and Read S3 File Content Source: https://github.com/liormizr/s3path/blob/master/README.rst Open an S3 file using a context manager and read its content. ```python >>> botocore_index_path = S3Path('/pypi-proxy/botocore/index.html') >>> with botocore_index_path.open() as f: >>> print(f.read()) """ Package Index botocore-1.4.93.tar.gz
""" ``` -------------------------------- ### Enabling Old Glob Algorithm Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Configures S3Path to use the older, pathlib-compatible glob algorithm. This is useful for specific compatibility needs but is deprecated in favor of the new algorithm. ```python register_configuration_parameter(path, glob_new_algorithm=False) ``` -------------------------------- ### Create PureS3Path from Bucket and Key Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Constructs a PureS3Path object from separate AWS S3 bucket and key names. ```python >>> PureS3Path.from_bucket_key('pypi-proxy', 'boto3/index.html') PureS3Path('/pypi-proxy/boto3/index.html') ``` -------------------------------- ### PureS3Path.as_uri Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Converts the path to an AWS S3 URI format. ```APIDOC ## PureS3Path.as_uri() ### Description Represents the path as a AWS S3 URI. ### Returns (string) The S3 URI representation of the path. ### Example ```python >>> p = PureS3Path('/pypi-proxy/boto3/') >>> p.as_uri() 's3://pypi-proxy/boto3/' >>> p = PureS3Path('/pypi-proxy/boto3/index.html') >>> p.as_uri() 's3://pypi-proxy/boto3/index.html' ``` ``` -------------------------------- ### Retrieve Subfolder Names in S3 Bucket Source: https://github.com/liormizr/s3path/blob/master/docs/comparison.rst Compares retrieving subfolder names within an S3 bucket prefix using S3Path and Boto3. ```python >>> from s3path import S3Path >>> for path in S3Path('/my-bucket/prefix-name-with-slash/').iterdir(): >>> if path.is_dir(): >>> print('sub folder : ', path) ``` ```python >>> import boto3 >>> s3_client = boto3.client('s3') >>> result = client.list_objects(Bucket='my-bucket', Prefix='prefix-name-with-slash/', Delimiter='/') >>> for o in result.get('CommonPrefixes'): >>> print('sub folder : ', o.get('Prefix')) ``` -------------------------------- ### VersionedS3Path.open() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Behaves the same as S3Path.open but uses the version_id to open a specific version of the object. ```APIDOC ## VersionedS3Path.open(mode='r', buffering=-1, encoding=None, errors=None, newline=None) ### Description Behaves the same as `S3Path.open` except that ``VersionedS3Path.version_id`` will be used to open the specified version of the object pointed to by the `VersionedS3Path` object. ### Method N/A (Method call) ### Endpoint N/A ### Parameters - **mode** (str): The mode in which to open the file (e.g., 'r', 'w', 'rb', 'wb'). Defaults to 'r'. - **buffering** (int): Optional buffering configuration. - **encoding** (str): The encoding to use for text mode. - **errors** (str): How encoding/decoding errors are handled. - **newline** (str): Controls universal newlines mode. ### Request Example N/A ### Response #### Success Response - **File-like object**: An object that supports read/write operations for a specific version. #### Response Example N/A ``` -------------------------------- ### S3Path.walk Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Walks the directory tree rooted at the current path, yielding directory and file names. This is useful for recursively processing contents of a bucket or prefix. ```APIDOC ## S3Path.walk(top_down=True, on_error=None, follow_symlinks=False) ### Description Walks the directory tree rooted at the current path, yielding directory and file names. This is useful for recursively processing contents of a bucket or prefix. ### Method Not applicable (Python method) ### Parameters #### Query Parameters - **top_down** (boolean) - Optional - If True, yields the directory path before its subdirectories. Defaults to True. - **on_error** (callable) - Optional - A function to call if an error occurs during the walk. - **follow_symlinks** (boolean) - Optional - If True, follows symbolic links (behavior may be limited in S3 context). Defaults to False. ### Response #### Success Response - Yields a 3-tuple for each directory: (dirpath, dirnames, filenames). - **dirpath** (string) - The path to the current directory. - **dirnames** (list of strings) - A list of subdirectory names within the current directory. - **filenames** (list of strings) - A list of file names within the current directory. ``` -------------------------------- ### List Specific File Types in S3 Source: https://github.com/liormizr/s3path/blob/master/README.rst Use glob to find all HTML files within a specified S3 path and its subdirectories. ```python >>> bucket_path = S3Path('/pypi-proxy/') >>> list(bucket_path.glob('**/*.html')) [S3Path('/pypi-proxy/requests/index.html'), S3Path('/pypi-proxy/boto3/index.html'), S3Path('/pypi-proxy/botocore/index.html')] ``` -------------------------------- ### PureS3Path.joinpath Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Joins path segments to the current path. Handles version_id propagation for PureVersionedS3Path. ```APIDOC ## PureS3Path.joinpath(*other) ### Description Joins path segments to the current path. If the final element of 'other' is a PureVersionedS3Path instance, the resulting object will also be a PureVersionedS3Path instance with version_id set to other[-1].version_id. Otherwise, the resulting object will be a PureS3Path instance. ### Parameters * other: Variable length argument list of path segments or path objects to join. ``` -------------------------------- ### PureVersionedS3Path.from_uri Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Creates a PureVersionedS3Path object from a versioned AWS S3 URI. ```APIDOC ## PureVersionedS3Path.from_uri(uri, *, version_id) ### Description Represents a versioned AWS S3 URI as a PureVersionedS3Path. ### Parameters * uri (string): The versioned AWS S3 URI to parse. * version_id (string): The version ID associated with the S3 object. ### Returns (PureVersionedS3Path) A PureVersionedS3Path object representing the URI and version ID. ### Example ```python >>> from s3path import PureVersionedS3Path >>> PureVersionedS3Path.from_uri('s3://pypi-proxy/boto3/index.html', version_id='') PureVersionedS3Path('/pypi-proxy/boto3/index.html', version_id='') ``` ``` -------------------------------- ### S3Path.write_text Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Opens the specified key in text mode, writes the provided string data to it, and then closes and saves the key. Encoding and error handling can be specified. ```APIDOC ## S3Path.write_text(data, encoding=None, errors=None, newline=None) ### Description Opens the specified key in text mode, writes the provided string data to it, and then closes and saves the key. Encoding and error handling can be specified. ### Method Not applicable (Python method) ### Parameters #### Request Body - **data** (string) - Required - The text data to write to the key. - **encoding** (string) - Optional - The encoding to use when writing the text. - **errors** (string) - Optional - Specifies how encoding errors are handled. - **newline** (string) - Optional - Specifies how newline characters are handled (Python 3.10+). ### Response #### Success Response Writes the provided text data to the specified S3 key. ``` -------------------------------- ### Query S3 Path Properties Source: https://github.com/liormizr/s3path/blob/master/README.rst Check if an S3 path exists and if it is a directory or a file. ```python >>> boto3_package_path = S3Path('/pypi-proxy/boto3/boto3-1.4.1.tar.gz') >>> boto3_package_path.exists() True >>> boto3_package_path.is_dir() False >>> boto3_package_path.is_file() True ``` -------------------------------- ### S3Path.is_dir() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Checks if the path points to a Bucket or a key prefix. Returns False if it points to a full key path or if the path does not exist. Other errors are propagated. ```APIDOC ## S3Path.is_dir() ### Description Returns ``True`` if the path points to a Bucket or a key prefix, ``False`` if it points to a full key path. ``False`` is also returned if the path doesn’t exist. Other errors (such as permission errors) are propagated. ### Method N/A (Method call) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response #### Success Response - **Boolean**: True if the path is a directory or prefix, False otherwise. #### Response Example N/A ``` -------------------------------- ### S3Path.exists() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Checks if the S3 path (bucket, key, or key prefix) exists. ```APIDOC ## S3Path.exists() ### Description Determines whether the path points to an existing Bucket, key, or key prefix in AWS S3. ### Response #### Success Response - `exists` (boolean) - True if the path exists, False otherwise. ### Request Example ```python S3Path('/pypi-proxy/boto3/index.html').exists() ``` ### Response Example ```json true ``` ``` -------------------------------- ### Unsupported cwd() Method in S3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Illustrates the NotImplementedError raised when attempting to use the cwd() method on an S3Path object, as AWS S3 does not support the concept of a current directory. ```python >>> S3Path('/test_bucket/test.txt').cwd() Traceback (most recent call last): File "", line 1, in File "/home/lior/lior_env/s3path/s3path.py", line 235, in cwd raise NotImplementedError(message) NotImplementedError: PathNotSupportedMixin.cwd is unsupported on AWS S3 service ``` -------------------------------- ### PureS3Path.from_uri Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Creates a PureS3Path object from an AWS S3 URI. ```APIDOC ## PureS3Path.from_uri(uri) ### Description Represents a AWS S3 URI as a PureS3Path. ### Parameters * uri (string): The AWS S3 URI to parse. ### Returns (PureS3Path) A PureS3Path object representing the URI. ### Example ```python >>> PureS3Path.from_uri('s3://pypi-proxy/boto3/index.html') PureS3Path('/pypi-proxy/boto3/index.html') ``` ``` -------------------------------- ### S3Path.is_file() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Checks if the path points to a Bucket key. Returns False if it points to a Bucket or a key prefix, or if the path does not exist. Other errors are propagated. ```APIDOC ## S3Path.is_file() ### Description Returns ``True`` if the path points to a Bucket key, ``False`` if it points to a Bucket or a key prefix. ``False`` is also returned if the path doesn’t exist. Other errors (such as permission errors) are propagated. ### Method N/A (Method call) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response #### Success Response - **Boolean**: True if the path is a file, False otherwise. #### Response Example N/A ``` -------------------------------- ### S3Path.iterdir() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst When the path points to a Bucket or a key prefix, yields path objects of the directory contents. ```APIDOC ## S3Path.iterdir() ### Description When the path points to a Bucket or a key prefix, yield path objects of the directory contents. ### Method N/A (Method call) ### Endpoint N/A ### Parameters N/A ### Request Example ```python bucket_path = S3Path('/pypi-proxy/') [path for path in bucket_path.iterdir() if path.is_dir()] ``` ### Response #### Success Response - **Iterator**: An iterator yielding S3Path objects for each item in the directory. #### Response Example ```python [S3Path('/pypi-proxy/requests/'), S3Path('/pypi-proxy/boto3/'), S3Path('/pypi-proxy/botocore/')] ``` ``` -------------------------------- ### Reading from an S3 Object Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Opens and reads the content of an S3 object using the S3Path interface. This utilizes the smart_open library for streaming and handles text content. ```python >>> with S3Path('/pypi-proxy/botocore/index.html').open() as f: >>> print(f.read()) ' Package Index botocore-1.4.93.tar.gz
' ``` -------------------------------- ### Writing and Reading Text Content Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Shows how to write text content to an S3 object and subsequently read it back. This method handles text encoding and decoding. ```python >>> S3Path('/test_bucket/test.txt').write_text('Text file contents') >>> S3Path('/test_bucket/test.txt').read_text() 'Text file contents' ``` -------------------------------- ### S3Path and VersionedS3Path Type Interactions Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Demonstrates the type returned when combining S3Path and VersionedS3Path objects with string paths. This shows how versioning information is handled during path concatenation. ```python >>> from s3path import S3Path, VersionedS3Path >>> str_path = "example/path" >>> s3_path = S3Path("example/path") >>> versioned_s3_path = VersionedS3Path("example/path", version_id="") >>> type(versioned_s3_path / versioned_s3_path) <<< s3path.VersionedS3Path >>> type(s3_path / versioned_s3_path) <<< s3path.VersionedS3Path >>> type(str_path / versioned_s3_path) <<< s3path.VersionedS3Path >>> type(versioned_s3_path / s3_path) <<< s3path.S3Path >>> type(versioned_s3_path / str_path) <<< s3path.S3Path ``` -------------------------------- ### Division Operator with PureVersionedS3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Describes the behavior of the division operator when used with PureVersionedS3Path instances. ```APIDOC ## Division Operator with PureVersionedS3Path ### Description The division of PureVersionedS3Path instances with other objects will yield the following types: * PureVersionedS3Path / PureVersionedS3Path -> PureVersionedS3Path * PureS3Path / PureVersionedS3Path -> PureVersionedS3Path * str / PureVersionedS3Path -> PureVersionedS3Path * PureVersionedS3Path / PureS3Path -> PureS3Path * PureVersionedS3Path / str -> PureS3Path ### Example ```python # Example usage would go here if provided in source ``` ``` -------------------------------- ### S3Path.read_text() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Returns the decoded contents of the Bucket key as a string. ```APIDOC ## S3Path.read_text(encoding=None, errors=None) ### Description Returns the decoded contents of the Bucket key as a string. ### Method N/A (Method call) ### Endpoint N/A ### Parameters - **encoding** (str): The encoding to use for decoding the text. Defaults to None. - **errors** (str): How decoding errors are handled. Defaults to None. ### Request Example ```python S3Path('/test_bucket/test.txt').write_text('Text file contents') S3Path('/test_bucket/test.txt').read_text() ``` ### Response #### Success Response - **str**: The decoded text content of the file. #### Response Example ```python 'Text file contents' ``` ``` -------------------------------- ### Create PureS3Path from S3 URI Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Parses an AWS S3 URI string into a PureS3Path object. ```python >>> PureS3Path.from_uri('s3://pypi-proxy/boto3/index.html') PureS3Path('/pypi-proxy/boto3/index.html') ``` -------------------------------- ### Recursive Globbing with S3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Use rglob to find files matching a pattern recursively within an S3 bucket or prefix. This is an alternative to glob_ with '**/' prepended to the pattern. ```python >>> bucket_path = S3Path('/pypi-proxy/') >>> list(bucket_path.rglob('*.html')) [S3Path('/pypi-proxy/requests/index.html'), S3Path('/pypi-proxy/index.html'), S3Path('/pypi-proxy/botocore/index.html')] ``` -------------------------------- ### S3Path.write_bytes Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Opens the specified key in bytes mode, writes the provided data to it, and then closes and saves the key. ```APIDOC ## S3Path.write_bytes(data) ### Description Opens the specified key in bytes mode, writes the provided data to it, and then closes and saves the key. ### Method Not applicable (Python method) ### Parameters #### Request Body - **data** (bytes) - Required - The binary data to write to the key. ### Response #### Success Response Writes the provided bytes data to the specified S3 key. ``` -------------------------------- ### Checking Path Equality with S3Path Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst The samefile method checks if two S3Path objects or a path object and a string refer to the same S3 bucket key. It returns True if they are the same, False otherwise. ```python >>> path = S3Path('/test_bucket/test.txt') >>> path.samefile(S3Path('/test_bucket/test.txt')) True >>> path.samefile('/test_bucket/fake') False ``` -------------------------------- ### S3Path.is_mount() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst AWS S3 Service does not have a mounting feature, so this method always returns False. ```APIDOC ## S3Path.is_mount() ### Description AWS S3 Service doesn't have mounting feature, There for this method will always return ``False``. ### Method N/A (Method call) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response #### Success Response - **Boolean**: Always False. #### Response Example N/A ``` -------------------------------- ### S3Path.samefile Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Checks if the current S3Path object points to the same S3 key as another path. The other path can be an S3Path object or a string. ```APIDOC ## S3Path.samefile(other_path) ### Description Checks if the current S3Path object points to the same S3 key as another path. The other path can be an S3Path object or a string. ### Method Not applicable (Python method) ### Parameters #### Path Parameters - **other_path** (S3Path or string) - Required - The path to compare against. ### Response #### Success Response - **True** if the paths point to the same S3 key. - **False** otherwise. ``` -------------------------------- ### S3Path.glob(pattern) Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Performs a glob search within the specified S3 path (bucket or key prefix) using a pattern. ```APIDOC ## S3Path.glob(pattern) ### Description Glob the given relative pattern in the Bucket or key prefix represented by this path, yielding all matching files. ### Parameters * `pattern` (string) - The glob pattern to match against S3 keys. ### Response #### Success Response - Yields `S3Path` objects for each matching file. ### Request Example ```python bucket_path = S3Path('/pypi-proxy/') matching_paths = [path for path in bucket_path.glob('boto*')] print(matching_paths) ``` ### Response Example ```json [ "S3Path('/pypi-proxy/boto3/')", "S3Path('/pypi-proxy/botocore/')" ] ``` ``` -------------------------------- ### S3Path.rmdir Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Removes an empty directory (key prefix) from S3. The directory must be empty for the operation to succeed. ```APIDOC ## S3Path.rmdir() ### Description Removes an empty directory (key prefix) from S3. The directory must be empty for the operation to succeed. ### Method Not applicable (Python method) ### Parameters None ### Response #### Success Response Removes the directory if it is empty. ``` -------------------------------- ### S3Path.replace() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Replaces the file or key at the target path. The behavior is similar to rename but may have different semantics depending on the underlying implementation. ```APIDOC ## S3Path.replace(target) ### Description Replaces this file or Bucket / key prefix / key to the given target. This method's behavior is analogous to `rename`, but it might offer different guarantees or semantics regarding existing targets. ### Method N/A (Method call) ### Endpoint N/A ### Parameters - **target** (str or S3Path): The new name or path for the file or prefix. ### Request Example N/A ### Response #### Success Response - **S3Path**: The target path object after replacement. #### Response Example N/A ``` -------------------------------- ### S3Path.owner() Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Returns the name of the user owning the Bucket or key, similar to boto3's ObjectSummary owner attribute. ```APIDOC ## S3Path.owner() ### Description Returns the name of the user owning the Bucket or key. Similarly to boto3's `ObjectSummary`_ owner attribute. ### Method N/A (Method call) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response #### Success Response - **str**: The name of the owner. #### Response Example N/A ``` -------------------------------- ### Create PureVersionedS3Path from Versioned S3 URI Source: https://github.com/liormizr/s3path/blob/master/docs/interface.rst Parses a versioned AWS S3 URI string into a PureVersionedS3Path object, including the version ID. ```python >>> from s3path import PureVersionedS3Path >>> PureVersionedS3Path.from_uri('s3://pypi-proxy/boto3/index.html', version_id='') PureVersionedS3Path('/pypi-proxy/boto3/index.html', version_id='') ```