### Install s3fs from GitHub Source

Source: https://s3fs.readthedocs.io/en/latest/_sources/install.rst

Instructions to download and install the s3fs library directly from its GitHub repository. This method is useful for developers or when needing the latest unreleased version.

```bash
git clone git@github.com:fsspec/s3fs
cd s3fs
python setup.py install
```

--------------------------------

### Install s3fs using pip

Source: https://s3fs.readthedocs.io/en/latest/install

Instructions to install the s3fs library using the pip package installer. This is a standard and widely used method for installing Python packages from PyPI.

```Shell
pip install s3fs
```

--------------------------------

### Install s3fs from source via Git

Source: https://s3fs.readthedocs.io/en/latest/install

Steps to download the s3fs library directly from its GitHub repository, navigate into the project directory, and install it using Python's setup.py script. This method is useful for developers or those needing the latest unreleased version.

```Shell
git clone git@github.com:fsspec/s3fs
cd s3fs
python setup.py install
```

--------------------------------

### Install s3fs using Pip

Source: https://s3fs.readthedocs.io/en/latest/_sources/install.rst

Instructions to install the s3fs library using the Python package installer, pip. This is a common method for installing Python packages.

```bash
pip install s3fs
```

--------------------------------

### Install Development and Test Dependencies

Source: https://s3fs.readthedocs.io/en/latest/_sources/development.rst

Installs all required Python packages for development and testing from the specified requirements files using pip. This ensures all necessary libraries are available for local development and test execution.

```bash
$ pip install -r requirements.txt -r test_requirements.txt
```

--------------------------------

### Install S3Fs Development Dependencies

Source: https://s3fs.readthedocs.io/en/latest/development

This command installs all necessary Python packages for S3Fs development and testing. It uses `pip` to read dependencies from both `requirements.txt` and `test_requirements.txt` files.

```bash
$ pip install -r requirements.txt -r test_requirements.txt
```

--------------------------------

### Install s3fs using Conda

Source: https://s3fs.readthedocs.io/en/latest/install

Instructions to install the s3fs library and its dependencies from the conda-forge repository using the conda package manager. This method is recommended for users who prefer Conda environments.

```Shell
$ conda install s3fs -c conda-forge
```

--------------------------------

### Python Example: S3FileSystem.set_session Usage

Source: https://s3fs.readthedocs.io/en/latest/api

Demonstrates how to initialize an `S3FileSystem` instance and use the `set_session` method asynchronously, as well as its blocking counterpart `connect`, to establish an S3 connection.

```python
s3 = S3FileSystem(profile="<profile name>")
# use in an async coroutine to assign the client object to a local variable
await s3.set_session()
# blocking version of set_session
s3.connect(refresh=True)
```

--------------------------------

### Install s3fs using Conda

Source: https://s3fs.readthedocs.io/en/latest/_sources/install.rst

Instructions to install the s3fs library and its dependencies from the conda-forge repository using the Conda package manager. This method ensures all necessary dependencies are handled automatically.

```bash
conda install s3fs -c conda-forge
```

--------------------------------

### s3fs.core.S3FileSystem.tree Method and Example

Source: https://s3fs.readthedocs.io/en/latest/api

Returns a string representation of the filesystem's directory structure, similar to the 'tree' command. It allows customization of the starting path, recursion depth, maximum number of items to display, and whether to show file sizes. Includes a Python example demonstrating its usage.

```APIDOC
tree(path: str = '/', recursion_limit: int = 2, max_display: int = 25, display_size: bool = False, prefix: str = '', is_last: bool = True, first: bool = True, indent_size: int = 4) -> str
  Return a tree-like structure of the filesystem starting from the given path as a string.
  Parameters:
    path (str): Root path to start traversal from
    recursion_limit (int): Maximum depth of directory traversal
    max_display (int): Maximum number of items to display per directory
    display_size (bool): Whether to display file sizes
    prefix (str): Current line prefix for visual tree structure
    is_last (bool): Whether current item is last in its level
    first (bool): Whether this is the first call (displays root path)
    indent_size (int): Number of spaces by indent
  Returns:
    str: A string representing the tree structure.
```

```python
from fsspec import filesystem
```

```python
fs = filesystem('ftp', host='test.rebex.net', user='demo', password='password')
tree = fs.tree(display_size=True, recursion_limit=3, indent_size=8, max_display=10)
print(tree)
```

--------------------------------

### Run S3Fs Unit Tests

Source: https://s3fs.readthedocs.io/en/latest/development

This command executes the test suite for the S3Fs project using `pytest`. Running tests is crucial to verify that changes to the codebase do not introduce regressions and that existing functionality works as expected.

```bash
$ pytest
```

--------------------------------

### Run Project Tests

Source: https://s3fs.readthedocs.io/en/latest/_sources/development.rst

Executes the project's test suite using the pytest framework. This command runs all discovered tests to verify the correctness and functionality of the codebase.

```bash
$ pytest
```

--------------------------------

### Python Example: S3FileSystem.split_path Basic Usage

Source: https://s3fs.readthedocs.io/en/latest/api

Demonstrates how `split_path` can be used to separate a standard S3 URI into its bucket and key components.

```python
split_path("s3://mybucket/path/to/file")
['mybucket', 'path/to/file', None]
```

--------------------------------

### Locate and Read a File with S3Fs

Source: https://s3fs.readthedocs.io/en/latest/_sources/index.rst

This example demonstrates how to initialize an anonymous S3FileSystem instance, list contents of a bucket, and open a file for binary reading using a context manager. It shows basic file access operations on S3.

```python
import s3fs
s3 = s3fs.S3FileSystem(anon=True)
s3.ls('my-bucket')
# Expected output: ['my-file.txt']
with s3.open('my-bucket/my-file.txt', 'rb') as f:
    print(f.read())
# Expected output: b'Hello, world'
```

--------------------------------

### Python Example: S3FileSystem.setxattr Usage

Source: https://s3fs.readthedocs.io/en/latest/api

Illustrates how to use `setxattr` to set custom attributes on an S3 file and how to pass additional parameters via `copy_kwargs` for the underlying S3 copy operation.

```python
mys3file.setxattr(attribute_1='value1', attribute_2='value2')
# Example for use with copy_args
mys3file.setxattr(copy_kwargs={'ContentType': 'application/pdf'},
    attribute_1='value1')
```

--------------------------------

### Open S3 File with S3FileSystem in Python

Source: https://s3fs.readthedocs.io/en/latest/api

This Python example demonstrates how to open a file on S3 using the `s3fs.core.S3FileSystem` and `s3fs.core.S3File` classes for reading. It shows the basic context manager usage to ensure proper file handling.

```python
s3 = S3FileSystem()
with s3.open('my-bucket/my-file.txt', mode='rb') as f:
    ...
```

--------------------------------

### S3FileSystem.connect Method and Example

Source: https://s3fs.readthedocs.io/en/latest/api

Establishes an S3 connection object. This asynchronous method can be called to refresh credentials or re-establish the session, providing a blocking version of `set_session`. The example demonstrates both asynchronous `set_session` and blocking `connect` usage.

```APIDOC
connect(refresh=False, kwargs={})
  refresh: bool (False) - If True, create a new session even if one already exists.
  kwargs: dict - Currently unused.
  Return type: Session to be closed later with await .close()
```

```Python
s3 = S3FileSystem(profile="<profile name>")
# use in an async coroutine to assign the client object to a local variable
await s3.set_session()
# blocking version of set_session
s3.connect(refresh=True)
```

--------------------------------

### Python Example: S3FileSystem.split_path with Version ID

Source: https://s3fs.readthedocs.io/en/latest/api

Shows how `split_path` handles S3 URIs that include a version ID, correctly extracting the bucket, key, and the version ID.

```python
split_path("s3://mybucket/path/to/versioned_file?versionId=some_version_id")
['mybucket', 'path/to/versioned_file', 'some_version_id']
```

--------------------------------

### Configure s3fs for MinIO with Auto-Discovery or Explicit Credentials

Source: https://s3fs.readthedocs.io/en/latest/_sources/index.rst

Demonstrates how to initialize `s3fs.S3FileSystem` for a self-hosted MinIO instance. Examples include relying on auto-discovery for credentials (requiring `anon=False` and `endpoint_url`) and explicitly passing `key` and `secret` along with the `endpoint_url`.

```python
# When relying on auto discovery for credentials
>>> s3 = s3fs.S3FileSystem(
      anon=False,
      endpoint_url='https://...'
   )
# Or passing the credentials directly
>>> s3 = s3fs.S3FileSystem(
      key='miniokey...',
      secret='asecretkey...',
      endpoint_url='https://...'
   )
```

--------------------------------

### Manage S3 Object Versions with s3fs

Source: https://s3fs.readthedocs.io/en/latest/index

This example illustrates how to enable version-aware support in `s3fs` to handle S3 buckets with object versioning. It shows how to initialize the filesystem for version awareness, open a file at its latest version, retrieve version information, and open a file at a specific historical version using its `version_id`.

```python
>>> s3 = s3fs.S3FileSystem(version_aware=True)
# Open the file at the latest version
>>> fo = s3.open('versioned_bucket/object')
>>> versions = s3.object_version_info('versioned_bucket/object')
# Open the file at a particular version
>>> fo_old_version = s3.open('versioned_bucket/object', version_id='SOMEVERSIONID')
```

--------------------------------

### Get Entry Details (s3fs.core.S3FileSystem)

Source: https://s3fs.readthedocs.io/en/latest/api

Provides detailed information about an entry at a given S3 path. Returns a dictionary with the same information as `ls` with `detail=True`. Some file systems might return `'size': None` if size cannot be determined. Keyword arguments are passed to `ls()`.

```APIDOC
s3fs.core.S3FileSystem.info(path: str, **kwargs)
  Description: Give details of entry at path. Returns a single dictionary, with exactly the same information as `ls` would with `detail=True`. The default implementation calls ls and could be overridden by a shortcut. kwargs are passed on to `ls()`. Some file systems might not be able to measure the file’s size, in which case, the returned dict will include 'size': None.
  Returns:
    dict: A dictionary with keys such as 'name' (full path), 'size' (in bytes, or None), 'type' (file, directory, etc.), and other FS-specific keys.
```

--------------------------------

### Configure s3fs for OVH S3 storage

Source: https://s3fs.readthedocs.io/en/latest/index

Provides an example for configuring `s3fs` to work with OVH's S3-compatible storage, including API key, secret, endpoint URL, region name, and signature version. The `config_kwargs` are crucial for specifying the correct signature version for OVH.

```python
s3 = s3fs.S3FileSystem(
   key='ovh-s3-key...',
   secret='ovh-s3-secretkey...',
   endpoint_url='https://s3.GRA.cloud.ovh.net',
   client_kwargs={
      'region_name': 'GRA'
   },
   config_kwargs={
      'signature_version': 's3v4'
   }
)
```

--------------------------------

### s3fs API Methods and Classes starting with 'M'

Source: https://s3fs.readthedocs.io/en/latest/genindex

This section lists s3fs API elements whose names begin with the letter 'M', including methods for bucket versioning, directory creation, merging, and metadata retrieval.

```APIDOC
s3fs.core.S3FileSystem.make_bucket_versioned()
s3fs.core.S3FileSystem.makedir()
s3fs.core.S3FileSystem.makedirs()
s3fs.core.S3FileSystem.merge()
s3fs.core.S3File.metadata()
s3fs.core.S3FileSystem.metadata()
s3fs.core.S3FileSystem.mkdir()
s3fs.core.S3FileSystem.mkdirs()
s3fs.core.S3FileSystem.modified()
s3fs.core.S3FileSystem.move()
s3fs.core.S3FileSystem.mv()
```

--------------------------------

### s3fs API Methods and Classes starting with 'P'

Source: https://s3fs.readthedocs.io/en/latest/genindex

This section lists s3fs API elements whose names begin with the letter 'P', including classes for parameter handling and methods for piping data and putting files/tags.

```APIDOC
s3fs.utils.ParamKwargsHelper (class)
s3fs.core.S3FileSystem.pipe()
s3fs.core.S3FileSystem.pipe_file()
s3fs.core.S3FileSystem.put()
s3fs.core.S3FileSystem.put_file()
s3fs.core.S3FileSystem.put_tags()
```

--------------------------------

### s3fs API Methods and Classes starting with 'S'

Source: https://s3fs.readthedocs.io/en/latest/genindex

This section lists s3fs API elements whose names begin with the letter 'S', including core classes like S3File and S3FileSystem, along with methods for seeking, session management, signing, sizing, and transaction handling.

```APIDOC
s3fs.core.S3File (class)
s3fs.core.S3FileSystem (class)
s3fs.mapping.S3Map()
s3fs.core.S3File.seek()
s3fs.core.S3File.seekable()
s3fs.core.S3FileSystem.set_session()
s3fs.core.S3File.setxattr()
s3fs.core.S3FileSystem.setxattr()
s3fs.core.S3FileSystem.sign()
s3fs.core.S3FileSystem.size()
s3fs.core.S3FileSystem.sizes()
s3fs.core.S3FileSystem.split_path()
s3fs.utils.SSEParams (class)
s3fs.core.S3FileSystem.start_transaction()
s3fs.core.S3FileSystem.stat()
```

--------------------------------

### Configure s3fs for Storj DCS with explicit or anonymous credentials

Source: https://s3fs.readthedocs.io/en/latest/index

Examples for connecting `s3fs` to Storj DCS via its S3-compatible Gateway, using either anonymous access or directly supplied access key and secret. This provides options for secure and flexible integration with Storj DCS.

```python
s3 = s3fs.S3FileSystem(
      anon=False,
      endpoint_url='https://gateway.storjshare.io'
   )
```

```python
s3 = s3fs.S3FileSystem(
      key='accesskey...',
      secret='asecretkey...',
      endpoint_url='https://gateway.storjshare.io'
   )
```

--------------------------------

### get

Source: https://s3fs.readthedocs.io/en/latest/api

Copies one or more files from the remote S3 filesystem to a local destination. It can handle single files, recursive directory copies, and glob patterns. If the local path ends with a '/', it's treated as a directory.

```APIDOC
s3fs.core.S3FileSystem.get(rpath, lpath, recursive=False, callback=<fsspec.callbacks.NoOpCallback object>, maxdepth
```

--------------------------------

### Configure s3fs for Storj DCS S3-compatible Gateway

Source: https://s3fs.readthedocs.io/en/latest/_sources/index.rst

Provides examples for connecting `s3fs` to Storj DCS via its S3-compatible Gateway. It covers both scenarios: relying on auto-discovery for credentials and explicitly providing `key` and `secret` along with the Storj gateway `endpoint_url`.

```python
# When relying on auto discovery for credentials
>>> s3 = s3fs.S3FileSystem(
      anon=False,
      endpoint_url='https://gateway.storjshare.io'
   )
# Or passing the credentials directly
>>> s3 = s3fs.S3FileSystem(
      key='accesskey...',
      secret='asecretkey...',
      endpoint_url='https://gateway.storjshare.io'
   )
```

--------------------------------

### s3fs API Methods and Classes starting with 'T'

Source: https://s3fs.readthedocs.io/en/latest/genindex

This section lists s3fs API elements whose names begin with the letter 'T', covering methods for tailing files, telling current position, converting to dictionary/JSON, touching files, and managing transactions.

```APIDOC
s3fs.core.S3FileSystem.tail()
s3fs.core.S3File.tell()
s3fs.core.S3FileSystem.to_dict()
s3fs.core.S3FileSystem.to_json()
s3fs.core.S3FileSystem.touch()
s3fs.core.S3FileSystem.transaction (property)
s3fs.core.S3FileSystem.transaction_type (attribute)
s3fs.core.S3FileSystem.tree()
s3fs.core.S3File.truncate()
```

--------------------------------

### Write Data to S3 with Blocked Caching

Source: https://s3fs.readthedocs.io/en/latest/_sources/index.rst

This example shows how to write large amounts of data to an S3 file using S3Fs with blocked caching. It initializes a non-anonymous S3FileSystem and writes data in chunks, demonstrating how data is flushed and the file size is reported.

```python
s3 = s3fs.S3FileSystem(anon=False) # uses default credentials
with s3.open('mybucket/new-file', 'wb') as f:
    f.write(2*2**20 * b'a')
    f.write(2*2**20 * b'a') # data is flushed and file closed
s3.du('mybucket/new-file')
# Expected output: {'mybucket/new-file': 4194304}
```

--------------------------------

### S3FileSystem Initialization Parameters

Source: https://s3fs.readthedocs.io/en/latest/api

Describes the various parameters available when initializing an S3FileSystem instance, including security, SSL usage, client configurations, caching, and concurrency settings.

```APIDOC
S3FileSystem Parameters:
  token (string, None): If not anonymous, use this security token, if specified.
  use_ssl (bool, True): Whether to use SSL in connections to S3; may be faster without, but insecure. If `use_ssl` is also set in `client_kwargs`, the value set in `client_kwargs` will take priority.
  s3_additional_kwargs (dict): Parameters that are used when calling s3 api methods. Typically used for things like "ServerSideEncryption".
  client_kwargs (dict): Parameters for the botocore client.
  requester_pays (bool, False): If RequesterPays buckets are supported.
  default_block_size (int, None): If given, the default block size value used for `open()`, if no specific value is given at all time. The built-in default is 50MB.
  default_fill_cache (Bool, True): Whether to use cache filling with open by default. Refer to `S3File.open`.
  default_cache_type (string, "readahead"): If given, the default cache_type value used for `open()`. Set to "none" if no caching is desired. See fsspec’s documentation for other available cache_type values. Default cache_type is “readahead”.
  version_aware (bool, False): Whether to support bucket versioning. If enable this will require the user to have the necessary IAM permissions for dealing with versioned objects. Note that in the event that you only need to work with the latest version of objects in a versioned bucket, and do not need the VersionId for those objects, you should set `version_aware` to False for performance reasons. When set to True, filesystem instances will use the S3 ListObjectVersions API call to list directory contents, which requires listing all historical object versions.
  cache_regions (bool, False): Whether to cache bucket regions or not. Whenever a new bucket is used, it will first find out which region it belongs and then use the client for that region.
  asynchronous (bool, False): Whether this instance is to be used from inside coroutines.
  config_kwargs (dict): Parameters passed to `botocore.client.Config`.
  kwargs (other parameters): Other parameters for core session.
  session (aiobotocore.AioSession): AioSession object to be used for all connections. This session will be used inplace of creating a new session inside S3FileSystem. For example: aiobotocore.session.AioSession(profile=’test_user’)
  max_concurrency (int, 10): The maximum number of concurrent transfers to use per file for multipart upload (`put()`) operations. Defaults to 10. When used in conjunction with `S3FileSystem.put(batch_size=...)` the maximum number of simultaneous connections is `max_concurrency * batch_size`. We may extend this parameter to affect `pipe()`, `cat()` and `get()`. Increasing this value will result in higher memory usage during multipart upload operations (by `max_concurrency * chunksize` bytes per file).
  fixed_upload_size (bool, False): Use same chunk size for all parts in multipart upload (last part can be smaller). Cloudflare R2 storage requires fixed_upload_size=True for multipart uploads.
  fsspec (parameters): The following parameters are passed on to fsspec.
  skip_instance_cache (control reuse): To control reuse of instances.
  use_listings_cache (control reuse): To control reuse of directory listings.
  listings_expiry_time (control reuse): To control reuse of directory listings.
  max_paths (control reuse): To control reuse of directory listings.
```

--------------------------------

### S3FileSystem Basic Usage - Listing Files

Source: https://s3fs.readthedocs.io/en/latest/api

Demonstrates how to initialize an anonymous S3FileSystem instance and list the contents of a specified S3 bucket.

```python
s3 = S3FileSystem(anon=False)
s3.ls('my-bucket/')
# Expected Output: ['my-file.txt']
```

--------------------------------

### Get Extended Attribute (s3fs.core.S3FileSystem)

Source: https://s3fs.readthedocs.io/en/latest/api

Retrieves an extended attribute from the metadata of an S3 object. Includes an example of usage.

```APIDOC
s3fs.core.S3FileSystem.getxattr(path: str, attr_name: str, **kwargs)
  Description: Get an attribute from the metadata.
  Examples:
    >>> mys3fs.getxattr('mykey', 'attribute_1')
    'value_1'
```

--------------------------------

### S3FileSystem.start_transaction Method API Documentation

Source: https://s3fs.readthedocs.io/en/latest/api

API documentation for the `start_transaction` method of `S3FileSystem`, which initiates a write transaction for deferring files outside of a context manager.

```APIDOC
S3FileSystem.start_transaction()
  Description: Begin write transaction for deferring files, non-context version
```

--------------------------------

### Get First Bytes of File (s3fs.core.S3FileSystem)

Source: https://s3fs.readthedocs.io/en/latest/api

Retrieves the first specified number of bytes from an S3 file.

```APIDOC
s3fs.core.S3FileSystem.head(path: str, size: int = 1024)
  Description: Get the first `size` bytes from file.
```

--------------------------------

### Get Delegated S3 Parameters (s3fs.core.S3FileSystem)

Source: https://s3fs.readthedocs.io/en/latest/api

Retrieves temporary credentials from AWS STS suitable for network transmission. This method is only relevant when key/secret credentials were explicitly provided.

```APIDOC
s3fs.core.S3FileSystem.get_delegated_s3pars(exp: int = 3600)
  Description: Get temporary credentials from STS, appropriate for sending across a network. Only relevant where the key/secret were explicitly provided.
  Parameters:
    exp (int): Time in seconds that credentials are good for (default: 3600).
  Returns:
    dict: A dictionary of parameters.
```

--------------------------------

### s3fs API Methods and Classes starting with 'O'

Source: https://s3fs.readthedocs.io/en/latest/genindex

This section lists s3fs API elements whose names begin with the letter 'O', specifically the 'open' method for file operations.

```APIDOC
s3fs.core.S3FileSystem.open()
```

--------------------------------

### s3fs.core.S3File.seek Method

Source: https://s3fs.readthedocs.io/en/latest/api

Sets the current file location (pointer) within the S3 object. The 'whence' parameter determines if the location is relative to the start, current position, or end of the file.

```APIDOC
seek(loc, whence=0)
  loc (int): The byte location to seek to.
  whence ({0, 1, 2}): Specifies the reference point for 'loc': 0 for start of file, 1 for current location, or 2 for end of file.
```

--------------------------------

### Configure s3fs for MinIO with explicit or anonymous credentials

Source: https://s3fs.readthedocs.io/en/latest/index

Demonstrates how to initialize `s3fs.S3FileSystem` for a self-hosted MinIO instance, either by relying on auto-discovery for credentials (anonymous access) or by directly providing API key and secret. This allows flexible authentication methods depending on security requirements.

```python
s3 = s3fs.S3FileSystem(
      anon=False,
      endpoint_url='https://...'
   )
```

```python
s3 = s3fs.S3FileSystem(
      key='miniokey...',
      secret='asecretkey...',
      endpoint_url='https://...'
   )
```

--------------------------------

### Locate and Read a File from S3 with s3fs

Source: https://s3fs.readthedocs.io/en/latest/index

Demonstrates how to initialize an anonymous S3FileSystem instance, list contents of an S3 bucket, and read a file from S3 using a context manager. Shows basic file system operations.

```python
>>> import s3fs
>>> s3 = s3fs.S3FileSystem(anon=True)
>>> s3.ls('my-bucket')
['my-file.txt']
>>> with s3.open('my-bucket/my-file.txt', 'rb') as f:
...     print(f.read())
b'Hello, world'
```

--------------------------------

### Utilize s3fs Asynchronous Operations

Source: https://s3fs.readthedocs.io/en/latest/index

Shows how to use `s3fs` in an asynchronous Python program. Demonstrates initializing `S3FileSystem` for async use, awaiting session creation, and closing the session, suitable for `asyncio` environments.

```python
async def run_program():
    s3 = S3FileSystem(..., asynchronous=True)
    session = await s3.set_session()
    ...  # perform work
    await session.close()

asyncio.run(run_program())  # or call from your async code
```

--------------------------------

### s3fs.core.S3File Class API Reference

Source: https://s3fs.readthedocs.io/en/latest/api

Comprehensive API documentation for the `s3fs.core.S3File` class, which represents an open S3 key as a file-like object. It covers the constructor parameters for instantiating an `S3File` and various methods for interacting with the S3 object, such as closing, committing, flushing, and retrieving metadata.

```APIDOC
s3fs.core.S3File:
  __init__(s3: S3FileSystem, path: string, mode: str = 'rb', block_size: int = 52428800, acl: str = False, version_id: str = None, fill_cache: bool = True, s3_additional_kwargs=None, autocommit: bool = True, cache_type: str = 'readahead', requester_pays: bool = False, cache_options=None, size=None)
    s3 (S3FileSystem): botocore connection
    path (string): S3 bucket/key to access
    mode (str): One of ‘rb’, ‘wb’, ‘ab’. These have the same meaning as they do for the built-in open function.
    block_size (int): read-ahead size for finding delimiters
    fill_cache (bool): If seeking to new a part of the file beyond the current buffer, with this True, the buffer will be filled between the sections to best support random access. When reading only a few specific chunks out of a file, performance may be better if False.
    acl (str): Canned ACL to apply
    version_id (str): Optional version to read the file at. If not specified this will default to the current version of the object. This is only used for reading.
    requester_pays (bool (False)): If RequesterPays buckets are supported.
    Description: Open S3 key as a file. Data is only loaded and cached on demand.

  close()
    Description: Close file. Finalizes writes, discards cache.

  commit()
    Description: Move from temp to final destination.

  discard()
    Description: Throw away temporary file.

  fileno()
    Description: Returns underlying file descriptor if one exists. OSError is raised if the IO object does not use a file descriptor.

  flush(force: bool = False)
    force (bool): When closing, write the last block even if it is smaller than blocks are allowed to be. Disallows further writing to this file.
    Description: Write buffered data to backend store. Writes the current buffer, if it is larger than the block-size, or if the file is being closed.

  getxattr(xattr_name, **kwargs)
    Description: Get an attribute from the metadata. See `getxattr()`.
    Example: mys3file.getxattr('attribute_1') returns 'value_1'

  info()
    Description: File information about this path.

  isatty()
    Description: Return whether this is an ‘interactive’ stream. Return False if it can’t be determined.

  metadata(refresh: bool = False, **kwargs)
    Description: Return metadata of file. See `metadata()`.
    Metadata is cached unless refresh=True.
```

--------------------------------

### Configure s3fs for Requester Pays Buckets

Source: https://s3fs.readthedocs.io/en/latest/index

This snippet demonstrates how to initialize an `s3fs.S3FileSystem` instance to enable the 'Requester Pays' feature for S3 buckets. This is required for accessing buckets where the data requester covers transfer fees, ensuring proper authentication and handling of potential charges.

```python
>>> s3 = s3fs.S3FileSystem(anon=False, requester_pays=True)
```

--------------------------------

### S3FileSystem.makedir Method

Source: https://s3fs.readthedocs.io/en/latest/api

This method serves as an alias for `AbstractFileSystem.mkdir`. It is used to create a directory at the specified path within the S3 file system.

```APIDOC
makedir(path, create_parents=True, **kwargs)
  Alias of AbstractFileSystem.mkdir.
```

--------------------------------

### Use S3Fs Asynchronously

Source: https://s3fs.readthedocs.io/en/latest/_sources/index.rst

This snippet demonstrates how to use S3Fs in an asynchronous context. It shows initializing an `S3FileSystem` for async operations, awaiting session creation, performing work, and closing the session, typically run within an `asyncio` event loop.

```python
async def run_program():
    s3 = S3FileSystem(..., asynchronous=True)
    session = await s3.set_session()
    # perform work
    await session.close()

asyncio.run(run_program()) # or call from your async code
```

--------------------------------

### Read Delimited Blocks from S3 File

Source: https://s3fs.readthedocs.io/en/latest/index

Illustrates reading a specific block of data from an S3 file, starting at a given offset and reading a specified length, with a delimiter for block termination. Useful for partial file reads.

```python
>>> s3.read_block(path, offset=1000, length=10, delimiter=b'\n')
b'A whole line of text\n'
```

--------------------------------

### current (classmethod)

Source: https://s3fs.readthedocs.io/en/latest/api

Returns the most recently instantiated FileSystem object. If no instance has been created yet, a new one will be initialized with default settings.

```APIDOC
s3fs.core.S3FileSystem.current()
  Returns (s3fs.core.S3FileSystem): The most recently instantiated FileSystem instance.
```

--------------------------------

### Read Byte Block from S3 File with s3fs.core.S3FileSystem.read_block

Source: https://s3fs.readthedocs.io/en/latest/api

Reads a specified number of bytes from a file on S3, starting at a given offset. Supports reading to delimiter boundaries and to the end of the file. The returned bytestring will include the end delimiter.

```APIDOC
read_block(fn, offset, length, delimiter=None)
  fn: string - Path to filename
  offset: int - Byte offset to start read
  length: int - Number of bytes to read. If None, read to end.
  delimiter: bytes (optional) - Ensure reading starts and stops at delimiter bytestring
```

```python
fs.read_block('data/file.csv', 0, 13)
b'Alice, 100\nBo'
```

```python
fs.read_block('data/file.csv', 0, 13, delimiter=b'\n')
b'Alice, 100\nBob, 200\n'
```

```python
fs.read_block('data/file.csv', 0, None, delimiter=b'n') # doctest: +SKIP
b'Alice, 100nBob, 200nCharlie, 300'
```

--------------------------------

### s3fs API Methods and Classes starting with 'R'

Source: https://s3fs.readthedocs.io/en/latest/genindex

This section lists s3fs API elements whose names begin with the letter 'R', covering various read operations for files and file systems, as well as rename and remove functionalities.

```APIDOC
s3fs.core.S3File.read()
s3fs.core.S3FileSystem.read_block()
s3fs.core.S3FileSystem.read_bytes()
s3fs.core.S3FileSystem.read_text()
s3fs.core.S3File.readable()
s3fs.core.S3File.readinto()
s3fs.core.S3File.readline()
s3fs.core.S3File.readlines()
s3fs.core.S3File.readuntil()
s3fs.core.S3FileSystem.rename()
s3fs.core.S3FileSystem.rm()
s3fs.core.S3FileSystem.rm_file()
s3fs.core.S3FileSystem.rmdir()
```

--------------------------------

### S3FileSystem.cp Method

Source: https://s3fs.readthedocs.io/en/latest/api

An alias for the `AbstractFileSystem.copy` method, providing a shorthand for copying operations within the filesystem.

```APIDOC
cp(path1, path2, **kwargs)
```

--------------------------------

### S3FileSystem.cat Method Documentation

Source: https://s3fs.readthedocs.io/en/latest/api

Detailed documentation for the `cat` method of `S3FileSystem`, explaining its purpose, parameters (recursive, on_error, kwargs), and return types for fetching file contents.

```APIDOC
cat(path, recursive=False, on_error='raise', **kwargs)
  Permalink: #s3fs.core.S3FileSystem.cat

  Description: Fetch (potentially multiple) paths’ contents

  Parameters:
    recursive (bool): If True, assume the path(s) are directories, and get all the contained files.
    on_error ("raise", "omit", "return"):
      - "raise": If raise, an underlying exception will be raised (converted to KeyError if the type is in self.missing_exceptions).
      - "omit": If omit, keys with exception will simply not be included in the output.
      - "return": If “return”, all keys are included in the output, but the value will be bytes or an exception instance.
    kwargs (passed to cat_file): Other parameters passed to cat_file.

  Returns: (Implicitly, the content of the paths)
```

--------------------------------

### Pass S3Fs Options to Pandas read_excel

Source: https://s3fs.readthedocs.io/en/latest/_sources/index.rst

This example illustrates how to pass S3Fs-specific configuration, such as anonymous access, directly through the `storage_options` argument when using libraries like pandas that accept S3 URLs. This allows fine-grained control over the S3 connection.

```python
df = pd.read_excel("s3://bucket/path/file.xls",
                           storage_options={"anon": True})
```

--------------------------------

### Create Key/Value Store Mapper (s3fs.core.S3FileSystem)

Source: https://s3fs.readthedocs.io/en/latest/api

Creates a MutableMapping interface to the file system at a given root path, allowing key/value store operations. Refer to `fsspec.mapping.FSMap` for more details.

```APIDOC
s3fs.core.S3FileSystem.get_mapper(root: str = '', check: bool = False, create: bool = False, missing_exceptions: Any = None)
  Description: Create key/value store based on this file-system. Makes a MutableMapping interface to the FS at the given root path. See `fsspec.mapping.FSMap` for further details.
```

--------------------------------

### Integrate s3fs with gzip and Pandas

Source: https://s3fs.readthedocs.io/en/latest/index

Demonstrates seamless integration of `s3fs` with Python's `gzip` module and the `pandas` library. Shows how to open a compressed CSV file from S3, decompress it on-the-fly, and load it into a Pandas DataFrame.

```python
>>> with s3.open('mybucket/my-file.csv.gz', 'rb') as f:
...     g = gzip.GzipFile(fileobj=f)  # Decompress data with gzip
...     df = pd.read_csv(g)           # Read CSV file with Pandas
```

--------------------------------

### s3fs.core.S3FileSystem and S3File Method Index

Source: https://s3fs.readthedocs.io/en/latest/genindex

This snippet provides an alphabetical listing of methods available within the s3fs.core.S3FileSystem and s3fs.core.S3File classes, as presented in the s3fs documentation index. It outlines the method names and their respective parent classes.

```APIDOC
s3fs.core.S3FileSystem:
  ukey()
  unstrip_protocol()
  upload()
  url()
  walk()
  write_bytes()
  write_text()
s3fs.core.S3File:
  url()
  writable()
  write()
  writelines()
```

--------------------------------

### s3fs Library Core API Reference

Source: https://s3fs.readthedocs.io/en/latest/_sources/api.rst

This section outlines the main classes and functions available in the s3fs library, including S3FileSystem for general file system operations, S3File for file-like object interactions, and utility components.

```APIDOC
Module: s3fs.core
  Class: S3FileSystem
    Description: Represents an S3-compatible file system.
    Members:
      - cat
      - du
      - exists
      - find
      - get
      - glob
      - info
      - ls
      - mkdir
      - mv
      - open
      - put
      - read_block
      - rm
      - tail
      - touch
  Class: S3File
    Description: Represents a file object within the S3 file system.
    Members:
      - close
      - flush
      - info
      - read
      - seek
      - tell
      - write

Module: s3fs.mapping
  Function: S3Map
    Description: Provides a mutable mapping interface to an S3 bucket or prefix.

Module: s3fs.utils
  Class: ParamKwargsHelper
    Description: Helper class for managing keyword arguments for S3 operations.
  Class: SSEParams
    Description: Class for handling Server-Side Encryption (SSE) parameters.
```

--------------------------------

### S3Fs Core API Methods and Properties Index

Source: https://s3fs.readthedocs.io/en/latest/genindex

An alphabetical index of methods and properties for the `s3fs.core.S3FileSystem` and `s3fs.core.S3File` classes, providing quick navigation to their detailed API documentation within the S3Fs library.

```APIDOC
s3fs.core.S3FileSystem:
  Methods:
    cat()
    cat_file()
    cat_ranges()
    checksum()
    chmod()
    clear_multipart_uploads()
    connect()
    copy()
    cp()
    created()
    delete()
    disk_usage()
    download()
    du()
    end_transaction()
    exists()
    expand_path()
    find()
    get()
    get_delegated_s3pars()
    get_file()
    get_mapper()
    get_tags()
    getxattr()
    glob()
    head()
    info()
    invalidate_cache()
    invalidate_region_cache()
    isdir()
    isfile()
    lexists()
    listdir()
    ls()
  Class Methods:
    clear_instance_cache()
    current()
  Static Methods:
    from_dict()
    from_json()
  Properties:
    fsid

s3fs.core.S3File:
  Methods:
    close()
    commit()
    discard()
    fileno()
    flush()
    getxattr()
    info()
    isatty()
```

--------------------------------

### S3FileSystem Basic Usage - Reading File Contents

Source: https://s3fs.readthedocs.io/en/latest/api

Illustrates how to open a file from an S3 bucket in binary read mode and print its contents using the S3FileSystem.

```python
with s3.open('my-bucket/my-file.txt', mode='rb') as f:
    print(f.read())
# Expected Output: b'Hello, world!'
```

--------------------------------

### from_dict (static)

Source: https://s3fs.readthedocs.io/en/latest/api

Recreates a filesystem instance from a dictionary representation. Users should be cautious as this method can import arbitrary modules based on the 'cls' key, potentially executing malicious code.

```APIDOC
s3fs.core.S3FileSystem.from_dict(dct: dict[str, Any]) -> AbstractFileSystem
  dct (Dict[str, Any]): The dictionary representation of the filesystem instance.
  Returns (AbstractFileSystem): A file system instance, not necessarily of this particular class.
```

--------------------------------

### S3FileSystem.sizes Method API Documentation

Source: https://s3fs.readthedocs.io/en/latest/api

API documentation for the `sizes` method of `S3FileSystem`, which returns the size in bytes for each file in a list of paths.

```APIDOC
S3FileSystem.sizes(paths)
  Description: Size in bytes of each file in a list of paths
```