### Install development version

Source: https://cloudpathlib.drivendata.org/stable

Installs the latest development version from GitHub with all dependencies.

```bash
pip install https://github.com/drivendataorg/cloudpathlib.git#egg=cloudpathlib[all]
```

--------------------------------

### Install cloudpathlib with extras

Source: https://cloudpathlib.drivendata.org/stable

Installs the library with specific cloud provider dependencies.

```bash
pip install cloudpathlib[s3,gs,azure]
```

```bash
pip install "cloudpathlib[s3,gs,azure]"
```

--------------------------------

### Install Development Requirements

Source: https://cloudpathlib.drivendata.org/stable/contributing

Install all development dependencies and an editable version of cloudpathlib by running 'make reqs'.

```bash
make reqs

```

--------------------------------

### Install cloudpathlib via conda

Source: https://cloudpathlib.drivendata.org/stable

Installs the library with a specific cloud provider dependency using conda.

```bash
conda install cloudpathlib-s3 -c conda-forge
```

--------------------------------

### Default Client Management

Source: https://cloudpathlib.drivendata.org/stable/api-reference/client

Methods for getting and setting the default client instance.

```APIDOC
## Default Client Management

### Description
Provides methods to access and manage the default client instance used by cloud path objects.

### Method
`get_default_client()`

### Description
Get the default client, which is the one that is used when instantiating a cloud path instance for this cloud without a client specified.

### Method
`set_as_default_client()`

### Description
Set this client instance as the default one used when instantiating cloud path instances for this cloud without a client specified.

### Request Example
```python
from cloudpathlib import Client

# Get the default client
default_client = Client.get_default_client()

# Create a new client and set it as default
new_client = Client()
new_client.set_as_default_client()
```

### Response
`get_default_client()` returns a `Client` instance.
`set_as_default_client()` returns `None`.
```

--------------------------------

### PR Checklist Example

Source: https://cloudpathlib.drivendata.org/stable/contributing

A checklist to ensure all requirements are met before submitting a Pull Request. This includes reading contribution guidelines, rebasing, testing, and updating documentation.

```markdown
 - [ ] I have read and understood `CONTRIBUTING.md`
 - [ ] Confirmed an issue exists for the PR, and the text `Closes #issue` appears in the PR summary (e.g., `Closes #123`).
 - [ ] Confirmed PR is rebased onto the latest base
 - [ ] Confirmed failure before change and success after change
 - [ ] Any generic new functionality is replicated across cloud providers if necessary
 - [ ] Tested manually against live server backend for at least one provider
 - [ ] Added tests for any new functionality
 - [ ] Linting passes locally
 - [ ] Tests pass locally
 - [ ] Updated `HISTORY.md` with the issue that is addressed and the PR you are submitting. If the top section is not `## UNRELEASED``, then you need to add a new section to the top of the document for your change.
```

--------------------------------

### Define a library function using built-in open

Source: https://cloudpathlib.drivendata.org/stable/patching_builtins

An example of a function that uses the standard built-in open, which would fail if passed a CloudPath without patching.

```python
# Imagine that deep in a third-party library a function is implemented like this
def library_function(filepath: str):
    with open(filepath, "w") as f:
        f.write("hello!")
```

--------------------------------

### Get Client Instance from Rig

Source: https://cloudpathlib.drivendata.org/stable/contributing

Obtain an instance of the client class associated with the test rig using `rig.client_class()`. This is useful for testing functionality directly on the client.

```python
new_client = rig.client_class()

```

--------------------------------

### Serve Documentation Locally

Source: https://cloudpathlib.drivendata.org/stable/contributing

Use `make docs-serve` to run a local development server for the documentation. This server auto-reloads for most changes. Note that changes to `index.md`, `HISTORY.md`, or `CONTRIBUTING.md` require running `make docs-setup` and restarting the server.

```bash
make docs-serve

```

--------------------------------

### Build Documentation

Source: https://cloudpathlib.drivendata.org/stable/contributing

Run `make docs` to build the latest version of the documentation. If you modify methods on `CloudPath` or its subclasses, also run `python docs/make_support_table.py` and update `README.md`.

```bash
make docs

```

--------------------------------

### Performance Test Report Example

Source: https://cloudpathlib.drivendata.org/stable/contributing

This is an example of the output generated by the performance tests, which should be included in PR descriptions when changes affect listing, globbing, or walking operations.

```text
                                  Performance suite results: (2023-10-08T13:18:04.774823)                                  
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ Test Name      ┃ Config Name                ┃ Iterations ┃           Mean ┃              Std ┃            Max ┃ N Items ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ List Folders   │ List shallow recursive     │         10 │ 0:00:00.862476 │ ± 0:00:00.020222 │ 0:00:00.898143 │   5,500 │
│ List Folders   │ List shallow non-recursive │         10 │ 0:00:00.884997 │ ± 0:00:00.086678 │ 0:00:01.117775 │   5,500 │
│ List Folders   │ List normal recursive      │         10 │ 0:00:01.248844 │ ± 0:00:00.095575 │ 0:00:01.506868 │   7,877 │
│ List Folders   │ List normal non-recursive  │         10 │ 0:00:00.060042 │ ± 0:00:00.003986 │ 0:00:00.064052 │     113 │
│ List Folders   │ List deep recursive        │         10 │ 0:00:02.004731 │ ± 0:00:00.130264 │ 0:00:02.353263 │   7,955 │
│ List Folders   │ List deep non-recursive    │         10 │ 0:00:00.054268 │ ± 0:00:00.003314 │ 0:00:00.062116 │      31 │
│ Glob scenarios │ Glob shallow recursive     │         10 │ 0:00:01.056946 │ ± 0:00:00.160470 │ 0:00:01.447082 │   5,500 │
│ Glob scenarios │ Glob shallow non-recursive │         10 │ 0:00:00.978217 │ ± 0:00:00.091849 │ 0:00:01.230822 │   5,500 │
│ Glob scenarios │ Glob normal recursive      │         10 │ 0:00:01.510334 │ ± 0:00:00.101108 │ 0:00:01.789393 │   7,272 │
│ Glob scenarios │ Glob normal non-recursive  │         10 │ 0:00:00.058301 │ ± 0:00:00.002621 │ 0:00:00.063299 │      12 │
│ Glob scenarios │ Glob deep recursive        │         10 │ 0:00:02.784629 │ ± 0:00:00.099764 │ 0:00:02.981882 │   7,650 │
│ Glob scenarios │ Glob deep non-recursive    │         10 │ 0:00:00.051322 │ ± 0:00:00.002653 │ 0:00:00.054844 │      25 │
│ Walk scenarios │ Walk shallow               │         10 │ 0:00:00.905571 │ ± 0:00:00.076332 │ 0:00:01.113957 │   5,500 │
│ Walk scenarios │ Walk normal                │         10 │ 0:00:01.441215 │ ± 0:00:00.014923 │ 0:00:01.470414 │   7,272 │
│ Walk scenarios │ Walk deep                  │         10 │ 0:00:02.461520 │ ± 0:00:00.031832 │ 0:00:02.539132 │   7,650 │
└────────────────┴────────────────────────────┴────────────┴────────────────┴──────────────────┴────────────────┴─────────┘
```

--------------------------------

### Constructor: __init__

Source: https://cloudpathlib.drivendata.org/stable/api-reference/azblobclient

Initializes the Azure Blob storage client with various authentication and configuration options.

```APIDOC
## __init__

### Description
Class constructor. Sets up a BlobServiceClient. Supports multiple authentication methods including environment variables, connection strings, account URLs, or existing service clients.

### Parameters
#### Request Body
- **account_url** (Optional[str]) - Optional - The URL to the blob storage account, optionally authenticated with a SAS token.
- **credential** (Optional[Any]) - Optional - Credentials with which to authenticate.
- **connection_string** (Optional[str]) - Optional - A connection string to an Azure Storage account.
- **blob_service_client** (Optional[BlobServiceClient]) - Optional - Instantiated BlobServiceClient.
- **data_lake_client** (Optional[DataLakeServiceClient]) - Optional - Instantiated DataLakeServiceClient.
- **file_cache_mode** (Optional[Union[str, FileCacheMode]]) - Optional - How often to clear the file cache.
- **local_cache_dir** (Optional[Union[str, PathLike]]) - Optional - Path to directory to use as cache for downloaded files.
- **content_type_method** (Optional[Callable]) - Optional - Function to call to guess media type (mimetype) when writing a file to the cloud.
```

--------------------------------

### Get path information with pathlib

Source: https://cloudpathlib.drivendata.org/stable/why_cloudpathlib

Demonstrates how to get various attributes of a resolved path, such as name, stem, suffix, parent, and read its content. This is useful for understanding and manipulating file paths.

```python
notebook = Path("why_cloudpathlib.ipynb").resolve()

print(f"{ 'Path:':15}{notebook}")
print(f"{ 'Name:':15}{notebook.name}")
print(f"{ 'Stem:':15}{notebook.stem}")
print(f"{ 'Suffix:':15}{notebook.suffix}")
print(f"{ 'With suffix:':15}{notebook.with_suffix('.cpp')}")
print(f"{ 'Parent:':15}{notebook.parent}")
print(f"{ 'Read_text:'}
{notebook.read_text()[:200]}
")
```

--------------------------------

### Available Developer Commands with Make

Source: https://cloudpathlib.drivendata.org/stable/contributing

Run 'make' to see a list of available developer commands for managing the project, such as cleaning artifacts, building, testing, and releasing.

```bash
❯ make
clean                remove all build, test, coverage and Python artifacts
clean-build          remove build artifacts
clean-pyc            remove Python file artifacts
clean-test           remove test and coverage artifacts
dist                 builds source and wheel package
docs-setup           setup docs pages based on README.md and HISTORY.md
docs                 build the static version of the docs
docs-serve           serve documentation to livereload while you work
format               run black to format codebase
install              install the package to the active Python's site-packages
lint                 check style with black, flake8, and mypy
release              package and upload a release
reqs                 install development requirements
test                 run tests with mocked cloud SDKs
test-debug           rerun tests that failed in last run and stop with pdb at failures
test-live-cloud      run tests on live cloud backends
perf                 run performance measurement suite for s3 and save results to perf-results.csv

```

--------------------------------

### GET /rglob

Source: https://cloudpathlib.drivendata.org/stable/api-reference/cloudpath

Recursively glob files matching a pattern.

```APIDOC
## GET rglob

### Description
Recursively glob files matching a pattern.

### Parameters
#### Query Parameters
- **pattern** (str) - Required - The glob pattern to match.
- **case_sensitive** (bool) - Optional - Whether the match is case sensitive.
- **recurse_symlinks** (bool) - Optional - Whether to recurse into symlinks.
```

--------------------------------

### Constructor: __init__

Source: https://cloudpathlib.drivendata.org/stable/api-reference/s3client

Initializes the S3 client with authentication, session, and caching configurations.

```APIDOC
## __init__

### Description
Class constructor. Sets up a boto3 Session. Directly supports the same authentication interface, as well as the same environment variables supported by boto3. If no authentication arguments or environment variables are provided, then the client will be instantiated as anonymous, which will only have access to public buckets.

### Parameters
#### Request Body
- **aws_access_key_id** (Optional[str]) - Optional - AWS access key ID.
- **aws_secret_access_key** (Optional[str]) - Optional - AWS secret access key.
- **aws_session_token** (Optional[str]) - Optional - Session key for your AWS account. This is only needed when you are using temporary credentials.
- **no_sign_request** (Optional[bool]) - Optional - If True, credentials are not looked for and we use unsigned requests to fetch resources. Default: False.
- **botocore_session** (Optional[Session]) - Optional - An already instantiated botocore Session.
- **profile_name** (Optional[str]) - Optional - Profile name of a profile in a shared credentials file.
- **boto3_session** (Optional[Session]) - Optional - An already instantiated boto3 Session.
- **file_cache_mode** (Optional[Union[str, FileCacheMode]]) - Optional - How often to clear the file cache.
- **local_cache_dir** (Optional[Union[str, PathLike]]) - Optional - Path to directory to use as cache for downloaded files.
- **endpoint_url** (Optional[str]) - Optional - S3 server endpoint URL to use for the constructed boto3 S3 resource and client.
- **boto3_transfer_config** (Optional[dict]) - Optional - Instantiated TransferConfig for managing s3 transfers.
- **content_type_method** (Optional[Callable]) - Optional - Function to call to guess media type (mimetype) when writing a file to the cloud. Default: mimetypes.guess_type.
- **extra_args** (Optional[dict]) - Optional - A dictionary of extra args passed to download, upload, and list functions as relevant.
```

--------------------------------

### GET /read_text

Source: https://cloudpathlib.drivendata.org/stable/api-reference/cloudpath

Reads the entire content of the cloud file as a string.

```APIDOC
## GET read_text

### Description
Reads the entire content of the cloud file as a string.

### Parameters
#### Query Parameters
- **encoding** (str) - Optional - The character encoding to use.
- **errors** (str) - Optional - How to handle encoding errors.
- **newline** (str) - Optional - How to handle newlines.

### Response
#### Success Response (200)
- **str** (str) - The content of the file.
```

--------------------------------

### Run Full Test Suite with Mocked Backends

Source: https://cloudpathlib.drivendata.org/stable/contributing

Execute the complete test suite using mocked, local backends by running 'make test'. This avoids network calls.

```bash
make test

```

--------------------------------

### GET /read_bytes

Source: https://cloudpathlib.drivendata.org/stable/api-reference/cloudpath

Reads the entire content of the cloud file as bytes.

```APIDOC
## GET read_bytes

### Description
Reads the entire content of the cloud file as bytes.

### Response
#### Success Response (200)
- **bytes** (bytes) - The content of the file.
```

--------------------------------

### Client Initialization

Source: https://cloudpathlib.drivendata.org/stable/api-reference/client

Initializes a Client instance with optional caching configurations.

```APIDOC
## Client Initialization

### Description
Initializes a Client instance with optional caching configurations.

### Method
`__init__`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from cloudpathlib import Client

# Initialize with default settings
client = Client()

# Initialize with custom cache directory and mode
client_custom = Client(file_cache_mode='persistent', local_cache_dir='/path/to/cache')
```

### Response
None (initializes the client object)

#### Success Response (200)
None

#### Response Example
None
```

--------------------------------

### Get Absolute Path

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gspath

Returns the absolute path representation of the object.

```python
def absolute(self) -> Self:
    return self
```

--------------------------------

### GET /public_url

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gsclient

Retrieves the public URL for a specified cloud path.

```APIDOC
## GET /public_url

### Description
Retrieves the public URL for a given cloud path object.

### Method
GET

### Endpoint
/public_url

### Parameters
#### Request Body
- **cloud_path** (GSPath) - Required - The path to the cloud object.
```

--------------------------------

### Run Live Cloud Backend Tests

Source: https://cloudpathlib.drivendata.org/stable/contributing

Execute tests against live cloud provider servers by running 'make test-live-cloud'. Ensure you have configured credentials for the providers you wish to test against.

```bash
make test-live-cloud

```

--------------------------------

### GET _get_metadata

Source: https://cloudpathlib.drivendata.org/stable/api-reference/s3client

Retrieves metadata for a specific cloud path object.

```APIDOC
## GET _get_metadata

### Description
Retrieves metadata for a given S3Path object, including last modified date, size, etag, and content type.

### Parameters
#### Path Parameters
- **cloud_path** (S3Path) - Required - The cloud path object to retrieve metadata for.

### Response
#### Success Response (200)
- **last_modified** (datetime) - The last modified timestamp.
- **size** (int) - The size of the object in bytes.
- **etag** (str) - The ETag of the object.
- **content_type** (str) - The MIME type of the object.
- **extra** (dict) - Additional metadata associated with the object.
```

--------------------------------

### Perform basic cloud path operations

Source: https://cloudpathlib.drivendata.org/stable

Demonstrates globbing, path joining, reading, writing, and deleting files in cloud storage.

```python
from cloudpathlib import CloudPath

# dispatches to S3Path based on prefix
root_dir = CloudPath("s3://drivendata-public-assets/")
root_dir
#> S3Path('s3://drivendata-public-assets/')

# there's only one file, but globbing works in nested folder
for f in root_dir.glob('**/*.txt'):
    text_data = f.read_text()
    print(f)
    print(text_data)
#> s3://drivendata-public-assets/odsc-west-2019/DATA_DICTIONARY.txt
#> Eviction Lab Data Dictionary
#>
#> Additional information in our FAQ evictionlab.org/help-faq/
#> Full methodology evictionlab.org/methods/
#>
#> ... (additional text output truncated)

# use / to join paths (and, in this case, create a new file)
new_file_copy = root_dir / "nested_dir/copy_file.txt"
new_file_copy
#> S3Path('s3://drivendata-public-assets/nested_dir/copy_file.txt')

# show things work and the file does not exist yet
new_file_copy.exists()
#> False

# writing text data to the new file in the cloud
new_file_copy.write_text(text_data)
#> 6933

# file now listed
list(root_dir.glob('**/*.txt'))
#> [S3Path('s3://drivendata-public-assets/nested_dir/copy_file.txt'),
#>  S3Path('s3://drivendata-public-assets/odsc-west-2019/DATA_DICTIONARY.txt')]

# but, we can remove it
new_file_copy.unlink()

# no longer there
list(root_dir.glob('**/*.txt'))
#> [S3Path('s3://drivendata-public-assets/odsc-west-2019/DATA_DICTIONARY.txt')]
```

--------------------------------

### Create Directory

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gspath

Placeholder method for directory creation, as empty directories are not supported on cloud storage.

```python
def mkdir(self, parents=False, exist_ok=False, mode: Optional[Any] = None):
    # not possible to make empty directory on cloud storage
    pass
```

--------------------------------

### GET /client/path

Source: https://cloudpathlib.drivendata.org/stable/api-reference/client

Generates a BoundedCloudPath object from a string or existing path.

```APIDOC
## GET /client/path

### Description
Creates a new BoundedCloudPath instance associated with the current client.

### Parameters
#### Query Parameters
- **cloud_path** (string/BoundedCloudPath) - Required - The base path string or object.
- **parts** (string) - Optional - Additional path components to join.
```

--------------------------------

### Initialize Azure Blob Client

Source: https://cloudpathlib.drivendata.org/stable/api-reference/azblobclient

Constructor for setting up the Azure Blob client with support for multiple authentication strategies and local caching configurations.

```python
def __init__(
    self,
    account_url: Optional[str] = None,
    credential: Optional[Any] = None,
    connection_string: Optional[str] = None,
    blob_service_client: Optional["BlobServiceClient"] = None,
    data_lake_client: Optional["DataLakeServiceClient"] = None,
    file_cache_mode: Optional[Union[str, FileCacheMode]] = None,
    local_cache_dir: Optional[Union[str, os.PathLike]] = None,
    content_type_method: Optional[Callable] = mimetypes.guess_type,
):
```

--------------------------------

### Get URI Representation

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gspath

Returns the URI string representation of the cloud path.

```python
def as_uri(self) -> str:
    return str(self)
```

--------------------------------

### Handle file: URI schemes with AnyPath

Source: https://cloudpathlib.drivendata.org/stable/anypath-polymorphism

Shows how to parse file: URIs into local paths and convert them back to URI strings using as_uri.

```python
from cloudpathlib import AnyPath

# hostname omitted variant
path = AnyPath("file:/root/mydir/myfile.txt")
path
#> PosixPath('/root/mydir/myfile.txt')

# explicit local path variant
path = AnyPath("file:///root/mydir/myfile.txt")
path
#> PosixPath('/root/mydir/myfile.txt')

# manipulate the path and return the file:// URI
parent_uri = path.parent.as_uri()
parent_uri
#> 'file:///root/mydir'
```

--------------------------------

### Validate cloud path

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gspath

Checks if a string or path object starts with the expected cloud prefix.

```python
@classmethod
def is_valid_cloudpath(
    cls, path: Union[str, "CloudPath"], raise_on_error: bool = False
) -> Union[bool, TypeGuard[Self]]:
    valid = str(path).lower().startswith(cls.cloud_prefix.lower())

    if raise_on_error and not valid:
        raise InvalidPrefixError(
            f"'{path}' is not a valid path since it does not start with '{cls.cloud_prefix}'"
        )

    return valid
```

--------------------------------

### GSClient Constructor

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gsclient

Initializes a new GSClient instance with specified authentication and configuration parameters.

```APIDOC
## GSClient Constructor

### Description
Initializes the GSClient, which sets up a Storage Client for Google Cloud Storage. It supports multiple authentication methods, including existing clients, OAuth2 credentials, service account files, or default application credentials.

### Parameters
#### Request Body
- **application_credentials** (Optional[Union[str, PathLike]]) - Optional - Path to Google service account credentials file.
- **credentials** (Optional[Credentials]) - Optional - The OAuth2 Credentials to use for this client.
- **project** (Optional[str]) - Optional - The project which the client acts on behalf of.
- **storage_client** (Optional[Client]) - Optional - Instantiated StorageClient.
- **file_cache_mode** (Optional[Union[str, FileCacheMode]]) - Optional - How often to clear the file cache.
- **local_cache_dir** (Optional[Union[str, PathLike]]) - Optional - Path to directory to use as cache for downloaded files.
- **content_type_method** (Optional[Callable]) - Optional - Function to call to guess media type (mimetype) when writing a file to the cloud.
- **download_chunks_concurrently_kwargs** (Optional[Dict[str, Any]]) - Optional - Keyword arguments to pass to download_chunks_concurrently for sliced parallel downloads.
- **timeout** (Optional[float]) - Optional - Cloud Storage timeout value.
- **retry** (Optional[Retry]) - Optional - Cloud Storage retry configuration.
```

--------------------------------

### Instantiate AnyPath for local and cloud paths

Source: https://cloudpathlib.drivendata.org/stable/anypath-polymorphism

Demonstrates how AnyPath automatically detects and creates the appropriate path object for local files and S3 URIs.

```python
from cloudpathlib import AnyPath

path = AnyPath("mydir/myfile.txt")
path
#> PosixPath('mydir/myfile.txt')

cloud_path = AnyPath("s3://mybucket/myfile.txt")
cloud_path
#> S3Path('s3://mybucket/myfile.txt')

isinstance(path, AnyPath)
#> True
isinstance(cloud_path, AnyPath)
#> True
```

--------------------------------

### Get Name of CloudPath

Source: https://cloudpathlib.drivendata.org/stable/api-reference/cloudpath

Returns the final component (name) of the CloudPath. Dispatches to the underlying pathlib path.

```python
@property
def name(self) -> str:
        return self._dispatch_to_path("name")
```

--------------------------------

### Get Public URL for Azure Blob

Source: https://cloudpathlib.drivendata.org/stable/api-reference/azblobclient

Retrieves the public URL for a blob in Azure Blob Storage.

```python
def _get_public_url(self, cloud_path: AzureBlobPath) -> str:
        blob_client = self.service_client.get_blob_client(
            container=cloud_path.container, blob=cloud_path.blob
        )
        return blob_client.url
```

--------------------------------

### Create and interact with an Azure Blob Storage file

Source: https://cloudpathlib.drivendata.org/stable/why_cloudpathlib

Demonstrates creating an AzureBlobPath object, checking its name, existence, writing text, and verifying its existence.

```python
from cloudpathlib import AzureBlobPath

azp = AzureBlobPath("az://cloudpathlib-test-container/file.txt")
azp.name
```

```python
azp.exists()
```

```python
azp.write_text("I'm on Azure, boss.")
```

```python
azp.exists()
```

--------------------------------

### Get Default Client

Source: https://cloudpathlib.drivendata.org/stable/authentication

Provides the method to retrieve a reference to the currently set default client instance.

```python
S3Client.get_default_client()
#> <cloudpathlib.s3.s3client.S3Client at 0x7feac3d1fb90>
```

--------------------------------

### Initialize GSClient with various authentication options

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gsclient

Instantiate the GSClient with different authentication methods. Supports pre-authenticated StorageClient, OAuth2 credentials, service account file paths, or default SDK credentials. If no method is provided, the client acts anonymously, accessing only public buckets.

```python
from cloudpathlib import GSClient

# Using default credentials (e.g., from gcloud CLI)
client = GSClient()

# Using a service account file
client = GSClient(application_credentials="/path/to/your/keyfile.json")

# Using OAuth2 credentials and project ID
from google.oauth2.credentials import Credentials
credentials = Credentials(...) # Your credentials object
client = GSClient(credentials=credentials, project="your-gcp-project-id")

# Using an already instantiated StorageClient
from google.cloud import storage
storage_client = storage.Client()
client = GSClient(storage_client=storage_client)

# With caching and custom content type method
import mimetypes
client = GSClient(
    local_cache_dir="/path/to/cache",
    file_cache_mode="ALWAYS_INVALIDATE",
    content_type_method=mimetypes.guess_type
)
```

--------------------------------

### Get Public URL in CloudPathlib

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gsclient

Retrieves the publicly accessible URL for a given GSPath in Google Cloud Storage.

```python
    def _get_public_url(self, cloud_path: GSPath) -> str:
        bucket = self.client.get_bucket(cloud_path.bucket)
        blob = bucket.blob(cloud_path.blob)
        return blob.public_url
```

--------------------------------

### Initialize GSPath Client

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gsclient

Configures the storage client using explicit credentials, service account files, or default SDK authentication.

```python
# don't check `GOOGLE_APPLICATION_CREDENTIALS` since `google_default_auth` already does that
        # use explicit client
        if storage_client is not None:
            self.client = storage_client
        # use explicit credentials
        elif credentials is not None:
            self.client = StorageClient(credentials=credentials, project=project)
        # use explicit credential file
        elif application_credentials is not None:
            self.client = StorageClient.from_service_account_json(application_credentials)
        # use default credentials based on SDK precedence
        else:
            try:
                # use `google_default_auth` instead of `StorageClient()` since it
                # handles precedence of creds in different locations properly
                credentials, default_project = google_default_auth()
                project = project or default_project  # use explicit project if present
                self.client = StorageClient(credentials=credentials, project=project)
            except DefaultCredentialsError:
                self.client = StorageClient.create_anonymous_client()

        self.download_chunks_concurrently_kwargs = download_chunks_concurrently_kwargs
        self.blob_kwargs: dict[str, Any] = {}
        if timeout is not None:
            self.timeout: float = timeout
            self.blob_kwargs["timeout"] = self.timeout
        if retry is not None:
            self.retry: Retry = retry
            self.blob_kwargs["retry"] = self.retry

        super().__init__(
            local_cache_dir=local_cache_dir,
            content_type_method=content_type_method,
            file_cache_mode=file_cache_mode,
        )
```

--------------------------------

### POST /client/init

Source: https://cloudpathlib.drivendata.org/stable/api-reference/client

Initializes the CloudPathLib client with specific caching configurations and local directory settings.

```APIDOC
## POST /client/init

### Description
Initializes the client instance, setting up the file cache mode and local cache directory. If no directory is provided, a temporary directory is used.

### Parameters
#### Request Body
- **file_cache_mode** (string/FileCacheMode) - Optional - The caching strategy to use (e.g., 'persistent', 'tmp_dir').
- **local_cache_dir** (string/os.PathLike) - Optional - The local directory path to use for caching files.
- **content_type_method** (Callable) - Optional - A function to determine content types, defaults to mimetypes.guess_type.
```

--------------------------------

### Initialize S3Path object

Source: https://cloudpathlib.drivendata.org/stable/why_cloudpathlib

Instantiate an S3Path object to interact with files in an S3 bucket. This requires the cloudpathlib library to be installed.

```python
from cloudpathlib import S3Path

s3p = S3Path("s3://cloudpathlib-test-bucket/why_cloudpathlib/file.txt")
s3p.name
```

--------------------------------

### Basic HttpPath Usage

Source: https://cloudpathlib.drivendata.org/stable/http

Demonstrates standard file operations like reading, joining paths, and listing directory contents using HttpPath.

```python
from cloudpathlib import CloudPath

# Create a path object
path = CloudPath("https://example.com/data/file.txt")

# Read file contents
text = path.read_text()
binary = path.read_bytes()

# Get parent directory
parent = path.parent  # https://example.com/data/

# Join paths
subpath = path.parent / "other.txt"  # https://example.com/data/other.txt

# Check if file exists
if path.exists():
    print("File exists!")

# Get file name and suffix
print(path.name)      # "file.txt"
print(path.suffix)    # ".txt"

# List directory contents (if server supports directory listings)
data_dir = CloudPath("https://example.com/data/")
for child_path in data_dir.iterdir():
    print(child_path)
```

--------------------------------

### Get Relative Path from CloudPath

Source: https://cloudpathlib.drivendata.org/stable/api-reference/cloudpath

Calculates the relative path from another CloudPath. Raises ValueError if paths are not of the same type or anchor.

```python
def relative_to(self, other: Self, walk_up: bool = False) -> PurePosixPath:
        # We don't dispatch regularly since this never returns a cloud path (since it is relative, and cloud paths are
        # absolute)
        if not isinstance(other, CloudPath):
            raise ValueError(f"{self} is a cloud path, but {other} is not")
        if self.anchor != other.anchor:
            raise ValueError(
                f"{self} is a {self.anchor} path, but {other} is a {other.anchor} path"
            )

        kwargs = dict(walk_up=walk_up)

        if sys.version_info < (3, 12):
            kwargs.pop("walk_up")

        return self._path.relative_to(other._path, **kwargs)  # type: ignore[call-arg]
```

--------------------------------

### Iterate Directory Contents

Source: https://cloudpathlib.drivendata.org/stable/api-reference/cloudpath

Use `iterdir()` to get a generator of all files and directories within the current directory. It excludes the directory itself.

```python
for f, _ in self.client._list_dir(self, recursive=False):
            if f != self:  # iterdir does not include itself in pathlib
                yield f
```

--------------------------------

### CloudPath Initialization (`__init__`)

Source: https://cloudpathlib.drivendata.org/stable/api-reference/azblobpath

Details the constructor for the CloudPath class, explaining how to initialize a CloudPath object with various arguments.

```APIDOC
## `__init__`

### Description
Initializes a CloudPath object. It can be initialized with a string, another CloudPath object, or a combination of a base path and additional parts. An optional client can also be provided.

### Method
`__init__`

### Parameters
- **cloud_path** (Union[str, Self, CloudPath]) - The base path or an existing CloudPath object.
- **parts** (str) - Additional path components to append.
- **client** (Optional[Client]) - An optional client instance to use for cloud operations.
```

--------------------------------

### Explicitly Instantiate Client

Source: https://cloudpathlib.drivendata.org/stable/authentication

Shows how to explicitly instantiate a client object for authentication using specific credentials. This client can then be used to create cloud path instances.

```python
from cloudpathlib import S3Client

client = S3Client(aws_access_key_id="myaccesskey", aws_secret_access_key="mysecretkey")

# these next two commands are equivalent
# use client's factory method
cp1 = client.CloudPath("s3://cloudpathlib-test-bucket/")
# or pass client as keyword argument
cp2 = CloudPath("s3://cloudpathlib-test-bucket/", client=client)
```

--------------------------------

### Get File Statistics

Source: https://cloudpathlib.drivendata.org/stable/api-reference/azblobpath

Retrieves metadata for a cloud path and returns an os.stat_result object. Raises NoStatError if the resource is not found.

```python
def stat(self, follow_symlinks=True):
    try:
        meta = self.client._get_metadata(self)
    except ResourceNotFoundError:
        raise NoStatError(
            f"No stats available for {self}; it may be a directory or not exist."
        )

    return os.stat_result(
        (
            None,  # mode
            None,  # ino
            self.cloud_prefix,  # dev,
            None,  # nlink,
            None,  # uid,
            None,  # gid,
            meta.get("size", 0),  # size,
            None,  # atime,
            meta.get("last_modified", 0).timestamp(),  # mtime,
            None,  # ctime,
        )
    )
```

--------------------------------

### GET /client/default

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gsclient

Retrieves the default client instance used when instantiating cloud path instances without an explicit client.

```APIDOC
## GET /client/default

### Description
Get the default client, which is the one used when instantiating a cloud path instance for this cloud without a client specified.

### Method
GET

### Endpoint
/client/default

### Response
#### Success Response (200)
- **client** (Client) - The default client instance.
```

--------------------------------

### Instantiate AzureBlobClient with Credentials

Source: https://cloudpathlib.drivendata.org/stable/api-reference/azblobclient

Instantiate an AzureBlobClient using account name and key. This client is used for interacting with Azure Blob Storage.

```python
self.service_client = BlobServiceClient(
 account_url=self.data_lake_client.url.replace(".dfs.", ".blob.", 1),
 credential=credential,
)
```

--------------------------------

### Initialize Cloud Storage Client

Source: https://cloudpathlib.drivendata.org/stable/api-reference/client

Instantiate a client for cloud storage operations. Configure file caching behavior, local cache directory, and content type detection. Defaults to temporary directory caching if no local directory is specified.

```python
client = Client(
    file_cache_mode=FileCacheMode.persistent,
    local_cache_dir="/path/to/local/cache",
    content_type_method=mimetypes.guess_type,
)
```

--------------------------------

### stat() - Get file statistics

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gspath

Retrieves file statistics, including size. Returns `NoStatError` if stats are unavailable (e.g., for directories).

```APIDOC
## stat(follow_symlinks=True)

### Description
Retrieves file statistics for the given path. Returns an `os.stat_result` object containing information like file size. Raises `NoStatError` if statistics are not available, which can happen for directories or non-existent paths.

### Method
GET (or equivalent metadata retrieval operation)

### Endpoint
`/{path}`

### Parameters
#### Query Parameters
- **follow_symlinks** (bool) - Optional - Defaults to `True`. Currently not used in the provided implementation.

### Request Example
```python
stats = cloud_path.stat()
print(stats.st_size)
```

### Response
#### Success Response (200)
- **os.stat_result** - An object containing file statistics, including:
  - **st_size** (int) - The size of the file in bytes.
  - Other fields may be `None` if not available.

#### Response Example
```json
{
  "st_mode": null,
  "st_ino": null,
  "st_dev": "gs://your-bucket/your-path",
  "st_nlink": null,
  "st_uid": null,
  "st_gid": null,
  "st_size": 1024,
  "st_atime": null,
  "st_mtime": 1678886400.0,
  "st_ctime": null
}
```

#### Error Response
- `NoStatError`: If no stats are available for the path.
```

--------------------------------

### Initialize S3Client

Source: https://cloudpathlib.drivendata.org/stable/api-reference/s3client

The constructor accepts various parameters for authentication, session management, and caching behavior. If no credentials are provided, it defaults to anonymous access for public buckets.

```python
def __init__(
    self,
    aws_access_key_id: Optional[str] = None,
    aws_secret_access_key: Optional[str] = None,
    aws_session_token: Optional[str] = None,
    no_sign_request: Optional[bool] = False,
    botocore_session: Optional["botocore.session.Session"] = None,
    profile_name: Optional[str] = None,
    boto3_session: Optional["Session"] = None,
    file_cache_mode: Optional[Union[str, FileCacheMode]] = None,
    local_cache_dir: Optional[Union[str, os.PathLike]] = None,
    endpoint_url: Optional[str] = None,
    boto3_transfer_config: Optional["TransferConfig"] = None,
    content_type_method: Optional[Callable] = mimetypes.guess_type,
    extra_args: Optional[dict] = None,
):
```

--------------------------------

### Get Default CloudPathLib Client

Source: https://cloudpathlib.drivendata.org/stable/api-reference/client

Retrieves the default client instance for a cloud. This client is used automatically when no specific client is provided during path instantiation.

```python
@classmethod
def get_default_client(cls) -> "Client":
    """Get the default client, which the one that is used when instantiating a cloud path
    instance for this cloud without a client specified.
    """
    if cls._default_client is None:
        cls._default_client = cls()
    return cls._default_client
```

--------------------------------

### Handle Invalid Persistent Cache Configuration

Source: https://cloudpathlib.drivendata.org/stable/caching

Shows an example of triggering an InvalidConfigurationException when the persistent cache mode is set without providing a local cache directory.

```python
import os

# set the mode here so that it will be used when we instantiate the client
os.environ["CLOUDPATHLIB_FILE_CACHE_MODE"] = "persistent"

tmp_dir_client = S3Client()
```

```python
import os # set the mode here so that it will be used when we instantiate the client os.environ["CLOUDPATHLIB_FILE_CACHE_MODE"] = "persistent" tmp_dir_client = S3Client()
```

--------------------------------

### Instantiate Default Client

Source: https://cloudpathlib.drivendata.org/stable/authentication

Demonstrates how a default client object is automatically created when a cloud path instance is first created. Subsequent instances of the same service's cloud paths will reuse this client.

```python
from cloudpathlib import CloudPath

cloud_path = CloudPath("s3://cloudpathlib-test-bucket/")   # same for S3Path(...)
cloud_path.client
#> <cloudpathlib.s3.s3client.S3Client at 0x7feac3d1fb90>
```

--------------------------------

### Instantiate Cloud Storage Client

Source: https://cloudpathlib.drivendata.org/stable/api-reference/gsclient

Instantiates a Cloud Storage client using explicit client, credentials, credential file, or default authentication. Handles project and timeout configurations.

```python
    # don't check `GOOGLE_APPLICATION_CREDENTIALS` since `google_default_auth` already does that
    # use explicit client
    if storage_client is not None:
        self.client = storage_client
    # use explicit credentials
    elif credentials is not None:
        self.client = StorageClient(credentials=credentials, project=project)
    # use explicit credential file
    elif application_credentials is not None:
        self.client = StorageClient.from_service_account_json(application_credentials)
    # use default credentials based on SDK precedence
    else:
        try:
            # use `google_default_auth` instead of `StorageClient()` since it
            # handles precedence of creds in different locations properly
            credentials, default_project = google_default_auth()
            project = project or default_project  # use explicit project if present
            self.client = StorageClient(credentials=credentials, project=project)
        except DefaultCredentialsError:
            self.client = StorageClient.create_anonymous_client()

    self.download_chunks_concurrently_kwargs = download_chunks_concurrently_kwargs
    self.blob_kwargs: dict[str, Any] = {}
    if timeout is not None:
        self.timeout: float = timeout
        self.blob_kwargs["timeout"] = self.timeout
    if retry is not None:
        self.retry: Retry = retry
        self.blob_kwargs["retry"] = self.retry

    super().__init__(
        local_cache_dir=local_cache_dir,
        content_type_method=content_type_method,
        file_cache_mode=file_cache_mode,
    )
```

--------------------------------

### Implement a custom Client class

Source: https://cloudpathlib.drivendata.org/stable/contributing

Create a new client class by inheriting from the base Client class.

```python
from cloudpathlib.client import Client

class MyClient(Client):
    # implementation here...
```

--------------------------------

### Open Binary File for Writing

Source: https://cloudpathlib.drivendata.org/stable/api-reference/s3path

Opens a binary file for writing. Supports default buffering. If the file exists, it will be truncated.

```python
open(
    mode: OpenBinaryModeWriting,
    buffering: Literal[-1, 1] = -1,
    encoding: None = None,
    errors: None = None,
    newline: None = None,
    force_overwrite_from_cloud: Optional[bool] = None,
    force_overwrite_to_cloud: Optional[bool] = None,
) -> BufferedWriter
```

--------------------------------

### stat() - Get File Status

Source: https://cloudpathlib.drivendata.org/stable/api-reference/cloudpath

Retrieves status information for a file or directory. Note that this might incur network costs and may be overridden by clients for efficiency.

```APIDOC
## stat(follow_symlinks: bool = True) -> os.stat_result

### Description
Retrieves status information (like size, modification time, etc.) for the path. For many clients, this method may not be implemented as a direct API call and might require downloading the file to calculate stats, potentially incurring significant network costs. A warning is issued if this is the case.

### Method
`stat`

### Endpoint
N/A (Method on CloudPath object)

### Parameters
#### Query Parameters
- **follow_symlinks** (bool) - Optional (default: True) - Whether to follow symbolic links.

### Request Example
```python
status = cloud_path.stat()
```

### Response
#### Success Response (os.stat_result)
- Returns an `os.stat_result` object containing file status information.

#### Response Example
```json
{
  "st_mode": 33188,
  "st_ino": 12345,
  "st_dev": 67,
  "st_nlink": 1,
  "st_uid": 1000,
  "st_gid": 1000,
  "st_size": 1024,
  "st_atime": 1678886400.0,
  "st_mtime": 1678886400.0,
  "st_ctime": 1678886400.0
}
```
```

--------------------------------

### Initialize Azure BlobServiceClient or DataLakeServiceClient

Source: https://cloudpathlib.drivendata.org/stable/api-reference/azblobclient

Instantiate a client using an existing BlobServiceClient or DataLakeServiceClient. If only a BlobServiceClient is provided, a DataLakeServiceClient will be created from it.

```python
super().__init__(
        local_cache_dir=local_cache_dir,
        content_type_method=content_type_method,
        file_cache_mode=file_cache_mode,
    )

    if connection_string is None:
        connection_string = os.getenv("AZURE_STORAGE_CONNECTION_STRING", None)

    self.data_lake_client: Optional[DataLakeServiceClient] = (
        None  # only needs to end up being set if HNS is enabled
    )

    if blob_service_client is not None:
        self.service_client = blob_service_client

        # create from blob service client if not passed
        if data_lake_client is None:
            credential = (
                blob_service_client.credential
                if not isinstance(
                    blob_service_client.credential, BlobSharedKeyCredentialPolicy
                )
                else AzureNamedKeyCredential(
                    blob_service_client.credential.account_name,
                    blob_service_client.credential.account_key,
                )
            )

            self.data_lake_client = DataLakeServiceClient(
                account_url=self.service_client.url.replace(".blob.", ".dfs.", 1),
                credential=credential,
            )
        else:
            self.data_lake_client = data_lake_client

    elif data_lake_client is not None:
        self.data_lake_client = data_lake_client

        if blob_service_client is None:

            credential = (
                data_lake_client.credential
                if not isinstance(
                    data_lake_client.credential, DataLakeSharedKeyCredentialPolicy
                )
                else AzureNamedKeyCredential(

```

--------------------------------

### Get Default Client Instance

Source: https://cloudpathlib.drivendata.org/stable/api-reference/client

Retrieve the globally configured default client instance. This client is used by CloudPath objects when no specific client is provided during instantiation.

```python
default_client = Client.get_default_client()
```

--------------------------------

### Initialize S3 Client with Boto3 Session

Source: https://cloudpathlib.drivendata.org/stable/api-reference/s3client

Instantiates the S3 client using an existing boto3 session. This is useful when you have pre-configured boto3 settings.

```python
if boto3_session is not None:
    self.sess = boto3_session
else:
    self.sess = Session(
        aws_access_key_id=aws_access_key_id,
        aws_secret_access_key=aws_secret_access_key,
        aws_session_token=aws_session_token,
        botocore_session=botocore_session,
        profile_name=profile_name,
    )
```

--------------------------------

### Configure ipytest for autoconfiguration

Source: https://cloudpathlib.drivendata.org/stable/testing_mocked_cloudpathlib

This snippet configures ipytest for automatic test discovery and execution within an interactive environment. It's a prerequisite for running the subsequent pytest examples.

```python
import ipytest

ipytest.autoconfig()
```