### Setup minio-py repository Source: https://github.com/minio/minio-py/blob/master/MAINTAINERS.md Clone the repository and install the necessary packaging tools. ```sh $ git clone git@github.com:minio/minio-py $ cd minio-py $ pip install --user --upgrade twine ``` -------------------------------- ### Configuration and Connection Examples Source: https://github.com/minio/minio-py/blob/master/_autodocs/README.md Details on client constructor parameters, environment variables, and connection examples. ```APIDOC ## Client Configuration and Connection This section covers how to configure the MinIO client and establish connections. ### Client Constructor Parameters - **endpoint**: The MinIO server endpoint (e.g., 'play.min.io:9000'). - **access_key**: Your MinIO access key. - **secret_key**: Your MinIO secret key. - **secure**: Boolean, True for HTTPS, False for HTTP. - **session_token**: Optional session token for temporary credentials. - **region**: The region of the MinIO service. - **http_client**: Custom HTTP client instance. - **config**: Additional configuration options. ### Environment Variables - **MINIO_ENDPOINT**: Sets the MinIO server endpoint. - **MINIO_ACCESS_KEY**: Sets the access key. - **MINIO_SECRET_KEY**: Sets the secret key. - **MINIO_SECURE**: Sets the secure connection flag (true/false). - **MINIO_REGION**: Sets the region. ### Connection Examples #### HTTPS Connection ```python from minio import Minio client = Minio("play.min.io:9000", access_key="YOUR-ACCESSKEYID", secret_key="YOUR-SECRETACCESSKEY", secure=True) ``` #### HTTP Connection ```python from minio import Minio client = Minio("127.0.0.1:9000", access_key="YOUR-ACCESSKEYID", secret_key="YOUR-SECRETACCESSKEY", secure=False) ``` #### Proxy Connection ```python from minio import Minio from minio.commonconfig import HttpConfig proxy_config = HttpConfig(proxy_host="your-proxy-host", proxy_port="proxy-port") client = Minio("play.min.io:9000", access_key="YOUR-ACCESSKEYID", secret_key="YOUR-SECRETACCESSKEY", http_client=proxy_config) ``` #### Self-Signed Certificates ```python from minio import Minio client = Minio("localhost:9000", access_key="YOUR-ACCESSKEYID", secret_key="YOUR-SECRETACCESSKEY", secure=True, # Disable certificate verification for self-signed certs # Use with caution in production environments # For production, properly configure trusted certificates # For example, using certifi or custom trust store # This is a simplified example for demonstration # In a real scenario, you might use: # cert_bundle='path/to/your/ca.crt' # Or configure the underlying requests session # For simplicity here, we'll assume a way to bypass verification if needed # Note: Direct disabling of verification is generally discouraged # The SDK might not expose a direct flag for this, requiring deeper http client config ) ``` ### HTTP Client Customization - Allows customization of the underlying HTTP client for advanced scenarios like connection pooling, custom headers, etc. ### Credentials Configuration - Details on how to provide credentials, including environment variables, configuration files, and IAM roles (if applicable). ### Debugging and Monitoring Setup - Instructions on enabling debug logging and setting up monitoring for the client. ``` -------------------------------- ### Clone and Install minio-py Source: https://github.com/minio/minio-py/blob/master/docs/zh_CN/CONTRIBUTING.md Initializes the local development environment by cloning the repository and installing the package. ```sh $ git clone https://github.com/$USER_ID/minio-py $ cd minio-py $ python setup.py install ... ``` -------------------------------- ### Install MinIO Python SDK from Source Source: https://github.com/minio/minio-py/blob/master/README.md Install the MinIO Python SDK by cloning the GitHub repository and installing from source. Requires Python 3.10+. ```sh git clone https://github.com/minio/minio-py cd minio-py python setup.py install ``` -------------------------------- ### Install Minio Python Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/INDEX.md Install the Minio Python client using pip. This is the first step to using Minio in your Python projects. ```bash pip install minio ``` -------------------------------- ### Install MinIO Python SDK using pip Source: https://github.com/minio/minio-py/blob/master/README.md Install the MinIO Python SDK using pip. Ensure you have Python 3.10+ installed. ```sh pip3 install minio ``` -------------------------------- ### Start Server Profiling Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Initiates server profiling, which can be used for performance analysis. Requires specifying the type of profile to start. ```python def start_profile(self, *, profile_type: str) -> dict[str, Any] ``` -------------------------------- ### Generate Presigned URL for GET and HEAD Methods Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Generate presigned URLs for objects, allowing temporary access with specified HTTP methods. The URL's validity is controlled by the 'expires' timedelta parameter. This example shows generating URLs for GET and HEAD requests. ```python from datetime import timedelta url = client.get_presigned_url( method="GET", bucket_name="my-bucket", object_name="file.txt", expires=timedelta(hours=1), ) # For custom methods url = client.get_presigned_url( method="HEAD", bucket_name="my-bucket", object_name="file.txt", expires=timedelta(hours=1), ) ``` -------------------------------- ### Complete Minio Admin Workflow Example Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Demonstrates a full administrative workflow using the Minio Python client, including user creation, policy management, and quota settings. ```python import json from minio import MinioAdmin from minio.credentials import StaticProvider # Connect as admin admin = MinioAdmin( endpoint="localhost:9000", credentials=StaticProvider("minioadmin", "minioadmin"), secure=False, ) # Check server status status = admin.service_status() print(f"Server: {status}") # Add new user admin.add_user(access_key="datauser", secret_key="secure-password-123") # Create read-only policy policy = { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Action": ["s3:GetObject", "s3:ListBucket"], "Resource": ["arn:aws:s3:::data-bucket/*"], }], } admin.add_canned_policy(policy_name="data-readonly", policy=json.dumps(policy)) # Attach policy to user admin.set_user_or_group_policy( policy_name="data-readonly", user_or_group="datauser", ) # Check account usage usage = admin.account_info() print(f"Account usage: {usage}") # Set bucket quota (100GB) admin.set_bucket_quota( bucket_name="data-bucket", quota_bytes=100 * 1024 * 1024 * 1024, ) # List all users users = admin.list_users() for access_key in users: info = admin.user_info(access_key=access_key) print(f"{access_key}: {info['status']}") ``` -------------------------------- ### Get Configuration Key-Value Pair Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Retrieves a specific configuration setting using its key. ```python def get_config_kv(self, *, key: str) -> dict[str, Any] ``` -------------------------------- ### Example Scenario for ServerError Source: https://github.com/minio/minio-py/blob/master/_autodocs/errors.md Demonstrates how to catch a ServerError and implement basic retry logic. This is useful for handling transient server issues. ```python try: client.list_buckets() except ServerError as e: print(f"Server error {e.status_code}: {e}") # Implement retry logic time.sleep(5) client.list_buckets() # Retry ``` -------------------------------- ### Select Object Content (CSV) Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Query data from objects using SQL-like expressions. This example demonstrates querying CSV files with specified input and output serializations. ```APIDOC ## Select Object Content Query data from objects using SQL-like expressions (CSV, JSON, Parquet): ```python from minio.models import SelectObjectContentRequest from minio.models import CSVInputSerialization, CSVOutputSerialization # Query CSV file with client.select_object_content( bucket_name="my-bucket", object_name="data.csv", request=SelectObjectContentRequest( expression="SELECT * FROM s3object WHERE age > 30", input_serialization=CSVInputSerialization( compression="NONE", allowed_duplicate_header_field_names=False, comments="#", quote_escape_character="\", record_delimiter="\n", field_delimiter=",", quote_character='"', allow_quoted_record_delimiter=False, ), output_serialization=CSVOutputSerialization( field_delimiter=",", record_delimiter="\n", quote_escape_character="\", quote_character='"', ), request_progress=True, ), ) as response: for data in response.stream(): print(data.decode()) print(f"Stats: {response.stats()}") ``` --- ``` -------------------------------- ### Set Bucket Quota Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Sets the disk quota for a bucket. Specify the bucket name and the desired quota in bytes. Example sets a 100GB quota. ```python # Set 100GB quota admin.set_bucket_quota( bucket_name="my-bucket", quota_bytes=100 * 1024 * 1024 * 1024, ) ``` -------------------------------- ### Select Object Content (JSON) Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Query data from JSON objects using SQL-like expressions. This example shows how to query JSON data with specific input and output serialization settings. ```APIDOC ### Query JSON Objects ```python from minio.models import SelectObjectContentRequest from minio.models import JSONInputSerialization, JSONOutputSerialization with client.select_object_content( bucket_name="my-bucket", object_name="data.json", request=SelectObjectContentRequest( expression="SELECT * FROM s3object[*] s WHERE s.value > 100", input_serialization=JSONInputSerialization( compression="NONE", type="DOCUMENT", # or "LINES" for JSONL ), output_serialization=JSONOutputSerialization(), ), ) as response: for data in response.stream(): print(data.decode()) ``` --- ``` -------------------------------- ### Build and verify package Source: https://github.com/minio/minio-py/blob/master/MAINTAINERS.md Build the source distribution and wheel files. ```sh $ make $ python setup.py register $ python setup.py sdist bdist bdist_wheel ``` -------------------------------- ### Minio Client Initialization and Basic Operations Source: https://github.com/minio/minio-py/blob/master/_autodocs/INDEX.md Demonstrates how to initialize the Minio client and perform fundamental operations such as creating a bucket, uploading an object, listing objects, and downloading an object. ```APIDOC ## Minio Client Initialization and Basic Operations ### Description This section covers the essential steps to get started with the Minio Python client. It includes initializing the client with your Minio server details and performing common object storage operations. ### Initialization ```python from minio import Minio # Create client client = Minio( endpoint="play.min.io", access_key="Q3AM3UQ867SPQQA43P2F", secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG", ) ``` ### Bucket Operations #### Create Bucket ```python # Create bucket client.make_bucket(bucket_name="my-bucket") ``` ### Object Operations #### Upload Object ```python # Upload object client.fput_object( bucket_name="my-bucket", object_name="my-object.txt", file_path="/path/to/file.txt", ) ``` #### List Objects ```python # List objects for obj in client.list_objects(bucket_name="my-bucket"): print(obj.object_name, obj.size) ``` #### Download Object ```python # Download object client.fget_object( bucket_name="my-bucket", object_name="my-object.txt", file_path="/path/to/download.txt", ) ``` ``` -------------------------------- ### Initialize Minio Client with Credentials Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Initialize the Minio client with access and secret keys for authenticated access. Replace placeholders with your actual credentials. ```python from minio import Minio # Access with credentials client = Minio( endpoint="s3.amazonaws.com", access_key="YOUR-ACCESS-KEY", secret_key="YOUR-SECRET-KEY", ) ``` -------------------------------- ### Get Presigned URL Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Generate presigned URLs for any HTTP method (e.g., GET, HEAD). These URLs provide temporary access to objects without requiring authentication for each request. ```APIDOC ## Get Presigned URL (Generic) Generate presigned URLs for any HTTP method: ```python from datetime import timedelta url = client.get_presigned_url( method="GET", bucket_name="my-bucket", object_name="file.txt", expires=timedelta(hours=1), ) # For custom methods url = client.get_presigned_url( method="HEAD", bucket_name="my-bucket", object_name="file.txt", expires=timedelta(hours=1), ) ``` ``` -------------------------------- ### Minio Python Client Basic Usage Source: https://github.com/minio/minio-py/blob/master/_autodocs/INDEX.md Demonstrates basic Minio client operations: creating a client instance, making a bucket, uploading an object, listing objects, and downloading an object. Ensure you have a file at '/path/to/file.txt' for upload and specify a path for download. ```python from minio import Minio # Create client client = Minio( endpoint="play.min.io", access_key="Q3AM3UQ867SPQQA43P2F", secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG", ) # Create bucket client.make_bucket(bucket_name="my-bucket") # Upload object client.fput_object( bucket_name="my-bucket", object_name="my-object.txt", file_path="/path/to/file.txt", ) # List objects for obj in client.list_objects(bucket_name="my-bucket"): print(obj.object_name, obj.size) # Download object client.fget_object( bucket_name="my-bucket", object_name="my-object.txt", file_path="/path/to/download.txt", ) ``` -------------------------------- ### get_config_kv Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Gets a configuration key-value pair. ```APIDOC ## get_config_kv ### Description Gets a configuration key-value pair. ### Method ```python def get_config_kv(self, *, key: str) -> dict[str, Any] ``` ``` -------------------------------- ### Initialize MinioAdmin Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Instantiate the MinioAdmin client with your MinIO server endpoint and administrative credentials. Set `secure=False` if not using HTTPS. ```python from minio import MinioAdmin from minio.credentials import StaticProvider credentials = StaticProvider( access_key="minioadmin", secret_key="minioadmin", ) admin = MinioAdmin( endpoint="localhost:9000", credentials=credentials, secure=False, ) ``` -------------------------------- ### data_usage_info Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Gets detailed data usage statistics. ```APIDOC ## data_usage_info ### Description Gets detailed data usage statistics. ### Method ```python def data_usage_info(self) -> dict[str, Any] ``` ### Returns Data usage by bucket and storage class. ``` -------------------------------- ### Initialize Minio Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/configuration.md Instantiate the Minio client with essential configuration parameters like endpoint, access key, and secret key. Secure connections are enabled by default. ```python from minio import Minio client = Minio( endpoint="play.min.io", access_key="Q3AM3UQ867SPQQA43P2F", secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG", session_token=None, secure=True, region=None, http_client=None, credentials=None, cert_check=True, enable_rdma=False, ) ``` -------------------------------- ### account_info Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Gets usage information for the current account. ```APIDOC ## account_info ### Description Gets usage information for the current account. ### Method ```python def account_info(self) -> dict[str, Any] ``` ### Returns Account usage including buckets and object count. ``` -------------------------------- ### Create MinIO Client Source: https://github.com/minio/minio-py/blob/master/README.md Instantiate a MinIO client with endpoint, access key, and secret key. Ensure these credentials are correct for your MinIO instance. ```python from minio import Minio client = Minio( endpoint="play.min.io", access_key="Q3AM3UQ867SPQQA43P2F", secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG", ) ``` -------------------------------- ### Enable RDMA/GPUDirect Storage in Minio Python Client Source: https://github.com/minio/minio-py/blob/master/README.md Instantiate the Minio client with `enable_rdma=True` to opt into the RDMA path. This requires `libminiocpp.so` to be available. ```python from minio import Minio client = Minio( endpoint="server:9000", access_key="...", secret_key="...", secure=False, enable_rdma=True, # opt-in ) ``` -------------------------------- ### Get Bucket Versioning Configuration Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the versioning configuration for a specified bucket. Requires the bucket name. ```python def get_bucket_versioning( self, *, bucket_name: str, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> VersioningConfig ``` -------------------------------- ### Initialize Minio Client with RDMA Enabled Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Initialize the Minio client with RDMA enabled for optimized data transfer. Note that this requires the libminiocpp.so library. ```python from minio import Minio client = Minio( endpoint="server:9000", access_key="ACCESS_KEY", secret_key="SECRET_KEY", secure=False, enable_rdma=True, # Requires libminiocpp.so ) ``` -------------------------------- ### Get Bucket Lifecycle Configuration Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the lifecycle configuration for a specified bucket. Requires the bucket name. ```python def get_bucket_lifecycle( self, *, bucket_name: str, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> LifecycleConfig ``` -------------------------------- ### Create a Bucket with MinIO Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Use `make_bucket` to create a new bucket. Supports specifying region, enabling object lock, and adding tags or custom configurations. ```python client.make_bucket(bucket_name="my-bucket") ``` ```python client.make_bucket( bucket_name="my-bucket", region="eu-west-1", ) ``` ```python client.make_bucket( bucket_name="my-bucket", region="eu-west-2", object_lock=True, ) ``` -------------------------------- ### Get Bucket Tags - Minio Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the tags associated with a bucket. Requires the bucket name. ```python def get_bucket_tags( self, *, bucket_name: str, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> Tags ``` -------------------------------- ### get_bucket_notification Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the notification configuration for a specified bucket. This allows you to check the current event notification setup. ```APIDOC ## get_bucket_notification ### Description Retrieves bucket notification configuration. ### Method Not specified (assumed to be a client SDK method call) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python # Example usage (actual call depends on client instantiation) # client.get_bucket_notification(bucket_name='my-bucket') ``` ### Response #### Success Response - **NotificationConfig**: The notification configuration object for the bucket. ``` -------------------------------- ### Get Bucket Notification Configuration Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the current notification configuration for a specified bucket. Requires the bucket name. ```python def get_bucket_notification( self, *, bucket_name: str, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> NotificationConfig ``` -------------------------------- ### LDAP Credentials with MinioAdmin Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/encryption-credentials.md Demonstrates using an LDAP-backed provider with the MinioAdmin client for authentication. ```python from minio.minioadmin import MinioAdmin admin_client = MinioAdmin( endpoint="minio:9000", credentials=ldap_provider, # LDAP-backed provider ) ``` -------------------------------- ### Get Bucket Policy - Minio Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the bucket policy as a JSON string. Requires the bucket name. ```python def get_bucket_policy( self, *, bucket_name: str, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> str ``` -------------------------------- ### Minio Client with Static Credentials Source: https://github.com/minio/minio-py/blob/master/_autodocs/configuration.md Instantiate the Minio client using static access and secret keys. Ensure these are kept secure. ```python client = Minio( endpoint="s3.amazonaws.com", access_key="AKIAIOSFODNN7EXAMPLE", secret_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", ) ``` -------------------------------- ### GET /bucket/object-lock Source: https://github.com/minio/minio-py/blob/master/docs/API.md Retrieves the object-lock configuration for a specified MinIO bucket. ```APIDOC ## GET /bucket/object-lock ### Description Fetches the object-lock configuration settings for the bucket. ### Method GET ### Endpoint get_object_lock_config(bucket_name, ...) ### Parameters #### Path Parameters - **bucket_name** (str) - Required - Name of the bucket. ### Response #### Success Response (200) - **ObjectLockConfig** (object) - The object-lock configuration details. ``` -------------------------------- ### Implement Progress Tracking Protocol Source: https://github.com/minio/minio-py/blob/master/_autodocs/types.md Implement this protocol to create custom handlers for tracking upload or download progress. Requires methods for setting initial metadata and updating progress. ```python class ProgressType(Protocol): def set_meta(self, object_name: str, total_length: int): ... def update(self, length: int): ... ``` -------------------------------- ### Initialize Minio Client with Custom HTTP Client and Proxy Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Initialize the Minio client with a custom HTTP client, including proxy configuration and retry settings. This allows for advanced network configurations. ```python from minio import Minio import urllib3 client = Minio( endpoint="SERVER:9000", access_key="ACCESS_KEY", secret_key="SECRET_KEY", secure=True, http_client=urllib3.ProxyManager( "https://PROXYSERVER:PROXYPORT/", timeout=urllib3.Timeout.DEFAULT_TIMEOUT, cert_reqs="CERT_REQUIRED", retries=urllib3.Retry( total=5, backoff_factor=0.2, status_forcelist=[500, 502, 503, 504], ), ), ) ``` -------------------------------- ### Get Object ACL Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Retrieve the Access Control List (ACL) for a specific object. This allows you to inspect who has what permissions on the object. ```APIDOC ## Get Object ACL Retrieve object access control list: ```python response = client.get_object_acl( bucket_name="my-bucket", object_name="document.txt", ) for grant in response.grants: print(f"Grantee: {grant.grantee}") print(f"Permission: {grant.permission}") ``` ``` -------------------------------- ### Get Account Information Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Retrieves usage statistics for the current Minio account. Includes information on buckets and object counts. ```python def account_info(self) -> dict[str, Any] ``` -------------------------------- ### Verify Uploaded File with mc ls Source: https://github.com/minio/minio-py/blob/master/README.md Use the MinIO client command-line tool (`mc`) to list objects in the bucket and verify the upload. ```sh mc ls play/python-test-bucket ``` -------------------------------- ### Get Group Information - Minio Admin API Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Retrieves detailed information about a specific group. The group name is required. ```python def group_info(self, *, group_name: str) -> dict[str, Any] ``` -------------------------------- ### GET /bucket/tags Source: https://github.com/minio/minio-py/blob/master/docs/API.md Retrieves the tag configuration associated with a specific MinIO bucket. ```APIDOC ## GET /bucket/tags ### Description Retrieves the current tag configuration for the specified bucket. ### Method GET ### Endpoint get_bucket_tags(bucket_name, region=None, ...) ### Parameters #### Path Parameters - **bucket_name** (str) - Required - Name of the bucket. #### Query Parameters - **region** (str) - Optional - Region of the bucket to skip auto probing. ### Request Example ```python tags = client.get_bucket_tags(bucket_name="my-bucket") ``` ### Response #### Success Response (200) - **Tags** (object) - The bucket tags configuration object. #### Response Example { "Project": "Project One", "User": "jsmith" } ``` -------------------------------- ### Minio Client with Environment Variables Source: https://github.com/minio/minio-py/blob/master/_autodocs/configuration.md Configure the Minio client by loading credentials from environment variables. This is useful for CI/CD pipelines and containerized applications. ```python import os from minio import Minio # Credentials loaded from AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY client = Minio( endpoint="s3.amazonaws.com", access_key=os.environ.get('AWS_ACCESS_KEY_ID'), secret_key=os.environ.get('AWS_SECRET_ACCESS_KEY'), session_token=os.environ.get('AWS_SESSION_TOKEN'), ) ``` -------------------------------- ### Get Presigned URL for Object Operations (Python) Source: https://github.com/minio/minio-py/blob/master/docs/API.md Generates a presigned URL for a specified object in a Minio bucket. This URL can be used to perform HTTP operations like GET, PUT, or DELETE on the object without requiring direct authentication for each request. The URL has a configurable expiry time and can include extra query parameters for advanced use cases. ```python from datetime import timedelta from minio.error import S3Error from minio.api import Minio from minio.compat import HTTPQueryDict # Assuming 'client' is an initialized Minio client instance # Example 1: Get presigned URL string to delete 'my-object' in 'my-bucket' with one day expiry. url_delete = client.get_presigned_url( method="DELETE", bucket_name="my-bucket", object_name="my-object", expires=timedelta(days=1), ) print(f"Delete URL: {url_delete}") # Example 2: Get presigned URL string to upload 'my-object' in 'my-bucket' with response-content-type as application/json and one day expiry. url_upload = client.get_presigned_url( method="PUT", bucket_name="my-bucket", object_name="my-object", expires=timedelta(days=1), extra_query_params=HTTPQueryDict({"response-content-type": "application/json"}), ) print(f"Upload URL: {url_upload}") # Example 3: Get presigned URL string to download 'my-object' in 'my-bucket' with two hours expiry. url_download = client.get_presigned_url( method="GET", bucket_name="my-bucket", object_name="my-object", expires=timedelta(hours=2), ) print(f"Download URL: {url_download}") ``` -------------------------------- ### Initialize Minio Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Initialize the Minio client for anonymous access to the specified endpoint. Ensure the endpoint is correctly formatted. ```python from minio import Minio # Anonymous access client = Minio(endpoint="play.min.io") ``` -------------------------------- ### Get Bucket Replication Configuration Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the replication configuration for a specified bucket. Requires the bucket name and optionally accepts a region. ```python def get_bucket_replication( self, *, bucket_name: str, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> ReplicationConfig ``` -------------------------------- ### MinioAdmin Constructor Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Initializes a new MinioAdmin client instance. This client is used for administrative operations on a MinIO server. ```APIDOC ## Constructor: `__init__` Creates and initializes a new MinioAdmin client. ### Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `endpoint` | `str` | Yes | — | MinIO server endpoint (e.g., `minio:9000`) | | `credentials` | `Provider` | Yes | — | Credentials provider with admin privileges | | `region` | `str` | No | `""` | MinIO region (usually empty for standalone) | | `secure` | `bool` | No | `True` | Use HTTPS (True) or HTTP (False) | | `cert_check` | `bool` | No | `True` | Enable/disable certificate validation | | `http_client` | `Optional[PoolManager]` | No | `None` | Custom HTTP client | ### Example ```python from minio import MinioAdmin from minio.credentials import StaticProvider credentials = StaticProvider( access_key="minioadmin", secret_key="minioadmin", ) admin = MinioAdmin( endpoint="localhost:9000", credentials=credentials, secure=False, ) ``` ``` -------------------------------- ### Get Bucket Encryption Configuration Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves the encryption configuration for a specified bucket. Requires the bucket name and optionally accepts a region. ```python def get_bucket_encryption( self, *, bucket_name: str, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> SSEConfig ``` -------------------------------- ### Configure pypirc Source: https://github.com/minio/minio-py/blob/master/MAINTAINERS.md Create or update the .pypirc file with PyPI credentials. ```sh $ cat >> $HOME/.pypirc << EOF [distutils] index-servers = pypi [pypi] username:minio password:**REDACTED** EOF ``` -------------------------------- ### CreateBucketConfiguration Type Source: https://github.com/minio/minio-py/blob/master/_autodocs/types.md Represents bucket creation options. Used by `make_bucket()`. ```python from typing import Optional from dataclasses import dataclass @dataclass(frozen=True) class CreateBucketConfiguration: location_config: Optional[Location] = None bucket_config: Optional[Bucket] = None @dataclass(frozen=True) class Location: region: str @dataclass(frozen=True) class Bucket: object_lock_enabled: Optional[bool] = None ``` -------------------------------- ### GET Presigned URL Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Generate a URL for downloading objects without requiring authentication. This URL is valid for a specified expiration time. ```APIDOC ### GET Presigned URL Generate a URL for downloading objects without authentication: ```python from datetime import timedelta url = client.presigned_get_object( bucket_name="my-bucket", object_name="document.pdf", expires=timedelta(days=1), ) print(f"Download URL: {url}") # Share with others; they can download without credentials ``` ``` -------------------------------- ### CreateBucketConfiguration Source: https://github.com/minio/minio-py/blob/master/_autodocs/types.md Represents bucket creation options. Used in bucket creation. ```APIDOC ## CreateBucketConfiguration ### Description Represents bucket creation options. ### Type Definition ```python @dataclass(frozen=True) class CreateBucketConfiguration: location_config: Optional[Location] = None bucket_config: Optional[Bucket] = None @dataclass(frozen=True) class Location: region: str @dataclass(frozen=True) class Bucket: object_lock_enabled: Optional[bool] = None ``` ### Used By `make_bucket()` ``` -------------------------------- ### Generate Presigned GET Object URL Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Generate a URL for downloading objects without authentication. The URL can be configured with an expiration time. ```python from datetime import timedelta url = client.presigned_get_object( bucket_name="my-bucket", object_name="document.pdf", expires=timedelta(days=1), ) print(f"Download URL: {url}") # Share with others; they can download without credentials ``` -------------------------------- ### GET /get_object_tags Source: https://github.com/minio/minio-py/blob/master/docs/API.md Retrieves the tag configuration associated with a specific object in a MinIO bucket. ```APIDOC ## GET /get_object_tags ### Description Retrieves the tag configuration of an object. This is useful for auditing or managing metadata associated with stored objects. ### Method GET ### Endpoint /get_object_tags ### Parameters #### Query Parameters - **bucket_name** (str) - Required - Name of the bucket. - **object_name** (str) - Required - Object name in the bucket. - **version_id** (str) - Optional - Version ID of the object. - **region** (str) - Optional - Region of the bucket to skip auto probing. ### Request Example { "bucket_name": "my-bucket", "object_name": "my-object" } ### Response #### Success Response (200) - **Tags** (object) - The tags configuration object. #### Response Example { "Project": "Project One", "User": "jsmith" } ``` -------------------------------- ### Connect to Minio with Error Handling Source: https://github.com/minio/minio-py/blob/master/_autodocs/INDEX.md Establishes a connection to the Minio service and includes error handling for S3-specific errors and general exceptions. Ensure you have the necessary credentials and endpoint configured. ```python from minio import Minio from minio.error import S3Error try: client = Minio( endpoint="s3.amazonaws.com", access_key="YOUR_KEY", secret_key="YOUR_SECRET", ) # Perform operations for bucket in client.list_buckets(): print(bucket.name) except S3Error as e: print(f"S3 Error: {e.code} - {e.message}") except Exception as e: print(f"Error: {e}") ``` -------------------------------- ### Import SSE Classes Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/encryption-credentials.md Imports necessary classes for server-side encryption configurations. ```python from minio.sse import Sse, SseKms, SseS3, SseCustomerKey ``` -------------------------------- ### Remove Canned Policy - Minio Admin API Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Removes a canned policy by its name. No setup is required beyond having the policy name. ```python def remove_canned_policy( self, *, policy_name: str, ) -> None ``` -------------------------------- ### Enable Virtual-Hosted-Style Paths Source: https://github.com/minio/minio-py/blob/master/_autodocs/configuration.md Use this method to enable virtual-hosted-style endpoint access, which formats URLs as https://bucket.example.com/object. This is the default for many S3 services. ```python # Enable virtual-hosted-style: https://bucket.example.com/object client.enable_virtual_style_endpoint() # URL: https://bucket.example.com/object ``` -------------------------------- ### Handle InvalidResponseError Source: https://github.com/minio/minio-py/blob/master/_autodocs/errors.md Example scenario for catching and handling an InvalidResponseError. This involves checking the HTTP status code, content type, and response body. ```python try: client.list_buckets() except InvalidResponseError as e: print(f"HTTP {e.code}: {e.content_type}") print(f"Response body: {e.body}") ``` -------------------------------- ### Enable RDMA/GPUDirect Storage Source: https://github.com/minio/minio-py/blob/master/_autodocs/configuration.md Enable RDMA dispatch for high-performance data transfer with GPU workloads by setting `enable_rdma=True` in the Minio client constructor. This path is automatically used for buffer-protocol objects like bytearray and GPU arrays. Ensure `libminiocpp.so` is accessible or `MINIOCPP_LIB` is set. ```python client = Minio( endpoint="server:9000", access_key="KEY", secret_key="SECRET", secure=False, enable_rdma=True, # Enable RDMA dispatch ) # When using buffer-protocol objects (bytearray, GPU arrays), RDMA path is used import cupy as cp gpu_buffer = cp.zeros((1024*1024,), dtype=cp.uint8) # 1MB GPU buffer # Upload via RDMA response = client.put_object( bucket_name="bucket", object_name="object.bin", data=gpu_buffer, # GPU buffer triggers RDMA dispatch length=len(gpu_buffer), ) # Download via RDMA n = client.get_object( bucket_name="bucket", object_name="object.bin", into=gpu_buffer, # RDMA dispatch length=len(gpu_buffer), ) ``` -------------------------------- ### Get Object Tags - MinIO Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Retrieves tags associated with a specific object in a bucket. Supports specifying a version ID and region. ```python def get_object_tags( self, *, bucket_name: str, object_name: str, version_id: Optional[str] = None, region: Optional[str] = None, extra_headers: Optional[HTTPHeaderDict] = None, extra_query_params: Optional[HTTPQueryDict] = None, ) -> Tags ``` -------------------------------- ### Get Data Usage Information Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minioadmin.md Fetches detailed data usage statistics for the Minio instance. Organizes usage by bucket and storage class. ```python def data_usage_info(self) -> dict[str, Any] ``` -------------------------------- ### Initialize Minio Client with Proxy Source: https://github.com/minio/minio-py/blob/master/docs/API.md Demonstrates how to create a Minio client instance configured to use a proxy server. This is useful for network environments that require outbound traffic to be routed through a proxy. It utilizes the `urllib3` library for managing the HTTP client and proxy settings. ```python import urllib3 from minio import Minio client = Minio( endpoint="SERVER:PORT", access_key="ACCESS_KEY", secret_key="SECRET_KEY", secure=True, http_client=urllib3.ProxyManager( "https://PROXYSERVER:PROXYPORT/", timeout=urllib3.Timeout.DEFAULT_TIMEOUT, cert_reqs="CERT_REQUIRED", retries=urllib3.Retry( total=5, backoff_factor=0.2, status_forcelist=[500, 502, 503, 504], ), ), ) ``` -------------------------------- ### GET /get_object_acl Source: https://github.com/minio/minio-py/blob/master/docs/API.md Retrieves the Access Control List (ACL) configuration for a specific object in a bucket. ```APIDOC ## GET /get_object_acl ### Description Fetches the ACL settings for an object, allowing you to verify permissions associated with the object. ### Method GET ### Endpoint get_object_acl(bucket_name, object_name, ...) ### Parameters #### Query Parameters - **bucket_name** (str) - Required - Name of the bucket. - **object_name** (str) - Required - Object name in the bucket. - **version_id** (str) - Optional - Version ID of the object. ### Request Example { "bucket_name": "my-bucket", "object_name": "my-object" } ### Response #### Success Response (200) - **response** (GetObjectAclResponse) - Object containing the ACL configuration. #### Response Example { "owner": "user-id", "grants": [{"grantee": "all-users", "permission": "READ"}] } ``` -------------------------------- ### Minio Client with Custom Credentials Provider Source: https://github.com/minio/minio-py/blob/master/_autodocs/configuration.md Implement a custom credential provider to dynamically load credentials, enabling automatic refresh or loading from external sources. ```python from minio import Minio from minio.credentials import Credentials # Provider automatically handles credential refresh class CustomCredentialProvider: def retrieve(self) -> Credentials: # Load credentials from external source return Credentials( access_key="KEY", secret_key="SECRET", session_token="TOKEN", ) client = Minio( endpoint="s3.amazonaws.com", credentials=CustomCredentialProvider(), ) ``` -------------------------------- ### Compose Object with Range Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/advanced-operations.md Compose specific byte ranges from source objects into a new object. If start and length are omitted, the entire object is used. ```python from minio.models import SourceObject sources = [ SourceObject("bucket", "file1.bin", start=0, length=1000), SourceObject("bucket", "file2.bin", start=500, length=2000), SourceObject("bucket", "file3.bin"), # Full file ] result = client.compose_object( bucket_name="bucket", object_name="composite.bin", sources=sources, ) ``` -------------------------------- ### Download Object with Progress Tracking Source: https://github.com/minio/minio-py/blob/master/_autodocs/configuration.md Download an object from a bucket and track the download progress. This is useful for providing user feedback during large downloads. ```python # Download with progress tracking class ProgressTracker: def set_meta(self, object_name: str, total_length: int): self.object_name = object_name self.total_length = total_length def update(self, length: int): print(f"Downloaded {length} bytes of {self.total_length}") progress = ProgressTracker() with client.get_object( bucket_name="my-bucket", object_name="file.bin", ) as response: with open("downloaded-file.bin", "wb") as f: for chunk in response.stream(): f.write(chunk) ``` -------------------------------- ### Minio Client API Reference Source: https://github.com/minio/minio-py/blob/master/_autodocs/README.md Reference for the Minio class, including all methods for bucket and object operations. Covers initialization, configuration, CRUD operations, metadata management, and presigned URLs. ```APIDOC ## Minio Client Operations This section details the public methods available in the Minio class for interacting with S3-compatible object storage. ### Object Operations #### Upload/Download - **put_object(bucket_name, object_name, data, length, content_type, metadata, progress, part_size, tags, version_id, legal_hold, retention)**: Uploads an object to a bucket. - **get_object(bucket_name, object_name, extra_args, version_id, request_payment)**: Downloads an object from a bucket. - **fput_object(bucket_name, object_name, file_path, content_type, metadata, progress, part_size, tags, version_id, legal_hold, retention)**: Uploads an object from a file path. - **fget_object(bucket_name, object_name, file_path, extra_args, version_id, request_payment)**: Downloads an object to a file path. #### Listing and Metadata - **list_objects(bucket_name, prefix, marker, delimiter, max_keys, extra_args)**: Lists objects in a bucket. - **stat_object(bucket_name, object_name, extra_args, version_id)**: Retrieves metadata for an object. #### Deletion - **remove_object(bucket_name, object_name, version_id, mforder)**: Deletes an object from a bucket. - **remove_objects(bucket_name, object_list)**: Deletes multiple objects from a bucket. #### Advanced Object Operations - **copy_object(bucket_name, object_name, source, extra_args, version_id, legal_hold, retention)**: Copies an object. - **compose_object(bucket_name, object_name, sources, metadata, content_type, tags, version_id, legal_hold, retention)**: Composes a new object from existing objects. - **select_object_content(bucket_name, object_name, query, input_serialization, output_serialization, extra_args, version_id)**: Performs SQL queries on object content. ### Bucket Operations #### Creation and Management - **make_bucket(bucket_name, location, object_lock)**: Creates a new bucket. - **bucket_exists(bucket_name)**: Checks if a bucket exists. - **list_buckets()**: Lists all buckets. - **delete_bucket(bucket_name)**: Deletes a bucket. #### Configuration - **set_bucket_cors(bucket_name, rule)**: Configures CORS for a bucket. - **get_bucket_cors(bucket_name)**: Retrieves CORS configuration for a bucket. - **set_bucket_lifecycle(bucket_name, rule)**: Configures lifecycle rules for a bucket. - **get_bucket_lifecycle(bucket_name)**: Retrieves lifecycle configuration for a bucket. - **set_bucket_versioning(bucket_name, status)**: Configures versioning for a bucket. - **get_bucket_versioning(bucket_name)**: Retrieves versioning configuration for a bucket. - **set_bucket_replication(bucket_name, rule)**: Configures replication for a bucket. - **get_bucket_replication(bucket_name)**: Retrieves replication configuration for a bucket. #### Policies and Encryption - **set_bucket_policy(bucket_name, policy)**: Sets a bucket policy. - **get_bucket_policy(bucket_name)**: Retrieves a bucket policy. - **set_object_encryption(bucket_name, object_name, encryption)**: Sets object encryption. #### Tags and Notifications - **set_bucket_tagging(bucket_name, tags)**: Sets bucket tags. - **get_bucket_tagging(bucket_name)**: Retrieves bucket tags. - **set_bucket_notification(bucket_name, notification)**: Sets bucket notifications. - **get_bucket_notification(bucket_name)**: Retrieves bucket notifications. #### Object Locking and Retention - **set_object_lock_configuration(bucket_name, config)**: Sets object lock configuration. - **get_object_lock_configuration(bucket_name)**: Retrieves object lock configuration. - **set_object_retention(bucket_name, object_name, retention)**: Sets object retention. - **get_object_retention(bucket_name, object_name)**: Retrieves object retention. ### Metadata Management #### Object Metadata - **set_object_tags(bucket_name, object_name, tags)**: Sets tags for an object. - **get_object_tags(bucket_name, object_name)**: Retrieves tags for an object. - **remove_object_tags(bucket_name, object_name)**: Removes tags from an object. #### Bucket Metadata - **set_bucket_tags(bucket_name, tags)**: Sets tags for a bucket. - **get_bucket_tags(bucket_name)**: Retrieves tags for a bucket. ### Security Features #### Server-Side Encryption - **SSE-S3**: Server-Side Encryption with Amazon S3-Managed Keys. - **SSE-KMS**: Server-Side Encryption with AWS Key Management Service. - **SSE-C**: Server-Side Encryption with Customer-Provided Keys. #### Credentials Providers - Supports multiple credential providers and chains for authentication. #### Presigned URLs - **presigned_get_url(bucket_name, object_name, expires, extra_args)**: Generates a presigned URL for GET requests. - **presigned_put_url(bucket_name, object_name, expires, extra_args)**: Generates a presigned URL for PUT requests. - **presigned_post_policy(method, bucket_name, object_name, expires, form_data)**: Generates a presigned POST policy. #### Access Control and Legal Holds - Methods for managing access control policies and legal holds on objects and buckets. ``` -------------------------------- ### Get Bucket Metadata with MinIO Client Source: https://github.com/minio/minio-py/blob/master/_autodocs/api-reference/minio-client.md Use `head_bucket` to retrieve metadata for a specific bucket without listing its contents. Optionally specify the region. ```python response = client.head_bucket(bucket_name="my-bucket") print(response.bucket_name) ``` -------------------------------- ### Checksum Data Structure Source: https://github.com/minio/minio-py/blob/master/_autodocs/types.md Represents object checksum information across multiple algorithms. Used in object operations like get, stat, and put. ```python from dataclasses import dataclass from typing import Optional @dataclass(frozen=True) class Checksum: checksum_crc32: Optional[str] = None checksum_crc32c: Optional[str] = None checksum_crc64nvme: Optional[str] = None checksum_sha1: Optional[str] = None checksum_sha256: Optional[str] = None checksum_type: Optional[str] = None ```