### Initialize ECMWF Data Stores Client (Python) Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/notebooks/quick_start Demonstrates how to import and initialize the ECMWF Data Stores client. This is the first step to interacting with ECMWF data stores. Ensure the 'ecmwf-datastores' library is installed. ```python from ecmwf.datastores import Client client = Client() ``` -------------------------------- ### Install ECMWF Data Stores Client with Pip Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Installs the ecmwf-datastores-client package using the pip package manager. This is a standard Python package installation method. ```shell pip install ecmwf-datastores-client ``` -------------------------------- ### ECMWF Data Stores Client Configuration File Example Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Example of a configuration file (.ecmwfdatastoresrc) for the ECMWF Data Stores Client. This file specifies the API URL and a user key, typically stored in the user's home directory. ```yaml url: https://cds.climate.copernicus.eu/api key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ``` -------------------------------- ### Install ECMWF Data Stores Client with Conda Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Installs the ecmwf-datastores-client package using the conda package manager from the conda-forge channel. Ensure conda is installed and configured. ```shell conda install -c conda-forge ecmwf-datastores-client ``` -------------------------------- ### Explore a Collection with ECMWF Datastores Client in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Provides examples of exploring a specific collection's metadata, such as its ID, title, description, timestamps, and bounding box, using the ECMWF Datastores Client. It also shows how to submit a request or apply constraints directly to a collection object. ```python collection = client.get_collection(collection_id) collection.id == collection_id collection.title collection.description collection.published_at collection.updated_at collection.begin_datetime collection.end_datetime collection.bbox collection.submit(request) collection.apply_constraints(request) ``` -------------------------------- ### Install ECMWF Data Stores Client with Pip Source: https://ecmwf.github.io/ecmwf-datastores-client/README Installs the ecmwf-datastores-client library using the pip package manager. This is a standard method for Python package installation. ```bash $ pip install ecmwf-datastores-client ``` -------------------------------- ### Instantiate and Check Authentication for ECMWF Data Stores Client Source: https://ecmwf.github.io/ecmwf-datastores-client/README Initializes the ECMWF Data Stores client and optionally verifies the authentication credentials. This requires the client to be installed and configured. ```python >>> from ecmwf.datastores import Client >>> client = Client() >>> client.check_authentication() # optional check {...} ``` -------------------------------- ### Install ECMWF Data Stores Client with Conda Source: https://ecmwf.github.io/ecmwf-datastores-client/README Installs the ecmwf-datastores-client library using the conda package manager from the conda-forge channel. Ensure you have conda installed and configured. ```bash $ conda install -c conda-forge ecmwf-datastores-client ``` -------------------------------- ### Client Initialization Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Initialize the DSS Client with optional parameters for URL, API key, TLS verification, timeouts, and progress display. ```APIDOC ## Client Initialization ### Description Initializes the ECMWF Data Stores Service (DSS) API Python client. ### Parameters - **url** (str or None) - API URL. Defaults to ECMWF_DATASTORES_URL or ECMWF_DATASTORES_RC_FILE if None. - **key** (str or None) - API Key. Defaults to ECMWF_DATASTORES_KEY or ECMWF_DATASTORES_RC_FILE if None. - **verify** (bool) - Whether to verify the TLS certificate. Defaults to True. - **timeout** (float or tuple[float, float]) - Connection and read timeout in seconds. Defaults to 60. - **progress** (bool) - Whether to display a progress bar during download. Defaults to True. - **cleanup** (bool) - Whether to delete requests after completion. Defaults to False. - **sleep_max** (float) - Maximum time to wait (in seconds) for status changes. Defaults to 120. - **retry_after** (float) - Time to wait (in seconds) between retries. Defaults to 120. - **maximum_tries** (int) - Maximum number of retries. Defaults to 500. - **session** (requests.Session) - A requests session object. ``` -------------------------------- ### Set up Conda Environment for Development Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Instructions for setting up a new Conda environment named 'DEVELOP' with Python 3.11, which is recommended for developing with the ECMWF Datastores Client. ```bash conda create -n DEVELOP -c conda-forge python=3.11 conda activate DEVELOP ``` -------------------------------- ### Instantiate ECMWF Datastores Client in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Initializes the ECMWF Datastores Client and optionally checks authentication. This is the first step before interacting with the datastores. ```python from ecmwf.datastores import Client client = Client() client.check_authentication() # optional check ``` -------------------------------- ### Developer Workflow Commands for ECMWF Datastores Client Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README A list of commands to run for quality assurance, testing, and documentation building during the development of the ECMWF Datastores Client. These commands ensure the code adheres to standards and is well-documented. ```bash make conda-env-update pip install -e . make template-update make qa make unit-tests make type-check make docs-build ``` -------------------------------- ### Client Initialization Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Initializes the ECMWF Data Stores Service (DSS) API Python client with various configuration options. ```APIDOC ## Client Initialization ### Description Initializes the ECMWF Data Stores Service (DSS) API Python client. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from ecmwf.datastores import Client client = Client( url=None, key=None, verify=True, timeout=60, progress=True, cleanup=False, sleep_max=120, retry_after=120, maximum_tries=500, session=None ) ``` ### Response *Note: This is an initialization, not a direct API call. No direct response.* ``` -------------------------------- ### retrieve Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Submit a request to retrieve data and download the results. ```APIDOC ## retrieve ### Description Submits a data retrieval request for a specified collection and downloads the results to a target location. ### Method POST ### Endpoint `/retrieve` (example endpoint, actual may vary) ### Parameters #### Path Parameters - **collection_id** (str) - Required - The ID of the collection to retrieve data from (e.g., "projections-cmip6"). #### Request Body - **request** (dict[str,Any]) - Required - The parameters defining the data retrieval request. #### Query Parameters - **target** (str or None) - Optional - The target path for the download. If None, downloads to the working directory. ### Response #### Success Response (200) - **downloaded_file_path** (str) - The path to the downloaded results file. ``` -------------------------------- ### Asynchronous Data Retrieval with ECMWF Datastores Client in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Shows alternative methods for retrieving data asynchronously using the ECMWF Datastores Client. This includes submitting a request, downloading results later, and waiting for results. ```python remote = client.submit(collection_id, request) # doesn't block remote.download("target_2.grib") # blocks results = client.submit_and_wait_on_results(collection_id, request) # blocks results.download("target_3.grib") client.download_results(remote.request_id, "target_4.grib") # blocks ``` -------------------------------- ### Submit Request Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Submits a request to the datastore client. ```APIDOC ## POST /submit ### Description Submits a request to the datastore client. This method is asynchronous and returns a reference to the submitted request. ### Method POST ### Endpoint /submit ### Parameters #### Request Body - **collection_id** (string) - Required - The ID of the collection to submit the request to (e.g., "projections-cmip6"). - **request** (object) - Required - A dictionary containing the request parameters. ``` -------------------------------- ### retrieve Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Submits a request to the Data Stores Service and downloads the results. ```APIDOC ## POST /retrieve ### Description Submits a request and retrieves the results. ### Method POST ### Endpoint `/retrieve` ### Parameters #### Path Parameters None #### Query Parameters - **target** (str, Optional) - The local path where the results should be saved. If None, downloads to the current working directory. #### Request Body - **collection_id** (str) - Required - Collection ID (e.g., "projections-cmip6"). - **request** (dict[str,Any]) - Required - Request parameters. ### Request Example ```json { "collection_id": "projections-cmip6", "request": { "variable": "t2m", "year": 2023 }, "target": "/data/downloaded_results" } ``` ### Response #### Success Response (200) - **str** - Path to the retrieved file on the local system. #### Response Example ``` /data/downloaded_results/t2m_2023_results.nc ``` ``` -------------------------------- ### Interact with Results using ECMWF Datastores Client in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Shows how to interact with data retrieval results obtained from the ECMWF Datastores Client. This includes checking content length, type, location, and downloading the results. ```python results = client.get_results(remote.request_id) results.content_length > 0 results.content_type results.location results.download("target_5.grib") ``` -------------------------------- ### Retrieve Data with ECMWF Datastores Client in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Demonstrates how to retrieve data from a specified collection using the ECMWF Datastores Client. It includes defining a request with various parameters and downloading the data to a local file. ```python collection_id = "reanalysis-era5-pressure-levels" request = { "product_type": ["reanalysis"], "variable": ["temperature"], "year": ["2022"], "month": ["01"], "day": ["01"], "time": ["00:00"], "pressure_level": ["1000"], "data_format": "grib", "download_format": "unarchived", } client.retrieve(collection_id, request, target="target_1.grib") # blocks ``` -------------------------------- ### Apply Constraints to Find Available Days in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Shows how to apply constraints to a collection request to determine the number of available days for a given month. This is useful for planning data requests. ```python month = {"year": "2000", "month": "02"} constrained_request = client.apply_constraints(collection_id, month) len(constrained_request["day"]) ``` -------------------------------- ### submit Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Submits a request to the Data Stores Service without waiting for results. ```APIDOC ## POST /submit ### Description Submits a request to the ECMWF Data Stores Service. ### Method POST ### Endpoint `/submit` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **collection_id** (str) - Required - Collection ID (e.g., "projections-cmip6"). - **request** (dict[str,Any]) - Required - Request parameters. ### Request Example ```json { "collection_id": "projections-cmip6", "request": { "variable": "mslp", "date": "2023-01-01" } } ``` ### Response #### Success Response (200) - **datastores.Remote** - An object representing the submitted request, including its ID and status. #### Response Example ```json { "request_id": "req_789", "status": "submitted" } ``` ``` -------------------------------- ### Submit and Wait for Results Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Submits a request and waits for the results to be ready. ```APIDOC ## POST /submit_and_wait_on_results ### Description Submits a request to the datastore client and blocks until the results are ready. This method is suitable for workflows where immediate access to results is required. ### Method POST ### Endpoint /submit_and_wait_on_results ### Parameters #### Request Body - **collection_id** (string) - Required - The ID of the collection to submit the request to (e.g., "projections-cmip6"). - **request** (object) - Required - A dictionary containing the request parameters. ### Response #### Success Response (200) - **results** (object) - The results of the request. The structure of the results depends on the specific datastore and request made. ``` -------------------------------- ### apply_constraints Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Apply constraints to request parameters for a specified collection. ```APIDOC ## apply_constraints ### Description Applies constraints to the parameters in a request for a given collection. ### Method POST ### Endpoint `/collections/{collection_id}/constraints` (example endpoint, actual may vary) ### Parameters #### Path Parameters - **collection_id** (str) - Required - The ID of the collection (e.g., "projections-cmip6"). #### Request Body - **request** (dict[str,Any]) - Required - The request parameters to apply constraints to. ### Response #### Success Response (200) - **valid_values** (dict[str,Any]) - A dictionary containing the valid values for the request parameters. ``` -------------------------------- ### datastores.Results.download() Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Results Downloads the results of a datastore job to a specified target path or the working directory if no target is provided. ```APIDOC ## datastores.Results.download() ### Description Downloads the results of a datastore job. If a target path is specified, the results will be saved to that path. Otherwise, the results will be downloaded to the current working directory. ### Method Not applicable (this is a method of the Results class) ### Endpoint Not applicable (this is a method of the Results class) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python results.download(target='/path/to/save/results.zip') ``` ### Response #### Success Response - **path** (str) - The path to the downloaded file. #### Response Example ```json { "path": "/path/to/save/results.zip" } ``` ``` -------------------------------- ### check_authentication Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Verify the client's authentication credentials. ```APIDOC ## check_authentication ### Description Verifies the authentication credentials of the client. ### Method GET ### Endpoint `/auth/check` (example endpoint, actual may vary) ### Response #### Success Response (200) - **response_content** (dict[str,Any]) - The content of the authentication response. #### Error Response - **HTTPError** - Raised if authentication fails. ``` -------------------------------- ### download_results Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Download the results of a specific request to a target location. ```APIDOC ## download_results ### Description Downloads the results of a specific request to a specified target path or the working directory. ### Method GET ### Endpoint `/results/{request_id}` (example endpoint, actual may vary) ### Parameters #### Path Parameters - **request_id** (str) - Required - The ID of the request whose results are to be downloaded. #### Query Parameters - **target** (str or None) - Optional - The target path for the download. If None, downloads to the working directory. ### Response #### Success Response (200) - **file_path** (str) - The path to the retrieved file. ``` -------------------------------- ### get_jobs Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Retrieve a list of submitted jobs with optional filtering and sorting. ```APIDOC ## get_jobs ### Description Retrieves a list of submitted jobs, with options for pagination, sorting, and filtering by status. ### Method GET ### Endpoint `/jobs` (example endpoint, actual may vary) ### Parameters #### Query Parameters - **limit** (int or None) - Optional - The number of jobs to return per page. - **sortby** ({None, 'created', '-created'}) - Optional - The field to sort the results by. - **status** (None or {'accepted', 'running', 'successful', 'failed', 'rejected'} or list) - Optional - Filter jobs by their status. ### Response #### Success Response (200) - **jobs_list** (datastores.Jobs) - An object containing a list of jobs. ``` -------------------------------- ### get_jobs Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Retrieves a list of submitted jobs with options for filtering and sorting. ```APIDOC ## GET /jobs ### Description Retrieves submitted jobs. ### Method GET ### Endpoint `/jobs` ### Parameters #### Path Parameters None #### Query Parameters - **limit** (int, Optional) - The maximum number of jobs to return per page. - **sortby** ({None, 'created', '-created'}, Optional) - The field to sort the results by. Use '-created' for descending order. - **status** (None or {'accepted', 'running', 'successful', 'failed', 'rejected'} or list, Optional) - Filter jobs by their status. #### Request Body None ### Request Example ``` GET /jobs?limit=20&status=successful&sortby=-created ``` ### Response #### Success Response (200) - **datastores.Jobs** - An object representing a list of jobs. #### Response Example ```json { "jobs": [ { "id": "job_abc", "status": "successful", "created_at": "2023-10-27T10:00:00Z" }, { "id": "job_def", "status": "running", "created_at": "2023-10-27T09:30:00Z" } ], "total": 150 } ``` ``` -------------------------------- ### Configure Logging Level with Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Configures the Python logging module to display messages at the INFO level. This is useful for monitoring the client's operations. ```python import logging logging.basicConfig(level="INFO") ``` -------------------------------- ### download_results Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Downloads the results of a specific request to a target location. ```APIDOC ## GET /results/{request_id} ### Description Downloads the results of a request. ### Method GET ### Endpoint `/results/{request_id}` ### Parameters #### Path Parameters - **request_id** (str) - Required - The ID of the request whose results are to be downloaded. #### Query Parameters - **target** (str, Optional) - The local path where the results should be saved. If None, downloads to the current working directory. #### Request Body None ### Request Example ``` GET /results/req_123?target=/path/to/save/results ``` ### Response #### Success Response (200) - **str** - The path to the retrieved file on the local system. #### Response Example ``` /path/to/save/results/results_for_req_123.zip ``` ``` -------------------------------- ### Configure ECMWF Data Stores Client with a Config File Source: https://ecmwf.github.io/ecmwf-datastores-client/README Demonstrates how to configure the ECMWF Data Stores client by creating a configuration file. This file, typically located at ~/.ecmwfdatastoresrc, specifies the API URL and key. ```yaml $ cat $HOME/.ecmwfdatastoresrc url: https://cds.climate.copernicus.eu/api key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ``` -------------------------------- ### submit_and_wait_on_results Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Submits a request and waits for the results to be ready, then returns them. ```APIDOC ## POST /submit_and_wait ### Description Submits a request and waits for the results to be ready. ### Method POST ### Endpoint `/submit_and_wait` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **collection_id** (str) - Required - Collection ID (e.g., "projections-cmip6"). - **request** (dict[str,Any]) - Required - Request parameters. ### Request Example ```json { "collection_id": "projections-cmip6", "request": { "variable": "u10", "time": "12:00" } } ``` ### Response #### Success Response (200) - **datastores.Results** - An object representing the results metadata once they are ready. #### Response Example ```json { "request_id": "req_abc", "status": "completed", "file_name": "u10_results.grib" } ``` ``` -------------------------------- ### check_authentication Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Verifies the client's authentication with the ECMWF Data Stores Service. ```APIDOC ## GET /check_authentication ### Description Verifies authentication with the ECMWF Data Stores Service. ### Method GET ### Endpoint `/check_authentication` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ``` GET /check_authentication ``` ### Response #### Success Response (200) - **dict[str,Any]** - Content of the response indicating successful authentication. #### Response Example ```json { "status": "authenticated", "message": "Authentication successful." } ``` #### Error Response - **requests.HTTPError** - If the authentication fails. ``` -------------------------------- ### List Successful Jobs with ECMWF Datastores Client in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Demonstrates how to list all successful jobs, sorted by newest first, using the ECMWF Datastores Client. It includes iterating through paginated job results to extract request IDs. ```python jobs = client.get_jobs(sortby="-created", status="successful") request_ids = [] while jobs is not None: # Loop over pages request_ids.extend(jobs.request_ids) jobs = jobs.next # Move to the next page ``` -------------------------------- ### apply_constraints Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Applies constraints to the parameters in a request for a given collection. ```APIDOC ## POST /apply_constraints ### Description Applies constraints to the parameters in a request. ### Method POST ### Endpoint `/apply_constraints` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **collection_id** (str) - Required - Collection ID (e.g., "projections-cmip6"). - **request** (dict[str,Any]) - Required - Request parameters. ### Request Example ```json { "collection_id": "projections-cmip6", "request": { "param1": "value1", "param2": "value2" } } ``` ### Response #### Success Response (200) - **dict[str,Any]** - Dictionary of valid values. #### Response Example ```json { "constrained_param1": "valid_value1", "constrained_param2": "valid_value2" } ``` ``` -------------------------------- ### List Collections with ECMWF Datastores Client in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Demonstrates how to list all available collection IDs, sorted by their last update time, using the ECMWF Datastores Client. It includes iterating through paginated results. ```python collections = client.get_collections(sortby="update") collection_ids = [] while collections is not None: # Loop over pages collection_ids.extend(collections.collection_ids) collections = collections.next # Move to the next page ``` -------------------------------- ### Download Job Results (Python) Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Results The `download` method allows users to download the results of a job. It can save the file to a specified target path or the current working directory if no target is provided. The method returns the path to the downloaded file. ```python from ecmwf_client.datastores import Results results = Results() results.download("/path/to/save/results.zip") ``` -------------------------------- ### Submit Request to Collection (Python) Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Collection The `submit` method is used to send a request to the datastore collection. It accepts a dictionary of request parameters and returns a `datastores.Remote` object, representing the submitted request. This is a core method of the `datastores.Collection` class. ```python def submit(request): # Submit a request. # :param request: Request parameters. # :type request: :py:class:`dict[str,Any]` # :rtype: :py:class:`datastores.Remote` pass ``` -------------------------------- ### Interact with a Submitted Job in Python Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/README Explains how to interact with a previously submitted job using its request ID. This includes retrieving job details, checking status, downloading results, and deleting the job. ```python remote = client.get_remote(remote.request_id) remote.collection_id == collection_id remote.request == request remote.status remote.results_ready remote.created_at remote.started_at remote.finished_at remote.updated_at == remote.finished_at remote.download("target_6.grib") remote.get_results() remote.delete() ``` -------------------------------- ### datastores.Results Properties Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Results Access metadata about the datastore job results, including content length, content type, JSON content, location, and URL. ```APIDOC ## datastores.Results Properties ### Description These properties provide metadata about the response received from the datastore job. ### Methods None (these are properties) ### Endpoints None (these are properties) ### Parameters None ### Request Example ```python results = datastores.Results(...) print(f"File size: {results.content_length} Bytes") print(f"MIME type: {results.content_type}") print(f"JSON content: {results.json}") print(f"File location: {results.location}") print(f"URL: {results.url}") ``` ### Response Properties - **content_length** (int) - The size of the file in Bytes. - **content_type** (str) - The MIME type of the file. - **json** (Any) - The content of the response parsed as JSON. - **location** (str) - The location of the file. - **url** (str) - The URL from which the file was downloaded. ``` -------------------------------- ### Collection - Submit Request Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Collection Submits a request to the collection and returns a `datastores.Remote` object. ```APIDOC ## POST /collections/submit ### Description Submit a request to the collection. ### Method POST ### Endpoint /collections/submit ### Parameters #### Request Body - **request** (dict[str,Any]) - Required - Request parameters. ### Response #### Success Response (200) - **Remote Object** (datastores.Remote) - The remote object representing the submitted request. #### Response Example ```json { "remote_url": "http://example.com/remote_resource" } ``` ``` -------------------------------- ### get_collections Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Retrieve a list of catalogue collections with optional filtering and sorting. ```APIDOC ## get_collections ### Description Retrieves a list of catalogue collections, with options for pagination, sorting, full-text search, and keyword filtering. ### Method GET ### Endpoint `/collections` (example endpoint, actual may vary) ### Parameters #### Query Parameters - **limit** (int or None) - Optional - The number of collections to return per page. - **sortby** ({None, 'id', 'relevance', 'title', 'update'}) - Optional - The field to sort the results by. - **query** (str or None) - Optional - A full-text search query to filter collections. - **keywords** (list[str] or None) - Optional - A list of keywords to filter collections by. ### Response #### Success Response (200) - **collections_list** (datastores.Collections) - An object containing a list of catalogue collections. ``` -------------------------------- ### get_collections Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Retrieves a list of catalogue collections, with options for filtering and sorting. ```APIDOC ## GET /collections ### Description Retrieves a list of catalogue collections. ### Method GET ### Endpoint `/collections` ### Parameters #### Path Parameters None #### Query Parameters - **limit** (int, Optional) - The maximum number of collections to return per page. - **sortby** ({None, 'id', 'relevance', 'title', 'update'}, Optional) - The field to sort the results by. - **query** (str, Optional) - A full-text search query to filter collections. - **keywords** (list[str], Optional) - A list of keywords to filter collections. #### Request Body None ### Request Example ``` GET /collections?limit=10&sortby=title&query=climate&keywords=CMIP6 ``` ### Response #### Success Response (200) - **datastores.Collections** - An object representing a list of collections. #### Response Example ```json { "collections": [ { "id": "projections-cmip6", "title": "CMIP6 Projections" }, { "id": "reanalysis-era5", "title": "ERA5 Reanalysis" } ], "total": 2 } ``` ``` -------------------------------- ### datastores.Collection Properties Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Collection Access metadata and details of a datastores collection. ```APIDOC ## datastores.Collection Properties ### Description These properties provide access to various metadata and attributes of a datastores collection. ### Methods N/A (These are properties) ### Endpoints N/A ### Properties - **bbox** (tuple[float, float, float, float]) - Bounding box of the collection (W, S, E, N). - **begin_datetime** (datetime.datetime | None) - Begin datetime of the collection. - **description** (str) - Description of the collection. - **end_datetime** (datetime.datetime | None) - End datetime of the collection. - **id** (str) - Collection ID. - **json** (Any) - Content of the response. - **published_at** (datetime.datetime) - When the collection was first published. - **title** (str) - Title of the collection. - **updated_at** (datetime.datetime) - When the collection was last updated. - **url** (str) - URL of the collection. ### Example Usage ```python collection = datastores.Collection.get("collection_id") print(collection.title) print(collection.bbox) ``` ``` -------------------------------- ### get_results Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Retrieve the results metadata for a specific request. ```APIDOC ## get_results ### Description Retrieves the metadata and information about the results of a specific request. ### Method GET ### Endpoint `/results/{request_id}` (example endpoint, actual may vary) ### Parameters #### Path Parameters - **request_id** (str) - Required - The ID of the request whose results metadata is to be retrieved. ### Response #### Success Response (200) - **results_metadata** (datastores.Results) - An object containing the results metadata. ``` -------------------------------- ### Interact with Submitted Jobs using datastores.Remote (Python) Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Remote The `datastores.Remote` class allows interaction with submitted jobs. It provides methods to delete jobs, download results to a specified target or the working directory, and retrieve job results. It also exposes properties for job status, collection ID, creation and update times, and request details. ```python class datastores.Remote: def delete() -> dict[str,Any] def download(target = None) def get_results() -> datastores.Results collection_id: str created_at: datetime.datetime finished_at: datetime.datetime | None json: dict[str, Any] request: dict[str, Any] request_id: str results_ready: bool started_at: datetime.datetime | None status: str updated_at: datetime.datetime ``` -------------------------------- ### Collection - Apply Constraints Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Collection Applies constraints to the parameters within a request and returns a dictionary of valid values. ```APIDOC ## POST /collections/apply_constraints ### Description Applies constraints to the parameters in a request. ### Method POST ### Endpoint /collections/apply_constraints ### Parameters #### Request Body - **request** (dict[str,Any]) - Required - Request parameters. ### Response #### Success Response (200) - **Valid Values** (dict[str,Any]) - Dictionary of valid values. #### Response Example ```json { "valid_values": { "param1": "value1", "param2": "value2" } } ``` ``` -------------------------------- ### Collection Properties Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Collection Provides access to various properties of the collection, such as bounding box, datetimes, ID, and URL. ```APIDOC ## Collection Properties ### Description Accesses properties of the collection. ### Properties - **bbox** (tuple[float, float, float, float]) - Bounding box of the collection (W, S, E, N). - **begin_datetime** (datetime.datetime | None) - Begin datetime of the collection. - **description** (str) - Description of the collection. - **end_datetime** (datetime.datetime | None) - End datetime of the collection. - **id** (str) - Collection ID. - **json** (Any) - Content of the response. - **published_at** (datetime.datetime) - When the collection was first published. - **title** (str) - Title of the collection. - **updated_at** (datetime.datetime) - When the collection was last updated. - **url** (str) - URL. ``` -------------------------------- ### datastores.Collection.submit Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Collection Submits a request to the datastores collection. This method is used to send a request with specified parameters. ```APIDOC ## datastores.Collection.submit ### Description Submit a request to the datastores collection. ### Method N/A (This is a class method) ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **request** (dict[str,Any]) - Required - Request parameters. ### Request Example ```json { "request": { "collection_id": "example_collection", "date": "2023-10-27" } } ``` ### Response #### Success Response (200) - **Remote Object** (datastores.Remote) - Represents the submitted request and its results. #### Response Example ```json { "status": "submitted", "job_id": "12345abcde" } ``` ``` -------------------------------- ### Explore a Specific Collection with ECMWF Data Stores Client Source: https://ecmwf.github.io/ecmwf-datastores-client/README Fetches detailed information about a specific collection, identified by its ID. This includes metadata such as title, description, dates, and bounding box. It also allows submitting new requests or applying constraints to the collection. ```python >>> collection = client.get_collection(collection_id) >>> collection.id == collection_id True >>> collection.title '...' >>> collection.description '...' >>> collection.published_at datetime.datetime(...) >>> collection.updated_at datetime.datetime(...) >>> collection.begin_datetime datetime.datetime(...) >>> collection.end_datetime datetime.datetime(...) >>> collection.bbox (...) >>> collection.submit(request) Remote(...) >>> collection.apply_constraints(request) {...} ``` -------------------------------- ### Remote Job Management API Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Remote This section covers the methods for managing a remote job, including deletion and downloading results. ```APIDOC ## DELETE /jobs/{job_id} ### Description Deletes the specified job. ### Method DELETE ### Endpoint /jobs/{job_id} ### Parameters #### Path Parameters - **job_id** (str) - Required - The unique identifier of the job to delete. ### Response #### Success Response (200) - **message** (str) - A confirmation message indicating the job was deleted. #### Response Example ```json { "message": "Job deleted successfully." } ``` ``` ```APIDOC ## POST /jobs/{job_id}/download ### Description Downloads the results of a specified job to a target location. ### Method POST ### Endpoint /jobs/{job_id}/download ### Parameters #### Path Parameters - **job_id** (str) - Required - The unique identifier of the job whose results are to be downloaded. #### Query Parameters - **target** (str) - Optional - The target path for the download. If not provided, results are downloaded to the current working directory. ### Response #### Success Response (200) - **file_path** (str) - The path to the downloaded file. #### Response Example ```json { "file_path": "/path/to/downloaded/results.zip" } ``` ``` ```APIDOC ## GET /jobs/{job_id}/results ### Description Retrieves the results of a specified job. ### Method GET ### Endpoint /jobs/{job_id}/results ### Parameters #### Path Parameters - **job_id** (str) - Required - The unique identifier of the job whose results are to be retrieved. ### Response #### Success Response (200) - **results** (object) - The results of the job, typically in a structured format like JSON or a specific object type defined by the datastores.Results class. #### Response Example ```json { "results": { "data": [ { "column1": "value1", "column2": "value2" } ], "metadata": { "total_records": 100 } } } ``` ``` -------------------------------- ### Remote Job Properties API Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Remote This section describes the properties available for accessing information about a remote job. ```APIDOC ## GET /jobs/{job_id} ### Description Retrieves metadata and status information for a specific job. ### Method GET ### Endpoint /jobs/{job_id} ### Parameters #### Path Parameters - **job_id** (str) - Required - The unique identifier of the job. ### Response #### Success Response (200) - **collection_id** (str) - The ID of the collection associated with the job. - **created_at** (datetime) - The timestamp when the job was created. - **finished_at** (datetime or null) - The timestamp when the job finished. Null if not yet finished. - **request_id** (str) - The unique identifier for the request. - **results_ready** (bool) - Indicates whether the job results are ready. - **started_at** (datetime or null) - The timestamp when the job started. Null if not yet started. - **status** (str) - The current status of the job (e.g., PENDING, RUNNING, COMPLETED, FAILED). - **updated_at** (datetime) - The timestamp when the job was last updated. - **request** (object) - The parameters used for the job request. - **json** (object) - The raw JSON content of the job response. #### Response Example ```json { "collection_id": "collection-abc-123", "created_at": "2023-10-27T10:00:00Z", "finished_at": null, "request_id": "req-xyz-789", "results_ready": false, "started_at": "2023-10-27T10:05:00Z", "status": "RUNNING", "updated_at": "2023-10-27T10:15:00Z", "request": { "param1": "value1" }, "json": { "some_response_key": "some_response_value" } } ``` ``` -------------------------------- ### Apply Constraints to Collection Request (Python) Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Collection The `apply_constraints` method modifies request parameters to align with the collection's valid values. It takes a dictionary of request parameters and returns a dictionary of validated values. This method is part of the `datastores.Collection` class. ```python def apply_constraints(request): # Apply constraints to the parameters in a request. # :param request: Request parameters. # :type request: :py:class:`dict[str,Any]` # :returns: Dictionary of valid values. # :rtype: :py:class:`dict[str,Any]` pass ``` -------------------------------- ### get_results Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Client Retrieves the results metadata for a specific request. ```APIDOC ## GET /requests/{request_id}/results ### Description Retrieves the results metadata of a request. ### Method GET ### Endpoint `/requests/{request_id}/results` ### Parameters #### Path Parameters - **request_id** (str) - Required - The ID of the request. #### Query Parameters None #### Request Body None ### Request Example ``` GET /requests/req_123/results ``` ### Response #### Success Response (200) - **datastores.Results** - An object representing the results metadata. #### Response Example ```json { "request_id": "req_123", "status": "completed", "file_size": 1024000, "file_name": "results_req_123.zip" } ``` ``` -------------------------------- ### Access Result Metadata (Python) Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Results The `datastores.Results` class provides properties to access various metadata about the job results. These include `content_length` (file size in bytes), `content_type` (MIME type), `json` (response content), `location` (file location), and `url` (the file's URL). ```python from ecmwf_client.datastores import Results results = Results() print(f"Content Length: {results.content_length}") print(f"Content Type: {results.content_type}") print(f"JSON Content: {results.json}") print(f"Location: {results.location}") print(f"URL: {results.url}") ``` -------------------------------- ### Remote Job Management API Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Remote This section details the operations available for managing a submitted datastore job, including deleting the job, downloading results, and retrieving job status. ```APIDOC ## datastores.Remote.delete() ### Description Deletes the submitted job. ### Method DELETE ### Endpoint `/jobs/{request_id}` (Assumed endpoint based on function behavior) ### Parameters #### Query Parameters None #### Request Body None ### Response #### Success Response (200) - **dict[str, Any]** - Content of the response. #### Response Example ```json { "message": "Job deleted successfully" } ``` ## datastores.Remote.download() ### Description Downloads the results of the job to a specified target path. ### Method GET ### Endpoint `/jobs/{request_id}/download` (Assumed endpoint based on function behavior) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```json { "target": "/path/to/save/results" } ``` ### Response #### Success Response (200) - **str** - Path to the retrieved file. #### Response Example ```json { "file_path": "/path/to/save/results/results.zip" } ``` ## datastores.Remote.get_results() ### Description Retrieves the results of the job. ### Method GET ### Endpoint `/jobs/{request_id}/results` (Assumed endpoint based on function behavior) ### Parameters None ### Response #### Success Response (200) - **datastores.Results** - The results object. #### Response Example ```json { "results": [ { "data": "...", "metadata": "..." } ] } ``` ## datastores.Remote Properties ### Description Properties providing information about the job. ### Properties Overview - **collection_id** (`str`): The ID of the collection associated with the job. - **created_at** (`datetime.datetime`): The timestamp when the job was created. - **finished_at** (`datetime.datetime` or `None`): The timestamp when the job finished. `None` if the job has not finished. - **json** (`dict[str, Any]`): The content of the response. - **request** (`dict[str, Any]`): The parameters used for the request. - **request_id** (`str`): The unique identifier for the request. - **results_ready** (`bool`): A flag indicating whether the results are ready. - **started_at** (`datetime.datetime` or `None`): The timestamp when the job started. `None` if the job has not started. - **status** (`str`): The current status of the request. - **updated_at** (`datetime.datetime`): The timestamp when the job was last updated. ``` -------------------------------- ### Submit Data Retrieval Request with ECMWF Data Stores Client (Non-blocking) Source: https://ecmwf.github.io/ecmwf-datastores-client/README Submits a data retrieval request asynchronously using the `submit` method. This method does not block and returns a `Remote` object that can be used to track and download the results later. ```python >>> remote = client.submit(collection_id, request) # doesn't block >>> remote Remote(...) >>> remote.download("target_2.grib") # blocks 'target_2.grib' ``` -------------------------------- ### Apply Constraints to Find Available Days in a Month with ECMWF Data Stores Client Source: https://ecmwf.github.io/ecmwf-datastores-client/README Applies constraints to a collection request to determine the number of available days within a specific month. This is useful for planning data retrievals. ```python >>> month = {"year": "2000", "month": "02"} >>> constrained_request = client.apply_constraints(collection_id, month) >>> len(constrained_request["day"]) 29 ``` -------------------------------- ### get_remote Source: https://ecmwf.github.io/ecmwf-datastores-client/_sources/_api/datastores/Client Retrieve the remote object associated with a request. ```APIDOC ## get_remote ### Description Retrieves the remote object details associated with a specific request. ### Method GET ### Endpoint `/requests/{request_id}/remote` (example endpoint, actual may vary) ### Parameters #### Path Parameters - **request_id** (str) - Required - The ID of the request for which to retrieve the remote object. ### Response #### Success Response (200) - **remote_object** (datastores.Remote) - An object representing the remote details of the request. ``` -------------------------------- ### Submit and Wait for Results with ECMWF Data Stores Client Source: https://ecmwf.github.io/ecmwf-datastores-client/README Submits a data retrieval request and waits for the results to be ready. This method blocks until the data is available and returns a `Results` object. ```python >>> results = client.submit_and_wait_on_results(collection_id, request) # blocks >>> results Results(...) >>> results.download("target_3.grib") 'target_3.grib' ``` -------------------------------- ### datastores.Collection.apply_constraints Source: https://ecmwf.github.io/ecmwf-datastores-client/_api/datastores/Collection Applies constraints to the parameters within a given request. This method helps in filtering and validating the request parameters before submission. ```APIDOC ## datastores.Collection.apply_constraints ### Description Apply constraints to the parameters in a request. ### Method N/A (This is a class method) ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **request** (dict[str,Any]) - Required - Request parameters. ### Request Example ```json { "request": { "param1": "value1", "param2": "value2" } } ``` ### Response #### Success Response (200) - **Valid Parameters** (dict[str,Any]) - Dictionary of valid values after applying constraints. #### Response Example ```json { "valid_param1": "processed_value1", "valid_param2": "processed_value2" } ``` ```