### Extract Tasktable Description with Python

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This Python snippet demonstrates how to use the `EOExtractRoiUnit` class from `eopf.computing.utils` to get available modes, the default mode, and the tasktable description for the default mode. It requires the `eopf` library to be installed.

```Python
from eopf.computing.utils import EOExtractRoiUnit

modes = EOExtractRoiUnit.get_available_modes()
default_mode = EOExtractRoiUnit.get_default_mode()
tasktable = EOExtractRoiUnit.get_tasktable_description(default_mode)
```

--------------------------------

### Install eopf-cpm with Multiple Optional Dependencies

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

This example demonstrates how to combine multiple optional dependency tags during installation. By listing tags separated by commas within the brackets, users can install dependencies for various functionalities simultaneously.

```bash
$ pip install eopf[doc,notebook] --extra-index-url --no-cache-dir
```

--------------------------------

### Request Tasktable Description via CLI

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This Bash command shows how to use the `eopf trigger tasktable` command-line tool to request the tasktable description for a specified module and processing unit. This is a direct execution example for the `eopf` command-line interface.

```Bash
eopf trigger tasktable "eopf.computing.utils" EOExtractRoiUnit
```

--------------------------------

### Launch Python Interactive Console

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

This command starts a Python interactive console directly in your terminal. It's a useful first step to verify the installation of Python libraries by attempting to import them.

```bash
$ python
```

--------------------------------

### Install Pre-commit Hooks

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/contributing

This command installs the Git pre-commit hooks defined in the `.pre-commit-config.yaml` file. It ensures that linting and formatting checks are run automatically before commits, reducing issues during pull requests. This should be run after installing and activating your Python environment.

```Shell
pre-commit install
```

--------------------------------

### Register mapping folder using setup.cfg (setuptools)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/store-developer-guide/mapping_file_registering

Example of adding a new mapping folder entry point in `setup.cfg` using the setuptools format. This method statically registers the mapping location for `eopf-cpm` at project setup.

```TOML
[options.entry_points.eopf.store]
mapping_folder = "path.to.my.mapping.folder"
```

--------------------------------

### Install eopf as Development Package

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/contributing

This command installs the `eopf` project in editable mode, including all development dependencies specified by the `[dev]` extra. It should be executed at the root level of the `eopf-cpm` project after activating your virtual environment.

```Shell
pip install -e ".[dev]"
```

--------------------------------

### Example Configuration for EOSafeInit

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/api/eopf.store

Illustrates the structure of a configuration dictionary used to define product recognition, initialization, and finalization functions within the EOPF framework. This specific example shows a mapping for a 'test_product.SEN3' product type, intended for use with the EOSafeInit class.

```JSON
{
    "recognition": {
        "filename_pattern": "test_product.SEN3",
        "product_type": "new_product_type"
    },
    "init_function": {
        "module": "tests.store.fake_init_function",
        "class": "default_init_function"
    },
    "finalize_function": {
        "module": "tests.store.fake_init_function",
        "function": "default_finalize_function"
    },
    "data_mapping": []
}
```

--------------------------------

### Create Python Virtual Environment with Venv

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

These bash commands guide the user through creating a dedicated Python virtual environment using `venv`, navigating to a desired folder, creating the environment, and activating it for isolated project dependencies.

```bash
$ cd path/to/folder
$ python |python_supported_version| -m venv cpm_env
$ source cpm_env/bin/activate
```

--------------------------------

### List Available eopf-cpm Versions via Pip

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

This command queries PyPI to display all available versions of the eopf-cpm package, helping users identify specific releases for installation.

```bash
pip install eopf== --no-cache-dir
```

--------------------------------

### Example Legacy Sentinel Product Filenames

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/eoproduct-user-guide/supported_formats

This list provides example filenames for legacy Sentinel-1, Sentinel-2, and Sentinel-3 products that are actively used for testing. These examples illustrate the naming conventions and types of product versions currently supported.

```Text
├── Sentinel-1
│   ├── S1A_EW_RAW__0SDH_20221111T114657_20221111T114758_045846_057C1E_9592.SAFE
│   ├── S1A_EW_RAW__0SDH_20230103T225516_20230103T225554_046625_059698_E497.SAFE
│   ├── S1A_IW_OCN__2SDV_20220412T054447_20220412T054512_042736_051989_2811.SAFE
│   ├── S1A_IW_RAW__0SDH_20220406T130116_20220406T130148_042653_0516C6_6FA3.SAFE
│   ├── S1A_S3_OCN__2SSH_20220708T000110_20220708T000129_044001_05409A_D34C.SAFE
│   └── S1A_S3_RAW__0SDV_20230103T171935_20230103T172005_046622_059673_0753.SAFE
├── Sentinel-2
│   ├── S2A_MSIL1C_20220306T100031_N0400_R122_T32TQQ_20220306T105713.SAFE
│   ├── S2A_MSIL2A_20220614T081611_N0400_R121_T36RVP_20220614T120009.SAFE
│   ├── S2B_MSIL1C_20220721T085559_N0400_R007_T35UPP_20220721T094331.SAFE
│   └── S2B_OPER_MSI_L0__DS_2BPS_20221223T230531_S20221223T220352_N05.09
└── Sentinel-3
    ├── S3A_DO_0_DOP____20240110T112733_20240110T130725_20240110T132417_5992_107_351______PS1_O_NR_004.SEN3
    ├── S3A_DO_0_NAV____20240112T053153_20240112T071333_20240112T072942_6100_107_376______PS1_O_AL_004.SEN3
    ├── S3A_GN_0_GNS____20240110T112732_20240110T130725_20240110T132417_5993_107_351______PS1_O_NR_004.SEN3
    ├── S3A_MW_0_MWR____20221101T092440_20221101T110518_20221101T112141_6037_091_307______PS1_O_NR_004.SEN3
    ├── S3A_OL_0_CR0____20220511T202328_20220511T202413_20220511T213302_0045_085_142______PS1_O_NR_002.SEN3
    ├── S3A_OL_0_EFR____20210629T044945_20210629T045145_20210629T061945_0119_073_247______LN1_O_NR_002.SEN3
    ├── S3A_OL_0_EFR____20221101T162118_20221101T162318_20221101T180111_0119_091_311______PS1_O_NR_002.SEN3
    ├── S3A_SL_0_SLT____20221101T204936_20221101T205436_20221101T212249_0299_091_314______PS1_O_NR_004.SEN3
    ├── S3A_SL_2_FRP____20231221T213647_20231221T213947_20231223T071844_0180_107_072_0540_PS1_O_NT_004.SEN3
    ├── S3A_SR_0_CAL____20240402T010212_20240402T010227_20240402T022326_0014_110_373______PS1_O_NR_004.SEN3
    ├── S3A_SR_0_SRA____20240402T000237_20240402T001237_20240402T004211_0599_110_372______PS1_O_NR_004.SEN3
    ├── S3A_SY_2_AOD____20191227T124211_20191227T124311_20230616T170045_0060_053_109______PS1_D_NR_002.SEN3
    ├── S3A_SY_2_SYN____20220607T043506_20220607T043806_20220607T155527_0179_086_133_1620_PS1_O_ST_002.SEN3
    ├── S3A_SY_2_V10____20231221T000000_20231231T235959_20240102T232539_EUROPE____________PS1_O_NT_002.SEN3
    ├── S3A_SY_2_VG1____20231221T000000_20231221T235959_20231223T201855_EUROPE____________PS1_O_NT_002.SEN3
    ├── S3A_SY_2_VGK____20191227T124211_20191227T124311_20230403T130523_0059_053_109______PS1_D_NR_002.SEN3
    ├── S3A_SY_2_VGP____20191227T124211_20191227T124311_20230403T130523_0059_053_109______PS1_D_NR_002.SEN3
    ├── S3A_TM_0_HKM____20231220T002042_20231220T020305_20231220T021924_6143_107_045______PS1_O_NR_004.SEN3
    ├── S3A_TM_0_HKM2___20231220T002043_20231220T020305_20231220T021924_6142_107_045______PS1_O_NR_004.SEN3
    ├── S3A_TM_0_NAT____20231220T020306_20231220T034540_20231220T040132_6154_107_046______PS1_O_AL_004.SEN3
    ├── S3B_OL_0_CR1____20180921T064333_20180921T081948_20180921T091503_5775_013_063______LN1_O_NR_002.SEN3
    ├── S3B_OL_1_EFR____20230506T015316_20230506T015616_20230711T065802_0180_079_117______LR1_D_NR_003.SEN3
    ├── S3B_OL_1_ERR____20230506T015316_20230506T015616_20230711T065804_0179_079_117______LR1_D_NR_003.SEN3
    ├── S3B_OL_2_LFR____20230724T155519_20230724T155538_20230725T103722_0019_082_097_1080_PS2_O_NT_002.SEN3
    ├── S3B_SL_1_RBT____20230315T095847_20230315T100147_20230316T030042_0179_077_150_4320_PS2_O_NT_004.SEN3
    └── S3B_SL_2_LST____20230822T052122_20230822T052422_20230822T072255_0180_083_119_2700_PS2_O_NR_004.SEN3
```

--------------------------------

### Example Configuration for EOSafeFinalize

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/api/eopf.store

Illustrates the structure of a configuration dictionary used to define product recognition, initialization, and finalization functions within the EOPF framework. This specific example shows a mapping for a 'test_product.SEN3' product type, intended for use with the EOSafeFinalize class.

```JSON
{
    "recognition": {
        "filename_pattern": "test_product.SEN3",
        "product_type": "new_product_type"
    },
    "init_function": {
        "module": "tests.store.fake_init_function",
        "class": "default_init_function"
    },
    "finalize_function": {
        "module": "tests.store.fake_init_function",
        "function": "default_finalize_function"
    },
    "data_mapping": []
}
```

--------------------------------

### Verify eopf-cpm Installation in Docker

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

This Python snippet demonstrates how to import the eopf library, confirming its successful installation and availability within the Docker container.

```python
import eopf
```

--------------------------------

### Install eopf-cpm with Optional Dependencies by Tag

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

This command allows installing eopf-cpm along with additional, optional dependencies by specifying a tag. Tags correspond to sections in the pyproject.toml file and provide specific sets of libraries for different use cases.

```bash
$ pip install -e ".[YOUR_TAG]" --no-cache-dir
```

--------------------------------

### Verify eopf-cpm Installation with EOProduct Import

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/quickstart/installation

After installing eopf-cpm, this Python snippet verifies the successful installation by attempting to import the EOProduct class. If no errors occur, the library is confirmed to be correctly set up and accessible.

```python
from eopf.product import EOProduct
```

--------------------------------

### Install eopf-cpm from GitLab Repository

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/quickstart/installation

Instructions to clone the `eopf-cpm` GitLab repository and install the library in editable mode. This method is suitable for contributors or those needing specific development versions, including installing optional dependencies.

```Bash
git clone https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm
```

```Bash
cd path/to/the/cloned/eopf-cpm/repo
```

```Bash
source cpm_env/bin/activate
```

```Bash
pip install -e . --no-cache-dir
```

```Bash
pip install -e ".[YOUR_TAG]" --no-cache-dir
```

```Bash
pip install -e ".[dev]" --no-cache-dir
```

```Bash
pip install -e ".[notebook]" --no-cache-dir
```

```Bash
pip install eopf[doc,notebook] --extra-index-url --no-cache-dir
```

--------------------------------

### Install eopf-cpm for Development (Contributor Tag)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

This specific installation command is intended for contributors to eopf-cpm. The '[dev]' tag installs a comprehensive set of additional dependencies required for development, testing, and code quality checks.

```bash
$ pip install -e ".[dev]" --no-cache-dir
```

--------------------------------

### Verify eopf-cpm Installation by Importing EOProduct

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

After launching the Python console, execute this Python code to import the EOProduct class. A successful import without errors indicates that the eopf-cpm library has been installed correctly and is accessible.

```python
from eopf.product import EOProduct
```

--------------------------------

### Install eopf-cpm for Notebook Usage

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/installation

This command installs eopf-cpm with the necessary dependencies to run and interact with its Jupyter notebooks. The '[notebook]' tag ensures all required libraries for notebook functionality are included.

```bash
$ pip install -e ".[notebook]" --no-cache-dir
```

--------------------------------

### Python Code to Load a Safe Product using EOPF Mapping Manager

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/store-developer-guide/mapping_file_samples

This Python example demonstrates how to load a 'safe product' using the EOPF library. It initializes an `EOPFMappingFactory` and `EOPFMappingManager` with a mapping file, then uses `EOSafeStore` to load the product into an `EOProduct` object, highlighting the process of accessing and handling product data, including potential issues with missing coordinate data.

```python
from eopf.common.file_utils import AnyPath
from eopf.store.mapping_factory import EOPFMappingFactory
from eopf.store.mapping_manager import EOPFMappingManager
from eopf.store.safe import EOSafeStore

#add tutorial mapping files to mapping manager
mapping_path = AnyPath("./docs/source/_data/mapping_tutorial/SAMPLE01_S03OLCLFR.json")
mf = EOPFMappingFactory(mapping_path=mapping_path)
mm = EOPFMappingManager(mf)

product_path = AnyPath("./docs/source/_data/mapping_tutorial/SAMPLE01_S3B_OL_2_LFR_VARIANT01.SEN3")
safe_store = EOSafeStore(product_path, mask_and_scale=False, mapping_manager=mm) # legacy store to access a file on the given URL
eop = safe_store.load(name = "SAMPLE01_S03OLCLFR") # create and return the EOProduct
eop
```

--------------------------------

### Initialize EOProduct and import modules

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This snippet demonstrates how to import necessary modules like EOProduct and numpy, then create a new EOProduct instance and set its name.

```Python
# The first thing to do is to import the EOProduct module from eopf.product
from eopf.product import *
import numpy as np

# Create an EOProduct and give it a name
name = "eoproduct.zarr"
new_eoproduct = EOProduct(name = "start_name")
# Change the name of the product
new_eoproduct.name = name
```

--------------------------------

### Write EOProduct to Zarr format

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This snippet shows the initial imports required to prepare for writing an EOProduct to disk in Zarr format using the EOZarrStore. It sets up the necessary modules for file system interaction and Zarr store functionality.

```Python
import os
import os.path as osp

# The first thing to do is importing the desired Store
from eopf.store.zarr import EOZarrStore
import eopf.common.constants
```

--------------------------------

### Import Libraries and Initialize EOPF Configuration

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/eoproduct-user-guide/product_use

This snippet imports essential Python libraries (numpy, os, pathlib) and core EOPF components (EOConfiguration, EOGroup, EOProduct, EOVariable, EOSafeStore, OpeningMode). It then demonstrates how to set up a test data folder, load a configuration file (eopf.toml), and register a logging directory, which are foundational steps for working with EOPF products.

```python
import numpy as np
import os
from pathlib import Path
from eopf.config import EOConfiguration
from eopf.product import EOGroup, EOProduct, EOVariable
from eopf.store.safe import EOSafeStore
from eopf.common.constants import OpeningMode

test_data_folder = (Path(os.getcwd()) / "tests" / "data" ).absolute()
print(test_data_folder)
conf = EOConfiguration()
conf.load_file(str(test_data_folder / "config" / "eopf.toml"))
conf.register_requested_parameter("logging_folder", str(test_data_folder / "logging"))
```

--------------------------------

### Automatically initialize a valid EOProduct

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This snippet demonstrates how to use the 'init_product' convenience function to automatically create a valid EOProduct instance, ensuring it contains all mandatory groups.

```Python
from eopf.product.conveniences import init_product
valid_product: EOProduct = init_product("product_test")
```

--------------------------------

### Basic Logger Usage Examples (Python)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/logging

Provides practical examples of common logging operations using a retrieved logger, including `info()`, `error()`, and `exception()`. It shows how to log messages, include variables, and capture exceptions for debugging and monitoring.

```Python
logger1.info("a random message")

a_parameter = "text of the parameter"
logger1.error(f"a random message with a parameter: {a_parameter}")

try:
    float("xxx")
except Exception as e:
    logger1.exception(f"Exception {e} encountered when converting to float")
```

--------------------------------

### Open a legacy SAFE EOProduct

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This snippet shows how to open a legacy Earth Observation product in SAFE format using EOSafeStore. It loads the product from a specified path, allowing it to be read and processed, including displaying its tree structure. It also uses os.path.join for path construction.

```Python
from eopf.product import EOProduct
from eopf.store.safe import EOSafeStore
import eopf.common.constants
import os # Required for os.path.join

# input is a product in zarr format on local disk
input_product_id = "new_product" # select a name for your product
path_to_safe  = os.path.join(os.getcwd(), 'docs', \
'source', '_data', 'S3B_OL_2_LFR____20230724T155519_20230724T155538_20230725T103722_0019_082_097_1080_PS2_O_NT_002.SEN3')

with EOSafeStore(url=path_to_safe).open(
    mode=eopf.common.constants.OpeningMode.OPEN
) as st:
    safe_eoproduct = st.load()
    # You can now read and use you product
    safe_eoproduct.tree()
```

--------------------------------

### Example Output: EOPF CPM Optional Configuration Parameters

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/conf-params

This snippet presents a structured example of the optional configuration parameters retrieved from the EOPF CPM system. Each entry details a parameter's default value, a brief description, its full name, and its optional status, offering a comprehensive overview of configurable settings.

```python
{'accessors.memmap.packet_cache_size': {'default': 1024,
                                        'description': 'Number of packet kept in cache for dask delayed',
                                        'name': 'accessors.memmap.packet_cache_size',
                                        'optional': true},
 'breakpoints.activate_all': {'default': false,
                              'description': 'Global ON/OFF switch for the breakpoints',
                              'name': 'breakpoints.activate_all',
                              'optional': true},
 'breakpoints.folder': {'default': '/builds/cpm/eopf-cpm/breakpoints',
                        'description': 'Folder to put breakpoints in',
                        'name': 'breakpoints.folder',
                        'optional': true},
 'breakpoints.storage_options': {'default': null,
                                 'description': 'storage options in case of S3',
                                 'name': 'breakpoints.storage_options',
                                 'optional': true},
 'dask_utils.compute.step': {'default': 9999,
                             'description': 'Number of dask future computed simultaneously in dask_utils',
                                            'name': 'dask_utils.compute.step',
                                            'optional': true},
 'dask_utils.retries': {'default': 3,
                        'description': 'Default number of retries',
                        'name': 'dask_utils.retries',
                        'optional': true},
 'dask_utils.timeout': {'default': 300,
                        'description': 'Default timeout on a submitted task',
                        'name': 'dask_utils.timeout',
                        'optional': true},
 'logging.folder': {'default': PosixPath('/builds/cpm/eopf-cpm/eopf/logging/conf'),
                    'description': 'Folder to look for json/yaml logging dictConfig files ',
                    'name': 'logging.folder',
                    'optional': true},
 'logging.level': {'default': 'INFO',
                   'description': 'Default logging level',
                   'name': 'logging.level',
                   'optional': true},
 'logging.obfuscate': {'default': true,
                       'description': 'Obfuscate or not the sensitive elements : password;usernam etc',
                       'name': 'logging.obfuscate',
                       'optional': true},
 'mapping.folder': {'default': 'store/mapping/',
                    'description': 'path to the folder where mappings are stored',
                    'name': 'mapping.folder',
                    'optional': true},
 'model.folder': {'default': '/builds/cpm/eopf-cpm/eopf/product/models',
                  'description': 'path to the folder where pydantic models are stored',
                  'name': 'model.folder',
                  'optional': true},
 'product.create_mask': {'default': false,
                         'description': 'Activate mask creation',
                         'name': 'product.create_mask',
                         'optional': true},
 'product.mask_and_scale': {'default': true,
                            'description': 'Activate default mask and scale on product read/write',
                            'name': 'product.mask_and_scale',
                            'optional': true},
 'qualitycontrol.folder': {'default': '/builds/cpm/eopf-cpm/eopf/qualitycontrol/config/',
                           'description': 'path to the folder where checklist *.json and inspections *_inspections.json are stored',
                           'name': 'qualitycontrol.folder',
                           'optional': true},
 'store.convert.use_multithreading': {'default': true,
                                      'description': 'Activate Dask LocalCluster if no Dask client detected',
                                      'name': 'store.convert.use_multithreading',
                                      'optional': true},
 'temporary.folder': {'default': '/tmp',
                      'description': 'Folder create the temporary folder in',
```

--------------------------------

### Get Mandatory Logging Parameters

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/api/eopf.config

Example demonstrating how to retrieve only the mandatory parameters related to logging using the `mandatory_list` method of the EOConfiguration class.

```Python
EOConfiguration().mandatory_list("logging")
```

--------------------------------

### Open an existing Zarr EOProduct

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This code demonstrates how to open an existing Earth Observation product stored in Zarr format. It specifies the product's URL and uses EOZarrStore with OpeningMode.OPEN to load the product for reading and further processing, including displaying its tree structure.

```Python
from eopf.product import EOProduct
from eopf.store.zarr import EOZarrStore
import eopf.common.constants

# input is a product in zarr format on local disk
input_product_id = "new_product" # select a name for your product
input_dir = DIR_TO_WRITE_YOUR_PRODUCT / 'new_zarr_product.zarr'

with EOZarrStore(url=input_dir).open(
    mode=eopf.common.constants.OpeningMode.OPEN
) as st:
    new_eoproduct = st.load(input_product_id)
    # You can now read and use you product
    new_eoproduct.tree()
```

--------------------------------

### Create and Open EOProduct with EOSafeStore in Python

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/developer-guide/store-developer-guide/mapping_accessor_tutorial

This snippet demonstrates how to initialize an EOProduct and then open it using an EOSafeStore. It requires a URL to the safe product and a mapping manager to correctly load the product data, enabling the integration of custom accessors and mapping files.

```python
product = eopf.EOProduct(
"test",

)
product = eopf.EOSafeStore(url=url_to_safe_product,mapping_manager=my_mapping_manager,).open().load("test")
```

--------------------------------

### Setup basic eopf-cpm logging configuration

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/logging

Configures a basic logging setup for eopf-cpm with a specified logging level (e.g., INFO) and an optional output file. This provides a quick way to get logging operational without requiring complex external configuration files, directing logs to a file or standard output.

```python
from logging import INFO
EOLogging().setup_basic_config(level=INFO, format=None, file="default_log_file.log")
```

--------------------------------

### Shell Example: Visualize Profiling Stats with Snakeviz

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/api/eopf.tracing

Provides commands to install Snakeviz and use it within an IPython environment to visualize profiling statistics generated by `single_thread_profiler`.

```Shell
pip install snakeviz
%load_ext snakeviz
%snakeviz stats
```

--------------------------------

### Get path suffix (Python AnyPath Property)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/api/eopf.common

Returns the shortest extension (suffix) of the path. For example, '.txt' for 'file.txt'. This property is part of the `AnyPath` class.

```APIDOC
AnyPath.suffix: str
  Returns: str - The shortest extension of the path.
```

--------------------------------

### Access and navigate EOProduct structure

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This snippet shows how to print the 'new_eoproduct' object to inspect its overall structure and content.

```Python
new_eoproduct
```

--------------------------------

### Define essential imports and input paths for eopf-cpm development

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/developer-guide/store-developer-guide/mapping_accessor_tutorial

This Python code snippet imports core modules required for working with `eopf-cpm`, including `os`, `logging`, `netCDF4`, and various `eopf` components like `EOConfiguration`, `EOLogging`, `EOVariable`, `EOPFMappingFactory`, `EOAccessor`, and `EOReadOnlyAccessor`. It also sets up a base directory path for tutorial input files, preparing the environment for subsequent development steps.

```python
import os
import os.path as osp
import logging
from netCDF4 import Dataset
from typing import TYPE_CHECKING, Any, Iterator, Optional

# eopf-cpm modules
import eopf
from eopf.config.config import EOConfiguration
from eopf.logging.log import EOLogging
from eopf.product.eo_variable import EOVariable
from eopf.store.mapping_factory import EOPFMappingFactory
from eopf.store.mapping_manager import EOPFMappingManager
from eopf.accessor import EOAccessor, EOAccessorFactory
from eopf.common.constants import OpeningMode
from eopf.formatting.decorators import (
    formatable_method,
    reverse_formatable_method,
    unformatable_method,
)
from eopf.accessor.abstract import EOReadOnlyAccessor
from eopf.exceptions import AccessorNotOpenError

if TYPE_CHECKING:  # pragma: no cover
    from eopf.product.eo_object import EOObject

# Define input path
path_to_mapping_tutorial = osp.abspath(os.path.join(os.getcwd(), 'docs', 'source', \
                               'developer-guide', 'store-developer-guide', 'tutorial_inputs'))
```

--------------------------------

### Start Python interpreter and import eopf-cpm inside Docker

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/quickstart/installation

Commands to enter the Python interactive shell within the Docker container and then import the eopf-cpm library, demonstrating its availability.

```Shell
python
```

```Python
import eopf
```

--------------------------------

### Get average Julian day from UTC time range (Python APIDOC)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/api/eopf.common

Calculates the average Julian day from a start and stop UTC time string.

```APIDOC
eopf.common.date_utils.get_average_julian_day(ptimestartutc: str, ptimestoputc: str)
Returns:
  float
```

--------------------------------

### Example Product File Structure for eopf-cpm

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/developer-guide/store-developer-guide/mapping_accessor_tutorial

Illustrates the directory and file structure of a "test_product.SEN3" product, including its XML manifest and NetCDF data files, intended for integration with the eopf-cpm platform.

```Plaintext
test_product.SEN3
├── xfdumanifest.xml
├── tie_meteo.nc
└── tie_geo_coordinates.nc
```

--------------------------------

### Get path suffixes (Python AnyPath Property)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/api/eopf.common

Returns a list of all extensions (suffixes) of the path. For example, ['.tar', '.gz'] for 'archive.tar.gz'. This property is part of the `AnyPath` class.

```APIDOC
AnyPath.suffixes: list[str]
  Returns: list[str] - A list of all extensions of the path.
```

--------------------------------

### Open and Explore Existing Zarr EOContainer in Python

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This snippet demonstrates how to open an existing Zarr container from disk. It specifies the container's URL, opens it in OPEN mode, loads the container, displays its tree structure, and iterates through its contained products.

```python
from eopf.product import EOProduct
from eopf.store.zarr import EOZarrStore
import eopf.common.constants

# input is a container in zarr format on local disk
input_container_id = "new_container" # select a name for your product
input_container_dir = osp.join(DIR_TO_WRITE_YOUR_PRODUCT, 'new_zarr_container.zarr')

with EOZarrStore(url=input_container_dir).open(
    mode=eopf.common.constants.OpeningMode.OPEN
) as st:
    new_eocontainer = st.load(input_container_id)
    # You can now read and use you product
    new_eocontainer.tree()
    # you can print the available products in this container
    for p in new_eocontainer:
       print(f"Available product: {p}")
```

--------------------------------

### Configure Pytest with TEST_OUTPUT_FOLDER

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/contributing

Examples demonstrating how to set the TEST_OUTPUT_FOLDER environment variable to define where test output data should be written. If this variable is not set, a temporary folder is used by default for test outputs.

```Shell
TEST_OUTPUT_FOLDER=/tmp/ouput_test pytest tests -m unit
```

```Shell
TEST_OUTPUT_FOLDER=.output_dir/ pytest tests
```

```Shell
TEST_OUTPUT_FOLDER=data_out/ pytest tests -m integration
```

--------------------------------

### Configure Pytest with Both Data and Output Folders

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/contributing

Examples showing how to configure both TEST_DATA_FOLDER and TEST_OUTPUT_FOLDER environment variables concurrently for pytest execution. This provides full control over both data input and output locations for tests.

```Shell
TEST_OUTPUT_FOLDER=/tmp/output_data TEST_DATA_FOLDER=local_data/ pytest tests -m unit
```

```Shell
TEST_OUTPUT_FOLDER=output_dir/ TEST_DATA_FOLDER=.data/ pytest tests
```

```Shell
TEST_OUTPUT_FOLDER=.data_out/ TEST_DATA_FOLDER=/home/user/data/ pytest tests -m integration
```

--------------------------------

### Define Imports and Input Path for eopf-cpm Tutorial

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/store-developer-guide/mapping_accessor_tutorial

This Python snippet imports necessary modules from the standard library (os, logging, netCDF4, typing) and the eopf-cpm framework, including components for configuration, logging, product handling, store management, and accessors. It also defines a base path for tutorial input files.

```Python
import os
import os.path as osp
import logging
from netCDF4 import Dataset
from typing import TYPE_CHECKING, Any, Iterator, Optional

# eopf-cpm modules
import eopf
from eopf.config.config import EOConfiguration
from eopf.logging.log import EOLogging
from eopf.product.eo_variable import EOVariable
from eopf.store.mapping_factory import EOPFMappingFactory
from eopf.store.mapping_manager import EOPFMappingManager
from eopf.accessor import EOAccessor, EOAccessorFactory
from eopf.common.constants import OpeningMode
from eopf.formatting.decorators import (
    formatable_method,
    reverse_formatable_method,
    unformatable_method,
)
from eopf.accessor.abstract import EOReadOnlyAccessor
from eopf.exceptions import AccessorNotOpenError

if TYPE_CHECKING:  # pragma: no cover
    from eopf.product.eo_object import EOObject

# Define input path
path_to_mapping_tutorial = osp.abspath(os.path.join(os.getcwd(), 'docs', 'source', \
	                           'developer-guide', 'store-developer-guide', 'tutorial_inputs'))
```

--------------------------------

### Configure Pytest with TEST_DATA_FOLDER

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/contributing

Examples demonstrating how to set the TEST_DATA_FOLDER environment variable to specify the location of external test data for pytest runs. This allows tests to access specific datasets from a custom path.

```Shell
TEST_DATA_FOLDER=local_data/ pytest tests -m unit
```

```Shell
TEST_DATA_FOLDER=.data/ pytest tests
```

```Shell
TEST_DATA_FOLDER=/home/user/data/ pytest tests -m integration
```

--------------------------------

### Creating and Opening EOProduct with Custom Accessor

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/store-developer-guide/mapping_accessor_tutorial

Shows how to create and open an EOProduct using a custom accessor and mapping file. It initializes an EOSafeStore with the product URL and mapping manager, then loads the product, integrating the custom accessor's capabilities.

```Python
url_to_safe_product = osp.join(
    path_to_mapping_tutorial,
    "test_product.SEN3",
)

product = eopf.EOProduct(
"test",

)
product = eopf.EOSafeStore(url=url_to_safe_product,mapping_manager=my_mapping_manager,).open().load("test")
```

--------------------------------

### Run Pytest with Default Data and Output Settings

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/contributing

Examples of running pytest without explicitly setting TEST_DATA_FOLDER or TEST_OUTPUT_FOLDER. In this scenario, test data is expected in the default 'data/' folder, and output is directed to a temporary directory.

```Shell
pytest tests -m unit
```

```Shell
pytest tests
```

```Shell
pytest tests -m integration
```

--------------------------------

### Minimal JSON Mapping File for Legacy Product Variables

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/store-developer-guide/mapping_file_samples

This JSON configuration defines how to map variables from a legacy product. It includes filename patterns for product recognition, and data mapping rules for 'otci' and other metadata, specifying source/target paths, accessor IDs, and optional accessor configurations.

```json
{
    "recognition": {
        "filename_pattern": "SAMPLE01_S3._OL_2_LFR.*SEN3|SAMPLE01_S03OLCLFR.*",
        "product_type": "SAMPLE01_S03OLCLFR",
        "processing_version": "0.0"
    },
    "data_mapping": [
        {
            "short_name": "otci",
            "target_path": "/measurements/otci",
            "source_path": "otci.nc:OTCI",
            "accessor_id": "netcdf"
        },
        {
            "target_path": "attrs:/:other_metadata",
            "source_path": "xfdumanifest.xml",
            "accessor_id": "xmlmetadata",
            "accessor_config": {
                "mapping": {"eopf_category": "Text(eoproduct)"},
                "namespaces": {},
                "path_template": {}
            }
        }
    ]
}
```

--------------------------------

### Open an Existing Zarr Product with EOPF

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/quickstart/start-with-code

Demonstrates how to open an existing Zarr product from a local disk using `EOZarrStore` in read mode. It shows how to load the product by its ID and inspect its structure using `new_eoproduct.tree()`, which outputs a hierarchical representation of the product's contents.

```Python
from eopf.product import EOProduct
from eopf.store.zarr import EOZarrStore
import eopf.common.constants

# input is a product in zarr format on local disk
input_product_id = "new_product" # select a name for your product
input_dir = DIR_TO_WRITE_YOUR_PRODUCT / 'new_zarr_product.zarr'

with EOZarrStore(url=input_dir).open(
    mode=eopf.common.constants.OpeningMode.OPEN
) as st:
    new_eoproduct = st.load(input_product_id)
    # You can now read and use you product
    new_eoproduct.tree()
```

--------------------------------

### Register mapping folder using pyproject.toml (flit)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/developer-guide/store-developer-guide/mapping_file_registering

Example of adding a new mapping folder entry point in `pyproject.toml` using the flit format. This method statically registers the mapping location for `eopf-cpm` at project setup.

```TOML
[project.entry-points.store.mapping_folder]
newfoldername = path.to.my.mapping.folder
```

--------------------------------

### Open EOProduct from Zarr Container in Python

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This code shows how to directly open an EOProduct that is part of a larger Zarr container. It constructs the product's path within the container, opens the store, loads the specific product, and displays its tree structure.

```python
# Opening an EOProduct written on disk which is part of a container
input_product_dir = osp.join(DIR_TO_WRITE_YOUR_PRODUCT, 'new_zarr_container.zarr', "new_product")

with EOZarrStore(url=input_product_dir).open(
    mode=eopf.common.constants.OpeningMode.OPEN
) as st:
    # name of the folder without .zarr extension
    new_product = st.load('product_inside_container')
    # You can now read and use you product
    new_product.tree()
```

--------------------------------

### Write a new EOProduct to Zarr

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This snippet demonstrates how to create a temporary directory and write a new Earth Observation product (EOProduct) in Zarr format. It utilizes EOTemporaryFolder for temporary storage and EOZarrStore with CREATE_OVERWRITE mode to manage the product writing process, ensuring automatic cleanup and proper context management.

```Python
from eopf.common.file_utils import AnyPath
from eopf.common.temp_utils import EOTemporaryFolder
from eopf.store.zarr import EOZarrStore
import eopf.common.constants

# Get a temporary folder to work in
# It will be automatically deleted at the end of execution
DIR_TO_WRITE_YOUR_PRODUCT = EOTemporaryFolder().get()
# write an EOProduct in zarr format
# Edit url with an existing path to make it work
# Edit the opening mode
with EOZarrStore(url=DIR_TO_WRITE_YOUR_PRODUCT).open(
    mode=eopf.common.constants.OpeningMode.CREATE_OVERWRITE
) as st:
    # Actually write the product to the store in DIR_TO_WRITE_YOUR_PRODUCT/new_zarr_product.zarr
    st["new_zarr_product"] = new_eoproduct
    # the store will be automatically closed as it is a context manager
    # but you can manually use st.close()
```

--------------------------------

### Run All Pre-commit Hooks on All Files

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/contributing

This command executes all defined pre-commit hooks on every file in the repository, regardless of whether they are staged for commit. It's useful for performing a full codebase check or for initial setup to ensure all files conform to quality standards.

```Shell
pre-commit run --all-files
```

--------------------------------

### Retrieve TaskTable Description Programmatically (Python)

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/processor-orchestration-guide/tasktables

This Python example demonstrates how to use the eopf.computing.utils module to programmatically access and retrieve information about a processing unit's tasktable. It shows how to get available modes, the default mode, and the full tasktable description for a given unit like EOExtractRoiUnit.

```python
from eopf.computing.utils import EOExtractRoiUnit

modes = EOExtractRoiUnit.get_available_modes()
default_mode = EOExtractRoiUnit.get_default_mode()
tasktable = EOExtractRoiUnit.get_tasktable_description(default_mode)
```

--------------------------------

### Install eopf-cpm Python Package with Pip

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/quickstart/installation

Commands to install the `eopf-cpm` library from PyPI using pip. This includes installing the latest version, checking available versions, and installing a specific version.

```Bash
pip install eopf --no-cache-dir
```

```Bash
pip install eopf== --no-cache-dir
```

```Bash
pip install eopf==1.5.2 --no-cache-dir
```

--------------------------------

### Initialize a new EOProduct instance in Python

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/quickstart/start-with-code

This snippet demonstrates how to import the necessary `eopf.product` module and `numpy`, then create an initial `EOProduct` object. It shows how to assign a name to the product after instantiation, setting up the basic container for Earth Observation data.

```Python
# The first thing to do is to import the EOProduct module from eopf.product
from eopf.product import *
import numpy as np

# Create an EOProduct and give it a name
name = "eoproduct.zarr"
new_eoproduct = EOProduct(name = "start_name")
# Change the name of the product
new_eoproduct.name = name
```

--------------------------------

### Append data to an existing Zarr EOProduct

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/_sources/quickstart/start-with-code

This example demonstrates how to open an existing Zarr product in append mode. This mode allows appending new data along the first dimension of a variable, such as adding more 'radiance' data. A warning notes that appending occurs on the first dimension of the variable.

```Python
from eopf.product import EOProduct
from eopf.store.zarr import EOZarrStore
import eopf.common.constants
import numpy as np # Required for np.random.randint
# Assuming EOGroup is imported or available in scope, e.g., from eopf.product.core import EOGroup

# input is a product in zarr format on local disk
input_product_id = "new_product" # select a name for your product
input_dir = DIR_TO_WRITE_YOUR_PRODUCT / 'new_zarr_product.zarr'

with EOZarrStore(url=input_dir).open(
    mode=eopf.common.constants.OpeningMode.APPEND
) as st:
   st['measurements/image/radiance'] = EOGroup("radiance", variables = { "sensor1" : np.random.randint(0,100,(1024,1024))})
```

--------------------------------

### Apply multiple profilers (single-thread and Dask) to a Python function

Source: https://cpm.pages.eopf.copernicus.eu/eopf-cpm/main/developer-guide/additional-modules/tracing

This Python example illustrates how to simultaneously apply both `single_thread_profiler` and `dask_profiler` decorators to a function. It's crucial that `single_thread_profiler` is applied as the outermost decorator. This setup allows for generating separate performance reports for both single-threaded operations and Dask-specific computations within the same function call.

```Python
@single_thread_profiler(report_name="stats_report.dump")
@dask_profiler(report_name="dask_report.html")
def s3_to_nc():
    ...

stats = s3_to_nc()
```