### ImporterExample Start Method Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_examples.rst The start method initializes the ImporterExample, setting up database tables for input elements. It defines the initial setup for the importer process. ```python def start(self) -> None: self.database.create_tables([InputElement]) ``` -------------------------------- ### Logging Setup Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/genindex Information about the logging setup utility within the MAFW execution scripts. ```APIDOC ## Logging Setup ### Description Utility function for configuring the logging system in MAFW execution scripts. ### Methods - **logger_setup()** (in module `mafw.scripts.mafw_exe`) ``` -------------------------------- ### MySQL Configuration and Password File Example Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/doc_database Demonstrates how to configure MySQL connection details within a configuration file and provides an example of a MySQL password file format. The configuration includes server URL, username, and a reference to a password file. ```ini [DBConfiguration] # Update the database server and the database name to reflect your configuration URL = "mysql://database-server:3306/database-name" # update to specify your username user = 'username' # update to specify the password file read_default_file = '~/.my.cnf' # if you want, you can leave the pragmas section from the SQLite default configuration because it # want be used. ``` ```ini [client] user=username password=password host=database-server ``` -------------------------------- ### MySQL Password File Example Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_database.rst This snippet provides an example of a MySQL client configuration file (e.g., ~/.my.cnf) for storing database credentials. It includes the user, password, and host information for secure connection. ```ini [client] user=username password=password host=database-server ``` -------------------------------- ### start() Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/generated/mafw.processor_library.importer Initializes the processor by creating the filename parser using the provided configuration. ```APIDOC ## start() ### Description The start method. The filename parser is created using the provided configuration file. ### Method POST ### Raises **ParserConfigurationError** – If the configuration file is not valid. ``` -------------------------------- ### Implement Importer start() Method Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/doc_examples The start method ensures the target database table exists by creating it if necessary. It calls the superclass's start method and then uses `self.database.create_tables()` to ensure the `InputElement` table is ready for data insertion. ```python def start(self) -> None: """ The start method. The filename parser is ready to use because it has been already configured in the super method. We need to be sure that the input table exists, otherwise we create it from scratch. """ super().start() self.database.create_tables([InputElement]) ``` -------------------------------- ### Install MAFW Plugin (Linux/MacOS) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_tutorial.rst Installs a MAFW plugin in a Python virtual environment on Linux or MacOS. It creates a virtual environment, activates it, and then uses pip to install the plugin from a local path. ```bash $ python -m venv my_venv $ source my_venv/bin/activate (my_venv) $ pip install -e /path/to/plug ``` -------------------------------- ### Install MAFW Plugin (Windows) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_tutorial.rst Installs a MAFW plugin in a Python virtual environment on Windows. It activates the virtual environment and then uses pip to install the plugin from a local path. ```doscon c:\my_venv> bin\activate (my_venv) c:\my_venv> pip install -e c:\path\to\plug ``` -------------------------------- ### Python: Implement Processor Workflow Methods (start, process, finish) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/doc_examples Demonstrates the core methods for a processor workflow: start, process, and finish. The 'start' method initializes the prime number list, 'process' checks for primality and appends to the list, and 'finish' logs summary information. It requires a 'is_prime' function and a 'log' object. ```python def start(self) -> None: """ Overload of the start method. **Remember:** to call the super method when you overload the start. In this specific case, we just make sure that the list of found prime numbers is empty. """ super().start() self.prime_num_found = [] def process(self) -> None: """ The process method. In this case, it is very simple. We check if :attr:`.Processor.item` is a prime number, if so we added to the list, otherwise we let the loop continue. """ if is_prime(self.item): self.prime_num_found.append(self.item) def finish(self) -> None: """ Overload of the finish method. **Remember:** to call the super method when you overload the finish method. In this case, we just print out some information about the prime number found in the range. """ super().finish() log.info( 'Found %s prime numbers in the range from %s to %s' % (len(self.prime_num_found), self.start_from, self.stop_at) ) if len(self.prime_num_found): log.info('The smallest is %s', self.prime_num_found[0]) log.info('The largest is %s', self.prime_num_found[-1]) ``` -------------------------------- ### Processor Start Method Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/processor The start method initializes the processor's status to 'Start' and performs an initial cleanup by removing orphan files. This method can be overloaded by users to include custom initialization steps, but should call the super method. ```python def start(self) -> None: """ Start method. The user can overload this method, including all steps that should be performed at the beginning of the operation. If the user decides to overload it, it should include a call to the super method. """ self.processor_status = ProcessorStatus.Start self._remove_orphan_files() ``` -------------------------------- ### Windows Environment Setup Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_tutorial.rst Commands to set up a Python virtual environment on Windows for running the MAFw analysis. ```doscon c:\\> python -m venv my_venv c:\\> cd my_venv ``` -------------------------------- ### Start Import Process Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/processor_library/importer Initializes the filename parser using a specified configuration file and starts the import process. It includes type checking for the parser configuration and raises a `ParserConfigurationError` if the configuration is invalid. ```python def start(self) -> None: """ The start method. The filename parser is created using the provided configuration file. :raise ParserConfigurationError: If the configuration file is not valid. """ super().start() if TYPE_CHECKING: assert isinstance(self.parser_configuration, str) self._filename_parser = FilenameParser(self.parser_configuration) ``` -------------------------------- ### Install Pre-commit Hooks Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_getting_started.rst Installs the pre-commit framework, which automatically runs checks before each commit to ensure code quality and style consistency. This command needs to be run once after setting up the development environment. It integrates with various linters and formatters configured in the project. ```doscon (dev.py3.13) D:\mafw> pre-commit install ``` ```bash (dev.py3.13) $ pre-commit install ``` -------------------------------- ### Example MAFw Steering File Configuration (TOML) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_runner.rst An example of a TOML steering file generated by MAFw. This file defines the analysis name, description, processors to run, and their specific configurations. It also includes options for user interface settings. ```toml # MAFw steering file generated on 2024-11-21 16:54:22.455297 # uncomment the line below and insert the processors you want to run from the available processor list # processors_to_run = [] # customise the name of the analysis analysis_name = "mafw analysis" analysis_description = "Summing up numbers" available_processors = ["AccumulatorProcessor", "GaussAdder", "ModifyLoopProcessor"] [AccumulatorProcessor] # A processor to calculate the sum of the first n values via a looping approach. last_value = 100 # Last value of the series [GaussAdder] # A processor to calculate the sum of the first n values via the so called *Gauss formula*. last_value = 100 # Last value of the series. [ModifyLoopProcessor] # Example processor demonstrating how it is possible to change the looping structure. item_to_abort = 65 # Item to abort items_to_skip = [12, 16, 25] # List of items to be skipped. total_item = 100 # Total item in the loop. [UserInterface] # Specify UI options interface = "rich" # Default "rich", backup "console" ``` -------------------------------- ### Initialize Prime Number List in Python Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/examples/loop_modifier Resets the list of found prime numbers to empty before starting a new prime number search. This method should be called after the superclass's start method. ```python def start(self) -> None: """ Overload of the start method. **Remember:** to call the super method when you overload the start. In this specific case, we just make sure that the list of found prime numbers is empty. """ super().start() self.prime_num_found = [] ``` -------------------------------- ### Initialize Prime Number Search Processor Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/examples/loop_modifier This method is called once before the main while loop starts. It initializes the list of found prime numbers, sets the target number of primes for the progress bar, and adjusts the starting number for the search. It ensures the starting number is odd if it's greater than 2. Dependencies: `super().start()`, `self.prime_num_found`, `self.n_item`, `self.prime_num_to_find`, `self.start_from`, `self.item`. ```python def start(self) -> None: """ The overload of the start method. **Remember:** The start method is called just before the while loop is started. So all instructions in this method will be executed only once at the beginning of the process execution. Always put a call to its `super` when you overload start. First, we empty the list of found prime numbers. It should not be necessary, but it makes the code more readable. Then set the :attr:`.Processor.n_item` to the total number of prime numbers we need to find. In this way, the progress bar will display useful progress. If the start value is smaller than 2, then let's add 2 to the list of found prime number and set our first item to check at 3. In principle, we could already add 3 as well, but maybe the user wanted to find only 1 prime number, and we are returning a list with two, that is not what he was expecting. Since prime numbers different from 2 can only be odd, if the starting number is even, increment it already by 1 unit. """ super().start() self.prime_num_found = [] self.n_item = self.prime_num_to_find if self.start_from < 2: self.prime_num_found.append(2) self.start_from = 3 if self.start_from % 2 == 0: self.item = self.start_from + 1 else: self.item = self.start_from ``` -------------------------------- ### Python Configuration Dictionary Example Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/generated/mafw.processor Demonstrates a configuration dictionary for a processor, specifying parameters like 'param1' and 'input_table'. This format is used when option 2 is selected for configuration. ```python conf_dict2 = {'param1': 5, 'input_table': 'my_table'} ``` -------------------------------- ### Find N Prime Numbers using While Loop in Python Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/examples/loop_modifier An example processor that finds a specified number of prime numbers starting from a given integer. It utilizes the `@while_loop` decorator and defines a `while_condition` to control the loop's execution. The `format_progress_message` method provides feedback on the search progress. ```python import logging import math from typing import Any, Collection from mafw.decorators import for_loop, while_loop from mafw.enumerators import LoopingStatus from mafw.processor import ActiveParameter, Processor log = logging.getLogger(__name__) @while_loop class FindNPrimeNumber(Processor): """ An example of Processor to search for N prime numbers starting from a given starting integer. This processor is meant to demonstrate the use of a while_loop execution workflow. Let us say we need to find 1000 prime numbers starting from 12347. One possible brute force approach to solve this problem is to start checking if the initial value is a prime number. If this is not the case, then check the next odd number. If it is the case, then add the current number to the list of found prime numbers and continue until the size of this list is 1000. This is a perfect application for a while loop execution workflow. """ prime_num_to_find = ActiveParameter( 'prime_num_to_find', default=100, help_doc='How many prime number we have to find' ) start_from = ActiveParameter('start_from', default=50, help_doc='From which number to start the search') def __init__(self, *args: Any, **kwargs: Any): """ Processor parameters: :param prime_num_to_find: The number of prime numbers to be found. :type prime_num_to_find: int :param start_from: The initial integer number from where to start the search. :type start_from: int """ super().__init__(*args, **kwargs) self.prime_num_found: list[int] = [] """The list with the found prime numbers""" def format_progress_message(self) -> None: self.progress_message = ( f'Checking integer number: {self.item}, already found {len(self.prime_num_found)} prime numbers' ) def while_condition(self) -> bool: """ Define the while condition. First, it checks if the prime_num_to_find is positive. Otherwise, it does not make sense to start. Then it will check if the length of the list with the already found prime numbers is enough. If so, then we can stop the loop return False, otherwise, it will return True and continue the loop. ``` -------------------------------- ### MAFw Processor Parameter Setting via Configuration Object Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_processor.rst Demonstrates setting processor parameters using a configuration object (dictionary). This approach is particularly useful when loading configurations from files. The example shows two ways to structure the configuration object and how to pass it to the processor using the `config` keyword argument. It also includes examples of dumping the current parameter configuration. ```python @single_loop class ConfigProcessor(Processor): p1 = ActiveParameter('param1', default='value') p2 = ActiveParameter('param2', default='value') cp = ConfigProcessor(config=dict(param1='new_value', param2='better_value', param3='do not exists')) assert cp.p1 == 'new_value' assert cp.p2 == 'better_value' dumped_config = cp.dump_parameter_configuration(option=2) assert dumped_config == dict(param1='new_value', param2='better_value') config = {'ConfigProcessor': {'param1': '1st', 'param2': '2nd'}} cp = ConfigProcessor(config=config) assert cp.p1 == '1st' assert cp.p2 == '2nd' dumped_config = cp.dump_parameter_configuration() assert config == dumped_config ``` -------------------------------- ### Get Git Tags (Python) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/generated/mafw.scripts.doc_versioning Returns a sorted list of (Version, tag) tuples from Git history. It can optionally filter tags starting from a minimum version. The list is sorted in ascending order. ```python from typing import List, Tuple, Any from packaging.version import Version # Assuming mafw.scripts.doc_versioning is available def get_git_tags(min_version: str | None = None) -> List[Tuple[Version, Any]]: # Implementation details... pass ``` -------------------------------- ### AccumulatorProcessor: Sum using a loop Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/examples/sum_processor Calculates the sum of the first n integers using a looping mechanism. It resets the accumulated value at the start and adds each item during processing. The result is stored in `accumulated_value`. ```python # Copyright 2025 European Union # Author: Bulgheroni Antonio (antonio.bulgheroni@ec.europa.eu) # SPDX-License-Identifier: EUPL-1.2 """ The module provides some examples for the user to develop their own processors. Those implemented here are mainly used in the test suit. """ from mafw.enumerators import LoopType from mafw.processor import ActiveParameter, Processor [docs] class AccumulatorProcessor(Processor): r""" A processor to calculate the sum of the first n values via a looping approach. In mathematical terms, this processor solves this easy equation: .. math:: N = \sum_{i=0}^{n}{i} by looping. It is a terribly inefficient approach, but it works as a demonstration of the looping structure. The user can get the results by retrieving the `accumulated_value` parameter at the end of the processor execution. """ last_value = ActiveParameter('last_value', default=100, help_doc='Last value of the series') def __init__(self, *args, **kwargs): """Constructor parameters: :param last_value: The `n` in the equation above. Defaults to 100 :type last_value: int :param accumulated_value: The `N` in the equation above at the end of the process. :type accumulated_value: int """ super().__init__(*args, **kwargs) self.accumulated_value: int = 0 [docs] def start(self): """Resets the accumulated value to 0 before starting.""" super().start() self.accumulated_value = 0 [docs] def get_items(self) -> list[int]: """Returns the list of the first `last_value` integers.""" return list(range(self.last_value)) [docs] def process(self): """Increase the accumulated value by the current item.""" self.accumulated_value += self.item ``` -------------------------------- ### main() Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/generated/mafw.scripts.update_changelog The main entry point for the script. ```APIDOC ## Function: mafw.scripts.update_changelog.main() ### Description Script entry point ### Method N/A (Function) ### Endpoint N/A ### Parameters None ### Request Example None ### Response #### Success Response (None) This function does not return any value. #### Response Example None ``` -------------------------------- ### HDFPlotterSNS with Steering File Parameters Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/doc_plotting This example demonstrates how to configure 'HDFPlotterSNS' to accept HDF5 filename and key from a steering file using 'ActiveParameter'. The parameters are assigned within the 'start' method. ```python @single_loop class HDFPlotterSNS(HDFDataRetriever, SNSPlotter): hdf_filename_from_steering = ActiveParameter('hdf_filename_from_steering', default='my_datafile.h5') hdf_key_from_steering = ActiveParameter('hdf_key_from_steering', default='my_key') def start(self): super().start() self.hdf_filename = self.hdf_filename_from_steering self.key = self.hdf_key_from_steering # let us execute the processor from within the mafw runner. ``` -------------------------------- ### FindNPrimeNumber.start Method (Python) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_examples.rst Initializes the FindNPrimeNumber processor. It sets the total number of items for progress tracking, resets the list of found primes and item counters. It also determines the starting number for prime checking based on the input parameters. ```python def start(self) -> None: self.n_item = self.prime_num_to_find self.prime_num_found = [] self.i_item = 0 if self.prime_num_to_find % 2 == 0: self.item = self.start_from if self.start_from % 2 != 0 else self.start_from + 1 else: self.item = self.start_from if self.start_from % 2 == 0 else self.start_from + 1 log.info('Start finding prime numbers from %s', self.item) ``` -------------------------------- ### For Loop Processor for Finding Prime Numbers in a Range (Python) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_examples.rst Implements a processor that finds all prime numbers within a specified range (start_from to stop_at). It utilizes the 'for_loop' decorator and requires overriding key methods like get_items, start, process, and finish. ```python @for_loop class FindPrimeNumberInRange(Processor): """Find prime numbers in a given range. Args: start_from: The starting integer of the range (inclusive). stop_at: The ending integer of the range (exclusive). """ def __init__(self, start_from: int, stop_at: int): super().__init__() self.start_from = start_from self.stop_at = stop_at self.prime_num_found = [] def get_items(self) -> Collection[Any]: """Return a list of integers in the specified range to iterate over.""" return list(range(self.start_from, self.stop_at, 2)) def start(self) -> None: """Initialize the list for found prime numbers and log the start.""" super().start() log.info('Starting to find prime numbers between %s and %s', self.start_from, self.stop_at) def process(self, item: int) -> None: """Check if the current item is prime and add it to the list if it is.""" if is_prime(item): self.prime_num_found.append(item) def finish(self) -> None: """Log the largest prime number found.""" super().finish() log.info('The largest is %s', self.prime_num_found[-1]) def format_progress_message(self) -> str: """Format a progress message for the user.""" return f'Found {len(self.prime_num_found)} prime numbers so far...' ``` -------------------------------- ### DBInit Processor - Initialization Steps Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/generated/mafw.processor_library.db_init Describes the steps involved in initializing or recreating database tables using the DBInit processor. It includes filtering by prefix, handling forced or soft recreation, creating tables, and initializing standard tables. ```APIDOC ## DBInit Processor - Initialization Steps ### Description This outlines the sequence of operations performed by the DBInit processor for database initialization or recreation. ### Method N/A (Internal processor logic) ### Endpoint N/A (Internal processor logic) ### Parameters #### Query Parameters - **apply_only_to_prefix** (list[str]) - Optional - Apply only to tables starting with the specified prefixes. Defaults to an empty list. - **force_recreate** (bool) - Optional - If True, all tables will be dropped and recreated. Use with extreme caution as this results in data loss. Defaults to False. - **soft_recreate** (bool) - Optional - If True, tables with existing models will be recreated safely (no drop, only new tables created or updated triggers). If False, only non-existing tables with models will be created. Defaults to True. - **remove_orphan_files** (bool) - Optional - Flag to remove or protect orphan files. Defaults to True. ### Request Example N/A (Internal processor logic) ### Response #### Success Response (200) N/A (Internal processor logic) #### Response Example N/A (Internal processor logic) ``` -------------------------------- ### Run Simple Processor List with mafw Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/examples/processor_list Demonstrates the simplest way to create and execute a ProcessorList. It appends two processors, AccumulatorProcessor and GaussAdder, to the list and then executes them. The example verifies that the accumulated values from both processors are consistent. ```python # Copyright 2025 European Union # Author: Bulgheroni Antonio (antonio.bulgheroni@ec.europa.eu) # SPDX-License-Identifier: EUPL-1.2 """ The module provides some examples on how to use ProcessorList to combine several processors. """ [docs] def run_simple_processor_list(): """Simplest way to run several processors in a go.""" from mafw.examples.sum_processor import AccumulatorProcessor, GaussAdder from mafw.processor import ProcessorList # create the list. name and description are optional new_list = ProcessorList(name='AddingProcessor', description='Summing up numbers') # append the processors. you can pass parameters to the processors in the standard way max_value = 120 new_list.append(AccumulatorProcessor(last_value=max_value)) new_list.append(GaussAdder(last_value=max_value)) # execute the list. This will execute all the processors in the list new_list.execute() # you can access single processors in the list, in the standard way. # remember that the ProcessorList is actually a list! assert new_list[0].accumulated_value == new_list[1].sum_value ``` -------------------------------- ### ImporterExample Get Items Method Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_examples.rst The get_items method retrieves a list of files from a specified input folder. It supports recursive searching and filtering based on file extensions and existing database entries. The new_only flag ensures only new or modified files are considered. ```python def get_items(self) -> Collection[Any]: return file_list ``` -------------------------------- ### Modify Loop Processor in Python Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/examples/loop_modifier Demonstrates a looping processor that can skip or abort items based on defined conditions. It uses ActiveParameters for configuration and manipulates the LoopingStatus enum to control loop flow. The 'start' method resets the skipped items list, and 'get_items' provides the range of items to process. ```python import logging import math from typing import Any, Collection from mafw.decorators import for_loop, while_loop from mafw.enumerators import LoopingStatus from mafw.processor import ActiveParameter, Processor log = logging.getLogger(__name__) class ModifyLoopProcessor(Processor): """ Example processor demonstrating how it is possible to change the looping structure. It is a looping processor where some events will be skipped, and at some point one event will trigger an abort. """ total_item: ActiveParameter[int] = ActiveParameter('total_item', default=100, help_doc='Total item in the loop.') items_to_skip: ActiveParameter[list[int]] = ActiveParameter( 'items_to_skip', default=[12, 16, 25], help_doc='List of items to be skipped.' ) item_to_abort: ActiveParameter[int] = ActiveParameter('item_to_abort', default=65, help_doc='Item to abort') def __init__(self, *args, **kwargs): """ Processor Parameters: :param total_item: The total number of items :type total_item: int :param items_to_skip: A list of items to skip. :type items_to_skip: list[int] :param item_to_abort: The item where to trigger an abort. :type item_to_abort: int """ super().__init__(*args, **kwargs) self.skipped_items: [list[int]] = [] """A list with the skipped items.""" def start(self): """Resets the skipped item container.""" super().start() self.skipped_items = [] def get_items(self) -> list[int]: """Returns the list of items, the range from 0 to total_item.""" return list(range(self.total_item)) def process(self): """Processes the item""" if self.item in self.items_to_skip: self.looping_status = LoopingStatus.Skip return if self.item == self.item_to_abort: self.looping_status = LoopingStatus.Abort return def skip_item(self): """Add skipped item to the skipped item list.""" self.skipped_items.append(self.item) ``` -------------------------------- ### Configure SQL Script Execution with SQLScriptRunner Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_database.rst This configuration example shows how to set up the SQLScriptRunner processor in a TOML file to execute a list of SQL scripts. It specifies the paths to the SQL files that will be processed sequentially within a single database transaction. ```toml [SQLScriptRunner] sql_files = ["./scripts/init_schema.sql", "./scripts/populate_data.sql"] ``` -------------------------------- ### Create and Execute MAFwApplication Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/generated/mafw.runner Demonstrates the creation and execution of a MAFwApplication instance. This involves importing necessary modules, setting up logging, and instantiating the MAFwApplication class. The application can be configured via a steering file, user interface, and plugin manager. ```python import logging from pathlib import Path from mafw.runner import MAFwApplication log = logging.getLogger(__name__) # Example usage (assuming steering_file and user_interface are defined elsewhere) # steering_file_path = Path("path/to/your/steering_file.yaml") # app = MAFwApplication(steering_file=steering_file_path) # app.run() ``` -------------------------------- ### Create and Run MAFwApplication Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/runner Demonstrates how to instantiate and execute a MAFwApplication using a steering file. It includes basic error handling for the execution process. Ensure the 'path_to_my_steering_file.toml' exists and is correctly configured. ```python import logging from pathlib import Path from mafw.runner import MAFwApplication log = logging.getLogger(__name__) # put here your steering file steering_file = Path('path_to_my_steering_file.toml') try: # create the app app = MAFwApplication(steering_file) # run it! app.run() except Exception as e: log.error('An error occurred!') log.exception(e) ``` -------------------------------- ### MAFw Processor Parameter Setting via Keyword Arguments Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_processor.rst Illustrates how to set parameter values for both Active and Passive parameters during processor initialization using keyword arguments. This method is straightforward and automatically handles parameter registration. The example defines a processor with two active parameters and initializes them with specific values. ```python class MyProcessor(Processor): active_param = ActiveParameter('value', default=0, help_doc='An active parameter') active_string = ActiveParameter('string', default=0, help_doc='An active parameter') my_p = MyProcessor(value=100, string='a better string', looper='single') print(my_p.active_param) # we get 100 assert my_p.active_param == 100 print(my_p.active_string) # we get 'a better string' assert my_p.active_string == 'a better string' ``` -------------------------------- ### Install MAFw with Seaborn (Optional) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/doc_getting_started Installs MAFw along with optional dependencies for enhanced graphical outputs using Seaborn. This command ensures that Seaborn, Matplotlib, and Pandas are also installed. ```bash pip install mafw[seaborn] ``` -------------------------------- ### Install MAFw using pip (Windows) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_getting_started.rst Installs MAFw in a separated virtual environment on Windows. It involves creating a virtual environment, navigating into it, activating it, and then installing MAFw using pip. ```doscon c:\> python -m venv mafw-env c:\> cd mafw-env c:\mafw-env> Scripts\activate (mafw-env)c:\mafw-env> pip install mafw ``` -------------------------------- ### ProcessorList Execution Example Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_database.rst Demonstrates how to set up and execute a ProcessorList with multiple processors (FileImporter, RowRemover, OrphanProtector, LazyProcessor) for testing orphan file management. It configures an in-memory SQLite database and asserts the successful completion of each processor. ```python db_conf = default_conf['sqlite'] db_conf['URL'] = 'sqlite:///:memory:' plist = ProcessorList(name='Orphan test', description='dealing with orphan files', database_conf=db_conf) importer = FileImporter(input_folder=tmp_path) remover = RowRemover(n_rows=n_delete) protector = OrphanProtector() lazy = LazyProcessor() plist.extend([importer, remover, protector, lazy]) plist.execute() assert importer.processor_exit_status == ProcessorExitStatus.Successful assert remover.processor_exit_status == ProcessorExitStatus.Successful assert protector.processor_exit_status == ProcessorExitStatus.Successful assert lazy.processor_exit_status == ProcessorExitStatus.Successful ``` -------------------------------- ### Initialize and Run MAFwApplication Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/generated/mafw.runner This snippet demonstrates how to initialize and run the MAFwApplication using a steering file. It includes basic error handling for exceptions during application execution. The MAFwApplication constructor takes a steering file path, and the run method executes the application's defined tasks. ```python from pathlib import Path from mafw import MAFwApplication steering_file = Path('path_to_my_steering_file.toml') try: # create the app app = MAFwApplication(steering_file) # run it! app.run() except Exception as e: # Assuming 'log' is a configured logger log.error('An error occurred!') log.exception(e) ``` -------------------------------- ### Start Trigger Processor Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/processor_library/db_init Starts the MAFw trigger processor by calling the parent's start method and then obtaining the appropriate database dialect. This ensures that the processor is ready to perform trigger manipulations. ```python def start(self) -> None: super().start() self.dialect = self.get_dialect() ``` -------------------------------- ### Verify MAFw Installation Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/doc_runner This command checks if the MAFw executable is installed and accessible in your system's PATH. It displays the installed version of MAFw, confirming that the environment is set up correctly for running processors. ```bash (env) C:\>mafw -v mafw, version 0.0.5 ``` -------------------------------- ### Initialize Application with Steering File (Python) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_modules/mafw/runner Initializes the application by parsing a steering file. This method can be called automatically by the constructor or manually at a later time. It takes a path to the steering file as input and configures the application based on its contents. Dependencies include `Path` from the `pathlib` module and `toml_tools` for loading the steering file. ```python def init(self, steering_file: Path | str) -> None: """ Initializes the application. This method is normally automatically invoked by the class constructor. It can be called in a later moment to force the parsing of the provided steering file. :param steering_file: The path to the steering file. :type steering_file: Path | str """ if isinstance(steering_file, str): steering_file = Path(steering_file) self.steering_file = steering_file self._configuration_dict = toml_tools.load_steering_file(steering_file) if self.user_interface is None: self.user_interface = self.get_user_interface(self._configuration_dict['UserInterface']['interface']) self._initialized = True self.name = self._configuration_dict.get('analysis_name', 'mafw analysis') ``` -------------------------------- ### Install MAFw using pip (Linux/MacOS) Source: https://mafw-e8abfd.pages.code.europa.eu/doc/latest/_sources/doc_getting_started.rst Installs MAFw in a separated virtual environment on Linux and MacOS. The process includes creating a virtual environment, changing the directory, activating the environment, and installing MAFw via pip. ```bash $ python -m venv mafw-env $ cd mafw-env $ source bin/activate (mafw-env) $ pip install mafw ```