Multiversum

https://simson.io/multiversum/
Admin
Multiversum is a Python package designed to make it easy to conduct, manage, and run parallel...
Tokens:32,311
Snippets:128
Trust Score:5.7
Update:1 week ago
Show doc for...
Tokens:
Raw
### Install multiversum package

Source: https://simson.io/multiversum

Installs the multiversum library using pip.

```bash
pip install multiversum

```

--------------------------------

### get_run_dir(sub_directory=None)

Source: https://simson.io/multiversum/api/multiverse_analysis

Gets the directory for the current run, optionally appending a sub-directory. Creates the directory if it does not exist.

```APIDOC
## get_run_dir(sub_directory=None)

### Description
Get the directory for the current run. Optionally appends a sub-directory to the run directory and creates the directory if it does not exist.

### Method
Method call (Python)

### Endpoint
`get_run_dir(sub_directory=None)`

### Parameters
#### Path Parameters
- **sub_directory** (Optional[str]) - Optional - An optional sub-directory to append to the run directory. Default is `None`.

### Returns
- **Path** - A Path object for the current run directory.

### Source Code
```python
def get_run_dir(self, sub_directory: Optional[str] = None) -> Path:
    """
    Get the directory for the current run.

    Args:
        sub_directory: An optional sub-directory to append to the run directory.

    Returns:
        A Path object for the current run directory.
    """
    run_dir = self.output_dir / "runs" / str(self.run_no)
    target_dir = run_dir / sub_directory if sub_directory is not None else run_dir
    target_dir.mkdir(parents=True, exist_ok=True)
    return target_dir
```
```

--------------------------------

### Universe.__init__

Source: https://simson.io/multiversum/api/universe

Initializes a Universe analysis instance. It parses the settings dictionary, handles seed overriding, and sets up dimensions, output directory, and metrics.

```APIDOC
## Universe.__init__

### Description
Initializes a Universe analysis instance. It parses the settings dictionary, handles seed overriding, and sets up dimensions, output directory, and metrics.

### Method
Constructor

### Endpoint
Universe.__init__

### Parameters
#### Constructor Parameters
- **settings** (dict or str) - Required - The settings for the universe analysis. This can usually just be passed along from the multiverse analysis. You only need to specify this yourself when developing / trying out an analysis. Possible keys in the dictionary are:
  - dimensions: The specified universe dimensions. This is the only required information.
  - run_no: The run number of the multiverse analysis.
  - seed: The seed to use for analyses.
  - universe_id: The id of the universe.
  - output_dir: The directory to which the output should be written.
- **metrics** (dict) - Required - A dictionary containing the metrics to be computed. Pass an empty dictionary to not compute any.
- **fairness_metrics** (dict) - Required - A dictionary containing the fairness metrics to be computed. (These are computed with awareness of groups.) Pass an empty dictionary to not compute any.
- **expand_dicts** (bool) - Optional - Whether to expand dictionaries in the dimensions i.e. if there are any dictionaries in the dimensions, expand them into separate dimensions of their own. Defaults to False.
- **seed** (int or str) - Optional - An optional seed value to override the one from settings. This can be either:
  - an integer: directly used as seed value
  - a string: interpreted as a dimension name whose value will be used as seed
- **set_seed** (bool) - Optional - Whether to use the seed provided in the settings. Defaults to True. Please note, that this only sets the seed in the Python random module and numpy.

### Request Example
```python
Universe(
    settings={
        "dimensions": {"age": 30, "gender": "female"},
        "run_no": 1,
        "seed": 42,
        "universe_id": "universe-1",
        "output_dir": "./output"
    },
    metrics={"accuracy": "accuracy_score"},
    fairness_metrics={"equalized_odds": "equalized_odds"},
    expand_dicts=False,
    seed=None,
    set_seed=True
)
```

### Response
#### Success Response
No return value. The constructor initializes the Universe instance with the provided settings.

#### Response Example
No response body.
```

--------------------------------

### get_execution_time()

Source: https://simson.io/multiversum/api/universe

Gets the execution time of the universe analysis. Returns the elapsed time in seconds as a float.

```APIDOC
## get_execution_time()

### Description
Gets the execution time of the universe analysis.

### Method
Function call

### Endpoint
`get_execution_time()`

### Parameters
None

### Returns
- **float** (float) - The execution time in seconds.

### Source Code
```python
def get_execution_time(self) -> float:
    """
    Gets the execution time of the universe analysis.

    Returns:
        float: The execution time in seconds.
    """
    if self.ts_end is None:
        print("Stopping execution_time clock.")
        self.ts_end = time.time()
    return self.ts_end - self.ts_start
```
```

--------------------------------

### MultiverseAnalysis __init__ constructor

Source: https://simson.io/multiversum/api/multiverse_analysis

Initializes a MultiverseAnalysis instance. Accepts dimensions, config (TOML/JSON/Python), universe file, output directory, run number, and other options. Searches for config files automatically if not provided, parses them, and converts to a Config object. Raises ValueError if dimensions are not specified.

```python
def __init__(
    self,
    dimensions: Optional[Dict] = None,
    config: Union[Path, Config, None] = None,
    universe: Path = None,
    output_dir: Path = Path("./output"),
    run_no: Optional[int] = None,
    new_run: bool = True,
    seed: Optional[int] = None,
    stop_on_error: Optional[bool] = None,
    cell_timeout: Optional[int] = None,
) -> None:
    """
    Initializes a new MultiverseAnalysis instance.

    Args:
        dimensions: A dictionary where keys are dimension names and values are lists of possible values for each dimension.
            Each dimension corresponds to a decision.
        config: A Path to a TOML, JSON or Python file containing the
            analysis configuration. Supported configuration options can be
            found in the Config class. If a Python file is used, it should
            contain a dictionary / config object named "config".
            Will automatically search for multiverse.toml / .json / .py.
        universe: The Path to the universe_file to run. Either an
            ipython / jupyter notebook (.ipynb) or a python script (.py).
        output_dir: The directory to store the output in.
        run_no: The number of the current run. Defaults to an automatically
            incrementing integer number if new_run is True or the last run if
            new_run is False.
        new_run: Whether this is a new run or not. Defaults to True.
        seed: The seed to use for the analysis.
        stop_on_error: Whether to stop the analysis if an error occurs.
        cell_timeout: A timeout (in seconds) for each cell in the notebook.
    """
    # Check for configuration file and parse it
    config_file = search_files(file=config, default_files=DEFAULT_CONFIG_FILES)
    if isinstance(config_file, Path):
        if config_file.suffix == ".toml":
            with open(config_file, "rb") as fp:
                config = tomllib.load(fp)
        elif config_file.suffix == ".json":
            with open(config_file, "r") as fp:
                config = json.load(fp)
        elif config_file.suffix == ".py":
            results = runpy.run_path(str(config_file))
            config = results["config"]
        else:
            raise ValueError(
                "Only .toml, .json and .py files are supported as config."
            )
    # Convert config to Config object
    if isinstance(config, dict):
        config = Config(**config)

    # Read settings from config (or args)
    self.read_config_value(config, "dimensions", dimensions)
    self.read_config_value(config, "seed", seed)
    self.read_config_value(config, "stop_on_error", stop_on_error)
    self.read_config_value(config, "cell_timeout", cell_timeout)
    self.read_config_value(config, "constraints")

    universe_file = search_files(
        file=universe, default_files=DEFAULT_UNIVERSE_FILES
    )
    self.universe_file = universe_file

    self.output_dir = output_dir
    if self.output_dir is not None:
        self.output_dir.mkdir(parents=True, exist_ok=True)

    self.run_no = (
        run_no if run_no is not None else self.read_counter(increment=new_run)
    )

    if self.dimensions is None:
        raise ValueError(
            "Dimensions need to be specified either directly or in a config."
        )

```

--------------------------------

### MultiverseAnalysis.__init__

Source: https://simson.io/multiversum/api/multiverse_analysis

Initializes a new MultiverseAnalysis instance. Accepts dimensions, configuration, universe file, and various run settings to set up the analysis.

```APIDOC
## MultiverseAnalysis.__init__

### Description
Initializes a new MultiverseAnalysis instance with the provided dimensions, configuration, universe file, and run settings.

### Method
Constructor

### Signature
```python
def __init__(
    self,
    dimensions: Optional[Dict] = None,
    config: Union[Path, Config, None] = None,
    universe: Path = None,
    output_dir: Path = Path("./output"),
    run_no: Optional[int] = None,
    new_run: bool = True,
    seed: Optional[int] = None,
    stop_on_error: Optional[bool] = None,
    cell_timeout: Optional[int] = None,
) -> None
```

### Parameters
- **dimensions** (Optional[Dict]) - Optional - A dictionary where keys are dimension names and values are lists of possible values for each dimension. Each dimension corresponds to a decision.
- **config** (Union[Path, Config, None]) - Optional - A Path to a TOML, JSON or Python file containing the analysis configuration. Supported configuration options can be found in the Config class. If a Python file is used, it should contain a dictionary / config object named "config". Will automatically search for multiverse.toml / .json / .py.
- **universe** (Path) - Optional - The Path to the universe_file to run. Either an ipython / jupyter notebook (.ipynb) or a python script (.py).
- **output_dir** (Path) - Optional - The directory to store the output in. Defaults to Path("./output").
- **run_no** (Optional[int]) - Optional - The number of the current run. Defaults to an automatically incrementing integer number if new_run is True or the last run if new_run is False.
- **new_run** (bool) - Optional - Whether this is a new run or not. Defaults to True.
- **seed** (Optional[int]) - Optional - The seed to use for the analysis.
- **stop_on_error** (Optional[bool]) - Optional - Whether to stop the analysis if an error occurs.
- **cell_timeout** (Optional[int]) - Optional - A timeout (in seconds) for each cell in the notebook.

### Description of Behavior
- Checks for a configuration file and parses it if provided. Supported formats: .toml, .json, .py. Raises ValueError for unsupported formats.
- Converts config to a Config object if a dictionary is provided.
- Reads settings from config or arguments for dimensions, seed, stop_on_error, cell_timeout, and constraints.
- Searches for the universe file using default universe files if not provided.
- Creates the output directory if it does not exist.
- Determines run_no based on run_no argument and new_run flag.
- Raises ValueError if dimensions are not specified either directly or in config.

### Raises
- **ValueError** - If the config file suffix is not .toml, .json, or .py.
- **ValueError** - If dimensions are not specified either directly or in config.
```

--------------------------------

### get_export_file_path(filename, mkdir=False)

Source: https://simson.io/multiversum/api/universe

Get the file path for exporting a file from this Universe. Returns a Path object representing the full export path.

```APIDOC
## get_export_file_path(filename, mkdir=False)

### Description
Get the file path for exporting a file from this Universe.

### Method
Function call

### Endpoint
`get_export_file_path(filename, mkdir=False)`

### Parameters
#### Path Parameters
- **filename** (str) - Required - The name of the file to export.
- **mkdir** (bool) - Optional - Whether to create the directory if it does not exist. Default: `False`

### Returns
- **Path** (Path) - The full path where the file should be exported.

### Source Code
```python
def get_export_file_path(self, filename: str, mkdir: bool = False) -> Path:
    """
    Get the file path for exporting a file from this Universe.

    Args:
        filename: The name of the file to export.
        mkdir: Whether to create the directory if it does not exist. (optional)

    Returns:
        Path: The full path where the file should be exported.
    """
    export_dir = (
        self.output_dir / "runs" / str(self.run_no) / "exports" / self.universe_id
    )
    if mkdir:
        export_dir.mkdir(parents=True, exist_ok=True)
    return export_dir / filename
```
```

--------------------------------

### MultiverseAnalysis.__init__

Source: https://simson.io/multiversum/api/multiverse_analysis

Initializes a new MultiverseAnalysis instance with the provided dimensions, configuration, universe file, and execution settings.

```APIDOC
## MultiverseAnalysis.__init__

### Description
Initializes a new MultiverseAnalysis instance. Accepts dimensions, an optional configuration file (TOML, JSON, or Python), a universe file (notebook or script), output directory, run number, and execution settings.

### Method
Constructor

### Signature
```python
MultiverseAnalysis(
    dimensions: Optional[Dict] = None,
    config: Union[Path, Config, None] = None,
    universe: Path = None,
    output_dir: Path = Path("./output"),
    run_no: Optional[int] = None,
    new_run: bool = True,
    seed: Optional[int] = None,
    stop_on_error: Optional[bool] = None,
    cell_timeout: Optional[int] = None,
) -> None
```

### Parameters
- **dimensions** (Optional[Dict]) - Optional - A dictionary where keys are dimension names and values are lists of possible values for each dimension. Each dimension corresponds to a decision.
- **config** (Union[Path, Config, None]) - Optional - A Path to a TOML, JSON or Python file containing the analysis configuration. Supported configuration options can be found in the Config class. If a Python file is used, it should contain a dictionary / config object named "config". Will automatically search for multiverse.toml / .json / .py.
- **universe** (Path) - Optional - The Path to the universe_file to run. Either an ipython / jupyter notebook (.ipynb) or a python script (.py).
- **output_dir** (Path) - Optional - The directory to store the output in. Defaults to Path("./output").
- **run_no** (Optional[int]) - Optional - The number of the current run. Defaults to an automatically incrementing integer number if new_run is True or the last run if new_run is False.
- **new_run** (bool) - Optional - Whether this is a new run or not. Defaults to True.
- **seed** (Optional[int]) - Optional - The seed to use for the analysis.
- **stop_on_error** (Optional[bool]) - Optional - Whether to stop the analysis if an error occurs.
- **cell_timeout** (Optional[int]) - Optional - A timeout (in seconds) for each cell in the notebook.

### Configuration File Support
- **.toml** - Parsed using tomllib.
- **.json** - Parsed using json.load.
- **.py** - Executed with runpy, expecting a variable named "config".
- Other extensions raise a ValueError.

### Notes
- The constructor reads settings from the config file (or arguments) for dimensions, seed, stop_on_error, cell_timeout, and constraints.
- The `constraints` attribute is an optional dictionary where keys are dimension names and values are lists of constraints. Each constraint is a dictionary with:
  - **value** - The value of the dimension that the constraint applies to.
  - **allowed_if** - A dictionary of dimension-value pairs that must be present for the constraint to be allowed.
  - **forbidden_if** - A dictionary of dimension-value pairs that must not be present for the constraint to be allowed.
  - Only one of allowed_if and forbidden_if can be present in a constraint.

```

--------------------------------

### read_counter(increment) method implementation

Source: https://simson.io/multiversum/api/multiverse_analysis

Reads the counter from counter.txt in the output directory. If the file does not exist, starts at 0. If increment is True, increments the counter before writing it back. Returns the current (or incremented) value.

```python
def read_counter(self, increment: bool) -> int:
    """
    Read the counter from the output directory.

    Args:
        increment: Whether to increment the counter after reading.

    Returns:
        The current value of the counter.
    """

    # Use a self-incrementing counter via counter.txt
    counter_filepath = self.output_dir / "counter.txt"
    if counter_filepath.is_file():
        with open(counter_filepath, "r") as fp:
            run_no = int(fp.read())
    else:
        run_no = 0
    if increment:
        run_no += 1
    with open(counter_filepath, "w") as fp:
        fp.write(str(run_no))

    return run_no

```

--------------------------------

### Universe.__init__

Source: https://simson.io/multiversum/api/universe

Initializes the Universe class with settings, optional metrics and fairness metrics, and options for expanding dictionaries and setting a seed. Arguments should be passed in from the larger multiverse analysis.

```APIDOC
## Universe.__init__

### Description
Initializes the Universe class with settings, optional metrics and fairness metrics, and options for expanding dictionaries and setting a seed. Arguments should be passed in from the larger multiverse analysis.

### Method
Constructor

### Signature
```python
def __init__(
    self,
    settings: Union[str, Dict[str, Any]],
    metrics: Optional[Dict[str, Callable]] = None,
    fairness_metrics: Optional[Dict[str, Callable]] = None,
    expand_dicts: bool = False,
    seed: Optional[Union[int, str]] = None,
    set_seed: bool = True,
) -> None
```

### Parameters
- **settings** (Union[str, Dict[str, Any]]) - Required - The settings for the universe.
- **metrics** (Optional[Dict[str, Callable]]) - Optional - A dictionary containing the metrics to be computed.
- **fairness_metrics** (Optional[Dict[str, Callable]]) - Optional - A dictionary containing the fairness metrics to be computed.
- **expand_dicts** (bool) - Optional - Whether to expand dictionaries in the settings.
- **seed** (Optional[Union[int, str]]) - Optional - The seed to use for reproducibility.
- **set_seed** (bool) - Optional - Whether to set the seed.

### Returns
None
```

--------------------------------

### Run multiverse analysis with uv

Source: https://simson.io/multiversum

Runs the multiverse analysis using the uv package manager.

```bash
uv run -m multiversum
```

--------------------------------

### MultiverseAnalysis.__init__

Source: https://simson.io/multiversum/api/multiverse_analysis

Initializes a new MultiverseAnalysis instance with configurable dimensions, configuration, universe file, output directory, run number, seed, error handling, and cell timeout.

```APIDOC
## MultiverseAnalysis.__init__

### Description
Initializes a new MultiverseAnalysis instance.

### Method
Constructor

### Signature
`__init__(dimensions=None, config=None, universe=None, output_dir=Path('./output'), run_no=None, new_run=True, seed=None, stop_on_error=None, cell_timeout=None)`

### Parameters
- **dimensions** (Optional[Dict]) - Optional - A dictionary where keys are dimension names and values are lists of possible values for each dimension. Each dimension corresponds to a decision.
- **config** (Union[Path, Config, None]) - Optional - A Path to a TOML, JSON or Python file containing the analysis configuration. Supported configuration options can be found in the Config class. If a Python file is used, it should contain a dictionary / config object named "config". Will automatically search for multiverse.toml / .json / .py.
- **universe** (Path) - Optional - The Path to the universe_file to run. Either an ipython / jupyter notebook (.ipynb) or a python script (.py).
- **output_dir** (Path) - Optional - The directory to store the output in. Defaults to Path('./output').
- **run_no** (Optional[int]) - Optional - The number of the current run. Defaults to an automatically incrementing integer number if new_run is True or the last run if new_run is False.
- **new_run** (bool) - Optional - Whether this is a new run or not. Defaults to True.
- **seed** (Optional[int]) - Optional - The seed to use for the analysis.
- **stop_on_error** (Optional[bool]) - Optional - Whether to stop the analysis if an error occurs.
- **cell_timeout** (Optional[int]) - Optional - A timeout (in seconds) for each cell in the notebook.

### Source Code
Source code in `multiversum/multiverse.py`
```

--------------------------------

### Universe class initialization and settings parsing

Source: https://simson.io/multiversum/api/universe

Initializes the Universe object by parsing settings, expanding dimensions, handling seed overrides, and setting up output directory and metrics. Warns when seed is overridden or dimensions are missing.

```python
# Extract settings
    parsed_settings = (
        json.loads(settings) if isinstance(settings, str) else settings
    )

    self.run_no = parsed_settings["run_no"] if "run_no" in parsed_settings else 0
    self.universe_id = (
        parsed_settings["universe_id"]
        if "universe_id" in parsed_settings
        else "no-universe-id-provided"
    )
    self.dimensions = parsed_settings["dimensions"]
    if expand_dicts:
        # Create a new dictionary to store expanded dimensions
        expanded_dimensions = {}
        # Process each key-value pair in dimensions
        for key, value in self.dimensions.items():
            if isinstance(value, dict):
                # For dictionary values, add all key-value pairs to root level
                # but don't expand nested dictionaries further
                expanded_dimensions.update(value)
            else:
                # For non-dictionary values, keep them as is
                expanded_dimensions[key] = value
        self.dimensions = expanded_dimensions

    # Handle seed overriding
    settings_seed = parsed_settings["seed"] if "seed" in parsed_settings else 0

    if seed is not None:
        # If seed is a string, try to interpret it as a dimension name
        if isinstance(seed, str):
            if seed in self.dimensions:
                seed_value = self.dimensions[seed]
                if "seed" in parsed_settings:
                    warnings.warn(
                        f"Seed from dimension '{seed}' ({seed_value}) is overriding seed from settings ({settings_seed})."
                    )
                self.seed = seed_value
            else:
                warnings.warn(
                    f"Dimension '{seed}' not found in dimensions. Using settings seed ({settings_seed}) instead."
                )
                self.seed = settings_seed
        else:
            # Seed is a direct value
            if "seed" in parsed_settings:
                warnings.warn(
                    f"Seed provided in constructor ({seed}) is overriding seed from settings ({settings_seed})."
                )
            self.seed = seed
    else:
        self.seed = settings_seed

    self.output_dir = (
        Path(parsed_settings["output_dir"])
        if "output_dir" in parsed_settings
        else Path("./output")
    )

    self.metrics = metrics
    self.fairness_metrics = fairness_metrics

    if self.dimensions is None:
        warnings.warn("No dimensions specified for universe analysis.")

    if set_seed:
        print(f"Setting seed to {self.seed} (in: [random, numpy.random]).")
        random.seed(self.seed)
        np.random.seed(self.seed)
```

--------------------------------

### Universe.__init__

Source: https://simson.io/multiversum/api/universe

Initializes the Universe class. The arguments should be passed in from the larger multiverse analysis.

```APIDOC
## Universe.__init__

### Description
Initializes the Universe class. The arguments should be passed in from the larger multiverse analysis.

### Method
Constructor

### Signature
```python
def __init__(
    self,
    settings: Union[str, Dict[str, Any]],
    metrics: Optional[Dict[str, Callable]] = None,
    fairness_metrics: Optional[Dict[str, Callable]] = None,
    expand_dicts: bool = False,
    seed: Optional[Union[int, str]] = None,
    set_seed: bool = True,
) -> None:
```

### Parameters
- **settings** (Union[str, Dict[str, Any]]) - Required - The settings for the universe analysis. This can usually just be passed along from the multiverse analysis. You only need to specify this yourself when developing / trying out an analysis. Possible keys in the dictionary are: dimensions (required), run_no, seed, universe_id, output_dir.
- **metrics** (Optional[Dict[str, Callable]]) - Optional - A dictionary containing the metrics to be computed. Pass an empty dictionary to not compute any.
- **fairness_metrics** (Optional[Dict[str, Callable]]) - Optional - A dictionary containing the fairness metrics to be computed. (These are cumputed with awareness of groups.) Pass an empty dictionary to not compute any.
- **expand_dicts** (bool) - Optional - Whether to expand dictionaries in the dimensions i.e. if there are any dictionaries in the dimensions, expand them into separate dimensions of their own. Defaults to False.
- **seed** (Optional[Union[int, str]]) - Optional - An optional seed value to override the one from settings. This can be either: an integer: directly used as seed value; a string: interpreted as a dimension name whose value will be used as seed.
- **set_seed** (bool) - Optional - Whether to use the seed provided in the settings. Defaults to True. Please note, that this only sets the seed in the Python random module and numpy.

### Returns
None
```

--------------------------------

### Distributed execution with --mode=partial-parallel and --mode=finalize

Source: https://simson.io/multiversum

Run a distributed multiverse analysis across multiple machines using partial-parallel mode. Use --partial with percentage ranges on each machine (e.g., 0%,50% and 50%,100%), then run --mode=finalize after all machines complete.

```shell
# On machine 1: Run first 33% of universes
python -m multiversum --mode=partial-parallel --partial="0%,50%"
# On machine 2: Run second 33% of universes
python -m multiversum --mode=partial-parallel --partial="50%,100%"

# After all machines complete, finalize the analysis
python -m multiversum --mode=finalize
```

--------------------------------

### execute_notebook_via_api(input_path, output_path, parameters)

Source: https://simson.io/multiversum/api/multiverse_analysis

Executes a notebook via the papermill Python API. This method takes an input notebook path, an output notebook path, and a dictionary of parameters, and runs the notebook with those parameters.

```APIDOC
## execute_notebook_via_api(input_path, output_path, parameters)

### Description
Executes a notebook via the papermill Python API.

### Method
Python method

### Endpoint
`execute_notebook_via_api(input_path, output_path, parameters)`

### Parameters
- **input_path** (str) - Required - The path to the input notebook.
- **output_path** (str) - Required - The path to the output notebook.
- **parameters** (Dict[str, str]) - Required - A dictionary containing the parameters for the notebook.

### Returns
- **None** - None

### Source Code
```python
def execute_notebook_via_api(
    self, input_path: str, output_path: str, parameters: Dict[str, str]
) -> None:
    """
    Executes a notebook via the papermill python API.

    Args:
        input_path: The path to the input notebook.
        output_path: The path to the output notebook.
        parameters: A dictionary containing the parameters for the notebook.

    Returns:
        None
    """
    pm.execute_notebook(
        input_path,
        output_path,
        parameters=parameters,
        progress_bar=False,
        kernel_manager_class="multiversum.IPCKernelManager.IPCKernelManager",
        execution_timeout=self.cell_timeout,
    )
```
```

--------------------------------

### python -m multiversum run options

Source: https://simson.io/multiversum/cli

Shows the options for the run subcommand.

```bash
  --mode [full|continue|test|partial-parallel|finalize]
                                  How to run the multiverse
                                  analysis.(continue: continue from previous
                                  run, full: run all universes, test: run a
                                  minimal set of universes where each unique
                                  option appears at least once, partial-
                                  parallel: run a partial range of universes
                                  in parallel see --partialfinalize: finalize
                                  the previous run e.g. after running in
                                  partial-parallel mode)
  --config PATH                   Relative path to a TOML, JSON or Python file
                                  with a config for the multiverse. Defaults
                                  to searching for multiverse.toml,
                                  multiverse.json, multiverse.py (in that
                                  order).
  --universe PATH                 Relative path to the universe file to run.
                                  Defaults to searching for universe.ipynb,
                                  universe.py (in that order).
  --output-dir PATH               Relative path to output directory for the
                                  results.
  --seed INTEGER                  The seed to use for the analysis (Defaults
                                  to 80539).
  --u-id TEXT                     Examine only a single universe with the
                                  given universe id (or starting with the
                                  provided id).
  --grid-only                     Only export the multiverse grid without
                                  running the analysis.
  --grid-format [json|csv|none]   Format of the exported multiverse grid
                                  (json, csv, or none to skip export).
  --n-jobs, --njobs INTEGER       Number of CPUs to use for parallel
                                  processing. -1 uses all CPUs, -2 uses all
                                  but one CPU (default), and 1 disables
                                  parallel processing.
  --partial TEXT                  Run only a specific percentage range of
                                  universes. Format: 'start%,end%' (e.g.
                                  '0%,50%' or '0,20%'). Set mode to 'partial-
                                  parallel' to run in parallel and avoid race
                                  conditions.
  --retry-errors                  When mode=continue, retry universes that
                                  previously resulted in errors.
  --retry-error-type TEXT         When mode=continue, retry only universes
                                  that failed with this specific error type.
  --help                          Show this message and exit.

```

--------------------------------

### Implement analysis in universe.py with multiversum and scikit-learn

Source: https://simson.io/multiversum

Implements the actual analysis in a Python file ('universe.py') using the multiversum Universe class and scikit-learn classifiers. It demonstrates how to access dimension settings dynamically and save results.

```python
# Implement the actual analysis

import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from multiversum import Universe

# Initialize the Universe with example settings for development
# These will automatically be overwritten by multiversum in the analysis
universe = Universe(settings = {
    # Automatically overwritten
    "classifier": "logistic",
    "feature_set": "standard",
})
seed = universe.seed

# Load the data
df = pd.read_csv("https://raw.githubusercontent.com/allisonhorst/palmerpenguins/main/inst/extdata/penguins.csv")

# Prepare data based on feature_set dimension
X = None
if universe.dimensions["feature_set"] == "minimal":
    X = df[["flipper_length_mm"]]
elif universe.dimensions["feature_set"] == "standard":
    X = df[["flipper_length_mm", "bill_length_mm"]]
elif universe.dimensions["feature_set"] == "full":
    X = df[["flipper_length_mm", "bill_length_mm", "bill_depth_mm"]]

y = df["species"]

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=seed)

# Choose classifier based on dimension
model = None
if universe.dimensions["classifier"] == "logistic":
    model = LogisticRegression(random_state=seed)
elif universe.dimensions["classifier"] == "random_forest":
    model = RandomForestClassifier(random_state=seed)

# Train model
model.fit(X_train, y_train)

# Evaluate
y_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)

# Prepare
results = pd.DataFrame({
    "accuracy": [accuracy],
    "feature_count": [len(X.columns)]
})

# Let multiversum handle the storage / saving
universe.save_data(results)

```

--------------------------------

### get_run_dir(sub_directory=None) method implementation

Source: https://simson.io/multiversum/api/multiverse_analysis

Returns a Path object for the current run directory, creating it if necessary. If sub_directory is provided, it is appended to the run directory. The run directory is constructed as output_dir/runs/run_no.

```python
def get_run_dir(self, sub_directory: Optional[str] = None) -> Path:
    """
    Get the directory for the current run.

    Args:
        sub_directory: An optional sub-directory to append to the run directory.

    Returns:
        A Path object for the current run directory.
    """
    run_dir = self.output_dir / "runs" / str(self.run_no)
    target_dir = run_dir / sub_directory if sub_directory is not None else run_dir
    target_dir.mkdir(parents=True, exist_ok=True)
    return target_dir
```

--------------------------------

### python -m multiversum --help output

Source: https://simson.io/multiversum/cli

Shows the help output for the main multiversum CLI command.

```bash
  --help  Show this message and exit.

```

--------------------------------

### Universe class definition and initializer

Source: https://simson.io/multiversum/api/universe

Defines the core Universe class with class-level defaults for metrics, fairness metrics, and timestamps, plus the __init__ method that accepts universe settings from the multiverse configuration. The settings parameter is typed as either a string or a dictionary.

```python
class Universe:
    """
    A class to help with running the analysis of a single universe contained
    within a multiverse analysis.

    Attributes:
        run_no: The run number of the multiverse analysis.
        universe_id: The id of the universe.
        universe: The universe settings.
        output_dir: The directory to which the output should be written.
        metrics: A dictionary containing the metrics to be computed.
        fairness_metrics: A dictionary containing the fairness metrics to be
            computed.
        ts_start: The timestamp of the start of the analysis.
        ts_end: The timestamp of the end of the analysis.
    """

    metrics = None
    fairness_metrics = None
    ts_start = None
    ts_end = None

    def __init__(
        self,
        settings: Union[str, Dict[str, Any]],
        metrics: Optional[Dict[str, Callable]] = None,
        fairness_metrics: Optional[Dict[str, Callable]] = None,
        expand_dicts: bool = False,
        seed: Optional[Union[int, str]] = None,
        set_seed: bool = True,
    ) -> None:
        """
        Initialize the Universe class.

        The arguments should be passed in from the larger multiverse analysis.
        """
```

--------------------------------

### visit_universe method implementation in multiversum/multiverse.py

Source: https://simson.io/multiversum/api/multiverse_analysis

Runs the complete analysis for a single universe. Generates a universe ID, removes old error files, builds the output path, and executes either a .ipynb notebook via API or a .py script. On error, renames the output file to indicate an error and either raises or logs and saves the error depending on stop_on_error.

```python
def visit_universe(self, universe_dimensions: Dict[str, str]) -> None:
    """
    Run the complete analysis for a single universe.

    Output from the analysis will be saved to a file in the run's output
    directory.

    Args:
        universe_dimensions: A dictionary containing the parameters
            for the universe.

    Returns:
        None
    """
    # Generate universe ID
    universe_id = generate_universe_id(universe_dimensions)
    logger.debug(f"Visiting universe: {universe_id}")

    # Clean up any old error fiels
    error_path = self._get_error_filepath(universe_id)
    if error_path.is_file():
        logger.warning(
            f"Removing old error file: {error_path}. This should only happen during a re-run."
        )
        error_path.unlink()

    universe_filetype = self.universe_file.suffix

    # Generate final command
    output_dir = self.get_run_dir(sub_directory="universes")
    output_filename = f"nb_{self.run_no}-{universe_id}{universe_filetype}"
    output_path = output_dir / output_filename

    # Ensure output dir exists
    output_dir.mkdir(parents=True, exist_ok=True)

    # Prepare settings dictionary
    settings = {
        "universe_id": universe_id,
        "dimensions": universe_dimensions,
        "run_no": self.run_no,
        "output_dir": str(self.output_dir),
        "seed": self.seed,
    }
    settings_str = json.dumps(settings, sort_keys=True)

    try:
        with warnings.catch_warnings(record=True) as w:
            warnings.simplefilter("always")
            if universe_filetype == ".ipynb":
                self.execute_notebook_via_api(
                    input_path=str(self.universe_file),
                    output_path=str(output_path),
                    parameters={
                        "settings": settings_str,
                    },
                )
            elif universe_filetype == ".py":
                self.execute_python_script(
                    input_path=str(self.universe_file),
                    output_path=str(output_path),
                    parameters=settings,
                )
            else:
                raise ValueError("Universe file must be a .ipynb or .py file.")

            for warning in w:
                logger.warning(
                    f"Warning in universe {universe_id}: {warning.message}"
                )
    except Exception as e:
        error_filename = "E_" + output_filename
        logger.error(f"Error in universe {universe_id} ({error_filename})")
        # Rename notebook file to indicate error
        error_output_path = output_dir / error_filename
        output_path.rename(error_output_path)
        if self.stop_on_error:
            raise e
        else:
            logger.exception(e)
            self.save_error(universe_id, universe_dimensions, e)

```

--------------------------------

### python -m multiversum aggregate-data options

Source: https://simson.io/multiversum/cli

Shows the options for the aggregate-data subcommand.

```bash
  --config PATH      Relative path to a TOML, JSON or Python file with a
                     config for the multiverse. Defaults to searching for
                     multiverse.toml, multiverse.json, multiverse.py (in that
                     order).
  --output-dir PATH  Relative path to output directory for the results.
  --run INTEGER      Run number to aggregate data for. Defaults to the latest
                     run.
  --help             Show this message and exit.

```

--------------------------------

### examine_multiverse() method implementation

Source: https://simson.io/multiversum/api/multiverse_analysis

Runs the analysis for all universes in the multiverse. Accepts an optional multiverse_grid and n_jobs parameter (default -2). If multiverse_grid is None, uses self.grid or generates a new grid. Handles single-threaded mode (n_jobs=1) and parallel mode using joblib's Parallel and rich_joblib for progress display.

```python
def examine_multiverse(
    self, multiverse_grid: List[Dict[str, Any]] = None, n_jobs: int = -2
) -> None:
    """
    Run the analysis for all universes in the multiverse.

    Args:
        multiverse_grid: A list of dictionaries containing the settings for different universes.
        n_jobs: The number of jobs to run in parallel. Defaults to -2 (all CPUs but one).
            Use -1 for all CPUs, positive integers for specific number of CPUs,
            or 1 to disable parallel processing.

    Returns:
        None
    """
    if multiverse_grid is None:
        multiverse_grid = self.grid or self.generate_grid(save_format="none")

    n_jobs = n_jobs if n_jobs > 0 else calculate_cpu_count(n_jobs)

    # Run analysis for all universes
    with Progress(
        TextColumn("[progress.description]{task.description}"),
        SpinnerColumn(),
        TaskProgressColumn(),
        BarColumn(bar_width=None),
        MofNCompleteColumn(),
        TextColumn("•"),
        TimeElapsedColumn(),
        TextColumn("•"),
        TimeRemainingColumn(),
        expand=True,
    ) as progress:
        task_id = progress.add_task("Running", total=len(multiverse_grid))
        if n_jobs == 1:
            logger.info("Running in single-threaded mode.")
            for universe_params in multiverse_grid:
                self.visit_universe(universe_params)
                progress.update(task_id, advance=1)
                # Somehow automatic updating is not working in single threaded mode, so we manually refresh
                progress.refresh()
        else:
            logger.info(f"Running in parallel mode (n_jobs = {n_jobs}).")
            with rich_joblib(progress, task_id):
                # For n_jobs below -1, (n_cpus + 1 + n_jobs) are used.
                # Thus for n_jobs = -2, all CPUs but one are used
                Parallel(n_jobs=n_jobs)(
                    delayed(self.visit_universe)(universe_params)
                    for universe_params in multiverse_grid
                )

```

--------------------------------

### examine_multiverse(multiverse_grid=None, n_jobs=-2)

Source: https://simson.io/multiversum/api/multiverse_analysis

Runs the analysis for all universes in the multiverse. Accepts an optional grid of universe settings and a parallelization parameter.

```APIDOC
## examine_multiverse(multiverse_grid=None, n_jobs=-2)

### Description
Run the analysis for all universes in the multiverse.

### Method
examine_multiverse(multiverse_grid=None, n_jobs=-2)

### Parameters
- **multiverse_grid** (List[Dict[str, Any]]) - Optional - A list of dictionaries containing the settings for different universes. Defaults to None.
- **n_jobs** (int) - Optional - The number of jobs to run in parallel. Defaults to -2 (all CPUs but one). Use -1 for all CPUs, positive integers for specific number of CPUs, or 1 to disable parallel processing.

### Returns
- **None** - None

### Example
```python
multiverse = Multiverse()
multiverse.examine_multiverse(multiverse_grid=[{"param": 1}], n_jobs=2)
```
```

--------------------------------

### Generate sub-universes from universe settings

Source: https://simson.io/multiversum/api/universe

Generates sub-universes by wrapping non-list dimension values with `list_wrap` and passing them to `generate_multiverse_grid`. Returns a list of dictionaries representing the sub-universes.

```python
def generate_sub_universes(self) -> List[dict]:
    """
    Generate the sub-universes for the given universe settings.

    Returns:
        A list of dictionaries containing the sub-universes.
    """
    # Wrap all non-lists in the universe to make them work with generate_multiverse_grid
    universe_all_lists = {k: list_wrap(v) for k, v in self.dimensions.items()}

    # Within-Universe variation
    return generate_multiverse_grid(universe_all_lists)

```

--------------------------------

### get_export_file_path(filename, mkdir=False)

Source: https://simson.io/multiversum/api/universe

Returns the full path for exporting a file from this Universe. Optionally creates the directory if it does not exist.

```APIDOC
## get_export_file_path(filename, mkdir=False)

### Description
Get the file path for exporting a file from this Universe.

### Method
Function call

### Endpoint
Universe.get_export_file_path(filename, mkdir=False)

### Parameters
#### Arguments
- **filename** (str) - Required - The name of the file to export.
- **mkdir** (bool) - Optional - Whether to create the directory if it does not exist. Default: False.

### Returns
- **Path** - The full path where the file should be exported.

### Example
```python
path = universe.get_export_file_path('output.csv', mkdir=True)
```
```

--------------------------------

### get_export_file_path() method implementation

Source: https://simson.io/multiversum/api/universe

Returns a Path object for the export file. If mkdir is True, it creates the export directory (including parents) if it does not exist.

```python
def get_export_file_path(self, filename: str, mkdir: bool = False) -> Path:
    """
    Get the file path for exporting a file from this Universe.

    Args:
        filename: The name of the file to export.
        mkdir: Whether to create the directory if it does not exist. (optional)

    Returns:
        Path: The full path where the file should be exported.
    """
    export_dir = (
        self.output_dir / "runs" / str(self.run_no) / "exports" / self.universe_id
    )
    if mkdir:
        export_dir.mkdir(parents=True, exist_ok=True)
    return export_dir / filename

```

--------------------------------

### python -m multiversum run usage

Source: https://simson.io/multiversum/cli

Shows the usage of the run subcommand.

```bash
python -m multiversum run [OPTIONS]

```
Tokens:
Raw