### Install ODAT-SE from Source Code

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/start.md

Clones the ODAT-SE repository from GitHub and installs it using pip. Requires pip version 19 or higher.

```bash
git clone https://github.com/issp-center-dev/ODAT-SE
python3 -m pip install ./ODAT-SE
```

--------------------------------

### Install ODAT-SE using PyPI

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/start.md

Installs the ODAT-SE Python package using pip. The `--user` flag installs it locally. Installing with `ODAT-SE[all]` includes optional packages.

```bash
python3 -m pip install ODAT-SE
python3 -m pip install ODAT-SE[all]
```

--------------------------------

### ODAT-SE Optimization Flow Example (Python)

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/customize/usage.md

This Python code snippet demonstrates the typical workflow for solving an optimization problem using ODAT-SE. It includes defining custom Solver and Algorithm classes, preparing input parameters from a TOML file using odatse.Info, instantiating the solver, runner, and algorithm, and finally invoking the main method to start the optimization.

```python
import sys
import odatse

# (1)
class Solver(odatse.solver.SolverBase):
    # Define your solver
    ...

class Algorithm(odatse.algorithm.AlgorithmBase):
    # Define your algorithm
    ...


# (2)
input_file = sys.argv[1]
info = odatse.Info.from_file(input_file)

# (3)
solver = Solver(info)
runner = odatse.Runner(solver, info)
algorithm = Algorithm(info, runner)

# (4)
result = algorithm.main()

```

--------------------------------

### Verify ODAT-SE Installation

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/start.md

Checks the installed ODAT-SE version and displays help information. These commands confirm that the installation was successful and the package is accessible.

```bash
odatse --version
odatse --help
```

--------------------------------

### Data File Format Example

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/post/tools/plt_1D_histogram.md

This example shows the expected format for input data files. Each line contains numerical data separated by whitespace, representing values for temperature/beta, function value, parameters, and weight.

```text
# Comment line (optional)
T (or beta)_value fx_value x1_value x2_value ... xN_value weight_value
T (or beta)_value fx_value x1_value x2_value ... xN_value weight_value
...
```

--------------------------------

### Example Output: SimplexData.txt

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/algorithm/minsearch.md

This is an example of the `SimplexData.txt` output file, which logs the step-by-step progress of the Nelder-Mead optimization. It includes the iteration step, the values of the variables (z1, z2, z3), and the corresponding objective function value (R-factor).

```text
#step z1 z2 z3 R-factor
0 5.25 4.25 3.5 0.015199251773721183
1 5.25 4.25 3.5 0.015199251773721183
2 5.229166666666666 4.3125 3.645833333333333 0.013702918021532375
3 5.225694444444445 4.40625 3.5451388888888884 0.012635279378225261
4 5.179976851851851 4.348958333333334 3.5943287037037033 0.006001660077530159
5 5.179976851851851 4.348958333333334 3.5943287037037033 0.006001660077530159
```

--------------------------------

### Execute ODAT-SE with MPI Parallelization (Bash)

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/input.md

Example command to run ODAT-SE using MPI for parallel computation. Specifies the number of processes to use (e.g., 4).

```bash
mpirun -np 4 odatse input.toml
```

--------------------------------

### Custom Algorithm Abstract Methods in Python

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/customize/algorithm.md

Outlines the abstract methods (_prepare, _run, _post) that must be implemented by custom Algorithm subclasses. It includes an example of how to call the runner to get objective function values within the _run method.

```python
def _prepare(self) -> None:
    # Describes pre-processes of the algorithm.
    pass

def _run(self) -> None:
    # Describes the algorithm body.
    # Example of calling runner:
    # args = (step, set)
    # fx = self.runner.submit(x, args)
    pass

def _post(self) -> dict:
    # Describes post-process of the algorithm.
    pass
```

--------------------------------

### Complete Programmatic Optimization Example (Python)

Source: https://context7.com/issp-center-dev/odat-se/llms.txt

A comprehensive Python script demonstrating how to set up and run an optimization process entirely through code using the ODAT-SE library. It defines a custom objective function, configures the optimization parameters, creates solver and runner instances, and executes the optimization algorithm.

```python
import numpy as np
import odatse
from odatse.solver.function import Solver as FunctionSolver
from odatse.algorithm.min_search import Algorithm as MinSearch

# Define custom objective function
def rosenbrock_nd(x: np.ndarray) -> float:
    """N-dimensional Rosenbrock function, minimum at [1,1,...,1]."""
    return np.sum(100.0 * (x[1:] - x[:-1]**2)**2 + (1 - x[:-1])**2)

# Configuration dictionary
config = {
    "base": {
        "dimension": 4,
        "output_dir": "rosenbrock_output"
    },
    "algorithm": {
        "name": "minsearch",
        "seed": 42,
        "param": {
            "min_list": [-5.0, -5.0, -5.0, -5.0],
            "max_list": [5.0, 5.0, 5.0, 5.0],
            "initial_list": [0.0, 0.0, 0.0, 0.0]
        },
        "minimize": {
            "xatol": 1e-6,
            "fatol": 1e-6,
            "maxiter": 10000
        }
    },
    "solver": {
        "name": "function"
    },
    "runner": {
        "log": {"interval": 50}
    }
}

# Create components
info = odatse.Info(config)
solver = FunctionSolver(info)
solver.set_function(rosenbrock_nd)
runner = odatse.Runner(solver, info)

# Run optimization
algorithm = MinSearch(info, runner, run_mode="initial")
result = algorithm.main()

# Report results
print(f"\n=== Optimization Complete ===")
print(f"Optimal x: {result['x']}")
print(f"Optimal f(x): {result['fx']:.10f}")
print(f"Expected minimum: f([1,1,1,1]) = 0")
print(f"Distance to optimum: {np.linalg.norm(result['x'] - 1.0):.6f}")
```

--------------------------------

### AlgorithmBase Core Methods in Python

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/customize/algorithm.md

Explains the core methods of the AlgorithmBase class: prepare for pre-algorithm setup, run for executing the main algorithm logic, post for post-processing and result collection, and main for orchestrating the entire process with different run modes.

```python
def prepare(self) -> None:
    # Prepares the algorithm.
    pass

def run(self) -> None:
    # Performs the algorithm.
    # Steps include entering proc_dir, running runner.prepare(), _run(), runner.post(), and returning.
    pass

def post(self) -> dict:
    # Runs a post process of the algorithm.
    pass

def main(self, run_mode) -> dict:
    # Calls prepare, run, and post.
    # Measures elapsed times and writes them to a file.
    # Takes run_mode argument: "initialize", "resume", "continue".
    pass
```

--------------------------------

### AlgorithmBase Initialization Parameters in Python

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/customize/algorithm.md

Details the initialization parameters for the AlgorithmBase class, including MPI communicator setup, random number generator, parameter space dimensions, directory paths, and checkpointing configurations.

```python
def __init__(self, info: odatse.Info, runner: odatse.Runner = None, mpicomm: Optional["MPI.Comm"] = None):
    # Reads the common parameters from info and sets the following instance variables:
    # self.mpicomm: Optional[MPI.Comm]
    # self.mpisize: int
    # self.mpirank: int
    # self.rng: np.random.Generator
    # self.dimension: int
    # self.label_list: list[str]
    # self.root_dir: pathlib.Path
    # self.output_dir: pathlib.Path
    # self.proc_dir: pathlib.Path
    # self.timer: dict[str, dict]
    # self.checkpoint: bool
    pass
```

--------------------------------

### Checkpointing and Resume Commands (Bash)

Source: https://context7.com/issp-center-dev/odat-se/llms.txt

Bash commands for managing ODAT-SE simulations with checkpointing. Shows how to start an initial run, resume an interrupted run from a checkpoint file, and continue a run with or without resetting the random seed.

```bash
# Start initial run
odatse input_checkpoint.toml

# Resume interrupted run (loads from status.pickle)
odatse --resume input_checkpoint.toml

# Continue with same parameters
odatse --cont input_checkpoint.toml

# Continue with new random seed
odatse --cont --reset_rand input_checkpoint.toml
```

--------------------------------

### Prepare Input Data (Text)

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/tutorial/linreg_with_noise.md

This is an example of a data file format expected by the ODAT-SE analysis script. It contains two columns of numerical data, representing x and y values, separated by whitespace. Each line corresponds to a data point.

```text
1.0 1.1
2.0 1.9
3.0 3.1
4.0 4.2
5.0 4.8
6.0 6.1
```

--------------------------------

### Example Output: res.txt

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/algorithm/minsearch.md

This is an example of the `res.txt` output file, which contains the final results of the Nelder-Mead optimization. It lists the minimum objective function value (`fx`) followed by the corresponding parameter values (z1, z2, z3) at that minimum.

```text
fx = 7.382680568652868e-06
z1 = 5.230524973874179
z2 = 4.370622919269477
z3 = 3.5961444501081647
```

--------------------------------

### Example time.log Output

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/output.md

This snippet shows an example of the `time.log` file format. It details the time spent on different stages of a calculation (prepare, run, post) for each MPI rank. The `run` section further breaks down time spent on file I/O and computation.

```default
#prepare
 total = 0.007259890999989693
#run
 total = 1.3493346729999303
 - file_CM = 0.0009563499997966574  # Time spent on file I/O
 - submit = 1.3224223930001244      # Time spent on calculation processing
#post
 total = 0.000595873999941432
```

--------------------------------

### Custom Algorithm Initialization in Python

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/customize/algorithm.md

Demonstrates the initialization of a custom Algorithm subclass, showing how to pass arguments to the base class constructor and how to handle the search region definition using either a provided domain or by creating one from info.

```python
def __init__(self, info: odatse.Info, runner: odatse.Runner = None, domain = None):
    super().__init__(info=info, runner=runner)
    # Reads info and sets information.
    # If domain is given, use it. Otherwise, create region or meshgrid from info.
    pass
```

--------------------------------

### AlgorithmBase Initialization in Python

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/customize/algorithm.md

Illustrates the initialization of the AlgorithmBase class in Python. It details the parameters accepted by the constructor, such as info, runner, and mpicomm, and explains how instance variables like mpicomm, mpisize, mpirank, rng, dimension, label_list, root_dir, output_dir, proc_dir, timer, and checkpoint settings are configured.

```python
def __init__(self, info: odatse.Info, runner: odatse.Runner = None, mpicomm: Optional["MPI.Comm"] = None):
    # Reads the common parameters from info and sets instance variables
    # ... (details of variable assignments)
    pass
```

--------------------------------

### Run ODAT-SE Analysis via Script

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/start.md

Executes an ODAT-SE analysis by directly running the main Python script. This is useful for development or when the 'odatse' command is not installed.

```bash
python3 src/odatse_main.py input.toml
```

--------------------------------

### Uninstall ODAT-SE

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/start.md

Removes the ODAT-SE module from the Python environment. Optional packages can be uninstalled similarly.

```bash
python3 -m pip uninstall ODAT-SE
```

--------------------------------

### AlgorithmBase Methods in Python

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/customize/algorithm.md

Demonstrates the core methods of the AlgorithmBase class in Python: prepare, run, post, and main. The 'prepare' method is for pre-processing, 'run' executes the main algorithm logic including runner interactions, 'post' handles finalization and result writing, and 'main' orchestrates the entire process based on the run_mode.

```python
def prepare(self) -> None:
    # Prepares the algorithm.
    pass

def run(self) -> None:
    # Performs the algorithm.
    # ... (steps: enter proc_dir, run runner.prepare, _run, runner.post, move back)
    pass

def post(self) -> dict:
    # Runs a post process of the algorithm.
    # ... (enters output_dir, calls _post, returns to original directory)
    pass

def main(self, run_mode) -> dict:
    # Calls prepare, run, and post.
    # Measures elapsed times and writes them to a file.
    # run_mode can be 'initialize', 'resume', or 'continue'.
    pass
```

--------------------------------

### Mesh Definition File Format

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/algorithm/mapper_mpi.md

Example of a mesh definition file used to specify candidate points for the parameter space. Each line represents a point, starting with its index followed by its coordinates. Lines starting with '#' are comments.

```text
1 6.000000 6.000000
2 6.000000 5.750000
3 6.000000 5.500000
4 6.000000 5.250000
5 6.000000 5.000000
6 6.000000 4.750000
7 6.000000 4.500000
8 6.000000 4.250000
9 6.000000 4.000000
...
```

--------------------------------

### Run ODAT-SE Analysis

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/en/source/start.md

Executes an ODAT-SE analysis using an input TOML configuration file. This command-line interface is the primary way to run analyses.

```bash
odatse input.toml
```

--------------------------------

### Example runner.log Output

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/output.md

This snippet illustrates the format of the `runner.log` file, which records information about solver calls for each MPI rank. It includes the serial number of the call, time elapsed since the last call, and time elapsed since the start of the calculation. This log is generated when `runner.log.interval` is a positive integer.

```default
# $1: num_calls
# $2: elapsed_time_from_last_call
# $3: elapsed_time_from_start

1 0.0010826379999999691 0.0010826379999999691
2 6.96760000000185e-05 0.0011523139999999876
3 9.67080000000009e-05 0.0012490219999999885
4 0.00011765699999999324 0.0013666789999999818
5 4.965899999997969e-05 0.0014163379999999615
6 8.666900000003919e-05 0.0015030070000000006
   ...
```

--------------------------------

### Install Scipy for Nelder-Mead

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/algorithm/minsearch.md

This command installs the Scipy library, which is a dependency for using the Nelder-Mead optimization method in ODAT-SE. Ensure you have Python 3 and pip installed.

```bash
python3 -m pip install scipy
```

--------------------------------

### Install PHYSBO for Bayesian Optimization

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/algorithm/bayes.md

This command installs the PHYSBO library, which is a dependency for the `bayes` Algorithm. PHYSBO provides the core functionality for Bayesian optimization. MPI parallel computing is available if mpi4py is also installed.

```bash
python3 -m pip install physbo
```

--------------------------------

### Solver Constructor and Instance Variables (Python)

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/customize/solver.md

Demonstrates the constructor for the Solver class, which initializes base class attributes and sets up essential instance variables like root directory, output directory, and process directory. It highlights how these are derived from the provided 'info' object.

```python
def __init__(self, info: odatse.Info):
    super().__init__(info)
    # self.root_dir: pathlib.Path
    # self.output_dir: pathlib.Path
    # self.proc_dir: pathlib.Path
```

--------------------------------

### Install mpi4py for MPI Parallelism

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/api/source/algorithm/mapper_mpi.md

Installs the mpi4py library, which is required for MPI parallelism with the mapper_mpi algorithm. This command should be run in a Python environment.

```bash
python3 -m pip install mpi4py
```

--------------------------------

### Main Script for ODAT-SE with Argument Parsing in Python

Source: https://github.com/issp-center-dev/odat-se/blob/main/doc/ja/source/tutorial/gpr.md

The main function of the sample script parses command-line arguments for an input TOML file, optional log file, and training/testing dataset sizes. It initializes ODAT-SE, generates data, trains a surrogate model, evaluates its performance, and runs the ODAT-SE solver, logging output to a file. Dependencies include argparse, sys, os, numpy, and ODAT-SE modules.

```python
if __name__ == "__main__":
        parser = argparse.ArgumentParser()
        parser.add_argument("--input", type=str, default="input.toml",
                                           help="ODAT-SE input configuration file path")
        parser.add_argument("--logfile", type=str, default=None,
                                           help="ODAT-SE run log file (default: odatse_run.log)")
        parser.add_argument("--ntrain", type=int, default=500,
                                           help="number of training data points (default: 500)")
        parser.add_argument("--ntest", type=int, default=500,
                                           help="number of test data points (default: 500)")
        args = parser.parse_args()

        # Initialize ODAT-SE to get output directory
        sys.argv = ["script.py", args.input, "--init"]
        info, run_mode = odatse.initialize()
        output_dir = info.base.get("output_dir", "./output")

        # Create output directory if it doesn't exist
        os.makedirs(output_dir, exist_ok=True)

        # Set log file path
        if args.logfile is None:
                args.logfile = os.path.join(output_dir, "odatse_run.log")

        np.random.seed(info.algorithm["seed"])
        xrange, yrange = zip(info.algorithm["param"]["min_list"],info.algorithm["param"]["max_list"])

        # Data
        X, t = make_data(himmelblau, args.ntrain + args.ntest, xrange, yrange, 0.01)

        # Split
        Xtr, ttr, Xte, _ = split_data(X, t, args.ntrain, args.ntest)
        conv = Converter(Xtr, ttr)
        Xtr_s = conv.convert_X(Xtr)
        ttr_s = conv.convert_t(ttr)

        predictor_raw = gplearn(Xtr_s, ttr_s, info.algorithm["seed"])
        predictor = conv.gen_predictor(predictor_raw)

        t_pred = predictor(Xte)
        t_true = himmelblau(Xte[:, 0], Xte[:, 1]).astype(np.float32)
        mse = float(np.mean((t_pred - t_true) ** 2))
        var_true = float(np.var(t_true))
        nmse = mse / (var_true + 1e-12)
        print(f"MSE (n_test={len(t_true)}): {mse:.6e}")
        print(f"NMSE: {nmse:.6e}")

        # Plot output
        plot_res(Xtr, predictor)

        original_stdout = sys.stdout
        original_stderr = sys.stderr

        # Run ODAT-SE
        with open(args.logfile, "w") as f:
                sys.stdout = f
                sys.stderr = f

                solver = PredictorSolver(info, predictor=predictor)
                runner = odatse.Runner(solver, info)
                alg_module = odatse.algorithm.choose_algorithm(info.algorithm["name"])
                alg = alg_module.Algorithm(info, runner, run_mode=run_mode)
                result = alg.main()

                sys.stdout = original_stdout
                sys.stderr = original_stderr
        print(f"Best solution: x^* = {result['x']}")
        print(f"Surrogate f(x^*) = {result['fx']}")
        print(f"True f(x^*) = {himmelblau(*tuple(result['x']))}")
```