### Starting the luigid central scheduler (local installation)

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/run_modes.md

If luigid is installed locally, use the path ~/.local/bin/luigid to start the central scheduler, specifying the desired port.

```bash
~/.local/bin/luigid --port PORT
```

--------------------------------

### Example Environment Setup Script

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/batch.md

Use this bash script to set up your virtual environment and export custom settings before a batch job runs. It's useful when the batch system doesn't support environment copying.

```bash
# Source my virtual environment
source venv/bin/activate
# Set some specific settings
export MY_IMPORTANT_SETTING 10
```

--------------------------------

### Install Requirements from File

Source: https://github.com/belle2/b2luigi/blob/main/examples/tutorial/README.rst

Install Python package requirements directly from the requirements.txt file. This is applicable if you have copied the examples from the Git repository.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install Publishing Dependencies

Source: https://github.com/belle2/b2luigi/blob/main/docs/development.md

Install necessary tools like setuptools, wheel, and twine for manually publishing a release to PyPI.

```bash
python -m pip install -U [ --user ] setuptools wheel twine
```

--------------------------------

### Install Tutorial Requirements

Source: https://github.com/belle2/b2luigi/blob/main/examples/tutorial/README.rst

Install all necessary Python packages for the tutorial, including b2luigi, pandas, pyarrow, uproot, matplotlib, and plothist. Use this command for a clean virtual environment.

```bash
pip install b2luigi pandas pyarrow uproot matplotlib plothist
```

--------------------------------

### Install Flit Builder

Source: https://github.com/belle2/b2luigi/blob/main/docs/development.md

Install the flit build tool, which is used instead of setuptools for building the b2luigi package.

```bash
pip3 [ --user ] install flit
```

--------------------------------

### Clone b2luigi Repository

Source: https://github.com/belle2/b2luigi/blob/main/examples/tutorial/README.rst

Clone the b2luigi repository from GitLab and navigate to the examples directory. This is the recommended way to obtain the example files.

```bash
git clone https://gitlab.desy.de/belle2/software/b2luigi.git
cd b2luigi/examples
```

--------------------------------

### Install b2luigi in Development Mode

Source: https://github.com/belle2/b2luigi/blob/main/docs/development.md

Install the cloned b2luigi package in development mode using flit, ensuring changes are immediately available.

```bash
flit install -s -deps=develop
```

--------------------------------

### Slurm Batch Processing Example

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/batch.md

This example shows how to configure b2luigi tasks to run on a Slurm cluster. It specifies Slurm-specific settings like 'export' and memory requirements directly within the task definition. Use this when targeting a Slurm environment.

```python
import b2luigi
import random


class MyNumberTask(b2luigi.Task):
    some_parameter = b2luigi.IntParameter()
    batch_system = "slurm"
    slurm_settings = {"export": "NONE", "ntasks": 1, "mem": "100MB"}

    def output(self):
        yield self.add_to_output("output_file.txt")

    def run(self):
        print("I am now starting a task")
        random_number = random.random()

        with open(self.get_output_file_name("output_file.txt"), "w") as f:
            f.write(f"{random_number}\n")


class MyAverageTask(b2luigi.Task):
    batch_system = "slurm"
    slurm_settings = {"export": "NONE", "ntasks": 1, "mem": "100MB"}

    def requires(self):
        for i in range(10):
            yield self.clone(MyNumberTask, some_parameter=i)

    def output(self):
        yield self.add_to_output("average.txt")

    def run(self):
        print("I am now starting the average task")

        # Build the mean
        summed_numbers = 0
        counter = 0
        for input_file in self.get_input_file_names("output_file.txt"):
            with open(input_file, "r") as f:
                summed_numbers += float(f.read())
                counter += 1

        average = summed_numbers / counter

        with open(self.get_output_file_name("average.txt"), "w") as f:
            f.write(f"{average}\n")


if __name__ == "__main__":
    b2luigi.process(MyAverageTask(), workers=200, batch=True)
```

--------------------------------

### Install b2luigi using pip

Source: https://github.com/belle2/b2luigi/blob/main/docs/installation.md

Install the b2luigi package from PyPI into your active Python virtual environment. This command fetches and installs the latest stable version.

```bash
pip3 install b2luigi
```

--------------------------------

### Setup BASF2 Environment Script

Source: https://github.com/belle2/b2luigi/blob/main/docs/examples/basf2-examples.md

Use this script to set up the BASF2 environment on worker nodes. Ensure the release version is correct.

```bash
source /cvmfs/belle.cern.ch/tools/b2setup release-XX-XX-XX
```

--------------------------------

### Starting the luigid central scheduler

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/run_modes.md

Execute the luigid command to start the central scheduler. The --port argument specifies the port to use.

```bash
luigid --port PORT
```

--------------------------------

### Install b2luigi Package

Source: https://github.com/belle2/b2luigi/blob/main/examples/tutorial/README.rst

Install the b2luigi package within the activated virtual environment. This command is used when the virtual environment is created by b2venv.

```bash
pip install b2luigi
```

--------------------------------

### run()

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_core.md

Executes the batch process. This method logs the start of the batch process, including the class name and associated task, and then initiates the job by calling `start_job`.

```APIDOC
## run()

### Description
Executes the batch process.

This method logs the start of the batch process, including the class name and associated task, and then initiates the job by calling `start_job`.
```

--------------------------------

### start_job

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Starts a job by creating and submitting an HTCondor submit file.

```APIDOC
## start_job()

### Description
Starts a batch job by generating an HTCondor submit file and submitting it using the `condor_submit` command.

### Method
instance

### Raises
**RuntimeError** – If the batch submission fails or the job ID cannot be extracted from the `condor_submit` output.
```

--------------------------------

### Install and Run Pre-commit Hooks

Source: https://github.com/belle2/b2luigi/blob/main/docs/development.md

Install pre-commit hooks to automatically check code for syntax and style errors using tools like ruff and PEP 8 compliance.

```bash
pre-commit install
```

```bash
pre-commit run --all-files
```

--------------------------------

### start_job()

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Starts a job by submitting the Slurm submission script. This method creates a Slurm submit file and submits it using the `sbatch` command, then parses the output to extract the batch job ID.

```APIDOC
## start_job()

### Description
Starts a job by submitting the Slurm submission script. This method creates a Slurm submit file and submits it using the `sbatch` command. After submission, it parses the output to extract the batch job ID.

### Raises
**RuntimeError** – If the batch submission fails or the job ID cannot be extracted.
```

--------------------------------

### Start Batch Job

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Starts a job by creating and submitting an HTCondor submit file using `_create_htcondor_submit_file` and then submitting the job with `condor_submit`. Raises RuntimeError if submission fails or job ID cannot be extracted.

```python
start_job()
```

--------------------------------

### HTCondor Batch Processing Example

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/batch.md

This example demonstrates how to define tasks for batch processing using HTCondor. It includes task dependencies, output generation, and HTCondor-specific settings like resource requests. Use this for running tasks on an HTCondor cluster.

```python
import b2luigi
import random


class MyNumberTask(b2luigi.Task):
    some_parameter = b2luigi.IntParameter()

    htcondor_settings = {"request_cpus": 1, "request_memory": "100 MB"}

    def output(self):
        yield self.add_to_output("output_file.txt")

    def run(self):
        print("I am now starting a task")
        random_number = random.random()

        if self.some_parameter == 3:
            raise ValueError

        with open(self.get_output_file_name("output_file.txt"), "w") as f:
            f.write(f"{random_number}\n")


class MyAverageTask(b2luigi.Task):
    htcondor_settings = {"request_cpus": 1, "request_memory": "200 MB"}

    def requires(self):
        for i in range(10):
            yield self.clone(MyNumberTask, some_parameter=i)

    def output(self):
        yield self.add_to_output("average.txt")

    def run(self):
        print("I am now starting the average task")

        # Build the mean
        summed_numbers = 0
        counter = 0
        for input_file in self.get_input_file_names("output_file.txt"):
            with open(input_file, "r") as f:
                summed_numbers += float(f.read())
                counter += 1

        average = summed_numbers / counter

        with open(self.get_output_file_name("average.txt"), "w") as f:
            f.write(f"{average}\n")


if __name__ == "__main__":
    b2luigi.process(MyAverageTask(), workers=200, batch=True)
```

--------------------------------

### Execute Simple Example Task via Bash

Source: https://github.com/belle2/b2luigi/blob/main/docs/quickstart.md

Executes the 'simple-example.py' script, submitting tasks to the LSF batch system. This command initiates the parallel processing of tasks defined in the Python script.

```bash
python simple-example.py --batch
```

--------------------------------

### Setup DIRAC Proxy

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Ensures a valid DIRAC proxy is initialized for the Belle II grid system. It checks for an existing proxy with sufficient lifetime and initializes a new one if necessary using `gb2_proxy_init`.

```APIDOC
## setup_dirac_proxy

### Description
Ensures that a valid DIRAC proxy is initialized for the Belle II grid system. If a proxy is already active and has sufficient remaining lifetime, no action is taken. Otherwise, the function initializes a new proxy using the `gb2_proxy_init` command.

### Parameters
* **gbasf2_setup_path** (*str*) – Path to the gbasf2 setup script. Defaults to “/cvmfs/belle.kek.jp/grid/gbasf2/pro/bashrc”.
* **task** (*optional*) – Task-specific context or configuration, if applicable.

### Behavior
- Checks if an active proxy exists and has sufficient lifetime remaining.
- If no valid proxy exists, initializes a new proxy using the `gb2_proxy_init` command.
- Prompts the user for the certificate password during proxy initialization.
- Validates the proxy initialization process and handles errors such as incorrect passwords or other initialization issues.

### Raises
**RuntimeWarning** – If the `gbasf2_proxy_lifetime` setting is not a positive integer.
```

--------------------------------

### Remove Output Directory

Source: https://github.com/belle2/b2luigi/blob/main/docs/quickstart.md

Before running the example, remove the previous output to ensure a clean run.

```bash
rm -rf results
```

--------------------------------

### Minimal gbasf2 Task Configuration

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/batch.md

This example shows the essential settings for a task to run on the gbasf2/grid batch system. Ensure `batch_system` is set to 'gbasf2' and provide necessary input dataset and project name prefix parameters.

```python
class MyTask(Basf2PathTask):
    batch_system = "gbasf2"
    gbasf2_project_name_prefix = b2luigi.Parameter(significant=False)
    gbasf2_input_dataset = b2luigi.Parameter(hashed=True)
```

--------------------------------

### start_job()

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_core.md

Override this function in your child class to start a job on the batch system. It is called exactly once. You need to store any information identifying your batch job on your own.

```APIDOC
## start_job()

### Description
Override this function in your child class to start a job on the batch system. It is called exactly once. You need to store any information identifying your batch job on your own.

You can use the `b2luigi.core.utils.get_log_file_dir` and the `b2luigi.core.executable.create_executable_wrapper` functions to get the log base name and to create the executable script which you should call in your batch job.

After the `start_job` function is called by the framework (and no exception is thrown), it is assumed that a batch job is started or scheduled.

After the job is finished (no matter if aborted or successful) we assume the stdout and stderr is written into the two files given by `b2luigi.core.utils.get_log_file_dir(self.task)`.
```

--------------------------------

### Define a Task with Output Generation

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/tasks.md

Example of a b2luigi.Task subclass demonstrating how to define requirements, output files using add_to_output, and the run logic for processing inputs and writing output.

```python
class MyAverageTask(b2luigi.Task):
    def requires(self):
        for i in range(100):
            yield self.clone(MyNumberTask, some_parameter=i)

    def output(self):
        yield self.add_to_output("average.txt")

    def run(self):
        # Build the mean
        summed_numbers = 0
        counter = 0
        for input_file in self.get_input_file_names("output_file.txt"):
            with open(input_file, "r") as f:
                summed_numbers += float(f.read())
                counter += 1

        average = summed_numbers / counter

        with self.get_output_file_name("average.txt").open("w") as f:
            f.write(f"{average}\n")
```

--------------------------------

### get_gbasf2_env

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Retrieves the gbasf2 environment variables as a dictionary, suitable for executing gbasf2 commands. It sources the specified setup file in a new shell.

```APIDOC
## get_gbasf2_env(gbasf2_setup_path, task=None)

### Description
Retrieve the `gbasf2` environment as a dictionary, which can be used to run `gbasf2` commands.

This function sets up the `gbasf2` environment by sourcing the specified setup file in a fresh shell
and capturing the resulting environment variables. It ensures that the environment is isolated,
except for the `HOME` variable, which is required for the setup process.

### Parameters
* **gbasf2_setup_path** (*str*) – Path to the gbasf2 setup file.
* **task** (*optional*) – Task parameter used to retrieve the `gbasf2_proxy_group` setting. Defaults to `"belle"` if not provided.

### Returns
A dictionary containing the environment variables set up by the gbasf2 setup file.

### Return type
dict

### Raises
**FileNotFoundError** – If the specified `gbasf2` setup file does not exist.
```

--------------------------------

### Basic b2luigi Task Example

Source: https://github.com/belle2/b2luigi/blob/main/docs/index.md

This snippet demonstrates how to define a simple Luigi task using b2luigi. It shows how to specify an output file and write content to it. To run this task in batch mode, use `b2luigi.process(MyTask(), batch=True)`.

```python
import b2luigi


class MyTask(b2luigi.Task):
    def output(self):
        return self.add_to_output("output_file.txt")

    def run(self):
        with open(self.get_output_file_name("output_file.txt"), "w") as f:
            f.write("This is a test\n")


if __name__ == "__main__":
    b2luigi.process(MyTask(), batch=True)
```

--------------------------------

### Uninstall b2luigi from PyPI

Source: https://github.com/belle2/b2luigi/blob/main/docs/development.md

Ensure a clean environment before local development by uninstalling any existing PyPI installation of b2luigi.

```bash
python -m pip uninstall b2luigi
```

--------------------------------

### Example Task dry_run Method

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/tasks.md

Demonstrates how to implement the dry_run method in a b2luigi Task to print output file names without executing the full task logic. This is useful for simulating task runs and inspecting expected outputs.

```python
class TheSuperFancyTask(b2luigi.Task):
    def dry_run(self):
        for name in self.get_all_output_file_names():
            print(f"              output: {name}")
```

--------------------------------

### Submit Analysis Path to Grid with gbasf2

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Example of a Luigi task that defines an analysis path and submits it to the grid using the gbasf2 batch system. Configure `batch_system` to 'gbasf2' and provide a `gbasf2_project_name_prefix`. The `create_path` method defines the analysis, and `output` specifies the generated files. The `AnalysisWrapperTask` demonstrates how to require multiple `AnalysisTask` instances with different parameters, each submitting a separate gbasf2 project.

```python
import b2luigi
from b2luigi.basf2_helper.tasks import Basf2PathTask

import example_mdst_analysis


class AnalysisTask(Basf2PathTask):
    # set the batch_system property to use the gbasf2 wrapper batch process for this task
    batch_system = "gbasf2"
    # Must define a prefix for the gbasf2 project name to submit to the grid.
    # b2luigi will then add a hash derived from the luigi parameters to create a unique project name.
    gbasf2_project_name_prefix = b2luigi.Parameter()
    gbasf2_input_dataset = b2luigi.Parameter(hashed=True)
    # Example luigi cut parameter to facilitate starting multiple projects for different cut values
    mbc_lower_cut = b2luigi.IntParameter()

    def create_path(self):
        mbc_range = (self.mbc_lower_cut, 5.3)
        return example_mdst_analysis.create_analysis_path(
            d_ntuple_filename="D_ntuple.root", b_ntuple_filename="B_ntuple.root", mbc_range=mbc_range
        )

    def output(self):
        yield self.add_to_output("D_ntuple.root")
        yield self.add_to_output("B_ntuple.root")


class AnalysisWrapperTask(b2luigi.WrapperTask):
    """
    We use the AnalysisWrapperTask to be able to require multiple analyse tasks with
    different input datasets and cut values. For each parameter combination, a
    different gbasf2 project will be submitted.
    """

    def requires(self):
        input_dataset = (
            "/belle/MC/release-04-01-04/DB00000774/SkimM13ax1/prod00011778/e1003/4S/r00000/mixed/11180100/udst/sub00/"
            "udst_000006_prod00011778_task10020000006.root"
        )
        # if you want to iterate over different cuts, just add more values to this list
        mbc_lower_cuts = [5.15, 5.2]
        for mbc_lower_cut in mbc_lower_cuts:
            yield AnalysisTask(
                mbc_lower_cut=mbc_lower_cut,
                gbasf2_project_name_prefix="luigiExample",
                gbasf2_input_dataset=input_dataset,
                max_event=100,
            )


if __name__ == "__main__":
    main_task_instance = AnalysisWrapperTask()
    n_gbasf2_tasks = len(list(main_task_instance.requires()))
    b2luigi.process(main_task_instance, workers=n_gbasf2_tasks)
```

--------------------------------

### Example Batch Task Command

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_core.md

Illustrates a typical command structure for executing a single Luigi task within a batch system. This command includes the path to the Python executable, the Luigi script, and a specific flag to indicate batch runner mode along with the task ID.

```default
/<path-to-your-exec>/python /your-project/some-file.py --batch-runner --task-id MyTask_38dsf879w3
```

--------------------------------

### Define Task Requirements and Process Inputs

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/tasks.md

Define task dependencies using the 'requires' method and process input files using 'get_input_file_names_from_dict'. This example shows how to handle multiple input sources ('a' and 'b') and combine their results.

```python
class TaskB(luigi.Task):

    def requires(self):
        return {
            "a": (TaskA(5.0, i) for i in range(100)),
            "b": (TaskA(1.0, i) for i in range(100)),
        }

    def run(self):
        result_a = do_something_with_a(
            self.get_input_file_names_from_dict("a")
        )
        result_b = do_something_with_b(
            self.get_input_file_names_from_dict("b")
        )

        combine_a_and_b(
            result_a,
            result_b,
            self.get_output_file_name("combined_results.txt")
        )

    def output(self):
        yield self.add_to_output("combined_results.txt")
```

--------------------------------

### Set Up b2luigi Virtual Environment

Source: https://github.com/belle2/b2luigi/blob/main/examples/tutorial/README.rst

Set up a virtual environment using b2setup and b2venv for the b2luigi tutorial. This ensures the correct Belle II software stack and necessary packages are available.

```bash
source /cvmfs/belle.cern.ch/tools/b2setup
b2venv release-09-00-00
```

--------------------------------

### Implement Get Job Status

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Implement this function to return the current job status (running, aborted, successful, or idle). It should be called after the job starts and may be called when the job is finished. Returns aborted if the task status is unknown.

```python
get_job_status()
```

--------------------------------

### terminate_job()

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_core.md

This command is used to abort a job started by the `start_job` function. It is only called once to abort a job, so make sure to either block until the job is really gone or be sure that it will go down soon. Especially, do not wait until the job is finished. It is called for example when the user presses `Ctrl-C`.

```APIDOC
## terminate_job()

### Description
This command is used to abort a job started by the `start_job` function. It is only called once to abort a job, so make sure to either block until the job is really gone or be sure that it will go down soon. Especially, do not wait until the job is finished. It is called for example when the user presses `Ctrl-C`.

In some strange corner cases it may happen that this function is called even before the job is started (the `start_job` function is called). In this case, you do not need to do anything (but also not raise an exception).
```

--------------------------------

### start_job()

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Submits a new gbasf2 project to the grid after checking for existence and preparing steering files. Raises an OSError for symlink issues.

```APIDOC
## start_job()

### Description
Submit a new gbasf2 project to the grid. This method checks if a project with the specified name already exists on the grid. If it does, the submission is skipped, and a message is printed. Otherwise, it prepares the necessary steering files and submits the project.

### Steps
: 1. Check if the project already exists on the grid. (see [`check_project_exists`](#b2luigi.batch.processes.gbasf2.check_project_exists))
  2. Prepare the steering file:
     : - If a custom steering file is provided, copy it. (see [`_copy_custom_steering_script`](#b2luigi.batch.processes.gbasf2.Gbasf2Process._copy_custom_steering_script))
       - Otherwise, create a wrapper steering file. (see: [`_create_wrapper_steering_file`](#b2luigi.batch.processes.gbasf2.Gbasf2Process._create_wrapper_steering_file))
  3. Build the gbasf2 submission command. (see [`_build_gbasf2_submit_command`](#b2luigi.batch.processes.gbasf2.Gbasf2Process._build_gbasf2_submit_command))
  4. Create a symlink for the pickle file to ensure it is included in the grid input sandbox.
  5. Submit the project using the `gbasf2` command. (see [`run_with_gbasf2`](#b2luigi.batch.processes.gbasf2.run_with_gbasf2))
  6. Clean up the symlink after submission.

### Raises
**OSError** – If there is an issue creating or removing the symlink.

### NOTE
If the project already exists, the user is advised to change the project name to submit a new project if needed.
```

--------------------------------

### b2luigi.process

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/top_level_functions.md

Call this function in your main method to tell `b2luigi` where your entry point of the task graph is. It is very similar to `luigi.build` with some additional configuration options.

```APIDOC
## b2luigi.process(task_like_elements, show_output=False, dry_run=False, test=False, batch=False, remove=[], remove_only=[], auto_confirm=False, keep_tasks=None, ignore_additional_command_line_args=False, **kwargs)

### Description
Call this function in your main method to tell `b2luigi` where your entry point of the task graph is. It is very similar to `luigi.build` with some additional configuration options.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **task_like_elements** ([`Task`](tasks.md#b2luigi.Task) or list) – Task(s) to execute with luigi. Can either be a list of tasks or a task instance.
* **show_output** (*bool* *,* *optional*) – Instead of running the task(s), write out all output files which will be generated marked in color, if they are present already. Good for testing of your tasks will do, what you think they should.
* **dry_run** (*bool* *,* *optional*) – Instead od running the task(s), write out which tasks will be executed. This is a simplified form of dependency resolution, so this information may be wrong in some corner cases. Also good for testing.
* **test** (*bool* *,* *optional*) – Does neither run on the batch system, with multiprocessing or dispatched (see [`DispatchableTask`](tasks.md#b2luigi.DispatchableTask)) but directly on the machine for debugging reasons. Does output all logs to the console.
* **batch** (*bool* *,* *optional*) – If set to False, the global settings of batch_system will be set to local. If set to True, task with no batch_system set, will be executed with the globally set batch_system. Refer to [Quick Start](../quickstart.md#quick-start-label) for more information. By default, the global batch system uses the auto setting, but this can be changed with the batch_system settings. See [`get_setting`](settings.md#b2luigi.core.settings.get_setting) on how to define settings.
* **remove** (*list* *,* *optional*) – If a single task is given, remove the output of this task. If a list of tasks is given, remove the output of all tasks in the list.
* **ignore_additional_command_line_args** (*bool* *,* *optional* *,* *default False*) – Ignore additional command line arguments. This is useful if you want to use this function in a file that also does some command line parsing.
* **&#x20&#x20kwargs** – Additional keyword arguments passed to `luigi.build`.

### WARNING
You should always have just a single call to `process` in your script. If you need to have multiple calls, either use a [`b2luigi.WrapperTask`](tasks.md#b2luigi.WrapperTask) or two scripts.
```

--------------------------------

### Settings Management

Source: https://github.com/belle2/b2luigi/blob/main/docs/api.md

Functions for getting, setting, and clearing B2Luigi settings.

```APIDOC
## get_setting()

### Description
Retrieves a B2Luigi setting.

### Method
`get_setting()`

### Parameters
None explicitly documented in this section.

### Request Example
```python
import b2luigi

setting_value = b2luigi.core.settings.get_setting('setting_name')
```

### Response
None explicitly documented in this section.
```

```APIDOC
## set_setting()

### Description
Sets a B2Luigi setting.

### Method
`set_setting()`

### Parameters
None explicitly documented in this section.

### Request Example
```python
import b2luigi

b2luigi.core.settings.set_setting('setting_name', 'setting_value')
```

### Response
None explicitly documented in this section.
```

```APIDOC
## clear_setting()

### Description
Clears a B2Luigi setting.

### Method
`clear_setting()`

### Parameters
None explicitly documented in this section.

### Request Example
```python
import b2luigi

b2luigi.core.settings.clear_setting('setting_name')
```

### Response
None explicitly documented in this section.
```

--------------------------------

### Build Documentation Locally

Source: https://github.com/belle2/b2luigi/blob/main/docs/development.md

Build the project documentation locally using sphinx-autobuild to preview changes before creating a pull request.

```bash
sphinx-autobuild docs build
```

--------------------------------

### Using RemoteTarget with Temporary Input

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/remote_targets.md

Demonstrates how to create a temporary local copy of a remote input file for processing. The temporary file is automatically cleaned up after use.

```python
target = RemoteTarget("remote://server/path/data.root", fs)
with target.get_temporary_input() as tmp_input:
    process_local_file(tmp_input)
# Temporary file is automatically cleaned up.
```

--------------------------------

### Task with RemoteTarget and temporary_path

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/remote_targets.md

Demonstrates how to use RemoteTarget with a temporary path for staging files before uploading to remote storage. Requires importing XRootDSystem and RemoteTarget.

```python
from b2luigi import XRootDSystem, RemoteTarget
from b2luigi.core.utils import create_output_filename
import b2luigi

class MyTask(b2luigi.Task):
    def run(self):
        file_name = "Hello_world.txt"
        target = self._get_output_target(file_name)
        with target.temporary_path() as temp_path:
            with open(temp_path, "w") as f:
                f.write("Hello World")

    def output(self):
        fs = XRootDSystem("root://eospublic.cern.ch")
        yield self.add_to_output("Hello_world.txt", RemoteTarget, file_system=fs)
```

--------------------------------

### Equivalent Command Line Call for Batch Processing

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/top_level_functions.md

This demonstrates how to achieve the same result as calling b2luigi.process with the batch=True argument by using command-line arguments.

```default
b2luigi.process(tasks, batch=True)
```

```default
python script.py --batch
```

--------------------------------

### Activate Virtual Environment

Source: https://github.com/belle2/b2luigi/blob/main/docs/installation.md

Activate the created virtual environment to ensure subsequent commands use the isolated Python installation. This command is for Unix-like systems.

```bash
source venv/bin/activate
```

--------------------------------

### Create and Activate Python Virtual Environment

Source: https://github.com/belle2/b2luigi/blob/main/docs/development.md

Set up a dedicated Python virtual environment for development using venv and activate it.

```bash
python3 -m venv venv
```

```bash
source venv/bin/activate
```

--------------------------------

### Get Filled Task Parameters

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/utilities.md

Retrieves a dictionary of parameter names and their values from a task. The task must have a `get_params` method and attributes corresponding to the parameter names.

```python
from b2luigi.core.utils import get_filled_params

# Assume 'task' is a task object
# params_dict = get_filled_params(task)
# print(params_dict)
```

--------------------------------

### Custom Hash Function for Dict Parameter

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/parameters.md

Provide a custom `hash_function` to a DictParameter to control how its value is hashed for the output name. This example uses the 'name' key from the dictionary.

```python
class MyTask(b2lujson.Task):
    my_parameter = b2luigi.DictParameter(hashed=True, hash_function=lambda x: x["name"])
```

--------------------------------

### Define an Apptainer Task

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/apptainer.md

Example of how to define a Luigi task that utilizes the ApptainerProcess for containerized execution. Configure the Apptainer image, mounts, and additional parameters as class attributes.

```python
class MyApptainerTask(luigi.Task):
    apptainer_image = "/cvmfs/belle.cern.ch/images/belle2-base-el9"
    apptainer_mounts = ["/cvmfs"]
    apptainer_mount_defaults = True
    apptainer_additional_params = "--cleanenv"

    <rest of the task definition>
```

--------------------------------

### Set Additional gbasf2 Parameters

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/batch.md

Append arbitrary command line arguments to the gbasf2 submission command using the gbasf2_additional_params setting. This example shows how to blacklist a grid site.

```python
b2luigi.set_setting("gbasf2_additional_params",  "--banned_site LCG.KEK.jp")
```

--------------------------------

### Get Setting

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/settings.md

Retrieves the value of a specific setting. It checks task properties, user-defined settings, and configuration files in order of precedence. A default value can be provided if the setting is not found.

```APIDOC
## get_setting

### Description
Retrieves the current value of a specific setting by its key. It supports checking task-specific properties, user-defined settings, and configuration files. A default value can be returned if the setting is not found, otherwise a ValueError is raised.

### Method Signature
`b2luigi.core.settings.get_setting(key: str, default: Any = None, task: object | None = None, deprecated_keys: List[str] | None = None) -> Any`

### Parameters
* **key** (`str`) - The name of the setting to query.
* **task** (`object | None`) - If provided, checks if the task has a property with this name.
* **default** (`Any`, optional) - The value to return if the setting is not found. If not provided and the setting is not found, a ValueError is raised.
* **deprecated_keys** (`List[str] | None`, optional) - A list of former names for this setting. A warning is issued if these are used.
```

--------------------------------

### Implementing remove_output method in a b2luigi Task

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/run_modes.md

Implement the remove_output method in your b2luigi.Task subclass to define actions for the 'remove' mode. This method is automatically executed when starting the processing in remove mode.

```python
class SomeTask(b2luigi.Task):
    [...] 
    def remove_output(self):
        # if a method with this name is provided, it will be executed
        # automatically when starting the processing in remove mode
        do_some_stuff_in_remove_mode()
```

--------------------------------

### Get Job Status for ID

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Determines the status of a batch job based on its HTCondor job status. Returns JobStatus.successful, JobStatus.running, or JobStatus.aborted. Raises ValueError for unknown HTCondor statuses.

```python
get_job_status_for_id(job_id)
```

--------------------------------

### show_all_outputs

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/runner.md

Displays all output files for a list of tasks, grouped by their respective keys. It collects and prints output files, indicating their existence with colored output.

```APIDOC
## b2luigi.cli.runner.show_all_outputs(task_list, *args, **kwargs)

### Description
Displays all output files for a list of tasks, grouped by their respective keys. This function iterates through a list of tasks, collects all output files from their dependency trees, and prints them grouped by their keys. The existence of each file is visually indicated using colored output (green for existing files, red for non-existing files).

### Parameters
**task_list** (*list*) – A list of tasks to process.

### Notes
- The function uses `get_all_output_files_in_tree` to retrieve output files for each task.
- The existence of files is determined but the task status is not checked.
```

--------------------------------

### RemoteTarget Class

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/remote_targets.md

The RemoteTarget class provides Luigi target implementations for remote file systems. It allows operations like creating directories, getting files, and accessing temporary copies.

```APIDOC
## RemoteTarget

### Description
Luigi target implementation for the remote file system.

### Methods

#### makedirs()

Create the target’s directory on the remote file system.

#### get(path: str = '~') -> str

A function to copy the file from the remote file system to the local file system.

* **Parameters:**
  **path** – Path to copy the file to.
* **Returns:**
  Path to the copied file.

#### open(mode: str) -> None

Raise NotImplementedError as open is not supported.

#### get_temporary_input(task: Task | None = None, **tmp_file_kwargs) -> Generator[str, None, None]

Create a temporary local copy of a remote input file.

Downloads the remote file to a temporary local directory for processing,
allowing safe concurrent access to the same remote input file by multiple tasks.

* **Parameters:**
  **task** (*Optional* *[*[*Task*](tasks.md#b2luigi.Task) *]*) – Task instance used to determine scratch directory settings.
  If None, uses default settings.
* **Yields:**
  *str* – Absolute path to the temporary local copy of the remote input file.

### Example

```python
target = RemoteTarget("remote://server/path/data.root", fs)
with target.get_temporary_input() as tmp_input:
    process_local_file(tmp_input)
# Temporary file is automatically cleaned up.
```
```

--------------------------------

### b2luigi.core.utils.create_output_dirs

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/utilities.md

Creates the necessary output directories for a given task.

```APIDOC
## b2luigi.core.utils.create_output_dirs(task)

### Description
Creates the necessary output directories for a given task.

This function takes a task object, retrieves its outputs, and ensures that
the directories required for those outputs exist by calling the `makedirs`
method on each target.

### Parameters
* **task** – The task object whose outputs need directories to be created.
```

--------------------------------

### Running BASF2 Tasks with HTCondor

Source: https://github.com/belle2/b2luigi/blob/main/docs/examples/basf2-examples.md

Example of a BASF2 Luigi task wrapper configured for HTCondor. It sets up the environment, specifies the executable, and defines the result directory. This is useful for distributed task execution.

```python
from time import sleep
import os

import b2luigi


class MyTask(b2luigi.Task):
    parameter = b2luigi.IntParameter()

    def output(self):
        yield self.add_to_output("test.txt")

    def run(self):
        sleep(self.parameter)

        with open(self.get_output_file_name("test.txt"), "w") as f:
            f.write("Test")


class Wrapper(b2luigi.Task):
    def requires(self):
        for i in range(10):
            yield MyTask(parameter=i)

    def output(self):
        yield self.add_to_output("test.txt")

    def run(self):
        with open(self.get_output_file_name("test.txt"), "w") as f:
            f.write("Test")


if __name__ == '__main__':
    # Choose htcondor as our batch system
    b2luigi.set_setting("batch_system", "htcondor")

    # Setup the correct environment on the workers
    b2luigi.set_setting("env_script", "setup_basf2.sh")

    # Most likely your executable from the submission node is not the same on
    # the worker node, so specify it explicitly
    b2luigi.set_setting("executable", ["python3"])

    # Where to store the results
    b2luigi.set_setting("result_dir", "results")

    b2luigi.process(Wrapper(), batch=True, workers=100)
```

--------------------------------

### RemoteFileSystem.mkdir

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/remote_targets.md

Creates a directory on the remote file system.

```APIDOC
## mkdir(path: str, parents: bool = True, raise_if_exists: bool = False) -> None

### Description
A function to create a directory on the remote file system.

### Parameters
#### Path Parameters
- **path** (str) - Required - The path of the directory to create.
- **parents** (bool) - Optional - Defaults to True.
- **raise_if_exists** (bool) - Optional - Defaults to False.
```

--------------------------------

### Add Multiple Output Files

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/tasks.md

Demonstrates how to use `add_to_output` multiple times within the `output()` method to specify several output files for a task. Each file will have its name automatically generated with parameter values.

```python
def output(self):
    yield self.add_to_output("some_file.txt")
    yield self.add_to_output("some_other_file.txt")
```

--------------------------------

### Get Unique Project Name

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/batch_processes.md

Combines the `gbasf2_project_name_prefix` setting with the `task_id` hash to create a unique project name for grid jobs, ensuring distinct project names for tasks with different parameters.

```APIDOC
## get_unique_project_name

### Description
Combine the `gbasf2_project_name_prefix` setting and the `task_id` hash to a unique project name. This is done to make sure that different instances of a task with different luigi parameters result in different gbasf2 project names. When trying to redoing a task on the grid with identical parameters, rename the project name prefix, to ensure that you get a new project.

### Parameters
**task** – A task. The task must have a `gbasf2_project_name_prefix` setting, parameter, or attribute.

### Returns
A unique project name combining the prefix and a hashed task ID.

### Return type
str

### Raises
* **Exception** – If the task does not have a `gbasf2_project_name_prefix` setting, parameter, or attribute.
* **ValueError** – If the generated project name exceeds the maximum allowed length (32 characters) or contains invalid characters (non-alphanumeric, excluding \_ and -).
```

--------------------------------

### Implementing dry_run method in a b2luigi Task

Source: https://github.com/belle2/b2luigi/blob/main/docs/features/run_modes.md

Define a dry_run method within your b2luigi.Task subclass to handle actions specific to the dry-run mode. This method is automatically executed when the processing is started in dry-run mode.

```python
class SomeTask(b2luigi.Task):
    [...] 
    def dry_run(self):
        # if a method with this name is provided, it will be executed
        # automatically when starting the processing in dry-run mode
        do_some_stuff_in_dry_run_mode()
```

--------------------------------

### Define and Process a Simple Task

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/top_level_functions.md

Use this snippet to define a basic task with parameters and then process it multiple times using b2luigi.process. Ensure all necessary imports are included.

```python
import b2luigi
import random

class MyNumberTask(b2luigi.Task):
    some_parameter = b2luigi.Parameter()

    def output(self):
        return b2luigi.LocalTarget(f"results/output_file_{self.some_parameter}.txt")

    def run(self):
        random_number = random.random()
        with self.output().open("w") as f:
            f.write(f"{random_number}\n")

if __name__ == "__main__":
    b2luigi.process([MyNumberTask(some_parameter=i) for i in range(100)])
```

--------------------------------

### Get Apptainer or Singularity Command

Source: https://github.com/belle2/b2luigi/blob/main/docs/api/utilities.md

Determines the command for containerization, preferring Apptainer over Singularity. It checks for `apptainer_cmd` setting, then `apptainer` or `singularity` in PATH. Raises ValueError if neither is available and no custom command is set.

```python
from b2luigi.core.utils import get_apptainer_or_singularity

# try:
#     container_cmd = get_apptainer_or_singularity()
#     print(f"Using container command: {container_cmd}")
# except ValueError as e:
#     print(e)
```