NVIDIA FLARE

https://nvflare.readthedocs.io/en/main/
Admin
NVIDIA FLARE is an open-source SDK for federated learning, enabling researchers and data scientists...
Tokens:265,664
Snippets:1,260
Trust Score:8.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install Python Dependencies for NVFlare Example

Source: https://nvflare.readthedocs.io/en/main/quickstart

This command uses pip to install all required Python packages listed in the `requirements.txt` file for the 'hello-pt' example. Ensure you have pip installed and an active Python environment.

```bash
$ pip install -r requirements.txt
```

--------------------------------

### Install NVFlare Example Dependencies

Source: https://nvflare.readthedocs.io/en/main/_sources/quickstart.rst

Installs all required Python packages for the 'hello-pt' federated learning example. This step uses pip to read and install dependencies listed in the 'requirements.txt' file, ensuring the environment is ready for execution.

```shell
$ pip install -r requirements.txt
```

--------------------------------

### Clone NVFlare Examples Repository

Source: https://nvflare.readthedocs.io/en/main/_sources/quickstart.rst

Clones the NVFlare repository from GitHub, which contains various examples, and then navigates into the specific 'hello-world/hello-pt' directory to prepare for running the PyTorch federated learning example.

```shell
$ git clone https://github.com/NVIDIA/NVFlare.git
$ cd NVFlare/examples/hello-world/hello-pt
```

--------------------------------

### Clone NVIDIA FLARE Examples with Git

Source: https://nvflare.readthedocs.io/en/main/quickstart

This command clones the NVIDIA FLARE examples repository from GitHub, specifically navigating into the 'hello-world/hello-pt' directory. This is the first step to access the sample federated learning projects.

```bash
$ git clone https://github.com/NVIDIA/NVFlare.git
$ cd NVFlare/examples/hello-world/hello-pt
```

--------------------------------

### Install Example-Specific Python Packages

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Upgrades `pip` within the active virtual environment and installs all required Python packages listed in the `requirements.txt` file. This ensures that all necessary dependencies for a specific NVFlare example are met.

```bash
pip install --upgrade pip
pip install -r requirements.txt
```

--------------------------------

### Start JupyterLab Server

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Launches the JupyterLab server in the current directory. This command opens JupyterLab in your web browser, providing an interface to manage files, run code, and work with Jupyter notebooks.

```bash
jupyter lab .
```

--------------------------------

### Install example-specific Python packages

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Upgrades pip within the active virtual environment and then installs all required Python packages listed in the `requirements.txt` file for a specific example.

```bash
pip install --upgrade pip
pip install -r requirements.txt
```

--------------------------------

### Install JupyterLab for Notebooks

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Installs the JupyterLab interactive development environment within the active Python virtual environment. JupyterLab is recommended for running NVFlare examples that include Jupyter notebooks.

```bash
pip install jupyterlab
```

--------------------------------

### Install Optional Plotting Requirements

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Installs additional Python packages from `plot-requirements.txt`. This step is optional and only necessary for examples that include scripts for plotting TensorBoard event files, providing visualization capabilities.

```bash
pip install -r plot-requirements.txt
```

--------------------------------

### Install optional plotting packages

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Installs additional Python packages from `plot-requirements.txt`, which are optional and typically used for plotting TensorBoard event files in certain examples.

```bash
pip install -r plot-requirements.txt
```

--------------------------------

### Install JupyterLab in virtual environment

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Installs the JupyterLab application within the active virtual environment, enabling interactive notebook development for NVIDIA FLARE examples.

```bash
pip install jupyterlab
```

--------------------------------

### Run NVFlare Federated Learning Example

Source: https://nvflare.readthedocs.io/en/main/_sources/quickstart.rst

Executes the PyTorch-based federated learning simulation script. This command initiates the training process, demonstrating how two clients collaboratively train a model using the NVFlare framework.

```shell
$ python fedavg_script_runner_pt.py
```

--------------------------------

### Install pip and virtualenv

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Installs or upgrades pip and then installs the virtualenv package globally for the current user, which is necessary for creating isolated Python environments.

```bash
python3 -m pip install --user --upgrade pip
python3 -m pip install --user virtualenv
```

--------------------------------

### Install Python Virtual Environment Tools

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Installs `pip` (Python package installer) and `virtualenv` globally for the current user. These tools are essential for creating and managing isolated Python environments, preventing conflicts between project dependencies.

```bash
python3 -m pip install --user --upgrade pip
python3 -m pip install --user virtualenv
```

--------------------------------

### Create a New Python Virtual Environment

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Creates a new isolated Python virtual environment named `nvflare_example` in the current working directory. This environment will contain its own Python interpreter and standard libraries, ensuring project-specific dependencies are isolated.

```bash
python3 -m venv nvflare_example
```

--------------------------------

### Activate Python Virtual Environment

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Activates the `nvflare_example` virtual environment. Once activated, the shell's `python` and `pip` commands will point to the versions within this specific environment, making its installed packages available for use.

```bash
source nvflare_example/bin/activate
```

--------------------------------

### Install Python Dependencies for NVFlare Example

Source: https://nvflare.readthedocs.io/en/main/examples/hello_pt_job_api

Command to install all necessary Python packages required to run the NVIDIA FLARE PyTorch example, specified in `requirements.txt`.

```bash
$ pip install -r requirements.txt

```

--------------------------------

### Start JupyterLab server

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Launches the JupyterLab server in the current directory, making it accessible via a web browser for working with notebooks and files.

```bash
jupyter lab .
```

--------------------------------

### Activate the virtual environment

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Activates the 'nvflare_example' virtual environment, making its Python interpreter and installed packages available in the current shell session.

```shell
$ source nvflare_example/bin/activate
```

--------------------------------

### Install Required Python Dependencies for Example

Source: https://nvflare.readthedocs.io/en/main/examples/tensorboard_streaming

Installs the necessary Python packages listed in the `requirements.txt` file located within the TensorBoard example directory, ensuring the environment is properly set up for execution.

```bash
python3 -m pip install -r requirements.txt
```

--------------------------------

### Create a Python virtual environment

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Creates a new virtual environment named 'nvflare_example' in the current working directory. This isolates project dependencies from the system-wide Python installation.

```shell
$ python3 -m venv nvflare_example
```

--------------------------------

### Register Virtual Environment as Jupyter Kernel

Source: https://nvflare.readthedocs.io/en/main/example_applications_algorithms

Registers the current virtual environment (`nvflare_example`) as a named kernel in Jupyter. This allows users to select `nvflare_example` from the kernel dropdown menu when opening notebooks in JupyterLab, ensuring they run within the correct environment.

```bash
python -m ipykernel install --user --name="nvflare_example"
```

--------------------------------

### Install NVIDIA FLARE from Source with Optional Dependencies

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

These commands show how to install NVIDIA FLARE from a local source repository with specified optional dependency groups, including all development dependencies or specific ones like PyTorch and Scikit-learn.

```shell
$ pip install -e ".[dev]"
$ pip install -e ".[PT,SKLEARN]"
```

--------------------------------

### Install NVFlare Development Dependencies

Source: https://nvflare.readthedocs.io/en/main/installation

Install all development-related dependencies for NVFlare. Separate commands are provided for Linux and macOS due to platform-specific requirements.

```bash
# On Linux
$ pip install nvflare[dev]
```

```bash
# On macOS
$ pip install nvflare[dev_mac]
```

--------------------------------

### Install NVIDIA FLARE from Source in Editable Mode

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

These commands clone the NVIDIA FLARE repository from GitHub and install it in editable mode, allowing for development and testing of custom builds directly from the source code.

```shell
$ git clone https://github.com/NVIDIA/NVFlare.git
$ cd NVFlare
$ pip install -e .
```

--------------------------------

### Install Python Dependencies for Building Wheels

Source: https://nvflare.readthedocs.io/en/main/installation

Install the necessary Python packages, `build` and `wheel`, required to create Python wheel distribution files.

```bash
$ pip install build wheel
```

--------------------------------

### Install NVFlare Wheel Package from Local Directory

Source: https://nvflare.readthedocs.io/en/main/installation

Install a previously built NVFlare wheel package from the local `dist/` directory using pip. This allows for installing custom or pre-built versions of the library.

```bash
$ pip install dist/nvflare-*.whl
```

--------------------------------

### NVFlare Code Pre-Installer Module API

Source: https://nvflare.readthedocs.io/en/main/apidocs/nvflare.tool.code_pre_installer.install

Comprehensive API documentation for the `nvflare.tool.code_pre_installer.install` module, covering functions for command-line argument parsing, general installation execution, specific application code installation from zip files, and Python package installation from requirements files.

```APIDOC
define_pre_install_parser(cmd_name: str, sub_cmd)
  - Description: Define parser for install command.
  - Parameters:
    - cmd_name (str): The name of the command.
    - sub_cmd: The subcommand object to define the parser for.

install(args)
  - Description: Run install command.
  - Parameters:
    - args: Arguments object containing parsed command-line arguments.

install_app_code(app_code_zip: Path, install_prefix: Path, site_name: str, target_shared_dir: str, delete: bool) -> None
  - Description: Install application code from zip file.
  - Parameters:
    - app_code_zip (Path): Path to application code zip file.
    - install_prefix (Path): Installation prefix directory.
    - site_name (str): Target site name.
    - target_shared_dir (str): Target shared directory path.
    - delete (bool): If True, delete the source application code zip after installation.
  - Returns: None

install_requirements(requirements_file: Path)
  - Description: Install Python packages from requirements.txt.
  - Parameters:
    - requirements_file (Path): Path to the requirements.txt file.
```

--------------------------------

### Start NVFlare Proof-of-Concept Components

Source: https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/poc_command

This command initiates all components of the NVFlare Proof-of-Concept (POC) environment, including client sites (site-1, site-2), the server, and the FLARE Console (Admin Client). All components are started within the default workspace at `/tmp/nvflare/poc`. This section also includes examples of successful output and common errors encountered during startup.

```bash
nvflare poc start
```

```log
start_poc at /tmp/nvflare/poc, gpu_ids=[], excluded = [], services_list=[]
WORKSPACE set to /tmp/nvflare/poc/example_project/prod_00/site-2/startup/..
WORKSPACE set to /tmp/nvflare/poc/example_project/prod_00/server/startup/..
WORKSPACE set to /tmp/nvflare/poc/example_project/prod_00/site-1/startup/..
PYTHONPATH is /local/custom:
PYTHONPATH is /local/custom:
start fl because of no pid.fl
start fl because of no pid.fl
start fl because of no pid.fl
new pid 24462
new pid 24463
new pid 24461
Waiting for SP....
Waiting for SP....
2023-07-20 16:29:32,709 - Cell - INFO - server: creating listener on grpc://0:8002
2023-07-20 16:29:32,718 - Cell - INFO - site-1: created backbone external connector to grpc://localhost:8002
2023-07-20 16:29:32,718 - Cell - INFO - site-2: created backbone external connector to grpc://localhost:8002
2023-07-20 16:29:32,719 - ConnectorManager - INFO - 24462: Try start_listener Listener resources: {'secure': False, 'host': 'localhost'}
2023-07-20 16:29:32,719 - ConnectorManager - INFO - 24463: Try start_listener Listener resources: {'secure': False, 'host': 'localhost'}
2023-07-20 16:29:32,719 - Cell - INFO - server: created backbone external listener for grpc://0:8002
2023-07-20 16:29:32,719 - ConnectorManager - INFO - 24461: Try start_listener Listener resources: {'secure': False, 'host': 'localhost'}
2023-07-20 16:29:32,719 - nvflare.fuel.f3.sfm.conn_manager - INFO - Connector [CH00002 PASSIVE tcp://0:31953] is starting
2023-07-20 16:29:32,719 - nvflare.fuel.f3.sfm.conn_manager - INFO - Connector [CH00002 PASSIVE tcp://0:22614] is starting
2023-07-20 16:29:32,720 - nvflare.fuel.f3.sfm.conn_manager - INFO - Connector [CH00002 PASSIVE tcp://0:41710] is starting
Trying to obtain server address
Obtained server address: localhost:8003
Trying to login, please wait ...
2023-07-20 16:29:33,220 - Cell - INFO - site-1: created backbone internal listener for tcp://localhost:31953
2023-07-20 16:29:33,220 - nvflare.fuel.f3.sfm.conn_manager - INFO - Connector [CH00001 ACTIVE grpc://localhost:8002] is starting
2023-07-20 16:29:33,220 - Cell - INFO - site-2: created backbone internal listener for tcp://localhost:22614
2023-07-20 16:29:33,220 - Cell - INFO - server: created backbone internal listener for tcp://localhost:41710
2023-07-20 16:29:33,220 - nvflare.fuel.f3.sfm.conn_manager - INFO - Connector [CH00001 PASSIVE grpc://0:8002] is starting
2023-07-20 16:29:33,220 - nvflare.fuel.f3.sfm.conn_manager - INFO - Connector [CH00001 ACTIVE grpc://localhost:8002] is starting
2023-07-20 16:29:33,221 - FederatedClient - INFO - Wait for engine to be created.
2023-07-20 16:29:33,221 - FederatedClient - INFO - Wait for engine to be created.
2023-07-20 16:29:33,222 - ServerState - INFO - Got the primary sp: localhost fl_port: 8002 SSID: ebc6125d-0a56-4688-9b08-355fe9e4d61a. Turning to hot.
deployed FL server trainer.
2023-07-20 16:29:33,229 - nvflare.fuel.hci.server.hci - INFO - Starting Admin Server localhost on Port 8003
2023-07-20 16:29:33,229 - root - INFO - Server started
2023-07-20 16:29:33,710 - ClientManager - INFO - Client: New client site-2@192.168.86.53 joined. Sent token: cbb4983f-c895-4364-8508-f58cca53dc31.  Total clients: 1
2023-07-20 16:29:33,711 - ClientManager - INFO - Client: New client site-1@192.168.86.53 joined. Sent token: e70a1568-2025-4d47-8e64-e3d1a3667a22.  Total clients: 2
2023-07-20 16:29:33,711 - FederatedClient - INFO - Successfully registered client:site-2 for project example_project. Token:cbb4983f-c895-4364-8508-f58cca53dc31 SSID:ebc6125d-0a56-4688-9b08-355fe9e4d61a
2023-07-20 16:29:33,712 - FederatedClient - INFO - Successfully registered client:site-1 for project example_project. Token:e70a1568-2025-4d47-8e64-e3d1a3667a22 SSID:ebc6125d-0a56-4688-9b08-355fe9e4d61a
2023-07-20 16:29:33,712 - FederatedClient - INFO - Got engine after 0.49114251136779785 seconds
2023-07-20 16:29:33,713 - FederatedClient - INFO - Got the new primary SP: grpc://localhost:8002
2023-07-20 16:29:33,714 - FederatedClient - INFO - Got engine after 0.49308180809020996 seconds
2023-07-20 16:29:33,714 - FederatedClient - INFO - Got the new primary SP: grpc://localhost:8002
Trying to login, please wait ...
Logged into server at localhost:8003 with SSID: ebc6125d-0a56-4688-9b08-355fe9e4d61a
Type ? to list commands; type "? cmdName" to show usage of a command.
>
```

```text
/tmp/nvflare/poc/project.yml is missing, make sure you have first run 'nvflare poc prepare'
```

--------------------------------

### Run NVFlare Federated Learning Simulation (PyTorch)

Source: https://nvflare.readthedocs.io/en/main/quickstart

This command executes the PyTorch-based federated learning simulation script. It initiates a federated learning process where two clients train a model collaboratively, demonstrating a basic FL workflow.

```bash
$ python fedavg_script_runner_pt.py
```

--------------------------------

### Install NVFlare from Source Repository

Source: https://nvflare.readthedocs.io/en/main/installation

Clone the NVFlare GitHub repository and install it in editable mode. This method is useful for accessing the latest features or for development, allowing for direct modification of the installed package. Specific optional dependencies can also be included during source installation.

```bash
$ git clone https://github.com/NVIDIA/NVFlare.git
$ cd NVFlare
$ pip install -e .  # Install in editable mode
```

```bash
$ pip install -e ".[dev]"  # Install all development dependencies
```

```bash
$ pip install -e ".[PT,SKLEARN]"  # Install specific optional dependencies
```

--------------------------------

### Update pip and setuptools in Virtual Environment

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

These commands update the `pip` and `setuptools` packages within the active virtual environment to their latest versions, ensuring proper package management and avoiding potential dependency issues.

```shell
(nvflare-env) $ python3 -m pip install -U pip
(nvflare-env) $ python3 -m pip install -U setuptools
```

--------------------------------

### Start NVFlare POC with Admin Console (Everything Else)

Source: https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/poc_command

Starts the entire NVFlare Proof-of-Concept environment, including the server and clients, and automatically launches the FLARE Console for administrative access. This command is useful for a complete setup in a single terminal.

```bash
nvflare poc start -ex admin@nvidia.com
```

--------------------------------

### Update pip and setuptools in virtual environment

Source: https://nvflare.readthedocs.io/en/main/installation

Updates the 'pip' and 'setuptools' packages within the activated 'nvflare-env' virtual environment to their latest versions. This ensures proper package management and avoids potential issues during installation.

```bash
python3 -m pip install -U pip
python3 -m pip install -U setuptools
```

--------------------------------

### NVFLARE Site Installation Kit Structure

Source: https://nvflare.readthedocs.io/en/main/programming_guide/provisioning_system

This snippet illustrates the directory structure of the Site Installation Kit generated by the NVFLARE provisioning process. It outlines the placement of startup files (Certs, keys, fed_*.json, shell scripts), local configuration files (resources.json, privacy.json, authorization.json), and the Readme.txt for installation and management instructions.

```text
Installation Kit
    startup
        Certs, private key, fed_[server|client].json, shell scripts
    local
        resources.json - used by main and job process (on client)
        privacy.json - used by job process only
        authorization.json - used by main process only
    Readme.txt: describe how to use scripts to install startup and site; how to manage content in the "site" folder
```

--------------------------------

### Build NVFlare Python Wheel Package

Source: https://nvflare.readthedocs.io/en/main/installation

Execute the Python build process to create a distributable wheel package for NVFlare. The generated `.whl` files will be placed in the `dist/` directory.

```bash
$ python -m build
```

--------------------------------

### Clone NVIDIA FLARE Repository

Source: https://nvflare.readthedocs.io/en/main/examples/hello_cross_val

This command clones the NVIDIA FLARE GitHub repository to your local machine. It's the first step to get the source code for the examples and applications required for the tutorial.

```Shell
git clone https://github.com/NVIDIA/NVFlare.git
```

--------------------------------

### Install Documentation Build Requirements for NVIDIA FLARE

Source: https://nvflare.readthedocs.io/en/main/_sources/contributing.rst

These commands upgrade the pip package installer and then install the necessary Python dependencies for building the NVIDIA FLARE documentation. The `-e .[doc]` flag installs the project in editable mode along with its documentation-specific dependencies.

```bash
python -m pip upgrade
python -m pip install -e .[doc]
```

--------------------------------

### Complete NVFlare Agent Lifecycle and Task Processing Example

Source: https://nvflare.readthedocs.io/en/main/programming_guide/execution_api_type/3rd_party_integration

This comprehensive Python example demonstrates the full lifecycle of an NVFlare agent, from initialization and starting to continuous task processing and graceful shutdown. It includes argument parsing for configuration, logging setup, and the core loop for `get_task` and `submit_result`, handling `AgentClosed` for termination.

```python
import argparse
import logging

from nvflare.client.flare_agent import AgentClosed, FlareAgentWithCellPipe

NUMPY_KEY = "numpy_key"

def main():

    logging.basicConfig()
    logging.getLogger().setLevel(logging.INFO)

    parser = argparse.ArgumentParser()
    parser.add_argument("--workspace", "-w", type=str, help="workspace folder", required=False, default=".")
    parser.add_argument("--site_name", "-s", type=str, help="flare site name", required=True)
    parser.add_argument("--agent_id", "-a", type=str, help="agent id", required=True)

    args = parser.parse_args()

    # 1. create the agent
    agent = FlareAgentWithCellPipe(
        root_url="grpc://server:8002",
        site_name=args.site_name,
        agent_id=args.agent_id,
        workspace_dir=args.workspace,
        secure_mode=True,
        submit_result_timeout=2.0,
        heartbeat_timeout=120.0,
    )

    # 2. start the agent
    agent.start()

    # 3. processing tasks
    while True:
        print("getting task ...")
        try:
            task = agent.get_task()
        except AgentClosed:
            print("agent closed - exit")
            break

        print(f"got task: {task}")
        result = train(task.data)  # perform train task
        submitted = agent.submit_result(result)
        print(f"result submitted: {submitted}")

    # 4. stop the agent
    agent.stop()

def train(model):
    print(f"training on {model}")
    return model

if __name__ == "__main__":
    main()
```

--------------------------------

### NVFlare Provision Command API Reference

Source: https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/provision_command

Comprehensive API documentation for the `nvflare provision` command, detailing its purpose, command signature, all available optional arguments, and how to use it for initial project setup as well as dynamically adding users and clients via YAML configuration files.

```APIDOC
Command: nvflare provision
Description: Manages the provisioning of NVIDIA FLARE projects, allowing for initial setup and dynamic addition of users and clients.

Signature: nvflare provision [-h] [-p PROJECT_FILE] [-w WORKSPACE] [-c CUSTOM_FOLDER] [--add_user ADD_USER] [--add_client ADD_CLIENT]

Arguments:
  -h, --help: show this help message and exit
  -p PROJECT_FILE, --project_file PROJECT_FILE: file to describe FL project
  -w WORKSPACE, --workspace WORKSPACE: directory used by provision
  -c CUSTOM_FOLDER, --custom_folder CUSTOM_FOLDER: additional folder to load python code
  --add_user ADD_USER: yaml file for added user.
    Example user.yaml:
      name: new_user@nvidia.com
      org: nvidia
      role: project_admin
  --add_client ADD_CLIENT: yaml file for added client.
    Example client.yaml:
      name: new-site
      org: nvidia
      components:
        resource_manager:    # This id is reserved by system.  Do not change it.
          path: nvflare.app_common.resource_managers.gpu_resource_manager.GPUResourceManager
          args:
            num_of_gpus: 0
            mem_per_gpu_in_GiB: 0
        resource_consumer:    # This id is reserved by system.  Do not change it.
          path: nvflare.app_common.resource_consumers.gpu_resource_consumer.GPUResourceConsumer
          args:

Usage Notes:
  - Running `provision` without any options and without a project.yml file in the current working directory will prompt to copy a default project.yml.
  - The `--add_user` and `--add_client` options allow for adding to an existing project by taking a YAML file to define the additional participant.
  - After running `nvflare provision` with `--add_user` or `--add_client`, the new user or client will be included in the `prod_NN` folder.
  - To permanently include users or clients, please update the `project.yml` file.
```

--------------------------------

### Register virtual environment kernel with JupyterLab

Source: https://nvflare.readthedocs.io/en/main/_sources/example_applications_algorithms.rst

Registers the 'nvflare_example' virtual environment as a named kernel in JupyterLab, allowing users to select it when opening notebooks.

```bash
python -m ipykernel install --user --name="nvflare_example"
```

--------------------------------

### Run NVIDIA FLARE Provisioning CLI Tool

Source: https://nvflare.readthedocs.io/en/main/real_world_fl/overview

This command executes the NVIDIA FLARE provisioning tool, using the specified `project.yml` file to generate startup kits. The `project.yml` defines the server, client sites, admin, and organization configurations for the federated learning project. Generated kits are created by default in a directory prefixed with 'prod_' within a folder of the project name in the workspace.

```bash
nvflare provision -p project.yml
```

--------------------------------

### NVFlare Code Pre-Installer Prepare Module API

Source: https://nvflare.readthedocs.io/en/main/apidocs/nvflare.tool.code_pre_installer.prepare

This section documents the core functions within the `nvflare.tool.code_pre_installer.prepare` module, which are essential for setting up and executing code pre-installation tasks for NVFLARE applications. It includes functions for command-line argument parsing, general preparation execution, and specific application code packaging.

```APIDOC
define_prepare_parser(*cmd_name: str*, *sub_cmd*)
  - Define parser for prepare command.

prepare(*args*)
  - Run prepare command.

prepare_app_code(*job_folder: Path*, *output_dir: Path*, *shared_lib: Path | None = None*, *requirements: Path | None = None*) → None
  - Package NVFLARE application code for pre-installation.
  - Parameters:
    - job_folder: Path to job folder (e.g., jobs/fedavg)
    - output_dir: Output directory for application.zip
    - shared_lib: Optional path to shared library folder
    - requirements: Optional path to requirements.txt file
```

--------------------------------

### Install Python Dependencies for MLflow Example

Source: https://nvflare.readthedocs.io/en/main/examples/fl_experiment_tracking_mlflow

This command installs all required Python packages listed in the 'requirements.txt' file for the MLflow experiment tracking example. It should be executed within the activated NVIDIA FLARE Python virtual environment.

```Shell
(nvflare-env) $ python3 -m pip install -r requirements.txt
```

--------------------------------

### Sample Client YAML for Dynamic Provisioning

Source: https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/provision_command

An example YAML file structure used with the `--add_client` option of the `nvflare provision` command to dynamically add a new client site to an existing NVIDIA FLARE project. It specifies the client's name, organization, and resource manager/consumer components.

```yaml
name: new-site
org: nvidia
components:
  resource_manager:    # This id is reserved by system.  Do not change it.
    path: nvflare.app_common.resource_managers.gpu_resource_manager.GPUResourceManager
    args:
      num_of_gpus: 0
      mem_per_gpu_in_GiB: 0
  resource_consumer:    # This id is reserved by system.  Do not change it.
    path: nvflare.app_common.resource_consumers.gpu_resource_consumer.GPUResourceConsumer
    args:
```

--------------------------------

### Install Stable NVIDIA FLARE Release

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

This command installs the latest stable release of NVIDIA FLARE from the Python Package Index (PyPI) into the active Python environment.

```shell
$ python3 -m pip install nvflare
```

--------------------------------

### Clone NVIDIA FLARE Repository

Source: https://nvflare.readthedocs.io/en/main/examples/hello_pt_job_api

Instructions to clone the NVIDIA FLARE repository from GitHub and navigate into the specific PyTorch example directory.

```bash
$ git clone https://github.com/NVIDIA/NVFlare.git
$ cd NVFlare/examples/hello-world/hello-pt

```

--------------------------------

### Install Python venv Package on Ubuntu

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

This command installs the `python3-venv` package on Ubuntu 20.04 or similar Debian-based systems, which is required to create Python virtual environments.

```shell
$ sudo apt update
$ sudo apt-get install python3-venv
```

--------------------------------

### Install Python venv package on Ubuntu

Source: https://nvflare.readthedocs.io/en/main/installation

Installs the Python venv package on Ubuntu 20.04, which is a prerequisite for creating and managing isolated Python virtual environments for NVIDIA FLARE.

```bash
sudo apt update
sudo apt-get install python3-venv
```

--------------------------------

### Install Build Dependencies for NVIDIA FLARE Wheels

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

This command installs the necessary Python packages (`build` and `wheel`) required to create wheel distribution packages for NVIDIA FLARE.

```shell
$ pip install build wheel
```

--------------------------------

### Install NVFlare from Local Wheel File

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

Installs the NVFlare package from a locally built wheel file using pip. This command assumes the wheel files are located in the `dist/` directory after a successful build process.

```shell
$ pip install dist/nvflare-*.whl
```

--------------------------------

### Install NVFlare with Optional Dependencies

Source: https://nvflare.readthedocs.io/en/main/installation

Install NVFlare with specific optional feature groups like Homomorphic Encryption (HE), Private Set Intersection (PSI), PyTorch (PT), Scikit-learn (SKLEARN), MLflow/Weights & Biases/TensorBoard (TRACKING), Datadog (MONITORING), or OmegaConf (CONFIG). Multiple groups can be installed simultaneously.

```bash
$ pip install nvflare[HE]
```

```bash
$ pip install nvflare[PSI]
```

```bash
$ pip install nvflare[PT]
```

```bash
$ pip install nvflare[SKLEARN]
```

```bash
$ pip install nvflare[TRACKING]
```

```bash
$ pip install nvflare[MONITORING]
```

```bash
$ pip install nvflare[CONFIG]
```

```bash
$ pip install nvflare[PT,SKLEARN,TRACKING]  # Install PyTorch, Scikit-learn, and tracking support
```

--------------------------------

### Sample User YAML for Dynamic Provisioning

Source: https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/provision_command

An example YAML file structure used with the `--add_user` option of the `nvflare provision` command to dynamically add a new user to an existing NVIDIA FLARE project. It defines the user's name, organization, and role.

```yaml
name: new_user@nvidia.com
org: nvidia
role: project_admin
```

--------------------------------

### NVFlare POC Command-Line Interface Reference

Source: https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/poc_command

Comprehensive documentation for the `nvflare poc` CLI commands, including `prepare`, `prepare-jobs-dir`, `start`, `stop`, and `clean`. These commands are used for setting up, managing, and cleaning Proof-of-Concept (POC) workspaces in NVFlare.

```APIDOC
nvflare poc prepare
  - Description: Prepares the NVFlare POC workspace by generating startup kits.
  - Usage: nvflare poc prepare [-h] [-n [NUMBER_OF_CLIENTS]] [-c [CLIENTS ...]] [-he] [-i [PROJECT_INPUT]] [-d [DOCKER_IMAGE]] [-debug]
  - Options:
    - -h, --help: show this help message and exit
    - -n [NUMBER_OF_CLIENTS], --number_of_clients [NUMBER_OF_CLIENTS]: number of sites or clients, default to 2
    - -c [CLIENTS ...], --clients [CLIENTS ...]: Space separated client names. If specified, number_of_clients argument will be ignored.
    - -he, --he: enable homomorphic encryption.
    - -i [PROJECT_INPUT], --project_input [PROJECT_INPUT]: project.yaml file path, If specified, 'number_of_clients','clients' and 'docker' specific options will be ignored.
    - -d [DOCKER_IMAGE], --docker_image [DOCKER_IMAGE]: generate docker.sh based on the docker_image, used in '--prepare' command. and generate docker.sh 'start/stop' commands will start with docker.sh
    - -debug, --debug: debug is on
  - Example Usage:
    nvflare poc prepare

nvflare poc prepare-jobs-dir
  - Description: Prepares the jobs directory for NVFlare POC.
  - Usage: nvflare poc prepare-jobs-dir [-h] [-j [JOBS_DIR]] [-debug]
  - Options:
    - -h, --help: show this help message and exit
    - -j [JOBS_DIR], --jobs_dir [JOBS_DIR]: jobs directory
    - -debug, --debug: debug is on
  - Note: The "-j" option is new in version 2.4 for linking to the job directory in the code base. It takes precedence over the NVFLARE_HOME environment variable.

nvflare poc start
  - Description: Starts the NVFlare POC services.
  - Usage: nvflare poc start [-h] [-p [SERVICE]] [-ex [EXCLUDE]] [-gpu [GPU ...]] [-debug]
  - Options:
    - -h, --help: show this help message and exit
    - -p [SERVICE], --service [SERVICE]: participant, Default to all participants
    - -ex [EXCLUDE], --exclude [EXCLUDE]: exclude service directory during 'start', default to , i.e. nothing to exclude
    - -gpu [GPU ...], --gpu [GPU ...]: gpu device ids will be used as CUDA_VISIBLE_DEVICES. used for poc start command
    - -debug, --debug: debug is on

nvflare poc stop
  - Description: Stops the NVFlare POC services.
  - Usage: nvflare poc stop [-h] [-p [SERVICE]] [-ex [EXCLUDE]] [-debug]
  - Options:
    - -h, --help: show this help message and exit
    - -p [SERVICE], --service [SERVICE]: participant, Default to all participants
    - -ex [EXCLUDE], --exclude [EXCLUDE]: exclude service directory during 'stop', default to , i.e. nothing to exclude
    - -debug, --debug: debug is on

nvflare poc clean
  - Description: Cleans up the NVFlare POC workspace.
  - Usage: nvflare poc clean [-h] [-debug]
  - Options:
    - -h, --help: show this help message and exit
    - -debug, --debug: debug is on
```

--------------------------------

### Install stable NVIDIA FLARE release via pip

Source: https://nvflare.readthedocs.io/en/main/installation

Installs the latest stable release of NVIDIA FLARE from PyPI using the pip package manager. This command should be executed within an activated virtual environment to ensure proper dependency management.

```bash
python3 -m pip install nvflare
```

--------------------------------

### Activate NVFlare Virtual Environment

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

This command activates the previously created `nvflare-env` virtual environment, making its Python interpreter and installed packages available in the current shell session.

```shell
$ source nvflare-env/bin/activate
```

--------------------------------

### Install NVIDIA FLARE Optional Dependencies

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

These commands demonstrate how to install NVIDIA FLARE with various optional dependency groups, such as Homomorphic Encryption (HE), Private Set Intersection (PSI), PyTorch (PT), Scikit-learn (SKLEARN), MLflow/W&B/TensorBoard tracking (TRACKING), Datadog monitoring (MONITORING), and OmegaConf configuration (CONFIG). You can also install multiple groups or development dependencies.

```shell
$ pip install nvflare[HE]

$ pip install nvflare[PSI]

$ pip install nvflare[PT]

$ pip install nvflare[SKLEARN]

$ pip install nvflare[TRACKING]

$ pip install nvflare[MONITORING]

$ pip install nvflare[CONFIG]

$ pip install nvflare[PT,SKLEARN,TRACKING]

# On Linux
$ pip install nvflare[dev]

# On macOS
$ pip install nvflare[dev_mac]
```

--------------------------------

### Create a Python Virtual Environment for NVFlare

Source: https://nvflare.readthedocs.io/en/main/_sources/installation.rst

This command creates a new Python virtual environment named `nvflare-env` in the current directory, isolating project dependencies and providing a clean installation space.

```shell
$ python3 -m venv nvflare-env
```

--------------------------------

### NVFlare Provisioning Command Line Interface

Source: https://nvflare.readthedocs.io/en/main/programming_guide/provisioning_system

This section details the command-line arguments for the `nvflare provision` tool, used to set up federated learning environments. It includes options for specifying project files, workspaces, custom folders, and dynamically adding users or clients via YAML files.

```APIDOC
usage: nvflare provision [-h] [-p PROJECT_FILE] [-w WORKSPACE] [-c CUSTOM_FOLDER] [--add_user ADD_USER] [--add_client ADD_CLIENT]

optional arguments:
-h, --help                                               show this help message and exit
-p PROJECT_FILE, --project_file PROJECT_FILE                 file to describe FL project
-w WORKSPACE, --workspace WORKSPACE                          directory used by provision
-c CUSTOM_FOLDER, --custom_folder CUSTOM_FOLDER    additional folder to load python code
--add_user ADD_USER                                             yaml file for added user
--add_client ADD_CLIENT                                       yaml file for added client
```

--------------------------------

### Activate NVIDIA FLARE virtual environment

Source: https://nvflare.readthedocs.io/en/main/installation

Activates the 'nvflare-env' virtual environment, making its Python interpreter and installed packages accessible in the current shell session. The shell prompt will typically change to indicate the active environment.

```bash
source nvflare-env/bin/activate
```

--------------------------------

### Initialize and Execute NVIDIA FLARE Provisioner

Source: https://nvflare.readthedocs.io/en/main/programming_guide/provisioning_system

This Python snippet illustrates the fundamental interaction with the `Provisioner` class from NVIDIA FLARE's Open Provision API. It first instantiates the `Provisioner` by providing the full path to the workspace and a collection of builder objects. Subsequently, it invokes the `provision` method, passing a `project` object, which initiates the entire provisioning workflow to generate necessary configuration artifacts for federated learning participants.

```python
provisioner = Provisioner(workspace_full_path, builders)

provisioner.provision(project)
```

--------------------------------

### Display NVFlare Provisioning CLI Help

Source: https://nvflare.readthedocs.io/en/main/release_notes/flare_220

This command is used to display the help message for the NVFlare provisioning command-line interface. It provides detailed information on how to use the `nvflare provision` command, including available options for configuring and distributing client startup kits, and supports both manual and dynamic provisioning methods.

```Bash
nvflare provision -h
```

--------------------------------

### Install NumPy in NVIDIA FLARE Environment

Source: https://nvflare.readthedocs.io/en/main/examples/hello_fedavg_numpy

This command installs the NumPy library within the activated NVIDIA FLARE Python virtual environment, which is a prerequisite for running the federated learning examples that use NumPy for array operations.

```bash
(nvflare-env) $ python3 -m pip install numpy
```

--------------------------------

### Install NumPy Dependency

Source: https://nvflare.readthedocs.io/en/main/examples/hello_cross_val

This command installs the NumPy library within the activated NVIDIA FLARE Python virtual environment. NumPy is a fundamental dependency for numerical operations used in the tutorial's examples, ensuring the application runs correctly.

```Shell
python3 -m pip install numpy
```

--------------------------------

### NVFlare Code Pre-Installer: Command Handling Functions

Source: https://nvflare.readthedocs.io/en/main/apidocs/nvflare.tool

Functions within the `nvflare.tool.code_pre_installer.pre_install_cmd` module responsible for defining argument parsers and handling the execution of pre-installation commands.

```APIDOC
def_pre_install_parser()
  - Defines the argument parser for pre-install commands.
handle_pre_install_cmd()
  - Handles the execution of pre-install commands.
```

--------------------------------

### NVFlare Provision Command Usage

Source: https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/provision_command

Displays the command-line arguments and options available for the `nvflare provision` command, which is used for setting up and managing NVIDIA FLARE projects.

```bash
usage: nvflare provision [-h] [-p PROJECT_FILE] [-w WORKSPACE] [-c CUSTOM_FOLDER] [--add_user ADD_USER] [--add_client ADD_CLIENT]

optional arguments:
-h, --help                                               show this help message and exit
-p PROJECT_FILE, --project_file PROJECT_FILE                 file to describe FL project
-w WORKSPACE, --workspace WORKSPACE                          directory used by provision
-c CUSTOM_FOLDER, --custom_folder CUSTOM_FOLDER    additional folder to load python code
--add_user ADD_USER                                             yaml file for added user
--add_client ADD_CLIENT                                       yaml file for added client
```

--------------------------------

### Install AWS EBS CSI Driver Addon on EKS

Source: https://nvflare.readthedocs.io/en/main/real_world_fl/kubernetes

Installs the AWS EBS CSI driver as an official EKS addon. This command links the addon to the previously created IAM service account role, enabling the cluster to dynamically provision and manage Amazon EBS volumes for persistent storage.

```bash
eksctl create addon --name aws-ebs-csi-driver \
--cluster nvflare-cluster \
--service-account-role-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):role/AmazonEKS_EBS_CSI_DriverRole \
--force
```

--------------------------------

### Execute NVIDIA FLARE PyTorch Federated Learning Example

Source: https://nvflare.readthedocs.io/en/main/examples/hello_pt_job_api

Command to run the main script that initiates the NVIDIA FLARE federated learning job for the PyTorch CIFAR10 example using the FL Simulator.

```bash
$ python fedavg_script_runner_pt.py

```

--------------------------------

### NVFLARE Cloud Component Deployment Commands

Source: https://nvflare.readthedocs.io/en/main/real_world_fl/cloud_deployment

Generic command line patterns for deploying NVFLARE components (FL Server, FL Client, Dashboard) to various cloud providers. Each command uses a startup script, specifies the cloud provider, and optionally includes a configuration file.

```Shell
FL Server:    <startup dir>/start.sh     –cloud <csp>  [ –config config.json ]
FL Client:      <startup dir>/start.sh     –cloud <csp>  [ –config config.json ]
Dashboard:   <startup dir>/start.sh     –cloud <csp>  [ –config config.json ]
```

--------------------------------

### Copy Startup Kits to Persistent Volume

Source: https://nvflare.readthedocs.io/en/main/real_world_fl/kubernetes

Copies local NVFlare startup kit directories (e.g., `server`, `site-1`, `site-2`, `admin@nvidia.com`) into the persistent volume mounted by the helper pod. This makes the necessary configuration and binaries available within the EKS cluster for NVFlare components.

```bash
kubectl cp workspace/example_project/prod_00/server <helper-pod>:/workspace/nvfl/
```

--------------------------------

### Switch NVFlare Git Repository Branch

Source: https://nvflare.readthedocs.io/en/main/installation

Change the active branch in a cloned NVFlare repository. This is useful for switching between different major release versions or development branches.

```bash
$ git switch 2.6  # Replace with desired version
```

--------------------------------

### NVFlare Lighter Proof-of-Concept Module API Reference

Source: https://nvflare.readthedocs.io/en/main/apidocs/nvflare

This module contains the `main()` function for the NVFlare Lighter proof-of-concept (PoC) execution. It serves as an entry point for demonstrating basic provisioning functionalities.

```APIDOC
nvflare.lighter.poc module:
  - main() (Function)
```
Tokens:
Raw