### Run Local MedPerf Server Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/setup.md Sets up and starts a local MedPerf server for tutorial purposes. This involves copying an environment file and executing a setup script. ```bash cd server cp .env.local.local-auth.sqlite .env sh setup-dev-server.sh ``` -------------------------------- ### Create Python Environment and Install Packages Source: https://github.com/mlcommons/medperf/blob/main/examples/ChestXRay/xrv_resnet/README.md Sets up a Python virtual environment and installs necessary packages, including MLCube and its Docker runner, from GitHub. ```bash virtualenv -p python3 ./env && source ./env/bin/activate pip install wheel git clone https://github.com/sergey-serebryakov/mlbox.git && cd mlbox && git checkout feature/configV2 cd ./mlcube && python setup.py bdist_wheel && pip install --force-reinstall ./dist/mlcube-* && cd .. cd ./runners/mlcube_docker && python setup.py bdist_wheel && pip install --force-reinstall --no-deps ./dist/mlcube_docker-* && cd ../../.. rm -fr mlbox ``` -------------------------------- ### Install Documentation Dependencies Source: https://github.com/mlcommons/medperf/blob/main/README.md Installs all required dependencies for serving MedPerf documentation using mkdocs. ```bash pip install -r docs/requirements.txt ``` -------------------------------- ### Check Singularity Installation Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/installation.md Verify that Singularity is installed on your system. ```bash singularity --version ``` -------------------------------- ### Setup Test Environment (No Docker) Source: https://github.com/mlcommons/medperf/blob/main/examples/fl_post/fl/README.md Run this script once to create necessary certificates and download required data for testing without Docker. ```bash sh setup_test_no_docker.sh ``` -------------------------------- ### Install Prerequisite Packages Source: https://github.com/mlcommons/medperf/blob/main/examples/ChestXRay/metrics/README.md Installs the 'wheel' package, a prerequisite for building Python packages. ```bash pip install wheel ``` -------------------------------- ### Reset Setup Artifacts Source: https://github.com/mlcommons/medperf/blob/main/examples/fl_post/fl/README.md Clear artifacts generated by the initial setup script, preparing for a re-run of the setup. ```bash sh setup_clean.sh ``` -------------------------------- ### MLCube Task CLI Interface Example Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/model_deepmedic/README.md Example of the expected command-line interface for an MLCube task, demonstrating how parameters are passed. ```bash python3 project/mlcube.py infer --data_path= --parameters_file= --greetings= --output_path= ``` -------------------------------- ### Production Data Configuration Example Source: https://github.com/mlcommons/medperf/blob/main/examples/cc/base_image/README.md Example JSON structure for configuring data settings in production mode. Ensure all GCP-related fields are correctly populated. ```json { "project_id": "gcp_project_id", "project_number": "gcp_project_number", "account": "gcp_account", "bucket": "gcp_bucket", "encrypted_asset_bucket_file": "gcp_encrypted_asset_bucket_file", "encrypted_key_bucket_file": "gcp_encrypted_key_bucket_file", "keyring_name": "gcp_keyring_name", "key_name": "gcp_key_name", "wip": "gcp_wip" } ``` -------------------------------- ### Check Docker Installation Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/installation.md Verify that Docker is installed on your system. Ensure you can run Docker as a non-root user if applicable. ```bash docker --version ``` -------------------------------- ### Install MLCube and Docker Runner from GitHub Source: https://github.com/mlcommons/medperf/blob/main/examples/ChestXRay/metrics/README.md Clones the mlbox repository, checks out a specific branch, and installs MLCube and its Docker runner from the local source. Users typically would use `pip install mlcube mlcube_docker`. ```bash git clone https://github.com/sergey-serebryakov/mlbox.git && cd mlbox && git checkout feature/configV2 cd ./mlcube && python setup.py bdist_wheel && pip install --force-reinstall ./dist/mlcube-* cd ../runners/mlcube_docker && python setup.py bdist_wheel && pip install --force-reinstall --no-deps ./dist/mlcube_docker-* cd ../../.. rm -fr mlbox ``` -------------------------------- ### MLCube Parameters Configuration Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/model_deepmedic/README.md Example configuration for parameters.yaml, used to parameterize model behavior. This specific example shows a boolean flag 'uppercase' for modifying Hello World output. ```yaml # Here you can store any key-value arguments that should be easily modifiable # by external users. E.g. batch_size # example argument for Hello World uppercase: false ``` -------------------------------- ### Install mlcube and mlcube-docker Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/data_prep/README.md Install the necessary mlcube and mlcube-docker Python packages using pip. ```bash pip install mlcube mlcube-docker ``` -------------------------------- ### Initiate Login via CLI Source: https://github.com/mlcommons/medperf/blob/main/docs/concepts/auth.md Use this command to start the login process from the command line. You will be prompted to enter your email address. ```bash medperf auth login ``` -------------------------------- ### MLCube Task Execution CLI Example Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/data_prep/README.md Demonstrates the command-line interface for executing MLCube tasks, such as 'prepare'. Input and output parameters are passed as arguments. This interface is expected by `mlcube.py`. ```bash python3 project/mlcube.py prepare --data_path= --labels_path= --parameters_file= --output_path= ``` -------------------------------- ### Install MedPerf from Source Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/installation.md Install MedPerf using pip in editable mode from the local source code. This command should be run after cloning the repository. ```bash pip install -e ./cli ``` -------------------------------- ### Start MedPerf Web UI Source: https://github.com/mlcommons/medperf/blob/main/docs/concepts/confidential_computing.md Run this command in your terminal to start the local web user interface for MedPerf. ```bash medperf_webui ``` -------------------------------- ### MLCube Directory Structure Example Source: https://github.com/mlcommons/medperf/blob/main/examples/SurgMLCube/surg_metrics/README.md This is the expected default directory tree for the SurgMLCube, including mlcube configuration, workspace, data, and predictions folders. ```tree surg_metrics ├── mlcube │ ├── workspace │ │ ├── data │ │ │ ├── frames │ │ │ └── data_csv │ │ │ │ │ ├── parameters.yaml │ │ │ │ │ └── predictions │ │ ├── some_video.csv │ │ ├── other_video.csv │ │ └ ... │ │ │ └── mlcube.yaml └── project ``` -------------------------------- ### Verify MedPerf Installation Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/installation.md Check if MedPerf has been installed correctly by running the version command. This confirms the installation was successful. ```bash medperf --version ``` -------------------------------- ### MLCube Evaluate CLI Interface Example Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/metrics/README.md Illustrates the expected command-line interface for executing the 'evaluate' task of an MLCube, passing input and output paths as arguments. ```bash python3 project/mlcube.py evaluate --predictions= --labels= --parameters_file= --output_path= ``` -------------------------------- ### Navigate to Data Preparator MLCube Folder Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/data_prep/README.md Change the current directory to the mlcube folder for the medperf data preparator example. ```bash cd medperf/data_preparator/mlcube ``` -------------------------------- ### Install MedPerf Server Requirements Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/setup.md Installs the necessary Python packages for the MedPerf server. Ensure you are in the MedPerf root directory before running. ```bash pip install -r server/requirements.txt pip install -r server/test-requirements.txt ``` -------------------------------- ### MLCube Final Directory Structure Example Source: https://github.com/mlcommons/medperf/blob/main/examples/SurgMLCube/surg_metrics/README.md This illustrates the final directory tree after the MLCube has run, including the generated results.yaml file. ```tree surg_metrics ├── mlcube │ ├── workspace │ │ ├── data │ │ │ ├── frames │ │ │ └── data_csv │ │ │ │ │ ├── parameters.yaml │ │ │ │ │ ├── predictions │ │ │ ├── some_video.csv │ │ │ ├── other_video.csv │ │ │ └ ... │ │ │ │ │ └── results.yaml │ │ │ └── mlcube.yaml └── project ``` -------------------------------- ### Check Python and Pip Versions Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/installation.md Verify that Python and pip are installed on your system. Use `python3` and `pip3` if your system is configured differently. ```bash python --version pip --version ``` ```bash python3 --version pip3 --version ``` -------------------------------- ### Navigate to Metrics MLCube Directory Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/metrics/README.md Change the current directory to the mlcube folder specific to the MedPerf metrics example. ```bash cd medperf/metrics/mlcube ``` -------------------------------- ### Login to Local MedPerf Server Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/shared/before_we_start.md Use this command to log in to the dummy local authentication system for tutorial purposes. For real servers, refer to the authentication guide. ```bash medperf auth login -e {{page.meta.email}} ``` -------------------------------- ### Radiotherapy Model Output Structure Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS2024/dummy_model/README.md Example directory structure for radiotherapy model predictions. Contains NIfTI files for segmentation. ```bash predictions ├── BraTS-MEN-RT-xxxx-x.nii.gz ├── BraTS-MEN-RT-yyyy-y.nii.gz └── BraTS-MEN-RT-zzzz-z.nii.gz ``` -------------------------------- ### Metrics Parameterization in parameters.yaml Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/metrics/README.md Example of a parameters.yaml file for MedPerf, specifying metrics to compute, label columns, and the common identifier column for matching labels and predictions. ```yaml # File for parametrizing your metrics calculations metrics: # List of metrics to run - ACC label columns: # Label columns that are going to be evaluated - greeting # Common identifier column for labels and predictions id column: id ``` -------------------------------- ### Example .json Label File Structure (Format 1) Source: https://github.com/mlcommons/medperf/blob/main/examples/SurgMLCube/surg_prep/README.md Labels in .json files can be structured as a list of dictionaries, where each dictionary contains 'timestamp', 'duration', and 'labelName'. Timestamps and durations are in milliseconds. ```json { 'timestamp' : 'duration' : 'labelName' : } ``` -------------------------------- ### Example .txt and .csv Label File Structure Source: https://github.com/mlcommons/medperf/blob/main/examples/SurgMLCube/surg_prep/README.md Labels in .txt or .csv files should have a header line and two columns separated by a delimiter (Tab or Comma). The columns represent Frame ID or timestamp and the label name. ```text ... ``` -------------------------------- ### Serve Local Documentation Source: https://github.com/mlcommons/medperf/blob/main/README.md Runs a local mkdocs server to visualize the documentation website. ```bash mkdocs serve ``` -------------------------------- ### Navigate to mlcube_examples Directory Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/data_prep/README.md Change the current directory to the mlcube_examples folder within the repository. ```bash cd mlcube_examples ``` -------------------------------- ### Create Python Environment and Activate Source: https://github.com/mlcommons/medperf/blob/main/examples/ChestXRay/metrics/README.md Sets up a dedicated Python virtual environment for the project and activates it. ```bash virtualenv -p python3 ./env && source ./env/bin/activate ``` -------------------------------- ### Download Tutorial Files Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/shared/before_we_start.md Execute this script from the MedPerf root folder to download all necessary files for the tutorial, creating a 'medperf_tutorial' workspace. ```bash sh tutorials_scripts/setup_{{page.meta.tutorial_id}}_tutorial.sh ``` -------------------------------- ### Initialize Local MedPerf Server and Database Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/shared/before_we_start.md Run this script from the MedPerf root folder to reset the database and seed it with demo data for the tutorial. ```bash cd server sh reset_db.sh python seed.py --demo {{page.meta.tutorial_id}} cd .. ``` -------------------------------- ### Build Project Scripts Source: https://github.com/mlcommons/medperf/blob/main/examples/fl_post/fl/README.md Execute the build script to compile project components. Use the -b flag to also rebuild the OpenFL base. ```bash sh build.sh ``` ```bash sh build.sh -b ``` -------------------------------- ### Run Test with Parameters Source: https://github.com/mlcommons/medperf/blob/main/examples/fl_post/fl/README.md Execute the test script with specified parameters for data and label paths. Ensure GPU IDs are correctly set in the script. ```bash bash test.sh --d1 absolute_path --l2 absolute_path ``` -------------------------------- ### Pathology Model Output Structure Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS2024/dummy_model/README.md Example CSV format for pathology model predictions. Includes SubjectID and Prediction columns. ```csv SubjectID,Prediction BraTSPath_cohort_xxxxxxx.png,A BraTSPath_cohort_yyyyyyy.png,B BraTSPath_cohort_zzzzzzz.png,C ``` -------------------------------- ### Run the 'prepare' Task with mlcube Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/data_prep/README.md Execute the 'prepare' task using mlcube to process and standardize data. ```bash mlcube run --task=prepare ``` -------------------------------- ### Clone MedPerf Repository Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/installation.md Clone the MedPerf repository from GitHub and navigate into the cloned directory. This is a necessary step before installing MedPerf from source. ```bash git clone https://github.com/mlcommons/medperf.git cd medperf ``` -------------------------------- ### Run Benchmark Models Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/data_owner_demo.md Executes all models for a specified benchmark. Requires the benchmark ID and the data owner's server UID. Use this command to initiate benchmark model executions. ```bash medperf benchmark run --benchmark 1 --data_uid 1 ``` -------------------------------- ### Sync Changes Source: https://github.com/mlcommons/medperf/blob/main/examples/fl_post/fl/README.md Run this script after modifying configuration files like training_config.yaml or additional_files to apply the changes. ```bash sync.sh ``` -------------------------------- ### Prepare Dataset Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/data_owner_demo.md Prepares a dataset for use by calculating statistics defined by the benchmark owner. Requires the dataset's UID. ```bash medperf dataset prepare --data_uid 1 ``` -------------------------------- ### Attribute Condition for Confidential Space Source: https://github.com/mlcommons/medperf/blob/main/examples/cc/base_image/README.md Example of an attribute condition used for filtering based on software name and support attributes, specifically for confidential computing environments. ```bash --attribute-condition="assertion.swname == 'CONFIDENTIAL_SPACE' \ && 'STABLE' in assertion.submods.confidential_space.support_attributes" ``` -------------------------------- ### Configure MLCube with Docker Build Strategy Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS2023/synthesis_metrics/project/README.md Navigates to the mlcube directory and configures the MLCube, specifying the Docker build strategy to 'always'. ```bash cd ../mlcube mlcube configure -Pdocker.build_strategy=always ``` -------------------------------- ### Define Demo Dataset Paths Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/benchmark_owner_demo.md Create a paths.yaml file to specify the location of data records and labels for the Data Preparator container. ```yaml data_path: demo_data/images labels_path: demo_data/labels ``` -------------------------------- ### Deploy MLCube with GaNDLF Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS2023/inpainting_metrics/setup/README.md Deploys an MLCube of type 'metrics' using the gandlf_deploy command. This command requires specifying template, output, and entrypoint paths. ```bash gandlf_deploy -t docker --mlcube-type metrics -r ./mlcube_template -o ../mlcube -e ./entrypoint.py ``` -------------------------------- ### Example .json Label File Structure (Format 2) Source: https://github.com/mlcommons/medperf/blob/main/examples/SurgMLCube/surg_prep/README.md An alternative .json structure for labels includes 'timestamp', 'duration', and a nested 'label' object with a 'name' field. Timestamps and durations are in milliseconds. ```json { 'timestamp' : 'duration' : 'label' : { 'name': } } ``` -------------------------------- ### JavaScript for Resuming Running Tasks Source: https://github.com/mlcommons/medperf/blob/main/cli/medperf/web_ui/templates/settings.html This JavaScript code adds event listeners to resume specific running tasks when the DOM is loaded. It targets forms for getting, deleting, or submitting client certificates. ```javascript document.addEventListener("DOMContentLoaded", function() { resumeRunningTask("#get-cert-form"); }); ``` ```javascript document.addEventListener("DOMContentLoaded", function() { resumeRunningTask("#delete-cert-form"); }); ``` ```javascript document.addEventListener("DOMContentLoaded", function() { resumeRunningTask("#submit-cert-form"); }); ``` -------------------------------- ### Package Demo Dataset Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/benchmark_owner_demo.md Compress the demo_data directory and paths.yaml file into a tarball for hosting. This command should be run from within the medperf_tutorial directory. ```bash cd medperf_tutorial tar -czf demo_data.tar.gz demo_data paths.yaml cd .. ``` -------------------------------- ### Build OpenFL Base Image Source: https://github.com/mlcommons/medperf/blob/main/examples/fl_post/fl/project/README.md Clone the OpenFL repository, checkout a specific commit, build the base Docker image, and clean up the repository. This is a prerequisite for building the MLCube. ```bash git clone https://github.com/securefederatedai/openfl.git cd openfl git checkout e6f3f5fd4462307b2c9431184190167aa43d962f docker build -t local/openfl:local -f openfl-docker/Dockerfile.base . cd .. rm -rf openfl ``` -------------------------------- ### Build MedPerf Container Source: https://github.com/mlcommons/medperf/blob/main/examples/fl/fl/project/README.md Navigate to the project directory and execute the build script to create the MedPerf container. Use the -b flag to also build the base container. ```bash cd .. bash build.sh ``` -------------------------------- ### Submitting a Model Container with Additional Files (Direct Download Link) Source: https://github.com/mlcommons/medperf/blob/main/docs/concepts/hosting_files.md When submitting a model container, use this flag to specify the direct download link for any additional hosted files, such as model weights or code archives. Ensure files have anonymous public read access. ```bash --additional-file ``` -------------------------------- ### Clone CheXpert Preparation Cube Source: https://github.com/mlcommons/medperf/blob/main/examples/ChestXRay/chexpert_prep/README.md Clones the medical repository, fetches a specific branch containing the CheXpert preparation cube, and checks out that branch. ```bash git clone https://github.com/mlcommons/medical.git && cd ./medical git fetch origin pull/3/head:cubes && git checkout cubes cd ./cubes/chexpert_prep/mlcube ``` -------------------------------- ### List Available Benchmarks Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/data_owner_demo.md Use this command to find the ID of the benchmark you wish to participate in. The ID is required for data registration and preparation. ```bash medperf benchmark ls ``` -------------------------------- ### List Submitted Benchmarks Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/benchmark_owner_demo.md Check the server UID of your submitted benchmark by listing your benchmarks. ```bash medperf benchmark ls --mine ``` -------------------------------- ### Register Training Container Hosted Files URL Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/webui_training.md Provide the URL to the hosted tarball file containing initial model weights for the training container. ```text https://storage.googleapis.com/medperf-storage/init_weights_flower.tar.gz ``` -------------------------------- ### Submit Cloud Build for Production Deployment Source: https://github.com/mlcommons/medperf/blob/main/flca/README.md Submits a cloud build job using a specified configuration file and substitutions. ```shell gcloud builds submit --config=cloudbuild.yaml --substitutions=SHORT_SHA=$TAG ``` -------------------------------- ### Build FeTS Tool Docker Image Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS2023/synthesis_metrics/project/README.md Builds the FeTS tool Docker image using the Dockerfile in the current directory and tags it as 'local/fets-tool:1.0.3'. ```bash docker build -t local/fets-tool:1.0.3 . ``` -------------------------------- ### Register Training Container Config Path Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/webui_training.md Specify the path to the container configuration file for the training container. ```yaml medperf_tutorial/fl_container/container_config.yaml ``` -------------------------------- ### Register Data Preparation Container Parameters Path Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/webui_training.md Specify the path to the parameters file for the data preparation container. ```yaml medperf_tutorial/data_preparator/workspace/parameters.yaml ``` -------------------------------- ### List My Submitted Containers Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/benchmark_owner_demo.md Check the server UIDs assigned to your submitted containers by running this command. ```bash medperf container ls --mine ``` -------------------------------- ### Benchmark Dataset URL Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/webui_full_tutorial.md Provides the URL for the Demo/Toy Dataset required for benchmark registration. ```url https://storage.googleapis.com/medperf-storage/chestxray_tutorial/demo_data.tar.gz ``` -------------------------------- ### Associate Dataset with Benchmark Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/data_owner_demo.md Use this command to associate your dataset with a specific benchmark. It first runs the benchmark's reference model for compatibility checks and then prompts for confirmation to send association request information. ```bash medperf dataset associate --benchmark_uid 1 --data_uid 1 ``` -------------------------------- ### List Datasets Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/data_owner_demo.md Lists all datasets registered by the current user. Useful for retrieving the data_uid needed for other operations. ```bash medperf dataset ls --mine ``` -------------------------------- ### Execute the 'statistics' Task Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/data_prep/README.md Run the 'statistics' task using mlcube to generate data insights. ```bash mlcube run --task=statistics ``` -------------------------------- ### Build Production Docker Image Source: https://github.com/mlcommons/medperf/blob/main/flca/README.md Builds the Docker image for the production environment using the Dockerfile.prod. ```shell docker build -t tmptag -f Dockerfile.prod . ``` -------------------------------- ### Run MLCube Tasks Source: https://github.com/mlcommons/medperf/blob/main/examples/ChestXRay/chexpert_prep/README.md Executes different tasks of the MLCube data preparation cube on a local machine using the Docker runner. Tasks include preprocessing, sanity checks, and statistics calculation. ```bash mlcube run --task preprocess # Creates new version of the data into /data mlcube run --task sanity_check # checks that the output format is okay mlcube run --task statistics # Calculates data statistics into statistics.yaml ``` -------------------------------- ### Register Data Preparation Container Config Path Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/webui_training.md Specify the path to the container configuration file for the data preparation container. ```yaml medperf_tutorial/data_preparator/container_config.yaml ``` -------------------------------- ### Show Local Benchmark Results Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/data_owner_demo.md Displays the local results of a benchmark execution using its result UID. Use this command to inspect the outcome of a specific benchmark run before submission. ```bash medperf result show_local_results 2 ``` -------------------------------- ### Check Resulting Data Source: https://github.com/mlcommons/medperf/blob/main/examples/BraTS/data_prep/README.md List the contents of the workspace/data directory to verify the prepared data. ```bash ls workspace/data ``` -------------------------------- ### Submit a Benchmark Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/benchmark_owner_demo.md Submit a new benchmark with its associated metadata and containers. The --operational flag indicates that the benchmark is ready for operational use. ```bash medperf benchmark submit \ --name tutorial_bmk \ --description "MedPerf demo bmk" \ --demo-url "{{ demo_url }}" \ --data-preparation-container 1 \ --reference-model 1 \ --evaluator-container 3 \ --operational ``` -------------------------------- ### Request Model Association with Benchmark Source: https://github.com/mlcommons/medperf/blob/main/docs/getting_started/model_owner_demo.md Request to associate your model with a specific benchmark. This command initiates a compatibility check and prompts for confirmation to send the association request. ```bash medperf model associate --benchmark 1 --model_uid 2 ```