### Initial Development Setup Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/CLAUDE.md Sets up a virtual environment and installs all packages in development mode. Use this for initial project setup. ```bash python -m venv idmtools idmtools\Scripts\activate # Windows python dev_scripts/bootstrap.py ``` -------------------------------- ### Install Dependencies Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/singularity/covasim/README.md Run this command after creating a virtual environment to install all necessary project dependencies. ```bash pip install -r requirements.txt ``` -------------------------------- ### Example IDMTools INI Configuration Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/idmtools_ini_alias/README.md Illustrates platform configuration when an idmtools.ini file is present. No specific setup is required beyond having the file. ```python example_idmtools_ini_config.py ``` -------------------------------- ### Install and Serve MkDocs Locally Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/development/release-process.md Install documentation dependencies and serve the documentation locally for development. Auto-reloads on changes. ```bash pip install -r docs/requirements.txt mkdocs build mkdocs serve ``` -------------------------------- ### Install Nano Text Editor Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/dev_scripts/linux-test-env/README.md Install the nano text editor within the container. This is an example of installing additional packages needed for development or debugging. ```bash sudo apt install nano ``` -------------------------------- ### Install idmtools File and Process Platform Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Installs packages for the File and Process platforms. ```bash pip install idmtools-platform-general ``` -------------------------------- ### Development Installation Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md Install the required packages for development. Use `make setup-dev` on Unix-like systems or `pymake setup-dev` on Windows. Alternatively, use the bootstrap script. ```bash make setup-dev ``` ```bash python dev_scripts/bootstrap.py ``` ```bash pymake setup-dev ``` ```bash python dev_scripts\bootstrap.py ``` -------------------------------- ### Install idmtools with specific platform support Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/faq/index.md Use this command to install idmtools with support for only the platforms you need, reducing installation size. ```bash pip install idmtools[platform] ``` -------------------------------- ### Platform Configuration File Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/platforms.md Example configuration file for setting up platforms. ```APIDOC ## Platform Configuration ### Configuration File Example `~/.idmtools/idmtools.ini`: ```ini [My_Container] type = Container max_workers = 4 job_dirrectory = /tmp/idmtools ``` ``` -------------------------------- ### Setup Development Environment Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/dev_scripts/linux-test-env/README.md Run this command before building to create necessary worker containers. ```bash make setup-dev ``` -------------------------------- ### Container Platform Configuration File Example Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/platforms.md An example INI configuration file for setting up a Container platform, specifying type, max workers, and job directory. ```ini [My_Container] type = Container max_workers = 4 job_dirrectory = /tmp/idmtools ``` -------------------------------- ### Install idmtools Platform Components from Development Branch Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md Use this command to install the idmtools_platform_comps package directly from the 'dev' branch of the GitHub repository. Ensure you have pip installed. ```bash pip install git+https://github.com/InstituteforDiseaseModeling/idmtools.git@dev#egg="idmtools_platform_comps&subdirectory=idmtools_platform_comps" ``` -------------------------------- ### Run Simple Sweep Example Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/parameter_sweeps/README.md Navigate to the example directory and run the basic parameter sweep script. This will create and run simulations locally, saving results to the idmtools working directory. ```bash cd examples/tutorials/parameter_sweeps python simple_sweep.py ``` -------------------------------- ### Initialize Container Platform Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_platform_container/README.md This is the basic example for initializing the Container Platform. Specify the job directory to store experiment outputs. ```python from idmtools.core.platform_factory import Platform platform = Platform('CONTAINER', job_directory='') ``` -------------------------------- ### Install idmtools Slurm Platform Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Installs packages for the Slurm platform. ```bash pip install idmtools-platform-slurm ``` -------------------------------- ### Install idmtools Local Container Platform Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Installs packages for the local Container platform. ```bash pip install idmtools-platform-container ``` -------------------------------- ### Install idmtools CLI Tools Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Installs the command-line interface tools for idmtools. ```bash pip install idmtools-cli ``` -------------------------------- ### Developer Installation from Source Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Installs the development version of idmtools from source code using the provided bootstrap script. ```bash python dev_scripts/bootstrap.py ``` -------------------------------- ### Install IDMtools with Platform Support Source: https://context7.com/institutefordiseasemodeling/idmtools/llms.txt Install IDMtools using pip, specifying desired platform support. Use '[full]' for all platforms. ```bash pip install idmtools[full] ``` ```bash pip install idmtools[idm] ``` ```bash pip install idmtools[slurm] ``` ```bash pip install idmtools[container] ``` -------------------------------- ### Install MkDocs and Plugins Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/SKILL.md Install MkDocs, the Material theme, and essential plugins for Python documentation, Markdown extensions, and diagram support. Use this command to set up the necessary tools for migrating your documentation. ```bash pip install mkdocs mkdocs-material mkdocstrings[python] \ mkdocs-gen-files mkdocs-literate-nav mkdocs-section-index \ mkdocs-autorefs pymdown-extensions ``` ```bash pip install mkdocs-plantuml-local # or handle via pre-rendered images ``` -------------------------------- ### Example Override Configuration (Calculon) Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/idmtools_ini_alias/README.md Demonstrates overriding platform configuration without an idmtools.ini file, specifically for the 'calculon' environment. This is useful for dynamic configuration. ```python example_override_config_calculon.py ``` -------------------------------- ### Example idmtools Version Output Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_cli/README.md This is an example of the output you can expect when running the 'idmtools version' command, showing versions of the core tool and its plugins. ```bash emodpy Version: 1.3.0 Plugins: EMODTask idmtools Version: 3.1.0 Plugins: CommandTask idmtools-cli Version: 3.1.0 idmtools-models Version: 3.1.0 Plugins: JSONConfiguredPythonTask JSONConfiguredRTask JSONConfiguredTask PythonTask RTask ScriptWrapperTask TemplatedScriptTask idmtools-platform-comps Version: 3.1.0 Plugins: COMPSPlatform SSMTPlatform idmtools-platform-slurm Version: 3.1.0 Plugins: SlurmPlatform ``` -------------------------------- ### Example Folder Structure with Suite Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_platform_container/README.md Illustrates the default directory structure when a suite and experiment are defined, showing name truncation and UUID inclusion. ```bash job_directory/ └── s_my_very_long_suite_name_for_m_12345678-9abc-def0-1234-56789abcdef0/ └── e_test_experiment_with_calibrati_abcd1234-5678-90ef-abcd-1234567890ef/ └── 7c9e6679-7425-40de-944b-e07fc1f90ae7/ ``` -------------------------------- ### Specify Docker Image (General) Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/container.md Example of how to specify a Docker image when creating a Platform instance. This is a general configuration option. ```python platform = Platform('Container', job_directory='any_dir', docker_image='my_docker_image:x.x.x') ``` -------------------------------- ### Install Covasim (Optional) Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_platform_comps/prototypes/covasim_model/README.md Install Covasim from PyPi if you need local development support. This is optional for running simulations on the cluster. ```bash pip install covasim ``` -------------------------------- ### Install idmtools packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/tutorials/python-models.md Install the idmtools package with the necessary extras for your target platform (COMPS, container, Slurm, or all). ```bash pip install idmtools[idm] # COMPS platform pip install idmtools[container] # Container platform (local Docker) pip install idmtools[slurm] # Slurm platform pip install idmtools[full] # All platforms ``` -------------------------------- ### Install Other Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md Placeholder for installing additional idmtools-related packages. Adapt the pattern shown for core and platform components. ```bash ... pip install other packages ``` -------------------------------- ### Install WSL2 on Windows Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md Commands to install and set WSL2 as the default version on Windows. This is often a prerequisite for containerized development. ```powershell wsl --install ``` ```powershell wsl --set-default-version 2 ``` ```powershell wsl --set-version 2 ``` -------------------------------- ### mpirun Output Example Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/platform_container/perf_tests/perf_test_result_burnin.txt Shows the mpirun output format when multiple tasks are contending for resources. ```text [0] Rank 0 ... [1] Rank 1 ... [2] Rank 2 ... [3] Rank 3 ... ``` -------------------------------- ### Install idmtools for COMPS Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Installs idmtools core, CLI, models, and COMPS platform packages for running simulations in COMPS. ```bash pip install idmtools[idm] ``` -------------------------------- ### Install and Configure mkdocs-redirects Plugin Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/SKILL.md Install the mkdocs-redirects plugin and configure it in mkdocs.yml to handle page redirects. The redirect map is typically parsed from a redirects.txt file. ```bash pip install mkdocs-redirects ``` ```yaml plugins: - redirects: redirect_maps: 'old-page.md': 'new-page.md' ``` -------------------------------- ### Example Output Files Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/platform_container/perf_tests/perf_test_result_burnin.txt Illustrates the typical state file output generated during a burn-in run. ```text state-03650.dtk 17 MB state-07300.dtk 17 MB ``` -------------------------------- ### Install Individual idmtools Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Install specific idmtools packages to create minimal environments. Available packages include core, CLI, platform support, models, and testing utilities. ```bash pip install idmtools ``` ```bash pip install idmtools-cli ``` ```bash pip install idmtools-platform-comps ``` ```bash pip install idmtools-models ``` ```bash pip install idmtools-platform-general ``` ```bash pip install idmtools-platform-slurm ``` ```bash pip install idmtools-platform-container ``` ```bash pip install idmtools-test ``` -------------------------------- ### Example Folder Structure without Suite Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_platform_container/README.md Shows the directory structure when only an experiment is defined, omitting the suite-specific directory. ```bash job_directory/ └── e_test_experiment_with_calibrati_abcd1234-5678-90ef-abcd-1234567890ef/ └── 7c9e6679-7425-40de-944b-e07fc1f90ae7/ ``` -------------------------------- ### Full idmtools Installation Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Installs all idmtools packages, excluding idmtools-test. Includes core, CLI, models, COMPS, General, Container, and Slurm platforms. ```bash pip install idmtools[full] ``` -------------------------------- ### Install idmtools for Docker Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Installs idmtools core, CLI, models, General, and Container platform packages for running simulations in Docker containers locally. ```bash pip install idmtools[container] ``` -------------------------------- ### Build and Serve Documentation Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md Build the documentation and launch a local web server to preview changes. The server watches for file modifications and reloads automatically. ```bash make build-docs ``` ```bash make build-docs-server ``` -------------------------------- ### Troubleshoot Stuck Dev Install Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md If the development installation gets stuck, it might be due to changed company credentials. Try installing the problematic package directly using `pip install -e .` within its directory. ```bash cd idmtools_platform_comps pip install -e . ``` -------------------------------- ### Install and Update Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_platform_container/docker_image/rocky_meta_runtime.txt Installs essential packages like rpm, dnf-plugins-core, python3, python3-devel, mpich, epel-release, wget, and nano, and updates existing packages. ```dockerfile RUN /bin/sh -c yum -y install rpm dnf-plugins-core && yum -y update && dnf -y install python3 python3-devel mpich epel-release wget nano # buildkit ``` -------------------------------- ### Get idmtools-cli Version Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_cli/README.md Use the 'version' command to check the version of idmtools and its installed plugins. ```bash idmtools version ``` -------------------------------- ### Basic Container Platform Usage Example Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/container.md Demonstrates initializing the ContainerPlatform, defining a command task, creating an experiment, and running it. Ensure Docker is running and accessible. ```python from idmtools.entities.command_task import CommandTask from idmtools.entities.experiment import Experiment from idmtools_platform_container.container_platform import ContainerPlatform # Initialize the platform from idmtools.core.platform_factory import Platform platform = Platform('Container', job_directory="destination_directory") # Or # platform = ContainerPlatform(job_directory="destination_directory") # Define task command = "echo 'Hello, World!'" task = CommandTask(command=command) # Run an experiment experiment = Experiment.from_task(task, name="example") experiment.run(platform=platform) ``` -------------------------------- ### Get Current Package Version Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/development/release-process.md Execute this script to display the current version of all installed packages, as determined by setuptools-scm. ```bash python dev_scripts/get_version.py ``` -------------------------------- ### Instantiate PlatformAnalysis with Analyzers Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/data-analysis/platform-analysis.md Example of instantiating PlatformAnalysis, passing analyzer classes and their arguments. Ensure analyzer classes are passed, not instances. ```python from idmtools.analysis.download_analyzer import DownloadAnalyzer from myanalyzers import MyCSVAnalyzer analysis = PlatformAnalysis( platform=platform, experiment_ids=['exp-id'], analyzers=[DownloadAnalyzer, MyCSVAnalyzer], # classes, not instances analyzers_args=[ {'filenames': ['output/InsetChart.json']}, {'filenames': ['output/data.csv'], 'output_path': 'results'}, ] ) ``` -------------------------------- ### Run Linux Test Environment Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/dev_scripts/linux-test-env/README.md Start the Linux test environment container. This command should be run from the directory containing the docker-compose file. ```bash docker-compose run linuxtst ``` -------------------------------- ### Build Documentation Locally Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Steps to build and serve the idmtools documentation locally. Requires creating a virtual environment, installing dependencies, and running build commands. ```bash pip install -r docs/requirements.txt python dev_scripts/bootstrap.py mkdocs build mkdocs serve ``` -------------------------------- ### Install idmtools Core from Development Branch Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md Use this command to install the idmtools_core package directly from the 'dev' branch of the GitHub repository. Ensure you have pip installed. ```bash pip install git+https://github.com/InstituteforDiseaseModeling/idmtools.git@dev#egg="idmtools&subdirectory=idmtools_core" ``` -------------------------------- ### Install Package Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/container.md Add this dependency to your Dockerfile to install a package. ```dockerfile RUN pip install my-package ``` -------------------------------- ### Install idmtools Core Package Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Installs only the core functionality of idmtools. ```bash pip install idmtools ``` -------------------------------- ### Resource Usage Over Time (Initialization Phase) Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/platform_container/perf_tests/perf_test_result_burnin.txt Tracks memory and CPU usage during the script start and initialization phase, showing significant memory growth. ```text Timestamp (approx) PID %CPU %MEM VSZ (virtual) RSS (resident) Comment / Interpretation 15:39 ( t + 0 min ) 2427662 101 35.8 6.7 GB 5.9 GB Script start – allocating data structures / loading input with "python example_container.py" 15:39 ( t + ~1 min ) 2427662 101 36.9 6.9 GB 6.0 GB Memory continues to grow during initialization. 15:39 ( t + ~2 min ) 2427662 101 38.9 7.2 GB 6.4 GB Peak allocation phase – approaching 40 % of system RAM. ``` -------------------------------- ### Install JFrog CLI on Windows Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/dev_scripts/jfrog_stage_to_prod.md Installs the JFrog CLI using PowerShell. Ensure you have administrator privileges. After installation, you will be prompted to log in via your web browser and configure a new server. ```powershell powershell "Start-Process -Wait -Verb RunAs powershell '-NoProfile iwr https://releases.jfrog.io/artifactory/jfrog-cli/v2-jf/[RELEASE]/jfrog-cli-windows-amd64/jf.exe -OutFile $env:SYSTEMROOT\system32\jf.exe'" ``` ```powershell jf login ``` ```powershell jf c add ``` ```powershell jf config show ``` -------------------------------- ### Create and Run an Experiment Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/core.md Demonstrates creating an Experiment from a SimulationBuilder and running it on a specified platform. Ensure the platform is correctly configured. ```python experiment = Experiment.from_builder(builder, task, name="My Experiment") experiment.run(platform=platform, wait_until_done=True) ``` -------------------------------- ### Install idmtools-test package Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Installs the idmtools-test package, which may be needed for running unittests. ```bash pip install idmtools-test ``` -------------------------------- ### Set up a virtual environment Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/container.md Create and activate a Python virtual environment for idmtools. This isolates project dependencies. ```bash python -m venv container_env # Windows container_env\Scripts\activate # Linux/macOS source container_env/bin/activate ``` -------------------------------- ### Common Development Commands Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/DEVELOPMENT_README.md Explore available development tasks by running `make help` or `pymake help` on Windows. These commands cover environment setup, cleaning, linting, and testing. ```bash setup-dev - Setup dev environment(assumes you already have a virtualenv) setup-dev-no-docker - Setup dev environment(assumes you already have a virtualenv) exlcuding docker builds for the local platform clean - Clean up temporary files clean-all - Deep clean project lint - Lint package and tests test - Run Unit tests test-all - Run Tests that require docker and external systems coverage - Run tests and generate coverage report that is shown in browser ``` -------------------------------- ### Development Installation of idmtools Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Clones the idmtools repository and installs it in editable mode for development purposes. ```bash git clone https://github.com/institutefordiseasemodeling/idmtools.git cd idmtools python dev_scripts/bootstrap.py ``` -------------------------------- ### Setuptools-scm Configuration Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/development/release-process.md TOML configuration for setuptools-scm to manage dynamic versioning based on git tags. ```toml [project] dynamic = ["version"] [tool.setuptools_scm] root = ".." version_scheme = "no-guess-dev" local_scheme = "no-local-version" fallback_version = "0.0.0.dev0" ``` -------------------------------- ### Install idmtools COMPS Platform Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Installs packages specifically for the IDM COMPS platform. ```bash pip install idmtools-platform-comps ``` -------------------------------- ### Create Platform Instance by Name Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/platforms.md Instantiate a platform using its name. Configuration can be provided directly or loaded from a config file. ```python from idmtools.core.platform_factory import Platform # Create platform by name platform = Platform("COMPS") # With configuration platform = Platform( "COMPS", endpoint="https://comps.idmod.org", environment="Bayesian" ) # From config file platform = Platform("COMPS", missing_ok=True) ``` -------------------------------- ### Install idmtools Python/R Models Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Installs packages for developing Python and R models with idmtools. ```bash pip install idmtools-models ``` -------------------------------- ### Remove Singularity Setup Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/container.md When migrating from COMPS to Container Platform, remove Singularity setup code. ```python # Remove this when using Container Platform emod_task.set_sif(sif_path) ``` -------------------------------- ### Verify idmtools Installation using Python Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Checks the installed version of idmtools by importing it in a Python script. ```python python -c "import idmtools; print(idmtools.__version__)" ``` -------------------------------- ### Create and Run a Simulation Suite Source: https://context7.com/institutefordiseasemodeling/idmtools/llms.txt This snippet demonstrates how to create a Suite, add experiments to it, and run all experiments within the suite. It also shows how to retrieve the suite ID and the status of individual experiments. ```python suite = Suite(name="Model Comparison Study") suite.update_tags({"study": "baseline_comparison", "version": "1.0"}) # Create suite on platform first platform.create_items([suite]) # Add experiments to suite suite.add_experiment(exp1) suite.add_experiment(exp2) # Run all experiments in suite suite.run(platform=platform, wait_until_done=True) print(f"Suite ID: {suite.id}") for exp in [exp1, exp2]: print(f" Experiment '{exp.name}': {exp.status}") ``` -------------------------------- ### Verify idmtools Installation using CLI Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/getting-started/installation.md Checks the installed version of idmtools using the command-line interface. ```bash idmtools --version ``` -------------------------------- ### Initialize Container Platform (Default) Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/platform_container/perf_tests/perf_test_result_burnin.txt Sets up the ContainerPlatform with default settings, suitable for balanced parallelization. The default max_job is 4. ```python platform = Platform("Container", job_directory="DEST") # default max_job = 4 ``` -------------------------------- ### Install Extra Python Packages at Runtime Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_platform_container/docker_image/debian/README.md When creating a Platform object, you can specify additional Python packages to be installed within the container at runtime using the `extra_packages` argument. Alternatively, use the `idmtools container install` CLI command. ```python from idmtools.core.platform_factory import Platform platform = Platform('CONTAINER', docker_image='debian12-env', extra_packages=['emodpy~=2.0.0', 'pytest']) ``` -------------------------------- ### Create and Run a Suite of Experiments Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/core.md Illustrates creating a Suite to group multiple experiments and then running the entire suite on a platform. This is useful for managing complex studies. ```python from idmtools.entities.suite import Suite suite = Suite(name="Complete Study") suite.add_experiment(exp1) suite.add_experiment(exp2) suite.run(platform=platform) ``` -------------------------------- ### Install Python Requirements Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/idmtools_platform_container/docker_image/rocky_meta_runtime.txt Installs Python dependencies from a requirements file. Ensure requirements.txt is present in the build context. ```dockerfile RUN CONTAINER_VERSION=0.0.1 /bin/sh -c bash -c "pip3 install -r /tmp/requirements.txt" # buildkit ``` -------------------------------- ### Instantiate a Platform Connection Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/core.md Provides an example of creating a platform instance using the Platform factory. Specify the platform name and any necessary connection details like endpoint and environment. ```python from idmtools.core.platform_factory import Platform platform = Platform("COMPS", endpoint="...", environment="...") ``` -------------------------------- ### Build Ubuntu Base Image Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/singularity/covasim/README.md Execute this script to build the base Ubuntu image for the project. ```python python3 build_ubuntu_base.py ``` -------------------------------- ### Build and Serve MkDocs Site Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/SKILL.md Commands to build the MkDocs static site, serve it locally with live reloading, and check for broken links using linkchecker. ```bash mkdocs build --strict ``` ```bash mkdocs serve ``` ```bash mkdocs build && linkchecker site/ ``` -------------------------------- ### Show Container Platform CLI Help Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/cli/index.md Use this command to display help information for the Container platform CLI, including available commands and options. ```bash idmtools container --help ``` -------------------------------- ### Example Override Configuration (Belegost) Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/examples/idmtools_ini_alias/README.md Shows how to override platform configuration without an idmtools.ini file, tailored for the 'belegost' environment. This method allows for flexible configuration management. ```python example_override_config_belegost.py ``` -------------------------------- ### Install idmtools with COMPS support Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/comps.md Install the necessary idmtools package for COMPS platform integration. This can be done using pip. ```bash pip install idmtools[idm] # or pip install idmtools-platform-comps ``` -------------------------------- ### Configure COMPS Platform Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/platforms.md Set up a COMPS platform instance with specific parameters like endpoint, environment, priority, and number of cores. ```python from idmtools.core.platform_factory import Platform platform = Platform( "COMPS", endpoint="https://comps.idmod.org", environment="Nibbler", priority="Highest", num_cores=1 ) ``` -------------------------------- ### Run Experiment on COMPS Platform Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/platforms.md Example of creating a COMPS platform instance and running an experiment, with options to wait for completion and filter simulations by tags. ```python # Create platform platform = Platform( "COMPS", endpoint="https://comps.idmod.org", environment="Calculon" ) # Run experiment experiment.run(platform=platform, wait_until_done=True) # Filter simulations by tags simulations = platform.filter_simulations_by_tags( suite_id, item_type=ItemType.SUITE, tags={"a": lambda v: 1 <= v <= 2, "sim_tag": "test_tag"}, entity_type=True) # Get experiment details exp = platform.get_item(exp_id, ItemType.EXPERIMENT) print(f"Status: {exp.status}") print(f"Simulations: {len(exp.simulations)}") ``` -------------------------------- ### Install idmtools[full] on macOS Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md On macOS, square brackets in pip install commands may need to be escaped with a backslash. ```bash pip install idmtools\[full\] ``` -------------------------------- ### Load Templates from Files Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/models.md This example shows how to define a template content that can be loaded from a file, demonstrating Jinja2 syntax for variable substitution in a JSON configuration. ```json # config_template.json """ { "population": {{population}}, "parameters": { "beta": {{beta}}, "gamma": {{gamma}} }, "output": { "format": "{{output_format}}", "directory": "{{output_dir}}" } } """ ``` -------------------------------- ### Force Start Container Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/container.md Forces the start of a new or previously stopped container. Use when a container needs to be explicitly restarted. ```python with Platform("Container", job_directory="DEST", force_start=True) as platform: experiment.run(wait_until_done=True) ``` -------------------------------- ### Create and Run a Basic Experiment Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/user-guide/creating-simulations.md This snippet demonstrates the basic workflow for creating a Python task, wrapping it in an experiment, and running it on a specified platform. Ensure 'model.py' exists locally. ```python from idmtools.core.platform_factory import Platform from idmtools.entities.experiment import Experiment from idmtools.entities.simulation import Simulation from idmtools_models.python.json_python_task import JSONConfiguredPythonTask # Create platform platform = Platform('Container', job_directory='anywhere') # Create a python task that runs a model.py. Make sure model.py exists in local dir task = JSONConfiguredPythonTask(script_path="model.py") # Add to experiment (required for commission) experiment = Experiment.from_task( task, name="My First Experiment" ) # Commission experiment.run(wait_until_done=True, platform=platform) # Check status print(f"Status: {experiment.status}") ``` -------------------------------- ### Install Package from TestPyPI Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/development/release-process.md Installs a specific version of an idmtools package from TestPyPI, falling back to the main PyPI if the version is not found. ```bash pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ idmtools==3.x.x ``` -------------------------------- ### Run Experiment on Local Slurm Platform Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/api/platforms.md Example of configuring a local Slurm platform with additional parameters and running an experiment. Includes refreshing the experiment status. ```python # Create platform platform = Platform("SLURM_LOCAL", job_directory=".", mail_user="test@test.com", account="test_acct", mail_type="begin", mem_per_cpu=2048) ) # Run experiment.run(platform=platform, wait_until_done=True) # Check job status experiment.refresh_status(platform) ``` -------------------------------- ### Install idmtools for Slurm Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/README.md Installs idmtools core, CLI, models, and Slurm platform packages for running simulations in a Slurm cluster. ```bash pip install idmtools[slurm] ``` -------------------------------- ### Install Extra Packages in Container Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/docs/platforms/container.md Installs specified Python packages into the container at runtime. Ensure packages are available on PyPI or a configured index. ```python from idmtools.assets import Asset, AssetCollection from idmtools.core.platform_factory import Platform from idmtools.entities.command_task import CommandTask from idmtools.entities.experiment import Experiment import os platform = Platform("Container", job_directory="DEST", new_container=True, extra_packages=['astor', 'numpy', 'pytest'], docker_image="ghcr.io/institutefordiseasemodeling/container-rocky-runtime:0.0.6") command = "python Assets/model_file.py" task = CommandTask(command=command) model_asset = Asset(absolute_path=os.path.join("inputs", "extra_packages", "model_file.py")) common_assets = AssetCollection() common_assets.add_asset(model_asset) experiment = Experiment.from_task(task, name="run_with_extra_packages", assets=common_assets) experiment.run(wait_until_done=True, platform=platform) ``` -------------------------------- ### Install idmtools Container Packages Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/MAC_README.md Install only the idmtools container-specific packages if your focus is on running simulations using the local Docker container platform. ```bash pip install idmtools[container] --full-reinstall --no-cache-dir --upgrade ``` -------------------------------- ### Install Full idmtools Package Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/MAC_README.md Install or update all idmtools related packages to the latest version. This command ensures you have the complete set of tools. ```bash pip install idmtools[full] --full-reinstall --no-cache-dir --upgrade ``` -------------------------------- ### Top-Level Development Commands Source: https://github.com/institutefordiseasemodeling/idmtools/blob/main/CLAUDE.md Common commands for setting up development environments, running tests, performing code quality checks, and building documentation from the repository root. ```bash # Setup make setup-dev # Full dev setup with Docker builds make setup-dev-no-docker # Dev setup without Docker builds # Testing make test # Run tests excluding COMPS and Docker make test-smoke # Run smoke tests (fastest) make test-all # Run all tests (serial + parallel) make test-failed # Rerun only failed tests make test-comps # Run COMPS platform tests make test-docker # Run Docker platform tests # Code Quality make lint # Run flake8 on all packages make clean # Clean temporary files make clean-all # Deep clean (requires re-running setup-dev) # Coverage make coverage # Generate coverage report make coverage-report # View coverage in browser # Documentation make build-docs # Build documentation make build-docs-server # Build docs with auto-reload at localhost:8000 ```