Imperial College London RCS User Guide

https://icl-rcs-user-guide.readthedocs.io/en/latest
Admin
A comprehensive user guide for Imperial College London's Research Computing Service, covering High...
Tokens:35,507
Snippets:305
Trust Score:9.6
Update:6 months ago
Show doc for...
Tokens:
Raw
### Activate Virtual Environment and Install Package (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/python

Activates a previously created virtual environment named 'example-env' and installs the 'mypy' package using pip. This allows for isolated package management within the activated environment.

```bash
#!/bin/bash
[user@login ~]$ source ~/venv/example-env/bin/activate
[user@login ~]$ python3 -m pip install mypy
```

--------------------------------

### Example TensorFlow Job Submission Script (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/tensorflow

This is an example PBS submission script for running a TensorFlow job on the HPC. It includes resource requests for CPUs, memory, and GPUs, loads the necessary TensorFlow module, and executes a Python script that utilizes TensorFlow. This script assumes the TensorFlow module has been loaded via EasyBuild.

```bash
#!/bin/bash
#PBS -l select=1:ncpus=4:mem=24gb:ngpus=1
#PBS -l walltime=01:00:00

module purge
module load tools/prod
module load TensorFlow/2.15.1-foss-2023a-CUDA-12.1.1

# Your python code utilising tensorflow e.g.
python tf2-benchmarks.py --model resnet50 --enable_xla --batch_size 256 --num_gpus 1
```

--------------------------------

### Run Hotspots Analysis with Vtune (PBS Script)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-vtune-profiler

An example PBS script to run the 'hotspots' analysis using Intel Vtune Profiler. It includes module loading, environment variable settings for threading and affinity, and the Vtune command to collect data for a LAMMPS simulation.

```bash
#PBS -l walltime=00:05:00
#PBS -l select=1:ncpus=64:mem=200gb
#PBS -N pinning_t2_c2_intelmpi
#PBS -o out.txt
#PBS -e error.txt

# Load the modules
module load imkl-FFTW/2023.1.0-iimpi-2023a
module load VTune/2022.3.0

# some pinning option (this will print some information in the output file about where the threads and processes were launched).
export I_MPI_DEBUG=5
export OMP_NUM_THREADS=64
export KMP_AFFINITY=verbose

cd $PBS_O_WORKDIR

# your job specific commands goes here.
lammps_path="/gpfs/home/lragta/RCS_help/1_janet/lkr/src/lmp_intel_cpu_intelmpi"

temp1=900
ex=0.0
ey=0.0
ez=0.0

# Vtune command goes here.
vtune -collect hotspots -knob sampling-mode=hw ${lammps_path} -l log.lammps -nocite -var temp1 ${temp1} -var ex ${ex} -var ey ${ey} -var ez ${ez} -in alkyl-amor-nvt.in
```

--------------------------------

### Time Command Output Example

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/faq/how-much-resource

Example output from the 'time' command, showing the real, user, and system execution times for a job. This information is crucial for estimating job duration and optimizing resource requests.

```text
real 0m3.743s
user 0m0.303s
sys 0m1.144s
```

--------------------------------

### Load TensorFlow Module via EasyBuild (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/tensorflow

This snippet demonstrates how to load the TensorFlow module using EasyBuild, which provides architecture-specific, GPU-compiled versions. It purges existing modules, loads a base toolchain, and then loads the desired TensorFlow version with its associated CUDA toolkit. This is the recommended installation method for optimal performance.

```bash
[user@login ~]$
module purge
module load tools/prod
module load TensorFlow/2.15.1-foss-2023a-CUDA-12.1.1
```

--------------------------------

### Get Vtune Help

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-vtune-profiler

Commands to retrieve help information for Vtune, either for the general 'collect' command or specific analysis types.

```bash
vtune -help collect
#or more specific
vtune -help collect <analysis_type>
```

--------------------------------

### Verify TensorFlow Installation (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/tensorflow

This script verifies a TensorFlow installation by attempting to list physical GPU devices. It assumes a Conda environment named 'tf2_env' has been activated. A successful execution should output a list of available GPUs, confirming that TensorFlow can utilize the GPU.

```bash
#!/bin/bash
#PBS -lselect=1:ncpus=4:mem=10gb:ngpus=1
#PBS -lwalltime=1:0:0

cd $PBS_O_WORKDIR

eval "$(~/miniforge3/bin/conda shell.bash hook)"
source activate tf2_env

## Verify install:
python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
```

--------------------------------

### Checking Loaded Modules

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/GCC

Demonstrates how to list currently loaded modules using 'module list'. This is useful for verifying that the correct modules, including 'binutils', are active.

```bash
# See all loaded modules
[user@login ~]$ module list
```

--------------------------------

### Install and Search Packages via Conda

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/openondemand

Commands to install R packages using Conda channels and search for available packages. Users are advised to avoid install.packages() to prevent dependency conflicts.

```bash
conda install -c conda-forge r-tidyverse
conda search "package_name"
```

--------------------------------

### Setup R Environment in Jupyter Lab (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/jupyter

This snippet outlines the bash commands to set up a Conda environment for R within a Jupyter Lab session. It includes loading the Conda module, creating a new environment with R and the IRkernel, activating the environment, and installing desired R packages. Dependencies include Conda and R packages like r-irkernel and r-base.

```bash
eval "$(~/miniforge3/bin/conda shell.bash hook)"
conda create -n ir413 r-base=4.1.3 r-irkernel jupyter_client -c conda-forge
source activate ir413
conda search "package_name"
conda install package_name[=version] -c conda-forge
R
```

--------------------------------

### Launch Vtune GUI

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-vtune-profiler

Command to launch the Intel Vtune Profiler graphical user interface. This is typically done after generating the analysis report to visualize the results.

```bash
vtune-gui
```

--------------------------------

### Generate Specific Vtune Reports

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-vtune-profiler

Examples of generating specific types of reports from Vtune results, such as 'gprof-cc' for call tree with time or 'summary' for an overview.

```bash
vtune -R gprof-cc -result-dir r001hs/
vtune -R summary -result-dir r001hs/
```

--------------------------------

### List Available MATLAB Versions

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/matlab

This command lists all installed versions of MATLAB available on the RDS. It requires loading the production EasyBuild stack first. The output shows available versions and their associated module names.

```bash
#!/bin/bash
[user@login ~]$ module load tools/prod
[user@login ~]$ module spider MATLAB
```

--------------------------------

### ParaView Client Configuration Settings

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/paraview

Configuration parameters required within the ParaView desktop client to connect to the HPC server instance.

```text
Name: Imperial RCS
Server Type: Client/Server (reverse connection)
Port: 11111
```

--------------------------------

### Load VTune Module

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-aps

Loads the Intel VTune Profiler module, which is required to use the APS tool. This is a prerequisite for running APS analysis.

```bash
module load VTune/2022.3.0
```

--------------------------------

### List Available ORCA Versions with module spider

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/orca

This command queries the module system to display all available versions of the ORCA software installed on the HPC. It also provides a brief description of ORCA's capabilities. Users can identify the specific version and its associated module name for loading.

```bash
[user@login ~]$
module spider orca

-------------------------------------------------------------------------------------------------------------------------------
  ORCA:
-------------------------------------------------------------------------------------------------------------------------------
    Description:
      ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on
      spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods
      ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat
      environmental and relativistic effects.

     Versions:
        ORCA/5.0.4-gompi-2023a
        ORCA/6.0.0-foss-2023b-avx2
        ORCA/6.0.0-foss-2023b
        ORCA/6.0.0-gompi-2023a-avx2
        ORCA/6.0.1-gompi-2023a-avx2

```

--------------------------------

### Loading binutils Module

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/GCC

Shows the command to load the 'binutils' module, which is a necessary dependency for GCC compilers to function correctly. This is typically loaded after 'GCCcore'.

```bash
[user@login ~]$ module load binutils/2.40-GCCcore-12.3.0
```

--------------------------------

### Configure Thread Pinning in PBS Job Script

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/thread-pinning

A template for a PBS job script demonstrating the configuration of Intel MPI and OpenMP environment variables to enforce thread affinity. This setup helps prevent thread migration across cores, potentially improving execution speed.

```bash
#PBS -l walltime=00:05:00
#PBS -l select=1:ncpus=64:mem=200gb:mpiprocs=16:ompthreads=4
#PBS -N pinning
#PBS -o output.txt
#PBS -e error.txt

# Load the modules
module load imkl-FFTW/2023.1.0-iimpi-2023a

# Pinning options
export I_MPI_DEBUG=5
export I_MPI_PIN_DOMAIN=omp
export I_MPI_PIN_ORDER=bunch

export OMP_NUM_THREADS=4
export KMP_AFFINITY=verbose, scatter

cd $PBS_O_WORKDIR
```

--------------------------------

### General Vtune Analysis Command

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-vtune-profiler

The general syntax for running Vtune analysis. It specifies the analysis type, Vtune options, the application to profile, and its parameters.

```bash
vtune -collect analysis_type <vtune specific options> -- <your_app> <your_app_parameters>
```

--------------------------------

### Searching for Available GCC Modules

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/GCC

Shows how to search for all available GCC compiler modules on the system using the 'module spider GCC' command. This helps in identifying compatible versions.

```bash
[user@login ~]$ module spider GCC
```

--------------------------------

### Example PBS Job Script for Python Environment (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/python

A sample PBS job script that requests resources (walltime, CPU, memory), loads production tools, loads a specific SciPy bundle, activates a virtual environment, changes directory to the job's working directory, and executes a Python script. This demonstrates how to run Python applications within a managed environment on the HPC.

```bash
#!/bin/bash
#PBS -l walltime=1:0:0
#PBS -lselect=1:ncpus=1:mem=4gb

module load tools/prod
module load SciPy-bundle/2022.05-foss-2022a
source ~/venv/example-env/bin/activate

cd $PBS_O_WORKDIR

python3 some_python_script.py
```

--------------------------------

### C++ Hello World Program

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/GCC

A simple C++ program that prints 'Hello' to the console. This serves as a basic test case for compilation.

```cpp
#include <iostream>

int main()
{
    std::cout << "Hello" << std::endl;
    return 0;
}
```

--------------------------------

### Display Conda Channels Configuration

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/conda

Command to display the current list of channels configured in Conda. This is useful for verifying channel settings and troubleshooting package installation issues. The output shows the order in which Conda searches for packages.

```bash
conda config --show channels
```

--------------------------------

### Conda Environment and Package Management Commands

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/conda

A collection of basic Conda commands for managing environments and packages. These commands allow users to list, create, activate, deactivate, search, install, and clean up Conda environments and their associated packages.

```bash
conda env list
conda create -n py39 python=3.9
source activate ENV_NAME
conda deactivate
conda list
conda search PACKAGENAME
conda install PACKAGENAME
conda clean
```

--------------------------------

### Load FreeSurfer Module and Run Binary (Shell)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/freesurfer

Demonstrates loading the FreeSurfer module and executing a binary, which may result in a 'library not found' error if dependent libraries are missing.

```shell
# Load the module
$ module load FreeSurfer/7.4.1-centos8_x86_64

# Use one of the binaries in the module
$ mri_robust_register -all-info
```

--------------------------------

### Generate Rank-to-Rank Communication Report

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-aps

Optional command to generate an interactive HTML report visualizing rank-to-rank communication. This can be useful for analyzing MPI communication patterns.

```bash
# Generate rank to rank html report. Optional and may be useful in some cases.
aps-report -x --format=html aps_result_20231122/
```

--------------------------------

### SSH with X11 Forwarding

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-vtune-profiler

Command to establish an SSH connection with X11 forwarding enabled, which is necessary for launching and using the Vtune GUI on a remote server from a local machine.

```bash
ssh -X username@hostname
```

--------------------------------

### Execute MATLAB Commands from Python

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/python

Demonstrates how to initialize the MATLAB engine, execute commands, call functions, and pass data between Python and MATLAB.

```python
import matlab.engine
import numpy as np

# Start MATLAB
eng = matlab.engine.start_matlab()

# Run a simple command
eng.eval("disp('Hello from MATLAB!')", nargout=0)

# Call a MATLAB function
result = eng.sqrt(16.0)
print(result)

# Pass Python data to MATLAB
x = matlab.double(np.linspace(0, 10, 11).tolist())
y = eng.sin(x)
print(y)

# Quit MATLAB
eng.quit()
```

--------------------------------

### Loading GCC Compiler Modules

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/GCC

Demonstrates the command-line steps to load the necessary GCC compiler modules. It includes loading the 'tools/prod' module and a specific GCC version (e.g., GCC/12.3.0) after purging existing modules.

```bash
[user@login ~]$ module purge
[user@login ~]$ module load tools/prod
[user@login ~]$ module load GCC/12.3.0

# Compile the program
[user@login ~]$ g++ -o HelloWorld HelloWorld.cpp
```

--------------------------------

### Install Bioconductor packages with Conda

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/R

Steps to install Bioconductor packages within an activated Conda R environment. It shows how to search for available Bioconductor packages and then install a specific one, like 'bioconductor-teqc'.

```bash
[user@login ~]$ conda search bioconductor-*
[user@login ~]$ conda install bioconductor-teqc
```

--------------------------------

### Install TensorFlow with Conda (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/tensorflow

This sequence of commands installs TensorFlow using Conda, including CUDA toolkit and cuDNN for GPU support. It creates a new conda environment, activates it, and then installs TensorFlow and necessary NVIDIA libraries. This method requires careful management of dependencies to avoid conflicts.

```bash
[user@login ~]$
conda create -n tf2_env -c conda-forge cudatoolkit=11.8 python=3.11
source activate tf2_env
conda install -c "nvidia/label/cuda-11.8.0" cuda-nvcc
python3 -m pip install nvidia-cudnn-cu11==8.6.0.163
mkdir -p $CONDA_PREFIX/etc/conda/activate.d
echo 'CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/:$CUDNN_PATH/lib' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
source $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
python3 -m pip install tensorflow==2.12.*
```

--------------------------------

### Stage workflow data via $TMPDIR in PBS script

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/best-practice

Demonstrates how to copy input files to the compute node's local $TMPDIR, execute a task (e.g., SAMtools), and copy results back to persistent storage to improve I/O performance.

```bash
#!/bin/bash
#PBS -l walltime=02:00:00
#PBS -l select=1:ncpus=2:mem=4gb
#PBS -N 1-10

module load SAMtools/1.19.2-GCC-13.2.0

# Copy input file to $TMPDIR
cp $HOME/project1/inputs/sample_${PBS_ARRAY_INDEX}.sam $TMPDIR

# Change to the $TMPDIR
cd $TMPDIR

# Run application
samtools view -S -b sample_${PBS_ARRAY_INDEX}.sam > sample_${PBS_ARRAY_INDEX}.bam

# Copy required files back
cp $TMPDIR/sample_${PBS_ARRAY_INDEX}.bam $HOME/project1/outputs/
```

--------------------------------

### Install and Use memory_profiler for Python

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/python

Installs the memory_profiler library via pip and demonstrates how to use the @profile decorator to track memory consumption line-by-line in a Python function.

```bash
python3 -m pip install memory-profiler
```

```python
from memory_profiler import profile

@profile
def func():
    x = [1] * (10 ** 7)
    y = [2] * (4 * 10 ** 8)
    y = y[:int(len(y)/2)]
    del x
    return y

if __name__ == '__main__':
    func()
```

--------------------------------

### Example of Modified Temporary Directory Path

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/freesurfer

This example demonstrates how the `TMPDIR` variable is modified by replacing square brackets with underscores, as performed by the line `FREESURFER_TMP="${TMPDIR//[[]]/_}"` in the script.

```text
/var/tmp/pbs.1148088_1_.pbs-7
```

--------------------------------

### Load SciPy Bundle and Create Virtual Environment (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/python

Loads a specific SciPy bundle which includes a compatible Python version, then creates a virtual environment named 'example-env' in the user's home directory using `virtualenv`. This is recommended for scientific packages.

```bash
#!/bin/bash
[user@login ~]$ module load SciPy-bundle/2022.05-foss-2022a
[user@login ~]$ mkdir ~/venv
[user@login ~]$ virtualenv ~/venv/example-env
```

--------------------------------

### Install R Kernel for Jupyter (R)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/jupyter

These R commands are executed within an R session to install the R kernel for Jupyter. This allows R code to be run within Jupyter notebooks. The `IRkernel::installspec` function registers the kernel with Jupyter, allowing it to be selected from the launcher. It requires the IRkernel package to be installed.

```r
IRkernel::installspec(user = TRUE)
IRkernel::installspec(name = 'ir413', displayname = 'R 4.1.3 (ir413)')
```

--------------------------------

### Sample Script for Hybrid MPI + OpenMP Application

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-aps

A sample PBS job script for running a hybrid MPI + OpenMP application with APS. It includes module loading, environment variable settings, and the mpirun command to launch APS analysis.

```bash
#PBS -l walltime=00:05:00
#PBS -l select=2:ncpus=64:mem=200gb:mpiprocs=64:ompthreads=2
#PBS -N pinning_t2_c2_intelmpi
#PBS -o out.txt
#PBS -e error.txt

# Load the modules
module load imkl-FFTW/2023.1.0-iimpi-2023a
module load VTune/2022.3.0

# some pinning option
export I_MPI_DEBUG=5
export OMP_NUM_THREADS=2
export KMP_AFFINITY=verbose

cd $PBS_O_WORKDIR

#your app specific
lammps_path="/gpfs/home/lragta/RCS_help/1_janet/lkr/src/lmp_intel_cpu_intelmpi"

temp1=900
ex=0.0
ey=0.0
ez=0.0

# This will generata the aps result folder.
mpirun -v6 -np 64 aps --stat-level=5  ${lammps_path} -l log.lammps -nocite -var temp1 ${temp1} -var ex ${ex} -var ey ${ey} -var ez ${ez} -in alkyl-amor-nvt.in
```

--------------------------------

### Visualize Intel Advisor Results via GUI

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-advisor

Commands to connect to a remote cluster with X11 forwarding enabled and launch the Intel Advisor graphical user interface to inspect generated performance reports.

```bash
ssh -X username@hostname
advisor-gui
```

--------------------------------

### Configure Custom Python Environment for JupyterHub

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/jupyter

Commands to initialize Conda, create a new environment, install necessary packages, and register the environment as a kernel accessible within the JupyterHub interface.

```bash
eval "$(~/miniforge3/bin/conda shell.bash hook)"
conda create -n test1 python=3.9 ipykernel jupyter_client
source activate test1
conda search "package_name"
conda install package_name[=version]
python -m ipykernel install --user --name python39_test1 --display-name "Python3.9 (test1)"
```

--------------------------------

### Compiling with GCCcore Module (Illustrating Errors)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/GCC

Illustrates the process of loading a 'GCCcore' module and attempting to compile a program, which is expected to fail due to missing dependencies. This highlights the limitations of using 'GCCcore' in isolation.

```bash
# First unload the current modules (if any)
[user@login ~]$ module purge

# Load the module
[user@login ~]$ module load GCCcore/12.3.0

# Compile the program
[user@login ~]$ g++ -o HelloWorld HelloWorld.cpp
```

--------------------------------

### Run LAMMPS Simulation with Specific Parameters (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/thread-pinning

This snippet demonstrates how to execute a LAMMPS simulation using `mpirun`. It specifies the LAMMPS executable path, number of MPI processes, log file, and simulation parameters like temperature and strain. Dependencies include an MPI implementation and the LAMMPS executable.

```bash
lammps_path="/gpfs/home/lragta/RCS_help/1_janet/lkr/src/lmp_intel_cpu_intelmpi"
temp1=900
ex=0.0
ey=0.0
ez=0.0

mpirun -v6 -np 16 ${lammps_path} -l log.lammps -nocite -var temp1 ${temp1} -var ex ${ex} -var ey ${ey} -var ez ${ez} -in alkyl-amor-nvt.in
```

--------------------------------

### R: Get available cores for parallel processing

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/R

R code snippet to correctly detect the number of available CPU cores for parallel processing using the 'future' package. This is crucial for efficient parallelization and avoids issues with `parallel::detectCores()` reporting incorrect values.

```r
library("future")
numOfCores = future::availableCores()
```

--------------------------------

### Load Default Module and List - Bash

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications

Loads the default version of a module (e.g., GROMACS) into the environment and then lists all currently loaded modules. The `module load` command is case-sensitive. Dependencies are automatically loaded.

```bash
[user@login ~]$ module load GROMACS
[user@login ~]$ module list

Currently Loaded Modules:
  1) tools/prod                        22) FFTW.MPI/3.3.10-gompi-2023b
  2) GCCcore/13.2.0                    23) ScaLAPACK/2.2.0-gompi-2023b-fb
  3) zlib/1.2.13-GCCcore-13.2.0        24) foss/2023b
  4) binutils/2.40-GCCcore-13.2.0      25) bzip2/1.0.8-GCCcore-13.2.0
  5) GCC/13.2.0                        26) ncurses/6.4-GCCcore-13.2.0
  6) numactl/2.0.16-GCCcore-13.2.0     27) libreadline/8.2-GCCcore-13.2.0
  7) XZ/5.4.4-GCCcore-13.2.0           28) Tcl/8.6.13-GCCcore-13.2.0
  8) libxml2/2.11.5-GCCcore-13.2.0     29) SQLite/3.43.1-GCCcore-13.2.0
  9) libpciaccess/0.17-GCCcore-13.2.0  30) libffi/3.4.4-GCCcore-13.2.0
 10) hwloc/2.9.2-GCCcore-13.2.0        31) Python/3.11.5-GCCcore-13.2.0
 11) OpenSSL/1.1                       32) gfbf/2023b
 12) libevent/2.1.12-GCCcore-13.2.0    33) cffi/1.15.1-GCCcore-13.2.0
 13) UCX/1.15.0-GCCcore-13.2.0         34) cryptography/41.0.5-GCCcore-13.2.0
 14) libfabric/1.19.0-GCCcore-13.2.0   35) virtualenv/20.24.6-GCCcore-13.2.0
 15) PMIx/4.2.6-GCCcore-13.2.0         36) Python-bundle-PyPI/2023.10-GCCcore-13.2.0
 16) UCC/1.2.0-GCCcore-13.2.0          37) pybind11/2.11.1-GCCcore-13.2.0
 17) OpenMPI/4.1.6-GCC-13.2.0          38) SciPy-bundle/2023.11-gfbf-2023b
 18) OpenBLAS/0.3.24-GCC-13.2.0        39) networkx/3.2.1-gfbf-2023b
 19) FlexiBLAS/3.3.1-GCC-13.2.0        40) mpi4py/3.1.5-gompi-2023b
 20) FFTW/3.3.10-GCC-13.2.0            41) GROMACS/2024.1-foss-2023b
 21) gompi/2023b
```

--------------------------------

### Setup and Create Conda Environment with Python and NumPy (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/python

Initializes Conda for use in the current shell session and then creates a new Conda environment named 'py39' with Python 3.9 and the 'numpy' package. This is a recommended method for managing complex Python dependencies.

```bash
[user@login ~]$ eval "$(~/miniforge3/bin/conda shell.bash hook)"
[user@login ~]$ conda create -n py39 python=3.9 numpy
[user@login ~]$ conda activate py39
```

--------------------------------

### Set up R environment with Conda

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/R

Instructions to create and activate a Conda environment for R. This involves setting up Conda hooks, creating an environment with a specific R version from conda-forge, and activating it. It also includes commands for searching and installing R packages from the 'r' or 'conda-forge' channels.

```bash
[user@login ~]$ eval "$(~/miniforge3/bin/conda shell.bash hook))"
[user@login ~]$ conda create -n r413 r-base=4.1.3 -c conda-forge
[user@login ~]$ source activate r413
```

```bash
[user@login ~]$ conda search -c r "r-*"
[user@login ~]$ conda install -c r r-png
```

```bash
[user@login ~]$ conda deactivate
```

--------------------------------

### Wrap Training Function with PyTorch Profiler

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/pytorch_profile

Demonstrates how to wrap a PyTorch training function (e.g., `trainer.fit`) with the profiler to capture CPU and CUDA activities, memory usage, and tensor shapes. Options can be adjusted for performance and detail.

```python
with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], with_stack=True, profile_memory=True, record_shapes=True) as prof:
    trainer.fit(model, dataloader_train_MICLe)
```

--------------------------------

### Reinstall Jupyter Kernel for Existing Conda Environment (Bash)

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/applications/guides/jupyter

This bash snippet demonstrates how to make a previously created Conda environment visible as a Jupyter kernel on a new JupyterHub service. It involves activating the desired Conda environment and then using `ipykernel install` to register it as a user-level kernel for Jupyter. This is useful when Conda environments do not automatically appear after a service migration.

```bash
[user@login ~]$ eval "$(~/miniforge3/bin/conda shell.bash hook)"
[user@login ~]$ conda activate python39_jupyter
[user@login ~]$ python -m ipykernel install --user --name python39_torch_env --display-name "Python3.9 (torch)"
```

--------------------------------

### Discard Job Logs with qsub

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/faq/controlling-job-log-files

This snippet illustrates how to discard job log files (stdout and stderr) after the job has finished, similar to redirecting output to `/dev/null`. It utilizes a combination of `-koed`, `-o /dev/null`, and `-e /dev/null` options with `qsub`. This is useful for jobs where log retention is not necessary, saving disk space. Examples are provided for both command-line execution and PBS directives.

```bash
[user@login ~]$ qsub -koed -o /dev/null -e /dev/null
```

```bash
#PBS -koed -o /dev/null -e /dev/null
```

--------------------------------

### Specify Log File Locations with qsub

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/faq/controlling-job-log-files

This section shows how to redirect the standard output and standard error log files to specific directories using the `-o` and `-e` options with `qsub`. The specified directories must exist before the job runs. This is useful for organizing logs or directing them to specific storage locations. The examples show both command-line usage and inclusion as PBS directives in the submission script.

```bash
[user@login ~]$ qsub -o /rds/general/user/home/project1/logs_project1/ -e /rds/general/user/home/project1/logs_project1/ myjob.pbs
```

```bash
#PBS -o /rds/general/user/home/project1/logs_project1/
#PBS -e /rds/general/user/home/project1/logs_project1/
```

--------------------------------

### Generate Roofline Analysis Data for MPI Applications

Source: https://icl-rcs-user-guide.readthedocs.io/en/latest/hpc/advanced/profiling/intel-advisor

A PBS job script template that loads necessary modules and executes two-step Intel Advisor data collection (survey and tripcounts) for an MPI-based application. It requires specifying a project directory and MPI rank range for output.

```bash
#PBS -l walltime=00:20:00
#PBS -l select=2:ncpus=64:mem=200gb:mpiprocs=64:ompthreads=2
#PBS -N pinning_t2_c2_intelmpi
#PBS -o out.txt
#PBS -e error.txt

module load imkl-FFTW/2023.1.0-iimpi-2023a
module load VTune/2022.3.0
module load Advisor/2023.2.0

export I_MPI_DEBUG=5
export OMP_NUM_THREADS=2
export KMP_AFFINITY=verbose

cd $PBS_O_WORKDIR

lammps_path="/gpfs/home/lragta/RCS_help/1_janet/lkr/src/lmp_intel_cpu_intelmpi"
temp1=900
ex=0.0
ey=0.0
ez=0.0

# Collect the survey
mpirun -gtool "advisor --collect=survey --project-dir=./my_results:0-63" -v6 -np 64 ${lammps_path} -l log.lammps -nocite -var temp1 ${temp1} -var ex ${ex} -var ey ${ey} -var ez ${ez} -in alkyl-amor-nvt.in

# Run trip counts
mpirun -gtool "advisor --collect=tripcounts --flop --project-dir=./my_results:0-63" -v6 -np 64 ${lammps_path} -l log.lammps -nocite -var temp1 ${temp1} -var ex ${ex} -var ey ${ey} -var ez ${ez} -in alkyl-amor-nvt.in
```
Tokens:
Raw