XMU Chenggroup Wiki

https://wiki.cheng-group.net/
Admin
The XMU Chenggroup Wiki is an internal knowledge base for Professor Cheng Jun's research group at...
Tokens:93,164
Snippets:678
Trust Score:6
Update:1 year ago
Show doc for...
Tokens:
Raw
### Prepare Source Code for Installation (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/install_from_src_in_conda

Navigate into the extracted source code directory and run autogen.sh to prepare the build system. Note that this step might vary depending on the specific software package; always consult its installation guide.

```shell
# enter the source code folder
cd valgrind-3.19.0
# NOTE: This is not a general procedure
# Please check the installation guide for your package
./autogen.sh

```

--------------------------------

### Prepare Valgrind Source for Configuration

Source: https://wiki.cheng-group.net/wiki/software_installation/install_from_src_in_conda

This code navigates into the extracted Valgrind source directory and runs `./autogen.sh` to prepare the build system. Note that `./autogen.sh` is specific to Valgrind and similar projects; other projects might have different preparation steps. Always refer to the specific software's installation guide.

```shell
# enter the source code folder
cd valgrind-3.19.0
# NOTE: This is not a general procedure
# Please check the installation guide for your package
./autogen.sh
```

--------------------------------

### Configure Installation Path (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/install_from_src_in_conda

Configure the build process to install the software into a specific virtual environment directory using the `--prefix` option. Replace the example path with your desired virtual environment location. This step is crucial for isolating the installation.

```shell
# configure with given installation path
./configure --prefix=/new_data/jxzhu/envs/test_env/

```

--------------------------------

### Configure and Source CP2K Environment

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Copies architecture files to the CP2K installation directory and sources the setup script to make the CP2K environment available. This is a crucial step after toolchain installation.

```shell
cp install/arch/local* /share/apps/cp2k/7.1/arch/
source /share/apps/cp2k/7.1/tools/toolchain/install/setup

```

--------------------------------

### Compile and Install Software (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/install_from_src_in_conda

Compile the source code using `make -j` for parallel compilation, speeding up the process. Then, install the compiled software into the specified prefix directory using `make install`. Ensure you have sufficient permissions for the target directory.

```shell
# make in parallel
make -j20
# install software
make install

```

--------------------------------

### Install and Configure wsl-vpnkit for Network Synchronization

Source: https://wiki.cheng-group.net/wiki/cluster_usage/wsl_usage

This snippet outlines the steps to install and configure `wsl-vpnkit` to synchronize Windows network environments with WSL. It includes installing dependencies, downloading and extracting the tool, editing the service file, placing files in appropriate directories, and enabling/starting the systemd service.

```bash
# 安装依赖
$ sudo apt-get install iproute2 iptables iputils-ping dnsutils wget

# 下载wsl-vpnkit并解压其中关键组件
$ VERSION=v0.4.x    # 目前最新版本为v0.4.1
$ wget https://github.com/sakai135/wsl-vpnkit/releases/download/$VERSION/wsl-vpnkit.tar.gz    # 可能需要科学上网下载再传到wsl中
$ tar --strip-components=1 -xf wsl-vpnkit.tar.gz \
    app/wsl-vpnkit \
    app/wsl-gvproxy.exe \
    app/wsl-vm \
    app/wsl-vpnkit.service
$ rm wsl-vpnkit.tar.gz
```

```bash
$ vi wsl-vpnkit.service

[Unit]
Description=wsl-vpnkit
After=network.target

[Service]
# for wsl-vpnkit setup as a distro
#ExecStart=/mnt/c/Windows/system32/wsl.exe -d wsl-vpnkit --cd /app wsl-vpnkit

# for wsl-vpnkit setup as a standalone script
ExecStart=/usr/local/bin/wsl-vpnkit/wsl-vpnkit
Environment=VMEXEC_PATH=/usr/local/bin/wsl-vpnkit/wsl-vm GVPROXY_PATH=/usr/local/bin/wsl-vpnkit/wsl-gvproxy.exe

Restart=always
KillMode=mixed

[Install]
WantedBy=multi-user.target
```

```bash
$ sudo mkdir /usr/local/bin/wsl-vpnkit
$ sudo mv wsl-vpnkit wsl-gvproxy.exe wsl-vm /usr/local/bin/wsl-vpnkit/
$ sudo mv wsl-vpnkit.service /etc/systemd/system/
```

```bash
$ sudo systemctl enable wsl-vpnkit    # 设置开机启动
$ sudo systemctl start wsl-vpnkit     # 开启服务
$ systemctl status wsl-vpnkit         # 查看运行状态
```

--------------------------------

### Compile and Install DeePMD-kit C++ Interface

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Compiles and installs the DeePMD-kit C++ interface. Verification commands are provided to check the installation of executables and libraries.

```bash
make
make install
```

```bash
$ ls $deepmd_root/bin
dp_ipi
```

```bash
$ ls $deepmd_root/lib
libdeepmd_ipi.so  libdeepmd_op.so  libdeepmd.so
```

--------------------------------

### Compile and Install Valgrind

Source: https://wiki.cheng-group.net/wiki/software_installation/install_from_src_in_conda

This snippet compiles the Valgrind source code using `make -j20` for parallel compilation (adjust the number of jobs based on your system's cores) and then installs it to the directory specified by the `--prefix` option during the configure step using `make install`.

```shell
# make in parallel
make -j20
# install software
make install
```

--------------------------------

### Install TensorFlow Python Interface

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Installs the GPU-enabled version of TensorFlow. Use the --upgrade option to overwrite if already installed, or --user for current account installation if permissions are insufficient.

```bash
pip install tensorflow-gpu==1.14.0
```

--------------------------------

### Configure Valgrind Installation Path

Source: https://wiki.cheng-group.net/wiki/software_installation/install_from_src_in_conda

This command configures the Valgrind build process, specifying the installation directory using the `--prefix` option. This is the crucial step for installing into a virtual environment. Replace `/new_data/jxzhu/envs/test_env/` with your desired virtual environment path.

```shell
# configure with given installation path
./configure --prefix=/new_data/jxzhu/envs/test_env/
```

--------------------------------

### Verify Valgrind Installation in Virtual Environment

Source: https://wiki.cheng-group.net/wiki/software_installation/install_from_src_in_conda

This sequence demonstrates how to verify the Valgrind installation. It first checks if `valgrind` is found in the default PATH (it shouldn't be if installed correctly in the virtual environment). Then, it activates the virtual environment using `conda activate` and checks `which valgrind` again, which should now point to the binary within the specified virtual environment.

```shell
(base) [jxzhu@login01:] /data/jxzhu/software/valgrind-3.19.0 $ which valgrind
/usr/bin/which: no valgrind in (...)
(base) [jxzhu@login01:] /data/jxzhu/software/valgrind-3.19.0 $ conda activate /new_data/jxzhu/envs/test_env/
(test_env) [jxzhu@login01:] /data/jxzhu/software/valgrind-3.19.0 $ which valgrind
/new_data/jxzhu/envs/test_env/bin/valgrind
```

--------------------------------

### Force Install Package with Pip

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

Installs a package, ignoring any previously installed versions. This can be useful when encountering issues with existing installations or when needing to ensure a fresh installation.

```bash
pip install some_package --ignore-installed

```

--------------------------------

### Install DeepMD-kit using pip

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

This command installs the DeepMD-kit package from the source code located in the current directory. It's typically run after cloning the repository and setting up necessary environment variables and compilers.

```bash
cd deepmd-kit
pip install .
```

--------------------------------

### Verify Installation in Virtual Environment (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/install_from_src_in_conda

This snippet demonstrates how to verify the installation. First, it shows that 'valgrind' is not found in the default PATH. Then, it activates the virtual environment and confirms that 'valgrind' is now accessible via its path within the environment.

```shell
(base) [jxzhu@login01:] /data/jxzhu/software/valgrind-3.19.0 $ which valgrind
/usr/bin/which: no valgrind in (...)
(base) [jxzhu@login01:] /data/jxzhu/software/valgrind-3.19.0 $ conda activate /new_data/jxzhu/envs/test_env/
(test_env) [jxzhu@login01:] /data/jxzhu/software/valgrind-3.19.0 $ which valgrind
/new_data/jxzhu/envs/test_env/bin/valgrind

```

--------------------------------

### Download and Decompress Valgrind Source Code

Source: https://wiki.cheng-group.net/wiki/software_installation/install_from_src_in_conda

This snippet shows how to download the Valgrind source code using `wget` and then decompress the downloaded archive using `tar`. Ensure you have `wget` installed. The URLs and filenames may change with new versions.

```shell
# download source code from official website
wget -c https://sourceware.org/pub/valgrind/valgrind-3.19.0.tar.bz2
# decompress
tar -jxvf valgrind-3.19.0.tar.bz2
```

--------------------------------

### Download and Decompress Source Code (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/install_from_src_in_conda

This snippet downloads the Valgrind source code using wget and then decompresses the downloaded tarball. Ensure wget is installed and the URL is valid. The output will be the decompressed source directory.

```shell
# download source code from official website
wget -c https://sourceware.org/pub/valgrind/valgrind-3.19.0.tar.bz2
# decompress
tar -jxvf valgrind-3.19.0.tar.bz2

```

--------------------------------

### Compile and Install DeePMD-kit C++ Interface

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Compiles the DeePMD-kit C++ interface using 'make' and then installs it to the specified location. This step makes the C++ library available for use in other applications.

```bash
make
make install
```

--------------------------------

### Install DP-CP2K

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

Instructions for installing the DP-CP2K module. This involves cloning the repository, running a toolchain script to set up the environment and dependencies (including DeePMD-kit and TensorFlow C++ interfaces), and then compiling the CP2K executable. Note that this version of DP-CP2K does not support MPI.

```shell
git clone https://github.com/Cloudac7/cp2k.git -b deepmd_latest --recursive --depth=1

cd tools/toolchain/
./install_cp2k_toolchain.sh --enable-cuda=no --with-deepmd=$deepmd_root --with-tfcc=$tensorflow_root --deepmd-mode=cuda --mpi-mode=no --with-libint=no --with-libxc=no --with-libxsmm=no

make -j 4 ARCH=local VERSION="ssmp sdbg"
```

--------------------------------

### Download and Install Bazel (Shell)

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

This code downloads a specific version of the Bazel build tool, makes it executable, and installs it for the current user. Bazel is required for compiling TensorFlow from source. It also adds the user's bin directory to the PATH.

```shell
cd /some/workspace
wget https://github.com/bazelbuild/bazel/releases/download/0.24.0/bazel-0.24.0-installer-linux-x86_64.sh
chmod +x bazel-0.24.0-installer-linux-x86_64.sh
./bazel-0.24.0-installer-linux-x86_64.sh --user
export PATH="$HOME/bin:$PATH"
```

--------------------------------

### Prepare for DeePMD-kit C++ Interface Compilation

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Sets up the build environment for the DeePMD-kit C++ interface. This involves navigating to the source directory, creating a build directory, and setting environment variables for the installation path and compiler.

```bash
deepmd_source_dir=`pwd`
cd $deepmd_source_dir/source
mkdir build 
cd build
export deepmd_root=/some/workspace/deepmd_root
export CC=`which gcc`
export CXX=`which g++`
```

--------------------------------

### Verify DeePMD-kit C++ Interface Installation

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Checks if the DeePMD-kit C++ interface was installed correctly by listing the contents of the 'bin' and 'lib' directories in the installation path. The presence of 'dp_ipi' and shared libraries indicates a successful installation.

```bash
$ ls $deepmd_root/bin
$ ls $deepmd_root/lib
```

--------------------------------

### Install DeePMD-kit Python Interface

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

This command compiles and installs the DeePMD-kit Python interface from the source code. Ensure the DP_VARIANT environment variable is set correctly (e.g., 'cuda' for GPU support) before running this command.

```bash
# Ensure DP_VARIANT is set, e.g., export DP_VARIANT=cuda
cd $deepmd_source_dir
pip install .
```

--------------------------------

### Install DeePMD-kit Python Dependencies and Package

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_ikkem

This section outlines the installation of essential Python packages for DeePMD-kit, including TensorFlow, scikit-build, and protobuf. It then navigates to the DeePMD source directory, sets the DP_VARIANT to 'cuda', and installs the DeePMD package using pip.

```bash
pip install tensorflow==2.7 --upgrade
pip install scikit-build ninja
pip install protobuf==3.20
cd $deepmd_source_dir
export DP_VARIANT=cuda
pip install .
```

--------------------------------

### README File Example (Markdown)

Source: https://wiki.cheng-group.net/wiki/teamwork/archive_rules

A README file should be included in directories to provide essential information about the contents, such as version details and conversion notes. Markdown is recommended for readability.

```markdown
# README
 converted from v1.2 pot
 compress input use that v1.2 training input

```

--------------------------------

### DP-CP2K Installation

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

Installs the DP-CP2K package, which integrates DeePMD with the CP2K molecular dynamics code. This involves cloning the repository, running a toolchain script for configuration, and compiling the CP2K executable.

```bash
git clone https://github.com/Cloudac7/cp2k.git -b deepmd_latest --recursive --depth=1
```

```bash
cd tools/toolchain/
./install_cp2k_toolchain.sh --enable-cuda=no --with-deepmd=$deepmd_root --with-tfcc=$tensorflow_root --deepmd-mode=cuda --mpi-mode=no --with-libint=no --with-libxc=no --with-libxsmm=no
```

```bash
make -j 4 ARCH=local VERSION="ssmp sdbg"
```

--------------------------------

### Upgrade Setuptools for Installation

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

Upgrades the 'setuptools' package without isolation. This can resolve installation errors related to package building, especially when dealing with older projects or specific build requirements.

```bash
pip install --upgrade setuptools --no-build-isolation

```

--------------------------------

### Upgrade pip and install TensorFlow Python interface

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

These commands first upgrade the pip package installer and then install the TensorFlow Python package matching a specific version (2.3.0). Use `--upgrade` to overwrite existing installations or `--user` for permission issues. Ensure the TensorFlow version matches the C++ interface.

```bash
pip install --upgrade pip
pip install tensorflow==2.3.0
```

--------------------------------

### Installing JupyterLab with Conda

Source: https://wiki.cheng-group.net/wiki/cluster_usage/open_ondemand

These commands demonstrate how to install JupyterLab within a Conda environment on the HPC cluster. It includes loading Miniconda, activating an environment, installing JupyterLab via pip or creating a new environment with conda.

```bash
module load miniconda/3
source activate <env> && pip install jupyterlab
```

```bash
conda create -n <env> python=3.9 jupyterlab
source activate <env>
```

```bash
pip install nbconvert==6.4.3
```

--------------------------------

### Load necessary modules for DeepMD-kit installation

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

This snippet demonstrates how to load essential modules like CUDA and GCC using the 'module' command, which is common in cluster environments for managing software versions. Ensure the specified versions are available on your system.

```bash
module avail
module add cuda/10.1
module add gcc/7.4.0
```

--------------------------------

### Download and Install Bazel for TensorFlow

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Downloads a specific version of the Bazel build tool (0.24.0) and makes it executable. It then installs Bazel for the current user and adds the user's bin directory to the PATH. Bazel version compatibility with TensorFlow is critical and should be checked against TensorFlow's official documentation.

```bash
cd /some/workspace
wget https://github.com/bazelbuild/bazel/releases/download/0.24.0/bazel-0.24.0-installer-linux-x86_64.sh
chmod +x bazel-0.24.0-installer-linux-x86_64.sh
./bazel-0.24.0-installer-linux-x86_64.sh --user
export PATH="$HOME/bin:$PATH"

```

--------------------------------

### Install LAMMPS DeePMD-kit Module (make)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

Builds the USER-DEEPMD LAMMPS extension package using the 'make' command. This process involves navigating to the build directory, compiling the LAMMPS module, and then copying the extension to the LAMMPS source directory.

```bash
cd $deepmd_source_dir/source/build
make lammps
```

```bash
cd /some/workspace
# Download Lammps latest release
wget -c https://lammps.sandia.gov/tars/lammps-stable.tar.gz
tar xf lammps-stable.tar.gz
cd lammps-*/src/
cp -r $deepmd_source_dir/source/build/USER-DEEPMD .
```

```bash
make yes-user-deepmd
make yes-kspace
```

```bash
make mpi -j4
```

--------------------------------

### Install DeePMD-kit Module for LAMMPS (CMake)

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

This section outlines the process of installing the DeePMD-kit module for LAMMPS using CMake. It involves configuring the build with specific CMake options, including enabling the USER-DEEPMD package and setting installation prefixes and compiler flags. This method is presented as a cleaner and faster alternative to the `make` approach.

```shell
conda install gsl

cd lammps-stable
mkdir build
cd build

cmake -C ../cmake/presets/most.cmake -C ../cmake/presets/nolib.cmake \
-D BUILD_MPI=yes -D BUILD_OMP=yes -D LAMMPS_MACHINE=mpi \
-D WITH_JPEG=no -D WITH_PNG=no -D WITH_FFMPEG=no \
-D PKG_PLUMED=yes -D PKG_COLVARS=yes -D PKG_USER-DEEPMD=ON \
-D CMAKE_INSTALL_PREFIX=/data/user/conda/env/deepmd \
-D CMAKE_CXX_FLAGS="-std=c++14 -DHIGH_PREC -DLAMMPS_VERSION_NUMBER=20220623 -I${deepmd_root}/include -I${tensorflow_root}/include -L${deepmd_root}/lib -L${tensorflow_root}/lib -Wl,--no-as-needed -ldeepmd_cc -ltensorflow_cc -ltensorflow_framework -Wl,-rpath=${deepmd_root}/lib -Wl,-rpath=${tensorflow_root}/lib"
../cmake

make -j 16
make install
```

--------------------------------

### List Conda Environments

Source: https://wiki.cheng-group.net/wiki/cluster_usage/conda

Lists all available conda environments on the system. This command is used to verify the setup and see existing environments.

```bash
conda env list
```

--------------------------------

### Load Modules and Add Paths (Shell)

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

This snippet demonstrates how to load necessary modules (CUDA, GCC) and set environment variables for library paths. It's crucial for ensuring that the system can find required libraries, especially in environments where direct internet access is restricted.

```shell
module avail
module add cuda/10.0
module add gcc/4.9.4

ln -s /share/cuda/10.0/lib64/stubs/libcuda.so /some/local/path/libcuda.so.1
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/cuda/10.0/lib64/stubs:/some/local/path
```

--------------------------------

### Configure, Compile, and Install GCC (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/gcc

Creates a build directory, configures the GCC build with specified languages and installation prefix, compiles the source code, and installs the compiled binaries. Requires standard build tools and a C/C++ compiler.

```shell
mkdir objdir
cd objdir
../gcc-5.5.0/configure --prefix=/share/apps/gcc/5.5.0 --enable-languages=c,c++,fortran,go --disable-multilib
make
make install
```

--------------------------------

### Install/Upgrade TensorFlow

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

This command installs or upgrades TensorFlow to a specific version (2.5.0 in this example) within the active Python environment. It's important to match the TensorFlow version with what DeePMD-kit expects or what was used in a conda-installed environment.

```bash
pip install --upgrade pip
pip install --upgrade tensorflow==2.5.0
```

--------------------------------

### Upgrade Setuptools for Build Isolation

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

This command upgrades the 'setuptools' package and enables the '--no-build-isolation' flag, which can help resolve build errors related to dependency management during package installation.

```bash
pip install --upgrade setuptools --no-build-isolation
```

--------------------------------

### Install Third-Party TensorFlow Interface

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_ikkem

This optional snippet details the steps to install the TensorFlow interface for DeePMD. It involves creating directories, creating symbolic links to TensorFlow libraries and headers within the custom paths, and then configuring and building DeePMD with CMake, enabling TensorFlow and CUDA support.

```bash
mkdir -p $tensorflow_root/lib 
cd $tensorflow_root
ln -s $CONDA_PREFIX/lib/python3.9/site-packages/tensorflow/include .
cd lib
ln -s $CONDA_PREFIX/lib/python3.9/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so libtensorflow_cc.so
ln -s $CONDA_PREFIX/lib/python3.9/site-packages/tensorflow/libtensorflow_framework.so.2 .
ln -s libtensorflow_framework.so.2 libtensorflow_framework.so

mkdir -p $deepmd_source_dir/source/build
mkdir -p $deepmd_root
cd $deepmd_source_dir/source/build
cmake -DLAMMPS_SOURCE_ROOT=/public/software/lammps/lammps-2022.6.23-intel -DUSE_TF_PYTHON_LIBS=TRUE -DUSE_CUDA_TOOLKIT=TRUE -DTENSORFLOW_ROOT=$tensorflow_root -DCMAKE_INSTALL_PREFIX=$deepmd_root ..
make -j20
make install
```

--------------------------------

### Clone DeePMD-kit Source Code

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

Downloads the DeePMD-kit source code repository, including all submodules, using Git. The `--recursive` flag is crucial for fetching necessary dependencies.

```bash
#(tf-dp)
git clone --recursive https://github.com/deepmodeling/DeePMD-kit.git DeePMD-kit

```

--------------------------------

### Configure, Compile, and Install GCC

Source: https://wiki.cheng-group.net/wiki/software_installation/gcc

This snippet details the process of configuring, compiling, and installing GCC. It specifies the installation prefix, enabled languages (C, C++, Fortran, Go), and disables multilib compilation. This is the core build process.

```shell
mkdir objdir
cd objdir
../gcc-5.5.0/configure --prefix=/share/apps/gcc/5.5.0 --enable-languages=c,c++,fortran,go --disable-multilib
make
make install

```

--------------------------------

### Clone DeePMD-kit Source Code

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Downloads the DeePMD-kit source code recursively to ensure all submodules are included. This is crucial for a successful compilation.

```bash
cd /some/workspace
git clone --recursive https://github.com/deepmodeling/deepmd-kit.git deepmd-kit
```

--------------------------------

### Link CUDA Libraries and Set Environment Path

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Manually links CUDA runtime libraries (libcuda.so and libcuda.so.1) to a local path and updates the LD_LIBRARY_PATH environment variable. This is necessary for nodes that cannot access the network. The example shows linking to `/some/local/path` and assumes CUDA 10.0 is installed.

```bash
ln -s /share/cuda/10.0/lib64/stubs/libcuda.so /some/local/path/libcuda.so.1
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/cuda/10.0/lib64/stubs:/some/local/path

```

--------------------------------

### Extract and Download GCC Prerequisites (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/gcc

Extracts the downloaded GCC source archive and then downloads any necessary build prerequisites for GCC. Requires tar and the download_prerequisites script.

```shell
tar -zxvf gcc-5.5.0.tar.gz
cd gcc-5.5.0
./contrib/download_prerequisites
cd ..
```

--------------------------------

### Recompile DeePMD-kit C++ Code

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_ikkem

This snippet demonstrates how to recompile the C++ components of DeePMD-kit after source code changes. It involves navigating to the build directory, using `make` to compile with multiple jobs for faster execution, and then `make install` to install the updated binaries and libraries.

```bash
cd $deepmd_source_dir/source/build
make -j20
make install
```

--------------------------------

### Manage WSL Distributions via Command Line

Source: https://wiki.cheng-group.net/wiki/cluster_usage/wsl_usage

Commands to shut down a specific WSL distribution or start one. This is useful for restarting a distribution without rebooting the entire system. Ensure you know the correct distribution name using `wsl -l -v`.

```bash
wsl --shutdown <distro_name>
distro_name    # 启动指定发行版
```

--------------------------------

### Download GCC Source Code (Shell)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/gcc

Downloads the GCC 5.5.0 source code tarball using wget. This is the initial step for manual installation. Ensure wget is installed.

```shell
wget https://mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-5.5.0/gcc-5.5.0.tar.gz
```

--------------------------------

### Install DeepMD-kit

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_new

Installs the DeepMD-kit package using pip after navigating to its source directory. This command compiles and installs the Python interface and associated C++ libraries.

```bash
cd deepmd-kit
pip install .

```

--------------------------------

### Force Install Package

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

If a package is already installed but causing issues, or if uninstalling fails, this command can be used to force pip to reinstall the package, potentially overwriting existing installations.

```bash
pip install some_package --ignore-installed
```

--------------------------------

### Compile DeePMD-kit C++ Interface

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

Steps to compile the DeePMD-kit C++ interface. This involves navigating to the source directory, creating a build directory, and running CMake with specified installation paths and dependencies. Ensure that the `LAMMPS_VERSION_NUMBER` and TensorFlow root directory are correctly set.

```shell
deepmd_source_dir=`pwd`
cd $deepmd_source_dir/source
mkdir build 
cd build

export deepmd_root=/some/workspace/deepmd_root

cmake -DLAMMPS_VERSION_NUMBER=<value> -DTENSORFLOW_ROOT=$tensorflow_root -DCMAKE_INSTALL_PREFIX=$deepmd_root ..

make
make install
```

--------------------------------

### Install/Upgrade TensorFlow

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

Upgrades pip to the latest version and installs TensorFlow version 2.5.0. This step is part of the manual installation process for the TF/Python interface.

```bash
#(tf-dp)
pip install --upgrade pip
pip install --upgrade tensorflow==2.5.0

```

--------------------------------

### Extract and Download GCC Prerequisites

Source: https://wiki.cheng-group.net/wiki/software_installation/gcc

This section shows how to extract the downloaded GCC tarball and then use the provided script to download necessary build prerequisites. This is a crucial step before compilation.

```shell
tar -zxvf gcc-5.5.0.tar.gz
cd gcc-5.5.0
./contrib/download_prerequisites
cd ..

```

--------------------------------

### Install Binaries and Clean Up Build Artifacts

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Creates a 'bin' directory, copies the compiled executables from the 'exe/local' directory into it, and then removes intermediate build files and directories to reduce disk space usage.

```shell
mkdir bin
cp ./exe/local/* ./bin
rm -rf tools/toolchain/build tools/toolchain/install

```

--------------------------------

### Configure TensorFlow C++ Build

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

This snippet shows the interactive configuration process for building the TensorFlow C++ interface. It prompts the user to specify paths for Python, CUDA, and other build options. The output details the choices made, such as enabling CUDA support and disabling others like OpenCL SYCL.

```bash
./configure
Please specify the location of python. [Default is xxx]:

Found possible Python library paths:
  /xxx/xxx/xxx
Please input the desired Python library path to use.  Default is [xxx]

Do you wish to build TensorFlow with XLA JIT support? [Y/n]:
XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Do you wish to build TensorFlow with TensorRT support? [y/N]:
No TensorRT support will be enabled for TensorFlow.

Found CUDA 10.0 in:
    /share/cuda/10.0/lib64
    /share/cuda/10.0/include
Found cuDNN 7 in:
    /share/cuda/10.0/lib64
    /share/cuda/10.0/include

Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 3.5,7.0]:

Do you want to use clang as CUDA compiler? [y/N]:
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /share/apps/gcc/4.9.4/bin/gcc]:

Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>":
    --config=mkl             # Build with MKL support.
    --config=monolithic      # Config for mostly static monolithic build.
    --config=gdr             # Build with GDR support.
    --config=verbs           # Build with libverbs support.
    --config=ngraph          # Build with Intel nGraph support.
    --config=numa            # Build with NUMA support.
    --config=dynamic_kernels    # (Experimental) Build kernels into separate shared objects.
    --config=v2              # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
    --config=noaws           # Disable AWS S3 filesystem support.
    --config=nogcp           # Disable GCP support.
    --config=nohdfs          # Disable HDFS support.
    --config=noignite        # Disable Apache Ignite support.
    --config=nokafka         # Disable Apache Kafka support.
    --config=nonccl          # Disable NVIDIA NCCL support.
Configuration finished

```

--------------------------------

### Redirect GitHub Downloads to a Mirror

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Replaces 'github.com' with 'hub.fastgit.org' in install scripts to use a mirror for faster downloads in mainland China. This can help avoid timeouts when downloading dependencies.

```shell
sed -i 's/github.com/hub.fastgit.org/g' scripts/install_*.sh

```

--------------------------------

### Install DeePMD-kit from Modified Source Code

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

This command installs the DeePMD-kit package after you have modified its source code. It compiles the modified code into a new executable version.

```bash
pip install .

```

--------------------------------

### Create and Activate Conda Environment

Source: https://wiki.cheng-group.net/wiki/cluster_usage/conda

Creates a new conda environment with a specified name and installs Python. It then activates the newly created environment for use.

```bash
conda create -n <your env name> python
conda activate <your env name>
```

--------------------------------

### Set Bazel Temporary Directory

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

This command sets the `TEST_TMPDIR` environment variable to specify a temporary directory for Bazel's compilation artifacts. This is useful when the default compilation directory (`~/.cache/bazel`) lacks sufficient disk space. The example uses `/data/user/.bazel` as the temporary directory.

```bash
export TEST_TMPDIR=/data/user/.bazel

```

--------------------------------

### Prepare CP2K Installation Directory

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Copies the downloaded CP2K source tarball to the installation path and extracts it. It also renames the extracted directory for consistency.

```shell
cp cp2k-7.1.tar.bz2 /share/apps/cp2k
cd /share/apps/cp2k/
tar -jxf cp2k-7.1.tar.bz2
mv cp2k-7.1 7.1

```

--------------------------------

### Configure DeePMD-kit C++ Interface with CMake

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Configures the C++ interface build using CMake, specifying the TensorFlow installation root and the desired installation prefix for DeePMD-kit. If multiple CMake versions are installed, use `cmake3`.

```bash
cmake -DTENSORFLOW_ROOT=$tensorflow_root -DCMAKE_INSTALL_PREFIX=$deepmd_root ..
```

--------------------------------

### Compile DeePMD-kit C++ Interface

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

Steps to compile the C++ interface for DeePMD-kit. This involves navigating to the source directory, creating a build directory, and configuring the build with CMake.

```bash
deepmd_source_dir=`pwd`
cd $deepmd_source_dir/source
mkdir build 
cd build
```

```bash
export deepmd_root=/some/workspace/deepmd_root
```

```bash
cmake -DLAMMPS_VERSION_NUMBER=<value> -DTENSORFLOW_ROOT=$tensorflow_root -DCMAKE_INSTALL_PREFIX=$deepmd_root ..
```

```bash
make
make install
```

```bash
$ ls $deepmd_root/bin
dp_ipi
$ ls $deepmd_root/lib
libdeepmd_ipi.so  libdeepmd_op.so  libdeepmd.so
```

--------------------------------

### Conda Installation of DeePMD-kit

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

Installs DeePMD-kit and its dependencies (including GPU support and LAMMPS) into a new conda environment named 'deepmd'. This method is suitable when source code modification is not required.

```bash
#(base)
conda create -n deepmd deepmd-kit=*=*gpu libdeepmd=*=*gpu lammps-dp cudatoolkit=11.3 horovod -c https://conda.deepmodeling.org

```

--------------------------------

### Clone TensorFlow Source Code

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_51

Clones the TensorFlow source code repository from GitHub, specifically checking out version v1.14.0 using a shallow clone (--depth=1) to save bandwidth and time. The code is cloned into the `/some/workspace` directory.

```bash
cd /some/workspace 
git clone https://github.com/tensorflow/tensorflow tensorflow -b v1.14.0 --depth=1
cd tensorflow

```

--------------------------------

### 启动 Mkdocs 服务 (Bash)

Source: https://wiki.cheng-group.net/en/wiki/how_to_edit/howtodo

运行 Mkdocs 的 serve 命令，在本地启动一个 Web 服务器来实时预览 Wiki 内容。用户可以在浏览器中访问指定地址查看修改效果。

```bash
mkdocs serve
```

--------------------------------

### Clone DeepMD-kit repository recursively

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

This command clones the DeepMD-kit repository from GitHub, including all submodules (essential for correct compilation). Replace `v1.3.3` with the desired version tag. The `--recursive` flag is crucial; use `git submodule update --init --recursive` if missed.

```bash
cd /some/workspace
git clone --recursive https://github.com/deepmodeling/deepmd-kit.git deepmd-kit -b v1.3.3
```

--------------------------------

### Install DeePMD-kit Python Interface

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

Installs the DeePMD-kit Python interface from the source code. Ensure the DP_VARIANT environment variable is set correctly (e.g., 'cuda' for GPU support) before running this command.

```bash
#(tf-dp)
cd $deepmd_source_dir
DP_VARIANT=cuda
pip install .

```

--------------------------------

### Install CMake via Pip

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

This command installs the CMake build system using pip. This is a fallback option if CMake is not available on the system's package manager or if a specific version is required.

```shell
pip install cmake
```

--------------------------------

### Download CP2K 7.1 Source Code

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Downloads the CP2K version 7.1 source code tarball using wget. The -c option allows resuming interrupted downloads.

```shell
wget -c https://github.com/cp2k/cp2k/releases/download/v7.1.0/cp2k-7.1.tar.bz2

```

--------------------------------

### Download GCC Source Code using wget

Source: https://wiki.cheng-group.net/wiki/software_installation/gcc

This snippet demonstrates how to download the GCC 5.5.0 source code tarball using the wget command. It's useful for users who find direct downloads from the GNU website slow and prefer using mirrors like tuna.

```shell
wget https://mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-5.5.0/gcc-5.5.0.tar.gz

```

--------------------------------

### Install LAMMPS DeePMD-kit Module (CMake)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

Installs the USER-DEEPMD LAMMPS extension package using CMake. This method offers a cleaner and faster compilation process, allowing for customization of build options and packages.

```bash
conda install gsl
```

```bash
set(STANDARD_PACKAGES
  ...
  USER-DEEPMD)
```

```bash
cd lammps-stable
mkdir build
cd build
```

```bash
cmake -C ../cmake/presets/most.cmake -C ../cmake/presets/nolib.cmake \
-D BUILD_MPI=yes -D BUILD_OMP=yes -D LAMMPS_MACHINE=mpi \
-D WITH_JPEG=no -D WITH_PNG=no -D WITH_FFMPEG=no \
-D PKG_PLUMED=yes -D PKG_COLVARS=yes -D PKG_USER-DEEPMD=ON \
-D CMAKE_INSTALL_PREFIX=/data/user/conda/env/deepmd \
-D CMAKE_CXX_FLAGS="-std=c++14 -DHIGH_PREC -DLAMMPS_VERSION_NUMBER=20220623 -I${deepmd_root}/include -I${tensorflow_root}/include -L${deepmd_root}/lib -L${tensorflow_root}/lib -Wl,--no-as-needed -ldeepmd_cc -ltensorflow_cc -ltensorflow_framework -Wl,-rpath=${deepmd_root}/lib -Wl,-rpath=${tensorflow_root}/lib"
../cmake
```

```bash
make -j 16
make install
```

--------------------------------

### Load Environment Modules for CP2K Installation

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Loads the required Intel compiler, MPI, and GCC modules necessary for compiling CP2K. These commands assume a module system is in use.

```shell
module load intel/17.5.239 mpi/intel/2017.5.239
module load gcc/5.5.0

```

--------------------------------

### Convert MKL Static Libraries to Dynamic Libraries

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Applies several sed commands to replace static MKL library references (.a) with dynamic library references (.so) in the install_mkl.sh script. This is necessary to avoid 'undefined reference' errors when using ELPA.

```shell
sed -i 's/a libmkl_core.a libmkl_sequential.a/so libmkl_sequential.so libmkl_core.so/g' scripts/install_mkl.sh
sed -i 's/libmkl_gf_lp64.a/libmkl_gf_lp64.so/g' scripts/install_mkl.sh
sed -i 's/libmkl_core.a/libmkl_sequential.so/g' scripts/install_mkl.sh
sed -i 's/libmkl_scalapack_lp64.a/libmkl_scalapack_lp64.so/g' scripts/install_mkl.sh
sed -i 's/libmkl_blacs_intelmpi_lp64.a/libmkl_blacs_intelmpi_lp64.so/g' scripts/install_mkl.sh
sed -i 's/libmkl_blacs_openmpi_lp64.a/libmkl_blacs_openmpi_lp64.so/g' scripts/install_mkl.sh
sed -i 's/libmkl_core.a/libmkl_sequential.so/g' scripts/install_mkl.sh

```

--------------------------------

### 克隆 Wiki 仓库 (Git)

Source: https://wiki.cheng-group.net/en/wiki/how_to_edit/howtodo

使用 Git 命令将 Wiki 的 GitHub 仓库克隆到本地。这是进行本地开发、编辑和预览 Wiki 内容的第一步。

```bash
git clone https://github.com/chenggroup/chenggroup.github.io.git
cd chenggroup.github.io
```

--------------------------------

### Search and Install TensorFlow C++ Interface (GPU)

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_191

This snippet shows how to search for available versions of the TensorFlow C++ interface (libtensorflow_cc) in a specific Conda channel and then install a particular version (2.3.0 for GPU with CUDA 10.1).

```shell
conda search libtensorflow_cc -c https://conda.deepmodeling.com
conda install libtensorflow_cc=2.3.0=gpu_cuda10.1_0 -c https://conda.deepmodeling.org
```

--------------------------------

### Create Symbolic Link for Module Version

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Creates a symbolic link from '.module' to '7.1' within the modulefile directory. This makes the CP2K 7.1 module accessible via the version number.

```shell
ln -s .module 7.1

```

--------------------------------

### GCC Environment Modulefile (Tcl)

Source: https://wiki.cheng-group.net/en/wiki/software_installation/gcc

A Tcl script to define environment variables for GCC 5.5.0. It sets up PATH, LD_LIBRARY_PATH, INCLUDE, INFOPATH, and MANPATH to make GCC accessible. This is used by module loading systems.

```tcl
#%Module1.0#####################################################################
##
## GCC modulefile
##
proc ModulesHelp { } {
        global version

        puts stderr "\tSets up environment for GCC v$version"
}

module-whatis   "sets up environment for GCC v5.5.0"

# for Tcl script use only
set     version 5.5.0
set     root    /share/apps/gcc/$version

prepend-path    INFOPATH        $root/share/info
prepend-path    LD_LIBRARY_PATH $root/lib64:$root/lib:$root/libexec
prepend-path    INCLUDE         $root/include
prepend-path    MANPATH         $root/share/man
prepend-path    PATH            $root/bin

```

--------------------------------

### Install CP2K Toolchain Dependencies

Source: https://wiki.cheng-group.net/en/wiki/software_installation/cp2k-7

Executes the install_cp2k_toolchain.sh script with specific options to install necessary dependencies for CP2K. This command disables GPU support, CUDA, and includes system MPI.

```shell
./install_cp2k_toolchain.sh --gpu-ver=no --enable-cuda=no --with-mpich=system --with-sirius=no --with-openmpi=no --with-spfft=no --with-hdf5=no

```

--------------------------------

### Install DeePMD-kit from Modified Source (Python)

Source: https://wiki.cheng-group.net/wiki/software_installation/deepmd-kit/deepmd-kit_installation_pc

This command installs the DeePMD-kit package from the current directory, which should contain modified source code. It assumes you have already set up your environment and dependencies. This process compiles the Python interface and any necessary C++ components.

```bash
pip install .
```

--------------------------------

### CP2K Input File Structure Example

Source: https://wiki.cheng-group.net/en/wiki/software_usage/cp2k/cp2k-reftraj

Demonstrates the basic syntax of a CP2K input file, illustrating the hierarchical structure of SECTIONS, SUBSECTIONS, and KEYWORDS. This format is fundamental for configuring any CP2K calculation.

```text
&SECTION
  &SUSECTION
  ...
  &END SUBSECTION
  KEYWORD1 <value>
  KEYWORD2 <value>
  ...
&END SECTION

```
Tokens:
Raw