OpenSCOW

https://pkuhpc.github.io/OpenSCOW
Admin
OpenSCOW is a supercomputing portal and management system that enables HPC users to access compute...
Tokens:109,561
Snippets:1,053
Trust Score:9.7
Update:6 months ago
Show doc for...
Tokens:
Raw
### Initialize OpenSCOW Configuration

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/install

Run this command to generate the main installation configuration file (`install.yaml`) and an example configuration directory (`config`). These files are essential for setting up OpenSCOW.

```bash
./cli init  

```

--------------------------------

### Install Golang Environment

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/cluster-monitor

Steps to download, install, and configure the Go environment on the management node.

```bash
# 下载go语言安装包，安装gocd download/  
wget https://golang.google.cn/dl/go1.19.7.linux-amd64.tar.gz  
tar -C /usr/local -xzf go1.19.7.linux-amd64.tar.gz  
  
# 在/etc/profile中设置环境变量  
export GOROOT=/usr/local/go  
export GOPATH=/usr/local/gopath  
export PATH=$PATH:/$GOROOT/bin:$GOPATH/bin  
  
# source环境变量  
source /etc/profile  
  
# 验证  
go version  
  
# 设置代理  
go env -w GOPROXY=https://goproxy.cn,direct  
  
# 开启go mod管理  
go env -w GO111MODULE=on
```

--------------------------------

### Build and Start Local Development Environment

Source: https://pkuhpc.github.io/OpenSCOW/docs/contribution/dev

Commands to build and start the Docker-based development environment for local testing. Ensure 'prepareDev' has been run before starting tests.

```bash
# Build and start the development environment
pnpm devenv

# Run tests (requires prepareDev to be completed first)
pnpm test

# Run tests for a specific project (e.g., @scow/portal-web)
pnpm --filter portal-web test

# Stop the development environment
pnpm devenv:stop

```

--------------------------------

### Install and Configure RNG Tools

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Install rng-tools to provide a source of entropy for Munge. Configure rngd to use /dev/urandom and ensure the service starts on boot.

```bash
yum install -y rng-tools
```

```bash
rngd -r /dev/urandom
```

```bash
vim /usr/lib/systemd/system/rngd.service
# Modify to:
[service]
ExecStart=/sbin/rngd -f -r /dev/urandom
```

```bash
systemctl daemon-reload
systemctl start rngd
systemctl enable rngd
```

--------------------------------

### Install Desktop Environments

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/desktop

Commands to install Xfce, KDE Plasma, MATE, and Cinnamon desktop environments using yum groupinstall or install.

```bash
yum groupinstall "Xfce" -y  
yum groupinstall "KDE Plasma Workspaces" -y  
yum groupinstall "MATE Desktop" -y  
yum install cinnamon -y  
```

--------------------------------

### Clone Repository and Install Dependencies

Source: https://pkuhpc.github.io/OpenSCOW/docs/contribution/dev

Clone the OpenSCOW repository, navigate into the directory, and install project dependencies using pnpm. This includes installing node-gyp globally if necessary.

```bash
git clone https://github.com/PKUHPC/OpenSCOW

cd OpenSCOW

# If using standalone installation or encountering node-gyp ENOET errors during pnpm i, run this global install. Only needs to be done once.
pnpm install -g node-gyp

# Install dependencies in the root directory
pnpm i

# Prepare development libraries and code: build dependencies, generate various code files
pnpm prepareDev

```

--------------------------------

### Install Modules Tool

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/module

Download, compile, and install the modules tool, linking it against the previously installed Tcl library.

```bash
mkdir -p /data/software/module/tools/modules  
```

```bash
# 下载，此处可能需要科学上网  
wget https://newcontinuum.dl.sourceforge.net/project/modules/Modules/modules-4.2.4/modules-4.2.4.tar.gz   
  
#解压  
tar -zxvf modules-4.2.4.tar.gz   
cd modules-4.2.4   
  
# 编译 & 安装,注意修改路径  
./configure --prefix=/data/software/module/tools/modules --with-tcl-lib=/data/software/module/tools/tcl/lib --with-tcl-inc=/data/software/module/tools/tcl/include    
make   
make install  
```

--------------------------------

### Minimal install.yaml Configuration Reference

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/v-1-0-0

A minimal configuration example for install.yaml, showing essential parameters.

```yaml
port: 80
basePath: /
imageTag: v1.0.0
portal:
  portMappings: {}
mis:
  dbPassword: must!chang3this
  portMappings: {}
log:
  fluentd:
    logDir: /var/log/fluentd
auth:
  portMappings: {}
audit:
  dbPassword: "must!chang3this"
gateway:
  proxyReadTimeout: 36000s
```

--------------------------------

### Install MariaDB for Slurm Accounting

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Install and start the MariaDB server on the management node. Configure a root password and create the Slurm accounting database.

```bash
yum -y install mariadb-server
systemctl start mariadb
systemctl enable mariadb
```

```bash
ROOT_PASS=$(tr -dc A-Za-z0-9 </dev/urandom | head -c 16)
mysql -e "CREATE USER root IDENTIFIED BY '${ROOT_PASS}'"
mysql -uroot -p$ROOT_PASS -e 'create database slurm_acct_db'
```

```bash
mysql -uroot -p$ROOT_PASS

create user slurm;

grant all on slurm_acct_db.* TO 'slurm'@'localhost' identified by '123456' with grant option;

flush privileges;
```

--------------------------------

### Download and Setup openscow-cli

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/install/scow-cli

Commands to download the binary, rename it, and set execution permissions.

```bash
# 如果没有wget，可以先install  
yum install wget  
  
# 下载对应的release版本。修改latest、cli-x64可下载指定版本cli  
# 修改latest为v0.4.0即可下载0.4.0版本的cli，如 https://github.com/PKUHPC/OpenSCOW/releases/download/v0.4.0/cli-x64  
# 可修改cli-x64为cli-arm64下载arm64版本，如 https://github.com/PKUHPC/OpenSCOW/releases/download/v0.4.0/cli-arm64  
wget https://github.com/PKUHPC/OpenSCOW/releases/download/latest/cli-x64  
  
# 重命名  
mv cli-x64 cli  
  
#修改文件权限  
chmod +x cli  
```

--------------------------------

### Example Configuration for User Dropdown Links

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/customization/custom-userlinks

An example of how to configure user links, including text, URL, and opening in a new page. This configuration will be applied to both the portal and management systems.

```yaml
userLinks:  
  # 链接名称，必填  
  - text: "百度搜索"  
    # 链接地址，必填  
    url: "https://www.baidu.com"  
    # 是否打开新的页面，可选填，默认值为false  
    openInNewPage: true  

```

--------------------------------

### Start and Enable Munge Service

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Start the Munge service on all nodes and enable it to start automatically on boot. Verify the service status.

```bash
systemctl start munge
systemctl enable munge
```

--------------------------------

### Anaconda Installation and Initialization

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/jupyter

Commands to download, verify, install, and manually initialize Anaconda on a cluster.

```bash
wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-2023.03-Linux-x86_64.sh  
```

```bash
sha256sum Anaconda3-2023.03-Linux-x86_64.sh  
```

```bash
bash Anaconda3-2023.03-Linux-x86_64.sh  
```

```bash
# 命令中/data/software/anaconda/3-2023.03根据实际安装路径进行替换  
# 命令中shell.bash根据实际使用会话shell替换成相应的shell  
eval "$(/data/software/anaconda/3-2023.03/bin/conda shell.bash hook)"  
conda init  
```

--------------------------------

### Install scow-sync on Transfer Node

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/transfer-cross-clusters

Install the scow-sync software on each designated transfer node. This involves cloning the repository, installing Python dependencies, and running the installation script. Ensure the transfer node has network access to the scow-sync GitHub repository.

```bash
# Log in as root to the transfer node
ssh root@[TransferNodeIP] -p [TransferNodePort]
# Clone scow-sync to the /data/software directory
cd /data/software
git clone https://github.com/PKUHPC/scow-sync.git

```

--------------------------------

### Execute Intel oneAPI Base Toolkit Installer

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/intel

Runs the offline installer script for the Intel oneAPI Base Toolkit. Follow the on-screen prompts for custom installation, specifying the prepared directory.

```bash
sh l_BaseKit_p_2022.2.0.262_offline.sh
```

--------------------------------

### Run Documentation Project in Development Mode

Source: https://pkuhpc.github.io/OpenSCOW/docs/contribution/dev

Navigate to the documentation directory and use pnpm to start the documentation development server.

```bash
cd docs
pnpm start

```

--------------------------------

### Install R Dependencies and Source

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/rstudio

Install system dependencies and compile R from source with shared library support.

```bash
yum -y install gcc-gfortran gcc-c++ glibc-headers java-1.8.0-openjdk java-1.8.0-openjdk-devel libX11-devel libXt-devel xz-devel curl-devel bzip2-devel readline-devel zlib-devel openssl-devel pcre2
```

```bash
# 下载R的源代码并编译安装  
curl -LJO https://cran.rstudio.com/src/base/R-4/R-4.2.3.tar.gz  
tar zxf R-4.2.3.tar.gz  
  
# 必须有--enable-R-shlib这个选项(lib64/R/lib/libR.so)  
./configure --prefix=/data/software/R/R-4.2.3 --enable-R-shlib  
make -j  
make install
```

--------------------------------

### Copy and Configure Configuration File

Source: https://pkuhpc.github.io/OpenSCOW/blog

Copies the example configuration file to a new file and then instructs to modify the parameters within the new configuration file based on provided documentation.

```bash
# 1. 复制配置文件
cp config-example.py config.py

# 2. 配置参数
# 根据和备份的.env文件和config.py中的参数说明，修改config.py文件中的参数
```

--------------------------------

### Download and Install code-server

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/vscode

Download the code-server binary, extract it, and move it to the specified software directory. Ensure you have the correct version.

```bash
# 下载所需版本的code-server二进制安装包  
wget https://github.com/coder/code-server/releases/download/v4.11.0/code-server-4.11.0-linux-amd64.tar.gz  
  
# 解压code-server二进制安装包  
tar zxf code-server-4.11.0-linux-amd64.tar.gz  
  
# 将code-server重命名至指定路径  
mkdir -p /data/software/code-server  
mv code-server-4.11.0-linux-amd64 /data/software/code-server/4.11.0  


```

--------------------------------

### Minimal install.yaml Configuration

Source: https://pkuhpc.github.io/OpenSCOW/blog/scow-update-to-v1.0

Reference for a minimal installation configuration file.

```yaml
port: 80  
basePath: /  
imageTag: v1.0.0  
portal:  
  portMappings: {}  
mis:  
  dbPassword: must!chang3this  
  portMappings: {}  
log:  
  fluentd:  
    logDir: /var/log/fluentd  
auth:  
  portMappings: {}  
audit:  
  dbPassword: "must!chang3this"  
gateway:  
  proxyReadTimeout: 36000s
```

--------------------------------

### Install TurboVNC

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/emacs

Installs the TurboVNC software. Ensure you have the necessary repository configuration.

```bash
wget https://turbovnc.org/pmwiki/uploads/Downloads/TurboVNC.repo --no-check-certificate
mv TurboVNC.repo /etc/yum.repos.d
# Install the latest version
yum install turbovnc -y
```

--------------------------------

### Initialize Go Module

Source: https://pkuhpc.github.io/OpenSCOW/docs/integration/scow-api-hook/examples/go

Initialize a new Go project module. Ensure you have Go installed and a project directory.

```bash
go mod init github.com/PKUHPC/scow-go-demo  

```

--------------------------------

### Start and Enable Slurm Services

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Start and enable the Slurm daemons (slurmdbd, slurmctld, slurmd) on the respective nodes. Verify their status using systemctl.

```bash
# On the controller node:
systemctl start slurmdbd
systemctl enable slurmdbd

systemctl start slurmctld
systemctl enable slurmctld

# On all nodes:
systemctl start slurmd
systemctl enable slurmd
```

--------------------------------

### Install Tcl Dependency

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/module

Download, compile, and install Tcl, which is a prerequisite for the module tool.

```bash
mkdir -p /data/software/module/tools/tcl  
```

```bash
# 下载  
wget https://cfhcable.dl.sourceforge.net/project/tcl/Tcl/8.5.9/tcl8.5.9-src.tar.gz  
  
# 解压  
tar -zxvf tcl8.5.9-src.tar.gz   
cd tcl8.5.9/unix   
  
# 编译 & 安装，注意修改此处目录  
./configure --prefix=/data/software/module/tools/tcl   
make   
make install  
```

--------------------------------

### Cluster Configuration Example

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/v-1-0-0

Example configuration for cluster settings, including display name, login nodes, and adapter URL. Remember to update these values with your actual cluster details.

```yaml
displayName: hpc01

loginNodes:
  - name: hpc01_login01
    address: hpc01_login01

adapterUrl: "192.168.188.102:8999"
```

--------------------------------

### Install Singularity from Source

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/rstudio

Commands to download, configure, and compile Singularity CE from source code.

```bash
export VERSION=3.9.2  
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-ce-${VERSION}.tar.gz  
tar -xzf singularity-ce-${VERSION}.tar.gz  
cd singularity-ce-${VERSION}  
module load go/1.20.3  
./mconfig --prefix=/data/software/singularity/${VERSION}  
cd builddir/  
make -j && make install
```

--------------------------------

### Create Intel oneAPI Installation Directory

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/intel

Creates a directory for installing the Intel oneAPI Base Toolkit. Ensure this directory is on NFS for cluster-wide availability and has sufficient space (approx. 30GB).

```bash
mkdir -p /data/software/intel/oneapi
```

--------------------------------

### Minimal mis.yaml Configuration Reference

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/v-1-0-0

A minimal configuration example for mis.yaml, including database settings and job fetching.

```yaml
db:
  host: db
  port: 3306
  user: root
  dbName: scow

fetchJobs:
  periodicFetch:
    enabled: true
    cron: "*/10 * * * *"

predefinedChargingTypes:
  - 测试

accountNamePattern:
  regex: "(a_)[a-z]+$"
```

--------------------------------

### Start and Stop Services

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/scow-deployment

Commands to start or stop services using the compose script.

```bash
# 启动服务  
./compose.sh up -d  
  
# 停止服务  
./compose.sh down
```

--------------------------------

### Verify scow-sync Installation

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/transfer-cross-clusters

Confirm that scow-sync has been installed correctly by running its command-line interface. The `scow-sync-start -h` command should display usage information, and `scow-sync-query` should return an empty list if no transfers are active.

```bash
[root@login01 ~]# scow-sync-start -h
usage: scow-sync-start [-h] [-a ADDRESS] [-u USER] [-s SOURCE]
                       [-d DESTINATION] [-m MAX_DEPTH] [-p PORT]
                       [-k SSHKEY_PATH] [-c]

argsparser for starting transferring files

optional arguments:
  -h, --help            show this help message and exit
  -a ADDRESS, --address ADDRESS
                        address of the server
  -u USER, --user USER  username for logging in
  -s SOURCE, --source SOURCE
                        path of the source file or directory
  -d DESTINATION, --destination DESTINATION
                        path of the destination directory
  -m MAX_DEPTH, --max-depth MAX_DEPTH
                        max parallel depth of the directory
  -p PORT, --port PORT  ssh port of the server
  -k SSHKEY_PATH, --sshkey-path SSHKEY_PATH
                        path of the private key
  -c, --check           check whether the key in scow-sync-ssh is right
    
[root@login01 ~]# scow-sync-query
[]
[root@login01 ~]# scow-sync-terminate -h
usage: scow-sync-terminate [-h] [-a ADDRESS] [-u USER] [-s SOURCE]

argsparser for terminating transferring files

optional arguments:
  -h, --help            show this help message and exit
  -a ADDRESS, --address ADDRESS
                        address of the server
  -u USER, --user USER  username for logging in to the server
  -s SOURCE, --source SOURCE
                        path to the source file or directory

```

--------------------------------

### Start OpenSCOW Services

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/install

After configuring the necessary YAML files, use this command to start the OpenSCOW system in detached mode (`-d`).

```bash
./cli compose up -d  

```

--------------------------------

### Compile and Install Exporter

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/cluster-monitor

Commands to build the exporter from source and register it as a systemd service.

```bash
# 下载代码  
git clone https://github.com/PKUHPC/prometheus-slurm-exporter.git  
cd prometheus-slurm-exporter  
// 编译  
make  
# 安装  
cp bin/prometheus-slurm-exporter /usr/bin/prometheus-slurm-exporter  
cp lib/systemd/prometheus-slurm-exporter.service /usr/lib/systemd/system  
# 启动exporter服务  
systemctl start prometheus-slurm-exporter.service
```

--------------------------------

### Run Portal Frontend in Development Mode

Source: https://pkuhpc.github.io/OpenSCOW/docs/contribution/dev

Navigate to the portal frontend directory and run the 'dev' script using pnpm to start the application in development mode.

```bash
cd apps/portal-web
pnpm dev

```

--------------------------------

### Start Monitoring Services with Docker Compose

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/cluster-monitor

Command to start the monitoring services defined in the docker-compose.yaml file. This should be run after the configuration file is edited.

```bash
cd /root/monitor
docker-compose -f monitor.yaml up -d
```

--------------------------------

### Install OpenSCOW Protobuf Files via npm

Source: https://pkuhpc.github.io/OpenSCOW/docs/integration/scow-api-hook/proto

Install OpenSCOW API proto files using npm. This allows access to the files for use in projects or for direct retrieval from the global module directory.

```bash
# Install the latest version of API files
npm install -g @scow/grpc-api

# Install a specific version (0.1.2) of the API
npm install -g @scow/grpc-api@0.1.2

# After installation, navigate to the npm global modules directory to find the package
cd $(npm root -g)/@scow/grpc-api

# If managing dependencies with npm in your project, install the package and access proto files from node_modules
npm install --save @scow/grpc-api
cd node_modules/@scow/grpc-api
```

--------------------------------

### Install Java 1.8

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/igv

Installs OpenJDK version 1.8 using yum. This is a prerequisite for running IGV.

```bash
yum install java-1.8.0-openjdk
```

--------------------------------

### Install TurboVNC

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/desktop

Commands to download the TurboVNC repository configuration and install the latest version via yum.

```bash
wget https://turbovnc.org/pmwiki/uploads/Downloads/TurboVNC.repo --no-check-certificate  
mv TurboVNC.repo /etc/yum.repos.d  
# 安装最新版本  
yum install turbovnc -y  
```

--------------------------------

### User Dropdown Link Configuration Examples

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/customization/custom-userlinks

Demonstrates different ways to configure user links, including external URLs, internal paths, and how they are handled during navigation. The system appends a token for tracking user state.

```yaml
  userLinks:  
    - text: "菜单1"  
      url: "http://www.navtest1.com"  
    - text: "菜单2"  
      url: "www.navtest2.com"  
    - text: "菜单3"  
      url: "/www.navtest3.com"  

```

--------------------------------

### Example Proxy Gateway URL Configuration

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/proxy-gateway

An example of how to specify the proxy gateway URL in the cluster configuration file, including the IP address and port of the Nginx listener.

```yaml
proxyGateway:
  url: http://192.168.88.100:12031
```

--------------------------------

### Install VSCode Extension from VSIX File

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/vscode

Install a VSCode extension by manually downloading its .vsix file and then using the --install-extension parameter in the terminal. Be cautious about compatibility with extensions downloaded from unofficial sources.

```bash
code-server --install-extension /path/to/extension.vsix  


```

--------------------------------

### Create Module Directory

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/module

Initialize the base directory for module installations on the shared storage.

```bash
mkdir /data/software/module  
```

--------------------------------

### Minimal mis.yaml Configuration Reference

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/scow

A simplified configuration example for `mis.yaml`, showing database settings, periodic job fetching, predefined charging types, and account name pattern.

```yaml
db:  
  host: db  
  port: 3306  
  user: root  
  dbName: scow  
  
fetchJobs:  
  periodicFetch:  
    enabled: true  
    cron: "*/10 * * * *"  
  
predefinedChargingTypes:  
  - 测试  
  
accountNamePattern:  
  regex: "(a_)[a-z]+$"  

```

--------------------------------

### Configure Audit Logs in install.yaml

Source: https://pkuhpc.github.io/OpenSCOW/blog/scow-update-to-v1.0

Add audit log configuration to the installation file.

```yaml
# 可添加审计日志功能，并配置审计日志数据库密码  
audit:  
  dbPassword: "must!chang3this"
```

--------------------------------

### Prepare Configuration File

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/scow-deployment

Copy and configure the deployment parameters.

```bash
# 1. 复制配置文件  
cp config-example.py config.py  
  
# 2. 配置参数  
# 根据和备份的.env文件和config.py中的参数说明，修改config.py文件中的参数
```

--------------------------------

### 配置安装文件 install.yaml

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/audit/intro

设置审计系统数据库的初始密码，该密码在系统首次启动后不可更改。

```yaml
# 确保审计系统会部署  
audit:  
  
  # dbPassword为审计系统数据库密码  
  # 在系统第一次启动前可自由设置，使用此密码可以以root身份登录数据库  
  # 一旦数据库启动后即不可修改  
  # 必须长于8个字符，并同时包括字母、数字和符号  
  dbPassword: "must!chang3this"  
```

--------------------------------

### Call OpenSCOW Account API in Go

Source: https://pkuhpc.github.io/OpenSCOW/docs/integration/scow-api-hook/examples/go

Example Go code to connect to the OpenSCOW API and retrieve account information. This requires the generated Go code from proto files and assumes the OpenSCOW server is running.

```go
package main  
  
import (  
	"context"  
	"log"  

	"github.com/PKUHPC/scow-go-demo/gen/go/server"  
	"google.golang.org/grpc"  
	"google.golang.org/grpc/credentials/insecure"  
)
  
func main() {  
  // 假设mis-server在192.168.88.100:7571上监听  
	conn, err := grpc.Dial("192.168.88.100:7571", grpc.WithTransportCredentials(insecure.NewCredentials()))  
  
  if err != nil {  
    panic(err)  
  }  
  
  // 生成AccountServiceClient  
	client := server.NewAccountServiceClient(conn)  
  
  // 调用`GetAccounts` RPC，获取所有账户  
	resp, err := client.GetAccounts(context.Background(), &server.GetAccountsRequest{})  
  
	if err != nil {  
		panic(err)  
	}
  
	log.Printf("Account list: %v", resp)
}

```

--------------------------------

### Create Dockerfile for RStudio Server

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/rstudio

Define the base image, copy the RPM, and install necessary dependencies for RStudio Server.

```dockerfile
FROM centos:7  
  
COPY package/rstudio-server-rhel-2023.03.0-386-x86_64-relwithdebinfo.rpm /  
  
RUN yum -y update && \
    yum -y install epel-release && \
    yum -y install which gcc-gfortran gcc-c++ glibc-headers java-1.8.0-openjdk java-1.8.0-openjdk-devel libX11-devel libXt-devel xz-devel curl-devel bzip2-devel readline-devel zlib-devel openssl-devel pcre2 initscripts postgresql-libs psmisc && \
    cd / && \
    rpm2cpio rstudio-server-rhel-2023.03.0-386-x86_64-relwithdebinfo.rpm | cpio -div && \
    mkdir data && \
    yum clean all
```

--------------------------------

### Install Python Dependencies for scow-sync

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/transfer-cross-clusters

Install the required Python libraries for scow-sync on the transfer node. Ensure your Python version is 3.6.0 or higher, and install specified versions of paramiko and psutil.

```text
python >= 3.6.0
paramiko >= 3.0.0
psutil >= 5.9.4
subprocess
concurrent

```

--------------------------------

### Configure Slurm Control Node

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Copy example Slurm configuration files to their respective locations on the control node. Review and customize slurm.conf and slurmdbd.conf.

```bash
cp /etc/slurm/cgroup.conf.example /etc/slurm/cgroup.conf
cp /etc/slurm/slurm.conf.example /etc/slurm/slurm.conf
cp /etc/slurm/slurmdbd.conf.example /etc/slurm/slurmdbd.conf
```

--------------------------------

### Set Up Slurm Directories and Permissions

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Create necessary spool and log directories for Slurm services and set the correct ownership to the 'slurm' user.

```bash
mkdir /var/spool/slurmd
chown slurm: /var/spool/slurmd
mkdir /var/log/slurm
chown slurm: /var/log/slurm

mkdir /var/spool/slurmctld
chown slurm: /var/spool/slurmctld
```

--------------------------------

### 配置安装文件 install.yaml

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/mis/intro

定义管理系统的部署路径和数据库初始密码。数据库密码在系统首次启动后不可修改。

```yaml
# 确保管理系统会部署  
mis:  
  # 如果管理系统部署在`/mis`路径下，设置"MIS.BASE_PATH"为 "/mis"。默认为"/"  
  basePath: "/mis"  
  # 如果将会部署在域名的根目录下，设置"MIS.BASE_PATH"为 "/"  
  basePath: "/"  
  
  # dbPassword为管理系统数据库密码  
  # 在系统第一次启动前可自由设置，使用此密码可以以root身份登录数据库  
  # 一旦数据库启动后即不可修改  
  # 必须长于8个字符，并同时包括字母、数字和符号  
  dbPassword: "must!chang3this"  
```

--------------------------------

### Create Nginx Configuration File

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/proxy-gateway

Use this command to create the Nginx configuration file for the proxy gateway. Ensure the path is correct for your Nginx setup.

```bash
touch /etc/nginx/conf.d/scow-portal-proxy-gateway.conf
```

--------------------------------

### 配置日志输出选项

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/ops

在 install.yaml 中设置日志等级和输出格式。

```yaml
log:  
  # LOG.LEVEL: 日志等级，可选trace, debug, info, warn, error。默认info  
  level: "info"  
  
  # LOG.PRETTY: 是否输出更适合直接读的log。如果为false，则输出json格式的日志。默认false  
  pretty: false  
```

--------------------------------

### Install Emacs

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/emacs

Installs the Emacs text editor using yum package manager.

```bash
yum install emacs -y
```

--------------------------------

### Configure VSCode Application with Custom Icon

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/configure-app-logo

This configuration example demonstrates how to set a custom icon for the VSCode application using the `logoPath` parameter. Ensure the image file is placed in the public directory and its path is correctly specified.

```yaml
# 这个应用的ID
id: vscode

# 这个应用的名字
name: VSCode

# 这个应用的图标图片在公共文件下的地址
logoPath: /apps/VScode.svg

# 指定应用类型为web
type: web

# Web应用的配置
web:

  # 指定反向代理类型
  proxyType: relative

  # 准备脚本
  beforeScript: |
    export PORT=$(get_port)
    export PASSWORD=$(get_password 12)

  # 运行任务的脚本。使用了用户在自定义表单中选择的选项
  script: |
    module load ${selectVersion}
    PASSWORD=$PASSWORD code-server -vvv --bind-addr 0.0.0.0:$PORT --auth password

  # 如何连接应用
  connect:
    method: POST
    path: /login
    formData:
      password: "{{ PASSWORD }}"

# 配置HTML表单，用户可以指定code-server版本        
attributes:
  - type: select
    name: selectVersion
    label: 选择版本
    required: true  # 用户必须选择一个版本
    placeholder: 选择code-server的版本  # 提示信息
    select:
      - value: code-server/4.8.0
        label: version 4.8.0
      - value: code-server/4.9.0
        label: version 4.9.0

```

--------------------------------

### 编写新版配置文件

Source: https://pkuhpc.github.io/OpenSCOW/blog/update-to-python-deployment

根据备份的.env文件内容修改新生成的config.py配置文件。

```bash
# 1. 复制配置文件  
cp config-example.py config.py  
  
# 2. 配置参数  
# 根据和备份的.env文件和config.py中的参数说明，修改config.py文件中的参数  
```

--------------------------------

### Download and Prepare OpenSCOW CLI

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/install

This snippet shows how to create a directory for OpenSCOW configuration, download the `openscow-cli` tool, and make it executable. Place the `scow-cli` binary in this directory.

```bash
# scow目录将会用于存在OpenSCOW相关的配置文件  
mkdir scow  
cd scow  

# 将下载的openscow-cli移动到scow目录下  
cp /path/to/scow-cli ./  
chmod +x scow-cli  

```

--------------------------------

### Prepare and Run Linkpack Benchmark

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/intel

This section covers setting up the environment for the Intel MPI library, building the xhpl executable from the provided benchmarks, and running it locally and across multiple nodes.

```bash
# 设置环境变量
source /data/software/intel/oneapi/mpi/2021.6.0/env/vars.sh

# 生成xhpl
cd /data/software/intel/oneapi/mkl/2022.1.0/benchmarks/mp_linpack
./build.sh

# 执行计算
mpirun -n 36 ./xhpl

# 多节点
mpirun -hosts cn1,cn2,cn3,cn4 -np 36 -ppn 4 /data/software/intel/oneapi/mkl/2022.1.0/benchmarks/mp_linpack/xhpl
```

--------------------------------

### Install Jupyter Notebook

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/jupyter

Command to install Jupyter Notebook if it is not already included with the Anaconda distribution.

```bash
conda install jupyter notebook  
```

--------------------------------

### Install Slurm on All Nodes

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Install the Slurm packages on all nodes using yum localinstall after copying the RPMs.

```bash
cd /root/rpmbuild/RPMS/x86_64/
yum localinstall slurm-*
```

--------------------------------

### 配置安装数据库密码

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/ai/intro

在 install.yaml 中设置 AI 系统数据库的初始 root 密码。

```yaml
# 其他配置省略  
# ...  
# 确保 AI 系统会部署  
ai:  
  # dbPassword 为 AI 系统数据库密码  
  # 在系统第一次启动前可自由设置，使用此密码可以以 root 身份登录数据库  
  # 一旦数据库启动后即不可修改  
  # 必须长于 8 个字符，并同时包括字母、数字和符号  
  dbPassword: "must!chang3this"  
```

--------------------------------

### Install JupyterLab via Conda

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/jupyterlab

Use this command to install JupyterLab if it was not automatically included with your Anaconda distribution.

```bash
conda install jupyterlab  
```

--------------------------------

### Install Octave via Conda

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/octave

Command to install the specific version of Octave within the active environment.

```bash
conda install octave=7.2.0  
```

--------------------------------

### Install Slurm Dependencies

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Install necessary development tools and libraries required for compiling and running Slurm on all nodes.

```bash
yum install gcc gcc-c++ readline-devel perl-ExtUtils-MakeMaker pam-devel rpm-build mysql-devel python3 -y
```

--------------------------------

### Initialize Database Schema

Source: https://pkuhpc.github.io/OpenSCOW/docs/hpccluster/cluster-monitor

SQL script to create the initial database tables and a bootstrap procedure for version tracking.

```sql
DROP PROCEDURE IF EXISTS bootstrap;  
  
DELIMITER //  
CREATE PROCEDURE bootstrap()  
BEGIN  
  SET @exists := (SELECT 1 FROM information_schema.tables I WHERE I.table_name = "Model" AND I.table_schema = database());  
  IF @exists IS NULL THEN  
  
    CREATE TABLE `Model` (  
      `ID` enum('1') NOT NULL,  
      `version` VARCHAR(20) NOT NULL,  
      PRIMARY KEY (`ID`)  
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;  
  
    INSERT INTO `Model` (`version`) VALUES ("0.0.1");  
  
  ELSE  
    SIGNAL SQLSTATE '42000' SET MESSAGE_TEXT='Model Table Exists, quitting...';  
  END IF;  
END;  
//  
DELIMITER ;  
  
-- Execute the procedure  
CALL bootstrap();  
  
-- Drop the procedure  
DROP PROCEDURE bootstrap;  
  
-- Create the rest of the tables  
CREATE TABLE `AlertGroup` (  
        `ID` INT NOT NULL AUTO_INCREMENT,  
        `time` TIMESTAMP NOT NULL,  
        `receiver` VARCHAR(100) NOT NULL,  
        `status` VARCHAR(50) NOT NULL,  
        `externalURL` TEXT NOT NULL,  
        `groupKey` VARCHAR(255) NOT NULL,  
        KEY `idx_time` (`time`) USING BTREE,  
    KEY `idx_status_ts` (`status`, `time`) USING BTREE,  
        PRIMARY KEY (`ID`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  
  
CREATE TABLE `GroupLabel` (  
        `ID` INT NOT NULL AUTO_INCREMENT,  
    `AlertGroupID` INT NOT NULL,  
    `GroupLabel` VARCHAR(100) NOT NULL,  
    `Value` VARCHAR(1000) NOT NULL,  
    FOREIGN KEY (AlertGroupID) REFERENCES AlertGroup (ID) ON DELETE CASCADE,  
        PRIMARY KEY (`ID`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  
  
CREATE TABLE `CommonLabel` (  
        `ID` INT NOT NULL AUTO_INCREMENT,  
    `AlertGroupID` INT NOT NULL,  
    `Label` VARCHAR(100) NOT NULL,  
    `Value` VARCHAR(1000) NOT NULL,  
    FOREIGN KEY (AlertGroupID) REFERENCES AlertGroup (ID) ON DELETE CASCADE,  
        PRIMARY KEY (`ID`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  
  
CREATE TABLE `CommonAnnotation` (  
        `ID` INT NOT NULL AUTO_INCREMENT,  
    `AlertGroupID` INT NOT NULL,  
    `Annotation` VARCHAR(100) NOT NULL,  
    `Value` VARCHAR(1000) NOT NULL,  
    FOREIGN KEY (AlertGroupID) REFERENCES AlertGroup (ID) ON DELETE CASCADE,  
        PRIMARY KEY (`ID`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  
  
CREATE TABLE `Alert` (  
        `ID` INT NOT NULL AUTO_INCREMENT,  
    `alertGroupID` INT NOT NULL,  
        `status` VARCHAR(50) NOT NULL,  
    `startsAt` DATETIME NOT NULL,  
    `endsAt` DATETIME DEFAULT NULL,  
        `generatorURL` TEXT NOT NULL,  
    FOREIGN KEY (alertGroupID) REFERENCES AlertGroup (ID) ON DELETE CASCADE,  
        PRIMARY KEY (`ID`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  
  
CREATE TABLE `AlertLabel` (  
        `ID` INT NOT NULL AUTO_INCREMENT,  
    `AlertID` INT NOT NULL,  
    `Label` VARCHAR(100) NOT NULL,  
    `Value` VARCHAR(1000) NOT NULL,  
    FOREIGN KEY (AlertID) REFERENCES Alert (ID) ON DELETE CASCADE,  
        PRIMARY KEY (`ID`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  
  
CREATE TABLE `AlertAnnotation` (  
        `ID` INT NOT NULL AUTO_INCREMENT,  
    `AlertID` INT NOT NULL,  
    `Annotation` VARCHAR(100) NOT NULL,  
    `Value` VARCHAR(1000) NOT NULL,  
    FOREIGN KEY (AlertID) REFERENCES Alert (ID) ON DELETE CASCADE,  
        PRIMARY KEY (`ID`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
```

--------------------------------

### Install R via Conda

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/rstudio

Use Conda to create a virtual environment and install a specific version of R.

```bash
# 载入conda到系统环境中，并初始化conda环境  
module load anaconda/3-2023.03  
eval "$($(which conda) shell.bash hook)"  
  
# 添加北大镜像源，查找相应版本的R进行安装  
conda config --add channels https://mirrors.pku.edu.cn/anaconda/pkgs/free/  
conda config --add channels https://mirrors.pku.edu.cn/anaconda/pkgs/main/  
conda config --add channels https://mirrors.pku.edu.cn/anaconda/cloud/conda-forge/  
conda search R  
  
  
# 根据查找到的R版本，创建Conda虚拟环境，并在创建的时候指定需要安装的R版本，这里选择安装R-3.6.0版本;  
conda create -n R-3.6.0 r=3.6.0 -y  
  
# 进入上一步创建好的虚拟环境；  
conda activate R-3.6.0  
  
# 安装R语言库，比如stringi  
conda install r-stringi -y  
  
# 安装完后就可以用R语言环境了，如果要退出当前环境则执行以下指令；  
conda deactivate
```

--------------------------------

### Install TurboVNC

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/igv

Installs the latest version of TurboVNC using yum. Ensure you have the TurboVNC repository file configured.

```bash
wget https://turbovnc.org/pmwiki/uploads/Downloads/TurboVNC.repo --no-check-certificate
mv TurboVNC.repo /etc/yum.repos.d
# 安装最新版本
yum install turbovnc -y
```

--------------------------------

### Minimal install.yaml Configuration

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/scow

A basic configuration reference for `install.yaml`, including port, basePath, imageTag, and settings for portal, mis, log, auth, audit, and gateway.

```yaml
port: 80  
basePath: /  
imageTag: v1.0.0  
portal:  
  portMappings: {}  
mis:  
  dbPassword: must!chang3this  
  portMappings: {}  
log:  
  fluentd:  
    logDir: /var/log/fluentd  
auth:  
  portMappings: {}  
audit:  
  dbPassword: "must!chang3this"  
gateway:  
  proxyReadTimeout: 36000s  

```

--------------------------------

### Install Munge Authentication Service

Source: https://pkuhpc.github.io/OpenSCOW/docs/slurm

Install Munge and its development libraries on all nodes. Munge is essential for authenticating processes between local or remote hosts.

```bash
yum install epel-release -y
yum install munge munge-libs munge-devel -y
```

--------------------------------

### Run OpenSCOW API Client

Source: https://pkuhpc.github.io/OpenSCOW/docs/integration/scow-api-hook/examples/go

Download Go module dependencies and run the OpenSCOW API client code. Ensure the Go environment is set up correctly.

```bash
go mod tidy  
go run api.go  

```

--------------------------------

### Start and Stop Services

Source: https://pkuhpc.github.io/OpenSCOW/blog

Manages the OpenCow services using Docker Compose. Use 'up -d' to start services in detached mode and 'down' to stop them.

```bash
# 启动服务
./compose.sh up -d

# 停止服务
./compose.sh down
```

--------------------------------

### Login/Compute节点镜像配置

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/get-started/vagrant/images

执行此脚本以配置基础系统环境、安装桌面环境、TurboVNC、Munge及Slurm。

```bash
# 1. 升级到centos7.9.2009  
yum update -y   
  
# 2. 关闭firewalld、dnsmasq、NetworkManager  
systemctl disable --now firewalld   
systemctl disable --now dnsmasq  
systemctl disable --now NetworkManager  
  
# 3. 关闭selinux  
setenforce 0  
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/sysconfig/selinux  
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config  
getenforce  
  
# 4. 关闭swap  
swapoff -a && sysctl -w vm.swappiness=0enforcing  
sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab  
  
  
# 5. 时间同步  
rpm -ivh http://mirrors.wlnmp.com/centos/wlnmp-release-centos.noarch.rpm  
yum install ntpdate vim -y  
  
# 同步时间。time2.aliyun.com外网，vineyard.pku.edu.cn内网  
# 时间同步配置如下：  
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime  
echo 'Asia/Shanghai' >/etc/timezone  
ntpdate time2.aliyun.com  
  
cat >>/var/spool/cron/root <<EOF  
*/5 * * * * /usr/sbin/ntpdate time2.aliyun.com  
EOF  
  
# 6. 安装epel-release  
yum install epel-release -y  
  
# 7. 安装emacs wget  
yum install emacs wget -y  
  
# 8. 安装turbovnc  
wget https://turbovnc.org/pmwiki/uploads/Downloads/TurboVNC.repo --no-check-certificate  
mv TurboVNC.repo /etc/yum.repos.d  
# 安装最新版本  
yum install turbovnc -y  
  
# 9. 安装桌面  
yum groupinstall "Xfce" -y  
yum groupinstall "KDE Plasma Workspaces" -y  
yum groupinstall "MATE Desktop" -y  
yum install cinnamon -y  
  
# 10. 安装rng-tools  
yum install -y rng-tools  
rngd -r /dev/urandom  
  
# 11. 安装munge  
yum install munge munge-libs munge-devel -y  
  
# 12. 安装slurm  
yum install gcc gcc-c++ readline-devel perl-ls-MakeMaker pam-devel rpm-build mysql-devel python3 -y  
# 此处是拷贝的slurm的编译文件  
cp -r /vagrant/slurm/rpmbuild/ /root/   
cd /root/rpmbuild/RPMS/x86_64/ && yum localinstall slurm-* -y  
  
# 13. 支持密码登录登录  
sed -i 's#PasswordAuthentication no#PasswordAuthentication yes#g' /etc/ssh/sshd_config  
systemctl restart sshd  
```

--------------------------------

### Backup .env Configuration File

Source: https://pkuhpc.github.io/OpenSCOW/blog/tags/scow

Steps to back up the `.env` configuration file. Create a backup directory and copy the `.env` file into it.

```bash
# 1. 创建备份目录  
mkdir /path/to/backup  
  
# 2. 备份部署的环境变量  
cp .env /path/to/backup  
  

```

--------------------------------

### Install RStudio Server (Older Versions)

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/rstudio

Use this script to download, copy, and extract RStudio Server RPM packages for versions prior to 1.4.1103. Ensure R is installed separately.

```bash
# 下载rpm包  
wget https://download2.rstudio.org/server/centos6/x86_64/rstudio-server-rhel-1.3.959-x86_64.rpm  
  
# 拷贝只共享存储  
mkdir -p /data/software/rstudio-server/1.3.959  
cp rstudio-server-rhel-1.3.959-x86_64.rpm /data/software/rstudio-server/1.3.959  
cd /data/software/rstudio-server/1.3.959  
  
# 解压安装  
rpm2cpio rstudio-server-rhel-1.3.959-x86_64.rpm | cpio -div  

```

--------------------------------

### Install VSCode Extension by Name/ID

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/apps/apps/vscode

Install a VSCode extension using its identifier or name directly from the terminal within code-server. Ensure the extension identifier is correct, typically in 'publisher.extension' format.

```bash
# <extension-name>替换为你想安装的扩展的标识符或名称。扩展的标识符通常是类似于publisher.extension的格式。例如，要安装 "Python" 扩展，扩展标识符为ms-python.python  
code-server --install-extension <extension-name>  


```

--------------------------------

### 启动门户服务

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/portal/intro

使用 CLI 工具启动门户系统容器。

```bash
./cli compose up -d
```

--------------------------------

### 更新 OpenSCOW 版本配置

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/ops

在 install.yaml 中指定 imageTag 以更新系统版本。

```yaml
# 使用v0.4.0版本  
imageTag: v0.4.0  
```

--------------------------------

### 重启系统服务

Source: https://pkuhpc.github.io/OpenSCOW/docs/deploy/config/auth/ssh

应用配置更改后，使用CLI工具重启系统。

```bash
./cli compose restart
```
Tokens:
Raw