### setup.config File Format Example

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=files-setupconfig

Illustrates the basic format of entries within the setup.config file. Each entry consists of a NAME followed by an equals sign and a string value, which may be enclosed in quotes if it contains multiple space-separated strings. Blank lines and comments starting with '#' are ignored.

```shell
NAME="STRING1 STRING2 ..."
```

--------------------------------

### Perform Silent LSF Installation and Uninstallation via msiexec

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=windows-installation-parameter-quick-reference

Demonstrates the command-line syntax for performing silent installations and uninstalls of LSF on Windows using the msiexec utility. It includes an example of specifying server hosts, host types, and installation directories.

```shell
msiexec /i \\hostB\download\lsf<version>_win-x64.msi SERVERHOSTS=hostM HOSTTYPE=Server INSTALLDIR=C:\LSF /quiet
msiexec /x package_file /quiet
```

--------------------------------

### Verify LSF Web Services Installation

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=services-installing-lsf-web-high-availability-environment

Commands to navigate to the installation directory and list files to confirm successful installation.

```bash
cd /opt/ibm/lsfsuite/ext
ls -l
```

--------------------------------

### Execute LSF Web Services Installer

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=services-installing-lsf-web-high-availability-environment

Command to initiate the installation script for LSF Web Services.

```bash
./lwsinstall.sh
```

--------------------------------

### Configure LSF Host Installation Options

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=queues-adding-host-your-cluster

Edit the `install.config` file to specify installation options for new host types. This file is crucial for customizing the installation process based on different host configurations. Refer to the IBM Spectrum LSF Configuration Reference for detailed options.

```text
3. Edit the install.config file to specify the options you want for new host types.
For more information about the install.config file, see the IBM® Spectrum LSF Configuration Reference. For information about the lsfinstall command, see Installing IBM Spectrum LSF on UNIX and Linux and the IBM Spectrum LSF Command Reference.
4. Run the `./lsfinstall -f install.config` command.
```

--------------------------------

### Execute Tasks with ClearCase Job Starter

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=clearcase-submitting-interactive-tasks-in

Demonstrates how to run a specific task on a target host using the configured clearstarter script via the lsrun command.

```shell
% lsrun -m hostA clearstarter mytask
```

--------------------------------

### Viewing Lower-Bound Job Start Time Estimation

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=dispatch-job-start-time-prediction

This section explains how to interpret the lower-bound job start time estimation when a precise start time cannot be determined. It details the output format and provides an example.

```APIDOC
## GET /websites/ibm_en_spectrum-lsf/bjobs-estimation/lower-bound

### Description
Retrieves lower-bound job start time estimation results when a precise start time could not be estimated. This endpoint is conceptual and represents the information provided by the `bjobs -l` command.

### Method
GET

### Endpoint
`/websites/ibm_en_spectrum-lsf/bjobs-estimation/lower-bound`

### Parameters
#### Query Parameters
- **job_id** (string) - Required - The ID of the job to query lower-bound estimation results for.

### Request Example
```
GET /websites/ibm_en_spectrum-lsf/bjobs-estimation/lower-bound?job_id=67890
```

### Response
#### Success Response (200)
- **estimation_details** (object) - Contains the detailed lower-bound estimation information.
  - **status** (string) - Indicates the status of the estimation ('LOWER_BOUND').
  - **lower_bound_time** (string) - The earliest possible start time.
  - **simulation_details** (string) - Information about the simulation process leading to the lower bound.

#### Response Example
```json
{
  "estimation_details": {
    "status": "LOWER_BOUND",
    "lower_bound_time": "Fri Aug 28 18:00:29",
    "simulation_details": "Started simulation-based estimation; Simulated job cannot start before <Fri Aug 28 18:00:29>;"
  }
}
```
```

--------------------------------

### Viewing Job Start Time Estimation Results

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=dispatch-job-start-time-prediction

This section details how to view successful job start time estimations using the `bjobs -l` command. It explains the output format and provides an example.

```APIDOC
## GET /websites/ibm_en_spectrum-lsf/bjobs-estimation

### Description
Retrieves detailed job start time estimation results from the `bjobs -l` command output. This endpoint is conceptual and represents the information provided by the command.

### Method
GET

### Endpoint
`/websites/ibm_en_spectrum-lsf/bjobs-estimation`

### Parameters
#### Query Parameters
- **job_id** (string) - Required - The ID of the job to query estimation results for.

### Request Example
```
GET /websites/ibm_en_spectrum-lsf/bjobs-estimation?job_id=12345
```

### Response
#### Success Response (200)
- **estimation_details** (object) - Contains the detailed estimation information.
  - **status** (string) - Indicates the status of the estimation (e.g., 'SUCCESS', 'LOWER_BOUND', 'NONE').
  - **estimated_start_time** (string) - The predicted start time if estimation was successful.
  - **allocated_hosts** (array) - A list of hosts allocated for the job.
  - **lower_bound_time** (string) - The earliest possible start time if a lower bound estimate is available.
  - **simulation_details** (string) - Information about the simulation process.

#### Response Example
```json
{
  "estimation_details": {
    "status": "SUCCESS",
    "estimated_start_time": "Thu Aug 27 18:00:02",
    "allocated_hosts": [
      "1* host1",
      "15* host2"
    ],
    "simulation_details": "Started simulation-based estimation; Simulated job start time <Thu Aug 27 18:00:02> on host(s) <1* host1> <15* host2>;"
  }
}
```
```

--------------------------------

### Example Host Partition Configuration in LSF

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=files-lsbhosts

Provides a complete example of a HostPartition section in LSF configuration. It defines the partition name, the hosts included in the partition, and the user share assignments for resource allocation.

```bash
Begin HostPartition
HPART_NAME = Partition1
HOSTS = hostA hostB
USER_SHARES = 
[groupA@, 3] [groupB, 7] [default, 1]
End HostPartition
```

--------------------------------

### Host Preference Determining Compute Unit Preference (Example 2)

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=units-control-job-locality-using-compute

This example shows a more complex scenario where multiple hosts within different compute units have assigned preferences. LSF determines the compute unit's preference by the highest preference of any host within it. `cu1` gets `10` from `h1+10`, and `cu2` gets `9` from `h3+9` and `h4+9`.

```bash
bsub -n 2 -m "h1+10 h2+1 h3+9 h4+9" -R "cu[pref=minavail]" ./app
```

--------------------------------

### Enable Fair Share Scheduling

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=files-installconfig

Example configuration for the lsb.queues file to enable fair share scheduling for user queues.

```text
Begin Queue
...
FAIRSHARE=USER_SHARES[[default, 1]]
...
End Queue
```

--------------------------------

### Navigate to LSF installation directory and configure automatic startup

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=linux-configuring-cluster

Changes the directory to the LSF installation path and prepares for configuring LSF daemons to start automatically at boot time. This is typically done on LSF server hosts.

```bash
# cd /usr/share/lsf/10.1.0/install
```

--------------------------------

### Default Host File Syntax Example

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=files-hosts

Illustrates the default syntax for a host file, mapping IP addresses to official host names and their aliases. This format can become unmanageable with numerous aliases or multihomed hosts.

```text
ip_address official_name [alias [alias ...]]
```

--------------------------------

### Configure Multi-User Cluster Permissions for LSF

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=iul-if-you-install-lsf-as-non-root-user

This snippet demonstrates the commands required to set up file ownership and permissions for a multi-user LSF cluster when installing as a non-root user. It ensures that LSF daemons can be started by non-root users and that other users can submit jobs.

```shell
# hostsetup --setuid
# chown root lsadmin badmin eauth swtbl_api ntbl_api
# chmod 4755 lsadmin badmin eauth swtbl_api ntbl_api
```

--------------------------------

### IBM Spectrum LSF Job Dependency Condition Example

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=arrays-setting-whole-array-dependency

This example demonstrates how to create a job dependency in IBM Spectrum LSF. It uses the 'numdone' condition to specify that a job should start only after a certain number of jobs in a job array have successfully completed. The condition is applied using the 'bsub -w' command.

```bash
bsub -w "numdone(123, >= 100)" myJob
```

--------------------------------

### Set Up New LSF Host

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=queues-adding-host-your-cluster

Use the `hostsetup` command to configure a new host for the LSF cluster. This command requires specifying the LSF top directory and whether to enable boot startup. Ensure you are logged in as root and in the LSF installation script directory.

```bash
# cd /usr/share/lsf/cluster1/10.1.0/install
# ./hostsetup --top="/usr/share/lsf/lsf_62" --boot="y"
```

--------------------------------

### Host Preference Determining Compute Unit Preference (Example 1)

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=units-control-job-locality-using-compute

This example illustrates how LSF calculates compute unit preference based on host preferences within those units. Here, `cu1` is assigned a preference of `10` due to `h1+10`, while `cu2` gets `0` as no hosts in `cu2` have explicit positive preferences.

```bash
bsub -n 2 -m "h1+10 others" -R "cu[pref=minavail]" ./app
```

--------------------------------

### Execute LSF Session Scheduler Installation

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=scheduler-installing-lsf-session

Command to run the ssinstall script using a specified configuration file. This script performs the actual installation of the cluster based on the parameters defined in install.config.

```bash
ssinstall -f install.config
```

--------------------------------

### LS_TOP Parameter Configuration Example

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=files-setupconfig

Illustrates the configuration of the LS_TOP parameter in setup.config. This parameter defines the absolute path to the top-level License Scheduler installation directory, which must be a shared directory accessible by all License Scheduler hosts.

```shell
LS_TOP="/usr/share/ls"
```

--------------------------------

### Requesting GPU Resources with bsub -R

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=jobs-submitting-that-require-gpu-resources

This example shows how to specify GPU resource requirements using the `bsub -R` command, including the number of physical GPUs, GPU model with memory, reserved GPU memory, and specific connection types like NVLink or xGMI.

```bash
bsub -R "span[gtile=N | '!'] rusage[ngpus_physical=X:gmodel=model[-mem]:gmem=M:glink=yes]"
```

--------------------------------

### LS_ADMIN Parameter Configuration Example

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=files-setupconfig

Demonstrates how to configure the LS_ADMIN parameter in setup.config to specify License Scheduler administrator accounts. The first user in the list is the primary administrator. User accounts must exist on all License Scheduler hosts prior to installation.

```shell
LS_ADMINS="lsadmin user1 user2"
```

--------------------------------

### Example: Percentage-based License Distribution

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=files-lsflicensescheduler

An example demonstrating the use of LS_FEATURE_PERCENTAGE=Y to configure license distribution where one project (p3) is always entitled to a specific percentage (20%) of total licenses and can preempt others if needed.

```configuration
Begin Feature
LS_FEATURE_PERCENTAGE = Y
DISTRIBUTION = LanServer (p1 1 p2 1 p3 1/20)
...
End Feature
```

--------------------------------

### Invoke LSF Job Starter in Windows

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=variables-environment-variable-reference

Illustrates how the LSF job starter is invoked on Windows, passing commands as arguments. It includes an example of setting the LSF_JOB_STARTER environment variable for DOS shell jobs.

```batch
LSF_JOB_STARTER command [argument...]

```

```batch
set LSF_JOB_STARTER=C:\cmd.exe /C
C:\> lsrun dir /p
C:\cmd.exe /C dir /p

```

--------------------------------

### Example user_data.sh for LSF Instance Configuration

Source: https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=reference-awsprov-templatesjson

This bash script is executed during instance startup. It reads environment variables like 'zone' and updates the LSF configuration file (lsf.conf) to define new resources and resource maps. This allows LSF to recognize and utilize instance-specific attributes like the zone.

```bash
#!/bin/bash
LSF_TOP=/usr/share/lsf
LSF_CONF_FILE=$LSF_TOP/conf/lsf.conf 

# run user script to enable selecting template based on zone
%EXPORT_USER_DATA%

logfile=/tmp/userscript.log
env > $logfile
if [ -n "${zone}" ]; then
    sed -i "s/(LSF_LOCAL_RESOURCES=.*)\"/1 [resource ${zone}] 
       [resourcemap ${zone}*zone]"/" $LSF_CONF_FILE
    echo "update LSF_LOCAL_RESOURCES lsf.conf successfully, 
       add [resource ${zone}] [resourcemap ${zone}*zone]" >> $logfile
else
    echo "zone doesn't exist in environnment variable" >> $logfile
fi

```