PySlurm

https://pyslurm.github.io/24.11
Admin
PySlurm is the Python client library for the Slurm Workload Manager, enabling programmatic...
Tokens:30,388
Snippets:185
Trust Score:9.7
Update:6 months ago
Show doc for...
Tokens:
Raw
### Configure Node Requirements

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Examples of specifying node counts using dictionaries, integers, or range strings.

```python
nodes = {
    "min": 3,
    "max": 6
}
```

```python
nodes = 3
```

```python
nodes = "1-5"
```

--------------------------------

### Configure CPU Frequency

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Examples of setting CPU frequency using dictionaries, strings, or numeric values.

```python
cpu_frequency = {
    "min": "Low",
    "max": "High",
    "governor": "UserSpace"
}
```

```python
cpu_frequency = "Performance"
or
cpu_frequency = {"governor": "Performance"}
```

```python
cpu_frequency = <frequency in kilohertz>
or either
cpu_frequency = {"max": <freq>} or cpu_freq = {"min": <freq>}
```

--------------------------------

### Install pyslurm using pip

Source: https://pyslurm.github.io/24.11

Install pyslurm directly using pip after cloning the repository. This method is a straightforward way to install the package.

```bash
pip install .
```

--------------------------------

### Set Slurm Installation Paths

Source: https://pyslurm.github.io/24.11

Specify custom include and library directories for Slurm installations when they are not in default locations. This is recommended before installing pyslurm.

```bash
export SLURM_INCLUDE_DIR=/opt/slurm/24.11/include
export SLURM_LIB_DIR=/opt/slurm/24.11/lib
```

--------------------------------

### Clone and Build pyslurm

Source: https://pyslurm.github.io/24.11

Clone the pyslurm repository and use the provided build script to install the library. Ensure Slurm header files and shared libraries are accessible.

```bash
git clone https://github.com/PySlurm/pyslurm.git && cd pyslurm
scripts/build.sh
```

--------------------------------

### Get collection values by key

Source: https://pyslurm.github.io/24.11/reference/xcollections

Retrieve values using a key or a (cluster, key) tuple.

```python
>>> job_id = 1
>>> job = data.get(job_id)
```

```python
>>> job_id = 1
>>> job = data.get(("REMOTE_CLUSTER", job_id))
```

--------------------------------

### GET pyslurm.deprecated.hostlist.get_list

Source: https://pyslurm.github.io/24.11/reference/hostlist

Retrieves the list of hostnames from a hostlist object.

```APIDOC
## get_list()

### Description
Get the list of hostnames composing the hostlist. For example, a hostlist created with "tux[1-3]" will return ['tux1', 'tux2', 'tux3'].

### Method
Method Descriptor

### Response
#### Success Response
- **list** (list) - The list of hostnames in case of success or None on error.
```

--------------------------------

### Get Job Information as Dictionary

Source: https://pyslurm.github.io/24.11/reference/job

The `to_dict` method retrieves all job information and formats it into a Python dictionary for easy access and manipulation.

```python
>>> # Example usage would typically involve creating a Job object first
>>> # job_info = pyslurm.Job(9999).to_dict()
```

--------------------------------

### Frontend API - Get Node Information

Source: https://pyslurm.github.io/24.11/reference/frontend

Retrieves information about the slurm front end nodes. The data is returned as a dictionary where keys are Topology IDs.

```APIDOC
## GET /frontend/get

### Description
Get front end node information.

### Method
GET

### Endpoint
/frontend/get

### Response
#### Success Response (200)
- **dict** - Dictionary whose key is the Topology ID
```

--------------------------------

### MultiClusterMap Data Structure Example

Source: https://pyslurm.github.io/24.11/reference/xcollections

Illustrates the internal data structure of a MultiClusterMap, showing how data from different clusters is organized. This structure is used by collections like pyslurm.db.Jobs.

```python
data = {
    "LOCAL_CLUSTER": {
        1: pyslurm.db.Job(1),
        2: pyslurm.db.Job(2),
        ...
    },
    "OTHER_REMOTE_CLUSTER": {
        100: pyslurm.db.Job(100),
        101, pyslurm.db.Job(101)
        ...
    },
    ...
}

```

--------------------------------

### Create a Partition

Source: https://pyslurm.github.io/24.11/reference/partition

Initializes a new partition instance and commits it to the Slurm controller.

```python
>>> import pyslurm
>>> part = pyslurm.Partition("debug").create()
```

--------------------------------

### Create a Node

Source: https://pyslurm.github.io/24.11/reference/node

Use the `create` method to add a new node to the system. The `state` parameter can be set to 'future' or 'cloud'.

```python
import pyslurm
node = pyslurm.Node("testnode").create()
```

--------------------------------

### Get Slurm Triggers

Source: https://pyslurm.github.io/24.11/reference/trigger

Retrieves information regarding all current Slurm triggers.

```APIDOC
## get()

### Description
Get the information on slurm triggers.

### Response
#### Success Response (200)
- **dict** - Dictionary, where keys are the trigger IDs
```

--------------------------------

### Open a PySlurm Database Connection

Source: https://pyslurm.github.io/24.11/reference/db/connection

Use the static `open()` method to establish a connection to slurmdbd. Ensure the connection is successfully opened before proceeding.

```python
>>> import pyslurm
>>> connection = pyslurm.db.Connection.open()
>>> print(connection.is_open)
True
```

--------------------------------

### Get Resource Layout API

Source: https://pyslurm.github.io/24.11/reference/job

API to retrieve the resource layout of a Slurm job on each node.

```APIDOC
## GET /job/{job_id}/resources

### Description
Retrieve the resource layout of this Job on each node.

### Method
GET

### Endpoint
`/job/{job_id}/resources`

### Parameters
#### Path Parameters
- **job_id** (int) - Required - The ID of the job whose resource layout to retrieve.

### Response
#### Success Response (200)
- **resource_layout** (dict) - Resource layout, where the key is the name of the node and the value another dict with the keys `cpu_ids`, `memory` and `gres`.

#### Response Example
```json
{
  "example": "{ \"node1\": { \"cpu_ids\": \"0-3\", \"memory\": \"4G\", \"gres\": \"gpu:2\" }, \"node2\": { \"cpu_ids\": \"4-7\", \"memory\": \"4G\", \"gres\": \"gpu:2\" } }"
}
```

### Warnings
- Return type may still be subject to change in the future.
```

--------------------------------

### Load a Partition

Source: https://pyslurm.github.io/24.11/reference/partition

Retrieves information for a specific partition by name.

```python
>>> import pyslurm
>>> part = pyslurm.Partition.load("normal")
```

--------------------------------

### Get Batch Script API

Source: https://pyslurm.github.io/24.11/reference/job

API to retrieve the content of the batch script for a Slurm batch job.

```APIDOC
## GET /job/{job_id}/script

### Description
Return the content of the script for a Batch-Job.

### Method
GET

### Endpoint
`/job/{job_id}/script`

### Parameters
#### Path Parameters
- **job_id** (int) - Required - The ID of the job whose batch script to retrieve.

### Request Example
```json
{
  "example": "import pyslurm\nscript = pyslurm.Job(9999).get_batch_script()"
}
```

### Response
#### Success Response (200)
- **script_content** (str) - The content of the batch script.

#### Response Example
```json
{
  "example": "#!/bin/bash\n#SBATCH --job-name=my_job\necho \"Hello, World!\""
}
```

### Errors
- **RPCError**: When retrieving the Batch-Script for the Job was not successful.
```

--------------------------------

### Submit a batch job

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Initialize a JobSubmitDescription and call submit to send the job to the scheduler.

```python
>>> import pyslurm
>>> desc = pyslurm.JobSubmitDescription(
...     name="test-job",
...     cpus_per_task=1,
...     time_limit="10-00:00:00",
...     script="/path/to/your/submit_script.sh")
>>>
>>> job_id = desc.submit()
>>> print(job_id)
99
```

--------------------------------

### Convert Node to Dictionary

Source: https://pyslurm.github.io/24.11/reference/node

Use the `to_dict` method to get the node's information as a Python dictionary.

```python
import pyslurm
my_node = pyslurm.Node.load("mynode")
my_node_dict = my_node.to_dict()
```

--------------------------------

### Partition Management Methods

Source: https://pyslurm.github.io/24.11/reference/partition

Methods for creating, deleting, loading, and modifying individual Slurm partitions.

```APIDOC
## create()

### Description
Create a new Partition by implementing the slurm_create_partition RPC.

### Returns
- **Partition** - The current Partition instance object.

### Raises
- **RPCError** - If creating the Partition was not successful.

## delete()

### Description
Delete a Partition by implementing the slurm_delete_partition RPC.

### Raises
- **RPCError** - When deleting the Partition was not successful.

## load(name)

### Description
Load information for a specific Partition.

### Parameters
#### Path Parameters
- **name** (str) - Required - The name of the Partition to load.

### Returns
- **Partition** - Returns a new Partition instance.

### Raises
- **RPCError** - If requesting the Partition information from the slurmctld was not successful.

## modify(changes)

### Description
Modify a Partition by implementing the slurm_update_partition RPC.

### Parameters
- **changes** (Partition) - Required - Another Partition object that contains all the changes to apply.

### Raises
- **RPCError** - When updating the Partition was not successful.
```

--------------------------------

### Load job configuration from environment

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Use load_environment to apply previously set environment variables to a JobSubmitDescription instance.

```python
>>> import pyslurm
>>> desc = pyslurm.JobSubmitDescription(...other args...)
>>> desc.load_environment()
>>> print(desc.name, desc.account, desc.is_requeueable)
myjobname, myaccount, False
```

--------------------------------

### Configure gpus_per_task

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Defines GPUs per task using dictionary, string, or integer formats. Requires ntasks or gpus to be set.

```python
gpus_per_task = {
    "tesla": 1,
    "volta": 5,
}
```

```python
gpus_per_task = "tesla:1,volta:5"
```

```python
gpus_per_task = 6
```

--------------------------------

### Frontend API - Get Node IDs

Source: https://pyslurm.github.io/24.11/reference/frontend

Returns the node IDs from the retrieved front end node data.

```APIDOC
## GET /frontend/ids

### Description
Return the node IDs from retrieved data.

### Method
GET

### Endpoint
/frontend/ids

### Response
#### Success Response (200)
- **dict** - Dictionary of node IDs
```

--------------------------------

### Load All Nodes

Source: https://pyslurm.github.io/24.11/reference/node

Use the static `load` method on the `Nodes` class to retrieve a collection of all nodes in the system. The `preload_passwd_info` parameter can optionally fetch user and group information.

```python
import pyslurm
nodes = pyslurm.Nodes.load()
```

--------------------------------

### Load All Job Steps

Source: https://pyslurm.github.io/24.11/reference/jobstep

Loads all job steps across the entire system. Returns a dictionary mapping Job IDs to their respective JobSteps objects.

```python
>>> import pyslurm
>>> all_steps = pyslurm.JobSteps.load_all()
```

--------------------------------

### Frontend API - Get Last Update Time

Source: https://pyslurm.github.io/24.11/reference/frontend

Retrieves the last update time for the front end node data in epoch seconds.

```APIDOC
## GET /frontend/lastUpdate

### Description
Return last time (sepoch seconds) the node data was updated.

### Method
GET

### Endpoint
/frontend/lastUpdate

### Response
#### Success Response (200)
- **int** - Epoch seconds
```

--------------------------------

### Configure GPU Allocation (Total)

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Specifies the total number of GPUs for the job. Can be a dictionary for specific types, a string, or an integer for any type. Corresponds to -G/--gpus in sbatch.

```python
gpus = {
    "tesla": 1,
    "volta": 5,
}
```

```python
gpus = "tesla:1,volta:5"
```

```python
gpus = 6
```

--------------------------------

### Adding a Job to a PySlurm Jobs Collection

Source: https://pyslurm.github.io/24.11/reference/xcollections

Example of adding a pyslurm.db.Job instance to a pyslurm.db.Jobs collection, which inherits from MultiClusterMap. Ensures the item added is of the correct type.

```python
>>> import pyslurm
>>> jobs = pyslurm.db.Jobs()
>>> job = pyslurm.db.Job(1)
>>> jobs.add(job)
>>> print(jobs)
pyslurm.db.Jobs({1: pyslurm.db.Job(1)})

```

--------------------------------

### Topology API Methods

Source: https://pyslurm.github.io/24.11/reference/topology

Methods for interacting with the Slurm topology configuration.

```APIDOC
## pyslurm.deprecated.topology

### Description
Access and update Slurm topology information. Note: This API is currently being reworked and is subject to removal.

### Methods

#### display()
Display topology information to standard output.

#### get()
Get Slurm topology information.
- **Returns**: (dict) Dictionary where the key is the Topology ID.

#### lastUpdate()
Get the time the retrieved data was updated.
- **Returns**: (int) Epoch seconds.

#### load()
Load Slurm topology information.
```

--------------------------------

### Get Batch Script Content with pyslurm

Source: https://pyslurm.github.io/24.11/reference/job

Retrieve the content of a batch script for a Slurm job using `get_batch_script()`. This method may raise an RPCError if the script cannot be retrieved.

```python
>>> import pyslurm
>>> script = pyslurm.Job(9999).get_batch_script()
```

--------------------------------

### Job Configuration Parameters

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

A collection of configuration parameters used to define job requirements, resource constraints, and execution behavior, mirroring standard sbatch functionality.

```APIDOC
## Job Configuration Parameters

### Description
These parameters define the resource requirements and execution environment for a Slurm job. They map directly to standard sbatch command-line arguments.

### Parameters
- **gpus_per_task** (Union[dict, str, int]) - Optional - GPUs to be allocated per task.
- **gres_per_node** (Union[dict, str]) - Optional - Generic resources to be allocated per node.
- **gpu_binding** (str) - Optional - GPU binding for the job.
- **ntasks** (int) - Optional - Maximum amount of tasks for the job.
- **ntasks_per_node** (int) - Optional - Amount of tasks to be invoked on each node.
- **ntasks_per_socket** (int) - Optional - Maximum amount of tasks to be invoked on each socket.
- **ntasks_per_core** (int) - Optional - Maximum amount of tasks to be invoked on each core.
- **ntasks_per_gpu** (int) - Optional - Amount of tasks to be invoked per GPU.
- **switches** (Union[dict, str, int]) - Optional - Maximum amount of leaf switches and wait time.
- **signal** (Union[dict, str]) - Optional - Warn signal to be sent to the job.
- **standard_in** (str) - Optional - Path to file for standard input.
- **standard_error** (str) - Optional - Path to file for standard error.
- **standard_output** (str) - Optional - Path to file for standard output.
- **kill_on_invalid_dependency** (bool) - Optional - Kill job if dependency is invalid.
- **spreads_over_nodes** (bool) - Optional - Spread job over as many nodes as possible.
- **use_min_nodes** (bool) - Optional - Prefer the minimum amount of nodes specified.
- **gres_binding** (str) - Optional - Generic resource task binding (enforce-binding, disable-binding).
- **gres_tasks_per_sharing** (str) - Optional - Shared GRES tasks (multiple, one).
- **temporary_disk_per_node** (Union[str, int]) - Optional - Temporary disk space per node.
- **min_cpus_per_node** (str) - Optional - Minimum CPUs required per node.
- **wait_all_nodes** (bool) - Optional - Wait for all nodes to be ready before execution.
```

--------------------------------

### Configure temporary_disk_per_node

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Specifies temporary disk space per node in MiB or using unit suffixes.

```python
# 2048 MiB
tmp_disk_per_node = "2G"

# 1024 MiB
tmp_disk_per_node = 1024
```

--------------------------------

### Configure switches

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Sets maximum leaf switches and optional wait time.

```python
switches = { "count": 5, "max_wait_time": "00:10:00" }
```

```python
switches = "5@00:10:00"
```

--------------------------------

### Load a Job Step

Source: https://pyslurm.github.io/24.11/reference/jobstep

Retrieves information for a specific job step using the slurm_get_job_steps RPC.

```python
>>> import pyslurm
>>> jobstep = pyslurm.JobStep.load(9999, 1)
```

--------------------------------

### Job Scheduling and Execution Configuration

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Configure job scheduling, execution, and runtime parameters.

```APIDOC
## Job Scheduling and Execution Configuration

### Description
This section covers parameters related to job scheduling, execution, and runtime behavior.

### Parameters

#### Scheduling
- **network** (str) - Required - Network types for the Job.
- **nice** (int) - Required - Adjusted scheduling priority for the Job.
- **overcommit** (bool) - Required - Whether resources should be overcommitted.
- **partitions** (Union[list, str]) - Required - List of partitions the Job may use.
- **qos** (str) - Required - Quality of Service for the Job.
- **requires_node_reboot** (bool) - Required - Force allocated nodes to reboot before the job starts.
- **is_requeueable** (bool) - Required - Whether the Job is eligible for requeuing.
- **reservations** (Union[list, str]) - Required - List of possible reservations the Job can use.
- **resource_sharing** (str) - Required - Controls resource sharing. Allowed values: `oversubscribe`, `user`, `mcs`, `no` or `exclusive`.

#### Execution
- **script** (str) - Required - Path or content of the batch script. Script arguments are not supported if content is provided as a string.
- **script_args** (str) - Required - Arguments passed to the batch script. Only applicable if `script` is a file path.
- **container** (str) - Required - Path to an OCI container bundle.

#### Runtime
- **log_files_open_mode** (str) - Required - Mode for opening log files. Valid options: `append`, `truncate`.
- **accounting_gather_frequency** (Union[dict, str]) - Required - Interval for accounting info gathering. Example: `{"energy": 60, "network": 20}` or `"energy=60,network=20"`.
- **environment** (Union[dict, str]) - Required - Environment variables to be set for the Job. Example: `{"VAR_NAME": "value"}` or `"VAR_NAME=value"`.
- **distribution** (str) - Required - Task distribution for the Job.
- **time_limit** (str) - Required - The time limit for the job.
- **time_limit_min** (str) - Required - A minimum time limit for the Job.

### Request Example
```json
{
  "network": "host",
  "nice": 10,
  "partitions": ["gpu", "debug"],
  "script": "/path/to/my/script.sh",
  "time_limit": "1:00:00"
}
```

### Response Example
```json
{
  "status": "scheduled",
  "message": "Job scheduled successfully."
}
```
```

--------------------------------

### Load Node Information

Source: https://pyslurm.github.io/24.11/reference/node

Use the static `load` method to retrieve information for a specific node by its name. This returns a new Node instance.

```python
import pyslurm
node = pyslurm.Node.load("localhost")
```

--------------------------------

### Job Resource Configuration

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Configure essential job resources such as memory, GPUs, and CPU allocation.

```APIDOC
## Job Resource Configuration

### Description
This section details parameters for configuring job resources like memory, GPUs, and CPU allocation.

### Parameters

#### Memory
- **memory_per_gpu** (str) - Required - Memory per GPU (e.g., "3G").

#### CPUs
- **cpus_per_task** (int) - Required - CPUs per task. Mutually exclusive with `cpus_per_gpu`.
- **cpus_per_gpu** (int) - Required - CPUs per allocated GPU. Mutually exclusive with `cpus_per_task`.

#### Sockets and Cores
- **sockets_per_node** (int) - Required - Minimum sockets per node.
- **cores_per_socket** (int) - Required - Minimum cores per socket.
- **threads_per_core** (int) - Required - Minimum threads per core.

#### GPUs
- **gpus** (Union[dict, str, int]) - Required - Total GPUs for the job. Can be a count, a dictionary of GPU types and counts (e.g., `{"tesla": 1, "volta": 5}`), or a string (e.g., `"tesla:1,volta:5"`).
- **gpus_per_socket** (Union[dict, str, int]) - Required - GPUs per socket. Requires `sockets_per_node` to be set. Accepts same formats as `gpus`.

### Request Example
```json
{
  "memory_per_gpu": "3G",
  "cpus_per_task": 4,
  "gpus": 2,
  "sockets_per_node": 1
}
```

### Response Example
```json
{
  "status": "configured",
  "message": "Job resources configured successfully."
}
```
```

--------------------------------

### Load Job Steps

Source: https://pyslurm.github.io/24.11/reference/job

Load all Job steps for a collection of Jobs. This method populates the `steps` attribute for each Job. Pending Jobs are ignored.

```python
# This function fills in the `steps` attribute for all Jobs in the collection.
# Pending Jobs will be ignored, since they don't have any Steps yet.
```

--------------------------------

### Configure GPUs Per Socket

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Specifies the number of GPUs to allocate per socket. Requires sockets_per_node to be set. Can be a dictionary for specific types, a string, or an integer. Corresponds to --gpus-per-socket in sbatch.

```python
gpus_per_socket = {
    "tesla": 1,
    "volta": 5,
}
```

```python
gpus_per_socket = "tesla:1,volta:5"
```

```python
gpus_per_socket = 6
```

--------------------------------

### pyslurm.JobSteps Methods

Source: https://pyslurm.github.io/24.11/reference/jobstep

Static methods for loading JobStep collections from the system.

```APIDOC
## load(job)

### Description
Load the Job Steps from the system for a specific job.

### Parameters
- **job** (Union[Job, int]) - Required - The Job for which the Steps should be loaded.

### Returns
- **JobSteps** - JobSteps of the Job.

## load_all()

### Description
Loads all the steps in the system.

### Returns
- **dict** - A dict where every JobID (key) is mapped with an instance of its JobSteps (value).
```

--------------------------------

### JobStep Methods

Source: https://pyslurm.github.io/24.11/reference/jobstep

Methods available for interacting with individual JobStep instances, including loading stats, modifying properties, and sending signals.

```APIDOC
## load_stats()

### Description
Load realtime stats for this Step. Returns live statistics and populates the stats and pids attributes of the instance.

### Returns
- **JobStepStatistics** - The statistics of the Step.

### Raises
- **RPCError** - When retrieving the stats for the Step failed.

## modify(changes)

### Description
Modify a job step by applying changes from another JobStep object. Implements the slurm_update_step RPC.

### Parameters
- **changes** (JobStep) - Required - Another JobStep object containing properties to modify.

### Raises
- **RPCError** - When updating the JobStep was not successful.

## send_signal(signal)

### Description
Send a signal to a running Job step. Implements the slurm_signal_job_step RPC.

### Parameters
- **signal** (Union[str, int]) - Required - Any valid signal (e.g., 'SIGUSR1' or 9).

### Raises
- **RPCError** - When sending the signal was not successful.
```

--------------------------------

### Partitions Collection Methods

Source: https://pyslurm.github.io/24.11/reference/partition

Methods for managing collections of Slurm partitions.

```APIDOC
## load()

### Description
Load all Partitions in the system.

### Returns
- **Partitions** - Collection of Partition objects.

### Raises
- **RPCError** - When getting all the Partitions from the slurmctld failed.

## modify(changes)

### Description
Modify all Partitions in a Collection.

### Parameters
- **changes** (Partition) - Required - Another Partition object that contains all the changes to apply.

### Raises
- **RPCError** - When updating at least one Partition failed.

## reload()

### Description
Reload the information for Partitions already in the collection.

### Returns
- **Partitions** - Returns self.

### Raises
- **RPCError** - When getting the Partitions from the slurmctld failed.
```

--------------------------------

### load(job_id, step_id)

Source: https://pyslurm.github.io/24.11/reference/jobstep

Loads information for a specific job step by implementing the slurm_get_job_steps RPC.

```APIDOC
## load(job_id, step_id)

### Description
Load information for a specific job step. Implements the slurm_get_job_steps RPC.

### Method
Static Method

### Parameters
#### Path Parameters
- **job_id** (Union[Job, int]) - Required - ID of the Job the Step belongs to.
- **step_id** (Union[int, str]) - Required - Step-ID for the Step to be loaded.

### Response
- **JobStep** - Returns a new JobStep instance

### Errors
- **RPCError** - When retrieving Step information from the slurmctld was not successful.

### Request Example
```python
import pyslurm
jobstep = pyslurm.JobStep.load(9999, 1)
```
```

--------------------------------

### submit

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Submits a batch job description to the SLURM system.

```APIDOC
## submit()

### Description
Submit a batch job description.

### Method
POST

### Endpoint
/

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
>>> import pyslurm
>>> desc = pyslurm.JobSubmitDescription(
...     name="test-job",
...     cpus_per_task=1,
...     time_limit="10-00:00:00",
...     script="/path/to/your/submit_script.sh")
>>> 
>>> job_id = desc.submit()
>>> print(job_id)
99
```

### Response
#### Success Response (200)
- **int** - The ID of the submitted Job.

#### Response Example
```json
99
```

### Error Handling
#### Raises
- **RPCError** - When the job submission was not successful.
```

--------------------------------

### Jobs.load_steps()

Source: https://pyslurm.github.io/24.11/reference/job

Loads all Job steps for a collection of Jobs. This method fills in the `steps` attribute for all Jobs in the collection.

```APIDOC
## Jobs.load_steps()

### Description
Load all Job steps for this collection of Jobs. This function fills in the `steps` attribute for all Jobs in the collection.

Note: Pending Jobs will be ignored, since they don't have any Steps yet.

### Method
`method descriptor`

### Parameters
None

### Raises
- **RPCError** - When retrieving the information for all the Steps failed.

### Request Example
```python
import pyslurm
jobs = pyslurm.Jobs.load()
jobs.load_steps()
# Now each job in the 'jobs' collection has a 'steps' attribute
```

### Response Example
```json
{
  "example": "No explicit response body, modifies existing job objects"
}
```
```

--------------------------------

### Load a Job from the Database

Source: https://pyslurm.github.io/24.11/reference/db/job

Load a specific job by ID. Note that script and environment data are not populated by default and must be explicitly requested.

```python
>>> import pyslurm
>>> db_job = pyslurm.db.Job.load(10000)
```

```python
>>> import pyslurm
>>> db_job = pyslurm.db.Job.load(10000, with_script=True)
>>> print(db_job.script)
```

--------------------------------

### Load all Jobs

Source: https://pyslurm.github.io/24.11/reference/job

Use this static method to retrieve all Jobs from the Slurm controller. Set `preload_passwd_info` to True to speed up attribute access by fetching user and group information.

```python
>>> import pyslurm
>>> jobs = pyslurm.Jobs.load()
>>> print(jobs)
pyslurm.Jobs({1: pyslurm.Job(1), 2: pyslurm.Job(2)})
>>> print(jobs[1])
pyslurm.Job(1)
```

--------------------------------

### load_sbatch_options

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Loads job attributes from #SBATCH options within a batch script.

```APIDOC
## load_sbatch_options(overwrite=False)

### Description
Load values from `#SBATCH` options in the batch script.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **overwrite** (bool) - Optional - If set to `True`, the value from an option found in the batch script will override the current value of the attribute in this instance. Default is `False`

### Request Example
None

### Response
#### Success Response (200)
None

#### Response Example
None
```

--------------------------------

### Configure signal

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Defines warning signals sent to the job before the time limit.

```python
signal = {
    "signal": "SIGKILL",
    "time": 120
}
```

```python
signal = "SIGKILL@120"
```

--------------------------------

### Partition Management API

Source: https://pyslurm.github.io/24.11/reference/partition

This section covers the core functionalities for managing individual partitions within PySlurm.

```APIDOC
## Partition

### Description
Manages individual Slurm partitions. This supersedes the `pyslurm.partition` class.

### Methods

#### `create`

##### Description
Creates a new partition.

##### Endpoint
`/partition/create` (Conceptual - actual implementation details may vary)

##### Parameters
* **partition_name** (string) - Required - The name of the partition to create.
* **settings** (dict) - Optional - A dictionary of settings for the new partition.

#### `delete`

##### Description
Deletes an existing partition.

##### Endpoint
`/partition/delete` (Conceptual - actual implementation details may vary)

##### Parameters
* **partition_name** (string) - Required - The name of the partition to delete.

#### `load`

##### Description
Loads partition information.

##### Endpoint
`/partition/load` (Conceptual - actual implementation details may vary)

##### Parameters
* **partition_name** (string) - Required - The name of the partition to load.

#### `modify`

##### Description
Modifies an existing partition.

##### Endpoint
`/partition/modify` (Conceptual - actual implementation details may vary)

##### Parameters
* **partition_name** (string) - Required - The name of the partition to modify.
* **settings** (dict) - Optional - A dictionary of settings to update.

#### `to_dict`

##### Description
Converts partition information to a dictionary.

##### Endpoint
`/partition/to_dict` (Conceptual - actual implementation details may vary)

##### Parameters
* **partition_name** (string) - Required - The name of the partition to convert.

### Response
* **Success Response (200)**: Returns details of the partition or a success status.
* **Error Response**: Indicates an error during the operation.
```

--------------------------------

### Job Configuration Schema

Source: https://pyslurm.github.io/24.11/reference/job

A comprehensive list of parameters used to define job requirements and environment settings.

```APIDOC
## Job Configuration Parameters

### Description
This schema defines the available fields for configuring a Slurm job, covering hardware requirements, resource limits, and job metadata.

### Parameters
- **sockets_per_board** (int) - Number of sockets per board.
- **sockets_per_node** (int) - Number of sockets per node.
- **cores_per_socket** (int) - Number of cores per socket.
- **threads_per_core** (int) - Number of threads per core.
- **ntasks** (int) - Number of parallel processes.
- **ntasks_per_node** (int) - Number of parallel processes per node.
- **ntasks_per_board** (int) - Number of parallel processes per board.
- **ntasks_per_socket** (int) - Number of parallel processes per socket.
- **ntasks_per_core** (int) - Number of parallel processes per core.
- **ntasks_per_gpu** (int) - Number of parallel processes per GPU.
- **delay_boot_time** (int) - Delay in minutes before booting.
- **constraints** (list) - Features the Job requires nodes to have.
- **cluster** (str) - Name of the cluster.
- **cluster_constraints** (list) - Features that a cluster should have.
- **reservation** (str) - Name of the reservation.
- **resource_sharing** (str) - Mode controlling resource sharing.
- **requires_contiguous_nodes** (bool) - Whether the Job has allocated contiguous nodes.
- **licenses** (list) - List of licenses needed.
- **network** (str) - Network specification.
- **command** (str) - Command to execute.
- **working_directory** (str) - Path to the working directory.
- **admin_comment** (str) - Administrator comment.
- **system_comment** (str) - System comment.
- **container** (str) - Container used.
- **comment** (str) - User comment.
- **standard_input** (str) - Path to standard input file.
- **standard_output** (str) - Path to standard output file.
- **standard_error** (str) - Path to standard error file.
- **required_switches** (int) - Number of switches required.
- **max_wait_time_switches** (int) - Seconds to wait for switches.
- **burst_buffer** (str) - Burst buffer specification.
- **burst_buffer_state** (str) - Burst buffer state.
- **cpu_frequency_min** (Union[str, int]) - Minimum CPU-Frequency.
- **cpu_frequency_max** (Union[str, int]) - Maximum CPU-Frequency.
- **cpu_frequency_governor** (Union[str, int]) - CPU-Frequency Governor.
- **billable_tres** (float) - Amount of billable trackable resources.
- **wckey** (str) - Name of the WCKey.
- **mail_user** (list) - Users to receive emails.
- **mail_types** (list) - Mail flags.
- **heterogeneous_id** (int) - Heterogeneous job id.
- **heterogeneous_offset** (int) - Heterogeneous job offset.
- **temporary_disk_per_node** (int) - Temporary disk space in Mebibytes per node.
- **array_id** (int) - Master Array-Job ID.
- **array_tasks_parallel** (int) - Max simultaneous array tasks.
- **array_task_id** (int) - Array Task ID.
- **array_tasks_waiting** (str) - Waiting array tasks.
- **end_time** (int) - Unix timestamp for end time.
- **run_time** (int) - Seconds running.
- **cores_reserved_for_system** (int) - Cores reserved for system.
- **threads_reserved_for_system** (int) - Threads reserved for system.
- **memory** (int) - Total memory in Mebibytes.
- **memory_per_cpu** (int) - Memory per CPU in Mebibytes.
- **memory_per_node** (int) - Memory per node in Mebibytes.
- **memory_per_gpu** (int) - Memory per GPU in Mebibytes.
- **gres_per_node** (dict) - Generic resources per node.
- **profile_types** (list) - Types for accounting data.
- **gres_binding** (str) - Binding enforcement of generic resource.
- **gres_tasks_per_sharing** (str) - Task sharing of generic resource.
- **kill_on_invalid_dependency** (bool) - Kill on invalid dependency.
- **spreads_over_nodes** (bool) - Spread over nodes.
- **is_cronjob** (bool) - Whether it is a cronjob.
- **cronjob_time** (str) - Cronjob time specification.
```

--------------------------------

### Node Class Methods

Source: https://pyslurm.github.io/24.11/reference/node

Methods for managing individual nodes.

```APIDOC
## POST /node/create

### Description
Create a node with an optional initial state.

### Method
POST

### Endpoint
/node/create

### Parameters
#### Query Parameters
- **state** (str) - Optional - An optional state the created Node should have. Allowed values are `future` and `cloud`. `future` is the default.

### Response
#### Success Response (200)
- **Node** (Node) - The newly created Node instance.

#### Error Response (400)
- **RPCError** - If creating the Node was not successful.

## DELETE /node/{name}

### Description
Delete a node.

### Method
DELETE

### Endpoint
/node/{name}

### Parameters
#### Path Parameters
- **name** (str) - Required - The name of the node to delete.

#### Error Response (400)
- **RPCError** - If deleting the Node was not successful.

## GET /node/{name}

### Description
Load information for a specific node.

### Method
GET

### Endpoint
/node/{name}

### Parameters
#### Path Parameters
- **name** (str) - Required - The name of the Node to load.

### Response
#### Success Response (200)
- **Node** (Node) - Returns a new Node instance with the loaded information.

#### Error Response (400)
- **RPCError** - If requesting the Node information from the slurmctld was not successful.

## PUT /node/{name}

### Description
Modify a node with specified changes.

### Method
PUT

### Endpoint
/node/{name}

### Parameters
#### Path Parameters
- **name** (str) - Required - The name of the node to modify.
#### Request Body
- **changes** (Node) - Required - Another Node object that contains all the changes to apply. Check the `Other Parameters` of the Node class to see which properties can be modified.

#### Error Response (400)
- **RPCError** - When updating the Node was not successful.

## GET /node/{name}/to_dict

### Description
Get node information formatted as a dictionary.

### Method
GET

### Endpoint
/node/{name}/to_dict

### Parameters
#### Path Parameters
- **name** (str) - Required - The name of the node to retrieve.

### Response
#### Success Response (200)
- **dict** (dict) - Node information as a dictionary.

```

--------------------------------

### JobSubmitDescription Class Methods

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Documentation for the methods available within the JobSubmitDescription class.

```APIDOC
## JobSubmitDescription

### Description
Represents a description for submitting a Slurm job.

### Methods

#### ``load_environment``

##### Description
Loads environment variables for the job submission.

##### Method
(Assumed to be a method of the JobSubmitDescription class, specific HTTP method not applicable)

#### ``load_sbatch_options``

##### Description
Loads sbatch options for the job submission.

##### Method
(Assumed to be a method of the JobSubmitDescription class, specific HTTP method not applicable)

#### ``submit``

##### Description
Submits the job using the configured description.

##### Method
(Assumed to be a method of the JobSubmitDescription class, specific HTTP method not applicable)

### Parameters
(No specific parameters detailed in the provided text for these methods)

### Request Example
(No request examples provided in the source text)

### Response
(No response details provided in the source text)
```

--------------------------------

### Load Job Step Statistics

Source: https://pyslurm.github.io/24.11/reference/jobstep

Loads realtime statistics for a job step. Populates the `stats` and `pids` attributes of the instance. Use to access CPU time and process IDs.

```python
>>> import pyslurm
>>> step = pyslurm.JobStep.load(9999, 1)
>>> stats = step.load_stats()
>>>
>>> # Print the CPU Time Used
>>> print(stats.total_cpu_time)
>>>
>>> # Print the Process-IDs for the Step, organized by hostname
>>> print(step.pids)
```

--------------------------------

### Configure memory_per_node

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Sets memory requirements per whole node. Mutually exclusive with memory_per_cpu and memory_per_gpu.

```python
# 1 MiB
memory_per_node = 1024

# 3 GiB
memory_per_node = "3G"
```

--------------------------------

### Configure gres_per_node

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Specifies generic resources per node. Supports dictionary, string, or simple integer formats.

```python
gres_per_node = {
    "gpu:tesla": 1,
    "gpu:volta": 5,
}
```

```python
gres_per_node = "gpu:tesla:1,gpu:volta:5"
```

```python
gres_per_node = "gpu:6"
```

--------------------------------

### JobStep Object

Source: https://pyslurm.github.io/24.11/reference/db/jobstep

Represents a Slurm Database JobStep with various attributes detailing its state and resource utilization.

```APIDOC
## JobStep

### Description
A Slurm Database JobStep object containing utilization statistics, resource allocation, and state information.

### Attributes
- **stats** (`JobStepStatistics`) - Utilization statistics for this Step.
- **num_nodes** (`int`) - Amount of nodes this Step has allocated.
- **cpus** (`int`) - Amount of CPUs the Step has/had allocated.
- **memory** (`int`) - Amount of memory the Step requested.
- **container** (`str`) - Path to OCI Container bundle.
- **elapsed_time** (`int`) - Amount of seconds elapsed for the Step.
- **end_time** (`int`) - When the Step ended, as a unix timestamp.
- **eligible_time** (`int`) - When the Step became eligible to run, as a unix timestamp.
- **start_time** (`int`) - Time when the Step started, as a unix timestamp.
- **exit_code** (`int`) - Exit code of the step.
- **ntasks** (`int`) - Number of tasks the Step uses.
- **cpu_frequency_min** (`str`) - Minimum CPU-Frequency requested for the Step.
- **cpu_frequency_max** (`str`) - Maximum CPU-Frequency requested for the Step.
- **cpu_frequency_governor** (`str`) - CPU-Frequency Governor requested for the Step.
- **nodelist** (`str`) - Nodes this Step is using.
- **id** (`Union[str, int]`) - ID of the Step.
- **job_id** (`int`) - ID of the Job this Step is a part of.
- **state** (`str`) - State of the Step.
- **cancelled_by** (`str`) - Name of the User who cancelled this Step.
- **submit_command** (`str`) - Full command issued to start the Step.
- **suspended_time** (`int`) - Amount of seconds the Step was suspended.
```

--------------------------------

### Send Signal to Job Step (String)

Source: https://pyslurm.github.io/24.11/reference/jobstep

Sends a signal to a running job step using its string representation. Ensure the signal string is valid.

```python
>>> import pyslurm
>>> pyslurm.JobStep(9999, 1).send_signal("SIGUSR1")
```

--------------------------------

### Configure memory_per_gpu

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Sets memory requirements per GPU. Mutually exclusive with memory_per_node and memory_per_cpu.

```python
# 1 MiB
memory_per_gpu = 1024
```

--------------------------------

### Configure Memory Per GPU

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Sets the memory allocation for each GPU. This corresponds to the --memory-per-gpu option in sbatch.

```python
memory_per_gpu = "3G"
```

--------------------------------

### Memory Configuration Parameters

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Configuration parameters for defining memory requirements in PySlurm jobs.

```APIDOC
## Memory Configuration Parameters

### Description
These parameters define the memory requirements for a job. They are mutually exclusive, meaning only one of these can be set for a given job configuration.

### Parameters
- **memory_per_cpu** (Union[str, int]) - Optional - Memory required per allocated CPU. Default unit is Mebibytes. Supports suffixes K, M, G, T. Equivalent to --mem-per-cpu.
- **memory_per_node** (Union[str, int]) - Optional - Memory required per whole node. Default unit is Mebibytes. Supports suffixes K, M, G, T. Equivalent to --mem.
- **memory_per_gpu** (Union[str, int]) - Optional - Memory required per GPU. Default unit is Mebibytes. Supports suffixes K, M, G, T. Equivalent to --mem-per-gpu.

### Request Example
```python
# Example configurations
memory_per_cpu = "3G"
memory_per_node = 1024
memory_per_gpu = 2048
```
```

--------------------------------

### Configure memory_per_cpu

Source: https://pyslurm.github.io/24.11/reference/jobsubmitdescription

Sets memory requirements per allocated CPU. Mutually exclusive with memory_per_node and memory_per_gpu.

```python
# 1 MiB
memory_per_cpu = 1024

# 3 GiB
memory_per_cpu = "3G"
```

--------------------------------

### Convert Job to Dictionary

Source: https://pyslurm.github.io/24.11/reference/db/job

Export the database job information into a standard Python dictionary.

```python
>>> import pyslurm
>>> myjob = pyslurm.db.Job.load(10000)
>>> myjob_dict = myjob.to_dict()
```

--------------------------------

### Convert Job Step to Dictionary

Source: https://pyslurm.github.io/24.11/reference/jobstep

Converts JobStep information into a dictionary format. Useful for programmatic access to step details.

```python
>>> import pyslurm
>>> step = pyslurm.JobStep.load(9999, 1)
>>> step.to_dict()
```

--------------------------------

### pyslurm.constants

Source: https://pyslurm.github.io/24.11/reference/constants

Reference for common constants used in PySlurm.

```APIDOC
## pyslurm.constants

### Description
Provides access to common constants used across the PySlurm library.

### Attributes
- **UNLIMITED** (string) - Represents an infinite or unlimited value. This is returned for specific attributes to indicate that there is no restriction applied.
```

--------------------------------

### Jobs Class Methods

Source: https://pyslurm.github.io/24.11/reference/db/job

Methods for managing collections of Slurm jobs.

```APIDOC
## Jobs Class Methods

### Description
Provides methods for managing and querying collections of Slurm jobs.

### Methods
- `calc_stats()`
- `load()`
- `modify()`
```

--------------------------------

### send_signal(signal, steps='children', hurry=False)

Source: https://pyslurm.github.io/24.11/reference/job

Sends a signal to a running job by implementing the slurm_signal_job RPC.

```APIDOC
## POST /job/signal

### Description
Send a signal to a running Job. Implements the slurm_signal_job RPC.

### Parameters
#### Request Body
- **signal** (Union[str, int]) - Required - Any valid signal (e.g., 'SIGUSR1' or 9).
- **steps** (str) - Optional - Selects which steps should be signaled (all, batch, children). Default is 'children'.
- **hurry** (bool) - Optional - If True, no burst buffer data will be staged out. Default is False.

### Response
#### Error Handling
- **RPCError** - Raised when sending the signal was not successful.
```

--------------------------------

### Convert Partition to Dictionary

Source: https://pyslurm.github.io/24.11/reference/partition

Exports partition data into a dictionary format.

```python
>>> import pyslurm
>>> mypart = pyslurm.Partition.load("mypart")
>>> mypart_dict = mypart.to_dict()
```
Tokens:
Raw