StableSwarmUI

https://github.com/stability-ai/stableswarmui
Admin
A modular Stable Diffusion Web-User-Interface focused on high performance, extensibility, and easy...
Tokens:51,313
Snippets:142
Trust Score:9.3
License:MIT
Update:1 year ago
Show doc for...
Tokens:
Raw
### ControlNet Model and Preprocessor Setup

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/ControlNet.md

Instructions for setting up ControlNet models and installing necessary preprocessors for StableSwarmUI.

```APIDOC
Model Installation:
  1. Download ControlNet models (e.g., SDXL Canny, Depth LoRAs) from sources like Hugging Face.
  2. Save models to the `(Swarm)/Models/controlnet` directory.
  3. Refresh the ControlNet model list in the UI using the dedicated refresh icon.

Preprocessor Installation:
  1. Click the 'Install Controlnet Preprocessors' button in the ControlNet parameter group.
  2. Confirm the installation prompt.
  3. The backend will download, install, and restart. Monitor server logs for progress.

Using Pre-processed Images:
  - If using pre-processed images (e.g., OpenPose line art), select the 'None' preprocessor and the corresponding ControlNet model.
```

--------------------------------

### Install StableSwarmUI on Windows using install-windows.bat

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Download and run the `install-windows.bat` script to automate the installation of StableSwarmUI on Windows. The script handles setup, launches the server, and opens the web UI. Ensure Git and DotNET 8 are installed if on Windows 10.

```batch
@echo off
REM This script automates the installation of StableSwarmUI.
REM Ensure Git and DotNET 8 are installed on Windows 10.
REM Download the latest install-windows.bat from the releases page.
REM Store it in a desired installation directory (not Program Files).
REM Run the script.
REM It may need to be run twice if it closes prematurely.
REM A desktop icon will be created to relaunch the server.
REM Follow on-page instructions after the server launches.
```

--------------------------------

### Install .NET SDK and Cloudflared

Source: https://github.com/stability-ai/stableswarmui/blob/master/colab/colab-notebook.ipynb

Installs the .NET SDK versions 7.0 and 8.0, and the Cloudflared daemon for secure tunneling. These are essential dependencies for running StableSwarmUI.

```shell
# Install dotnet dependencies
!wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh
!chmod +x dotnet-install.sh
!./dotnet-install.sh --channel 7.0
!./dotnet-install.sh --channel 8.0

# Install Clouldflared
!wget https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb
!dpkg -i cloudflared-linux-amd64.deb
```

--------------------------------

### Manual Windows Install Steps

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Provides steps for a manual installation of StableSwarmUI on Windows, requiring pre-installation of Git and the DotNET 8 SDK.

```batch
REM Install Git from https://git-scm.com/download/win
REM Install DotNET 8 SDK (x64 for Windows) from https://dotnet.microsoft.com/en-us/download/dotnet/8.0

REM Open a terminal in the desired installation folder
REM git clone https://github.com/Stability-AI/StableSwarmUI

REM Navigate into the cloned folder
REM cd StableSwarmUI

REM Run the launch script
REM launch-windows.bat
```

--------------------------------

### Set SWARMPATH for Temporary Install

Source: https://github.com/stability-ai/stableswarmui/blob/master/colab/colab-notebook.ipynb

Configures the installation path for StableSwarmUI when using a temporary directory. This sets the SWARMPATH environment variable to '/content/'.

```python
SWARMPATH = '/content/'
```

--------------------------------

### Download and Execute Linux Installer Script

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Downloads the StableSwarmUI Linux installation script using wget, makes it executable with chmod, and then runs the script to automate the installation process.

```bash
wget https://github.com/Stability-AI/StableSwarmUI/releases/download/0.6.1-Beta/install-linux.sh -O install-linux.sh
chmod +x install-linux.sh
./install-linux.sh
```

--------------------------------

### Launch StableSwarmUI

Source: https://github.com/stability-ai/stableswarmui/blob/master/colab/colab-notebook.ipynb

Navigates to the StableSwarmUI directory, pulls the latest changes, and launches the application using a bash script. It specifies the launch mode and the path to the Cloudflared executable.

```bash
# Alright, launch it! Watch the output for a Cloudflare URL
%cd $SWARMPATH/StableSwarmUI/
!git pull

!bash ./launch-linux.sh --launch_mode none --cloudflared-path cloudflared
```

--------------------------------

### Manual Linux Install: Clone and Launch

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Clones the StableSwarmUI repository from GitHub, navigates into the directory, and executes the launch script for a manual installation process on Linux.

```bash
git clone https://github.com/Stability-AI/StableSwarmUI
cd StableSwarmUI
./launch-linux.sh
```

--------------------------------

### Configure Google Drive Install and Install Requirements

Source: https://github.com/stability-ai/stableswarmui/blob/master/colab/colab-notebook.ipynb

Mounts Google Drive for persistent storage and installs Python dependencies for the ComfyUI backend if a requirements file is found. This setup is for installations within Google Drive.

```python
from google.colab import drive
drive.mount('/content/drive')
SWARMPATH = '/content/drive/MyDrive'

# Colab breaks venvs, and doesn't save anything valid to drive, so just screw it do a global install of pip reqs if we used a comfy backend in the drive
!if [[ -f "$SWARMPATH/StableSwarmUI/dlbackend/ComfyUI/requirements.txt" ]]; then rm -rf $SWARMPATH/StableSwarmUI/dlbackend/ComfyUI/venv/; pip install -r $SWARMPATH/StableSwarmUI/dlbackend/ComfyUI/requirements.txt; fi
```

--------------------------------

### Clone and Launch StableSwarmUI on macOS

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Clones the StableSwarmUI repository, changes the directory, and executes the macOS launch script to begin the installation and setup process.

```bash
git clone https://github.com/Stability-AI/StableSwarmUI
cd StableSwarmUI
./launch-macos.sh
```

--------------------------------

### Backend Configuration Types

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Basic Usage.md

Lists common backend types and configurations for connecting additional GPUs or remote machines to StableSwarmUI for distributed image generation. Includes examples for local and remote setups.

```Configuration
Backend Type: Swarm-API-Backend
Description: For remote hosts. Requires StableSwarmUI setup on the remote machine with a working connection.
Configuration: Provide the URL to the remote backend.
```

```Configuration
Backend Type: ComfyUI Self-Starting
Description: For additional GPUs on the same machine.
Configuration: Use multiple instances, changing the GPU ID for each.
```

--------------------------------

### Launch StableSwarmUI on Linux

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Executes the launch script for StableSwarmUI on Linux systems after installation, typically used to start the application or re-launch it.

```bash
./launch-linux.sh
```

--------------------------------

### Prompt Syntax Examples

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Basic Usage.md

Demonstrates special syntax elements that can be used within prompts for enhanced control, such as random selection of options or image interpretation via ReVision.

```PromptSyntax
<random:cat, dog>
```

```PromptSyntax
Use ReVision by dragging an image into the prompt box.
```

--------------------------------

### API InstallConfirmWS Endpoint

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/BasicAPIFeatures.md

Confirms installation via WebSocket. Requires theme, installed_for, backend, stability_api_key, models, install_amd, and language.

```APIDOC
WebSocket Route /API/InstallConfirmWS

Description:
(ROUTE DESCRIPTION NOT SET)

Parameters:
| Name | Type | Description | Default |
| --- | --- | --- | --- |
| theme | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| installed_for | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| backend | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| stability_api_key | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| models | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| install_amd | Boolean | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| language | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |

Return Format:
```js
(RETURN INFO NOT SET)
```
```

--------------------------------

### Generate Image Example with Bash and Curl

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/API.md

Demonstrates how to obtain a session ID and generate an image using bash curl commands against the StableSwarmUI API. It includes steps for getting a session, making the generation request, and downloading the output.

```bash
# First, get a usable session ID:
curl -H "Content-Type: application/json" -d "{}" -X POST http://localhost:7801/API/GetNewSession
# EXAMPLE REPLY:
#        {"session_id":"9D3534E30DA38499DE782BC38211976A58555AA6","user_id":"local","output_append_user":true,"version":"0.6.3.0.GIT-5ee406ba","server_id":"058716b5-c6f5-49ed-9ca3-be20d82e4c5f","count_running":1}
# Copy the "session_id" value above to reuse

# Now, generate the image
curl -H "Content-Type: application/json" -d '{"session_id":"9D3534E30DA38499DE782BC38211976A58555AA6","images":1,"prompt":"a cat","model":"OfficialStableDiffusion/sd_xl_base_1.0","width":1024,"height":1024}' -X POST http://localhost:7801/API/GenerateText2Image
# EXAMPLE REPLY:
#        {"images":["View/local/raw/2024-05-19/a cat-OfficialStableDiffusionsd_xl_base_10s-1872258705.png"]}
# Copy the image URL

# Now download the image to look at it
wget "http://localhost:7801/View/local/raw/2024-05-19/a cat-OfficialStableDiffusionsd_xl_base_10s-1872258705.png"
```

--------------------------------

### Clone and Launch StableSwarmUI via Docker

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Clones the StableSwarmUI repository, navigates into the directory, and executes the Docker launch script to start the application within a containerized environment.

```bash
git clone https://github.com/Stability-AI/StableSwarmUI
cd StableSwarmUI
./launch-docker.sh
```

--------------------------------

### Set Environment Variables and Download StableSwarmUI

Source: https://github.com/stability-ai/stableswarmui/blob/master/colab/colab-notebook.ipynb

Sets the SWARMPATH environment variable and changes the current directory to the specified path. It also disables virtual environment creation and clones the StableSwarmUI repository from GitHub.

```python
# use the chosen path
import os
os.environ['SWARMPATH'] = SWARMPATH
%cd $SWARMPATH

# Colab breaks venv, so, tell swarm to not make a venv
os.environ['SWARM_NO_VENV'] = 'true'

# Download StableSwarmUI
url = "https://github.com/Stability" + "-AI/StableSwarmUI"
!git clone $url
```

--------------------------------

### Install Linux Dependencies (apt)

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Installs essential packages like git, python3, pip, and venv required for StableSwarmUI on Debian-based Linux distributions using the apt package manager.

```bash
sudo apt install git python3-pip python3-venv
```

--------------------------------

### StableSwarmUI API Overview and Key Routes

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/API.md

Provides an overview of the StableSwarmUI API, including general request formats, session management, error handling, and lists major API route categories. It details common POST and GET endpoints for managing the instance and generating content.

```APIDOC
StableSwarmUI Network API Overview:

General Request Format:
- Most API routes use POST requests to `(your server)/API/(route)`.
- Requests typically contain JSON formatted inputs and receive JSON formatted outputs.
- WebSocket routes are designated with a `WS` suffix.

Session Management:
- All API routes, except `GetNewSession`, require a `session_id` in the JSON input.
- Obtain a session ID by calling `/API/GetNewSession`.

Error Handling:
- API routes can return an `error` or `error_id`.
- `invalid_session_id`: Requires recalling `/API/GetNewSession`.
- Other `error_id`s are contextual.
- `error`: Display text for end-users.

Key API Route Categories:
- AdminAPI: Administrative server management.
- BackendAPI: Backend management.
- UtilAPI: General utility APIs.
- ModelsAPI: Handling models (loras, wildcards, etc.).
- T2IAPI: Text2Image APIs.
- BasicAPIFeatures: User and session handling.
- Built-in-Extension APIs: APIs for extensions like GridGenerator, ComfyUIWebAPI, ImageBatchTool.

Specific API Calls:

POST /API/GetNewSession
  Description: Obtains a new session ID for API interactions.
  Input:
    (empty JSON object is acceptable)
  Returns:
    session_id: Unique identifier for the session.
    user_id: Identifier for the user.
    output_append_user: Boolean indicating output append behavior.
    version: Server version.
    server_id: Unique server identifier.
    count_running: Number of currently running processes.

POST /API/GenerateText2Image
  Description: Generates images based on text prompts.
  Input:
    session_id: The session identifier.
    images: Number of images to generate.
    prompt: The text prompt for image generation.
    model: The model to use for generation (e.g., "OfficialStableDiffusion/sd_xl_base_1.0").
    width: The width of the generated image.
    height: The height of the generated image.
  Returns:
    images: A list of URLs to the generated images.

GET Routes:
- /Text2Image: HTML main page.
- /Install: HTML installer page.
- /Error/*: HTML error pages.
- /css/*.css: Stylesheets.
- /js/*.js: JavaScript files.
- /fonts/*.woff2, /imgs/*.jpg, /imgs/*.png, /favicon.ico: Asset files.
- /View/*.*: Returns saved outputs (e.g., /View/(user)/raw/(date)/(file).(ext)).
- /Output/*.*: Legacy output call format (deprecated).
- /ExtensionFile/(extension)/*.*: Gets web asset files from extensions.
- /ComfyBackendDirect/*.*: Direct pass-through to a Comfy instance (requires ComfyUI Backend Extension).
```

--------------------------------

### Install .NET and Python Dependencies on macOS

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Installs the .NET SDK and specific Python versions (3.10) along with virtualenv using Homebrew, required for running StableSwarmUI on macOS.

```bash
brew install dotnet
brew install python@3.10
brew install virtualenv
```

--------------------------------

### ControlNet Parameters and Configuration

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/ControlNet.md

Details on configuring ControlNet settings within the StableSwarmUI interface, including start/end times and multi-ControlNet setups.

```APIDOC
ControlNet Parameters:
  - ControlNet Start: (Optional) Specifies the diffusion step at which the ControlNet begins to influence the generation. Defaults to 0.
  - ControlNet End: (Optional) Specifies the diffusion step at which the ControlNet stops influencing the generation. Defaults to 1.
  - ControlNet Two: Enables a second ControlNet instance.
  - ControlNet Three: Enables a third ControlNet instance.

Usage:
  These parameters are found within the 'ControlNet' parameter grouping in the UI.
  'ControlNet Start' and 'End' allow fine-tuning the application of ControlNet guidance throughout the diffusion process.
  Enabling 'ControlNet Two' and 'Three' allows for complex multi-ControlNet workflows.
```

--------------------------------

### Workflow Node Examples for GPU Assignment

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Using More GPUs.md

Illustrates how node connections and output node coloring affect GPU assignment. A single `LoadCheckpoint` feeding multiple `KSampler` nodes, each with a unique output color, ensures efficient separation of tasks across GPUs.

```UI Interaction
`LoadCheckpoint` -> `KSampler` -> `Preview Image` (Color A)
```

```UI Interaction
`LoadCheckpoint` -> `KSampler` -> `Preview Image` (Color B)
```

```UI Interaction
`LoadCheckpoint` -> `KSampler` -> `Preview Image` (Color C)
```

--------------------------------

### StableSwarmUI Command Line Arguments

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Command Line Arguments.md

Configuration options for launching and running StableSwarmUI via the command line. These arguments allow for overriding default settings, specifying file paths, and controlling the application's environment and network behavior. Invalid inputs will prevent the program from starting.

```APIDOC
StableSwarmUI Command Line Arguments:

Usage Example:
  .\launch-windows.ps1 --host * --port 7850 --environment development --launch_mode web

Arguments:

--data_dir
  Default: `Data`
  Description: Override the default data directory.

--settings_file
  Default: `Data/Settings.fds`
  Description: Specify the path to the settings file if it's not in the default location.

--backends_file
  Default: `Data/Backends.fds`
  Description: Specify the path to the backends file if it's not in the default location.

--environment
  Default: `Production`
  Description: Sets the ASP.NET Web Environment. Can be `development` (detailed logs) or `production` (optimized).

--host
  Default: `localhost`
  Description: Overrides the 'Network.Host' server setting.

--port
  Default: `7801`
  Description: Overrides the 'Network.Port' server setting.

--asp_loglevel
  Default: `warning`
  Description: Sets the minimum log level for the ASP.NET web logger. Accepted values: `Trace`, `Debug`, `Information`, `Warning`, `Error`, `Critical`, `None`.

--loglevel
  Default: `Info`
  Description: Sets the minimum StableSwarmUI log level. Accepted values: `Debug`, `Info`, `Init`, `Warning`, `Error`, `None`.

--user_id
  Default: `local`
  Description: Sets the local user's default UserID for single-user mode.

--lock_settings
  Default: `false`
  Description: If enabled, prevents in-UI editing of server settings by admins. Settings can only be modified by editing the settings file and restarting.

--ngrok-path
  Default: (None)
  Description: Path to an `ngrok` executable. Enables automatic loading and configuration of ngrok to share the UI instance via a public URL.

--cloudflared-path
  Default: (None)
  Description: Path to a `cloudflared` executable. Enables automatic loading and configuration of TryCloudflare to share the UI instance via a public URL.

--proxy-region
  Default: (None)
  Description: Sets the region for the proxy (ngrok/cloudflared). Defaults to the closest region if unspecified.

--ngrok-basic-auth
  Default: (None)
  Description: Sets a basic authentication requirement for accessing the ngrok-proxied UI.

--launch_mode
  Default: `none`
  Description: Overrides the 'LaunchMode' server setting.
```

--------------------------------

### Use Workflow in Generate Tab

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Using More GPUs.md

Recommended method for multi-GPU usage. This action transfers the current Comfy workflow to the main Generate tab, allowing queued generations to be managed there, which is simpler for multi-GPU setups.

```UI Interaction
Click `Use This Workflow in Generate Tab`
```

--------------------------------

### Update and Verify Homebrew Packages on macOS

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Ensures Homebrew package manager is up-to-date and checks its installation integrity, which are prerequisites for installing other dependencies on macOS.

```bash
brew update
brew doctor
```

--------------------------------

### API ListBackendTypes: List Backend Types

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/BackendAPI.md

Provides a list of all available backend types that can be instantiated on the server. Includes details like name, description, and configurable settings.

```APIDOC
HTTP Route /API/ListBackendTypes

Description:
Returns of a list of all available backend types.

Parameters:
None.

Return Format:
{
    "list": [
        {
            "id": "idhere",
            "name": "namehere",
            "description": "descriptionhere",
            "settings": [
                {
                    "name": "namehere",
                    "type": "typehere",
                    "description": "descriptionhere",
                    "placeholder": "placeholderhere"
                }
            ],
            "is_standard": false
        }
    ]
}
```

--------------------------------

### Feature Combination

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Basic Usage.md

Illustrates the capability of StableSwarmUI to combine multiple features simultaneously, such as IP-Adapter and Video generation, with automatic handling of interactions.

```FeatureExample
Combine IP-Adapter with Video generation.
Enable multiple features simultaneously.
Swarm automatically handles feature combination.
```

--------------------------------

### ComfyUI Self-Start Configuration

Source: https://github.com/stability-ai/stableswarmui/blob/master/src/BuiltinExtensions/ComfyUIBackend/README.md

Configuration details for enabling ComfyUI's self-starting backend within StableSwarmUI. This involves specifying the path to the ComfyUI main script.

```APIDOC
Server Backend Configuration:
  ComfyUI Self-Starting:
    StartScript: Path to the ComfyUI main script (e.g., dlbackend/comfy/ComfyUI/main.py).
    Other values can be left default or configured to your preference.
```

--------------------------------

### ListRecentLogMessages API

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/AdminAPI.md

Returns a list of recent server log messages. Optionally, specify 'last_sequence_ids' to set the starting point for fetching logs.

```APIDOC
HTTP Route /API/ListRecentLogMessages

Description:
Returns a list of recent server log messages.

Parameters:
| Name | Type | Description | Default |
| --- | --- | --- | --- |
| raw | JObject | Optionally input "last_sequence_ids": { "info": 123 } to set the start point. | **(REQUIRED)**

Return Format:
```js
  "last_sequence_id": 123,
  "data": {
        "info": [
            {
                "sequence_id": 123,
                "timestamp": "yyyy-MM-dd HH:mm:ss.fff",
                "message": "messagehere"
            }, ...
        ]
    }
```
```

--------------------------------

### Migrate to Independent SwarmUI Repository

Source: https://github.com/stability-ai/stableswarmui/blob/master/README.md

Instructions for migrating from the Stability AI maintained version to the independent `mcmonkeyprojects/SwarmUI` repository using Git commands.

```git
# For Windows users: Run migrate-windows.bat after updating.
# For Linux, Mac, or manual Windows migration:
# Open a terminal in your Swarm folder
# git remote set-url origin https://github.com/mcmonkeyprojects/SwarmUI
```

--------------------------------

### Get Models API

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/ModelsAPI.md

Returns a list of models available on the server within a given folder, with their metadata. Supports filtering by subtype and sorting options.

```APIDOC
API Endpoint: (Implied, based on description and parameters)

Description:
Returns a list of models available on the server within a given folder, with their metadata.

Parameters:
- path (String): What folder path to search within. Use empty string for root. (REQUIRED)
- depth (Int32): Maximum depth (number of recursive folders) to search. (REQUIRED)
- subtype (String): Model sub-type - `LoRA`, `Wildcards`, etc. (Default: `Stable-Diffusion`)
- sortBy (String): What to sort the list by - `Name`, `DateCreated`, or `DateModified`. (Default: `Name`)
- sortReverse (Boolean): If true, the sorting should be done in reverse. (Default: `False`)

Return Format:
{
  "folders": ["folder1", "folder2"],
  "files":
  [
    {
      "name": "namehere",
      // etc., see `DescribeModel` for the full model description
    }
  ]
}
```

--------------------------------

### ControlNet Preview and Troubleshooting

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/ControlNet.md

Guidance on previewing ControlNet preprocessor results and resolving common errors.

```APIDOC
Previewing Preprocessors:
  - Click the 'Preview' button within the ControlNet parameter group.
  - This runs the selected preprocessor on the input image and displays the result.
  - Useful for verifying preprocessor output before generation.

Troubleshooting Common Errors:
  - Error: 'mat1 and mat2 shapes cannot be multiplied'
    - Cause: Mismatch between ControlNet model version (SDv1 vs. SDXL) and the Stable Diffusion model.
    - Solution: Ensure ControlNet models are compatible with the SD version being used.

  - Error: 'Cannot preview a ControlNet preprocessor without any preprocessor enabled.'
    - Cause: A ControlNet model was selected that does not auto-detect its preprocessor, and no preprocessor was manually chosen.
    - Solution: Manually select the appropriate preprocessor from the list for the chosen ControlNet model.
```

--------------------------------

### Loading LORAs in ComfyUI Workflows

Source: https://github.com/stability-ai/stableswarmui/blob/master/src/BuiltinExtensions/ComfyUIBackend/README.md

Example of using a specific node, SwarmLoraLoader, within a custom ComfyUI workflow to load LORAs. This is relevant for custom workflows used in the main Generate tab of StableSwarmUI.

```json
{
  "nodes": [
    {
      "id": 1,
      "type": "LoadCheckpoint",
      "params": {
        "ckpt_name": "sd_xl_base_1.0.safetensors"
      }
    },
    {
      "id": 2,
      "type": "SwarmLoraLoader",
      "params": {
        "lora_name": "my_lora.safetensors",
        "strength_model": 0.8,
        "strength_clip": 0.8
      }
    }
  ],
  "links": [
    {
      "from_node": 1,
      "from_slot": "model",
      "to_node": 2,
      "to_slot": "model"
    }
  ]
}
```

--------------------------------

### ComfyUI Web API HTTP Endpoints

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/ComfyUIWebAPI.md

Documentation for various HTTP endpoints of the ComfyUI Web API, including operations for managing workflows, refreshing states, and installing features. Each endpoint details its parameters and expected return format.

```APIDOC
HTTP Route /API/ComfyDeleteWorkflow
  Description: (ROUTE DESCRIPTION NOT SET)
  Parameters:
    - name (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
  Return Format:
    ```js
    (RETURN INFO NOT SET)
    ```

HTTP Route /API/ComfyEnsureRefreshable
  Description: (ROUTE DESCRIPTION NOT SET)
  Parameters:
    None.
  Return Format:
    ```js
    (RETURN INFO NOT SET)
    ```

HTTP Route /API/ComfyGetGeneratedWorkflow
  Description: (ROUTE DESCRIPTION NOT SET)
  Parameters:
    - rawInput (JObject): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
  Return Format:
    ```js
    (RETURN INFO NOT SET)
    ```

HTTP Route /API/ComfyInstallFeatures
  Description: (ROUTE DESCRIPTION NOT SET)
  Parameters:
    - feature (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
  Return Format:
    ```js
    (RETURN INFO NOT SET)
    ```

HTTP Route /API/ComfyListWorkflows
  Description: (ROUTE DESCRIPTION NOT SET)
  Parameters:
    None.
  Return Format:
    ```js
    (RETURN INFO NOT SET)
    ```

HTTP Route /API/ComfyReadWorkflow
  Description: (ROUTE DESCRIPTION NOT SET)
  Parameters:
    - name (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
  Return Format:
    ```js
    (RETURN INFO NOT SET)
    ```

HTTP Route /API/ComfySaveWorkflow
  Description: (ROUTE DESCRIPTION NOT SET)
  Parameters:
    - name (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
    - workflow (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
    - prompt (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
    - custom_params (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
    - param_values (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
    - image (String): (PARAMETER DESCRIPTION NOT SET) - **(REQUIRED)**
    - description (String): (PARAMETER DESCRIPTION NOT SET) - (Empty String)
    - enable_in_simple (Boolean): (PARAMETER DESCRIPTION NOT SET) - `False`
  Return Format:
    ```js
    (RETURN INFO NOT SET)
    ```
```

--------------------------------

### Grid Generator Usage

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Basic Usage.md

Instructions on how to use the Grid Generator tool to test multiple parameter combinations simultaneously. It involves selecting axes, filling values, and generating a grid of images.

```UIInstructions
Navigate to Tools -> Grid Generator.
Select an axis to compare (e.g., Sampler, CFG Scale).
Fill axis values manually or use the 'fill with examples' button.
Add multiple axes as needed.
Click 'Generate Grid' to start the process.
```

--------------------------------

### API AddNewPreset Endpoint

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/BasicAPIFeatures.md

Adds a new preset to the system. Requires title, description, and raw data. An optional preview image and editing state can be provided.

```APIDOC
HTTP Route /API/AddNewPreset

Description:
(ROUTE DESCRIPTION NOT SET)

Parameters:
| Name | Type | Description | Default |
| --- | --- | --- | --- |
| title | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| description | String | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| raw | JObject | (PARAMETER DESCRIPTION NOT SET) | **(REQUIRED)** |
| preview_image | String | (PARAMETER DESCRIPTION NOT SET) | (null) |
| is_edit | Boolean | (PARAMETER DESCRIPTION NOT SET) | `False` |
| editing | String | (PARAMETER DESCRIPTION NOT SET) | (null) |

Return Format:
```js
(RETURN INFO NOT SET)
```
```

--------------------------------

### Lightning Models Configuration

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Model Support.md

Basic configuration guidelines for Lightning models in StableSwarmUI, specifying CFG Scale and Steps. Sampling specifics are noted as pending.

```APIDOC
Lightning Models Configuration:
  General Settings:
    - `CFG Scale`: Set to `1`.
    - `Steps`: (TODO: Sampling specifics for lightning).
  Notes: Work the same as regular models, with specified CFG Scale and Steps.
```

--------------------------------

### Preset Injection Syntax

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/Prompt Syntax.md

Allows for the dynamic inclusion of predefined prompt presets into the current prompt. This can be combined with other syntaxes for dynamic preset selection.

```APIDOC
Preset Injection:

Syntax: <preset:presetname>

Description:
  Injects the content of a named preset into the prompt.

Parameters:
  - presetname: The name of the preset to inject.

Examples:
  - <preset:cinematic_lighting>
  - <preset:<random:style_a, style_b>> (dynamically selects a preset)

Notes:
  - This syntax is useful for programmatically or dynamically altering generation parameters via presets.
```

--------------------------------

### Adding Custom Tabs

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Making Extensions.md

Explains the convention for adding custom tabs to the StableSwarmUI interface by creating specific folder structures within the extension directory.

```html
src/Extensions/MyExtension/Tabs/Text2Image/Your Tab Name.html
```

--------------------------------

### Textual Inversion Embeddings Syntax

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/Prompt Syntax.md

Provides a way to directly reference and use Textual Inversion embeddings within the prompt by their filename.

```APIDOC
Textual Inversion Embeddings:

Syntax: <embed:filename>

Description:
  Injects a Textual Inversion embedding into the prompt at the specified location.

Parameters:
  - filename: The name of the Textual Inversion embedding file (without extension).

Example:
  - <embed:my_style_embedding>
```

--------------------------------

### From-To Text Swapping Syntax

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/Prompt Syntax.md

Facilitates swapping one phrase for another after a specified timestep during generation. The timestep can be an absolute step number or a fractional value representing progress.

```APIDOC
From-To Text Swapping:

Syntax: <fromto[timestep]:before, after>

Description:
  Swaps the 'before' text with the 'after' text at a specific point in the generation process.

Parameters:
  - timestep: The point at which the swap occurs. Can be an integer step number (e.g., 10) or a float representing progress (e.g., 0.5 for halfway).
  - before: The text to be replaced.
  - after: The text to replace 'before' with.

Examples:
  - <fromto[0.5]:cat, dog> (swaps 'cat' to 'dog' halfway through generation)
  - <fromto[10]:blue sky, red sky> (swaps 'blue sky' to 'red sky' at step 10)

Separators:
  - Must have exactly two entries separated by `,`.
  - Similar to 'alternate', `|` or `||` can be used as separators if the 'before' or 'after' text contains commas.
```

--------------------------------

### Launch StableSwarmUI with Ngrok

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Advanced Usage.md

Configure StableSwarmUI to use Ngrok for secure, remote access without port forwarding. This involves specifying the path to the Ngrok executable.

```bash
stable-swarm-ui --ngrok-path /path/to/ngrok
```

--------------------------------

### Wildcard Selection Syntax

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/Prompt Syntax.md

Allows random selection from a predefined text file (wildcard file), where each line represents a possible option. These files can be managed in the UI or placed in a specific directory.

```APIDOC
Wildcard Selection:

Syntax: <wildcard:wildcard_file_name>
Syntax for multiple selections: <wildcard[count]:wildcard_file_name>

Description:
  Randomly selects one or more lines from a specified wildcard text file.

Parameters:
  - wildcard_file_name: The name of the text file containing options, one per line.
  - count (optional): Specifies how many lines to select from the file. Can be a single number or a range (e.g., 1-3).

Examples:
  - <wildcard:animals> (selects one random animal from the 'animals' wildcard file)
  - <wildcard[1-3]:animals> (selects 1 to 3 random animals from the 'animals' file)

File Management:
  - Wildcard files can be created and edited in the 'Wildcards' tab of the UI.
  - Custom wildcard files can be imported by placing them in the 'Data/Wildcards' folder.
```

--------------------------------

### Fancy Prompt Syntax - Timestep Alternating/FromTo

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Why Use Swarm.md

Enables advanced prompt engineering techniques directly within the generate tab, such as timestep prompt alternating and 'from-to' syntax, which are typically difficult to implement in standard ComfyUI.

```text
Type a `<` in the generate tab prompt box to access fancy prompt syntax.
Supports timestep prompt alternating/fromto.
```

--------------------------------

### Grid Generation Input Syntax

Source: https://github.com/stability-ai/stableswarmui/blob/master/src/BuiltinExtensions/GridGenerator/README.md

Provides syntax rules for generating image grids, including parameter filling, skipping values, prompt separation, and prompt replacement.

```APIDOC
Grid Generation Input Syntax:

Parameter Filling:
  Use '..' between numbers to automatically fill a sequence.
  Requires two starting numbers (to identify start and step) and one ending number.
  Example: '1, 2, .., 10' fills to '1, 2, 3, 4, 5, 6, 7, 8, 9, 10'
  Example: '1, 3, 5, 6, 6.5, .., 9, 11, 13' fills to '1, 3, 5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 11, 13'

Skipping Values:
  Prefix a value with 'SKIP:' (all caps) to exclude it from generation while maintaining sequence.
  Example: '1, 2, SKIP: 3, 4' generates images for 1, 2, and 4, but not 3.
  Useful for placeholders or overwriting specific images in saved configurations.

Prompt Separation:
  Use '||' to separate multiple text prompts when a prompt itself contains commas.
  Example: 'a cat, red || a cat, blue' is parsed as two distinct prompts.

Prompt Replacement:
  Allows replacing parts of a prompt or entire prompts.
  Simple replacement: 'cat, dog, wolf' applied to 'a photo of a cat' yields 'a photo of a cat', 'a photo of a dog', 'a photo of a wolf'.
  Key-value replacement: 'cat=cat, cat=dog, photo=drawing' applied to 'a photo of a cat' yields 'a photo of a cat', 'a photo of a dog', 'a drawing of a cat'.
  Placeholder replacement: 'mylora, myotherlora, mythirdlora' can replace placeholders like '<lora:mylora>' in prompts.
  Combination: 'a photo of a cat <lora:mylora>' with replacement 'mylora, myotherlora' yields 'a photo of a cat <lora:mylora>' and 'a photo of a cat <lora:myotherlora>'.

Preset Usage:
  Reference saved configurations using '<preset:preset_name>'.
  Can be combined: 'mypreset || mypreset, mypreset2 || mypreset2' to compare presets and their combinations.
```

--------------------------------

### LoRA Application Syntax

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/Prompt Syntax.md

Enables the application of LoRAs (Low-Rank Adaptation) directly within the prompt, including specifying a weight. While convenient, using the GUI is generally preferred.

```APIDOC
LoRA Application:

Syntax: <lora:filename:weight>

Description:
  Applies a LoRA model to the generation process with a specified weight.

Parameters:
  - filename: The name of the LoRA file.
  - weight: The strength of the LoRA's influence (e.g., 0.7).

Examples:
  - <lora:my_style_lora:0.7>

Notes:
  - It is generally recommended to use the GUI for selecting LoRAs.
  - LoRA placement typically does not affect its application, unless used within `<segment:...>` or `<object:...>` syntax, where it applies only to that specific segment or object.
```

--------------------------------

### Registering Custom Web Assets

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Making Extensions.md

Illustrates how to register custom JavaScript files and CSS stylesheets for use within the StableSwarmUI interface by overriding specific properties in the Extension class.

```csharp
public override List<string> ScriptFiles => new List<string> { "path/to/your/script.js" };
public override List<string> StyleSheetFiles => new List<string> { "path/to/your/style.css" };
```

--------------------------------

### Latency Consistency Models (LCM) Configuration

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Model Support.md

Instructions for configuring Latency Consistency Models (LCM) in StableSwarmUI, including CFG Scale, Steps, and Sampler settings.

```APIDOC
Latency Consistency Models (LCM) Configuration:
  General Settings:
    - `CFG Scale`: Set to `1`.
    - `Steps`: Set to `4`.
  Sampler:
    - Parameter: `Sampler` (under `Sampling` group).
    - Setting: Set to `lcm`.
```

--------------------------------

### Repetition Syntax

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/Prompt Syntax.md

Enables repeating a specific word or phrase a set number of times, or a random number of times within a range. Can be combined with other syntaxes like random.

```APIDOC
Repetition Syntax:

Syntax: <repeat:count, text>

Description:
  Repeats the specified 'text' 'count' number of times.

Parameters:
  - count: The number of times to repeat the text. Can be a fixed number or a range (e.g., 1-3).
  - text: The word or phrase to repeat.

Examples:
  - <repeat:3, cat> (results in 'cat cat cat')
  - <repeat:1-3, <random:cat, dog>> (results in 1 to 3 repetitions of either 'cat' or 'dog', e.g., 'cat dog cat')

Notes:
  - The 'text' parameter can itself be another syntax, allowing for dynamic repetition.
```

--------------------------------

### T2IAPI: List T2I Parameters

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/T2IAPI.md

Fetches a list of available Text-to-Image parameters and their configurations.

```APIDOC
HTTP Route /API/ListT2IParams

Description:
List available T2I parameters.

Parameters:
None

Return Format:
(JSON object detailing available T2I parameters, their types, defaults, and options)
```

--------------------------------

### API AddNewBackend: Add New Backend

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/APIRoutes/BackendAPI.md

Adds a new backend to the server. Requires specifying the type of backend to instantiate. The response includes details of the newly created backend.

```APIDOC
HTTP Route /API/AddNewBackend

Description:
Add a new backend of the specified type.

Parameters:
- type_id (String): ID of what type of backend to add (see `ListBackendTypes`). (REQUIRED)

Return Format:
{
    "id": "idhere",
    "type": "typehere",
    "status": "statushere",
    "settings": {
        "namehere": valuehere
    },
    "modcount": 0,
    "features": [ "featureidhere", ... ],
    "enabled": true,
    "title": "titlehere",
    "can_load_models": true,
    "max_usages": 0
}
```

--------------------------------

### Image Segmentation and Refining Directives

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Features/Prompt Syntax.md

Allows automatic segmentation and refining of specific image regions using text prompts or YOLOv8 models. Supports customization of creativity and segmentation thresholds, and selection of specific YOLO matches.

```APIDOC
Image Segmentation Directives:

Use directives within prompts to control image manipulation.

1. CLIP Segmentation:
   - Syntax: `<segment:texthere>`
     - Automatically refines a part of the image based on the provided text description.
     - Example: `<segment:face>`

   - Syntax with parameters: `<segment:texthere,creativity,threshold>`
     - `texthere`: Text description for segmentation (e.g., 'face').
     - `creativity`: Inpaint strength, a float value (e.g., 0.8). Defaults to 0.6.
     - `threshold`: Segmentation minimum threshold, a float value (e.g., 0.5). Defaults to 0.5.
     - Example: `<segment:face,0.8,0.5>`

   - Dependencies: Requires CLIP Segmentation model (automatically downloaded on first use).

2. YOLOv8 Segmentation:
   - Syntax: `<segment:yolo-modelnamehere>`
     - Uses a specified YOLOv8 segmentation model to segment the image.
     - Models should be stored in `(Swarm)/Models/yolov8`.
     - Example: `<segment:yolo-face_yolov8m-seg_60.pt>`

   - Syntax with match index: `<segment:yolo-modelnamehere-N>`
     - Selects a specific detected object match (N) from the YOLO segmentation.
     - N is a 1-based index (e.g., `-1` for the first match, `-2` for the second).
     - Example: `<segment:yolo-face_yolov8m-seg_60.pt-1>`

   - Syntax with creativity for YOLO: `<segment:yolo-modelnamehere-N,creativity,1>`
     - Applies a specific creativity value to the YOLO segmentation refinement.
     - The `,1` is a placeholder for the threshold parameter when only creativity is specified for YOLO.
     - Example: `<segment:yolo-face_yolov8m-seg_60.pt-1,0.8,1>`

   - Behavior without index: If multiple objects are detected and no index is specified, a bulk segmented refine is applied to all detected objects.

3. Advanced Settings:
   - `Segment Model` (under Regional Prompting): Allows customization of the base model used for segment processing.
   - `Save Segment Mask`: Option to save a preview copy of the generated mask.

4. LORA Integration:
   - You can insert a `<lora:...>` directive within the prompt area of a segment directive to apply a LORA model to that specific segment.
```

--------------------------------

### SwarmKSampler Node - Advanced Control

Source: https://github.com/stability-ai/stableswarmui/blob/master/docs/Why Use Swarm.md

Introduces the `SwarmKSampler` node, an enhanced version of ComfyUI's sampler, offering additional power-user options and dynamic control over preview methods, which are not directly available as nodes in standard ComfyUI.

```text
Control preview method on the fly dynamically.
`SwarmKSampler` node provides extra power-user options.
```
Tokens:
Raw